JP2007221294A - Image processing apparatus, image processing method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method capable of generating output information in which postscript information and its history can be discriminated. <P>SOLUTION: A changed region including a pixel region to be discriminated as postscript information is extracted, and the output information enabling to discriminate history of generation of history information corresponding to the changed region, i.e. a postscript history is generated. For example, a color applied to the postscript information itself or the background of the postscript information is set at different lightness or saturation corresponding to the postscript history. This configuration enables a user to discriminate the history of postscript information on the basis of the output information. Further, in an example of configuration in which different colors are set depending on the kind of postscript information, e.g. for each user who gives postscript or the postscript kind such as postscript contents, the postscript history and the postscript kind can be easily discriminated. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer program. More specifically, additional information such as notes written on a document composed of text or image data, for example, a memo written on a printed document, is extracted, and a plurality of additional writings executed at different timings are distinguished. In addition, the present invention relates to an image processing apparatus, an image processing method, and a computer program that can generate output information that enables grasping of an additional recording history and present it to a user.

  In the field of image processing, for example, various researches and developments have been conducted on a process of extracting additional information such as a memo written by a user in a document output from a printer or a FAX. In general, such additional record extraction processing is performed by comparing original document data with an output document that may be added.

  For example, Patent Document 1 discloses a configuration in which an original document and a new manuscript document that can be additionally written are stored as electronic document data, and a difference extraction between these electronic documents is performed to extract additional information. Has been.

  In addition, there are, for example, Patent Documents 2 to 11 as conventional techniques that disclose the configuration of the additional information extraction process. However, in the systems disclosed heretofore, including the above-described Patent Document 1, only the additional information is specified and extracted. For example, even when multiple additional recordings are executed at different timings, No additional information is distinguished, and there has been no disclosure of a configuration that makes it possible to grasp the additional recording history such that the second additional recording is performed with respect to the first additional recording.

  When a note is added to some original document, the first addition is performed based on the information described in the original document, and the added document including the additional information is an information processing apparatus such as a PC. In addition, the additional document may be printed at a later date, and then a second additional recording may be performed. The user who performs additional writing may be a plurality of different users, and the second additional writing may be a comment corresponding to the first additional writing.

When such a document containing multiple times of postscript information is printed out or presented on a display screen, it is difficult to grasp the order of postscript only by displaying the original information and postscript information separately. There is a problem that it is not possible to determine which postscript comment is the final postscript comment.
Japanese Patent Laid-Open No. 10-255027 JP 11-272654 A JP 2000-112646 A JP 2002-352217 A JP 2003-67363 A JP 2004-80601 A JP 2004-302617 A JP-A-2005-74724 JP 2005-122682 A JP 2005-149097 A JP 2005-175565 A

  The present invention has been made in view of the above-described problems, and extracts additional information such as a memo written on a document composed of text, image data, and the like, for example, a printed document. , Image processing apparatus, image processing method, and computer program capable of distinguishing a plurality of additional recordings executed at different timings, generating output information that enables grasping of the additional recording history, and presenting it to the user The purpose is to provide.

The first aspect of the present invention is:
A change area determination means for executing a determination process of a change area determined as additional write information based on a document image;
Input the determination result information of the change area determination means, change area history determination means for determining the additional writing history of additional information,
Change area color conversion means for inputting the determination result information of the change area history determination means and performing different display mode setting processing according to the additional writing history as an output mode corresponding to each of the determined additional writing information;
Output means for generating output information according to the display mode corresponding to the additional information set in the change area color conversion means;
An image processing apparatus characterized by comprising:

  Furthermore, in an embodiment of the image processing apparatus of the present invention, the image processing apparatus further includes a change area type determination unit that determines the type of the additional information based on the document image, and the change area color The conversion means is configured to perform a setting process of a different display mode corresponding to the additional recording history and the additional recording type as an output mode corresponding to each additional recording information.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the change area determination means executes a change area determination process including a pixel area determined as additional write information based on a difference between a plurality of input image data. It is the structure.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the change area determination means determines a change area including a pixel area that is determined as additional information based on color information included in one input image data. It is the structure which performs.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the change area color conversion means sets the saturation to be set to at least one of the postscript information itself or the background color of the postscript information as a different display mode according to the postscript history. Alternatively, at least one of the lightnesses is configured to perform a display mode setting process corresponding to the additional writing information, which is changed according to the additional recording history.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the change area color conversion means sets the display mode according to the type of the postscript information as at least one of the postscript information itself or the background color of the postscript information. The present invention is characterized in that a display mode setting process corresponding to additional information is performed by changing at least one of saturation and lightness in accordance with the type of additional information.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the output means is configured to generate output information including guide information indicating the meaning of the display mode of the additional information.

Furthermore, the second aspect of the present invention provides
In the image processing apparatus, an image processing method for generating output information that enables determination of a postscript history of postscript information,
In the change area determination means, a change area determination step for executing a change area determination process determined as additional information based on the document image;
In the change area history determination means, input the determination result information of the change area determination means, a change area history determination step for determining the additional write history of the additional information,
In the change area color conversion means, the determination result information of the change area history determination means is input, and a change area for performing a different display mode setting process according to the additional writing history as an output mode corresponding to each of the determined additional writing information A color conversion step;
In the output means, an output step for generating output information in accordance with the display mode corresponding to the additional information set in the change area color conversion means;
An image processing method characterized by comprising:

  Furthermore, in one embodiment of the image processing method of the present invention, the image processing method further includes a change region type determination step of determining a type of the additional information based on the document image in a change region type determination unit. And the change area color conversion step is a step of performing a setting process of different display modes according to the additional recording history and the additional recording type as an output mode corresponding to each additional recording information.

  Furthermore, in an embodiment of the image processing method of the present invention, the change area determination step executes a change area determination process including a pixel area determined as additional information based on a difference between a plurality of input image data. It is a step.

  Furthermore, in an embodiment of the image processing method of the present invention, the change area determination step includes determining a change area including a pixel area that is determined as additional information based on color information included in one input image data. It is a step for executing processing.

  Furthermore, in one embodiment of the image processing method of the present invention, the change area color conversion step sets the saturation to be set to at least one of the postscript information itself and the background color of the postscript information as a different display mode according to the postscript history. Alternatively, it is a step of performing a display mode setting process corresponding to additional recording information in which at least one of the brightness values is changed according to the additional recording history.

  Furthermore, in an embodiment of the image processing method according to the present invention, the change area color conversion step sets at least one of the postscript information itself and the background color of the postscript information as a different display mode according to the type of the postscript information. It is a step of performing a display mode setting process corresponding to additional recording information in which at least one of saturation and lightness is changed according to the type of additional recording information.

  Furthermore, in an embodiment of the image processing method of the present invention, the output step is a step of generating output information including guide information indicating the meaning of the display mode of the additional information.

Furthermore, the third aspect of the present invention provides
In the image processing apparatus, a computer program that executes image processing for generating output information that enables determination of a postscript history of postscript information,
A change area determination step for executing a change area determination process determined as additional information based on the document image in the change area determination means;
In the change area history determination means, input the determination result information of the change area determination means, a change area history determination step for determining the additional write history of the additional information,
In the change area color conversion means, the change result information of the change area history determination means is input, and a change is made to perform a different display mode setting process according to the additional writing history as an output mode corresponding to each of the determined additional writing information An area color conversion step;
In the output means, an output step for generating output information in accordance with the display mode corresponding to the additional information set in the change area color conversion means;
In a computer program characterized by causing

  Note that the computer program of the present invention is a recording medium provided in a computer-readable format for a computer system capable of executing various program codes, for example, a recording medium such as a CD, FD, or MO. A computer program that can be provided by a medium or a communication medium such as a network. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of the present invention, in the image processing apparatus that executes the process of extracting and presenting additional information, the modified area including the pixel area that is determined as the additional information is extracted, and the additional information corresponding to the modified area is generated. Output information capable of discriminating the additional history, that is, the additional recording history is generated. Specifically, for example, since the color applied to the postscript information itself or the background of the postscript information is set to a different brightness or saturation according to the postscript history, the user can record the history of the postscript information based on the output information. Can be determined. Furthermore, in the configuration example in which different colors are set according to the type of additional writing information, for example, for each additional writing user, additional writing content, or additional writing type such as additional writing color, it becomes possible to easily determine the additional writing history and the additional writing type. .

  Hereinafter, an image processing apparatus, an image processing method, and a computer program according to embodiments of the present invention will be described in detail with reference to the drawings.

  First, an outline of processing to which the image processing apparatus of the present invention is applied will be described with reference to FIG. An original document 11 shown in FIG. 1 is a document as, for example, a paper medium, which is output through a printer 102, electronic data stored in an information processing apparatus 101 such as a PC. The original document 11 is a document that is read by a data reading processing device such as a scanner and stored as electronic data in the storage unit of the information processing device 101. The original document 11 may be image data, text data, or the like. In some cases, it is taken in as data. In the present invention, the document to be processed may be either image data or text data.

  The original document is not limited to a document read by a data reading device such as a scanner, but may be a document as electronic data generated by an information processing device 101 such as a PC and stored in a database such as a hard disk. For example, it may be a document generated by a document creation application provided with various attribute data such as layout information.

  Document data stored in a database such as a hard disk of the information processing apparatus 101 is output as an original document 11 via the printer 102 and distributed to users. The output document output from the printer is an original document 11 shown in the figure.

  The user writes notes on the original document 11. So-called postscript is performed. As a result, the first postscript document 12 shown in the figure is generated. In the first additional writing document 12, first additional writing information 13 including memos such as comments by the user is additionally written. The first additional document 12 is read using, for example, the scanner 103, and the read data is stored in the image processing apparatus 200. Further, after the first additional writing document 12 is generated, new additional writing is performed on the first additional writing document 12. This additional recording document is the second additional recording document 14 shown in the figure, and the second additional recording document 14 is recorded with second additional recording information 15 such as memo writing as a new additional recording. The second additional document 14 is also read using the scanner 103 and the read data is stored in the image processing apparatus 200.

  The generation of the second additional writing document 14 is performed after the creation of the first additional writing document. For example, the image data of the first additional document 12 read by the scanner 103 is recorded in the storage unit of the image processing apparatus 200, the data is transferred to the information processing apparatus 101, printed out, and printed out. A new additional recording (second additional recording information) is recorded on the document, and a second additional recording document 14 is generated.

  In the image processing apparatus 200, the data of the first additional document 12 acquired through the scanner 103, the data of the second additional document 14, and the data of the original document 11 are acquired from the information processing apparatus 101 via a network, for example. The original document 11 and each additional document 12 and 14 are compared with each other, and processing such as difference extraction is executed to generate output data 20 and present an analysis result via a display or the like, or a printer (not shown) The analysis result is output via. If the original document data cannot be acquired, the postscript information is extracted only from the postscript document. These processes will be described later.

  The image processing apparatus 200 according to the present invention not only extracts the postscript information by analyzing the difference between the original document 11 and each of the postscript documents 12 and 14, but also analyzes the postscript history as the order in which the postscript information was recorded, Output data 20 is generated so that the additional recording history can be grasped. For example, in the output data 20 shown in the figure, two additional write information executed at a plurality of different timings, that is, first additional write information 21 and second additional write information 22 are shown. The postscript information generated at different timings is output with the display mode changed. For example, output information that enables each additional information to be distinguished is generated by processing such as changing the display color or the background color.

  Furthermore, a specific processing example executed by the image processing apparatus of the present invention will be described with reference to FIGS.

  FIG. 2 shows document transitions along the time axis (t). The original document 201 at the left end shown in FIG. 2 is, for example, a paper document obtained by printing out original document data that has been stored in an image processing apparatus. For example, the user adds a note to the original document 201. The resulting document is the first additional writing document 221. The character string “Original 1” shown on the sheet of the original document 201 indicates information recorded in the original document 201, and the characters “Addition 1” shown on the sheet of the first additional document 221. The column indicates the first additional information. Hereinafter, similarly, the character string of [Additional n] indicated on the sheet of the nth additional writing document indicates the nth additional writing information.

  The first additional information (additional writing 1) in the first additional writing document 221 is additional writing performed with reference to the information recorded in the original document 201. At time (t1), the first additional document 221 is scanned by a scanner, and data is stored as first additional document image data in a storage unit in the image processing apparatus.

  Further, after the generation of the first additional writing document 221, the second additional writing document 222 in which the new additional writing (addition 2) is recorded is generated with reference to the first additional writing document 221. In addition, the user who performs each additional recording is various, such as a case where it is the same user, a case where it is a different user. The second additional writing information (additional writing 2) in the second additional writing document 222 is a new addition made by referring to the original information (original 1) and the first additional writing information (additional writing 1). At time (t2), the second additional document 222 is scanned by the scanner, and is stored as second additional document image data in the storage unit in the image processing apparatus.

  Furthermore, after the second additional writing document 222 is generated, a third additional writing document 223 in which a new additional writing (Addition 3) is recorded is generated with reference to the second additional writing document 222. The third postscript information (postscript 3) in the third postscript document 223 was performed with reference to the original information (original 1), the first postscript information (postscript 1), and the second postscript information (postscript 2). It is a new postscript. At time (t3), the third additional record document 223 is scanned by a scanner, and data is stored as third additional document image data in a storage unit in the image processing apparatus.

The image processing apparatus has different appendable documents recorded in the storage unit, that is,
(1) 1st recordable document (2) 2nd recordable document (3) 3rd recordable document Using these recordable document image data, and further reporting as necessary, applying the image data of the original document Are extracted and the history of additional writing is analyzed, output information 224 is generated and printed or displayed on the display unit.

  The image processing apparatus determines the additional writing history by analyzing the first to third additional writing documents, and generates output information in different display modes for the plurality of additional writing information according to the additional writing order. In the figure, as output information 224, two different display examples (display examples 1 and 2) are shown.

In display example 1, first, in order to make the postscript information distinguishable from the original information, the display color of the postscript information is displayed as a color (for example, blue) different from the original information (for example, black). It is the example of a display which changed the saturation of the postscript information output in blue. In particular,
(A) First additional note: blue with low saturation,
(B) Second additional note: Blue in saturation,
(C) Third note: Highly saturated blue,
In this way, the output is applied by applying a color in which the saturation of the older (preceding) additional writing is lowered and the saturation of the newer (later) additional writing is increased.

In the display example 2, the background color (for example, blue) is set only for the postscript information so that it can be distinguished from the original information. Further, according to the postscript history, the brightness of the background color of the postscript information output in blue It is the example of a display which changed. In particular,
(A) Background of the first postscript: Blue with low brightness,
(B) Background of the second postscript: Blue in lightness,
(C) Background of the third postscript: light blue,
In this way, the lightness of the background color of the older (preceding) additional recording is lowered, and the color with the lightness of the background color of the newer (following) additional recording is applied and output.

  By these display outputs, the user can accurately grasp not only the original information and the additional information but also the order of the additional information.

  Note that this processing example is an example of processing using three additional writing documents that have been subjected to three different additional writing processes, but the same processing can be performed for any number of additional writing documents of two or more times.

  FIG. 3 is a processing example in which additional writing executed based on the same document is also displayed separately, and different additional writing information executed based on the same document can be discriminated. For example, it is possible to distinguish additional writings performed by different users A and B, or to distinguish additional writing as a comment and additional writing as a correction even for the same user.

  FIG. 3 also shows the document transition along the time axis (t), as in FIG. The original document 230 at the left end shown in FIG. 3 is, for example, a paper document obtained by printing out original document data that has been stored in the image processing apparatus. Different additions are made to the original document 231. Here, an example of processing in which user A and user B perform additional writing will be described.

  User A performs additional writing [additional writing 1r], for example, with a red pen. User B performs additional writing [additional writing 1b], for example, with a blue pen. The resulting document is the first additional record document 231.

  The character string “Original 1” shown on the paper of the original document 230 indicates information recorded in the original document 230, and the character “Addition 1r” shown on the paper of the first additional document 231. The column indicates the first additional write information made in red (Red), and the character string [Addition 1b] shows the first additional write information made in blue (Blue). In the same manner, the character string “additional nr” shown on the sheet of the nth additional writing document indicates the nth red additional information, and the character string “addition nb” indicates the nth blue additional information. ing.

  A plurality of different first additional information (additional writing 1r, additional writing 1b) in the first additional writing document 231 are additional writings having different meanings with reference to information recorded in the original document 230, for example. Here, it is assumed that the additional writing of the user A is made in red (Red) [Additional writing 1r] and the additional writing made by the user B is made in blue (Blue) [Additional writing 1b].

  At time (t1), the first additional document 231 is scanned by a scanner, and data is stored as first additional document image data in a storage unit in the image processing apparatus.

  Further, after the generation of the first additional document 231, a second additional document 232 in which a new additional recording (additional recording 2) is recorded is generated with reference to the first additional recording document 231. A plurality of pieces of second additional information (additional writing 2r, additional writing 2b) in the second additional writing document 232 are newly made by referring to the original information (original 1) and the first additional writing information (additional writing 1r, additional writing 1b). It is an additional note. Here, it is assumed that the additional writing of the user A is made in red (Red) [Additional writing 2r] and the additional writing made by the user B is made in blue (Blue) [Additional writing 2b].

  At time (t2), the second additional document 232 is scanned by the scanner and stored as second additional document image data in the storage unit in the image processing apparatus.

  Furthermore, after the second additional writing document 232 is generated, the third additional writing document 233 in which the new additional writing (addition 3) is recorded is generated with reference to the second additional writing document 232. The plurality of third additional information (additional writing 3r, additional writing 3b) in the third additional writing document 233 includes original information (original 1), first additional writing information (additional writing 1r, additional writing 1b), and second additional writing information (additional writing). 2r, additional writing 2b). Here, it is assumed that the additional writing of user A is made in red (Red) [additional writing 3r], and the additional writing made by user B is made in blue (blue) [additional writing 3b].

  At time (t3), the third additional document 233 is scanned by the scanner, and data is stored as second additional document image data in the storage unit in the image processing apparatus.

The image processing apparatus has different appendable documents recorded in the storage unit, that is,
(1) 1st recordable document (2) 2nd recordable document (3) 3rd recordable document Using these recordable document image data, and further reporting as necessary, applying the image data of the original document Is extracted and the history of additional writing is executed, and output information 234 is generated and printed out or displayed on the display unit.

  The image processing apparatus determines the postscript history and the postscript type, here the user, by analyzing the first to third postscript documents, and outputs output information in different display modes for a plurality of postscript information according to the order of the postscript. Generate. Note that the additional recording type is not only the additional user as described above, but may also be different types of additional writing such as comment addition and correction addition. In the present processing example, output information that distinguishes the additional recording history and the additional recording type is generated.

FIG. 3 shows two different display examples (display examples 1 and 2) as the output information 234. In display example 1, first, in order to make it possible to distinguish the postscript information from the original information, the display color of the postscript information is displayed as a color (for example, red and blue) different from the original information (for example, black). Here, different colors are set according to the type of additional information (for example, the type of user). For example, user A's additional writing (additional writing 1r, 2r, 3r) is red, and user B's additional writing (additional writing 1b, 2b, 3b) is blue. Furthermore, it is the example of a display which changed the saturation of additional recording information according to the additional recording log | history of each additional recording. In particular,
(A) First additional recording of additional recording type 1 (user A): red with low saturation,
(B) Second additional recording of additional recording type 1 (user A): red in saturation,
(C) 3rd additional recording of additional recording type 1 (user A): red with high saturation,
(D) First additional recording of additional recording type 2 (user B): blue with low saturation,
(E) Second additional recording of additional recording type 2 (user B): blue in saturation,
(F) Third additional recording of additional recording type 2 (user B): high saturation blue,

  In this way, the color of the older (advanced) postscript is reduced, and the newer (later) postscript is increased in color and output, and different colors are applied depending on the type of postscript. The output information that enables the type of additional writing to be discriminated is generated.

Display example 2 sets the background color (for example, red and blue) only for the postscript information so that it can be distinguished from the original information, and has a different background color depending on the type of postscript information (for example, the type of user). It is set. For example, user A's additional writing (additional writing 1r, 2r, 3r) is red, and user B's additional writing (additional writing 1b, 2b, 3b) is a blue background color. Furthermore, it is the example of a display which changed the lightness of the background color of additional recording information according to the additional recording log | history of each additional recording. In particular,
(A) Background of the first additional recording of the additional recording type 1 (user A): red with low brightness,
(B) Background of second additional recording of additional recording type 1 (user A): red in lightness,
(C) Background of the third postscript of the postscript type 1 (user A): red with high brightness,
(D) Background of the first additional recording of the additional recording type 2 (user B): light blue,
(E) Background of second additional recording of additional recording type 2 (user B): light blue
(F) Background of the third additional recording of the additional recording type 2 (user B): light blue

  In this way, the lightness of the background color of the older (preceding) additional writing is lowered, the newer (follower) additional writing background color is applied and output, and the background varies depending on the type of additional writing. By applying the color, output information that enables the type of additional writing to be determined is generated.

  With these display outputs, the user can accurately grasp not only the original information and the additional information but also the order of the additional information and the type of additional information.

  Note that this processing example is also a processing example using three additional writing documents that have been subjected to three different additional writing processes, but the same processing can be performed for any number of additional writing documents of two or more times.

  FIG. 4 shows an example of setting colors used in this processing example. FIG. 4 (1) shows an example of colors set according to the type of additional write information. For example, the color to be set according to the type of additional recording information, such as [Red] for the type A of additional recording information, [Blue] for the type B, [Green] for the type C, [Purple] for the type D, etc. Make it different.

  Furthermore, as shown in FIG. 4 (2), even in the case of the same type of postscript information, the order in which the postscript was executed can be determined by changing the saturation or brightness according to the postscript history. And

  In the output information described in the processing examples of FIGS. 2 and 3, the setting of changing the saturation of the additional information itself or the brightness of the background color of the additional information to indicate the history of the additional information. The brightness of itself may be changed, or a setting for changing the saturation of the background color of the postscript information may be used. In addition, it is good also as a structure which changes lightness and saturation together.

  Next, the configuration and processing of the image processing apparatus of the present invention will be described with reference to FIG. FIG. 5 is a block diagram showing the configuration of the image processing apparatus 200 of the present invention. The image processing apparatus 400 includes an image input unit 410, a change region determination unit 420, a change region history determination unit 430, a change region type determination unit 440, a change region color conversion unit 450, and an image / data output unit 460. Note that the change area type determination unit 440 may be omitted.

  The image input unit 410 inputs image data to be analyzed, for example, an additional document, or an original document and an additional document. These data are input from a database such as a scanner or a hard disk. Note that the input document includes various data such as image data, text data, and a document including attribute data such as layout information.

  Each document data input to the document input unit 410 is input to the change area determination unit 420. The change area determination unit 420 selects the image data necessary for extracting the additional write information and executes a process for acquiring the additional write information.

Additional information can be extracted by the change area determination unit 420 and various processing modes.
(Processing example 1)
The first additional information is extracted from the difference between the original document and the first additional document, the first additional information and the second additional information are extracted from the difference between the original document and the second additional document, and the original document and the third additional document are extracted. The first to third additional write information can be extracted from the difference between the first additional write information, the second additional write information, and the third additional write information from the difference between the extracted information.

(Processing example 2)
Alternatively, the first additional information is extracted from the difference between the original document and the first additional document, the second additional information is extracted from the difference between the first additional document and the second additional document, and the second additional document and the third additional document are extracted. It is good also as a structure which extracts 3rd postscript information from the difference with a document.

  Further, even if there is no original, depending on the recording process format of the additional information, the additional information can be extracted only from the additional document.

A detailed configuration of the change area determination unit 420 will be described with reference to FIG. As shown in FIG. 6, the processing configuration of the change area determination unit 420 is as follows.
(A) If the original document or image exists,
(B) If the original document or image does not exist,
In these cases (a) and (b), different processing configurations are used.

  First, (a) a case where an original document or an image exists will be described. When the original document or image exists, the additional information is extracted by extracting the difference between the original document image and the additional document image.

  First, the image information alignment unit 421 executes alignment processing of two images of the original document image and the postscript document image. Various existing methods can be applied as a method for aligning the two images. Basically, similar feature regions are extracted from each image, and similar feature regions are associated with each other. Based on the corresponding feature region, the amount of deviation between the two images is calculated, and based on the amount of deviation, alignment is performed while correcting the image data.

  The image information difference calculation means 422 is a process for extracting the difference between the original document image data and the additionally-written document image data that have been aligned. This process is the same as the conventional process. For example, the presence / absence of a difference between the coordinate position (x, y) of the original document image data and the corresponding coordinate position (x, y) of the postscript document image data is detected. Only when data exists, data extraction is executed with this as additional information.

  The change area output unit 423 selects and outputs an area including the additional recording information extracted by the image information difference calculation unit 422. This region extraction is performed in accordance with a predetermined region selection algorithm such as pixel extraction in which difference information exists or a rectangular region including difference pixels. The area data including the extracted difference image is output to the change area history determination unit 430.

The change area determination unit 420 executes these processes in any one of the above-described processing examples 1 and 2 to distinguish and extract the additional write information 1 to 3.
In the case of the processing example 1, the original document and the first additional document are input, the first additional information is extracted from the difference, and the first additional information and the second additional information are calculated from the difference between the original document and the second additional document. The information is extracted, the first to third additional information is extracted from the difference between the original document and the third additional document, and the first additional information, the second additional information, and the third additional information are obtained from the difference between the extracted information. Can be determined.

  In the case of the above processing example 2, the first additional information is extracted from the difference between the original document and the first additional document, and the second additional information is extracted from the difference between the first additional document and the second additional document. The third additional information is extracted from the difference between the second additional document and the third additional document.

  Next, the processing of the change area determination unit 420 when there is no original document or image in FIG. 6B will be described. When the original document or image does not exist, it is not possible to extract the difference between the original document image and the postscript document image. Therefore, a process of extracting data that is determined to be postscript information from only the postscript document is executed. Here, it is assumed that the additional recording information is recorded in a color different from the recording color of the original document, and the additional recording information is extracted by color determination. Specifically, for example, when the original document is monochrome print data, and the additional writing information included in the additional writing document is a color other than black (for example, a color such as red or blue), the non-black color data is added from the additional writing document. Extraction is performed by extracting.

  First, the used color extraction unit 425 performs image analysis (color analysis) of the additional writing document, and extracts colors used in the additional writing document. For example, analysis results such as black, red and blue can be obtained.

  Next, the changed color information input unit 426 inputs color information to be identified as additional write information. For example, color information (red or red and blue) is input by an operator. For example, color information other than black may be set as additional information without inputting color information by the operator. In the case of this configuration, it is not necessary to input color information from the outside.

  The change area output unit 427 performs the additional writing based on the image analysis (color analysis) result of the additional writing document of the used color extracting unit 425 and the color information input by the changed color information input unit 426 or preset color information. Extract pixels with specified color from document. Alternatively, a rectangular area including pixels having a designated color is extracted. The changed area output unit 427 performs changed area extraction according to a predetermined area selection algorithm. The extracted area data is output to the change area history determination means 430.

  The change area history determination unit 430 determines the generation order of the additional information extracted by the change area determination unit 420, that is, the additional information generation history. The generation history of the postscript information can be determined based on the storage time information of each of the postscript documents 1 to n. In the image processing apparatus, the scanner capture date / time information of each additional document is stored in the storage unit as attribute information of each additional document, and the generation order of the additional information of each additional document can be determined based on the date / time information. .

  Alternatively, the additional writing documents 1 to n may be arranged in the order in which the additional writing information increases, and the generation order of the additional writing information may be determined. That is, it is possible to determine that the additional writing document with less additional writing information recorded in the additional writing document is the preceding additional writing document, and based on this determination, the generation order of each additional writing information is determined.

  The change area type determination unit 440 is different in the type of postscript information, that is, the postscript executed by different users, the comment and the corrected text, or the postscript color as in the processing example described above with reference to FIG. A process of detecting additional information belonging to the category is executed. This is because, for example, in the process example described with reference to FIG. 3, when additional information is recorded in advance on a document, additional writing for each type is performed in different modes (for example, different colors) depending on the type of additional information to be recorded. It becomes effective when is executed.

The change area type determination means 440 determines the type (category) for each additional write information extracted from each additional write document. For example, in the processing example of FIG.
(A) Category 1: Addition of user A made in red (b) Category 2: Addition of user B made in blue These two types of additional information determination processing are executed.

  Next, processing of the change area color conversion unit 450 will be described. As described above with reference to FIG. 4, the change area color conversion unit performs output color setting in which the output color mode is changed according to the history of additional write information or the history and type of additional write information. That is, as shown in FIG. 4A, the color set as the additional information or the background color of the additional information is changed according to the type of additional information. For example, the color to be set according to the type of additional recording information, such as [Red] for the type A of additional recording information, [Blue] for the type B, [Green] for the type C, [Purple] for the type D, etc. Make it different. Further, as shown in FIG. 4 (2), the saturation or brightness of the color set as the background color of the additional recording information or the additional recording information is changed according to the additional recording history even for the same type of additional recording information.

  As described above, the change area color conversion unit 450 performs color setting in a different manner according to the type and history of the additional information. This process makes it possible to determine the order and type of additional writing. As described above, in the output information described in the processing examples of FIGS. 2 and 3, the saturation of the postscript information itself or the lightness of the background color of the postscript information is changed to indicate the history of the postscript information. Although it is set, the lightness of the postscript information itself may be changed, or the background color saturation of the postscript information may be changed. In addition, it is good also as a structure which changes lightness and saturation together.

  The image / data output unit 460 generates output information according to the color information set by the change area color conversion unit 450, and performs print output or display output. This output information corresponds to the output information 224 described above with reference to FIG. 2 or the output information 234 described with reference to FIG.

That is, in the processing example of FIG. 2, the output information is such that only the additional recording history can be grasped, and the display color of the additional recording information is different from the original information (for example, black) in order to distinguish the additional recording information from the original information (for example, black). For example, it is displayed as blue), and further, the display is performed by changing the saturation of the additional information output in blue according to the additional recording history. In particular,
(A) First additional note: blue with low saturation,
(B) Second additional note: Blue in saturation,
(C) Third note: Highly saturated blue,
Output information of such setting is output. Alternatively, the display is performed by changing the brightness of the background color of the additional information output in blue according to the additional recording history. In particular,
(A) Background of the first postscript: Blue with low brightness,
(B) Background of the second postscript: Blue in lightness,
(C) Background of the third postscript: light blue,
By executing such display processing, the user can accurately grasp not only the original information and the additional information but also the order of the additional information.

Further, as described with reference to FIG. 3, it is possible to generate output information in which not only the generation history of additional information but also the type of additional information can be grasped. That is, in order to make it possible to distinguish the postscript information from the original information, the display color of the postscript information is set as a color (for example, red and blue) different from the original information (for example, black), and the type of the postscript information (for example, user or postscript information) Set different colors according to the content. Further, for the same type of additional writing, a display in which the saturation of the additional information is changed according to the additional recording history is performed. In particular,
(A) First additional recording of additional recording type 1 (user A): red with low saturation,
(B) Second additional recording of additional recording type 1 (user A): red in saturation,
(C) 3rd additional recording of additional recording type 1 (user A): red with high saturation,
(D) First additional recording of additional recording type 2 (user B): blue with low saturation,
(E) Second additional recording of additional recording type 2 (user B): blue in saturation,
(F) Third additional recording of additional recording type 2 (user B): high saturation blue,
Output information of such setting is output. Alternatively, a display in which the brightness of the background color of the additional writing information is changed according to the additional recording history is performed. In particular,
(A) Background of the first additional recording of the additional recording type 1 (user A): red with low brightness,
(B) Background of second additional recording of additional recording type 1 (user A): red in lightness,
(C) Background of the third postscript of the postscript type 1 (user A): red with high brightness,
(D) Background of the first additional recording of the additional recording type 2 (user B): light blue,
(E) Background of second additional recording of additional recording type 2 (user B): light blue
(F) Background of the third additional recording of the additional recording type 2 (user B): light blue
Output information of such setting is output.

  With these display outputs, the user can accurately grasp not only the original information and the additional information but also the order of the additional information and the type of additional information.

  Next, a processing sequence executed by the image processing apparatus of the present invention will be described with reference to a flowchart shown in FIG. In step S101, image data stored in a storage unit inside the image processing apparatus or a scanned image is input. This is processing in the image input means 410 shown in FIG. Image data to be analyzed, for example, a postscript document, or an original document and a postscript document are input.

  Step S102 is a process of the change area determination means 420 shown in FIG. The change area determination unit 420 is one of (a) the process when the original document or image exists, or (b) the process when the original document or image does not exist, described with reference to FIG. Execute.

  For example, (a) as processing when the original document or image exists, alignment between the postscript document image data and the original document image data is executed, a difference between these image data is calculated, and a pixel in which the difference information exists Alternatively, an area including pixels is determined as a change area, that is, a recording area for additional information.

  When the original document or image does not exist, it is not possible to extract the difference between the original document image and the postscript document image. Therefore, a process of extracting data that is determined to be postscript information from only the postscript document is executed. In other words, the postscript information is extracted by performing image analysis (color analysis) of the postscript document and extracting the color used in the postscript document.

  Step S103 is a process of the change area history determination means 430 shown in FIG. In step S103, the change area history determination unit 430 analyzes the change history of the change area. That is, the generation order of the additional writing information extracted by the change area determination unit 420, that is, the additional writing information generation history is determined. As described above, the generation history of the postscript information is determined based on the storage time information of each of the postscript documents 1 to n. Alternatively, the additional writing documents 1 to n are arranged in the order in which the additional writing information increases, and the generation order of the additional writing information is determined.

Step S104 is a process of the change area type determination means 440 shown in FIG. The change area type determination unit 440 adds the type of additional information, that is, the additional writing belonging to different categories, such as additional writing executed by different users as in the processing example described above with reference to FIG. Detect information. However, this is because, for example, in the processing example described with reference to FIG. 3, when additional information is recorded in advance on a document, it is classified according to type in different modes (for example, different colors) depending on the type of additional information to be recorded. This process is executed when the additional recording is executed, and may be omitted.
In the processing example of FIG.
(A) Category 1: Additional writing of user A in red (b) Category 2: Additional writing of user B in blue These two types of additional information determination processing are executed.

  Step S105 is processing of the change area color conversion means 450 shown in FIG. The change area color conversion means performs output color setting in which the aspect of the output color is changed according to the history of additional information or the history and type of additional information. That is, as shown in FIG. 4A, the color set as the additional information or the background color of the additional information is changed according to the type of additional information. For example, the color to be set according to the type of additional recording information, such as [Red] for the type A of additional recording information, [Blue] for the type B, [Green] for the type C, [Purple] for the type D, etc. Make it different. Further, as shown in FIG. 4 (2), the saturation or brightness of the color set as the background color of the additional recording information or the additional recording information is changed according to the additional recording history even for the same type of additional recording information.

  Step S106 is processing of the image / data output means 460 shown in FIG. The image / data output unit 460 generates output information according to the color information set by the change area color conversion unit 450, and performs print output or display output. This output information corresponds to the output information 224 described above with reference to FIG. 2 or the output information 234 described with reference to FIG. In addition, it is good also as a setting which includes the guide information of the setting color shown previously demonstrated in FIG. 4 in output information so that a user may understand easily.

  By presenting such output information, the user can grasp not only the original information and the additional information but also the order of the additional information. Furthermore, in the processing example in which step S104 is executed, it is possible to accurately grasp the type of additional writing.

  Finally, with reference to FIG. 8, a hardware configuration example of the image processing apparatus of the present invention will be described. The configuration illustrated in FIG. 8 is an image processing apparatus configured by, for example, a PC, and illustrates a hardware configuration example including a data reading unit 531 such as a scanner and a data output unit 532 such as a printer.

  A CPU (Central Processing Unit) 501 performs various data processing described in the above-described embodiments, that is, a change area determination process, a change area history determination process, a change area type determination process, a change area color conversion process, an output information generation process, and the like. It is a control part which performs the process according to the computer program which described the execution sequence of each process of these.

  A ROM (Read Only Memory) 502 stores programs used by the CPU 501, calculation parameters, and the like. A RAM (Random Access Memory) 503 stores programs used in the execution of the CPU 501, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 504 including a CPU bus.

  The host bus 504 is connected to an external bus 506 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 505.

  A keyboard 508 and a pointing device 509 are input devices operated by the user. The display 510 includes a liquid crystal display device, a CRT (Cathode Ray Tube), or the like, and displays various information as text or image information.

  An HDD (Hard Disk Drive) 511 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 501 and information. The hard disk stores the input original document and additional document data. Further, various computer programs such as various data processing programs are stored.

  The drive 512 reads data or a program recorded on a removable recording medium 521 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out from the interface 507 and the external bus 506. , And supplied to the RAM 503 connected via the bridge 505 and the host bus 504. The removable recording medium 521 can also be used as a data recording area similar to a hard disk.

  The connection port 514 is a port for connecting the external connection device 522 and has a connection unit such as USB or IEEE1394. The connection port 514 is connected to the CPU 501 and the like via the interface 507, the external bus 506, the bridge 505, the host bus 504, and the like. The communication unit 515 is connected to a network and executes data communication processing with the outside. The data reading unit 531 executes document reading processing, and the data output unit 532 executes document data output processing.

  Note that the hardware configuration example of the image processing apparatus shown in FIG. 8 shows one example of the apparatus, and the image processing apparatus of the present invention is not limited to the configuration shown in FIG. Any configuration capable of executing the process may be used.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

  The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, or a semiconductor memory. It can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. The computer can receive the program transferred in this manner and install it on a recording medium such as a built-in hard disk.

  Note that the various processes described in the specification are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Further, in this specification, the system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to being in the same casing.

  As described above, according to the configuration of the present invention, in the image processing apparatus that executes the extraction and presentation process of additional write information, the change area including the pixel area determined as the additional write information is extracted, and the change area is further changed to the change area. A history in which the corresponding postscript information is generated, that is, output information that can discriminate the postscript history is generated. Specifically, for example, since the color to be applied to the postscript information itself or the background of the postscript information is set to different brightness or saturation depending on the postscript history, the user can record the history of the postscript information based on the output information. Can be determined. Furthermore, in the configuration example in which a different color is set according to the type of additional write information, for example, for each additional write user or the additional write type such as additional write content, the additional write history and the additional write type can be easily determined.

It is a figure explaining the outline | summary of the process to which the image processing apparatus of this invention is applied. It is a figure explaining the outline | summary and document of a process to which the image processing apparatus of this invention is applied. It is a figure explaining the outline | summary and document of a process to which the image processing apparatus of this invention is applied. It is a figure explaining the output aspect of the postscript information which the image processing apparatus of this invention performs. It is a figure explaining the structure of the image processing apparatus of this invention. It is a figure explaining the process which the change area determination means of the image processing apparatus of this invention performs. It is a figure which shows the flowchart explaining the process sequence which the image processing apparatus of this invention performs. It is a figure explaining the hardware structural example of the image processing apparatus of this invention.

Explanation of symbols

11 Original document 12 First additional document 13 First additional information 14 Second additional document 15 Second additional information 20 Output data 21 First additional information 22 Second additional information 101 Information processing device 102 Printer 103 Scanner 200 Image processing device 201 Original Documents 221 to 223 First to third additional documents 224 Output information 230 Original document 231 to 233 First to third additional documents 234 Output information 400 Image processing apparatus 410 Image input means 420 Change area determination means 421 Image information alignment means 422 Image information Difference calculation means 423 Change area output means 425 Used color extraction means 426 Change area output means 430 Change area history determination means 440 Change area type determination means 450 Change area color conversion means 460 Image / data Output means 501 CPU (Central Processing Unit)
502 ROM (Read-Only-Memory)
503 RAM (Random Access Memory)
504 Host bus 505 Bridge 506 External bus 507 Interface 508 Keyboard 509 Pointing device 510 Display 511 HDD (Hard Disk Drive)
512 drive 514 connection port 515 communication unit 521 removable recording medium 522 external connection device 531 data reading unit 532 data output unit

Claims (15)

A change area determination means for executing a determination process of a change area determined as additional write information based on a document image;
Input the determination result information of the change area determination means, change area history determination means for determining the additional writing history of additional information,
Change area color conversion means for inputting the determination result information of the change area history determination means and performing different display mode setting processing according to the additional writing history as an output mode corresponding to each of the determined additional writing information;
Output means for generating output information according to the display mode corresponding to the additional information set in the change area color conversion means;
An image processing apparatus comprising: The image processing apparatus further includes:
Based on the document image, it has a change area type determination means for determining the type of the additional information,
The change area color conversion means includes:
The image processing apparatus according to claim 1, wherein the output mode corresponding to each additional recording information is configured to perform different display mode setting processing according to the additional recording history and the additional recording type. The change area determination means includes
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to execute a change area determination process including a pixel area determined as additional information based on a difference between a plurality of input image data. The change area determination means includes
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to execute a determination process of a change area including a pixel area determined as additional write information based on color information included in one input image data. The change area color conversion means includes:
As a different display mode according to the additional recording history, a display mode corresponding to the additional recording information in which at least one of the saturation and the brightness set in at least one of the background color of the additional recording information itself or the additional recording information is changed according to the additional recording history The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to perform setting processing. The change area color conversion means includes:
As a different display mode according to the type of postscript information, the postscript information in which at least one of saturation or lightness set in at least one of the background color of the postscript information itself or the postscript information is changed according to the type of the postscript information The image processing apparatus according to claim 2, wherein the image processing apparatus is configured to perform a corresponding display mode setting process. The output means includes
The image processing apparatus according to claim 1, wherein the image processing apparatus is configured to generate output information including guide information indicating a meaning of a display mode of the additional recording information. In the image processing apparatus, an image processing method for generating output information that enables determination of a postscript history of postscript information,
In the change area determination means, a change area determination step for executing a change area determination process determined as additional information based on the document image;
In the change area history determination means, input the determination result information of the change area determination means, a change area history determination step for determining the additional write history of the additional information,
In the change area color conversion means, the determination result information of the change area history determination means is input, and a change area for performing a different display mode setting process according to the additional writing history as an output mode corresponding to each of the determined additional writing information A color conversion step;
In the output means, an output step for generating output information in accordance with the display mode corresponding to the additional information set in the change area color conversion means;
An image processing method comprising: The image processing method further includes:
The change area type determination means includes a change area type determination step for determining the type of the additional recording information based on the document image.
The change area color conversion step includes:
9. The image processing method according to claim 8, wherein the output mode corresponding to each additional recording information is a step of performing a setting process of a different display mode according to the additional recording history and the additional recording type. The change area determination step includes:
The image processing method according to claim 8, wherein the image processing method is a step of executing a determination process of a change area including a pixel area determined as additional write information based on a difference between a plurality of input image data. The change area determination step includes
9. The image processing method according to claim 8, which is a step of executing a determination process of a change area including a pixel area determined as additional write information based on color information included in one input image data. The change area color conversion step includes:
As a different display mode according to the additional recording history, a display mode corresponding to the additional recording information in which at least one of the saturation and the brightness set in at least one of the background color of the additional recording information itself or the additional recording information is changed according to the additional recording history The image processing method according to claim 8, wherein the setting process is a step. The change area color conversion step includes:
As a different display mode according to the type of postscript information, the postscript information in which at least one of saturation or lightness set in at least one of the background color of the postscript information itself or the postscript information is changed according to the type of the postscript information The image processing method according to claim 9, which is a step of performing a corresponding display mode setting process. The output step includes
9. The image processing method according to claim 8, which is a step of generating output information including guide information representing a meaning of a display mode of the additional information. In the image processing apparatus, a computer program that executes image processing for generating output information that enables determination of a postscript history of postscript information,
A change area determination step for executing a change area determination process determined as additional information based on the document image in the change area determination means;
In the change area history determination means, input the determination result information of the change area determination means, a change area history determination step for determining the additional write history of the additional information,
In the change area color conversion means, the change result information of the change area history determination means is input, and a change is made to perform a different display mode setting process according to the additional writing history as an output mode corresponding to each of the determined additional writing information An area color conversion step;
In the output means, an output step for generating output information in accordance with the display mode corresponding to the additional information set in the change area color conversion means;
A computer program for executing