JP2011141599A - Image processing method, image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display writing information related to image data when displaying the image data included in an image data group. <P>SOLUTION: A first image data group and a second image data group are compared, and a first additional object is extracted. A second additional object of the image data projected before projection of target image data that are image data from which the first additional object is extracted. The first additional object and the second additional object are compared, and it is decided whether or not an object included in the first additional object is a new object. When it is the new object, the first additional object is added such that the first additional object is displayed in the target image data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing apparatus, and a program.

  In recent years, the use of digital cameras has expanded due to their high performance, and office cameras have come to be used for capturing conference records. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for capturing a conference record written on a whiteboard with a video camera, separating the content written on the whiteboard from a photographed image, and storing it as electronic data.

  In addition, the technique disclosed in Patent Document 2 is an image obtained by photographing a screen in which writing is performed on a document image projected by a projector, and handwriting obtained by photographing the screen using a projector projected image as a white image. The image is linked and stored. As a result, the information written on the screen and the document image can be linked and stored.

JP 2005-253067 A JP 2008-152766 A

  In the conventional technique, when an image written on an image projected on a screen is photographed by a projector, all entries of the photographed image are recognized and processed as entry information for the original image of the photographed image. It was.

  However, the image projected by the projector and the entry information on the screen are different. For this reason, there is a considerable phenomenon of mismatch between the projected image and the entry on the screen, such as the entry information for the projected image before switching, even though the projected image has been switched. Was.

  An image processing apparatus according to the present invention includes a first image data group and an image data group obtained by capturing an image written on an image obtained by sequentially projecting each of the image data constituting the first image data group on a screen. Means for comparing the second image data group, a means for extracting the writing information as a first additional object based on the comparison result, and the first additional object of the second image data group. Means for obtaining a second additional object extracted from an image projected before the image extracted is compared with the first additional object and the second additional object; Means for determining whether an object included in the object is a new object or the same object; and the determined new object and the same object. Means for determining a respective storage location of the-objects, characterized in that it comprises a means for storing the new object and the same object each having the acquired storage destination described above decision.

  According to the present invention, it is possible to provide an image processing apparatus, an image processing method, and a program capable of displaying writing information related to image data when displaying image data included in the image data group. it can.

1 is a diagram illustrating a system configuration of Embodiment 1. FIG. FIG. 6 is a diagram illustrating state transition of image data according to the first exemplary embodiment. 3 is a flowchart showing processing of Example 1; FIG. 6 is a diagram illustrating image comparison processing according to the first exemplary embodiment. It is a figure which shows the additional object in the previous image data. It is a flowchart which shows an additional object comparison process. It is a figure which shows the comparison process of the position and size of an additional object. It is a figure which shows the comparison process of the density of an additional object. It is a figure which shows the comparison process of the density distribution of an additional object. It is a figure which shows the storing process of an additional object. It is a figure which shows the display switching process of an additional object. 12 is a flowchart illustrating processing for determining the same object addition destination according to the second embodiment. It is a figure which shows the additional object determination result of Example 2. FIG. It is explanatory drawing of the object influence degree of Example 2. FIG. It is a figure which shows the output result in Example 2. FIG. 14 is a flowchart illustrating processing for determining a new object addition destination according to the third embodiment.

[Example 1]
FIG. 1 is a system configuration example according to an image processing apparatus for carrying out a first embodiment of the present invention. Reference numeral 101 denotes a CPU that executes a program and controls each component of the system. Reference numeral 102 denotes a storage device that stores image data, programs, and the like to be processed in the present embodiment. Reference numeral 103 denotes an input device for inputting image data to be processed in the present embodiment. The input device 103 includes a digital camera, a data input terminal, and the like. Reference numeral 104 denotes an output device for outputting an image processed in the present embodiment to the outside. Examples of the output device include a projector and a data output terminal. Reference numeral 105 denotes a ROM (Read Only Memory). Reference numeral 106 denotes a RAM (Random Access Memory). Programs stored in the storage device 102, the ROM 105, and the like are expanded in the RAM 106 and executed by the CPU 101.

  A storage device 102 mounted with a hard disk or the like stores a first image data group composed of a plurality of image data. The CPU 101 sequentially outputs the output target images to a screen or the like by the image output device 104 such as a projector while switching the output target images included in the first image data group stored in the storage device 102.

  The target image projected on the screen is read by the image input device 103 such as a digital camera together with the writing information directly written on the projected screen. The writing information is added to the read image data by the execution of the program by the CPU 101, and a plurality of image data including the image data is stored again in the storage device as the second image data group.

FIG. 2 is a diagram illustrating the state transition of the image information in the conference according to the first embodiment of the present invention.
Target image data 1 (201) included in the first image data group is projected on the screen as a projected image 11 (202) by an image output device 104 such as a projector.

  Writing information 1 is written on the screen on which the projection image 11 (202) is projected, and a projection image 12 (203) is created.

  An image on which the projection image 12 (203) is projected is captured as a conference record by the image input device 103 such as a digital camera, and the image data 1 projected on the screen 11 (202) and the writing information 1 are combined. The photographed image data 1 (204) photographed in this way is created.

  Next, the projection image is switched. The projection image by the image output device 104 is switched from the image data 1 (201) to the image data 2 (205), and the projection image 21 (206) is projected on the screen.

  Writing information 2 is written on the screen on which the projection image 21 (206) is projected, and a projection image 22 (207) is created.

  An image on which the projected image 22 (207) is projected is captured by the image input device 103 as a conference record, and the image information 2 (206) projected on the screen is combined with the writing information 1 and the writing information 2. The photographed image data 2 (208) photographed in this way is created.

  An image data group including the captured image data 1 (204) and the captured image data 2 (208) is stored in the storage device 102 as a second image data group.

  FIG. 3 is a process flowchart of the embodiment of the present invention. With reference to FIG. 3, details of processing when the system according to the present embodiment performs the processing shown in FIG. 2 will be described.

  In step S <b> 301, the system inputs the target image data and the captured image data that have been projected on the screen as input data. Here, the case where the target image data 2 (205) and the currently captured image data 2 (208) in FIG. 2 are used as the input image data of this embodiment will be described.

  In S302, the CPU 101 compares the first image data group and the second image data group and extracts the difference. That is, the target image data included in the first image data group that is input image data is compared with the captured image data included in the second image data group, and a difference image is extracted.

  FIG. 4 is a conceptual diagram of comparison processing of input image data.

  4-A image data is electronic data, while 4-B is captured image data. For this reason, in addition to information 4-B3 and 4-B4 written to the user, as shown in 4-B1, noise on the screen and noise generated during shooting are included in the 4-B captured image data. Exists. In addition, since the original image data is electronic data, edges such as lines are clear as shown in 4-D, whereas in the captured image data, edges are blurred due to blurring or blurring as shown in 4-E. End up.

  Therefore, when image differences are extracted, as shown in a 4-C difference image, differences due to noise such as 4-C1 and differences due to line blurring and blurring such as 4-C2 are extracted. . A differential image extraction method for write information is realized by Japanese Patent Application Laid-Open No. 2007-41832.

  In S303, the CPU 101 extracts write information as an additional object (first additional object) from the difference image. Here, the first additional object includes one or more objects. If the image data is not written, no additional object is extracted. The same applies to the second additional object described later.

  In the image comparison performed in S <b> 302, even small differences such as noise and blur are acquired. Therefore, in the additional object extraction processing, in this embodiment, the difference generated around the target image data or the size of the difference data is equal to or smaller than the threshold value. (Noise removal process) is ignored. As a result, as shown in 4-F, an additional object that is a difference by writing to the current target image data can be acquired from the 4-C difference image.

  In step S304, the CPU 101 acquires an additional object (second additional object) extracted from an image processed before the currently processed image (that is, an image projected before the image to be processed is projected). To do. That is, also in the image comparison between the target image data (201) of the preprocessed image and the captured image data (204), image comparison and additional object extraction processing are performed as in S302 to S303, as shown in FIG. Additional objects with different differences are extracted.

  In S305, the CPU 101 compares the additional objects acquired in S303 and S304.

  FIG. 6 is a flowchart of the additional object comparison process. Details of the additional object comparison process will be described using this flowchart.

  In S601, if the additional object is a multi-valued image, a binarization process is performed in which the additional object is black and the background is white.

  In S602, the circumscribed rectangle of the additional object is acquired. The circumscribed rectangle is expressed by position and size. The minimum value (Xmin) of the X coordinate where the pixel of the object exists, the minimum value (Ymin) of the Y coordinate, the maximum value (Xmax) of the X coordinate, and the maximum value (Ymax) of the Y coordinate are acquired. Further, a circumscribed rectangle is expressed by the coordinate value (Xmin, Ymin) at the upper left of the object, the pixel width (Xmax−Xmin + 1) of the object, and the pixel height (Ymax−Ymin + 1) of the object.

  In step S603, the positions of the additional objects are compared. The upper left position (X, Y) of the circumscribed rectangle is compared from the upper left (image origin) of the image.

  The additional object (A1) of the additional object data (7-A) extracted from the photographed image data 1 of FIG. 7 is compared with the additional object (B1) of the additional object data (7-B) of the current image.

  First, the difference between the positions of the additional object (A1) and the additional object (B1) is acquired. The difference between each position X (7-A1) -X (7-B1), Y (7-A1) -Y (7-B1) is calculated, and the difference between the threshold values X (T) and Y (T) is greater than If there is, it is determined as a mismatch. Since X (T)> | X (7-A1) -X (7-B1) |, Y (T) <| Y (7-A1) -Y (7-B1), the additional object (A1) It is determined that the additional object (B1) is a different object.

  Next, the difference between the positions of the additional object (A1) and the additional object (B2) is acquired. X (7-A1) -X (7-B2), Y (7-A1) -Y (7-B2) are calculated, and if they are equal to or greater than the respective threshold values X (T) and Y (T), they are regarded as inconsistent. judge. In this example, X (T)> | X (7-A1) -X (7-B2) |, Y (T) <| Y (7-A1) -Y (7-B2). If it is determined that there is a possibility that the same object exists, S603 is performed.

  In step S604, the object sizes of the objects determined to match in step S603 are compared. If the width (W) and the height (H) are compared as the size of the circumscribed rectangle and the threshold values W (T) and H (T) are less than or equal to each other, there is a possibility of the same object. Judged to be an object. The size difference between the additional object (A1) and the additional object (B2) is calculated, and W (T)> | W (7-A1) −W (7-B2) |, H (T) <| H ( 7-A1) -H (7-B2), and it is determined that there is a possibility of the same object, and S605 is performed.

  In S605, the pixel densities are compared. As shown in FIG. 8, the pixel density (D) = the number of pixels of the additional object (Obj (P)) / the number of pixels of the circumscribed rectangle (R (P)). If the pixel density is compared with the threshold value D (T) or less, it is determined that there is a possibility of the same object. The pixel density of each additional object is calculated. The calculation result is D (T) <| D (A1) −D (B1), and S605 is performed assuming that there is a possibility of the same object. Accordingly, although there is a high possibility that the additional object A1 of (8-A) and the additional object B2 of (8-B) are the same, the additional object of (8-C) determined to have the same position and size. Different data can be identified.

  In S606, the pixel density distributions are compared. As shown in FIG. 9, the image is divided into fine regions, and it is determined whether the difference in pixel density in each region is equal to or less than a threshold value. By comparing pixel density distributions, (9-A) and (9-B) can be determined to be different objects from (9-C), which are determined to be the same.

  As described above, in the processing from S601 to S606, it is possible to determine that the additional object A1 is the same object (the same object) as the additional object B2. Further, it can be seen that the same object for the additional object B1 does not exist in the previous captured image data, and it can be determined that the additional object B1 is an object (new object) that newly appears in the current captured image. That is, according to the processing of S601 to S606, it can be determined whether the object included in the first additional object is a new object or the same object.

  In this embodiment, in order to increase the accuracy of the same determination, the stroke length of the connected pixels, the number of loops, and the like can be used in the determination method for the same object.

  In S306, based on the comparison result of the additional object, it is determined whether it is a new object newly written with the current photographed image data or the same object existing from the previous photographed image data.

  In the object comparison process of S306, the same object addition destination (storage destination) determination process of S308 is performed for the additional object determined to be the same object. In the example shown in FIG. 7, the same object addition destination determination process of S308 is performed on the additional object B2 determined to be the same object as the additional object A1.

  Further, a new object addition destination determination process in S307 is performed on the additional object determined to be a new object. In the example illustrated in FIG. 7, the new object addition destination determination process of S307 is performed on the additional object B1 determined to be a new object.

  In step S307, new object addition destination determination processing is performed. It is determined that the new object is writing to the target image data 2 currently processed (that is, the target image data 2 is the addition destination image data), and is added to the display layer of the target image data 2. It is determined.

  By deciding to add a new object to the display layer in this way, the new object can be added to be displayed in the target image data 2 as will be described later. Further, by adding a new object to the target image data 2, the new object can be used as the second additional object in the object comparison process (S305) performed later on the other image data.

  In S308, the same object addition destination determination process is performed. The same object is an object that also existed in the previous photographed image, and there is a high possibility that entry information has been forgotten to be deleted. Therefore, it is determined that the same object is not written to the currently processed image data 2 and is determined to be added to the non-display layer of the target image data 2.

  By deciding to add the same object to the non-display layer in this way, the same object can be added so as not to be displayed in the target image data 2 as will be described later.

In step S309, additional object storage (addition) processing is performed.
With reference to FIG. 10, a method for storing additional objects will be described.
The target image data (10-A) as the storage destination has a layer that displays object data (display layer) and a layer that does not display object data (non-display layer), and finally overlaps the layer structure (layer structure). To generate display data.

  In the previous process, a new object determined to be added as display data is added to the display layer as shown in 10-B. The same object that is determined to be added as non-display data is added to the non-display layer as shown in 10-C.

  A 10-D output result can be obtained by overlapping a display layer and a non-display layer on the target image data 10-A.

  As shown in FIG. 11, the output image data can be switched between display and non-display of objects for each layer by switching the layers to be displayed.

  In accordance with an instruction from a user interface provided in the image display device, 11-A, which is the optimal display in the present embodiment, and 11-B, which is the display of all objects (displays all additional objects at the time of shooting), can be easily switched. Is possible.

  As described above, according to this embodiment, when displaying image data included in an image data group, it is possible to display only writing information related to the image data. In other words, only information written to an image on which certain image data is projected can be displayed together with the image data.

[Example 2]
In the second embodiment, a method different from the first embodiment will be described with respect to the same object addition destination determination process.

  With respect to the processing shown in FIG. 3, in the present embodiment, the same processing as in the first embodiment is performed until the same object determination processing in S <b> 306.

  FIG. 12 is a flowchart of the same object addition destination determination process representing the process of the second embodiment. A second embodiment will be described with reference to the flowchart of FIG.

  FIG. 13 shows the results obtained by the processing up to S306. The additional object obtained from the previous photographed image data 13-A and the additional object obtained from the current photographed image data 13-B are compared to obtain the same object 1, the same object 2, and the new object 1, respectively. .

  In S1201, the influence degree of the same object on the content included in the previous target image data is calculated. Here, the previous target image data is previously projected image data. The content is an object such as a figure, a table, a character, or a photograph that is included in the target image data in advance, and is an object of the target image data other than the background. The degree of influence is the degree of influence that the same object has on the display of target image data.

  FIG. 14 is a diagram showing the positional relationship between the object and the content. The influence degree of the same object 1 and the content 14A is calculated. The degree of influence is proportional to the distance between the content and the circumscribed rectangle of the object. That is, the influence degree is calculated according to the distance between the content and the object. When the distance equal to or greater than the predetermined threshold value D (T) is separated, the influence degree is determined to be small. An arbitrary value is preset for the threshold value D.

  In this embodiment, it is determined that D (T) = 0, and the degree of influence is large when the content and the circumscribed rectangle are in contact, and the degree of influence is small when they are separated. The method for determining the degree of influence may be dynamically changed according to the size of the content to be compared and the size of the object. The distance of the circumscribed rectangle of the same object 1 as the content 14A is D1, and it is determined that D1> 0 and the influence degree is small. The distance between the circumscribed rectangle of the same object 2 and the content 14A is D2, and it is determined that D2> 0 and the degree of influence is small.

  In step S1202, the degree of influence between the content of the current processing target image data (current target image data) and the same object is calculated. The distance between the circumscribed rectangles of the same object 1 as the content 14B is D3, and it is determined that D3> 0 and the degree of influence is small. The distance between the circumscribed rectangle of the same object 2 and the content 14A is D4, and since the content and the object overlap, it is determined that D4 <0 and the influence degree is large.

  In S1203, the addition destination is determined based on the degree of influence. Since the same object 1 has a small influence on both the previous target image data and the current target image data, it is determined that the same object 1 is an object having a small influence even if it is continuously added. On the other hand, since the same object 2 has a large influence on the current target image data, the same object 2 is determined to be an object that has a large influence when continuously added.

  In S1204, it is determined whether to add an object based on the determination result, and it is determined that the same object 1 having a small influence degree is added to the current target image data so as to be displayed. That is, when the degree of influence is smaller than a predetermined threshold, it is determined that the same object 1 is added to be displayed in the current target image data. Further, it is determined that the same object 2 having a large influence degree cannot be added (or added as non-display).

  FIG. 15 shows a storage result of the additional object in the second embodiment. After the above determination, the same object 1 determined to be able to be added to the target image data of 15-A by the image data storage process of S309 in FIG. 3 is displayed on the display layer (15-B ). The same object 2 determined not to be added is not added to the current target image data (or stored in the non-display layer (15-C)). The output result shown in 15-D is output as the output result.

  In this embodiment, the degree of influence of the same object on the contents of the previous target image data and the current target image data is calculated, and a method for adding (storing) the same object to the current target image data is determined according to the degree of influence. However, it is not limited to this method. Only the influence degree of the same object on the content of the current target image data may be calculated to determine a method of adding the same object to the current target image data.

  As described above, according to the present embodiment, when displaying image data included in an image data group, the same object having a small influence on the content of the image data is added to be displayed on the image data. can do. Further, the same object having a large influence degree can be added to the image data so as not to be displayed.

[Example 3]
In the third embodiment, a method different from that of the first embodiment will be described with respect to the process for determining a new object object addition destination.

  The processing shown in FIG. 3 is the same as that in the first embodiment up to the same object determination processing in step S306 in the present embodiment, and a description thereof will be omitted.

  FIG. 16 is a flowchart of a new object addition destination determination process representing the process of the third embodiment. Embodiment 3 will be described with reference to the flowchart of FIG.

  In step S1601, the type of the input object (new object) is determined. In this embodiment, it is determined whether the input object is a character or a non-character. The object type is determined by taking out the connected pixels of the object and determining that the size of each connected pixel is uniform to some extent and that the pixels are regularly arranged. Further, if the object is recognized with a certain threshold by executing OCR, it is determined as a character object. The determination whether the input object is a character or a non-character is not limited to this method, and can be performed by an arbitrary method.

  In S1602, the relevance of the non-character object with the current target image data is calculated. The degree of relevance is considered to be highest for a slide in which a new target object appears for the first time. However, it is considered that there are many uses for modifying the content projected on the screen as a feature of the non-character object. Therefore, the non-character object has a higher degree of association as the overlap with the content increases, and the degree of association tends to decrease as the distance from the content increases.

The relevance of non-character objects is calculated using the following formula.
Object area (A)
Overlap area of object and content (A1)
Distance between object and content (D)
Relevance (R) = A1 / A (when distance D ≦ 0 (overlap from contact))
Relevance (R) =-D (when distance D> 0)

  In S1603, the relevance of the character object with the current target image data is calculated. In contrast to non-character objects, character objects are often written independently of the content projected on the screen. This is because the character object itself cannot be read because it overlaps the content. Therefore, the relevance level of the character object decreases as the distance from the content increases, and the relevance level decreases as the overlap with the content increases.

The relevance of the character object is calculated using the following formula.
Object area (A)
Overlap area of object and content (A1)
Distance between object and content (D)
Relevance (R) = 1− (A1 / A) (when distance D ≦ 0 (overlap from contact))
Relevance (R) = 1 / D (when distance D> 0)

  Note that the above calculation formula used in the processing of S1602 and S1603 is merely an example, and the relevance is calculated so as to be determined according to the distance and overlap area between the new object and the content included in the current target image data. Just do it. In the case of a non-character object, the relevance may be calculated so as to increase as the overlap with the content increases and to decrease as the distance from the content increases. Further, in the case of a character object, the relevance may be calculated so as to decrease as the distance from the content increases and to decrease as the overlap with the content increases.

  The degree of association between the new object and the previous target image data is also calculated by the same method.

  S1604 is an addition destination target image data determination process. The addition destination image data is determined based on the relevance calculated in S1602 or S1603.

  Here, if (relevance of the previous target image data)> (relationship of the current target image data), the character object is added to the display layer of the previous target image data, and the non-display layer of the current target image data is added. to add. If (relationship of previous target image data) ≦ (relationship of current target image data), a character object is added to the display layer of the current target image data.

  That is, in this embodiment, when the relevance level of the current target image data is equal to or higher than the relevance level of the previous target image data, a character object is added to be displayed in the current target image data. However, the present invention is not limited to this method, and whether or not to add a character object to be displayed in the current target image data is determined depending on whether or not the relevance of the current target image data is greater than a predetermined threshold. good.

  As described above, according to the present embodiment, when displaying image data included in an image data group, a new object having a high degree of association with the image data is displayed together with the image data and has a low degree of association. New objects can be hidden.

(Other embodiments)
The present invention can be applied to a system constituted by a plurality of devices (for example, a computer, an interface device, a reader, a printer, etc.), or can be applied to an apparatus (multifunction device, printer, facsimile machine, etc.) comprising a single device. Is also possible.

  The processing method for storing the program for operating the configuration of the above-described embodiment so as to realize the function of the above-described embodiment in a storage medium, reading the program stored in the storage medium as a code, and executing the program on the computer is also described above. It is included in the category of the embodiment. That is, a computer-readable storage medium is also included in the scope of the embodiments. In addition to the storage medium storing the computer program, the computer program itself is included in the above-described embodiment.

  As such a storage medium, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, and a ROM can be used.

  In addition, the processing is not limited to the single program stored in the above-described storage medium, but operates on the OS in cooperation with other software and expansion board functions to execute the operations of the above-described embodiments. This is also included in the category of the embodiment described above.

101 CPU
102 Storage Device 103 Input Device 104 Output Device 105 ROM
106 RAM

Claims (13)

A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. Means for comparing;
Means for extracting the writing information as a first additional object based on the comparison result;
Means for obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Means for comparing the first additional object with the second additional object to determine whether the object included in the first additional object is a new object or the same object;
Means for determining a storage location of each of the determined new object and the same object;
An image processing apparatus comprising: means for storing the acquired new object and the same object in the determined storage destination.   2. The image processing according to claim 1, wherein the means for determining whether the object included in the first additional object is a new object or the same object is determined by comparing at least a position and a size of each additional object. apparatus.   The means for determining the storage destination of the new object and the same object stores the new object in the display layer of the image data related to the first image data group, and the non-display of the image related to the first image data group The image processing apparatus according to claim 1, wherein the image processing apparatus determines to store in a layer. A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. Means for comparing;
Means for extracting the writing information as a first additional object based on the comparison result;
Means for obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Means for comparing the first additional object with the second additional object to determine whether the objects included in the first additional object are the same object;
Means for calculating the influence of the object determined to be the same object on the content included in advance in the image data related to the first image data group according to the distance between the content and the object;
An image processing apparatus comprising: means for storing the same object so as to be displayed in the image data relating to the first image data group when the calculated influence degree is smaller than a predetermined threshold value. A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. Means for comparing;
Means for extracting the writing information as a first additional object based on the comparison result;
The second additional object extracted from the previous target image data that is the image projected before the current target image data that is the image data from which the first additional object is extracted from the second image data group. Means for obtaining
Means for comparing the first additional object and the second additional object to determine whether the object included in the first additional object is a new object;
Means for calculating a degree of association between the object determined to be the new object and the current target image data according to a distance and an overlapping area between the new object and the content included in the current target image data;
Means for calculating a degree of association between the object determined to be the new object and the previous target image data according to a distance and an overlapping area between the new object and the content included in the previous target image data;
Means for storing the new object to be displayed in the current target image data when the calculated relevance level with the current target image data is equal to or higher than the relevance level with the previous target image data. An image processing apparatus.   In the case where the new object is a non-character object, the means for calculating the degree of association with the current target image data and the degree of association with the previous target image data increases as the overlap with the content increases and the distance from the content increases. The image processing apparatus according to claim 5, wherein the degree of relevance is calculated so as to become lower according to:   When the new object is a character object, the means for calculating the degree of association with the current target image data and the degree of association with the previous target image data decreases as the distance from the content increases, and overlaps with the content. The image processing apparatus according to claim 5, wherein the calculation is performed such that the larger the value is, the lower the value is. A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
Obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Comparing the first additional object with the second additional object to determine whether the object included in the first additional object is a new object or the same object;
Determining a storage location for each of the determined new object and the same object;
Storing the acquired new object and the same object in the determined storage location, respectively. A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
Obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Comparing the first additional object and the second additional object to determine whether the objects included in the first additional object are the same object;
Calculating the degree of influence of the object determined as the same object on the content included in advance in the image data related to the first image data group according to the distance between the content and the object;
An image processing method comprising: storing the same object so as to be displayed in image data relating to the first image data group when the calculated influence degree is smaller than a predetermined threshold value. A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
The second additional object extracted from the previous target image data that is the image projected before the current target image data that is the image data from which the first additional object is extracted from the second image data group. Step to get the
Comparing the first additional object and the second additional object to determine whether the object included in the first additional object is a new object;
Calculating a degree of association between the object determined to be the new object and the current target image data according to a distance and an overlapping area between the new object and the content included in the current target image data;
Calculating the degree of association between the object determined to be the new object and the previous target image data according to a distance and an overlapping area between the new object and the content included in the previous target image data;
Storing the new object to be displayed in the current target image data when the calculated relevance level with the current target image data is equal to or higher than the relevance level with the previous target image data. An image processing method characterized by the above. On the computer,
A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
Obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Comparing the first additional object with the second additional object to determine whether the object included in the first additional object is a new object or the same object;
Determining a storage location for each of the determined new object and the same object;
A program for executing the step of storing the acquired new object and the same object in the determined storage location. On the computer,
A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
Obtaining a second additional object extracted from an image projected before the image from which the first additional object is extracted is projected from the second image data group;
Comparing the first additional object and the second additional object to determine whether the objects included in the first additional object are the same object;
Calculating the degree of influence of the object determined as the same object on the content included in advance in the image data related to the first image data group according to the distance between the content and the object;
When the calculated influence degree is smaller than a predetermined threshold value, a program for executing the step of storing the same object to be displayed in the image data related to the first image data group. On the computer,
A first image data group, and a second image data group that is an image data group obtained by capturing images written on images obtained by sequentially projecting the image data constituting the first image data group onto a screen. A step of comparing;
Extracting the writing information as a first additional object based on the comparison result;
The second additional object extracted from the previous target image data that is the image projected before the current target image data that is the image data from which the first additional object is extracted from the second image data group. Step to get the
Comparing the first additional object and the second additional object to determine whether the object included in the first additional object is a new object;
Calculating a degree of association between the object determined to be the new object and the current target image data according to a distance and an overlapping area between the new object and the content included in the current target image data;
Calculating the degree of association between the object determined to be the new object and the previous target image data according to a distance and an overlapping area between the new object and the content included in the previous target image data;
And storing the new object to be displayed in the current target image data when the calculated degree of association with the current target image data is greater than or equal to the degree of association with the previous target image data. Program for.

Images