JP2014119809A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To update information by attaching paper tags on a screen to be overlapped with one another, in a system capable of electronizing paper tags.SOLUTION: An information processing apparatus includes: holding means for extracting, from a first area included in first image data and having information described therein, the information and holding the information; acquisition means for acquiring second image data including the first area and a second area having information described therein; detection means for detecting whether or not the second area is overlapped on the first area in an image in the second image data; and updating means for updating, when the second area is overlapped on the first area in the image in the second image data, the information extracted from the first area held by the holding means by using the information extracted from the second area.

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program for controlling display of electronic tag data obtained by digitizing a paper tag attached on a screen.

  A method called brainstorming is known as a method for creating new knowledge by a plurality of people. In brainstorming, you write ideas one at a time on cards and share them to diverge the ideas. As a method for converging these ideas, a method called KJ method is known. The KJ method is a method of organizing relationships between groups by repeatedly grouping ideas that are similar to each other by collecting and classifying them. And in this work, discovery of the essence of ideas and creation of new ideas are performed.

  In the brainstorming or KJ method, a sticky sticky note (such as Post-It (registered trademark)) is used as the aforementioned card, and the sticky sticky note is pasted on a board surface such as a white board, and the method is often used. In addition, during the brainstorming process, sticking sticky notes over other sticky notes may be added to the contents of the underlying paper sticky notes or express the intention to overwrite them. .

  As a method for electronically supporting brainstorming, Patent Document 1 proposes a method for recognizing a board surface by discriminating a sticking position and contents of a paper sticky note and digitizing the paper sticky note. This technique discloses recognizing the position of a paper sticky note from an image captured by a digital camera, and storing the recognized position and the image of the paper sticky note in association with each other.

  On the other hand, Patent Document 2 discloses a system that associates an object placed on a screen with an object (such as a window) displayed below the object. For example, when a sticky note is pasted on a window displayed on the screen, it is detected that an object is placed on the window, and the window is displayed using the sticky note by associating the window with the sticky note. It can be controlled (limitation of window movement, etc.).

JP 2006-059020 A JP 2010-146338 A

  When the method of Patent Literature 2 is applied to a method of digitizing a paper sticky note as in Patent Literature 1, the paper sticky attached to the screen may be digitized while corresponding to the object displayed on the screen. Conceivable. In this case, an image (electronic sticky note) obtained by digitizing the attached paper sticky note can be associated with another electronic object (for example, electronic sticky note) that has already been displayed. However, the technique of Patent Document 2 associates a paper sticky note with an electronic object displayed on the screen. When a paper sticky note is superimposed on a paper sticky note, the two cannot be associated with each other. Further, Patent Document 1 discloses grouping a plurality of paper sticky notes, but it does not take into account correcting the description content described on the paper sticky notes with a simple operation.

  In order to solve the above problems, the present invention has the following configuration. That is, an information processing apparatus that extracts information from the image data including the area where the information is described and converts the information into data, and includes the first area including the information included in the first image data. A holding unit that extracts and holds information; an acquisition unit that acquires second image data including the first region; and a second region in which the information is described; and an image of the second image data And detecting means for detecting whether or not the second region overlaps the first region, and in the image of the second image data, the second region overlaps the first region. And updating means for updating information extracted from the first area held by the holding means using information extracted from the second area.

  According to the present invention, when a medium is newly overlaid on a medium whose contents are already registered as digitized data, the electronic data corresponding to the medium on which the information on the overlaid medium is overlaid can be updated. .

The figure which shows the structure of the information processing system in this embodiment. The figure which shows the structure of the server (information processing apparatus) 111. FIG. 3 is a flowchart illustrating a processing flow according to the first embodiment. The figure which shows an example of arrangement | positioning of a tag. The figure which shows the detailed procedure of a tag image extraction process (S302). The figure which shows the data structure 601 of an electronic tag. The figure which shows the structure of the electronic tag data 701 of the paper tag 401. FIG. The figure which shows the structure of the electronic tag data 801 of the paper tag 405. FIG. The figure which shows the structure of the electronic sticky note data 901 of the paper sticky note 401 after additional writing. 9 is a flowchart showing a process flow of Embodiment 2. The figure which shows an example of how to stick a tag. The figure which shows the structure of the electronic tag data 1201 of the paper tag 401 after overwriting. 10 is a flowchart illustrating a process flow according to the third embodiment.

<Example 1>
[System configuration]
FIG. 1 is a diagram showing a configuration of an information processing system in the present embodiment. The information processing system includes a server 111 as an information processing device, a digital camera 113 as a photographing device, and a display device 115. The display device 115 further includes a projector 112 and a whiteboard 114, and an image projected from the projector 112 is displayed on the whiteboard 114. The information processing system according to the present embodiment can be used in conferences such as brainstorming, the KJ method, or the like, but the usage is not limited thereto.

  In the present embodiment, a case where a sticky tag with glue (Post-it (registered trademark), etc.) is pasted on a screen such as a whiteboard will be described, but the present invention is not limited to a sticky tag with glue. . For example, the present invention can be similarly applied to a case where a document or card is pasted using a tape or the like, or a case where the document or card is pasted by a suction means such as a magnet. Further, the material of the sticky note, the document, and the card is not limited to paper, and may be a medium of other materials (plastic, cloth, etc.).

  In the following embodiments, a sticky note or document pasted on a whiteboard (screen) is referred to as a paper sticky note (or paper document) regardless of the material. Further, the sticky note data digitized by photographing or scanning the paper sticky note is referred to as an electronic sticky note (or electronic document) to be distinguished.

  FIG. 2 is a diagram illustrating a configuration of the server (information processing apparatus) 111. The server 111 includes a CPU 201, a RAM 202, a ROM 203, a keyboard I / F 205, a display I / F 206, an external memory I / F 207, and an external I / F 208, which are connected by a system bus 204. A keyboard 209 connected via the keyboard I / F 205 receives various operations from the user. A projector 112 is connected to the display I / F 206, and the server 111 projects an image on the whiteboard 114 via the projector 112. The user can paste a paper sticky note on the whiteboard 114 (screen) on which the projector 112 projects an image.

  A digital camera 113 is connected to the external I / F 208, and the server 111 can capture an image of a paper sticky note photographed by the digital camera 113 and create an electronic sticky note (sticky note data). In the present embodiment, electronic data input by the user using the keyboard 209 is also handled in the same way as an electronic sticky note created from an image of a paper sticky note, and these can be mixed and displayed on the projector 112. To do.

  A CPU (processor) 201 controls the entire server 111. The CPU 201 reads a program stored in the ROM 203 or the external memory 211 into the RAM 202 and executes various processes. That is, the CPU 201 of the server 111 functions as a processing unit (processing unit) that executes each step of a flowchart described later by executing a computer program stored in a computer-readable storage medium. The ROM 203 stores various data including a program for starting the CPU 201. The RAM 202 is used as a work memory.

[Processing flow]
Next, the operation of this system will be described. FIG. 3 is a flowchart showing a processing flow of the present system. The procedure shown in this flowchart is executed by the CPU 201 with a corresponding program stored in one of the ROM 203, RAM 202, and external memory (HDD, etc.) 211 of the server 111.

  In general, brainstorming is performed by writing user ideas on paper sticky notes and pasting them on the whiteboard 114. First, FIG. 4A shows a state where three sticky notes 401 to 403 are pasted on the whiteboard 114. Image data corresponding to the image shown in FIG. 4A is set as first image data. From this state, the description will be made on the assumption that the white board 114 is in a state as shown in FIG. 4B by pasting the paper sticky note 405 so as to overlap the paper sticky note 401. Image data corresponding to the image shown in FIG. 4B is set as second image data. Accordingly, it is assumed that electronic data obtained by digitizing the paper sticky notes 401 to 403 is already stored in the storage unit of the server 111 at the stage where processing starts. In the example of FIG. 4, the area indicated by the paper sticky note 401 is the first area, and the area of the paper sticky note 405 superimposed on the paper sticky note 401 is the second area.

  First, the digital camera 113 that has detected a shooting request from the user takes an image of the whiteboard 114 (S301). Image data captured by the digital camera 113 is stored in the RAM 202 through the external I / F 208 of the server 111. The captured image includes a paper sticky note image and an electronic sticky note image being projected by the projector 112.

  In S302, the server 111 extracts a tag image from the captured image data, and generates electronic tag data from the extracted tag image. By generating this electronic sticky note data, information (character information or the like) indicated on the sticky note is converted into data. The data structure of the electronic sticky note including the sticky note image will be described later.

  FIG. 5 is a diagram showing a detailed procedure of the tag image extraction process (S302). When the CPU 201 of the server executes the program, it functions as the processing units 502 to 505 in FIG. The processing method in each processing unit is not particularly limited, and a general method may be used.

  First, contour extraction processing by the contour extraction unit 502 and color determination processing by the color determination unit 503 are performed on the captured image data 501 captured by the digital camera 113 and stored in the RAM 202 of the server 111. In the contour extraction process, by performing a known edge extraction or contour tracking process, the contour of an object existing in the image (the contour of a paper sticky note, the contour of a character image written on a paper sticky note, etc.) can be extracted. Further, the color of each object (paper tag or character) can be extracted by the color determination process.

  Next, a sticky note area determination process by the sticky note area determination unit 504 is performed. In the sticky note area determination process, an area of a rectangular outline object having a size satisfying a size condition of a paper sticky note assumed in advance is determined as a sticky note area. At this time, if it is configured to determine whether or not a character-size outline is included in the rectangular outline, the sticky note area can be determined more accurately.

  Then, in the sticky note region cutout unit 505, a sticky note region cutout process for cutting out partial images (sticky note images) of the sticky note region corresponding to each of the paper sticky notes is performed, and electronic sticky note data is based on the extracted sticky note image. 506 is generated. At this time, the text data of the character image written in the tag may be obtained by performing OCR processing on the tag image.

  FIG. 6 shows a data structure 601 of the generated electronic sticky note. Each electronic sticky note has a header portion 602 and a data portion 603. The header section 602 includes information on ID, pasting time, pasting state ON / OFF, pasting place, sticky note area, character image area, and sticky note area. The ID is an identifier for identifying each electronic sticky note. The pasting time indicates the time when the corresponding paper sticky note is pasted. The sticking state ON / OFF indicates whether the corresponding paper sticky note is stuck on the whiteboard or peeled off. The sticking location indicates the sticking location of the paper sticky note. The sticky note area indicates the position of the sticky note area on the whiteboard. The character image area indicates a relative position of the character image area to the tag. The sticky note area indicates the size of the sticky note.

  The sticky note area includes a set of coordinates for the upper left, upper right, lower left, and lower right of the sticky note in a coordinate system with the lower left of the whiteboard as the origin. The character image area includes a set of coordinates for the upper left, upper right, lower left, and lower right of the character image in a coordinate system with the origin at the lower left of the tag area.

  The data portion 603 includes image data of a sticky note image cut out from the captured image data 501 captured by the digital camera 113. Note that when the image data is subjected to OCR processing, the data portion 603 further stores OCR data indicating the text of the OCR processing result.

  Specific examples of the data structure of the electronic sticky note are shown in FIGS. Here, electronic sticky note data 701 is generated from the sticky note image of the paper sticky note 401 in FIG. Also, electronic sticky note data 801 is generated from the sticky note image of the paper sticky note 405 in FIG.

  FIG. 7 shows the structure of the electronic sticky note data 701 of the paper sticky note 401. The contents of the header portion 702 of the electronic sticky note data 701 are as follows. The ID is “1”. The pasted time is “10: 0: 0”. Since the corresponding paper sticky note 401 is still attached, the attached state is “ON”. The affixing location is “Conference Room 1”. The sticky note area is (X1, Y2) at the upper left of the tag, (X2, Y2) at the upper right, (X1, Y1) at the lower left, and (X2, Y1) at the lower right. In the character image area, the upper left is (X3, Y4), the upper right is (X4, Y4), the lower left is (X3, Y3), and the lower right is (X4, Y3). The sticky note area is “20”. The contents of the data portion 703 are such that image data is stored with a file name “data1.jpg”, and “ABCD” is stored in the OCR data when OCR processing is performed.

  FIG. 8 shows the structure of the electronic sticky note data 801 of the paper sticky note 405. The contents of the header portion 802 of the electronic sticky note data 801 are as follows. The ID is “2”. The pasted time is “10: 5: 0”. Since the corresponding paper sticky note 405 is still attached, the attached state is “ON”. The affixing location is “Conference Room 1”. The sticky note area is (X5, Y6) at the upper left, (X6, Y6) at the upper right, (X5, Y5) at the lower left, and (X6, Y5) at the lower right. In the character image area, the upper left is (X7, Y8), the upper right is (X8, Y8), the lower left is (X7, Y7), and the lower right is (X8, Y7). The sticky note area is “20”. The contents of the data portion 803 are such that image data is stored with a file name “data2.jpg”, and “EFG” is stored in the OCR data when OCR processing is performed.

  In S303 of FIG. 3, the server 111 determines whether all electronic sticky note data extracted in S302 has been processed. If it is determined that there is other unprocessed electronic sticky note data (YES in S303), the process proceeds to S303. If it is determined that all electronic sticky note data has been processed (NO in S303), server 111 ends this processing flow.

  In S304, the server 111 compares each of the sticky note images included in the electronic sticky note generated in S302 with the electronic sticky note data already registered in the storage device (external memory or RAM) of the server 111 in order. Then, the server 111 collates whether or not electronic sticky note data of the same sticky note image has already been registered. Specifically, the comparison of the pasting positions of each other and the comparison of the sticky image data of each other by a known similar image determination technique are performed, and if both are determined to be the same, it is determined that they have been registered. If it is determined that the sticky note image extracted in S302 is different from the electronic sticky note registered in the storage device of the server 111 based on the collation result in S304 (YES in S305), it is determined that a new paper sticky note has been added to S306. move on. Conversely, if it is determined that the sticky note image extracted in S302 is the same as the electronic sticky note data registered in the storage area of the server 111 (NO in S305), the process returns to S303.

  In S306 to S309, the server 111 determines whether the newly added paper sticky note is superimposed on another sticky note. When it is determined that they are stacked (YES in any of S306 to S309), a sticky note (hereinafter referred to as a lower sticky note) located under the paper sticky note is extracted. In S <b> 306, the server 111 determines that any one of the electronic sticky notes among the coordinates in the upper left of the sticky note area stored in the header part 602 of the electronic sticky note data of the newly added sticky note has been registered in the storage area of the server 111. It is determined whether or not it is included inside the sticky note area. If it is determined that the tag is included (YES in S306), the server 111 determines that the upper left corner of the tag is superimposed on another tag, and the server 111 sets the electronic tag having a tag area including the upper left coordinates inside as the lower tag S311. Proceed to If it is determined that it is not included (NO in S306), the process proceeds to S307.

  In step S <b> 307, the server 111 selects one of the electronic sticky notes from which the upper right coordinates of the sticky note area stored in the header part 602 of the newly added electronic sticky note data are registered in the storage area of the server 111. It is determined whether or not it is included inside the sticky note area. If it is determined that it is included (YES in S307), it is determined that the upper right side of the sticky note is overlaid on another sticky note, and the server 111 uses electronic sticky note data having a sticky note area that includes the upper right coordinates inside as a lower sticky note. The process proceeds to S311. If it is determined that it is not included (NO in S307), the process proceeds to S308.

  In S <b> 308, the server 111 selects one of the electronic sticky notes from among the ones in which the lower left coordinates of the sticky note area stored in the header part 602 of the newly added electronic sticky note data have been registered in the storage area of the server 111. It is determined whether or not it is included inside the sticky note area. If it is determined that the tag is included (YES in S308), it is determined that the lower left corner of the sticky note is superimposed on another sticky note, and the server 111 sets an electronic sticky note having a sticky note area including the lower left coordinates inside as a lower sticky note S311. Proceed to If it is determined that it is not included (NO in S308), the process proceeds to S309.

  In S <b> 309, the server 111 determines that any of the electronic coordinates among the coordinates in the lower right of the sticky note area stored in the header part 602 of the electronic sticky note data of the newly added sticky note has been registered in the storage area of the server 111. It is determined whether or not the tag is included within the tag area of the tag. If it is determined that the tag is included (YES in S309), the server 111 determines that the lower right corner of the tag is superimposed on another tag, and the server 111 stores the electronic tag data having the tag area including the lower right coordinates on the lower side. The process proceeds to S311 as a tag. If it is determined that it is not included (NO in S309), the server 111 determines that the paper sticky note has not been superimposed on another sticky note, and proceeds to S310. In S310, the server 111 registers the electronic sticky note data created based on the paper sticky note image as new electronic sticky note data, and returns to S303.

  If it is determined that the newly added paper sticky note is superimposed on another sticky note (YES in any of S306 to S309), the server 111 in S311 newly adds the paper added to the data portion 603 of the lower sticky note Add the contents of the tag (hereinafter referred to as the above tag). In the assumption of this embodiment, the contents of the data portion 803 of the upper paper sticky note 405 are added to the data portion 703 of the electronic sticky note data 701 of the lower paper sticky note 401, and the electronic sticky note data after the additional writing as shown in FIG. 901 is created. As a result, the contents of the electronic sticky note data held are updated.

  In FIG. 9, the header portion 902 is not changed from the header portion 702 of the electronic sticky note data of the lower sticky note. On the other hand, in each element of the data portion 903, the contents of the data portion 803 of the electronic sticky note data of the upper sticky note are added. In step S311, the sticky note data is added to the server data, and the process returns to step S303.

  As described above, according to the present embodiment, by overlapping a paper sticky note on a sticky note that has already been pasted and whose description content is registered as electronic data, the content of the paper sticky note overlapped is superimposed. Can be added to the electronic data corresponding to the attached paper sticky note.

<Example 2>
In the first embodiment, the method of performing additional writing on the electronic data corresponding to the stacked paper sticky notes by superimposing the paper sticky notes on the sticky notes has been described. A method of updating by switching between appending and overwriting will be described.

  The operation of this system will be described. FIG. 10 is a flowchart showing the flow of processing of this system. Note that the processing in S301 to S310 and S311 is the same as that in the flowchart of FIG. Also in this flowchart, a paper sticky note 405 is newly attached so as to overlap the paper sticky note 401 from a state where three paper sticky notes are attached to the whiteboard 114 as shown in FIG. The description will be made on the assumption that the state shown in FIG.

  When it is determined that the newly added paper sticky note is superimposed on another sticky note (YES in any of S306 to S309), in S1001, the server 111 defines an area shared by the two sticky notes. That is, the part where two sticky notes overlap is specified. For example, when the paper sticky note 405 is overlapped so that the contents of the lower paper sticky note 401 can be seen as shown in FIG. 11A, a portion (1101) represented by shading in FIG. Becomes a shared area. In addition, when the paper sticky note 405 is overlaid so as to cover the contents of the lower paper sticky note 401 so that the contents of the lower paper sticky note 401 cannot be seen as shown in FIG. 11D, they are represented by shading in FIG. The existing part (1103) is a shared area.

  Thereafter, the server 111 calculates the area of the shared region in S1002. Subsequently, in S1003, the server 111 compares the sticky note areas in the header portions of the electronic sticky note data of the upper sticky note and the lower sticky note. As a result of the comparison, if it is determined that the sticky area of the upper sticky note is smaller (YES in S1003), the server 111 sets the sticky note area of the upper sticky note as a reference area in S1004, and proceeds to S1006. If it is not determined that the sticky area of the upper sticky note is smaller (NO in S1003), the server 111 sets the sticky note area of the lower sticky note as a reference area in S1005, and proceeds to S1006. The reference area is used when determining how to superimpose a sticky note by comparing with the area of the shared region calculated in S1002.

  After determining the reference area, the server 111 calculates a value obtained by dividing the area of the shared region by the reference area in S1006, and determines whether the value is smaller than a predetermined threshold. If it is smaller than the threshold value (YES in S1006), the server 111 determines that the upper sticky note is overlapped so that the contents of the lower sticky note can be seen as shown in FIG.

  For example, consider a case where the lower paper sticky note 401 and the upper paper sticky note 405 are overlapped as shown in FIG. 11A and the threshold used in S1006 is “0.5”. As the reference area (since the areas of the two sticky notes are equal), “20”, which is the area of the lower paper sticky note 401, is set in S1005. If the area of the shared area 1101 is calculated to be “8” in S1002, the determination in S1006 is (8/20) <0.5, and the upper sticky note is shifted so that the contents of the lower sticky note can be seen. It is determined that

  Thereafter, the server 111 appends the contents of the upper sticky note to the electronic sticky note data of the lower sticky in S311 and returns to S303.

  On the other hand, when the value obtained by dividing the area of the shared region by the reference area is equal to or larger than the threshold (NO in S1006), the server 111 cannot see the contents of the lower tag as shown in FIG. It is determined that they are stacked so as to cover from above. For example, consider a case where the lower paper sticky note 401 and the upper paper sticky note 405 are overlapped as shown in FIG. 11D and the threshold used in S1006 is “0.5”. As the reference area (since the areas of the two sticky notes are equal), “20”, which is the area of the lower paper sticky note 401, is set in S1005. Assuming that the area of the shared area 1103 is calculated to be “16” in S1002, the determination in S1006 is (16/20)> 0.5, so that the upper tag cannot be seen from the contents of the lower tag. It is determined that they are stacked so as to cover them.

  Thereafter, the server 111 overwrites the data portion 603 of the lower sticky note with the content of the upper sticky note in S1007. In the assumption of this embodiment, the data portion 703 of the electronic sticky note data 701 of the lower paper sticky note 401 is overwritten with the contents of the data portion 803 of the upper paper sticky note 405, and the electronic sticky note data after additional writing as shown in FIG. 1201 is created.

  In FIG. 12, the pasting time of the header portion 1202 is overwritten with the content of the pasting time of the header portion 802 of the electronic sticky note data of the upper sticky note. On the other hand, each element of the data portion 1203 is overwritten with the contents of the data portion 803 of the electronic sticky note data of the upper sticky note. In step S1007, the server data is overwritten with the tag data above, and the process returns to step S303.

  As described above, according to the present embodiment, the area of the overlapping area of the sticky notes is used to determine how to overlap the sticky notes (the degree of overlapping), and the contents of the sticky notes superimposed on the electronic data based on the result. Can be added or overwritten. This makes it possible to switch between appending and overwriting a sticky note depending on how the sticky note is superimposed while simply determining how to superimpose the sticky note.

<Example 3>
In the second embodiment, the method of switching between appending and overwriting electronic data according to how the sticky notes are overlaid has been described. In the second embodiment, the area of the shared area of the sticky notes overlapped when determining how to overlap the sticky notes is used. In the third embodiment, a method of using the legibility of the lower sticky note after the sticky note is overlaid for this determination will be described. The legibility here means how much information on the lower sticky note appears on the surface (can be recognized) in a state where the sticky note is overlapped.

  The operation of this system will be described. FIG. 13 is a flowchart showing the flow of processing of this system. Note that the processing of S301 to S310 and S311 is the same processing as the flowchart of FIG. 3, and S1007 is the same processing as the flowchart of FIG. Also in this flowchart, a paper sticky note 405 is newly attached so as to overlap the paper sticky note 401 from a state where three paper sticky notes are attached to the whiteboard 114 as shown in FIG. The description will be made on the assumption that the state shown in FIG.

  When it is determined that the newly added paper sticky note is superimposed on another sticky note (YES in any of S306 to S309), the server 111 reads the character from the header part 602 of the electronic sticky note data of the lower sticky in S1301. Get the coordinates of the image area. In step S <b> 1302, the server 111 acquires sticky image data from the electronic sticky note data portion 603 of the lower sticky note. Thereafter, in S1303, the server 111 cuts out the image data of the lower tag acquired in S1302 at the coordinates of the character image information acquired in S1301. Thereby, for example, in the case of the paper sticky note 401, the character image area 404 of the lower sticky note before the sticky note is overlaid can be cut out as an image.

  In step S <b> 1304, the server 111 converts the character image area of the lower sticky note into absolute coordinates with the lower left corner of the whiteboard 114 as the origin. Specifically, by adding the lower left coordinates of the sticky note area included in the header portion 602 to the coordinates of the character image area included in the header portion 602 of the lower sticky note, the lower left corner of the whiteboard 114 is set as the origin. Convert to absolute coordinates. Subsequently, in step S1305, the server 111 acquires captured image data of the whiteboard 114 captured by the digital camera 113 captured in step S301. Thereafter, in S1306, the server 111 cuts out the captured image data of the whiteboard 114 acquired in S1305 using the coordinates of the character image information of the lower tag acquired in S1304 (absolute coordinates of the lower left origin of the whiteboard 114). Thus, for example, when the paper sticky note is overlapped so that the contents of the lower paper sticky note 401 can be seen as shown in FIG. 11C, the character image area 1102 of the lower sticky note after the sticky note is overlapped is cut out. I can do things. In addition, when the sticky note is overlaid so as to cover the content of the lower paper sticky note 401 from the top as shown in FIG. 11F, the character image area 1104 of the lower sticky note after the sticky note is overlaid is displayed. Can be cut out.

  Thereafter, the server 111 performs a well-known similarity determination process in S1307, and the image of the character image area of the lower sticky note before the sticky note obtained in S1303 and the lower note after the sticky note obtained in S1306 are superimposed. The similarity with the image of the character image area of the tag is calculated. Subsequently, in step S1308, the server 111 determines whether the similarity obtained in step S1307 is greater than a predetermined threshold. If it is larger than the threshold (YES in S1308), server 111 determines that the lower sticky is highly readable even after the sticky notes are overlaid, and the upper sticky note is a lower sticky note as shown in FIG. It is determined that the contents are overlapped so that the contents can be seen.

  For example, consider the case where the lower paper sticky note 401 and the upper paper sticky note 405 are overlapped as shown in FIG. 11A, and the threshold used in S1308 is defined in advance as “80”. Assume that the similarity between the character image area 404 of the lower sticky note before the sticky note is superimposed and the character image region 1102 of the lower sticky note after the sticky note is calculated as “90” in S1307. In this case, since 90> 80, it is determined that the upper sticky note is overlapped so that the content of the lower sticky note can be seen.

  Thereafter, the server 111 appends the content of the upper sticky note to the electronic sticky note data of the lower sticky note in S311, and returns to S303.

  On the other hand, when the similarity obtained in S1307 is equal to or less than a predetermined threshold (NO in S1308), server 111 determines that the readability of the lower tag after the tag is stacked is low. As a result, it is determined that the upper sticky note is overlapped so as to cover from above so that the content of the lower sticky note cannot be seen as shown in FIG.

  For example, consider a case where the lower paper sticky note 401 and the upper paper sticky note 405 are overlapped as shown in FIG. 11D, and the threshold used in S1308 is “80”. If it is calculated in S1307 that the similarity between the character image area 404 of the lower sticky note before the sticky note is superimposed and the character image region 1104 of the lower sticky note after the sticky note is superimposed is “10”, 10 <90, it is determined that the upper sticky note is overlapped so as to cover the lower sticky note.

  Thereafter, the server 111 overwrites the electronic sticky note data of the lower sticky note with the content of the upper sticky note in S1007, and returns to S303.

  As described above, according to the present embodiment, the way in which the sticky notes are overlaid is determined based on the legibility of the lower sticky notes after the sticky notes are overlaid, and the contents of the sticky notes overlaid on the electronic data are added as a result. You can switch between overwriting and overwriting. Accordingly, it is possible to provide switching of appending and overwriting of a sticky note depending on how the sticky notes are overlaid in a form close to the user's feeling that when a sticky note is overlaid for the purpose of appending, the sticky note is overlaid so that it can be read.

<Other examples>
In the embodiment described above, the degree of overlap is determined by the similarity of the character image area. In addition to this, the result of applying the OCR to the character image area before and after the sticky note is overlaid is compared, and how much the character string read before the sticky note is overlaid after the sticky note is overlaid (above You may judge by whether it is hidden by the sticky note). In this case, even if the similarity of the character image area is low before and after the sticky notes are overlapped, if the area where the sticky notes are overlapped has a range (ratio) that does not affect the interpretation of the characters, it can be treated as additional writing.

  In the embodiment described above, the CPU (processor) of the server 111 executes a program to function as each processing unit. However, part or all of the processing is performed by a dedicated electronic circuit (hardware). You may make it comprise.

  The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (10)

An information processing apparatus that extracts data from image data including an area where information is described and converts the information into data,
Holding means for extracting and holding the information from the first area in which the information is included, included in the first image data;
Obtaining means for obtaining second image data including the first region and a second region in which information is described;
Detecting means for detecting whether or not the second region overlaps the first region in the image of the second image data;
In the image of the second image data, when the second region overlaps the first region, the information extracted from the second region is used to hold the first An information processing apparatus comprising: update means for updating information extracted from a region. The detection means further determines how the second region overlaps the first region,
The information processing apparatus according to claim 1, wherein the update unit switches an update method according to the way of overlapping. The detection means determines the area of the overlapping portion of the first region and the second region,
When the area is smaller than a predetermined threshold, the update unit adds information extracted from the second region to information extracted from the first region,
The update means overwrites the information extracted from the first area with the information extracted from the second area when the area is equal to or greater than the predetermined threshold. Information processing device. The detection means determines the similarity between the first region and the second region,
If the similarity is greater than a predetermined threshold, the updating means adds information extracted from the second area to information extracted from the first area,
The update means overwrites information extracted from the first area with information extracted from the second area when the similarity is equal to or less than the predetermined threshold. The information processing apparatus described. The detection means determines the ratio of the information described in the first area being hidden by the second area,
When the ratio is smaller than a predetermined threshold, the updating unit adds the information extracted from the second area to the information extracted from the first area,
The update means overwrites information extracted from the first area with information extracted from the second area when the ratio is equal to or greater than the predetermined threshold. Information processing device.   6. The information processing apparatus according to claim 1, wherein the first and second areas are indicated by an image of a medium on which information is described.   The information processing apparatus according to claim 6, wherein the medium is a tag. A photographing device for photographing images;
A projection device for projecting an image photographed by the photographing device;
Connected to
The acquisition unit obtains image data obtained by photographing the image with the photographing device when a user overlaps the medium on the image of the first image data projected with the projection device. The information processing apparatus according to claim 6, wherein the information processing apparatus is acquired as the second image data. An information processing method for extracting data from image data including an area where information is described and converting the information into data,
A holding step in which the holding unit extracts the information from the first region in which the information is described and is stored in the storage unit, which is included in the first image data;
An acquisition step in which the acquisition means acquires second image data including the first region and a second region in which information is described;
A detecting step for detecting whether or not the second region overlaps the first region in the image of the second image data;
When the second area overlaps the first area in the image of the second image data, the update unit uses the information extracted from the second area and holds the information in the storage unit And an updating process for updating information extracted from the first area. Computer
Holding means for extracting and holding the information from the first region in which the information is included, included in the first image data;
Acquisition means for acquiring second image data including the first region and a second region in which information is described;
Detecting means for detecting whether or not the second region overlaps the first region in the image of the second image data;
In the image of the second image data, when the second region overlaps the first region, the information extracted from the second region is used to hold the first A program for functioning as update means for updating information extracted from an area.

Images