JP2010176603A - Device, support device and search program for meeting document search, and meeting support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow a document used in a meeting to be found out easily. <P>SOLUTION: A meeting support system includes: a difference extraction section 13 which extracts difference between a taken image of a note at beginning of display of a slide and a taken image of the note at changing of the display in accordance with changing the display of the slide during the meeting, and generates difference information containing domain image data of the difference and the taken image at the changing of the display; an associating section 16 which associates the difference information with a slide ID of the slide where the display sent from a display control section 15 is switched, and registers them in an associating information storage section 22; a comparing section 33 which, after the meeting, compares the taken images of the note and the difference information with each other and identifies the difference information of the domain image data contained in the taken images, to identify the associated slide ID; and a meeting document obtaining section 34 which takes out the slide identified by the identified slide ID from a meeting document storing section 21 and makes a display section 35 to display it. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a conference material search device, a conference material search support device, a conference material search support program, a conference material search program, and a conference support system.

  In recent meetings, paper materials are often displayed on a projector or an electronic blackboard without distributing paper materials from the viewpoint of security and ecology. In some cases, the materials used in the conference are stored in a document sharing server or the like so that the participants can freely view them later. Conventionally, there has been proposed a technique for improving searchability by associating a meeting date and time, an organizer, and a participant as specific information with an image created and displayed during a meeting (Patent Document 1).

Japanese Patent Laid-Open No. 2000-165577 JP 2002-032397 A Japanese Patent Laid-Open No. 11-272679

  However, it is not always possible for the participants of the conference to remember items (search keywords) for specifying materials to be browsed such as the date and time of the meeting and the participants. In regular meetings and the like, the participants, the day of the week, and the date and time are often the same every time, so these pieces of information are not necessarily effective information for finding the target material.

  An object of the present invention is to easily find materials used in a conference.

  The meeting material search support device according to the present invention detects that the photographing means for always photographing a memo medium in which a memo is written by a meeting participant during the meeting and the display of the meeting material image displayed during the meeting are switched. Switching detection means for acquiring, acquisition means for acquiring the specific information of the conference material image displayed at the display switching time detected by the switching detection means, and the photographing means at the display switching time detected by the switching detection means And a photographic image taken at the start of taking in the memo medium taken by the photographing means at the start of display of the conference material image displayed at the time of switching the display. Difference information generating means for generating difference information for specifying a difference portion, the difference information and the identification information of the meeting material image are associated with each other to store the association information. And having registration means for registering the means.

  The conference material retrieval apparatus according to the present invention includes a photographing unit for photographing a memo medium in which a memo is written in a conference by a conference participant in accordance with a conference material image extraction instruction, and identification of a conference material image displayed during the conference Difference information stored in association information storage means in association with information, at the display start time of the conference material image, and at the end of display of the captured image of the memo medium and the conference material image captured by the imaging means The difference information included in the photographed image is identified by comparing the difference information that identifies the difference between the photographed image of the memo medium photographed by the photographing means and the photographed image acquired from the photographing means. And an output means for acquiring and outputting a meeting material image based on the identification information of the meeting material image associated with the specified difference information. Characterized in that it.

  Further, the difference information includes the switching-time captured image and region information for specifying a region including a difference between the switching-time captured image and the capture start-time captured image. And

  Further, the page turning detection means for detecting that the memo medium has been turned is provided, and the acquisition means specifies the meeting material image displayed when the page turning is detected by the page turning detection means. Information is acquired, and the difference information generating means is a captured image of the memo medium taken at the start time of display of the conference material image displayed at the time of page turning detection by the page turning detection means, and a page turning detection time point. Difference information for identifying a difference portion from the photographed image of the memo medium photographed in step 1 is generated.

  In addition, the output unit outputs a plurality of meeting material images collectively when the specifying unit specifies a plurality of pieces of difference information from the captured images acquired by the imaging unit.

  The output means outputs each of the plurality of meeting material images acquired based on the plurality of difference information to a position corresponding to the writing position of each memo on the photographed image of the memo medium.

  The output means outputs each of a plurality of meeting material images acquired based on a plurality of difference information in a size proportional to a writing area of each memo on a captured image of the memo medium.

  In the meeting material search support program according to the present invention, the computer is switched to the photographing means for always photographing the memo medium in which the memo is written by the meeting participant during the meeting, and the display of the meeting material image displayed during the meeting is switched. A switching detection means for detecting, an acquisition means for acquiring specific information of a conference material image that has been displayed at the display switching time detected by the switching detection means, and the photographing means at the display switching time detected by the switching detection means And a photographic image taken at the start of taking in the memo medium taken by the photographing means at the start of display of the conference material image displayed at the time of switching the display. Difference information generating means for generating difference information for specifying a difference portion, associating the difference information with the identification information of the meeting material image Registration means for registering in the communication with the information storage means, to function as a.

  According to the present invention, there is provided a conference material retrieval program for photographing a memo medium in which a memo is written in a conference by a conference participant according to a conference material image extraction instruction, and a conference material image displayed during the conference. Difference information stored in the correspondence information storage means in association with the specific information of the conference material image captured by the photographing means at the start of display of the conference material image and display end of the conference material image The difference information included in the photographed image is compared with the photographed image obtained from the photographing means by comparing the difference information that identifies the difference between the photographed image of the memo medium photographed by the photographing means at the time point. A means for identifying the meeting material, and acquiring and outputting the meeting material image based on the identifying information of the meeting material image associated with the identified difference information Power means, to function as.

  A conference support system according to the present invention includes a conference material search support device, a conference material search device, and an association information storage unit, and the conference material search support device receives a memo by a conference participant during the conference. First imaging means for always photographing the written memo medium, switching detection means for detecting that the display of the conference material image displayed during the meeting is switched, and at the time of display switching detected by the switching detection means. An acquisition means for acquiring specific information of the conference material image whose display has been completed; a photographic image at the time of switching of the memo medium taken by the first photographing means at the time of display switching detected by the switching detection means; The difference from the captured image at the start of taking in the memo medium taken by the first photographing means at the display start time of the meeting material image that has been displayed at the display switching time Difference information generating means for generating difference information to be specified; and registration means for associating the difference information with the identification information of the meeting material image and registering it in the association information storage means, the meeting material search device Is associated with the second photographing means for photographing the memo medium on which the memo is written by the conference participant according to the conference material image extraction instruction, and the specific information of the conference material image displayed during the conference. The difference information stored in the association information storage means, and the captured image of the memo medium photographed by the second photographing means at the display start time of the conference material image and the display end time of the conference material image Difference information that identifies a difference between the photographic image taken by the second photographing unit and the photographed image of the memo medium is collated with the photographed image acquired from the second photographing unit. And specifying means for specifying the difference information included in the captured image; and output means for acquiring and outputting the meeting material image based on the specifying information of the meeting material image associated with the specified difference information; It is characterized by having.

  According to the first and eighth aspects of the present invention, it is possible to easily find the conference material displayed when the memo is written on the memo medium by the conference participants.

  According to the second, ninth, and tenth aspects of the present invention, the conference material displayed when the photographed memo is written on the memo medium can be browsed.

  According to the third aspect of the present invention, it is possible to specify the writing area of a memo written while a certain conference material image is displayed and its image.

  According to the fourth aspect of the present invention, the conference material displayed when the memo is written can be found even when the memo medium is turned over.

  According to the fifth aspect of the present invention, it is possible to cope with a case where a plurality of conference materials are associated with a memo medium photographed in response to a conference material image extraction instruction.

  According to the sixth aspect of the present invention, it is possible to easily grasp which memo on the memo medium corresponds to the conference material.

  According to the seventh aspect of the present invention, it is possible to output the conference material in a size corresponding to the writing amount of the memo.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram of an overall configuration showing a conference support system according to the present embodiment. FIG. 1 shows a conference terminal device 10 installed in the conference room, a database (DB) server 20 installed in the computer room 2, and a user terminal device 30 installed in the office 3. The conference terminal device 10 is a computer used for conference progress, schedule management, etc. in the conference room 1, and corresponds to the conference material search support device of the present invention. The display board 4 connected to the conference terminal device 10 displays the image data of the conference material shared with the conference participants in the conference room 1. The camera 5 is a photographing means for always photographing the notebook 9 brought by each participant to write notes individually. In this embodiment, a case where a note 9 is used as a memo medium in which a memo is written by a participant is taken as an example. As will be described later, the database (DB) server 20 stores electronic data of conference materials and association information characteristic of the present embodiment so as to be accessible from the conference terminal device 10 and the user terminal device 30.

  The user terminal device 30 is a computer used by a conference participant and corresponds to the conference material search device of the present invention. Connected to the user terminal device 30 are a display 7 on which a conference material image is displayed and a camera 8 that captures the notebook 9 brought into the conference by the participant.

  FIG. 2 is a hardware configuration diagram of a computer forming the conference terminal device 10 according to the present embodiment. In the present embodiment, the basic configuration of the computer forming the conference terminal device 10 can be realized by a general-purpose hardware configuration that has existed in the past. That is, the computer corresponds to the CPU 41, the ROM 42, the RAM 43, the HDD controller 45 connected to the hard disk drive (HDD) 44, the mouse 46 and keyboard 47 provided as input means, and the display board 4 as shown in FIG. An input / output controller 50 for connecting each display 48 and a network controller 51 provided as communication means are connected to an internal bus 52. In this embodiment, a camera 49 corresponding to the camera 5 shown in FIG. 1 is further connected to the input / output controller 50.

  Since the user terminal device 30 in the present embodiment is also a computer, its hardware configuration can be illustrated in the same manner as in FIG. The database server 20 is similar, but the camera 49 need not be installed.

  FIG. 3 is a block diagram showing the conference support system in the present embodiment. In FIG. 3, among the arrows connecting the blocks, a solid line indicates a data flow, and a broken line indicates a control flow. The conference terminal device 10 includes a photographing unit 11, an image capturing unit 12, a difference extraction unit 13, a page turning detection unit 14, a display control unit 15, an association unit 16, and a display unit 17. The photographing unit 11 is realized by a cooperative operation with the camera 5 and always photographs a note 9 in which a memo is written by a conference participant during the conference. The image capturing unit 12 continuously captures captured images sent from the camera 5. That is, the captured image is acquired as a moving image. The difference extraction unit 13 switches the display data of the note 9 captured by the camera 5 at the time of switching the display when the meeting material image displayed on the display board 4 is switched, and the display switching. Difference information for identifying a difference portion from the image data of the note 9 (captured image at the start of capturing) captured by the camera 5 at the display start time of the conference material image that has been displayed at the time is generated. The difference extraction unit 13 also generates difference information even when page turning is detected. The page turning detection unit 14 detects that the note 9 has been turned by analyzing the moving image sent from the image capturing unit 12. The display control unit 15 performs display control of the conference material image displayed on the display board 4. The associating unit 16 associates the difference information generated by the difference extracting unit 13 when the display of the conference material image is switched with the specific information of the conference material image. The display unit 17 displays the conference material image on the display board 4 under the control of the display control unit 15.

  Each component 11 to 17 in the conference terminal device 10 is realized by a cooperative operation of a computer that forms the conference terminal device 10 and a program that operates on the CPU 41 mounted on the computer.

  The database server 20 includes a conference material storage unit 21 and an association information storage unit 22. The conference material storage unit 21 stores an image of conference material used in the conference. In the association information storage unit 22, the difference information associated with the association unit 16 and the specific information of the conference material image are stored as association information. The conference material storage unit 21 and the association information storage unit 22 are realized by the HDD 44 mounted on the database server 20. Strictly speaking, software executed by the CPU 41 of the database server 20 performs data transmission / reception and data reading / writing in response to an access request from each of the devices 10 and 30, but this is omitted for the sake of simplicity. .

  The user terminal device 30 includes a photographing unit 31, an image acquisition unit 32, a collation unit 33, a conference material acquisition unit 34, and a display unit 35. The photographing unit 31 is realized by a cooperative operation with the camera 8 and photographs a memo written in the note 9 in response to a conference material image extraction instruction. The image acquisition unit 32 acquires the image of the note 9 photographed by the photographing unit 31. In this case, a still image is acquired. The collation unit 33 identifies the difference information included in the photographed image by collating the difference information stored in the association information storage unit 22 with the photographed image. The conference material acquisition unit 34 acquires the conference material image based on the specific information of the conference material image associated with the specified difference information. The display unit 35 displays the acquired conference material image on the display 7.

  The components 31 to 35 in the user terminal device 30 are realized by a cooperative operation of a computer that forms the user terminal device 30 and a program that operates on the CPU 41 mounted on the computer.

  Further, the program used in this embodiment can be provided not only by communication means but also by storing it in a computer-readable recording medium such as a CD-ROM or DVD-ROM. The program provided from the communication means or the recording medium is installed in the computer, and various processes are realized by the CPU of the computer sequentially executing the installation program.

  Next, the operation in this embodiment will be described. First, processing in the conference terminal device 10 that operates during the conference will be described using the flowchart shown in FIG.

  During the conference, the display control unit 15 takes out the conference material designated in accordance with the operation instruction of the conference proceeding person from the conference material storage unit 21 and causes the display unit 17 to display the conference material. The conference participant writes some memo in the notebook 9 brought into the conference room 1 while referring to the display content of the display board 4. The photographing unit 11 always photographs the note 9. Then, the image capturing unit 12 continues to capture the captured image captured by the capturing unit 11. Here, for convenience of explanation, explanation will be made by paying attention to the note 9 of one conference participant. In addition, the associating unit 16 waits for notification of the occurrence of an event such as switching of conference materials from the display control unit 15 or page turning from the page turning detection unit 14. Here, first, the case where the switching of the display of the conference material image occurs will be described.

  The display control unit 15 takes out the image data of the designated conference material from the conference material storage unit 21 and displays it on the display unit 17 in accordance with a display switching instruction from the conference proceeding person, thereby displaying the conference on the display board 4. Switch the image of the material. At this time, the display control unit 15 sends to the associating unit 16 identification information of the conference material image whose display is terminated by the display switching and the display switching instruction. In addition, assuming that the conference material is composed of a plurality of pages, the image data of each page constituting the conference material is referred to as “slide”. That is, the display control unit 15 sends the identification information (slide ID) of the slide whose display is ended by the display switching instruction to the associating unit 16 together with the display switching. In the case of switching conference materials, the slide ID notification itself may be a display switch.

  The associating unit 16 detects that the display content of the display board 4 has been switched by a notification from the display control unit 15 (step 101), and acquires a slide ID that can identify the conference material whose display has ended (step 102). ), And notifies the difference extraction unit 13 of a difference extraction instruction. Upon receiving this notification, the difference extraction unit 13 extracts the difference as follows and generates difference information. The generation of the difference information will be described with reference to FIGS. 5A to 5C.

  First, as illustrated in FIG. 5B, the difference extraction unit 13, as illustrated in FIG. 5B, the still image data (shooting image at the time of switching) of the notebook 9 captured by the imaging unit 11 at the display switching time detected by the association unit 16, and FIG. As illustrated, a difference portion from the still image data (captured image at the start of capture) of the note 9 captured by the camera 5 at the display start time of the slide that has been displayed at the display switching time is extracted. The difference extraction unit 13 processes the time point when the notification from the associating unit 16 is received as the display switching time point. Alternatively, the association unit 16 may notify the difference extraction unit 13 of the display switching time by including the time when the switching notification from the display control unit 15 is received. The captured image at the start of capture is captured at the time when the display of the previous slide is completed (or immediately after that time, or in the case of the first slide, when the display of the slide is started). It is only necessary to identify the captured image data (still image) captured at that time from the captured image data (moving image) and mark it as a captured image at the start of capture. Then, as illustrated in FIG. 5C, the rectangular area 61 including the extracted difference portion is specified on the switching captured image, and coordinate data of each corner of the rectangular area is obtained. And the difference extraction part 13 will transmit this to the correlation part 16, if the difference information containing the coordinate data of the picked-up image and rectangular area 61 is produced | generated (step 103). The difference information generated by the above processing is information indicating the entry range of the memo written by the conference participant while the slide to be displayed is switched.

  When difference information is sent from the difference extraction unit 13, the associating unit 16 associates the difference information with the slide ID received from the display control unit 15 to generate association information and registers it in the association information storage unit 22. (Step 104). In the present embodiment, the difference extraction unit 13 generates the difference information. However, when the association unit 16 is configured to capture the imaging data from the image capturing unit 12 or the imaging data holding unit (not illustrated). The associating unit 16 may generate the difference information.

  FIG. 6 is a diagram illustrating a data configuration example of the association information recorded in the association information storage unit 22 according to the present embodiment. The difference information included in the association information includes a switching-time captured image and coordinate data for specifying the rectangular area 61 that is written while the slide is displayed in the image. In the present embodiment, the switching-time captured image itself is stored in the association information storage unit 22, but the switching-time captured image is stored in another means, and the difference information stores the switching-time captured image. You may comprise so that previous information may be included. The slide ID is composed of a file name including the slide and a page number in the file. If each slide is composed of only one page file, the page number information is not necessary.

  Next, a case where the page turning of the note 9 occurs will be described.

  When the page turning detection unit 14 detects that the note 9 has been turned by analyzing the moving image sent from the image capturing unit 12, the page turning detection unit 14 notifies the association unit 16 of the fact. For detecting that the page is turned, for example, a marker is added to each page, and when this marker cannot be confirmed from the imaging data, it is recognized that the page is turned. Alternatively, a data code including a page number may be added to each page, and it may be recognized that the page is turned by a change in the page number of the data code included in the imaging data. Alternatively, an operation indicating that the conference participant has turned the page may be input.

  When the association unit 16 detects that the page of the note 9 has been turned by the notification from the page turning detection unit 14 (step 101), the association unit 16 notifies the difference extraction unit 13 of a difference extraction instruction. At the same time, the associating unit 16 sends a notification instruction of the slide ID of the conference material currently displayed to the display control unit 15. In response to this instruction, the slide ID sent from the display control unit 15 is acquired (step 102).

  Upon receiving the difference extraction instruction from the associating unit 16, the difference extracting unit 13 captures the image data of the note 9 (photographed image at the time of page turning) taken by the photographing unit 11 at the time of page turning detection, and at the time of page turning detection. A difference portion from the image data of the note 9 (captured image at the start of capturing) captured by the camera 5 at the display start time of the displayed slide is extracted. The imaging data of each note 9 is still image data as described above. The difference extracting unit 13 processes the time point when the notification from the associating unit 16 is received as the page turning detection time point. Alternatively, the association unit 16 may notify the difference extraction unit 13 of the page turning detection time by including the time when the notification from the page turning detection unit 14 is received. The captured image at the start of capture is a captured image of the note 9 captured at the display start time of the slide displayed at the time of page turning detection as described above. In other words, since the image data was captured at the time when the display of the previous slide was completed (or immediately after that time), the captured image data (still image) captured at that time from the captured image data (moving image). Mark as a captured image at the start of capture.

  By the way, since the page turning is detected when the page is actually turned, it is not possible to photograph the page before turning from the imaging data after the page is turned. Therefore, the imaging data at the time of detecting the page turning is not the time when the action of turning the page by the conference participant is completed, but the imaging data immediately before the action is started. Therefore, the difference extracting unit 13 extracts the difference by tracing back the moving image captured by the image capturing unit 12 to the imaging data at the time when the page turning action is started when the page turning is actually detected. It is necessary to extract the imaging data (photographed image at the time of page turning) used for the above.

  In this manner, when the captured start-time captured image and the page-turned captured image corresponding to FIGS. 5A and 5B are extracted from the moving image, the difference extraction unit 13 performs the page-turning as illustrated in FIG. 5C. On the photographed image, a rectangular area 61 including the extracted difference portion is specified, and coordinate data of each corner of the rectangular area is obtained. And the difference extraction part 13 will transmit this to the correlation part 16, if the difference information containing the coordinate data of the picked-up image at the time of page turning and the rectangular area 61 is produced | generated (step 103). Thereafter, as in the slide display switching described above, the associating unit 16 generates association information by associating the difference information with the slide ID received from the display control unit 15 and stores it in the association information storage unit 22. Register (step 104).

  Next, a material search process performed when a conference participant wants to view the conference material used in the conference after the conference ends will be described with reference to the flowchart shown in FIG.

  Suppose that a participant of a conference wants to browse a slide corresponding to a memo written in the note 9 while referring to the memo. At this time, the conference participant opens the page in which the memo is written while the slide is displayed, and instructs the camera 8 to take a picture by instructing to take out a predetermined conference material image. Note 9 that the notebook 9 is normally used in a two-page spread state, so the photographed image is as shown in FIG. FIG. 8 shows an example in which memos for four slides are written on two pages of spread, and the memos are classified into four areas 62 to 65 corresponding to the respective slides.

  When the image acquisition unit 32 acquires imaging data captured by the imaging unit 11 in response to a conference participant image retrieval instruction from a conference participant (step 111), the collation unit 33 sequentially determines difference information from the association information storage unit 22. And the area image data specified by the area data of the captured image included in the read difference information is collated with the captured image. If the area image data is included in the captured image, more precisely, if the area image data matches the image data of the area of the captured image, the image data specified from the difference information, for example, FIG. It can be determined that image data corresponding to the rectangular area 61 in 5C is included in the captured image. Thereby, the collation part 33 specifies the difference information of the area | region image data contained in the picked-up image (step 112). The collation unit 33 repeatedly performs this collation process on all the difference information, thereby specifying the difference information of the area image data that matches the captured images in the areas 62 to 65.

  When the difference information is specified in this way, the collation unit 33 reads out and acquires the slide ID associated with the difference information from the association information storage unit 22 (step 113). Then, the acquired slide ID is sent to the conference material acquisition unit 34. According to the example of FIG. 8, four slide IDs corresponding to the regions 62 to 65 are sent to the conference material acquisition unit 34.

  Upon receiving the slide ID from the collation unit 33, the conference material acquisition unit 34 takes out the slide specified from each slide ID from the conference material storage unit 21 and displays it on the display unit 35 (step 35). An example of the conference material display screen displayed on the display 7 by the above processing is shown in FIG.

  As illustrated in FIG. 9, corresponding slides are displayed at positions corresponding to the areas 62 to 65 in which the memos are written. In particular, in the case of the present embodiment, the size of the slide to be displayed is adjusted in proportion to the size of each of the regions 62 to 65. In this embodiment, a plurality of slides are displayed together on one screen in this way. Of course, one slide may be displayed while being switched in order.

  In the above description, four slides corresponding to the memos written on the entire two open pages are displayed. For example, the conference participant can select the slide corresponding to the memo in the area 62 in FIG. Suppose you only want to browse. In this case, the conference participant gives an instruction to take out the conference material image after covering the areas 63 to 65 with hands. If it does so, only the memo of the area | region 62 will be image | photographed, and the memo of the other areas 63-65 will not be image | photographed the whole. Since the area image data corresponding to the captured image of the area 62 is included in the difference information, the slide ID can be specified from the association information as described above. On the other hand, the area image data that coincides with each captured image of the areas 63 to 65 covered with hands or the like does not exist in the difference information. That is, the slide ID corresponding to the areas 63 to 65 cannot be extracted from the association information. As a result, only the slide corresponding to the memo in the area 62 is displayed on the display 7. This display may be displayed on the entire screen of the display 7 or may be displayed in a size corresponding to the size of the area 62.

  As is clear from the above description, if the entire region 62 is photographed by the camera 8, the slide corresponding to the memo written in the region 62 can be viewed. On the other hand, if at least a part of the areas 63 to 65 is covered, each captured image in the areas 63 to 65 does not match any area image data obtained from the difference information, and thus cannot be a display target.

  In the present embodiment, since the corresponding slide can be browsed based on the writing to the physical note 9, it is not necessary to transfer the content of the slide to the note 9. However, for this purpose, it is necessary to reliably extract the image data of the note 9 from the data captured by the cameras 5 and 8. You may also take notes from multiple meeting participants at the same time. Therefore, in order for this embodiment to operate correctly, it is necessary to be able to correctly extract the note 9 from the imaging data. For this purpose, the image of the note 9 may be extracted from the imaged data using image analysis processing, or the image position of the note 9 included in the imaged data may be determined by providing an installation base for the note 9 or the like. It may be fixed. Alternatively, the image of the note 9 may be easily detected from the captured data by adding a marker to the note 9. Further, a plurality of cameras 5 may be installed in the conference room 1 so that a plurality of conference participants can be handled.

  In this embodiment, a note having two spread pages as an example of a memo medium has been described as an example. However, the present invention is not limited to this.

  In the present embodiment, the output destination of the slide is the display 7, but it may be printed on a paper medium.

It is the schematic of the whole structure which showed the meeting assistance system in this Embodiment. It is a hardware block diagram of the computer which forms the conference terminal device in this Embodiment. It is the block block diagram which showed the meeting assistance system in this Embodiment. It is the flowchart which showed the process implemented during a meeting in this Embodiment. It is the figure which showed the example of the picked-up image acquired at the display start time of the slide in this Embodiment. It is the figure which showed the example of the picked-up image acquired at the display switching time of the slide in this Embodiment. It is the figure which showed the area | region image which shows the difference of each picked-up image shown to FIG. 5A and FIG. 5B. It is the figure which showed the example of a data structure of the correlation information recorded on the correlation information storage part in this Embodiment. It is the flowchart which showed the material search process performed after the meeting completion | finish in this Embodiment. It is the figure which showed the example of the captured image of the notebook image | photographed after the meeting in this Embodiment. It is the figure which showed the example of the meeting material display screen which displays a slide in this Embodiment.

  1 conference room, 2 computer room, 3 office, 4 display board, 5, 8, 49 camera, 7, 48 display, 10 conference terminal device, 11, 31 photographing unit, 12 image capturing unit, 13 difference extracting unit, 14 Page turning detection unit, 15 display control unit, 16 association unit, 17, 35 display unit, 20 database server, 21 conference material storage unit, 22 association information storage unit, 30 user terminal device, 32 image acquisition unit, 33 verification unit, 34 meeting material acquisition unit, 41 CPU, 42 ROM, 43 RAM, 44 hard disk drive (HDD), 45 HDD controller, 46 mouse, 47 keyboard, 50 input / output controller, 51 network controller, 52 internal bus.

Claims (10)

Photographing means for always photographing a memo medium in which a memo is written by a participant during the meeting;
A switching detection means for detecting that the display of the meeting material image displayed during the meeting is switched;
Acquisition means for acquiring specific information of a conference material image whose display has been terminated at the time of display switching detected by the switching detection means;
The photographic medium at the time of display switching detected by the switching detection means and the photographic medium at the time of switching, and the photographic medium at the display start time of the conference material image displayed at the display switching time. Difference information generating means for generating difference information for identifying a difference portion from a captured image at the start of capturing of the captured memo medium;
Registration means for associating the difference information with the specific information of the meeting material image and registering it in the association information storage means;
A conference material search support apparatus characterized by comprising: Photographing means for photographing a memo medium in which a memo is written at a meeting by a participant of the meeting in accordance with a meeting material image take-out instruction;
Photographing of the memo medium, which is difference information stored in the association information storage means in association with the specific information of the meeting material image displayed during the meeting and photographed by the photographing means at the start of displaying the meeting material image By comparing the difference information that identifies the difference between the image and the captured image of the memo medium captured by the imaging unit at the end of the display of the meeting material image with the captured image acquired from the imaging unit, A specifying means for specifying the difference information included in the captured image;
Output means for acquiring and outputting a conference material image based on the specific information of the conference material image associated with the identified difference information;
A conference material retrieval apparatus characterized by comprising: The conference material search support apparatus according to claim 1,
The difference information includes the switching-time captured image and area information for specifying an area including a difference between the switching-time captured image and the capture start-time captured image. Meeting material search support device. The conference material search support apparatus according to claim 1,
Page turning detecting means for detecting that the memo medium has been turned;
The acquisition unit acquires specific information of the conference material image displayed when the page turning is detected by the page turning detection unit,
The difference information generation means is a photographed image of the memo medium taken at the display start time of the conference material image displayed at the time of page turning detection by the page turning detection means, and the image taken at the time of page turning detection. A conference material search support apparatus, characterized in that it generates difference information for identifying a difference between a memo medium and a captured image. In the meeting material search device according to claim 2,
The meeting material search apparatus characterized in that the output means collectively outputs a plurality of meeting material images when the specifying means specifies a plurality of pieces of difference information from the photographed images acquired by the photographing means. The conference material search device according to claim 5,
The output means outputs a plurality of meeting material images acquired based on a plurality of difference information to a position corresponding to a writing position of each memo on a photographed image of the memo medium. . In the meeting material search device according to claim 5 or 6,
The output means outputs each of a plurality of meeting material images acquired based on a plurality of difference information in a size proportional to a writing area of each memo on a photographed image of the memo medium. apparatus. Computer
Shooting means for always shooting a memo medium in which a memo is written by a participant during the meeting,
Switching detection means for detecting that the display of the meeting material image displayed during the meeting is switched,
Acquisition means for acquiring specific information of a meeting material image whose display has been terminated at the time of display switching detected by the switching detection means;
The photographic medium at the time of display switching detected by the switching detection means and the photographic medium at the time of switching, and the photographic medium at the display start time of the conference material image displayed at the display switching time. Difference information generating means for generating difference information for identifying a difference portion from a captured image at the start of capture of the captured memo medium;
Registration means for associating the difference information with the specific information of the meeting material image and registering it in the association information storage means;
Meeting material search support program to function as. Computer
Photographing means for photographing a memo medium in which a memo is written at a meeting by a participant of the meeting in accordance with a meeting material image extraction instruction;
Photographing of the memo medium, which is difference information stored in the correspondence information storage means in association with the specific information of the meeting material image displayed during the meeting and photographed by the photographing means at the start of displaying the meeting material image By comparing the difference information that identifies the difference between the image and the captured image of the memo medium captured by the imaging unit at the end of the display of the meeting material image with the captured image acquired from the imaging unit, A specifying means for specifying difference information included in the photographed image;
An output means for acquiring and outputting a conference material image based on the specific information of the conference material image associated with the identified difference information;
Meeting material search program to function as. A meeting material search support device;
A conference material search device;
Association information storage means;
Have
The conference material search support device includes:
A first photographing means for always photographing a memo medium in which a memo is written by a participant of the meeting during the meeting;
A switching detection means for detecting that the display of the meeting material image displayed during the meeting is switched;
Acquisition means for acquiring specific information of a conference material image whose display has been terminated at the time of display switching detected by the switching detection means;
At the time of display switching detected by the switching detection means, the photographic image taken at the time of switching of the memo medium photographed by the first photographing means, and at the display start time of the conference material image displayed at the time of display switching. Difference information generating means for generating difference information for identifying a difference portion from the captured image at the start of capturing the memo medium imaged by the first imaging means;
Registration means for associating the difference information with the specific information of the meeting material image and registering it in the association information storage means;
Have
The conference material search device
A second photographing means for photographing a memo medium in which a memo is written at the meeting by a participant of the meeting in response to the instruction to take out the meeting material image;
The difference information stored in the association information storage means in association with the specific information of the meeting material image displayed during the meeting, which is taken by the second photographing means at the start of displaying the meeting material image Difference information for identifying a difference between the photographed image of the memo medium and the photographed image of the memo medium photographed by the second photographing means at the end of display of the conference material image is acquired from the second photographing means. Identifying means for identifying the difference information included in the captured image by collating with the captured image,
Output means for acquiring and outputting a conference material image based on the specific information of the conference material image associated with the identified difference information;
A conference support system characterized by comprising:

Images