JP2018006950A - Conference related information reproduction device, conference related information reproduction method and conference related information reproduction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily reproduce much more information related to a conference.SOLUTION: A portable information device comprises: a reference information acquisition part 120 for acquiring reference information; a grouping part 130 for sorting multiple pieces of reference information for which positions of additional information are included within a predetermined range in presentation data, among multiple pieces of reference information into the same group; and a reference information display part 150 for displaying at least one of multiple pieces of reference information belonging to a selected group. The reference information acquisition part 120 includes: a read data acquisition part 121 for acquiring read data that are obtained by reading a presentation document; an additional information extraction part 122 for extracting an image that is not present in the presentation data, as additional information from the read data; a position determination part 123 for determining a position in the presentation data corresponding to a position of the additional information as a position where the additional information is added; and a recording medium correspondence generation part 124 for generating reference information corresponding to the presentation document based on the additional information and the determined position.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a conference related information reproduction device, a conference related information reproduction method, and a conference related information reproduction program, and in particular, a conference related information reproduction device and a conference related information reproduction method for reproducing information related to a conference in which a plurality of users participate. And a conference related information reproduction program.

  2. Description of the Related Art A conference system that displays data prepared as materials for presentation in a conference or the like on an electronic terminal such as a portable information device carried by all participants is known. For example, Japanese Patent Application Laid-Open No. 2014-098972 discloses an electronic conference system including an image display device and one or more electronic terminals, and the electronic terminal displays a plurality of images displayed on the image display device. The image / operation recording unit includes an image / operation recording unit that records on a recording medium in association with operation information indicating a user's operation on the plurality of images, and the image display device communicates with the electronic terminal, and the image / operation recording unit An electronic conference system is described that includes image generation means for generating a display image based on a plurality of images recorded on a recording medium and operation information associated with the plurality of images.

According to the above-described electronic conference system, the participants of the conference can reconfirm the content of the conference by reproducing the user's operation related to the conference using the display image. On the other hand, when a user who does not carry the electronic terminal participates in the conference, the user participates in the conference while referring to the paper on which the material is printed. In this case, in the electronic conference system described in Japanese Patent Application Laid-Open No. 2014-098972, there is a problem in that information generated by an operation by a user who does not carry the electronic terminal, for example, an operation of handwriting on a sheet cannot be collected. There is.
JP 2014-098972 A

  The present invention has been made to solve the above-described problems, and one of the objects of the present invention is to provide a conference related information reproducing apparatus capable of easily reproducing more information related to a conference. It is to be.

  Another object of the present invention is to provide a conference related information reproduction method capable of easily reproducing more information related to a conference.

  Still another object of the present invention is to provide a conference related information reproduction program capable of easily reproducing more information related to a conference.

  According to an aspect of the present invention to achieve the above-described object, the conference related information reproduction device includes reference information acquisition means for acquiring reference information generated by either the own device or one or more other devices, The reference information includes additional information added to the presentation data that can be accessed by the own device and one or more other devices, and position information indicating a position where the additional information is added in the presentation data. A grouping means for classifying a plurality of reference information whose additional information is within a predetermined range in the presentation data from among the plurality of reference information acquired by the method, and one of one or more groups is selected. And a reference information display means for displaying at least one of the plurality of reference information belonging to the selected group. Reading data acquisition means for acquiring reading data obtained by reading a recording medium on which an image based on the data is formed, and additional information for extracting an image that does not exist in the presentation data from the reading data acquired by the reading data acquisition means as additional information Extracted by the extracting means, the position determining means for determining the position in the presentation data corresponding to the position of the extracted additional information in the read data as the added position of the extracted additional information, and the additional information extracting means Recording medium correspondence generating means for generating reference information corresponding to the recording medium based on the additional information and the position determined by the position determining means.

  According to this aspect, reference information generated by either the own device or one or more other devices is acquired, and an image that does not exist in the presentation data from read data obtained by reading a recording medium on which an image based on the presentation data is formed Is extracted as additional information, the position in the presentation data corresponding to the position of the extracted additional information is determined as the additional position of the extracted additional information, and based on the extracted additional information and the determined position Reference information corresponding to the recording medium is generated. Then, among the plurality of reference information, a plurality of reference information whose additional information positions are within a predetermined range in the presentation data are classified into the same group, and one of the one or more groups is selected. At least one of the plurality of reference information belonging to the selected group is displayed. For this reason, additional information that is extracted from read data obtained by reading a recording medium on which an image based on the presentation data is read and that is not present in the presentation data is added as additional information to the presentation data by the own device or another device. Can be reproduced with. As a result, more information related to the conference can be easily reproduced.

  Preferably, the reference information acquisition unit further includes a size determination unit that determines the size of the additional information extracted by the additional information extraction unit based on at least one of the read data and the presentation data.

  According to this aspect, since the extracted additional information is displayed in a size corresponding to the size of the additional information added to at least one of the read data and the presentation data, the size variation of the displayed additional information is reduced. can do.

  Preferably, the reference information acquisition unit determines the size of the additional information extracted by the additional information extraction unit based on the size of the additional information added to the presentation data by either the own device or one or more other devices. Size determining means is further provided.

  According to this aspect, since the extracted additional information is displayed in a size corresponding to the size of the additional information added to the presentation data by either the own device or one or more other devices, the size of the additional information varies. Can be reduced.

  Preferably, when the additional information extracted by the additional information extraction unit does not exist in any of the images of the plurality of pages included in the presentation data, the position determination unit adds the extracted additional information from the plurality of pages included in the presentation data. The page corresponding to the image having the closest information and the relative position of the extracted additional information with respect to the corresponding page are determined as the position of the additional information.

  According to this aspect, the position of the additional information extracted from the read data obtained by reading the recording medium on which the image based on the presentation data is read is extracted from the page corresponding to the image whose additional information is closest to the additional data. Since the relative position of the information is determined, the additional information extracted from the read data can be classified into one or more groups. For this reason, the additional information extracted from the read data can be reproduced together with the additional information added to the presentation data by the own device or another device.

  Preferably, the reference information acquisition unit determines whether the additional information extracted by the additional information extraction unit includes a condition image extending from within any of the plurality of pages of images included in the presentation data. The position determination unit further includes an image determination unit, and when the additional information includes a condition image, the position determination unit includes an end portion of the condition image of the page in the page of the image that overlaps the condition image among a plurality of pages included in the presentation data. Is determined as the position where the additional information is added.

  According to this aspect, when the condition information exists in the additional information, the position of the end of the condition image is determined as the position where the additional information is added in the page of the image overlapping the condition image. Can be determined at an appropriate position within the page of the presentation data.

  Preferably, the conference related information reproduction device can select a conference information acquisition unit that acquires information on a conference state as conference information, a conference information output unit that outputs at least a part of the conference information, and one or more groups. And a group selection means for selecting any one of the one or more groups displayed by the group list display means, and the conference information output means includes at least one group selection means. Presentation data part display means for displaying a presentation part in the presentation data corresponding to the selected group in response to selection of any one of the groups, and the reference information display means is the presentation data part display means One or more additional information respectively included in one or more reference information belonging to the selected group is included in the presentation part displayed by Additional information display means for superimposing and displaying the additional information when at least a part of the additional information extracted by the additional information extraction means does not exist in any of the multiple page images included in the presentation data An indicator display means for superimposing an indicator indicating the presence of the indicator on the presentation part instead of the additional information, and an outer side that accepts an operation of the indicator by the user and displays additional information corresponding to the indicator when the indicator is operated Additional information display means.

  According to this aspect, when any one of the one or more groups is selected, the presentation part in the presentation data corresponding to the selected group is added to one or more reference information belonging to the selected group. One or more additional information included in each is superimposed and displayed. Thus, the user can easily recognize the presence of additional information added to the announcement part while referring to the announcement part in the announcement data. In addition, instead of additional information that does not exist in any of the multiple-page images included in the presentation data, an indicator indicating the presence of the additional information is displayed superimposed on the presentation part, and if the indicator is operated, the indicator Additional information corresponding to is displayed. This makes it possible to display additional information that does not exist in any of the images of a plurality of pages.

  Preferably, each of the plurality of reference information corresponding to the presentation data including the additional information added to the presentation data includes an input period that defines a start time and an end time of input of the additional information. A plurality of reference information corresponding to the presentation data acquired by the information acquisition means, and a plurality of reference information whose additional information is within a predetermined range in the presentation data and whose start time is within a predetermined period belong to the same group. Each of the plurality of reference information corresponding to the recording medium acquired by the reference information acquisition unit is classified as follows, and the position of the additional information of the reference information is one or more of the plurality of groups. The information is classified into a group included in a predetermined range from any position of the plurality of reference information included in the group.

  According to this aspect, each of the plurality of reference information corresponding to the presentation data is classified into one or more groups based on the position of the additional information and the input period. Further, each of the plurality of reference information corresponding to the recording medium has one or a plurality of groups of the plurality of groups in which the position of the additional information of the reference information is one of the plurality of reference information included in the group. Are classified into groups included in a predetermined range from the position of. Thereby, the plurality of reference information corresponding to the presentation data and the plurality of reference information corresponding to the recording medium can be appropriately classified into a plurality of groups.

  Preferably, the conference information changes with the passage of time, and the conference-related information reproduction device performs group periods based on an input period of each of one or more reference information belonging to the group for each of one or more groups. Group period determining means for determining the reproduction period, and reproduction period determining means for determining the reproduction period as the group period determined for the selected group in response to selection of one of the one or more groups. In addition, the conference information output means outputs a portion of the conference information corresponding to the determined reproduction period.

  According to this aspect, for each of one or more groups, a group period is determined based on an input period of each of the one or more reference information belonging to the group, and any one of the one or more groups is selected. In response, the group period determined for the selected group is determined as the playback period, and the portion of the conference information corresponding to the determined playback period is output. For this reason, it is possible to reproduce the conference state in a period during which additional information is input by a user who operates either the own device or one or more other devices.

  Preferably, the reference information further includes user identification information for identifying the user who added the additional information, and the reference information acquisition unit includes the additional information included in the read data together with the read data acquired by the read data acquisition unit. Further includes user identification information acquisition means for acquiring the user identification information of the user who added the information, and the recording medium correspondence generation means is based on the user identification information acquired by the user identification information acquisition means, and the reference information corresponding to the recording medium When displaying at least one of the plurality of reference information, the reference information display means simultaneously displays the additional information included in each reference information and the user identification information corresponding to the additional information.

  According to this aspect, user identification information for identifying a user who has added additional information to the recording medium is acquired together with the read data. Based on the acquired user identification information, reference information corresponding to the recording medium is generated. When at least one of the plurality of reference information is displayed, the additional information included in each reference information and the user identification information corresponding to the additional information are displayed at the same time. Thereby, it is possible to notify the user of who added the additional information regarding the reference information corresponding to the recording medium.

  According to another aspect of the present invention, the conference related information reproduction method includes a reference information acquisition step of acquiring reference information generated by either the own device or one or more other devices, and the reference information includes the own device and A plurality of references acquired by reference information acquisition means, including additional information added to presentation data accessible by each of one or more other devices, and position information indicating a position where the additional information is added in the presentation data Among the information, a grouping step for classifying a plurality of reference information in which the position of the additional information is within a predetermined range in the presentation data into the same group, and when any one of the one or more groups is selected, A reference information display step for displaying at least one of the plurality of reference information belonging to the selected group, wherein the reference information acquisition step is based on the presentation data. A read data acquisition step of acquiring read data obtained by reading a recording medium on which an image is formed; an additional information extraction step of extracting an image that does not exist in the presentation data from the read data acquired by the read data acquisition step; A position determination step for determining a position in the presentation data corresponding to a position of the extracted additional information in the read data as position information indicating a position where the extracted additional information is added, and an additional information extraction step. A recording medium correspondence generating step for generating reference information corresponding to the recording medium based on the additional information and the position determined by the position determining step.

  According to this aspect, it is possible to provide a conference related information reproduction method capable of easily reproducing more information related to a conference.

  According to another aspect of the present invention, the conference related information reproduction program acquires the reference information generated by either the own device or one or more other devices, and the reference information includes the own device and A plurality of references acquired by reference information acquisition means, including additional information added to presentation data accessible by each of one or more other devices, and position information indicating a position where the additional information is added in the presentation data Among the information, a grouping step for classifying a plurality of reference information in which the position of the additional information is within a predetermined range in the presentation data into the same group, and when any one of the one or more groups is selected, A reference information display step for displaying at least one of a plurality of reference information belonging to the selected group; and a control for controlling the own device and one or more other devices. The reference information acquisition step is executed by the computer, and the read data acquisition step for acquiring the read data obtained by reading the recording medium on which the image based on the announcement data is formed, and the read data acquired by the read data acquisition step are changed to the announcement data. As an additional information extraction step for extracting an image that does not exist as additional information, and a position in the presentation data corresponding to the position of the extracted additional information in the read data as position information indicating the position where the extracted additional information is added A position determining step for determining, and a recording medium correspondence generating step for generating reference information corresponding to the recording medium based on the additional information extracted by the additional information extracting step and the position determined by the position determining step.

  According to this aspect, it is possible to provide a conference related information reproduction program that can easily reproduce more information related to a conference.

It is a figure which shows an example of the whole outline | summary of the conference system containing the portable information device which concerns on embodiment of this invention. 2 is a block diagram showing an outline of a hardware configuration of an MFP according to the present embodiment. FIG. It is a block diagram which shows the outline | summary of the hardware constitutions of the portable information device in this Embodiment. It is a block diagram which shows an example of the function which CPU with which a portable information device is provided has. It is a figure which shows an example of the format of audio | voice related information. It is a figure which shows an example of the format of presentation image related information. It is a figure which shows an example of the format of pointer related information. It is a figure which shows an example of the format of the reference information corresponding to presentation data. It is a figure which shows an example of the format of the reference information corresponding to a recording medium. It is a figure which shows an example of the display state of a display part when the user A operates a portable information device and participates in a meeting. It is a figure which shows an example of the reference information corresponding to the announcement data produced | generated in the user's A portable information device. It is a figure which shows an example of the reference information corresponding to the announcement data produced | generated in the portable information device of the user B. It is a figure which shows an example of the reference information corresponding to the announcement data produced | generated in the portable information device of the user C. It is a figure which shows an example of the reference information corresponding to the announcement data produced | generated in the portable information device of the user D. It is a figure which shows an example of the reference information corresponding to the presentation data produced | generated in the portable information device of the user E. It is a schematic diagram for demonstrating an example of the reference information corresponding to the user's F presentation manuscript. It is a schematic diagram for demonstrating an example of the reference information corresponding to the user's G presentation manuscript. It is a schematic diagram for demonstrating an example of the reference information corresponding to the user's H presentation manuscript. It is a schematic diagram which shows the example of grouping of the reference information corresponding to presentation data. It is a schematic diagram which shows the example of grouping of the reference information based on a presentation manuscript. It is a figure which shows an example of a group list. It is a figure which shows an example of a group list. It is a figure which shows the example of a display when one group identification information is selected from a group list. It is a display example when the reference image of the user F is selected. It is a display example when the reference image of the user G is selected. FIG. 25 is a display example when a character size button is operated in the display state of FIG. 24. It is a figure which shows the other example of a display when one group identification information is selected from a group list. It is a display example when the reference image of the user H is selected. It is a flowchart which shows an example of the flow of a meeting information acquisition process. It is a flowchart which shows an example of the flow of a presentation image related process. It is a flowchart which shows an example of the flow of the reference information acquisition process corresponding to presentation data. It is a flowchart which shows an example of the flow of the reference information acquisition process corresponding to a presentation manuscript. It is a flowchart which shows an example of the flow of a grouping process. It is a flowchart which shows an example of the flow of a reproduction process. It is a flowchart which shows an example of the flow of a 1st reference information display process. It is a flowchart which shows an example of the flow of a 2nd reference information display process.

  Hereinafter, a conference related information reproducing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. Hereinafter, a portable information device will be described as an example of a conference related information reproduction device.

  FIG. 1 is a diagram showing an example of an overall outline of a conference system including a portable information device according to an embodiment of the present invention. Referring to FIG. 1, a conference system 1 includes an MFP (Multi Function Peripheral) 400, which is a multifunction device, and a plurality (five in this example) of portable information devices 200A, 200B, 200C, 200D, and 200E. . MFP 400 and portable information devices 200 </ b> A to 200 </ b> E may be arranged at the same location or at different locations. When MFP 400 and portable information devices 200A to 200E are arranged at a plurality of bases, the distances between the plurality of bases are not particularly limited.

  MFP 400 and wireless station 3A are connected to network 2A. The portable information devices 200A and 200B can be connected to the network 2A by communicating with the wireless station 3A. A wireless station 3B is connected to the network 2B. The portable information devices 200C, 200D, and 200E can be connected to the network 2B by communicating with the wireless station 3B. Each of the networks 2A and 2B is connected to the Internet 5. Therefore, MFP 400 and portable information devices 200A to 200E can communicate with each other.

  The networks 2A and 2B are LANs (local area networks), and the connection form may be wired or wireless. The networks 2A and 2B are not limited to a LAN, but may be a WAN (wide area network), a PSTN (public switched telephone network), the Internet, or the like.

  Each of the portable information devices 200A to 200E is a personal computer. Since the portable information devices 200A to 200E have the same hardware configuration and functions, the portable information device 200A will be described as an example unless otherwise specified. In addition, each of portable information device 200A-200E should just have the function of FIG. Therefore, each of portable information devices 200A to 200E may be a portable information terminal having a function similar to the function of FIG. Examples of such a portable information terminal include a smartphone, a tablet terminal, and a PDA (Personal Digital Assistant).

  FIG. 2 is a block diagram illustrating an example of an outline of a hardware configuration of the MFP 400. Referring to FIG. 2, MFP 400 includes a main circuit 410. The main circuit 410 includes a CPU 411, a communication interface (I / F) unit 412, a ROM (Read Only Memory) 413, a RAM (Random Access Memory) 414, and a hard disk drive (HDD) 415 as a mass storage device. , A facsimile unit 416 and an external storage device 417. CPU 411 is connected to automatic document feeder 420, document reading unit 430, image forming unit 440, sheet feeding unit 450, and operation panel 460, and controls the entire MFP 400.

  The ROM 413 stores a program executed by the CPU 411 or data necessary for executing the program. The RAM 414 is used as a work area when the CPU 411 executes a program. The RAM 414 temporarily stores the read images continuously sent from the document reading unit 430.

  The facsimile unit 416 is connected to the public switched telephone network (PSTN) and transmits facsimile data to the PSTN or receives facsimile data from the PSTN. The facsimile unit 416 stores the received facsimile data in the HDD 415 or converts it into print data that can be printed by the image forming unit 440 and outputs the print data to the image forming unit 440. As a result, the image forming unit 440 forms an image based on the facsimile data received by the facsimile unit 416 on the paper supplied by the paper feeding unit 450. In addition, the facsimile unit 416 converts the data read by the document reading unit 430 or the data stored in the HDD 415 into facsimile data, and transmits the facsimile data to a facsimile machine connected to the PSTN.

  Communication I / F unit 412 is an interface for connecting MFP 400 to network 2A. The communication I / F unit 412 communicates with the portable information devices 200A to 200E connected to the networks 2A and 2B using a communication protocol such as TCP (Transmission Control Protocol) or FTP (File Transfer Protocol).

  The external storage device 417 is controlled by the CPU 411, and a CD-ROM (Compact Disk Read Only Memory) 418 or a semiconductor memory is mounted. The CPU 411 can access the CD-ROM 418 or the semiconductor memory via the external storage device 417. The CPU 411 loads the program recorded on the CD-ROM 418 or the semiconductor memory mounted on the external storage device 417 into the RAM 414 and executes it. Note that the program executed by the CPU 411 is not limited to the program recorded in the CD-ROM 418, and the program stored in the HDD 415 may be loaded into the RAM 414 and executed. In this case, another computer connected to the network rewrites the program stored in the HDD 415 of the MFP 400 or adds a new program and writes it via the network connected to the communication I / F unit 412. You may do it. Further, MFP 400 may download a program from another computer connected to the network and store the program in HDD 415. The program here includes not only a program directly executable by the CPU 411 but also a source program, a compressed program, an encrypted program, and the like.

  The medium for storing the program executed by the CPU 411 is not limited to the CD-ROM 418, but an optical disc (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC card, optical card, It may be a semiconductor memory such as a mask ROM, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM).

  Operation panel 460 includes a display unit 461 and an operation unit 463. The display unit 461 is a display such as a liquid crystal display (LCD) or an organic ELD (Electro-Luminescence Display), and displays an instruction menu for the user, information about acquired image data, and the like. The operation unit 463 includes a touch panel 465 and a hard key unit 467 including a plurality of keys. Each of the plurality of keys included in the hard key unit 467 includes a contact switch and is connected to the CPU 411.

  When a plurality of keys of the hard key unit 467 are pressed, the operation unit 463 receives input of data such as instructions, characters, numbers, and the like corresponding to the pressed keys. The touch panel 465 is provided on the upper surface or the lower surface of the display unit 461, and outputs the coordinates of the position designated by the operation user to the CPU 411. The touch panel 465 detects a position designated by the operation user with a finger or a stylus pen, and outputs the coordinates of the detected position to the CPU 411.

  The touch panel 465 is preferably the same size or larger than the display surface of the display unit 461. Since the touch panel 465 is provided so as to overlap with the display unit 461, the touch panel 465 indicates the coordinates of the position designated by the operation user on the display surface of the display unit 461 when the operation user points to the display surface of the display unit 461. It outputs to CPU411. For the touch panel 465, for example, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, or a capacitance method can be used, and the method is not limited.

  The operations that can be accepted by the operation unit 463 include an operation of pressing a plurality of keys of the hard key unit 467, a single touch operation that instructs the touch panel 465, and a multi-touch operation.

  The HDD 415 stores in advance a schedule to which one or more conference dates and times are assigned and announcement data that serves as material used in each conference. The presentation data is data shared by one or more users participating in the conference, and can be accessed by each of the portable information devices 200A to 200E. Moreover, CPU411 manages the user who participates in a meeting for every predetermined meeting using portable information device 200A-200E. For example, each user of portable information devices 200 </ b> A to 200 </ b> E requests the CPU 411 to participate in the conference using a user ID (identifier) and a password given in advance at the start of the conference. In HDD 415, user IDs and passwords respectively corresponding to a plurality of users are stored as management information. When there is a request for participation in the conference from portable information devices 200 </ b> A to 200 </ b> E, CPU 411 determines whether or not to permit the participation request by comparing the user ID and password input together with the participation request with management information. .

  When the CPU 411 permits a participation request to the conference from any of the portable information devices 200A to 200E, the CPU 411 causes the portable information device to download conference presentation data corresponding to the participation request. In addition, the CPU 411 manages the operation of one or more portable information devices that have permitted the participation request among the portable information devices 200A to 200E.

  In the conference system 1 according to the present embodiment, each of a plurality of users participating in the conference using the portable information devices 200A to 200E can request the CPU 411 to be a presenter. When the CPU 411 receives a request to become a presenter from a certain user, the CPU 411 determines whether to permit the request from the user according to a predetermined condition.

  For example, when permitting a request to become a presenter from any one of the portable information devices 200A to 200E, for example, the portable information device 200A, the CPU 411 grants a presentation authority to the portable information device. On the other hand, when the CPU 411 does not permit the request from the portable information device 200A to be a presenter, the CPU 411 does not grant the presentation authority to the portable information device 200A. Presentation authority specifies the image (hereinafter referred to as the presentation image) that should be presented to all users participating in the conference out of the presentation data for one conference, and outputs its own remarks to all users. Means the right to do.

  An arbitrary user can instruct the MFP 400 to create a manuscript based on the presentation data stored in the HDD 415 (hereinafter referred to as a presentation manuscript). For example, this is a case where a user who does not carry the band information devices 200A to 200E participates in a conference using a sheet on which an image based on presentation data is formed. When the user inputs an operation for designating presentation data by operating the operation panel 460, the CPU 411 controls the image forming unit 440 to form an image based on the presentation data on a sheet. In this way, a presentation manuscript is created by the MFP 400. The presentation manuscript is an example of a recording medium on which an image based on the presentation data is formed. The size of the presentation manuscript is a size designated by the user who has instructed the creation of the presentation manuscript. For example, when the paper size of A4 is designated, the size of the announcement manuscript is A4, and when the paper size of A3 is designated, the size of the announcement manuscript is A3. Further, when the presentation data is composed of a plurality of pages, the presentation manuscript is composed of the same number of sheets as the number of pages of the plurality of pages when formed on a sheet for each of the plurality of pages included in the presentation data. The presentation manuscript is composed of a smaller number of sheets than the number of pages of a plurality of pages when two or more images of a plurality of pages included in the presentation data are formed on one sheet. For example, when 10 pages are included in the presentation data and 2 pages are formed on one sheet, the presentation manuscript is composed of 5 sheets. When 4 pages are formed on one sheet, The presentation manuscript consists of three sheets.

  A user who has obtained a presentation manuscript may add a handwritten memo to the presentation manuscript while participating in a conference. Here, the image of the presentation manuscript to which the handwritten memo has been added is called a presentation manuscript image. In this case, for example, after the end of the meeting, the user can log in to the MFP 400 by operating the operation panel 460 and allow the MFP 400 to read the presentation manuscript with the memo added. Specifically, when the user places an announcement document on the automatic document feeder 420 and inputs an operation to instruct the operation panel 460 to read, the CPU 411 controls the document reading unit 430 to display the announcement document image. And the read data read is stored in the HDD 415 in association with the user identification information of the logged-in user.

  Further, the user who has read the presentation original image by MFP 400 can cause MFP 400 to transmit a set of read data and user identification information to any of portable information devices 200A to 200E. For example, a user who causes the MFP 400 to read a presentation document image operates the operation panel 460 to specify one of the portable information devices 200A to 200E and input an operation to instruct transmission of read data stored in the HDD 415 Then, the CPU 411 transmits the read data and the user identification information to the designated portable information device. In addition, when there is a request for transmission of the read data stored in the HDD 415 from any of the portable information devices 200A to 200E, the CPU 411 includes the requested read data and user identification information associated with the read data. Is transmitted to the portable information device that has transmitted the transmission request.

  FIG. 3 is a block diagram showing an outline of the hardware configuration of portable information device 200A in the present embodiment. Referring to FIG. 3, portable information device 200A according to the present embodiment stores a central processing unit (hereinafter referred to as “CPU”) 201 for controlling overall portable information device 200A, a program executed by CPU 201, and the like. ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203 used as a work area of the CPU 201, a flash memory 204 for storing data in a nonvolatile manner, a display unit 205 for displaying information, a user's An operation unit 206 that accepts operations, an audio input unit 207 that receives audio around the portable information device 200A, an audio output unit 208 that outputs audio, a wireless LAN I / F (wireless local area network interface) 209, and an external storage Device 210.

  The flash memory 204 stores a program executed by the CPU 201 or data necessary for executing the program. The CPU 201 loads the program recorded in the flash memory 204 to the RAM 203 and executes it. When the CPU 201 executes a program recorded in the flash memory 204, reference information and conference information described later are generated, and the generated reference information and conference information are stored in the flash memory 204.

  The display unit 205 is an LCD (liquid crystal display device), an organic ELD (organic electroluminescence display device), or the like. The operation unit 206 includes a main key 206A, a touch panel 206B, a keyboard 206C, and a mouse 206D. The main key 206A is a hard key composed of, for example, a contact switch. The touch panel 206 </ b> B is provided to overlap the display unit 205 on the upper surface or the lower surface of the display unit 205. The touch panel 206 </ b> B detects a position designated by the user on the display surface of the display unit 205, and outputs the detected position of the display surface to the CPU 201.

  Based on the screen displayed on the display unit 205 and the position detected by the touch panel 206B, the CPU 201 accepts input of various instructions, data such as characters and numbers by user operations. For example, when a screen including a numeric keypad image is displayed on the display unit 205, a number corresponding to the key displayed at the position detected by the touch panel 206B is received. Further, the CPU 201 can detect drawing based on position information input from the touch panel 206B. Specifically, the CPU 201 generates drawing data indicating a handwritten image in which the pixel value of a pixel detected by the touch panel 206B is “1” and the pixel value of another pixel is “0”.

  The keyboard 206C includes a plurality of alphanumeric keys. The CPU 201 detects an alphanumeric key pressed by the user and accepts input of data such as various characters and numbers by the user's operation. The mouse 206D includes a position indicating mechanism for indicating an arbitrary position with a mouse pointer displayed on the display unit 205, and one or more hard keys. The position instruction mechanism of the mouse 206D outputs position information instructed by the user to the CPU 201. The hard key of the mouse 206D outputs an on / off signal to the CPU 201 by a user operation. The CPU 201 detects a position designated by the user on the display surface of the display unit 205 based on position information input from the mouse 206D. Further, the CPU 201 detects an operation state (for example, a click operation, a drag operation, etc.) of the mouse 206D by the user based on an on / off signal input from the mouse 206D. Further, the CPU 201 can detect drawing based on position information and an operation state detected by position information and an on / off signal input from the mouse 206D. Specifically, the CPU 201 indicates a handwritten image in which the pixel value of the pixel at the position on the display surface of the display unit 205 indicated by the mouse 206D is “1” and the pixel values of the other pixels are “0”. Drawing data can also be generated.

  The audio input unit 207 is a microphone, for example, and the audio output unit 208 is a speaker, for example. The voice input unit 207 collects voices of users around the portable information device 200 </ b> A, converts the collected sounds into voice data, and outputs the voice data to the CPU 201. The audio output unit 208 is controlled by the CPU 201, reproduces audio data input from the CPU 201, and generates sound.

  The wireless LAN I / F 209 is an interface for communicating with the wireless station 3A and connecting the portable information device 200A to the network 2A. By registering the IP (Internet Protocol) address of MFP 400 in portable information device 200A, portable information device 200A can communicate with MFP 400 and transmit / receive data. In the present embodiment, a case where portable information device 200A communicates with MFP 400 using wireless LAN I / F 209 is described as an example, but communication may be performed using another communication method. Specifically, when portable information device 200A and MFP 400 are mounted with a short-range wireless device such as Bluetooth (registered trademark), for example, portable information device 200A communicates with MFP 400 on a one-to-one basis. Also good.

  The external storage device 210 is detachable from the portable information device 200A, and can be loaded with a CD-ROM 210A in which a conference related information reproduction program is stored. The CPU 201 can access the CD-ROM 210 </ b> A via the external storage device 210. The CPU 201 can load the conference related information reproduction program recorded on the CD-ROM 210 </ b> A attached to the external storage device 210 into the RAM 203 and execute it. Further, the CPU 201 may store the conference related information reproduction program recorded on the CD-ROM 210 </ b> A in the ROM 202 or the flash memory 204. In this case, the CPU 201 can load the conference related information reproduction program stored in the ROM 202 or the flash memory 204 into the RAM 203 and execute it.

  The program executed by the CPU 201 is not limited to a program stored in advance in the ROM 202, the flash memory 204, and the CD-ROM 210A. The CPU 201 may execute, for example, a program stored in the flash memory 204 and rewritten by another computer connected to the networks 2A and 2B or the Internet 5. Further, the CPU 201 may execute a new program additionally written in the flash memory 204 by another computer connected to the networks 2A and 2B or the Internet 5. Further, the CPU 201 may execute a program downloaded from another computer connected to the network 2 or the Internet 5. The program here includes not only a program directly executable by the CPU 201 but also a source program, a compressed program, an encrypted program, and the like.

  The medium for storing the program executed by the CPU 201 is not limited to the CD-ROM 210A, but may be an optical disc such as an MO (Magnetic Optical disc), an MD (Mini Disc), or a DVD (Digital Versatile Disc). The medium for storing the program executed by the CPU 201 may be a semiconductor memory such as an IC card, an optical card, a mask ROM, an EPROM (Erasable Programmable ROM), and an EEPROM (Electrically EPROM).

  FIG. 4 is a block diagram illustrating an example of functions of the CPU 201 provided in the portable information device 200A. In FIG. 4, portable information devices 200B to 200E are shown as other devices.

  The functions shown in FIG. 4 are functions formed in the CPU 201 when the CPU 201 included in each of the portable information devices 200A to 200E executes various programs stored in the ROM 202, the flash memory 204, and the CD-ROM 210A. The functions of the CPU 201 provided in each of the portable information devices 200A to 200E are the same. Hereinafter, the functions of the CPU 201 of the portable information device 200A will be described on behalf of the CPU 201 of the portable information devices 200A to 200E.

  Referring to FIG. 4, CPU 201 obtains conference information acquisition unit 110, reference information acquisition unit 120, grouping unit 130, group list display unit 131, group period determination unit 133, selection unit 134, and playback. A period determining unit 138, a reference information display unit 150, a conference information output unit 160, a reproduction operation instruction unit 170, and a timer unit 190 are included. The timekeeping unit 190 acquires the current time information and outputs the acquired time information to the conference information acquisition unit 110 and the reference information acquisition unit 120.

  The meeting information acquisition unit 110 acquires information regarding the state of the meeting that changes over time as meeting information. The meeting information is generated by any of portable information devices 200A to 200E of users who participate in the meeting. The conference information acquisition unit 110 generates conference information in a state where the device mode is the announcement mode, and transmits the generated conference information to the other portable information devices 200B to 200E via the wireless LAN I / F 209. On the other hand, the conference information acquisition unit 110 receives the conference information transmitted from the portable information device in the announcement mode among the other portable information devices 200B to 200E in a state where the device mode is in the participation mode. Since the conference information acquisition unit 110 generates conference information in the announcement mode and receives the conference information in the participation mode, the conference information acquisition unit 110 continuously acquires the conference information during the conference in which the user of the portable information device 200A participates.

  Here, a case will be described in which conference information is generated by portable information device 200A in which the device mode is the announcement mode among portable information devices 200A to 200E of users participating in the conference. In this case, the conference information acquisition unit 110 included in the portable information device 200A generates conference information and transmits the conference information to the other portable information devices 200B to 200E via the wireless LAN I / F 209. The conference information acquisition unit 110 included in each of the other portable information devices 200B to 200E acquires conference information by receiving the conference information generated by the portable information device 200A from the portable information device 200A.

  The meeting information includes audio related information, presentation image related information, and pointer related information. The voice related information is information in which the voice of the presenter to whom the presentation authority is given is associated with the time when the voice is generated. The presentation image related information is information in which the presentation part designated by the presenter in the presentation data is associated with the time at which the presentation part is designated. The pointer related information is information in which the position of the mouse pointer designated by the presenter is associated with the time when the position is designated.

  The conference information acquisition unit 110 switches the device mode between the presentation mode and the participation mode while the user is participating in the conference. Specifically, the conference information acquisition unit 110 sets the device mode to the participation mode by default. When the presentation mode is selected by the operation unit 206 in a state where the apparatus mode is the participation mode, the conference information acquisition unit 110 controls the wireless LAN I / F 209 and sends a request signal indicating that it should be a presenter. Send to MFP 400. The conference information acquisition unit 110 switches the device mode to the presentation mode on condition that the MFP 400 permits the request to be a presenter, that is, when the presentation authority is given to the user of the portable information device 200A. On the other hand, when the request for becoming a presenter is not permitted by the MFP 400, that is, when the presentation authority is not granted to the user of the portable information device 200A, the conference information acquisition unit 110 does not switch the device mode to the presenter mode. Maintain participation mode.

  The portable information device 200A can set the presentation part in a state where the device mode is switched to the presentation mode. The presentation part is a part displayed on each of the portable information devices 200A to 200E of the users participating in the conference among the presentation data used for the conference. Portable information device 200A cannot set the presentation part in the participation mode.

  The conference information acquisition unit 110 acquires audio data input from the audio input unit 207. The conference information acquisition unit 110 detects the presentation start time and the presentation end time based on the time information acquired by the timer unit 190 in the presentation mode. The presentation start time is the time when the device mode is switched from the participant mode to the presentation mode, and is the time when the user of the portable information device 200A becomes the presenter. The presentation end time is the time when the device mode is switched from the presentation mode to the participant mode, and is the time when the user of the portable information device 200A is no longer a presenter. The conference information acquisition unit 110 acquires audio data between the presentation start time and the presentation end time. The conference information acquisition unit 110 generates voice related information by associating the voice data with the detected presentation start time and presentation end time.

  The conference information acquisition unit 110 detects the position of the mouse pointer when the presenter operates the mouse 206D in the state of the presentation mode. The conference information acquisition unit 110 generates pointer-related information by associating the detected position of the mouse pointer with the time information acquired by the time measuring unit 190 in the presentation mode.

  The meeting information acquisition unit 110 includes a presentation part setting unit 111. When the presenter specifies the presentation page by operating the operation unit 206 in the state of the presentation mode, the presentation part setting unit 111 extracts the designated presentation page and sets the entire presentation page as the presentation part. . In addition, when the presenter expands or reduces the presentation page by operating the operation unit 206 in a state where the presentation page is extracted, the presentation part setting unit 111 sets the expanded or reduced area in the presentation page as the presentation part. Set. The conference information acquisition unit 110 generates the presentation image related information by associating the announcement part set by the announcement part setting unit 111 in the announcement mode with the time information acquired by the timer unit 190.

  In response to acquiring the conference information, the conference information acquisition unit 110 displays an image of the presentation part specified by the presentation image related information included in the conference information on the display unit 205 and refers to the acquired conference information The information is output to the information acquisition unit 120, the group list display unit 131, and the conference information output unit 160.

  FIG. 5 is a diagram illustrating an example of a format of audio related information. Referring to FIG. 5, the voice related information includes a voice data item, a presentation start time item, and a presentation end time item. In the audio data item, audio data obtained by converting the presenter's audio into electronic data is set. The time at which the device mode is switched to the presentation mode is set in the presentation start time item, and the time at which the device mode is switched from the presentation mode to the participant mode is set in the presentation end time item.

  FIG. 6 is a diagram illustrating an example of the format of the presentation image related information. Referring to FIG. 6, the presentation image related information includes a page data item, a partial data item, and a time information item. In the page data item, page identification information for identifying a presentation page designated by the presenter among a plurality of pages included in the presentation data used for the conference is set. In the partial data item, area identification information for identifying an area designated by the presenter in the presentation page is set. In the time information item, the time when the region is designated by the presenter in the presentation page is set.

  FIG. 7 is a diagram illustrating an example of a format of pointer related information. Referring to FIG. 7, the pointer related information includes a position item and a time information item. In the position item, position data indicating the position at which the presenter points on the screen of the display unit 205 with the mouse pointer is set. In the time information item, the time at which the position indicated by the position data is specified is set. Is done.

  Returning to FIG. 4, the reference information acquisition unit 120 acquires reference information. Here, the reference information according to the present embodiment includes reference information corresponding to the presentation data and reference information corresponding to the presentation manuscript.

  First, reference information corresponding to the presentation data will be described. The reference information corresponding to the presentation data is the additional information input by each user of the portable information devices 200A to 200E participating in the conference, the position in the presentation data where the additional information is input, and the time when the additional information is input. Information associated with. Specifically, the reference information includes user identification information for identifying a user who has input additional information, additional information, position information indicating a position of the additional information in the announcement data, and an input period.

  When the operation unit 206 receives additional information, the reference information acquisition unit 120 generates reference information corresponding to the presentation data, and transmits the generated reference information to another portable information device 200B via the wireless LAN I / F 209. To each 200E. The additional information includes information that is input by operating the operation unit 206 by the user operating the portable information device 200A. For example, when the keyboard 206C or the soft keyboard receives a character string, the reference information acquisition unit 120 receives the received character string as additional information. Further, when the touch panel 206B detects the position on the surface of the display unit 205 that the user designates with a finger or a stylus pen, the reference information acquisition unit 120 uses handwritten drawing as additional information based on the detected position. Accept.

  When receiving the additional information, the reference information acquisition unit 120 determines a user who operates the portable information device 200A, a position in the presentation data where the additional information is input, and an input period. Specifically, the reference information acquisition unit 120 determines a user who has logged into the portable information device 200A as a user who operates the portable information device 200A. In addition, the reference information acquisition unit 120 adds the presentation part specified by the presentation image related information included in the meeting information input from the meeting information acquisition part 110 and the presentation part image displayed on the display unit 205. The position of the area where information is input is determined as position information. The position information includes page identification information for identifying a page including a presentation part among a plurality of pages included in the conference data, and area position information indicating a position of an area where additional information is arranged in the page. Including. Further, the reference information acquisition unit 120 determines the input period based on the time information acquired by the time measuring unit 190. The input period is a period from the input start time when input of additional information is started to the input end time when input of additional information is completed.

  When the wireless LAN I / F 209 receives reference information corresponding to the announcement data from any of the other portable information devices 200B to 200E, the reference information acquisition unit 120 acquires the received reference information. For this reason, the reference information acquisition part 120 produces | generates the reference information corresponding to the additional information which the user who operates 200A of portable information inputs, and respond | corresponds to the additional information which the user who operates portable information device 200B-200E inputs. Since the reference information is received, the reference information corresponding to each additional information input by all users participating in the conference is continuously acquired during the conference period. The reference information acquisition unit 120 outputs reference information corresponding to the acquired presentation data to the grouping unit 130 and the reproduction period determination unit 138.

  FIG. 8 is a diagram illustrating an example of a format of reference information corresponding to presentation data. Referring to FIG. 8, the reference information corresponding to the presentation data includes a user ID item, an additional information item, a position information item, and an input period item. In the user ID item, user identification information for identifying the user is set. In the additional information item, additional information input by the user specified by the user identification information set in the user ID item is set. In the position information item, position information indicating the position of an area where additional information is input in the conference data is set. The input period item includes a start time item and an end time item. The start time item is set to the input start time when the input of the additional information is started, and the end time item is set to the input end time when the input of the additional information is ended.

  As described above, the additional information corresponding to the presentation data is input on the image of the presentation part displayed on the display unit 205. Here, it is assumed that the presentation data includes a predetermined reference image (for example, a frame) for defining the reference size of the page for each page. In this case, the relative size of the additional information with respect to the reference size is the ratio of the size of the announcement portion to the reference size, the image size of the announcement portion displayed on the display unit 205, and the additional information input on the image of the announcement portion. It is uniquely determined based on the size.

  Next, reference information corresponding to the presentation manuscript will be described. The reference information corresponding to the presentation manuscript is basically data which is a memo handwritten on the presentation manuscript by a user who participated in the conference using the presentation manuscript.

  Referring to FIG. 4, reference information acquisition section 120 generates reference information corresponding to the presentation manuscript based on the read data of the presentation manuscript. The reference information acquisition unit 120 includes a read data acquisition unit 121, an additional information extraction unit 122, a position determination unit 123, a recording medium correspondence generation unit 124, a condition image determination unit 125, a user identification information acquisition unit 126, and a size determination unit 127. .

  For example, when the user operates operation panel 460 of MFP 400 to instruct transmission of read data to portable information device 200A, MFP 400 corresponds to the read data stored in HDD 415 and the read data as described above. User identification information is transmitted to portable information device 200A. Read data acquisition unit 121 acquires read data received from MFP 400. Further, user identification information acquisition unit 126 acquires user identification information received together with the read data from MFP 400.

  As described above, the size of the presentation manuscript is not necessarily constant. Further, in the presentation manuscript, the number of pages formed on one sheet is not always constant. Therefore, in the read data, there is a high possibility that the ratio of the image size per page to the reference size differs for each published manuscript.

  Therefore, the additional information extraction unit 122 according to the present embodiment first extracts one or more pages included in the presentation manuscript from the read data corresponding to one presentation manuscript. This extraction process is executed, for example, by detecting a reference image provided for each page of the presentation manuscript using pattern matching or the like.

  Thereafter, the additional information extraction unit 122 extracts an image that does not exist in the presentation data from the read data as additional information by comparing the extracted read data for each page with the announcement data for each page. That is, the additional information extraction unit 122 extracts an image that exists in the presentation manuscript image but does not exist in the presentation manuscript as additional information corresponding to the presentation manuscript. At this time, the additional information extraction unit 122 may vectorize the extracted additional information. Thereby, the image quality of the additional information when displaying the additional information can be improved, and the data amount of the additional information can be reduced.

  Further, the additional information extraction unit 122 defines the image size for one page of the read data as the read reference size based on the reference image included in the read data for each extracted page, and then adds the read data to the read reference size. The ratio of the information size is calculated as the enlargement / reduction ratio.

  The presentation manuscript includes a blank area surrounding the presentation image area in addition to the presentation image area in which the presentation image is formed. Therefore, the user who uses the presentation manuscript may write a memo in the blank area using a lead line extending from the presentation image area to the blank area. Therefore, an image having a specific shape that is considered to indicate that there is a memo in the blank area is set as the condition image. In the present embodiment, an image that extends outward from any of a plurality of pages of images included in the presentation data, for example, a linear image that extends from the presentation image area to the blank area in the presentation document image is set as the condition image. To do.

  The condition image determination unit 125 determines whether or not a condition image exists in the additional information extracted by the additional information extraction unit 122.

  The position determination unit 123 determines a position in the presentation data corresponding to the position of the extracted additional information in the read data as an added position of the extracted additional information, and adds information indicating the determined position The position information corresponds to the information. In addition, when the condition image determination unit 125 determines that the condition image exists, the position determination unit 123 determines the condition image of the page in the page of the announcement image that overlaps the condition image among a plurality of pages included in the presentation data. The position of the end in the presentation image area is determined as the position where the additional information is added. Thereby, the position of the additional information can be determined as an appropriate position within the page of the presentation data.

  Furthermore, when the additional information extracted by the additional information extraction unit 122 does not exist in any of the images of the plurality of pages included in the presentation data, the position determination unit 123 is extracted from the plurality of pages included in the presentation data. The page corresponding to the image with the closest additional information and the relative position of the extracted additional information with respect to the corresponding page are determined as the position of the additional information.

  As described above, the additional information extracted for the page corresponding to the image whose additional information is closest to the position in the presentation data of the additional information extracted from the read data obtained by reading the presentation manuscript on which the image based on the presentation data is read. Since the relative position is determined, the additional information extracted from the read data can be classified into one or more groups. For this reason, the additional information extracted from the read data can be reproduced together with the additional information added to the presentation data.

  Here, the size of the additional information written on the presentation image or the presentation manuscript varies depending on the user. When the additional information is characters, if there is a variation in the size of the plurality of additional information written by a plurality of users, each additional information is confirmed when displaying the plurality of additional information on the display unit 205 as described later. Hard to do.

  Each of the plurality of characters included in the additional information can be detected based on, for example, a histogram of pixel values representing a drawing image including the additional information, and the size of one character can be detected. Therefore, the size determination unit 127 is based on at least one of the read data and the presentation data so that each of the plurality of additional information can be displayed on the display unit 205 with an appropriate character size. For each additional information, an ideal size of the additional information with respect to the reference size is determined as an appropriate character size. In this case, the appropriate character size may be, for example, the same size as the maximum, minimum, or average size of a plurality of characters described in the presentation manuscript corresponding to the read data, or a plurality of characters in the presentation data. It may be the same size as the maximum, minimum or average size. As a result, the extracted additional information is displayed in a size corresponding to the size of each page or character of the presentation data, so that variation in the size of the displayed additional information can be reduced.

  Alternatively, the size determination unit 127 adds the additional information to the presentation image in any of the portable information devices 200A to 200E so that each of the plurality of additional information can be displayed on the display unit 205 with an appropriate character size. An appropriate character size may be determined based on the size of the added information. In this case, the appropriate character size may be, for example, the same size as the maximum, minimum, or average size of the plurality of additional information added to the presentation image in any of the portable information devices 200A to 200E. Thereby, since the extracted additional information is displayed in a size corresponding to the size of the additional information added to the presentation data by any of the portable information devices 200A to 200E, the variation in the size of the additional information is reduced. Can do.

  The recording medium correspondence generation unit 124 includes user identification information acquired by the user identification information acquisition unit 126, additional information extracted by the additional information extraction unit 122, an enlargement / reduction ratio calculated by the additional information extraction unit 122, and a position determination unit. Based on the position information indicating the position determined by 123, the determination result by the condition image determination unit 125, and the appropriate character size determined by the size determination unit 127, reference information corresponding to the presentation manuscript is generated.

  FIG. 9 is a diagram illustrating an example of a format of reference information corresponding to a presentation manuscript. Referring to FIG. 9, reference information corresponding to the presentation manuscript includes user ID items, additional information items, position information items, enlargement / reduction ratio items, appropriate character size items, and condition images. And a determination result item. In the user ID item, user identification information acquired by the user identification information acquisition unit 126 is set. In the additional information item, additional information extracted by the additional information extracting unit 122 is set. In the position information item, position information indicating the position of the additional information determined by the position determination unit 123 is set. An enlargement / reduction ratio calculated by the additional information extraction unit 122 is set in the item of enlargement / reduction ratio. In the item “appropriate character size”, an appropriate character size determined by the size determining unit 127 is set. In the condition image determination result item, a determination result by the condition image determination unit 125 is set. The reference information acquisition unit 120 outputs reference information corresponding to the acquired presentation manuscript to the grouping unit 130.

  The reference information corresponding to the presentation manuscript need only include at least items of additional information and position information, and does not need to include items of user ID, enlargement / reduction ratio, appropriate character size, and conditional image determination result.

  Returning to FIG. 4, the grouping unit 130 classifies the plurality of reference information input from the reference information acquisition unit 120 into one or more groups. In the present embodiment, the grouping unit 130 first has a plurality of reference information corresponding to the presentation data, the position indicated by the position information is within a predetermined range in the presentation data, and the input start time is within a predetermined period. One or more reference information is classified into the same group. Note that one or more pieces of reference information whose input start time is within a predetermined period may be classified into the same group. Here, the predetermined range and the predetermined period may be set by a user who operates portable information device 200 </ b> A, or may be predetermined in conference system 1. When predetermined in the conference system 1, the predetermined range and the predetermined period have the same value in the portable information devices 200A to 200D. The predetermined range and the predetermined period are values set by the user of the portable information device 200A. The shorter the predetermined range or the shorter the predetermined period, the larger the number of groups. Conversely, the longer the predetermined range or the longer the predetermined period, the smaller the number of groups. In this example, the grouping unit 130 classifies one or more pieces of reference information whose position indicated by the position information is within a predetermined range in the presentation data and whose input start time is within a predetermined period into the same group. Explained.

  As described above, the reference information corresponding to the presentation image does not include the input period. Therefore, after grouping a plurality of reference information corresponding to the presentation data, the grouping unit 130 converts each of the plurality of reference information corresponding to the presentation manuscript into a plurality of reference information corresponding to the presentation data. Among the groups, the position of the additional information of the reference information is one or a plurality of groups of the plurality of groups and included in a predetermined range from any position of the plurality of reference information included in the group Classify in all of Thereby, the plurality of reference information corresponding to the presentation data and the plurality of reference information corresponding to the recording medium can be appropriately classified into a plurality of groups. In addition, when there is no group set based on the presentation data within a predetermined range from the position of the reference information corresponding to the presentation manuscript, the grouping unit 130 refers to the group closest to the reference information. Classify information. In addition, when the input from the reference information acquisition unit 120 is only the reference information corresponding to the presentation image, the grouping unit 130 has one or more pieces of reference information in which the position indicated by the position information is within a predetermined range in the presentation data. Are classified into the same group.

  After grouping the plurality of reference information into one or more groups, the grouping unit 130 displays one or more reference information for each of the one or more groups as a group list display unit 131, a group period determination unit 133, and a reference information display unit 150. Output to.

  The group period determination unit 133 receives one or more pieces of reference information corresponding to the presentation data for each of one or more groups from the grouping unit 130. The group period determining unit 133 determines a group period for each of one or more groups. The group period determination unit 133 determines a group period for each of one or more groups based on input periods included in one or more reference information belonging to the group. Specifically, the group period determining unit 133 includes one or more belonging to the group to be processed from the earliest time among one or more input start times respectively included in one or more reference information belonging to the group to be processed. The period from the one or more input end times included in the reference information to the latest time is determined as the group period. The group period determining unit 133 outputs the group period determined for each of one or more groups to the reproduction period determining unit 138.

  Here, when the reference information input from the reference information acquisition unit 120 to the grouping unit 130 is only the reference information corresponding to the presentation image, the reference information input to the group period determination unit 133 has no input period. . Therefore, the group period determination unit 133 determines, for example, the group period of all groups as a period from the presentation start time to the presentation end time when there is no input period in all of the input reference information.

  The group list display unit 131 receives one or more pieces of reference information for each of one or more groups from the grouping unit 130 and receives meeting information from the meeting information acquisition unit 110. The group list display unit 131 displays group identification information for identifying each of one or more groups on the display unit 205 in a selectable manner. The group list display unit 131 generates group identification information for each of the one or more groups based on the one or more reference information for each of the one or more groups from the grouping unit 130. The group identification information includes one or more additional information included in one or more reference information belonging to the group and a position of an area specified by one or more position information included in one or more reference information belonging to the group. It is at least one of the information to show and the group period of the group.

  Here, as an example, the group identification information is one or more additional information included in one or more reference information belonging to the group and one or more position information included in one or more reference information belonging to the group. explain. The group list display unit 131 corresponds to one or more pieces of reference information based on one or more pieces of reference information belonging to the group and presentation image related information included in the meeting information input from the meeting information acquisition unit 110. The above reference image is generated. The reference image includes user identification information included in the reference information, additional information, and an image of an area specified by the position information included in the reference information in the announcement portion specified by the presentation image related information. Specifically, the group list display unit 131 adds the image included in the reference information to the image of the portion including the region specified by the position information included in the reference information in the presentation portion specified by the presentation image related information. The reference image is generated by adding the image of the user identification information included in the reference information to the image obtained by combining the image of the information with the region specified by the position information included in the reference information. The group list display unit 131 generates one or more reference images respectively corresponding to one or more reference information belonging to the group, and generates an image in which the generated one or more reference images are arranged as group identification information. The group list display unit 131 displays on the display unit 205 a group list in which one or more group identification information generated corresponding to one or more groups is arranged.

  The selection unit 134 includes a group selection unit 135, a reference information selection unit 136, and an index selection unit 137. The group selection unit 135 selects one of one or more pieces of group identification information displayed on the display unit 205 so as to be selectable by the group list display unit 131 based on an operation that the user inputs to the operation unit 206. Specifically, for example, the group selection unit 135 receives a click operation of the mouse 206D in a state where the mouse pointer is placed at any position of one or more group identification information displayed on the display unit 205, for example. The group identified by the group identification information displayed at the position where the mouse pointer is placed is selected. As a result, the user can easily select a desired group. The group selection unit 135 outputs group identification information of the selected group to the reproduction period determination unit 138 and the reference information display unit 150 in response to the user selecting a group.

  The reference information selection unit 136 is based on an operation in which the user inputs one of one or more reference images included in the selected group identification information to the operation unit 206 in a state where the group is selected by the group selection unit 135. To select. Specifically, the reference information selection unit 136 receives a click operation of the mouse 206D in a state where the mouse pointer is placed at any position among one or more reference images included in the group identification information. Reference information corresponding to the reference image displayed at the position where the mouse pointer is placed is selected. The reference information selection unit 136 outputs reference information identification information for identifying the selected reference information to the reproduction period determination unit 138 and the reference information display unit 150 in response to the reference information being selected by the user.

  As will be described later, when reference information is selected by the reference information selection unit 136, additional information of the selected reference information is displayed on the display unit 205 by the reference information display unit 150. At this time, when the reference information includes a determination result indicating the presence of the condition image, an indicator indicating the presence of the reference information is displayed on the display unit 205 instead of the additional information.

  In the state where one or more indicators are displayed on the display unit 205 when the reference information selection unit 136 selects the reference information, the index selection unit 137 allows the user to select one of the one or more indicators on the operation unit 206. Select an indicator based on the input operation. Specifically, the index selection unit 137 receives the click operation of the mouse 206D in a state where the mouse pointer is placed at any position among one or more indices included in the reference information, so that the mouse pointer is An index corresponding to the index displayed at the position to be arranged is selected. The index selection unit 137 outputs index identification information for identifying the selected index to the reference information display unit 150 in response to the index being selected by the user.

  The reproduction period determination unit 138 receives the group periods of one or more groups from the group period determination unit 133 and receives group identification information from the selection unit 134. The reproduction period determination unit 138 receives the group identification information, the group period of the group specified by the group identification information from among the group periods of one or more groups input from the group period determination unit 133 And the determined group period is determined as the reproduction period. The reproduction period indicates a period from the reproduction start time to the reproduction end time. When the group identification information is given from the selection unit 134, the reproduction period determination unit 138 reproduces the group period of the group indicated by the group identification information from among the group periods of one or more groups input from the group period determination unit 133. Decide on the period.

  Further, the reproduction period determination unit 138 may receive the reference information identification information after the group identification information is input from the selection unit 134. When the reference information identification information is given from the selection unit 134, the reproduction period determination unit 138 identifies the reference information indicated by the reference identification information from among the plurality of reference information input from the reference information acquisition unit 120. The input period included in the reference information is determined as the playback period. The playback period determination unit 138 outputs the determined playback period to the conference information output unit 160.

  The conference information output unit 160 receives the conference information from the conference information acquisition unit 110 and the playback period from the playback period determination unit 138. The conference information output unit 160 outputs a portion corresponding to the reproduction period in the conference information. Thereby, it is possible to reproduce the conference state in a period during which the additional information is input in a limited manner. The conference information output unit 160 includes a presentation data partial display unit 161, and includes a conference audio output unit, a text conversion output unit, a static output control unit, and a dynamic output control unit (not shown).

  The presentation data portion display unit 161 determines in advance on the display screen of the display unit 205 a presentation portion corresponding to the presentation period as a presentation image based on the presentation image related information of the conference information acquired by the conference information acquisition unit 110. Displayed in the specified reproduction area. Specifically, the presentation data partial display unit 161 extracts the presentation image related information in which the time within the reproduction period is set in the item of time information in the presentation image related information. The presentation data part display unit 161 displays the presentation part in the presentation data specified by the page identification information set in the page data item of the extracted presentation image related information and the area identification information set in the partial data item. Identify.

  The conference audio output unit outputs the audio of the presenter during the reproduction period from the audio output unit 208 based on the audio related information of the conference information acquired by the conference information acquisition unit 110. Specifically, the conference audio output unit extracts all audio related information in which the time within the reproduction period is set in the audio data item. The conference audio output unit outputs the audio data set in the audio data item of the extracted audio-related information from the audio output unit 208. Thereby, the user can grasp the contents of the presentation at the conference in more detail.

  The text conversion output unit converts voice data of the voice related information acquired from the conference information acquisition unit 110 into text data by voice recognition. The text conversion output unit displays a character string of text data obtained by character recognition of the voice during the reproduction period in a predetermined area on the display surface of the display unit 205. Specifically, the text conversion output unit extracts all audio related information in which the time within the playback period is set in the audio data item. The text conversion output unit converts the extracted voice data of the voice related information into text data, and displays a character string of the converted text data on the display unit 205. Thereby, the voice of the presenter during the meeting can be visually recognized by the user.

  The text conversion output unit may display not only the character string of the text data during the reproduction period but also the character string of the text data in the period other than the reproduction period on the display unit 205. In this case, for example, the text conversion output unit highlights the character string of the text data during the reproduction period in a display mode different from the character string of the text data during the period other than the reproduction period. Thereby, the user can easily recognize the voice of the presenter during the playback period.

  The static output control unit displays a trajectory image indicating the trajectory of the mouse pointer during the reproduction period on the display unit 205 based on the pointer-related information of the conference information acquired by the conference information acquisition unit 110 on the presentation data partial display unit 161. Is superimposed on the presentation image displayed. Specifically, the static output control unit extracts all the pointer related information in which the time within the reproduction period is set in the item of time information among the pointer related information. The static output control unit displays a trajectory image including the position specified by the position data set in the item of the position of the extracted pointer related information in the reproduction region. Since the image of the presentation part during the reproduction period is displayed in the reproduction area, the user can easily recognize the locus of the presenter's mouse pointer along with the presentation part during the reproduction period. The static output control unit changes the display mode of the mouse pointer trajectory image as time elapses and displays it. Thereby, the user can recognize the operation state such as the moving direction and moving speed of the presenter's mouse pointer during the reproduction period.

  The dynamic output control unit superimposes and displays a moving image indicating the movement of the mouse pointer during the reproduction period on the reproduction area of the display unit 205 based on the pointer related information of the conference information acquired by the conference information acquisition unit 110. Specifically, the dynamic output control unit extracts all the pointer related information in which the time within the reproduction period is set in the time information item of the pointer related information. The dynamic output control unit displays an image of the mouse pointer at a position specified by the position data set in the item of position of the extracted pointer related information in the reproduction area. Since the image of the presentation part during the reproduction period is displayed in the reproduction area, the user can easily recognize the operation state such as the position, movement direction, and movement speed of the presenter's mouse pointer during the reproduction period. .

  When the presenter operates the operation unit 206 to instruct an operation related to the reproduction of conference information, the reproduction operation instruction unit 170 receives the instructed operation. Further, the reproduction operation instruction unit 170 controls the dynamic output control unit of the conference information output unit 160 based on the instructed operation. For example, when the reproduction operation instruction unit 170 is instructed to reproduce the conference information, the reproduction operation instruction unit 170 causes the dynamic output control unit to reproduce a moving image indicating the movement of the pointer during the reproduction period. In addition, when an instruction to stop reproduction of conference information is given during reproduction of conference information, the reproduction operation instruction unit 170 causes the dynamic output control unit to stop the moving image being reproduced. Furthermore, when the reverse reproduction of the conference information is instructed, the reproduction operation instruction unit 170 reversely reproduces the moving image indicating the pointer movement during the reproduction period to the dynamic output control unit. In addition, when an instruction to stop reverse playback of conference information is given during reverse playback of conference information, the playback operation instruction unit 170 causes the dynamic output control unit to stop the reverse playback video.

  Note that the reproduction operation instruction unit 170 may control the conference audio output unit by the dynamic output control unit based on the instructed operation. In this case, when the reproduction operation instruction unit 170 is instructed to reproduce the conference information, the dynamic output control unit causes the conference audio output unit to reproduce the presenter's voice during the reproduction period. Further, when the stop of the reproduction of the conference information is instructed during the reproduction of the conference information, the reproduction operation instruction unit 170 stops the audio reproduced by the conference audio output unit by the dynamic output control unit.

  The conference information output unit 160 may reproduce the conference information portion corresponding to the reproduction period determined by the reproduction period determination unit 138 only once. Alternatively, the conference information output unit 160 may repeatedly reproduce the portion of the conference information corresponding to the reproduction period determined by the reproduction period determination unit 138.

  The reproduction period determination unit 138 is a case where the second group is selected after the group selection unit 135 selects the first group, and the first group period and the first group corresponding to the first group are selected. When the second group period corresponding to the second group overlaps with the second group period, after the first group period is determined as the reproduction period, a portion of the second group period that does not overlap with the first group period becomes the reproduction period. decide. In this case, conference information corresponding to a period in which the first group period and the second group period overlap is not repeatedly output. Therefore, the output of conference information is not redundant.

  Furthermore, the reproduction period determining unit 138 is a case where the second reference information is selected after the first reference information is selected by the reference information selecting unit 136, and the first reference information corresponding to the first reference information is selected. When the input period overlaps with the second input period corresponding to the second reference information, after the first input period is determined as the reproduction period, the first input period does not overlap with the first input period. The part is determined as the playback period. Even in this case, the conference information corresponding to the period in which the first input period and the second input period overlap is not repeatedly output. Therefore, the output of conference information is not redundant.

  The reference information display unit 150 receives one or more reference information for each of one or more groups from the grouping unit 130, and receives group identification information from the selection unit 134. The reference information display unit 150 may receive index identification information from the selection unit 134. The reference information display unit 150 includes an additional information display unit 151, an index display unit 152, and an outer side additional information display unit 153. In response to the input of group identification information from the selection unit 134, the additional information display unit 151 adds 1 to the reproduction region of the display unit 205 based on one or more reference information belonging to the group specified by the group identification information. One or more additional information respectively included in the above reference information is displayed. Specifically, the additional information display unit 151 displays an image of the additional information included in the reference information at a position specified by the position information included in the reference information in the reproduction area of the display unit 205. In the reproduction area of the display unit 205, the presentation data image is displayed by the presentation data part display unit 161, so the additional information display unit 151 displays the additional information image superimposed on the presentation part image. The additional information display unit 151 does not display the additional information included in the reference information including the determination information indicating the presence of the condition image or the reference information including the position information in the blank area. This is because such additional information is not located in the reproduction region.

  When the reference information identification information is input from the selection unit 134, the additional information display unit 151 displays an image of the additional information included in the reference information specified by the reference information identification information as another image of the additional information. The display mode is different from the display mode. Thereby, it is possible to notify the user of the position in the image of the presentation portion of the additional image corresponding to the reference image selected by the user.

  The index display unit 152 belongs to a group specified by the group identification information when at least a part of one or more reference information belonging to the group specified by the group identification information includes a determination result indicating the presence of the condition image. When at least a part of one or more pieces of reference information includes additional information located in the blank area, an indicator indicating the presence of the reference information is displayed on the display unit 205. Specifically, the index display unit 152 is a position specified by the position information included in the reference information in the reproduction area of the display unit 205, or in the vicinity of the position specified by the position information and the reproduction area (presentation image area). The indicator indicating the presence of the reference information is superimposed and displayed at a position in Thus, the user can easily recognize the presence of additional information added to the announcement part while referring to the announcement part in the announcement data.

  The outer additional information display unit 153 displays additional information corresponding to the index specified by the index identification information on the display unit 205 in response to the index identification information input from the selection unit 134. This additional information includes the image written in the margin area of the presentation manuscript. At this time, the outside additional information display unit 153 may superimpose the additional information on the image of the presentation part displayed on the display unit 205, or may be different from the image of the presentation part on the screen of the display unit 205. You may display in a position. In this way, additional information that does not exist in any of the images of a plurality of pages in the presentation data can be displayed.

  In the following description, the respective users of portable information devices 200A, 200B, 200C, 200D, and 200E participating in a certain conference are referred to as user A, user B, user C, user D, and user E. The three users who participated in the conference using the presentation manuscript are referred to as user F, user G, and user H.

  FIG. 10 is a diagram illustrating an example of a display state of the display unit 205 when the user A operates the portable information device 200A to participate in the conference. Referring to FIG. 10, participation button 221, presentation button 222, and list button 223 are displayed on screen of display unit 205 of portable information device 200 </ b> A, and a presentation image display area 231 is provided. The user A can operate the participation button 221, the presentation button 222, and the list button 223 using the operation unit 206.

  The participation button 221 displays either a character string “participating” or a character string “personal mode”. The announcement button 222 displays a character string “personal mode” by default. When user A operates participation button 221 on which the character string “personal mode” is displayed, portable information device 200A requests MFP 400 to participate in the conference using the user ID and password. When the MFP 400 permits the request for participation in the conference, the participation button 221 displays a character string “participating”. At this time, the presentation image is displayed in the presentation image display area 231. On the other hand, if the MFP 400 does not permit a request for participation in the conference, the character string “personal mode” is displayed on the participation button 221. When the character string of “personal mode” is displayed on the announcement button 222, it indicates that the user A of the portable information device 200A has not participated in the conference.

  The announcement button 222 displays either a character string “Announcement” or a character string “Write”. The announcement button 222 displays a character string “write” by default. When the user A operates the announcement button 222 on which the character string “write” is displayed, the CPU 201 requests the MFP 400 to become the presenter. When a request to become a presenter is granted by the MFP 400 and a presentation authority is granted, a character string “presentation” is displayed on the presentation button 222. If the MFP 400 does not permit the request to become a presenter and the presentation authority is not granted, the character string “write” remains displayed on the announcement button 222.

  A specific example of reference information corresponding to the presentation data acquired by the CPU 201 will be described. FIG. 11 is a diagram illustrating an example of reference information corresponding to the presentation data generated in the portable information device 200A of the user A. Referring to FIG. 11, a presentation image is displayed in presentation image display area 231, and a handwritten drawing image p <b> 1 is arranged as additional information in the upper left area of the presentation image. Here, the input period of additional information is from 13:00:00 to 13:00:07.

  FIG. 12 is a diagram illustrating an example of reference information corresponding to the presentation data generated in the portable information device 200B of the user B. Referring to FIG. 12, a presentation image is displayed in presentation image display area 231 and an image p2 including a balloon and a character string “erroneous” as additional information is arranged in the upper left area of the presentation image. Here, the additional information input period is from 13:00:05 to 13:00:09.

  FIG. 13 is a diagram illustrating an example of reference information corresponding to the presentation data generated in the portable information device 200C of the user C. Referring to FIG. 13, the presentation image is displayed in the presentation image display area 231, and the handwritten drawing image p <b> 3 is arranged as additional information in the upper right area of the presentation image. Here, the additional information input period is from 13:00:08 to 13:00:28.

  FIG. 14 is a diagram illustrating an example of reference information corresponding to the presentation data generated in the portable information device 200D of the user D. Referring to FIG. 14, a presentation image is displayed in presentation image display area 231 and a handwritten drawing image p4 is arranged as additional information in the lower left area of the presentation image. Here, the additional information input period is from 13:00:30 to 13:00:38.

  FIG. 15 is a diagram illustrating an example of reference information corresponding to the presentation data generated in the portable information device 200E of the user E. Referring to FIG. 15, a presentation image is displayed in presentation image display area 231 and a handwritten drawing image p5 is arranged as additional information in the lower left area of the presentation image. Here, the additional information input period is from 13:00:25 to 13:00:29.

  A specific example of the reference information corresponding to the presentation manuscript generated after the conference will be described. FIG. 16 is a schematic diagram for explaining an example of reference information corresponding to the user F's presentation manuscript. FIG. 16 shows a presentation manuscript in which the presentation images shown in FIGS. 11 to 15 are printed on A4 size paper. In the presentation manuscript of FIG. 16, the boundary line between the presentation image area and the blank area is indicated by a one-dot chain line. In the upper left area in the presentation image area, a drawing image p6 made up of a handwritten character string “with error” is arranged as additional information.

  FIG. 17 is a schematic diagram for explaining an example of the reference information corresponding to the user G's presentation manuscript. FIG. 17 shows a presentation manuscript in which the announcement image shown in FIGS. 11 to 15 and the announcement image of the next page are printed on A3 size paper. In the presentation manuscript of FIG. 17, the boundary line between the presentation image area and the blank area is indicated by a one-dot chain line. Further, in this presentation manuscript, a drawing image p7 made up of a handwritten character string “Naosu” is arranged as additional information in the upper left area of the margin area.

  FIG. 18 is a schematic diagram for explaining an example of reference information corresponding to the user H's presentation manuscript. FIG. 18 shows a presentation manuscript in which the presentation images shown in FIGS. 11 to 15 are printed on A4 size paper. In the presentation manuscript of FIG. 18, the boundary line between the presentation image area and the blank area is indicated by a one-dot chain line. A lead line is arranged as additional information so as to extend from the blank portion to the lower left area in the presentation image area. The lead lines in FIG. 18 correspond to the above condition image. A handwritten character string “point” is arranged as additional information in the vicinity of one end portion of the lead line located in the blank area. A drawn image p8 is formed by the lead line and the handwritten character string.

  FIG. 19 is a schematic diagram illustrating an example of grouping of reference information corresponding to presentation data. FIG. 19 illustrates an example in which the reference information of each of the users A to E illustrated in FIGS. 11 to 15 is grouped. Referring to FIG. 19, reference information input periods of users A to E are indicated by white lines on the time axis. As for the reference information of each of the users A to E, one or more reference information whose input start time is within a predetermined period and whose position indicated by the position information is within a predetermined range in the presentation image is classified into the same group. .

  Specifically, for example, n (n is a natural number) seconds is set in advance as the predetermined period. As the predetermined range, for example, a range of m pixels (m is a natural number) centering on a position where additional information is input is set. In this case, each of the five reference information respectively corresponding to the users A to E has an input start time within n seconds from the input start time included in the reference information, and the additional information included in the reference information Is classified into the same group as the reference information. As described above, the predetermined period is relatively determined based on the input start time of the reference information to be processed. The predetermined range is relatively determined based on the position of the additional information included in the reference information to be processed. In addition, it may be set in advance as a condition that satisfies at least a part of different additional information within a predetermined range.

  11 and 12, it is assumed that the position of the additional information of user A and the position of the additional information of user B are substantially equal, and the additional information of user B is within a predetermined range from the additional information of user A. . Further, the difference between the input start time included in the reference information of the user A and the input start time included in the reference information of the user B is within 10 seconds, and is within a predetermined period. In this case, referring to FIG. 19, the reference information of user A and the reference information of user B are classified into the first group. In addition, the group period α of the first group is determined from the input start time 13: 00: 0 included in the reference information of the user A to the input end time 13:00:09 included in the reference information of the user B.

  Referring to FIG. 13, the additional information of the reference information of user C is greatly separated from the additional information of the reference information of users A, B, D, and E. The additional information of users A, B, D, and E is It is assumed that it is not within the predetermined range from the additional information of user A. In this case, referring to FIG. 19, only the reference information of user C is classified into the second group. Further, the group period β of the second group is determined from the input start time 13:00:08 included in the reference information of the user C to the input end time 13:00:28 included in the reference information of the user C.

  14 and 15, it is assumed that the position of the additional information of user D and the position of the additional information of user E are substantially equal, and the additional information of user E is within a predetermined range from the additional information of user D. . Further, the difference between the input start time included in the reference information of the user D and the input start time included in the reference information of the user E is within 10 seconds, and is within a predetermined period. In this case, referring to FIG. 19, CPU 201 classifies the reference information of user D and the reference information of user E into the third group. Also, the group period γ of the third group is determined from the input start time 13:00:25 included in the reference information of the user E to the input end time 13:00:38 included in the reference information of the user D.

  FIG. 20 is a schematic diagram illustrating an example of grouping of reference information based on a presentation manuscript. FIG. 20 illustrates an example in which the reference information of each of the users F to H illustrated in FIGS. 16 to 18 is grouped. Referring to FIG. 20, the occupied areas on the image corresponding to the first to third groups in which the reference information of users A to F is classified by the grouping of FIG. 19 are indicated by two-dot chain lines. The occupied areas on the images corresponding to the first to third groups are areas indicating that they are within a predetermined range from the position of one or more additional information included in each group. In this case, each of the plurality of reference information of the users F to H based on the presentation manuscript is classified into one of the first to third groups.

  In this example, a part of the drawing image p6 corresponding to the user F is included in the occupied area on the image corresponding to the first group. Thereby, the reference information of the user F is classified into the first group.

  In this example, the drawing image p8 corresponding to the user H includes a lead line that is a condition image. In this case, the position information corresponding to the drawn image p8 is set in the vicinity of the character string “sound reproduction” in the presentation image area. Therefore, a part of the drawing image p8 corresponding to the user H is included in the occupied area on the image corresponding to the third group. Thereby, the reference information of the user H is classified into the third group.

  By the way, in this example, the drawing image p7 corresponding to the user G is not included in the occupied area on the image corresponding to any of the first to third groups. In this case, the reference information of the user G is classified into the first group that is closest to the drawing image p7 among the first to third groups.

  Referring to FIG. 10, when user A operates list button 223, a group list in which group identification information of each of the first to third groups is arranged is displayed on display unit 205. 21 and 22 are diagrams illustrating an example of the group list. Referring to FIGS. 21 and 22, a group list display area 241 is provided on the side of the presentation image display area 231. In addition, a character size button 291 and a return button 292 are displayed above the group list display area 241.

  A slider 249 is provided in the group list display area 241. The slider 249 expands an area where the group list can be displayed in the group list display area 241. In FIG. 21, the group list of the first group is mainly displayed when the slider 249 is positioned above the screen. On the other hand, in FIG. 22, the group list of the third group is mainly displayed when the slider 249 is positioned at the bottom of the screen.

  In the group list display area 241, group identification information 242 for identifying each of the first, second, and third groups is displayed. In each group identification information 242, reference images 243 of reference information belonging to the group corresponding to the group identification information 242 are arranged. Each reference image 243 includes user identification information included in the reference information, additional information, and an image of an area specified by position information included in the reference information in the announcement part specified by the presentation image related information. Including. Thereby, the user A can identify one or more groups by the additional information. The group identification information 242 may include a group period of one or more groups. In this case, the user A can identify one or more groups by the group period.

  Further, in the group identification information 242, a character string indicating whether the reference information represented by the reference image 243 corresponds to the presentation data or the presentation manuscript is displayed in the vicinity of each reference image 243. Therefore, the user A can easily identify the type of each reference information.

  User A can select one of a plurality of group identification information 242 displayed in the group list display area 241 by operating the operation unit 206. FIG. 23 is a diagram illustrating a display example when one group identification information 242 is selected from the group list. When the user A selects one group identification information 242, an audio text display area 251 and a reproduction operation display area 261 are provided below the presentation image display area 231. At this time, the group period corresponding to the selected group identification information 242 is determined as the reproduction period.

  Referring to FIG. 23, in the group list display area 241, the portion other than the arranged reference image 243 in the group identification information 242 selected by the user A is the other group identification information 242 that is not selected. Displayed in a different manner. In FIG. 23, a portion excluding the arranged reference images 243 in the group identification information 242 of the first group is indicated by hatching. Thereby, the user A can easily identify the group identification information 242 selected by the user A.

  In the presentation image display area 231, a presentation image 401 corresponding to the group period of the selected first group is displayed. On the presentation image 401, a trajectory image 402 indicating the trajectory of the mouse pointer during the group period of the selected first group is displayed. On the presentation image 401, the drawn images p1, p2, and p6 are displayed in a translucent state as one or more additional information included in one or more reference information belonging to the selected first group.

  At this time, the size of the presentation images p1 and p2, which are additional information corresponding to the presentation data, on the presentation image is uniquely determined based on the size of the additional information input on the presentation portion image. On the other hand, the size of the drawn image p6, which is additional information corresponding to the presentation manuscript, is determined based on the enlargement / reduction ratio calculated by the additional information extraction unit 122 for the additional information. Accordingly, the drawing image p6 of the user F is displayed on the presentation image so that the presentation document of the user F is reproduced on the presentation image display area 231.

  Incidentally, the drawing image p7 of the user G belonging to the first group is located in the blank area. In the example of FIG. 23, only the display image area is displayed in the presentation image display area 231. Therefore, in this example, instead of the drawing image p7, a sticky note-like index p11 indicating that at least a part of the additional information (drawing image p7) exists in the blank area is in the vicinity of the additional information and within the display image region. Is displayed in a semi-transparent state.

  In the speech text display area 251, a character string of text data obtained by character recognition of speech during the group period of the selected first group is displayed. Furthermore, the playback operation display area 261 includes operation buttons for instructing start of playback of conference information, stop of playback, start of reverse playback, and stop of reverse playback, and a time slider indicating the playback time of conference information. Is displayed.

  In this state, the user A operates an operation button displayed in the reproduction operation display area 261, for example, displays a moving image showing the movement of the mouse pointer during the group period of the selected first group in the presentation image display area 231. Can be superimposed on the presentation image 401 displayed on the screen. Also, the audio during the group period of the selected first group can be reproduced from the audio output unit 208.

  In this example, the speech text display area 251 displays a character string of text data obtained by character recognition of speech during the group period of the selected first group, but the present invention is not limited to this. In the speech text display area 251, a character string of text data obtained by character recognition of speech during a period including the group period of the first group and including periods before and after the group period may be displayed. In this case, the character string of the text data during the group period of the first group is highlighted in a display mode different from the character string of the text data during the period before and after the group period. Thereby, the user A can easily identify the voice of the presenter during the group period of the first group.

  The user A further operates the operation unit 206 to select the user A reference image 243, the user B reference image 243, the user F reference image 243, the user A reference image 243 arranged in the group identification information 242 of the first group. By selecting one of the E reference images 243, one of a plurality of reference information belonging to the first group can be selected. Alternatively, the user A further operates the operation unit 206 to select any one of the drawn images p1, p2, p6 and the index p11 displayed on the presentation image 401, thereby selecting the selected image or index. The reference information corresponding to can be selected. At this time, the input period included in the selected reference information is determined as the reproduction period.

  FIG. 24 is a display example when the reference image 243 of the user F is selected. Referring to FIG. 24, when user A selects user F's reference image 243 or drawing image p6, the selected user F's reference image 243 is the unselected user's A, B, G reference images 243 and Are displayed in different ways. In FIG. 24, the reference image 243 of the user F is shown by hatching. Thereby, the user A can easily identify the reference image 243 selected by the user A.

  The reference information of the user F does not include an input period because it corresponds to the presentation manuscript. In this case, in the presentation image display area 231, the corresponding presentation image 401 is displayed during the group period of the selected first group. On the presentation image 401, a locus image 402 indicating the locus of the mouse pointer during the group period is displayed. On the presentation image 401, a drawing image p6 that is additional information included in the reference information of the selected user F is displayed in an emphasized state. At this time, similarly to the example of FIG. 23, the drawn images p1 and p2 of the users A and B that are not selected are displayed on the presentation image 401 in a translucent state. In addition, the index p11 corresponding to the reference information of the user G that has not been selected is displayed in a translucent state. In the speech text display area 251, a character string of text data obtained by character recognition of speech during the group period of the first group selected previously is displayed.

  FIG. 25 is a display example when the reference image 243 of the user G is selected. Referring to FIG. 25, when user A selects reference image 243 or index p11 of user G, the selected reference image 243 of user G is the reference image 243 of unselected users A, B, and F. Displayed in a different manner. In FIG. 25, the reference image 243 of the user G is indicated by hatching. Thereby, the user A can easily identify the reference image 243 selected by the user A.

  The reference information of the user G, like the reference information of the user F, does not include an input period because it corresponds to the presentation manuscript. In this case, in the presentation image display area 231, the corresponding presentation image 401 is displayed during the group period of the selected first group.

  Here, the additional information included in the reference information of the user G is located on the margin area of the presentation manuscript. As described above, when the additional information of the selected reference information is located on the blank area, the blank area image 403 surrounding the presentation image 401 is displayed in the presentation image display area 231 in addition to the presentation image 401. In addition, on the presentation image display area 231, the drawing image p7 of the user G is displayed in an emphasized state on the blank area image 403 so that the presentation manuscript of the user G is reproduced. At this time, similarly to the example of FIG. 23, the drawn images p1, p2, and p6 of the users A, B, and F are displayed in a semi-transparent state and the mouse pointer locus image 402 is displayed on the presentation image 401. The When the blank area image 403 is displayed, the index p11 shown in FIGS. 23 and 24 displayed on the presentation image 401 is deleted. In the speech text display area 251, a character string of text data obtained by character recognition of speech during the group period of the first group selected previously is displayed.

  When the user A re-selects the reference image 243 of the user F in the display state of FIG. 25, the display unit 205 transitions to the display state of FIG. Thereby, in the presentation image display area 231, the blank area image 403 is not displayed, and the index p11 is displayed instead of the drawing image p6 of the user G.

  The user A operates the character size button 291 after selecting the group identification information 242, thereby adjusting the size of the additional information of the reference information belonging to the selected group and corresponding to the presentation manuscript to the appropriate character size of the additional information. can do.

  FIG. 26 shows a display example when the character size button 291 is operated in the display state of FIG. In the example of FIG. 26, when the character size button 291 is operated, the size of the character string included in the drawing image p6 that is the additional information of the user F is changed to the character string of the presentation data included in the presentation image 401 and the presentation data. Substantially matches the size of the drawn images p1 and p2 of the reference information based on. Thereby, the visibility of the drawn image p6 is improved.

  FIG. 27 is a diagram showing another display example when one group identification information 242 is selected from the group list. The difference between the display example of FIG. 27 and the display example of FIG. 23 will be described.

  In the display example of FIG. 27, when the user A selects the third group, the part other than the arranged reference image 243 in the group identification information 242 of the third group is not selected as other group identification information 242. Is displayed in a different manner. In FIG. 27, a portion excluding the arranged reference images 243 in the group identification information 242 of the third group is indicated by hatching.

  In the presentation image display area 231, a corresponding presentation image 401 is displayed during the group period of the selected third group. On the presentation image 401, a locus image 402 indicating the locus of the mouse pointer during the group period of the selected third group is displayed. On the presentation image 401, the drawn images p4 and p5 are displayed in a translucent state as one or more additional information included in one or more reference information belonging to the selected first group.

  By the way, the drawing image p8 of the user H belonging to the third group is located so as to straddle the presentation image area and the blank area, including the leader line as the condition image. In the example of FIG. 27, only the display image area is displayed in the presentation image display area 231. Therefore, in this example, instead of the drawing image p8, a position where a sticky note-like index p12 indicating that at least part of the additional information (drawing image p8) exists in the blank area is represented by the position information of the additional information. It is displayed in a semi-transparent state at the end position of the leader line. In the speech text display area 251, a character string of text data obtained by character recognition of speech during the group period of the third group is displayed.

  Also in the example of FIG. 27, the user A has the reference image 243 of the user D, the reference image 243 of the user E, and the user H arranged in the group identification information 242 of the third group selected by operating the operation unit 206. By selecting one of the reference images 243, one of a plurality of reference information belonging to the third group can be selected. Alternatively, the user A further corresponds to the selected image or index by selecting any one of the drawn images p4 and p5 and the index p12 displayed on the presentation image 401 by operating the operation unit 206. Reference information to be selected can be selected.

  FIG. 28 is a display example when the reference image 243 of the user H is selected. Referring to FIG. 28, when user A selects reference image 243 or index p12 of user H, the selected reference image 243 of user H is different from the reference images 243 of unselected users D and E. Is displayed. In FIG. 28, the reference image 243 of the user H is indicated by hatching.

  Since the reference information of the user H corresponds to the presentation manuscript, the input period is not included. In this case, in the presentation image display area 231, the corresponding presentation image 401 is displayed during the group period of the selected third group.

  As described above, the additional information of the user H includes the condition image, and a part thereof is located on the blank area of the presentation manuscript. In this case, a blank area image 403 surrounding the presentation image 401 is displayed in the presentation image display area 231 in addition to the presentation image 401. Further, the presentation image p8 of the user H is displayed in an emphasized state on the presentation image 401 and the blank area image 403 so that the presentation document of the user H is reproduced on the presentation image display area 231. At this time, similarly to the example of FIG. 27, the drawn images p4 and p5 of the users D and E are displayed in a semi-transparent state and the locus image 402 of the mouse pointer is displayed on the presentation image 401. When the blank area image 403 is displayed, the index p12 shown in FIG. 27 displayed on the presentation image 401 is deleted. In the speech text display area 251, a character string of text data obtained by character recognition of speech during the group period of the third group selected previously is displayed.

  In the above example, when the reference information including the additional information is selected in order to present the user A with additional information at least part of which is present in the margin area, the presentation image 401 and the margin are displayed in the presentation image display area 231. The area image 403 is displayed, and additional information is superimposed on these images, but the present invention is not limited to this.

  In the state where only the presentation image 401 is displayed in the presentation image display area 231, when reference information including additional information at least part of which is present in the blank area is selected, only the additional information is displayed on the screen of the display unit 205. May be displayed separately from the display of the presentation image display area 231.

  Alternatively, when at least one additional information among the plurality of reference information displayed in the group list exists in the blank area, the presentation image 401 and the blank area image 403 are displayed in the presentation image display area 231 and these images are displayed. All additional information belonging to the group selected above may be displayed in a superimposed manner.

  FIG. 29 is a flowchart illustrating an example of the flow of conference information acquisition processing. The conference information acquisition process is a process executed by the CPU 201 when the CPU 201 included in each of the portable information devices 200A to 200E executes a conference information acquisition program stored in the ROM 202, RAM 203, flash memory 204, or CD-ROM 210A. is there. The conference information acquisition program is a part of the conference related information reproduction program. In an initial state of various processes described below, it is assumed that presentation data is downloaded from the MFP 400 in advance to the portable information device 200A. Further, it is assumed that user A's user identification information is stored in advance in the flash memory 204 of the portable information device 200A.

  Referring to FIG. 29, in step S01, the apparatus mode is set to the participation mode, and the process proceeds to step S02. In step S02, it is determined whether or not the presentation mode has been selected. If the announcement mode is selected, the process proceeds to step S03. If not, the process proceeds to step S11.

  In step S03, a presenter request signal for requesting that the user should become a presenter is transmitted to MFP 400, and the process proceeds to step S04. In step S04, it is determined whether a permission signal for the request to become a presenter is received from MFP 400 or not. If the permission signal is received from MFP 400, the process proceeds to step S05. If not, the process returns to step S02.

  In step S05, the apparatus mode is set to the presentation mode. In step S06, it is determined whether conference information has been generated. Specifically, it is determined whether or not any one of the presentation image related information, the pointer related information, and the voice related information is generated by at least one of a presentation image related process, a pointer related process, and a voice related process described later. . If conference information is generated, the process proceeds to step S07; otherwise, the process proceeds to step S08.

  In step S07, the generated conference information is transmitted to portable information devices 200B to 200E of all users B to E participating in the conference, and the process proceeds to step S08. In step S08, it is determined whether or not a participation mode has been selected. If the participation mode is selected, the process proceeds to step S09; otherwise, the process proceeds to step S06. In step S09, the apparatus mode is set to the participation mode, and the process proceeds to step S02.

  In step S11, it is determined whether meeting information has been received from any of other portable information devices 200B to 200E. If conference information is received, the process proceeds to step S12. If not, the process returns to step S02. In step S12, the received conference information is stored in RAM 203 or flash memory 204, and the process returns to step S02.

  FIG. 30 is a flowchart illustrating an exemplary flow of a presentation image related process. The presentation image related process is a process executed by the CPU 201 when the CPU 201 included in each of the portable information devices 200A to 200E executes a presentation image related program stored in the ROM 202, RAM 203, flash memory 204, or CD-ROM 210A. is there. The presentation image related program is a part of the conference related information reproduction program. Here, a case where the CPU 201 included in the portable information device 200A executes the presentation image related program will be described as an example.

  Referring to FIG. 30, in step S21, it is determined whether or not the device mode is set to the presentation mode. The process waits until it is determined that the device mode is set to the presentation mode (NO in step S21). If it is determined that the device mode is set to the presentation mode (YES in step S21), the process is performed in step S22. Proceed to In step S22, a presentation part is set in the presentation data, an image of the set presentation part is displayed on the display unit 205 as a presentation image, and the process proceeds to step S23. Specifically, the region specified by the presentation page and region designated by the user A who is the presenter operating the operation unit 206 is set as the presentation part. Furthermore, the image of the presentation part set in the presentation data is displayed on the display unit 205 as a presentation image.

  In step S23, presentation image related information is generated, the generated presentation image related information is stored in RAM 203 or flash memory 204, and the process proceeds to step S24. The page identification information of the announcement page specified in the announcement part set in step S22 is set in the page data item, and the area identification information indicating the position in the announcement page of the area specified in the announcement part is the item of partial data. And present image related information is generated by setting the current time in the item of time information. In step S24, the announcement part set in step S22 is transmitted to portable information devices 200B to 200E of all users B to E, and the process proceeds to step S25.

  In step S25, it is determined whether or not the announcement part has been changed. Specifically, it is determined whether or not the user A has newly designated a region different from the presentation portion set in step S22. If it is determined that the announcement part has been changed, the process returns to step S22; otherwise, the process returns to step S21.

  FIG. 31 is a flowchart illustrating an example of a flow of reference information acquisition processing corresponding to presentation data. In this reference information acquisition process, the CPU 201 included in each of the portable information devices 200A to 200E executes a reference information acquisition program corresponding to the presentation data stored in the ROM 202, RAM 203, flash memory 204, or CD-ROM 210A. The process executed by The reference information acquisition program corresponding to the presentation data is a part of the conference related information reproduction program. Here, a case where the CPU 201 included in the portable information device 200A executes a reference information acquisition program corresponding to presentation data will be described as an example.

  Referring to FIG. 31, in step S31, user identification information for identifying a user who operates portable information device 200 is acquired, and the process proceeds to step S32. Here, since the user identification information of the user who operates portable information device 200A is stored in flash memory 204, the user identification information stored in flash memory 204 is read out. Note that user identification information input to the operation unit 206 by a user operating the portable information device 200A may be acquired. In the next step S32, it is determined whether or not the input of additional information on the presentation image is started by the user A operating the operation unit 206. If input of additional information is started, the process proceeds to step S33; otherwise, the process proceeds to step S39.

  In step S33, the current time is set as the input start time, and the process proceeds to step S34. In step S34, it is determined whether or not the input of additional information on the presentation image by user A has been completed. It is in a standby state until it is determined that the input of additional information on the presentation image by user A has been completed (NO in step S34), and when it is determined that the input of additional information on the presentation image by user A has been completed (step S34). The process proceeds to step S35.

  In step S35, the current time is set as the input end time, and the process proceeds to step S36. In step S36, the position of the region where the additional information is input in the presentation image is determined as position information, and the process proceeds to step S37. In step S37, reference information corresponding to the presentation data of user A is generated, and the generated reference information is stored in RAM 203 or flash memory 204, and the process proceeds to step S38. Specifically, the user identification information acquired in step S31 is set in the user ID item, the time set in the input start time in step S33 is set in the start time item in the input period item, and step S35. The time set as the input end time is set as the end time item of the input period item, and the additional information whose input is started in step S32 and input is ended in step S34 is set as the additional information item. Then, the reference information in which the position information determined in step S36 is set in the item of position information is generated. In step S38, the reference information corresponding to the generated presentation data is transmitted to the portable information devices 200B to 200E of all users B to E participating in the conference, and the process proceeds to step S39.

  In step S39, it is determined whether or not reference information corresponding to the presentation data is received from any of the other portable information devices 200B to 200E. If wireless LAN I / F 209 receives reference information from any of other portable information devices 200B to 200E, the process proceeds to step S40. If not, the process returns to step S32. In step S40, the received reference information is stored in RAM 203 or flash memory 204, and the process returns to step S32.

  FIG. 32 is a flowchart illustrating an example of a flow of reference information acquisition processing corresponding to a presentation manuscript. In this reference information acquisition process, the CPU 201 included in each of the portable information devices 200A to 200E executes the reference information acquisition program corresponding to the presentation manuscript stored in the ROM 202, the RAM 203, the flash memory 204, or the CD-ROM 210A. The process executed by The reference information acquisition program corresponding to the presentation manuscript is a part of the conference related information reproduction program. Here, a case where the CPU 201 included in the portable information device 200A executes a reference information acquisition program corresponding to presentation data will be described as an example.

  Referring to FIG. 32, in step S41, it is determined whether or not the read data of the presentation manuscript and the corresponding user identification information are received from MFP 400. The wireless LAN I / F 209 is in a standby state until it determines that it has received read data and user identification information from the MFP 400 (NO in step S41), and determines that the wireless LAN I / F 209 has received read data and user identification information from the MFP 400 ( In step S41, YES), the process proceeds to step S42.

  The presentation data according to the present embodiment includes a predetermined reference image (for example, a frame or the like) for defining the reference size of the page for each page. Therefore, a reference image is formed for each page of the presentation data even in the presentation manuscript.

  In step S42, a page is extracted by detecting a reference image formed for each page of the presentation data from the received read data using pattern matching or the like, and the process proceeds to step S43. As a result, the image size for one page of the presentation data formed on one sheet of the presentation manuscript can be defined as the reading reference size.

  In step S43, the ratio of the size of the additional information to the reading reference size is calculated as the enlargement / reduction ratio, and the process proceeds to step S44. In step S44, each of the one or more images that do not exist in the presentation data is extracted as additional information from the read data by comparing the extracted read data for each page with the presentation data for each page, and the process is performed. Proceed to S45. When the process proceeds to step S45, one or more additional information included in the read data is extracted.

  In step S45, one additional information to be processed is selected from the extracted one or more additional information, and the process proceeds to step S46. In step S46, it is determined whether or not a predetermined condition image exists in the additional information to be processed. As described above, in the present embodiment, the condition image is set as a linear image extending from the presentation image area to the blank area in the presentation document image. If the condition image does not exist, the process proceeds to step S47. If the condition image exists, the process proceeds to step S48.

  In step S47, the position of the region where the additional information is input in the presentation document image is determined as position information, and the process proceeds to step S49. In step S48, the position of the area where the end of the condition image is input in the presentation document image is determined as the position information of the additional information, and the process proceeds to step S49. In step S49, an appropriate character size of the additional information is determined based on the read data and the presentation data, and the process proceeds to step S50.

  In step S50, reference information corresponding to the presentation manuscript is generated, and the generated reference information is stored in the RAM 203 or the flash memory 204. Specifically, the user identification information received in step S41 is set in the item of user ID, the additional information extracted in step S43 and selected in step S45 is set in the item of additional information, and steps S47 and S48. The position information determined in any of the above is set in the position information item, the enlargement / reduction ratio calculated in step S43 is set in the enlargement / reduction ratio item, and the determination result in step S46 is set as the condition image determination result item. The reference information is generated by setting the appropriate character size determined in step S49 in the item of the appropriate character size. Thereafter, the process proceeds to step S51.

  In step S51, for one or more additional information extracted in step S44, it is determined whether there is additional information not yet selected in step S45. If unselected additional information exists, the process returns to step S45; otherwise, the process returns to step S41.

  In addition, when additional information is not extracted in said step S44, you may return to the process of step S41 noting that additional information does not exist.

  In the present embodiment, when at least a part of the additional information is in the blank area, the position of the additional information may be represented only by the page number. For example, in step S46 described above, instead of determining whether or not a condition image exists in the additional information, it is determined whether or not at least a part of the additional information exists in the blank area. Accordingly, when at least a part of the additional information does not exist in the blank area, the process proceeds to step S47. On the other hand, when at least a part of the additional information exists in the blank area, the page including the additional information is determined as the position information of the reference information, and the process proceeds to step S47.

  Further, in the present embodiment, when there is no condition image in the additional information and all of the additional information is in the blank area, the position closest to the additional information in the presentation image area adjacent to the blank area is You may determine as a position of additional information.

  FIG. 33 is a flowchart illustrating an example of the flow of grouping processing. The grouping process is a process executed by the CPU 201 when the CPU 201 included in each portable information device 200A to 200E executes a grouping program stored in the ROM 202, the RAM 203, the flash memory 204, or the CD-ROM 210A. The grouping program is part of the conference related information reproduction program. Here, a case where CPU 201 provided in portable information device 200A executes a grouping program will be described as an example.

  Referring to FIG. 33, in step S61, reference information corresponding to the presentation data stored in RAM 203 or flash memory 204 is read out by the reference information acquisition process shown in FIG. 31, and the process proceeds to step S62. In step S62, one reference information to be processed is selected from the read one or more reference information, and the process proceeds to step S63.

  In step S63, reference information whose input start time is within a predetermined period from the input start time of the reference information to be processed is extracted from the one or more reference information read in step S61, and the process is performed in step S64. Proceed to In step S64, the reference information in which the position indicated by the position information from the reference information extracted in step S63 is within a predetermined range from the position indicated by the position information of the reference information to be processed is extracted and processed. Proceed to step S65.

  In step S65, the reference information to be processed and the one or more reference information extracted in step S64 are classified into the same group and stored in the RAM 203 or the flash memory 204. In the next step S66, the group period is determined, the determined group period is stored in the RAM 203 or the flash memory 204, and the process proceeds to step S67. Specifically, one or more reference information belonging to the same group stored in step S65 is a processing target, and one or more from the earliest time among one or more input start times respectively included in the one or more reference information. The period from the one or more input end times included in the reference information to the latest time is determined as the group period.

  In step S67, for one or more reference information read in step S61, it is determined whether there is reference information that has not yet been selected in step S62. If unselected reference information exists, the process returns to step S62; otherwise, the process proceeds to step S68.

  In step S68, the reference information corresponding to the presentation manuscript stored in the RAM 203 or the flash memory 204 is read out by the reference information acquisition process shown in FIG. 32, and the process proceeds to step S69. In step S69, one reference information to be processed is selected from the one or more read reference information, and the process proceeds to step S70.

  In step S70, based on the position information of the selected reference information, it is determined whether any group classified in the processes of steps S61 to S67 is within a predetermined range from the position of the additional information of the reference information. to decide. If any group is within the predetermined range, the process proceeds to step S71; otherwise, the process proceeds to step S72.

  In step S71, the reference information selected in step S59 is classified into a group within a predetermined range from the position of the additional information of the reference information, stored in the RAM 203 or the flash memory 204, and the process proceeds to step S63. On the other hand, in step S72, the reference information selected in step S69 is classified into a group closest to the position of the additional information of the reference information, stored in the RAM 203 or the flash memory 204, and the process proceeds to step S73. .

  In step S73, for one or more reference information read in step S68, it is determined whether there is reference information not yet selected in step S69. If unselected reference information exists, the process returns to step S69. If not, the process ends.

  When at least part of the additional information is in the blank area, the position of the additional information is specified only by the page number (when the position information is expressed only by the page number), the selected reference information This page may contain multiple groups. In this case, the selected reference information may be classified into all of a plurality of groups set on the page indicated by the position information.

  Note that there is a possibility that only the reference information corresponding to the presentation data is stored in the RAM 203 or the flash memory 204 at the start of the grouping process, and the reference information corresponding to the presentation manuscript is not stored. In this case, steps S68 to S73 in the grouping process of FIG. 33 are omitted. Further, there is a possibility that only the reference information corresponding to the presentation manuscript is stored in the RAM 203 or the flash memory 204 at the start of the grouping process, and the reference information corresponding to the presentation data is not stored. In this case, the processing of steps S61 to S67 is omitted from the grouping processing of FIG. 33, and instead of the processing of steps S70 to S72, reference information within a predetermined range is extracted from the selected reference information and extracted. The obtained reference information is classified into the same group and stored in the RAM 203 or the flash memory 204. Further, the group period of each group is determined and stored as a period from the presentation start time to the presentation end time.

  FIG. 34 is a flowchart illustrating an example of the flow of reproduction processing. The reproduction process is a process executed by the CPU 201 when the CPU 201 included in each portable information device 200A to 200E executes a reproduction program stored in the ROM 202, the RAM 203, the flash memory 204, or the CD-ROM 210A. The reproduction program is a part of the conference related information reproduction program. Here, a case where the CPU 201 included in the portable information device 200A executes the reproduction program will be described as an example.

  Referring to FIG. 34, in step S81, the group list is displayed on display unit 205, and the process proceeds to step S82. Specifically, one or more reference information classified into the group is stored in the RAM 203 or the flash memory 204 for each group of one or more by the grouping process shown in FIG. The CPU 201 displays on the display unit 205 a group list in which one or more group identification information for identifying one or more groups stored in the RAM 203 or the flash memory 204 is arranged. The group identification information is an image in which one or more reference images respectively corresponding to one or more reference information belonging to the group are arranged. The reference image includes user identification information included in the reference information, additional information, and an image of an area specified by the position information included in the reference information in the announcement portion specified by the presentation image related information. Specifically, the CPU 201 combines the image of the additional information included in the reference information with the image of the portion including the area specified by the position information set in the position information item of the reference information in the presentation data. In addition, a reference image to which an image of user identification information included in the reference information is added is generated. The CPU 201 generates, as group identification information, an image in which one or more reference images respectively corresponding to one or more reference information belonging to a group are arranged.

  In step S82, it is determined whether any one of one or more groups has been selected. Specifically, when any one of one or more group identification information indicated in the group list is selected by the user A, it is determined that a group corresponding to the selected group identification information has been selected. If none of the one or more pieces of group identification information is selected, it is determined that no group is selected. If a group is selected, the process proceeds to step S83; otherwise, the process ends.

  In step S83, the group period determined by the grouping process shown in FIG. 33 corresponding to the selected group is determined as the reproduction period, and the process proceeds to step S84. In step S84, a first reference information display process is performed, and the process proceeds to step S85. Through the first reference information display process, the presentation image during the reproduction period is displayed on the display unit 205. Details of the first reference information display process will be described later.

  In step S85, a static output control process is performed, and the process proceeds to step S86. In this static output control process, a trajectory image indicating the trajectory of the mouse pointer during the reproduction period is displayed on the presentation image displayed during the reproduction period displayed on the display unit 205. In addition, a character string representing the sound during the reproduction period is displayed on the display unit 205.

  In step S86, it is determined whether an instruction for a reproduction operation has been accepted. Specifically, when touch panel 206B detects that user A has instructed an operation button displayed in reproduction operation display area 261, an instruction for a reproduction operation is accepted. If a reproduction operation instruction has been accepted, the process proceeds to step S87; otherwise, the process proceeds to step S88. In step S87, a dynamic output control process is performed, and the process proceeds to step S88. In this dynamic output control process, a moving image of the mouse pointer is displayed so that the movement of the mouse pointer during the reproduction period is reproduced on the presentation image during the reproduction period. Also, the sound during the playback period is played back.

  In step S88, it is determined whether any one of the one or more reference information belonging to the group selected in step S82 has been selected. Specifically, the reference information corresponding to the selected reference image when any one of the one or more reference images arranged in the group identification information of the group selected in step S82 is selected by the user A. Is determined to have been selected. In the first reference information display process of step S84, the presentation image corresponding to the display unit 205 during the reproduction period determined in step S83 is displayed and one or more additional information or index corresponding to the reproduction period is displayed. Is displayed on the presentation image. In step S88, when the additional information or index displayed on the presentation image is selected, it is determined that the reference information including the additional information or the reference information including the additional information corresponding to the index is selected. Further, when none of the one or more reference images is selected, it is determined that the reference information is not selected. If reference information is selected, the process proceeds to step S89; otherwise, the process returns to step S82. Since the reference information includes additional information, the selection of reference information is the same as the selection of additional information.

  In step S89, a second reference information display process is performed, and the process proceeds to step S90. By the second reference information display processing, the presentation image during the reproduction period is displayed on the display unit 205, and the selected additional information is emphasized and displayed in the presentation image more than the additional information of the other reference information. . Alternatively, the presentation image and the blank area image during the reproduction period are displayed on the display unit 205, and the selected additional information is emphasized and displayed in the presentation image and the blank area image more than the additional information of the other reference information. To do. Details of the second reference information display process will be described later.

  In step S90, it is determined whether or not the reference information selected in step S88 or step S99 described later corresponds to the presentation data. If the reference information corresponds to the presentation data, the process proceeds to step S91. If not (the reference information corresponds to the presentation manuscript), the process proceeds to step S92.

  In step S91, the input period of the reference information selected in step S88 or step S99 described later is determined as the reproduction period, and the process proceeds to step S93. On the other hand, in step S92, the group period of the group selected in the immediately preceding step in step S82 and step S100 described later is determined as the reproduction period, and the process proceeds to step S93. In step S93, the reproduction period determined in either step S91 or step S92 is stored in RAM 203 or flash memory 204, and the process proceeds to step S94.

  In the static output control process in step S85, the character string of the corresponding voice during the playback period determined in step S83 is displayed on the display unit 205, in other words, the voice of the group in the group selected in step S82. A character string is displayed. In this case, the voice character string displayed on the display unit 205 includes the voice character string during the input period of the reference information selected in step S88.

  In step S94, it is determined whether or not an instruction to appropriately adjust the character size is received for the additional information corresponding to the presentation manuscript among the additional information displayed on the display unit 205 in the processes of steps S84 and S89. This instruction is generated, for example, when the user A operates the character size button 291 in FIG. If an instruction to appropriately adjust the character size is accepted, the process proceeds to step S95; otherwise, the process proceeds to step S96. In step S95, the character size of the additional information corresponding to the presentation manuscript among the additional information displayed on the display unit 205 is adjusted to an appropriate character size.

  In step S96, among the voice character strings displayed on the display unit 205, the voice character string during the reference information input period selected in step S88 is selected during the period other than the selected reference information input period. The voice character string is emphasized and displayed, and the process proceeds to step S97.

  In step S97, as in step S86, it is determined whether or not an instruction for a reproduction operation has been accepted. If a reproduction operation instruction is accepted, the process proceeds to step S98; otherwise, the process proceeds to step S99. In step S98, a dynamic output control process is performed as in step S87, and the process proceeds to step S100.

  In step S99, it is determined whether any one of the one or more reference information belonging to the group selected in step S82 is newly selected. If reference information is selected, the process returns to step S89; otherwise, the process proceeds to step S100.

  In step S100, as in step S82, it is determined whether any one of one or more groups has been selected. If a group has been selected, the process returns to step S83; otherwise, the process returns to step S97.

  FIG. 35 is a flowchart showing an exemplary flow of a first reference information display process. The first reference information display process is a process executed in step S84 of the reproduction process shown in FIG. A group is selected by the user A at a stage before the first reference information display process is executed.

  Referring to FIG. 35, in step S101, corresponding presentation image related information is read out during the reproduction period (in this example, the group period of the group selected in step S82 in FIG. 34), and the process proceeds to step S102. . The presentation image related information whose time set in the item of time information is read from the one or more presentation image related information stored in the RAM 203 or the flash memory 204 is read. In step S102, the presentation image during the reproduction period is displayed on the display unit 205 based on the read presentation image related information, and the process proceeds to step S103. Specifically, according to the page identification information set in the page data item of the presentation image related information and the location information set in the partial data item, the presentation part in the presentation data is identified and the image of the presentation part is announced. The image is displayed on the display unit 205 as an image.

  In step S103, one or more pieces of reference information classified into the group selected by user A are read, and the process proceeds to step S104. All the reference information classified into the group selected by the user A is read out from one or more pieces of reference information stored in the RAM 203 or the flash memory 204.

  In step S104, one reference information to be processed is selected from the read one or more reference information, and the process proceeds to step S105. In step S105, it is determined whether a condition image exists in the additional information of the selected reference information. If the condition image does not exist, the process proceeds to step S107. If the condition image exists, the process proceeds to step S106. In step S106, an index indicating that the additional information is present in the blank area is superimposed on the position of the additional information (the position of the end of the condition image indicated by the position information) on the presentation image, and the process is performed. Proceed to S110.

  In step S107, it is determined whether or not at least part of the additional information of the selected reference information exists in the blank area. If the additional information does not exist in the blank area, the process proceeds to step S109. If the additional information exists in the blank area, the process proceeds to step S108. In step S108, an index indicating that additional information is present in the blank area is superimposed on a position near the additional information on the presentation image, and the process proceeds to step S110.

  In step S109, an image of additional information included in the read one or more pieces of reference information is superimposed on the presentation image, and the process proceeds to step S110. At this time, simultaneously with the display of the additional information, the user identification information corresponding to the additional information may be superimposed on the presentation image, or the user identification information corresponding to the additional information other than the presentation image of the display unit 205 may be displayed. You may display in a position. Thereby, not only the reference information corresponding to the presentation data but also the reference information corresponding to the presentation manuscript can be notified to the user who added the additional information.

  In step S110, for one or more reference information read in step S103, it is determined whether there is reference information not yet selected in step S104. If unselected reference information exists, the process returns to step S103; otherwise, the process proceeds to step S75 of the reproduction process shown in FIG.

  When at least a part of the additional information is in the blank area, the position of the additional information is specified only by the page number (when the position information is expressed only by the page number), in steps S106 and S108. The indicator may be displayed at a predetermined position on the presentation image.

  FIG. 36 is a flowchart illustrating an example of the flow of the second reference information display process. The second reference information display process is a process executed in step S79 of the reproduction process shown in FIG.

  Referring to FIG. 36, in step S111, the reference information selected in step S78 of FIG. 34 is read, and the process proceeds to step S112. In step S112, it is determined whether a condition image exists in the additional information of the selected reference information. If the condition image does not exist, the process proceeds to step S113. If the condition image exists, the process proceeds to step S114.

  In step S113, it is determined whether or not at least part of the additional information of the selected reference information exists in the blank area. If the additional information does not exist in the blank area, the process proceeds to step S113. If the condition image exists, the process proceeds to step S114.

  Here, in the first reference information display process, an index is displayed on the presentation image when the condition image exists in the additional information and when the additional information exists in the blank area. In step S114, the index displayed in steps S105 and S107 of FIG. 35 is deleted and the presentation image and the blank area image are displayed on the display unit 205, and the process proceeds to step S115.

  In step S115, the additional information of the selected reference information is highlighted at the position indicated by the position information, and the process proceeds to step S80 of the reproduction process shown in FIG. At this time, the user identification information corresponding to the additional information may be highlighted at the same time as the additional information is highlighted. As a result, the user can be notified of who added the additional information regarding the reference information corresponding to the selected reference information.

  As described above, portable information device 200A in the present embodiment functions as a conference-related information reproduction device, and is reference information generated by one of the own device and other portable information devices 200B to 200E. Reference information including additional information such as a memo added to the presentation data and position information indicating a position where the additional information is added in the presentation data is acquired. In addition, portable information device 200A acquires read data obtained by MFP 100 reading an announcement document on which an image based on the announcement data is formed, and extracts an image that does not exist in the announcement data as additional information from the acquired read data. When a user draws a memo by handwriting on a presentation manuscript on which an image based on the presentation data is formed, the image of the memo drawn by the user is extracted as additional information. The portable information device 200A determines the position in the presentation data corresponding to the position of the extracted additional information as the added position of the extracted additional information, and based on the extracted additional information and the determined position, Reference information corresponding to the recording medium is generated. 200 A of portable information devices classify | categorize the some reference information in which the position of additional information exists in the predetermined range in presentation data among the some reference information acquired as mentioned above in the same group, and 1 or more groups In response to the user selecting any one of these, at least one of a plurality of reference information belonging to the selected group is displayed.

  Therefore, portable information device 200A provides a memo that does not exist in the presentation data, extracted from the read data output by MFP 100 by reading the presentation manuscript, and is presented in either of its own device or other portable information devices 200B to 200E. It can be reproduced with notes added to. As a result, more information related to the conference can be easily reproduced.

  In the above embodiment, the CPU 201 of the portable information device 200A generates the conference information during the period from the presentation start time to the presentation end time while the user A is a presenter, but the present invention is not limited to this. Not. The CPU 201 may generate the conference information only when the user A is the presenter and the reference information is acquired from the portable information devices 200B to 200E of the other users B to E. Thereby, the data amount of the meeting information memorize | stored in each portable information device 200A-200E can be reduced.

  In the above-described embodiment, the portable information devices 200A to 200E have been described as examples of the conference related information reproduction device. However, the processing illustrated in FIGS. 29 to 36 is performed on each of the portable information devices 200A to 200E. Needless to say, the invention can be understood as a conference related information reproduction method to be executed and a conference related information reproduction program to be executed by the CPU 201 included in each of the portable information devices 200A to 200E.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

<Appendix>
(1) The conference related information reproduction device according to any one of claims 1 to 9, wherein the reference information acquisition unit vectorizes the additional information extracted by the additional information extraction unit. According to this aspect, the image quality when displaying the additional information is improved, and the amount of data representing the additional information is reduced.

  (2) The conference related information reproduction device according to claim 6, wherein the group list display unit displays, for each of the one or more groups, any additional information of one or more reference information belonging to the group. According to this aspect, the additional information of the reference information belonging to the group is displayed in the group list, so that the group to be selected can be easily identified.

  (3) The group list display means displays one or more reference information belonging to one or more groups in a selectable manner, and selects any one of the one or more reference information displayed by the group list display means. The reference information selecting means for further displaying, wherein the group list display means displays the reference information selected by the reference information selecting means in a display mode different from the display mode of other reference information. Conference related information reproduction device. According to this aspect, since the selected reference information is displayed in a display mode different from the display mode of other reference information, the selected reference information can be easily identified.

  (4) The additional information display means is selected from the one or more additional information displayed by the additional information display means when any one of the one or more reference information is selected by the reference information selection means. The conference related information reproduction device according to claim 6, wherein the additional information included in the reference information is displayed in a display mode different from other additional information. According to this aspect, since the additional information included in the selected reference information is displayed in a display mode different from the display mode of other additional information, the selected reference information can be easily identified.

  1 Conference system, 2 network, 2A, 2B network, 3A, 3B wireless station, 5 Internet, 100 MFP, 110 conference information acquisition unit, 111 presentation part setting unit, 120 reference information acquisition unit, 121 read data acquisition unit, 122 added Information extraction unit, 123 position determination unit, 124 recording medium correspondence generation unit, 125 condition image determination unit, 126 user identification information acquisition unit, 127 size determination unit, 130 grouping unit, 131 group list display unit, 133 group period determination unit , 134 selection unit, 135 group selection unit, 136 reference information selection unit, 137 index selection unit, 138 playback period determination unit, 150 reference information display unit, 151 additional information display unit, 152 index display unit, 153 outer additional information display unit , 160 Conference information output unit, 161 Announcement Data portion display unit, 170 playback operation instruction unit, 190 timing unit, 200A, 200B, 200C, 200D, 200E portable information device, 201 CPU, 202 ROM, 203 RAM, 204 flash memory, 205, 461 display unit, 206, 463 operation unit, 206A main key, 206B, 465 touch panel, 206C keyboard, 206D mouse, 207 audio input unit, 208 audio output unit, 209 wireless LAN I / F, 210 external storage device, 210A CD-ROM, 221 participation button, 222 Presentation button, 223 list button, 231 presentation image display area, 241 group list display area, 242 group identification information, 243 reference image, 249 slider, 251 voice text display area, 261 playback operation display area, 291 Character size button, 292 Back button, 400 MFP, 401 Presentation image, 402 Trajectory image, 403 Margin area image, 410 Main circuit, 411 CPU, 412 Communication I / F unit, 413 ROM, 414 RAM, 415 HDD, 416 Facsimile , 417 external storage device, 418 CD-ROM, 420 automatic document feeder, 430 document reading unit, 440 image forming unit, 450 paper feed unit, 460 operation panel, 461 display unit, 463 operation unit, 465 touch panel, 467 hardware Key part, p1, p2, p3, p4, p5, p6, p7, p8 Drawing image.

Claims (11)

Reference information acquisition means for acquiring reference information generated by either the own device or one or more other devices;
The reference information includes additional information added to presentation data accessible by each of the device and one or more other devices, and position information indicating a position where the additional information is added in the presentation data,
Among the plurality of reference information acquired by the reference information acquisition means, grouping means for classifying a plurality of reference information in which the position of additional information is within a predetermined range in the presentation data, into the same group;
Reference information display means for displaying at least one of a plurality of reference information belonging to the selected group in response to any one of the one or more groups being selected,
The reference information acquisition means includes
Read data acquisition means for acquiring read data obtained by reading a recording medium on which an image based on the announcement data is formed;
Additional information extracting means for extracting, as the additional information, an image that does not exist in the presentation data from the read data acquired by the read data acquiring means;
Position determining means for determining a position in the presentation data corresponding to a position of the extracted additional information in the read data as an added position of the extracted additional information;
And a recording medium correspondence generating unit that generates reference information corresponding to the recording medium based on the additional information extracted by the additional information extracting unit and the position determined by the position determining unit. .   The said reference information acquisition means is further provided with the size determination means which determines the size of the additional information extracted by the said additional information extraction means based on at least one of the said reading data and the said announcement data. Meeting related information reproduction device.   The reference information acquisition unit determines the size of the additional information extracted by the additional information extraction unit based on the size of the additional information added to the presentation data by either the own device or one or more other devices. The conference related information reproduction device according to claim 1, further comprising a size determination unit.   The position determining means extracts the extracted information from the plurality of pages included in the presentation data when the additional information extracted by the additional information extraction means does not exist in any of the images of the plurality of pages included in the presentation data. The page corresponding to the image having the closest additional information and the relative position of the extracted additional information with respect to the corresponding page are determined as the position of the additional information. The conference related information reproduction device described. The reference information acquisition means is a condition for determining whether or not there is a condition image extending from any of a plurality of images included in the presentation data in the additional information extracted by the additional information extraction means Further comprising image determination means,
The position determining means, when the condition image is present in the additional information, in the page of the image that overlaps the condition image among a plurality of pages included in the presentation data, at the end of the condition image of the page The conference related information reproduction device according to claim 1, wherein the position is determined as a position where the additional information is added. Meeting information acquisition means for acquiring information on the state of the meeting as meeting information;
Meeting information output means for outputting at least a part of the meeting information;
Group list display means for selectively displaying the one or more groups;
Group selection means for selecting any one of one or more groups displayed by the group list display means;
The conference information output means includes
A presentation data part display means for displaying a presentation part in the presentation data corresponding to the selected group in response to any one of the one or more groups being selected by the group selection means,
The reference information display means includes
Additional information display means for superimposing and displaying one or more additional information respectively included in one or more reference information belonging to the selected group on the presentation part displayed by the presentation data part display means;
When at least a part of the additional information extracted by the additional information extracting unit does not exist in any of the images of the plurality of pages included in the presentation data, an index indicating the presence of the additional information is used as the additional information. Instead, indicator display means for superimposing and displaying on the presentation part,
6. The conference related information reproduction device according to claim 1, further comprising an outside additional information display unit that receives an operation of the index by a user and displays additional information corresponding to the index when the index is operated. Each of the plurality of reference information corresponding to the announcement data including additional information added to the announcement data includes an input period that defines a start time and an end time of input of additional information,
The grouping means includes
A plurality of reference information corresponding to the presentation data acquired by the reference information acquisition means is a plurality of reference information whose additional information is within a predetermined range in the presentation data and whose start time is within a predetermined period. Classify them to belong to the same group,
Each of a plurality of pieces of reference information corresponding to the recording medium acquired by the reference information acquisition unit has a position of additional information of the reference information being one or a plurality of groups of a plurality of groups. The conference related information reproduction device according to claim 6, wherein the conference related information reproduction device classifies the group into a group included in a predetermined range from any position of the plurality of reference information included. The conference information changes over time,
A group period determining means for determining a group period for each of the one or more groups based on an input period of each of the one or more reference information belonging to the group;
Replay period determining means for determining, as a replay period, a group period determined for the selected group in response to selection of any one of the one or more groups,
The conference related information reproduction device according to claim 7, wherein the conference information output unit outputs a portion of the conference information corresponding to the determined reproduction period. The reference information further includes user identification information for identifying a user who has added additional information,
The reference information acquisition means further includes user identification information acquisition means for acquiring user identification information of a user who has added additional information included in the read data together with the read data acquired by the read data acquisition means,
The recording medium correspondence generating unit generates reference information corresponding to the recording medium based on the user identification information acquired by the user identification information acquiring unit,
The reference information display means simultaneously displays additional information included in each reference information and user identification information corresponding to the additional information when displaying at least one of the plurality of reference information. The conference related information reproduction device according to any one of? A reference information acquisition step of acquiring reference information generated by either the own device or one or more other devices;
The reference information includes additional information added to presentation data accessible by each of the device and one or more other devices, and position information indicating a position where the additional information is added in the presentation data,
A grouping step of classifying a plurality of reference information in which the position of additional information is within a predetermined range in the presentation data among the plurality of reference information acquired by the reference information acquisition unit into the same group;
A reference information display step of displaying at least one of a plurality of reference information belonging to the selected group in response to any one of the one or more groups being selected,
The reference information acquisition step includes
A read data acquisition step of acquiring read data obtained by reading a recording medium on which an image based on the announcement data is formed;
An additional information extraction step of extracting, as the additional information, an image that does not exist in the presentation data from the read data acquired by the read data acquisition step;
A position determining step for determining a position in the presentation data corresponding to a position of the extracted additional information in the read data as position information indicating a position where the extracted additional information is added;
And a recording medium correspondence generating step for generating reference information corresponding to the recording medium based on the additional information extracted by the additional information extracting step and the position determined by the position determining step. . A reference information acquisition step of acquiring reference information generated by either the own device or one or more other devices;
The reference information includes additional information added to presentation data accessible by each of the device and one or more other devices, and position information indicating a position where the additional information is added in the presentation data,
A grouping step of classifying a plurality of reference information in which the position of additional information is within a predetermined range in the presentation data among the plurality of reference information acquired by the reference information acquisition unit into the same group;
A reference information display step for displaying at least one of a plurality of reference information belonging to the selected group in response to selection of any one of the one or more groups; Let the computer that controls the other device execute it,
The reference information acquisition step includes
A read data acquisition step of acquiring read data obtained by reading a recording medium on which an image based on the announcement data is formed;
An additional information extraction step of extracting, as the additional information, an image that does not exist in the presentation data from the read data acquired by the read data acquisition step;
A position determining step for determining a position in the presentation data corresponding to a position of the extracted additional information in the read data as position information indicating a position where the extracted additional information is added;
And a recording medium correspondence generating step for generating reference information corresponding to the recording medium based on the additional information extracted by the additional information extracting step and the position determined by the position determining step. .

Images