JP2010261989A - Image processing device, display history confirmation support method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow participants to confirm content of main points in presentation and the like more easily in comparison with a conventional way. <P>SOLUTION: An image forming apparatus 1 includes a pointer position discrimination part 105 and a document screen display processing part 108. The pointer position discrimination part 105 detects a pointer placed over a document displayed on a screen by a projector and the position of the pointer in the document. The document screen display processing part 108 displays options for selecting the position of the detected pointer on a touch panel, and each time when a selection is made among the options, displays the document, for which the pointer is detected in the position pointed by an icon, on the touch panel while placing the icon so that it points the position of the pointer relating to the selected option. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for providing an image such as a document to a projector or the like.

  In recent years, image forming apparatuses having various functions such as copying, network printing, scanning, faxing, and file servers have become widespread. Such an image forming apparatus is sometimes called an “MFP (Multi Function Peripherals)” or “multifunction machine”. Image forming apparatuses are installed and used in various organizations such as companies, government offices, and educational institutions.

  In addition, projectors having a function of projecting a screen displayed on a personal computer onto a screen or the like have become widespread. In recent years, projectors have a function to output an image using image data in a format such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or TIFF (Tagged Image File Format). . As a result, an image can be projected without a personal computer.

  If the image forming apparatus and the projector described above are used, the image data managed by the image forming apparatus can be distributed to the projector, and the image can be displayed on the screen by the projector. In particular, if image data is transmitted to a projector provided in a conference room, an image can be easily shown to attendees as materials for presentations, and facilitators can easily make presentations.

  In addition, by pointing a specific portion of an image displayed on the screen, that is, a material using a pointing stick or a laser pointer, attendees can be promoted to understand the content of the presentation.

  Patent Documents 1 to 4 disclose systems and apparatuses that handle images for presentations and the like.

  In the presentation system described in Patent Document 1, a presentation image data output from a notebook PC is projected onto a screen by a projector, and a laser on a presentation image is obtained from image data in a projection range photographed by a camera unit. The pointer irradiation position by the pointer is detected. Then, a text portion or an image portion on the presentation image corresponding to the pointer irradiation position is extracted as a target portion to be pointed by the pointer, and an underline image perfectly aligned with the target portion is combined and projected and displayed. The surrounding line image QC surrounding the target portion is synthesized and projected and displayed.

  The projection apparatus described in Patent Literature 2 images a screen by an imaging unit during projection. The position information of the part designated by the laser pointer is acquired from the captured image by the image recognition unit. Further, the character recognition unit recognizes a character existing in the designated portion of the captured image, and acquires the character size. The control unit calculates an enlargement magnification such that the relative ratio to the size of the image being projected (pixel size in the vertical and horizontal directions) is an appropriate ratio determined in advance, and the projection magnification is calculated as an enlargement magnification. change. Just by designating an arbitrary part with the laser pointer, the designated part is projected in a state where characters existing in the part are enlarged to be easily seen.

  The presentation data processing apparatus described in Patent Document 3 includes a distributor, display means for displaying display data on a screen, first photographing means for photographing a display image on the screen, and second means for photographing a presenter. Image capturing means, display data capturing means, display part extracting means in the display image, coordinate system setting means for setting the coordinate system of the display part, indicated location coordinate calculating means for calculating the coordinate position of the indicated location, display It has presentation data synthesis means for synthesizing the designated portion in the data, and screen synthesis means for synthesizing the synthesized image of the presenter image and presentation data on one screen.

  The system described in Patent Document 4 obtains the importance for each piece of information included in the presentation material based on the position information of the pointer in the recorded image of the presentation performed using the presentation material in the image format. In accordance with the degree, the search suitability for each keyword in the presentation material is set, and the position information of the pointer including search means for using the search suitability for searching for information in the presentation material is used.

JP 2008-27080 A JP 2008-170742 A JP 2007-233647 A JP 2006-228059 A

  By the way, in the presentation, there are cases where the material is explained without skipping all parts. It is convenient for participants to easily know which part was used in order to recall later the contents of the presentation that progressed in this way.

  However, in the prior art, the voice of the speaker in the presentation must be recorded or screened and recorded, and then played back and confirmed. Even if it is confirmed by so-called double speed reproduction, it is inconvenient.

  In view of such problems, it is an object of the present invention to make it possible for a participant or the like to more easily check the contents that are important in a presentation or the like than in the past.

  An image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs support for confirming an image selectively displayed on a first display device, and is displayed by the first display device. Option display processing for displaying a pointer pointed to the image, pointer detection means for detecting the position of the pointer in the image, and an option for selecting the detected position of the pointer on the second display device And each time the option is selected, the image in which the pointer is detected at the position is displayed in a state where the mark is arranged to indicate the position of the pointer related to the selected option. Image re-display processing means for displaying on the display device.

  Preferably, the image redisplay processing unit displays the first material as the image, and then displays the second material as the image, and then displays the first material again as the image. Is displayed together with a second mark for indicating the existence of the second material.

  Alternatively, the image redisplay processing means displays the second material as the image when the second mark is selected.

  Or, when the mark is selected, the voice storage means for storing the voice recording the presentation performed by displaying the image on the first display device, and the position of the mark is pointed by the pointer Sound reproducing means for reproducing the sound at the time.

  Alternatively, the image re-display processing means arranges a mark including text representing the content of sound recorded when the position of the mark is pointed by the pointer.

  According to the present invention, the user can more easily confirm the contents that have become the main points in a presentation or the like.

It is a figure which shows the example of the whole structure of an organization network system. 2 is a diagram illustrating an example of a hardware configuration of an image forming apparatus. FIG. It is a figure which shows the example of an operation panel. 2 is a diagram illustrating an example of a functional configuration of an image forming apparatus. FIG. It is a figure which shows the example of arrangement | positioning of each apparatus of a presentation system. It is a figure which shows the example of the hardware constitutions of a projector. It is a figure which shows the example of a functional structure of a projector. It is a figure which shows the example of display switching log | history data. It is a flowchart explaining the example of the flow of a pointer position determination process. It is a figure which shows the example of a pointer table. It is a figure which shows the example of a document browser screen. It is a figure which shows the example of a document browser screen. It is a figure which shows the example of a document browser screen. 6 is a flowchart illustrating an example of the overall processing flow of the image forming apparatus.

  1 is a diagram illustrating an example of the overall configuration of the organization network system SY, FIG. 2 is a diagram illustrating an example of a hardware configuration of the image forming apparatus 1, FIG. 3 is a diagram illustrating an example of an operation panel 10h, and FIG. 5 is a diagram showing an example of the functional configuration of the image forming apparatus 1, FIG. 5 is a diagram showing an example of the arrangement of each device of the presentation system 2, FIG. 6 is a diagram showing an example of the hardware configuration of the projector 21, and FIG. It is a figure which shows the example of a 21 functional structure.

  As shown in FIG. 1, the organization network system SY includes an image forming apparatus 1, a presentation system 2, a personal computer 3, a communication line 4, and the like.

  The organization network system SY is provided in an organization such as a company, a government office, or a school, and is used for exchanging information between members of the organization. Hereinafter, an organization network system SY provided in a certain company will be described as an example.

  A user who is an employee of the company can transmit image data stored in the image forming apparatus 1 to a projector 21 (to be described later) and project an image by the projector 21 when giving a presentation at a meeting or the like. Furthermore, it is possible to create a progress log by recording the explanation given in the presentation and to refer to it after the presentation.

  The image forming apparatus 1, each apparatus (a projector 21, a video camera 22, and a recording apparatus 23 described later) constituting the presentation system 2, and the personal computer 3 can communicate with each other via the communication line 4. As the communication line 4, a so-called LAN (Local Area Network) line, a public line, the Internet, or the like is used.

  The image forming apparatus 1 is an apparatus generally called a multifunction peripheral or MFP (Multi Function Peripherals), and is an apparatus that integrates functions such as copying, network printing (PC printing), fax, scanner, box, and file transmission. is there.

  The “box function” is a storage area called a “box” corresponding to a folder or directory in a personal computer for each user, and the user stores image data or the like in his / her storage area. It is a function that allows you to

  The image forming apparatus 1 is installed on a floor where desks of a plurality of employees (users) gather, and is shared by a plurality of users.

  As shown in FIG. 2, the image forming apparatus 1 includes a CPU (Central Processing Unit) 10a, a RAM (Random Access Memory) 10b, a ROM (Read Only Memory) 10c, a hard disk 10d, a scanner 10e, a printing device 10f, and a network controller 10g. In addition to the operation panel 10h, the modem 10i, and the sound reproduction unit 10j, the control panel 10h is used.

  The scanner 10e is an apparatus that reads an image such as a photograph, a document, an illustration, a chart, and the like written on an original sheet and generates image data.

  The printing device 10f prints an image read by the scanner 10e or an image indicated by image data received from another device.

  As shown in FIG. 3, the operation panel 10h includes a touch panel 10h1, a key group 10h2, and the like.

  On the touch panel 10h1, a screen for giving a message to the user, a screen showing a processing result, a screen for the user to input an instruction to the image forming apparatus 1, and the like are displayed. A material browser screen 6 (see FIG. 11) described later is also displayed on the touch panel 10h1.

  The touch panel 10h1 detects the touched (pressed) position and notifies the CPU 10a of the position.

  The key group 10h2 includes a numeric key and a job start key, and mode selection keys for switching to screens corresponding to various function modes such as fax and scan. Each mode selection key is provided with an LED (Light Emitting Diode), and the mode selection key LED of the mode relating to the screen currently displayed on the touch panel 10h1 is lit.

  The user operates the operation panel 10 h to instruct the image forming apparatus 1 to transmit image data to the projector 21, select image data to be transmitted, and check a progress log. Can be.

  The network controller 10g is a NIC (Network Interface Card) for communicating with other devices such as a personal computer via the communication line 4.

  The modem 10i is a device for exchanging image data with other fax terminals using a protocol such as G3 via a fixed telephone network.

  The audio reproduction unit 10j includes a sound board and a speaker, and reproduces audio based on audio data.

  The ROM 10c or the hard disk 10d stores a document image data storage unit 101, a document image data distribution unit 102, a recording start command unit 103, a progress record data receiving unit 104, a pointer position determination unit 105, and a progress record data storage as shown in FIG. Programs for realizing the unit 106, the voice text conversion unit 107, the material screen display processing unit 108, the voice reproduction processing unit 109, and the like are stored. These programs are loaded into the RAM 10b as necessary and executed by the CPU 10a.

  All or some of the functions of the units illustrated in FIG. 4 may be realized only by hardware such as a circuit.

  Returning to FIG. 1, the presentation system 2 includes a projector 21, a video camera 22, a recording device 23, a screen 24, a laser point device 25, and the like. The presentation system 2 is provided in a conference room or the like, and is used to display presentation materials or record the contents of presentations as shown in FIG.

  The projector 21 projects an image based on RGB (Red Blue Green) image signals input from a personal computer or the like onto a screen 24 using a liquid crystal panel. The projector 21 may be generally called an “image projector” or a “liquid crystal projector”. The projector 21 may be a projector of a method other than the liquid crystal method. The screen 24 is a curtain on which an image is projected.

  The projector 21 can project an image based on image data such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), or TIFF (Tagged Image File Format). Therefore, it is possible to directly receive image data from the image forming apparatus 1 and project an image without using the personal computer 3.

  As shown in FIG. 6, the projector 21 includes a CPU 21a, a RAM 21b, a ROM 21c, an auxiliary storage device 21d, a control circuit 21e, an operation panel 21f, a network controller 21g, a projection unit 21h, and the like.

  The auxiliary storage device 21d is a non-volatile storage device that retains the stored contents even when the power is turned off. As the auxiliary storage device 21d, a semiconductor memory such as a flash memory or a magnetic storage device such as a hard disk is used.

  The auxiliary storage device 21d includes a material image data receiving unit 211, a material image data storage unit 212, a display material designation receiving unit 213, a display page designation receiving unit 214, an image display control unit 215, and a display history storage as shown in FIG. A program for realizing the unit 216, the display history notification processing unit 217, and the like is stored. These programs are loaded into the RAM 21b as necessary and executed by the CPU 21a.

  All or part of the functions of the units illustrated in FIG. 7 may be realized only by hardware such as a circuit.

  The control circuit 21e is a circuit for controlling the auxiliary storage device 21d, the operation panel 21f, the network controller 21g, the projection unit 21h, and the like. Note that a part or all of the functions of the control circuit 21e may be assigned to the CPU 21a.

  The operation panel 21f includes a display and various keys. The screen to be displayed on the display may be displayed (projected) on the screen by the projection unit 21h. In this case, it is not necessary to provide a display. The user can designate materials and pages to be displayed on the projector 21 by operating the keys.

  The network controller 21g is a NIC for communicating with the image forming apparatus 1 and the personal computer 3 using a protocol such as TCP / IP.

  The projection unit 21h separates the white light from the light source into an R (Red) color component, a G (Green) color component, and a B (Blue) color component, and synthesizes the separated light after transmitting the light to the liquid crystal panel. Then, the synthesized light is projected through the projection lens. An image based on the input image data is projected onto the screen 24 by controlling the display elements in the liquid crystal panel based on a command from the CPU 21a or the like.

  1 and 5, the laser point device 25 is a device about the size of a pen that emits laser light of a predetermined color such as red or green. Generally called “laser pointer”. A user giving a presentation (hereinafter referred to as “facilitator user”) irradiates a material displayed (projected) on the screen 24 by the projector 21 by irradiating a laser beam from the laser point device 25. You can let participants know where to pay attention.

  The video camera 22 shoots the screen 24 during the presentation, and transmits image data (hereinafter referred to as “captured image data DTS”) obtained thereby to the image forming apparatus 1.

  The recording device 23 records the voice of the facilitating user during the presentation, and transmits the voice data obtained thereby (hereinafter referred to as “recording data DTV”) to the image forming apparatus 1. . The recording device 23 is provided with a microphone, and is used by being placed near the facilitator user or attached to the chest of the facilitator user during the presentation.

  Returning to FIG. 1, the personal computer 3 has drivers for controlling the image forming apparatus 1, the projector 21, the video camera 22, and the recording apparatus 23, and controls these apparatuses as will be described later. Used for.

  As the personal computer 3, a desktop personal computer or a notebook personal computer is used. In the latter case, the facilitating user can bring it into the conference room and use it during the presentation.

  8 is a diagram showing an example of the display switching history data DTR, FIG. 9 is a flowchart for explaining an example of the flow of the pointer position determination process, FIG. 10 is a diagram showing an example of the pointer table TL1, and FIGS. It is a figure which shows the example of the screen.

  Next, details of each part of the image forming apparatus 1 shown in FIG. 4, each part of the projector 21 shown in FIG. 6, the video camera 22, the recording device 23, and the personal computer 3 will be described in detail.

  In the image forming apparatus 1, the document image data storage unit 101 in FIG. 4 describes image data of a document (document) composed of an image such as a document, an illustration, a photograph, or a chart (hereinafter referred to as “document image data DTG”). ) Is stored and managed by the box. The document image data DTG is used to display (project) the document used in the presentation with the projector 21 as described later. The document image data DTG is generated by scanning the paper of the document with the scanner 10e. Or it may be obtained by downloading from a Web server on the Internet. As the format of the document image data DTG, JPEG, GIF, TIFF, or the like is used.

  The document image data storage unit 101 stores document image data DTG in units of files. A unique file name is assigned to each document image data DTG. The file name is also used as the name of the material (material name).

  The facilitator user prepares to display the material used in the presentation on the projector 21 as follows. The facilitator user goes to the image forming apparatus 1 and operates the printing apparatus 10f (see FIG. 3) to log in to the image forming apparatus 1.

  When the material image data DTG of the material to be used is already stored in the box of the material image data storage unit 101, the facilitator user switches to the box mode with the mode selection key, and the material image data of the material from the list. Select DTG.

  On the other hand, when the document image data DTG of the material to be used is not yet present, the facilitator user switches to the scan mode, sets the material sheet on the document table, and causes the image forming apparatus 1 to scan the material. Then, the image forming apparatus 1 reads the material by the scan function as usual, generates the material image data DTG, and stores it in a predetermined box of the material image data storage unit 101.

  Then, the facilitator user switches to the box mode and selects the material image data DTG of the material from the list.

  The document image data distribution unit 102 reads the document image data DTG selected by the facilitating user by the above operation from the document image data storage unit 101 and distributes (transmits) the document image data to the projector 21.

  The facilitator user directly operates the image forming apparatus 1 and selects the document image data DTG. However, the facilitator user may operate the personal computer 3 or the projector 21 to select the document image data DTG remotely. Further, the facilitating user can transmit a plurality of document image data DTG to the projector 21 as necessary.

  A material image data receiving unit 211 of the projector 21 illustrated in FIG. 7 receives the material image data DTG from the image forming apparatus 1. The document image data storage unit 212 stores the received document image data DTG.

  The facilitator user goes to the conference room when the material image data DTG of the material used in the presentation can be prepared. When a predetermined time (hereinafter referred to as “start time T0”) is reached, the presentation is started.

  The recording start command unit 103 sends a command to start recording to the projector 21, the video camera 22, and the recording device 23 when the start time T0 is reached.

  Then, the video camera 22 starts photographing the screen 24, and the recording device 23 starts recording the voice of the facilitating user.

  The facilitator user can advance the presentation while displaying (projecting) the document on the projector 21.

  In displaying the material, the facilitator user operates the operation panel 21f to designate the material to be displayed to the projector 21.

  When the document image data DTG of a plurality of materials is stored in the projector 21, the facilitator user can change the materials to be displayed as appropriate during the presentation. Also in this case, the facilitator user only has to operate the operation panel 21f and specify the material to be displayed next to the projector 21.

  The display material designation receiving unit 213 receives a material designation by the facilitator user. At this time, the identifier of the designated material (for example, the file name of the material image data DTG of the material) and the received time are stored in the display history storage unit 216.

  When the document is composed of a plurality of pages, the facilitator user can operate the operation panel 21f and specify the page to be displayed next.

  Then, the display page designation receiving unit 214 receives the designation. At this time, the identifier (for example, page number) of the designated page and the received time are stored in the display history storage unit 216.

  In this manner, the designated history is stored in the display history storage unit 216 each time the facilitator user designates the material or page. For example, a material with the file name “main material A” is designated 5 seconds after the start time T0, the second page is designated 30 seconds after the start time T0, and the third page is designated 2 minutes 15 seconds after the start time T0. When a material with the file name “reference material B” is designated 3 minutes and 30 seconds after the start time T0, a history as shown in FIG. 8 is stored in the display history storage unit 216.

  When a laser pointer with a remote controller is used as the laser point device 25 and the projector 21 is equipped with a wireless receiver, the facilitating user operates a button provided on the laser point device 25 to operate the document and You may specify a page. The display material designation accepting unit 213 and the display page designation accepting unit 214 receive the designation of the material and the designation of the page by receiving a signal from the laser point device 25, respectively. Alternatively, when a notebook personal computer is used as the personal computer 3 and brought into the conference room, the facilitating user may use the personal computer 3 to specify materials and pages.

  When the designation of the material is accepted by the display material designation accepting unit 213, the image display control unit 215 reads the material image data DTG of the designated material from the material image data storage unit 212, and the default page of the material (for example, (First page) is displayed (projected) on the screen 24.

  Alternatively, the image display control unit 215 projects the designated page onto the screen 24 when the display page designation receiving unit 214 receives the designation of another page while projecting a certain page of a certain material.

  The display history notification processing unit 217 transmits display switching history data DTR indicating the history stored in the display history storage unit 216 to the image forming apparatus 1 after the end of the presentation. That is, the display history is notified to the image forming apparatus 1. Alternatively, notification may be made every time a document or page is designated, instead of collectively giving notification after the presentation ends.

  The progress record data receiving unit 104 of the image forming apparatus 1 receives data related to the progress of the presentation from the projector 21, the video camera 22, and the recording device 23. Specifically, the display switching history data DTR is received from the projector 21, the captured image data DTS is received from the video camera 22, and the recording data DTV is received from the recording device 23.

  By the way, as described above, the facilitator user appropriately writes the laser beam of the laser point device 25 on the material displayed on the screen 24 as being related to the matter being spoken (location, Proceed with the presentation. Therefore, the captured image data DTS records not only the video of the material but also the pointer of the laser beam.

  The pointer position discriminating unit 105 uses the method shown in FIG. 9 to indicate when and on which page of which material the pointer of the laser beam is applied based on the captured image data DTS and the display switching history data DTR. Determine.

  The pointer position determination unit 105 prepares a pointer table TL1 as shown in FIG. At this time, nothing is stored in the pointer table TL1.

  The pointer position determination unit 105 performs the following process for each frame of the video shot by the video camera 22. First, the pointer position determination unit 105 pays attention to the first frame based on the captured image data DTS (# 501 in FIG. 9).

  By comparing the focused frame (hereinafter referred to as “focused frame”) with the original frame of the focused frame, that is, the frame not pointed to by the laser point device 25, and identifying the position of the difference, The pointer and its position are detected (# 502). If there is no difference between them, the position of the pointer is not detected.

  It should be noted that the original frame of the noticeable frame specifies the material and page displayed on the screen 24 at the time when the noticeable frame was shot based on the display switching history data DTR received by the progress record data receiving unit 104. It can be obtained from the material image data DTG of the identified material.

  If the pointer and its position can be detected (Yes in # 503), the material and page of the original frame of the current frame of interest are not the same as the material and page of the original frame of the previous frame of attention (No in # 504), the pointer position determination unit 105 determines that the pointer points to a new part (# 509), the material, page, and position (detected position) pointed to by the pointer and the start time Pointer information PJ indicating the elapsed time from T0 is generated and stored in the pointer table TL1 (# 510). Hereinafter, this position is referred to as “initial position”.

  If the first frame is the target frame, there is no previous target frame, so if the pointer can be detected in step # 502, the pointer is necessarily a new pointer. The pointer information PJ of the pointer is added to the pointer table TL1.

  On the other hand, if the pointer could not be detected in step # 502 (No in # 503), neither processing for generating and adding pointer information PJ nor processing for updating pointer information PJ, which will be described later, is performed.

  Then, if the next frame of the frame of interest remains (No in # 512), the pointer position determination unit 105 newly pays attention to the next frame (# 511), and performs the processing of steps # 502 to # 510. Perform as appropriate:

  The pointer position determination unit 105 compares the next frame (that is, here the target frame) with the original frame of the target frame and identifies the position of the difference, thereby determining the pointer and the position from the target frame. Is detected (# 502).

  If the pointer and its position can be detected (Yes in # 503), the material and page of the original frame of the current frame of interest are not the same as the material and page of the original frame of the previous frame of attention (No in # 504), the pointer position determination unit 105 determines that the pointer is a new pointer, the material, page, and position (detected position) pointed to by the pointer and the elapsed time from the start time T0. Is generated and stored in the pointer table TL1 (# 510). That is, when there is a change in the displayed material or page, new pointer information PJ is generated and added to the pointer table TL1.

  If the material and page of the original frame of the current frame of interest are the same as the material and page of the original frame of the previous frame of interest (Yes in # 504), the pointer of the pointer information PJ added recently The difference between the initial position and the detected position of the current frame of interest is calculated in the X-axis direction and the Y-axis direction (# 505).

  If both the difference in the X-axis direction and the difference in the Y-axis direction are equal to or greater than a threshold value α (for example, 50 dots) (Yes in # 506), the pointer position determination unit 105 determines that the pointer has moved to a new part. (# 509). Pointer information PJ indicating the material, page and position (detected position) pointed to by the pointer and the elapsed time from the start time T0 is generated and stored in the pointer table TL1 (# 510).

  When at least one of the difference in the X-axis direction and the difference in the Y-axis direction is less than the threshold α (No in # 506), the pointer position determination unit 105 continues to determine that the pointer is pointing to the same part. (# 507). In this case, the pointer position determination unit 105 determines an average position between the position of the pointer detected from the current frame of interest and the position of each pointer detected from another frame that has been determined to indicate the same part. (That is, the position of the approximate center of the pointer movement range in these frames) is calculated, and the pointer position of the latest pointer information PJ is updated to the calculated position (# 508).

  For example, there are already three pointers that are determined to point to the same part, the positions of which are (x1, y1), (x2, y2), and (x3, y3), and the position of the pointer detected this time Is (x4, y4), the pointer position determination unit 105 updates the “pointer position (x, y)” of the latest pointer information PJ to ((x1 + x2 + x3 + x4) / 4, (y1 + y2 + y3 + y4) / 4). .

  By performing the above processing for each frame of the captured video, pointer information PJ for each portion pointed to by the pointer is obtained.

  4 and 6, the progress record data storage unit 106 stores the pointer table TL1 obtained by the pointer position determination unit 105 and the recording data DTV obtained by the progress record data reception unit 104. That is, the minutes of the presentation are saved.

  The voice text conversion unit 107 converts the voice of the facilitating user into text data based on the recording data DTV in response to a request from the material screen display processing unit 108 described below. That is, voice recognition processing is performed.

  The document screen display processing unit 108 performs a process of displaying a document browser screen 6 on the operation panel 10h for a user such as a facilitating user or a participant to view the document used in the presentation.

  As shown in FIG. 11, the document browser screen 6 includes a pointer transition display area 6A and a document display area 6B.

  In the material display area 6B, a list of the parts pointed to by the pointers shown in the pointer information PJ stored in the pointer table TL1 is arranged. When the user selects any part from the list, the material screen display processing unit 108 displays the material and page including the part in the material display area 6B based on the material image data DTG of the material. Further, the icon 6C is arranged so as to point to the position indicated by “pointer position (x, y)” of the pointer information PJ of the pointer on the document and the page.

  The user can easily confirm the portion explained by the pointer during the presentation by selecting the pointer transition display area 6A.

  In addition, the document screen display processing unit 108 displays a certain document page in the document display area 6B, then displays another page with another document, and then redisplays the previous document and page. The material 6 and the page icon 6D are arranged to point to the position of the pointer.

  For example, when the sixth item from the top of the pointer transition display region 6A is selected, the material screen display processing unit 108 displays the eighth page of the main material A in the material display region 6B. Next, when the seventh item from the top is selected, the tenth page of the reference material B is displayed. When the sixth item from the top is selected again, the icon 6D on the 10th page of the reference material B is arranged so as to indicate the same position as the icon 6C as shown in FIG. When the icon 6D is selected, the material screen display processing unit 108 displays the material and page corresponding to the icon 6D, that is, the 10th page of the reference material B again.

  When the user selects the icon 6C, the voice reproduction processing unit 109 reproduces the voice of the facilitating user when the pointer points to the icon 6C based on the recording data DTV. The reproduction start time can be found by referring to the elapsed time indicated in the pointer information PJ of the pointer. For example, if the elapsed time is “10 minutes 21 seconds”, the sound may be reproduced from the point where 10 minutes 21 seconds have elapsed from the start time T0. Further, the reproduction time is a predetermined number of seconds. Alternatively, the reproduction may be continued until the user issues a stop command.

  An icon having a text (character string) representing the voice generated by the voice recognition processing of the voice text conversion unit 107 and having the text in the shape of a balloon as shown in FIG. May be displayed as the icon 6C. The text may be displayed when the icon 6C is selected, or may be displayed from the beginning.

  FIG. 14 is a flowchart illustrating an example of the overall processing flow of the image forming apparatus 1.

  Next, the overall processing flow for presentation material presentation of the image forming apparatus 1 will be described with reference to the flowchart of FIG.

  The image forming apparatus 1 transmits the material image data DTG of the material designated by the facilitator user to the projector 21 before the presentation starts (# 1 in FIG. 14). When the start time T0 is reached, a command to start recording is given to the projector 21, the video camera 22, and the recording device 23 (# 2).

  Then, the projector 21 records the material and page designation by the facilitating user, and generates display switching history data DTR as shown in FIG. The video camera 22 captures the screen 24 to generate captured image data DTS. The recording device 23 generates recording data DTV by recording the voice of the facilitating user.

  The image forming apparatus 1 receives display switching history data DTR from the projector 21 as presentation progress log data, receives photographed image data DTS from the video camera 22, and receives recording data DTV from the recording device 23 ( # 3).

  Then, the image forming apparatus 1 determines the position of the pointer by the laser point device 25 based on the received photographed image data DTS and display switching history data DTR, and the position transition data, that is, as shown in FIG. Pointer information PJ is generated (# 4). The procedure for generating the pointer information PJ is as described above with reference to FIG.

  When the user inputs a predetermined command after the presentation is finished, the image forming apparatus 1 displays a document browser screen 6 as shown in FIG. 11 (# 5). In the material display area 6B, the material and page indicated by the pointer information PJ at the top of the pointer table TL1, that is, the material and page initially displayed on the screen 24 in the presentation are displayed as default materials and pages. To do. In the pointer transition display area 6A, a plurality of parts pointed by the pointer are shown, but at this point, the uppermost part is selected.

  Here, the user can select an arbitrary portion from the pointer transition display area 6A. Alternatively, the icon 6C can be selected. Or when the icon 6D is displayed, it can be selected.

  When any part is selected from the pointer transition display area 6A (Yes in # 7), the image forming apparatus 1 selects the material display area 6B instead of the currently displayed material and page. The material and page including the marked portion are newly displayed (# 10). However, if the material including the selected portion is different from the material currently displayed in the material display area 6B and has been displayed in the material display area 6B before (Yes in # 8), as shown in FIG. The material and page including the selected portion are displayed with the icon 6D arranged (# 9).

  Alternatively, when the icon 6D is selected (No in # 7, Yes in # 11), the image forming apparatus 1 displays the icon 6D in the material display area 6B instead of the currently displayed material and page. The associated material and page are displayed (# 12).

  Or, when the icon 6C is selected (No in # 7, No in # 11, Yes in # 13), the voice of the facilitating user when the part indicated by the icon 6C is pointed by the pointer is reproduced ( # 14).

  The processes of steps # 9, # 12, and # 14 are appropriately executed according to the operation by the user.

  According to the present embodiment, the user can more easily confirm the contents that are important points in the presentation, for example, the contents of the discussion and the action items.

  In the present embodiment, a liquid crystal projector is used as an apparatus for displaying a document.

  In the present embodiment, a laser pointer is used as a device for pointing a material, but a pointer may be used. In this case, the tip of the pointer may be detected as a pointer. Alternatively, a mouse pointer may be used. When the mouse pointer is used, the position of the pointer can be detected by inquiring the position of the mouse pointer to the operating system of the personal computer 3.

  In the present embodiment, shooting of the screen 24 and recording of the voice of the facilitating user are started when the start time T0 is reached, but may be started when the facilitating user receives a signal. In this case, the time when the signal is given may be set as the start time T0.

  The present invention can be used not only for presentations but also for classes and seminars.

  In the present embodiment, the document browser screen 6 is displayed on the touch panel 10h1 of the image forming apparatus 1, but may be displayed on the display of the personal computer 3.

  In addition, the organization network system SY, the image forming apparatus 1, the presentation system 2, the configuration of the whole or each part of the projector 21, the processing content, the processing order, the data configuration, the value of the threshold value α, and the like are appropriately determined in accordance with the spirit of the present invention. Can be changed.

1 Image forming device (image processing device)
104 Progress record data receiver (voice storage means)
105 Pointer position determination unit (pointer detection means)
108 Document screen display processing unit (option display processing means, image redisplay processing means)
109 Audio reproduction processing unit (audio reproduction means)
10h1 touch panel (second display device)
10j Audio reproduction unit (audio reproduction means)
24 screen (first display device)
6A Pointer transition display area 6B Material display area 6C Icon (mark)
6D icon (second landmark)

Claims (7)

An image processing device that provides support for confirming an image selectively displayed on a first display device,
A pointer pointed to the image displayed by the first display device and pointer detection means for detecting the position of the pointer in the image;
Option display processing means for displaying an option for selecting the position of the detected pointer on the second display device;
Each time the option is selected, the image in which the pointer is detected at the position is displayed on the second display device in a state where a mark is arranged to indicate the position of the pointer related to the selected option. Image redisplay processing means to be displayed;
An image processing apparatus comprising: The image re-display processing means displays the first material as the image, and then displays the second material as the image, and then displays the first material again as the image. Display with a second landmark to convey the presence of the second material,
The image processing apparatus according to claim 2. The image redisplay processing means displays the second material as the image when the second mark is selected,
The image processing apparatus according to claim 2. Audio storage means for storing audio recording a presentation made by displaying the image on the first display device;
Sound reproduction means for reproducing sound when the position of the mark is pointed by the pointer when the mark is selected;
The image processing apparatus according to claim 1. The image re-display processing means arranges a mark including text representing the content of sound recorded when the position of the mark is pointed by the pointer, as the mark.
The image processing apparatus according to claim 1. A display history confirmation support method for performing support for confirming an image selectively displayed on a first display device,
Causing the computer to detect the pointer pointed to the image displayed by the first display device and the position of the pointer in the image;
An option for selecting the detected position of the pointer is displayed on the second display device,
Each time the option is selected, the image in which the pointer is detected at the position is displayed on the second display device in a state where a mark is arranged to indicate the position of the pointer related to the selected option. Display,
A display history check support method characterized by that. A computer program used for a computer that provides support for confirming an image selectively displayed on a first display device,
In the computer,
Processing for detecting a pointer pointed to the image displayed by the first display device and a position of the pointer in the image;
A process of displaying on the second display device an option for selecting the detected position of the pointer;
Each time the option is selected, the image in which the pointer is detected at the position is displayed on the second display device in a state where a mark is arranged to indicate the position of the pointer related to the selected option. Processing to display,
A computer program for executing

Images