JP2012150616A - Presentation support system, presentation support device, image forming device, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a presentation environment which supports a presenter who came to his/her wits' end for making explanation.SOLUTION: A presentation support system comprises: a display processing unit that performs display processing to cause a display device to display a plurality of images for presentation one by one in sequence, and has, as display processing modes for displaying variable images associated with supplementary information, a normal mode that shows no supplementary information and a support mode that shows the supplementary information together with the variable images; a voice analysis unit for detecting a silent state in which volume of input voice stays equal to or less than a threshold for a predetermined period; and a display processing control unit for, when a silent state is detected while variable images are displayed in the normal mode, controlling the display processing unit to switch the normal mode to the support mode.

Description

  The present invention relates to a system and apparatus for supporting a presenter who makes a presentation.

  Generally, in a presentation, a presenter advances a presentation while showing a plurality of presentation images (slides) prepared in advance to an audience. There is widely known a presentation form in which an image is displayed in a large size using a projector or a large screen display, and an utterance is made while pointing a point to be noticed in the image using a pointer or a laser pointer.

  Conventionally, there has been proposed a presentation support system that automatically controls the display of slides so that the presentation ends on schedule (Patent Document 1). This system judges the progress status by comparing the scheduled elapsed time and presentation elapsed time set for each of multiple slides, and displays the low importance slides indicated by the attribute information when the progress is delayed. Or cancel the time-consuming animation display. The system also displays supplemental material for time adjustment associated with the displayed slide when progress is earlier than scheduled. This allows the presenter to finish the presentation almost as scheduled without being aware of the progress.

JP 2008-033550 A

  The presenter may forget what the presenter should explain or be silent because of tension and stop the presentation. It is only necessary for the presenter to be able to resume the explanation immediately, but there are cases where the presenter is impatient and it is difficult to resume the explanation. In such a situation, the above-described prior art has not been effective. That is, even if the time adjustment is performed by automatically thinning the display of the image (slide) that was originally scheduled to be displayed, the presenter may not be able to explain the original after the non-thinned image is displayed. Absent. You may not remember what you should talk about when the display changes.

  In view of such circumstances, an object of the present invention is to provide a presentation environment that supports presenters who have been unable to explain.

  The presentation support system that achieves the above object is responsible for display processing for displaying a plurality of presentation images one by one on the display device one by one, and as a display processing mode for a variable image that is an image associated with supplementary information. A display processing unit having a normal mode in which information is not displayed and a support mode in which the supplementary information is displayed together with the variable image, and a silence state in which the volume of sound input by the microphone is below a threshold for a certain time is detected. And causing the display processing unit to switch from the normal mode to the support mode when the silence state is detected during a period in which the variable image is displayed in the normal mode. A display processing control unit.

  In another aspect, the presentation support system detects an immobile state in which a presenter points to the same part in an image displayed by the display device over a certain period of time based on a captured image input by a camera. Is further provided. The display processing control unit switches the normal mode to the support mode when the immobility state is detected during the period in which the variable image is displayed in the normal mode. Let the department execute.

  According to the present invention, when the presenter gets stuck with the explanation, the supplementary information is displayed together with the image being displayed to increase the amount of visual information, thereby supplementing the explanation.

It is a figure which shows the hardware constitutions of the information system which concerns on embodiment of this invention. It is a figure which shows the structure of a presentation assistance system. It is a figure which shows the 1st example of the display content in each of normal mode and assistance mode. It is a figure which shows an example of the user interface used for preparation of a variable image. It is a figure which shows the 2nd example of the display content in each of normal mode and assistance mode. It is a figure which shows the 3rd example of the display content in each of normal mode and assistance mode. It is a figure which shows the 4th example of the display content in assistance mode. It is a figure which shows the example of a display of the mark for the switching instruction | indication of display processing mode. It is a figure which shows the example of the setting of a mark coordinate. It is a figure which shows the example of a display of the mark in the monitor display. FIG. 10 is a diagram illustrating a flow of operation of the MFP in the presentation support system. It is a figure which shows the flow of operation | movement of the personal computer in a presentation assistance system. It is a figure which shows the other example of the flow of operation | movement of a presentation assistance system. It is a figure which shows the function structure of the presentation assistance apparatus 2b.

  The presentation is performed using a plurality of devices illustrated in FIG. At the time of presentation, an image 8 is projected onto the screen 7 by the projector 3 connected to the personal computer 2. The presenter 9 points to a place in the image 8 where attention should be paid to the audience by using a pointer or a laser pointer (not shown) as necessary while speaking to the audience. The personal computer 2 is installed with software for presentation that causes the projector 3 to display each page of a document having a plurality of pages as a slide. In the present embodiment, the presenter 9 is supported by the cooperation of the personal computer 2 and the MFP (Multifunction Peripheral) 1 via the LAN (Local Area Network) 6.

  The MFP 1 is a business-use information device having various functions such as image input (scanning), facsimile communication, e-mail transmission / reception, and document storage in addition to copying and network printing, which are functions as an image forming apparatus. . The MFP 1 is configured to be able to input sound by the microphone 4 and input of a photographed image by the digital camera 5.

  FIG. 2 shows the configuration of the presentation support system 100. The presentation support system 100 includes a voice input unit 14, a camera control unit 15, a voice analysis unit 16, an image analysis unit 17, a notification processing unit 18, a display processing unit 21, and a display processing control unit 22. Among these elements, the voice input unit 14, the camera control unit 15, the voice analysis unit 16, the image analysis unit 17, and the notification processing unit 18 are executed by a CPU (central processing unit) provided in the MFP 1 to execute a predetermined computer program. Is a functional element realized by. The display processing unit 21 and the display processing control unit 22 that are the remaining elements are functional elements that are realized when the CPU of the personal computer 2 executes a predetermined computer program.

  The voice input unit 14 sequentially stores voice data input from the microphone 4 via the voice input interface in a hard disk drive (hereinafter referred to as HDD) 19 built in the MFP 1 as storage. Here, it is assumed that the microphone 4 is arranged in the presentation hall so as to mainly capture the voice of the presenter 9. The type of the microphone 4 may be a stationary type or a portable type that can be worn on the clothes of the presenter 9, and a signal transmission form may be wired or wireless.

  The camera control unit 15 instructs the digital camera 5 to shoot, and stores shooting data input from the digital camera 5 in the HDD 19. Here, it is assumed that the digital camera 5 is arranged so as to photograph the screen 7. A USB cable can be used for connection between the digital camera 5 and the MFP 1.

  The voice analysis unit 16 reads the voice data stored in the voice input unit 14 from the HDD 19 and detects a silence state in which the magnitude of the input voice is equal to or less than a threshold value for a certain time. The silence state is a state in which the presenter 9 regards it as being stuck with explanation. For example, when the input voice is low for 3 to 5 seconds, the voice analysis unit 16 determines that it is in a silence state and informs the notification processing unit 18 accordingly.

  The image analysis unit 17 detects a non-moving state in which the presenter 9 indicates the same portion in the image displayed on the screen 7 for a certain period of time based on the shooting data periodically stored by the camera control unit 15. The non-moving state is also a state that the presenter 9 regards as being stuck in the explanation. For example, when the position of the tip of the pointer or the spot of the laser pointer stays within a predetermined small area for 3 to 5 seconds, the image analysis unit 17 determines that it is in an immobile state and notifies that to the notification processing unit. Tell 18 In addition, the image analysis unit 17 determines whether or not there is an operation of pointing the mark to the presenter 9 during a period in which the mark that is an operation button for instructing the presenter 9 to switch the display processing mode is displayed on the screen 7. The determination result is transmitted to the notification processing unit 18.

  The notification processing unit 18 notifies the display processing control unit 22 of the personal computer 2 via the communication interface as a supplementary information display instruction as a supplementary information display instruction when a silence state is detected and when a stationary state is detected. To do. Further, the notification processing unit 18 passes the mark coordinates, which are the display position information of the mark notified from the display processing control unit 22, to the image analysis unit 17, and a mark indication notification indicating the determination result regarding the mark from the image analysis unit 17. Is sent to the display processing control unit 22.

  The display processing unit 21 of the personal computer 2 is responsible for display processing for sequentially displaying a plurality of presentation images (including the image 8) on the projector 3 and the monitor display 27 one by one. The display processing unit 21 displays the normal mode in which the supplementary information is not displayed and the supplementary information together with the variable image as a display processing mode for the variable image that is an image associated with supplementary information to be described later related to the support operation. Has support mode. Note that the normal mode is applied to images that are not variable images.

  Then, the display processing control unit 22 causes the display processing unit 21 to switch the display processing mode when a silence state or a non-moving state is detected during a period in which the variable image is displayed in the normal mode. However, in this embodiment, the display processing control unit 22 causes the display processing unit 21 to execute a process of displaying the mark prior to switching the display processing mode, and then determines that there has been an operation of pointing to the mark. In addition, the display processing unit 21 is caused to switch from the normal mode to the support mode. The mark is displayed on the screen 7 and the display surface of the display 27, respectively, and an operation for pointing the mark displayed on the display 27 with a pointing device such as a mouse 29, and the display processing control unit 22 points to the mark on the screen 7. Treat it like an operation.

  FIG. 3 shows a first example of display contents in each of the normal mode and the support mode. An image 8 illustrated in FIG. 3 is a text image that conveys information mainly by characters, and corresponds to one page of a document having a plurality of pages. In the normal mode, a character string 201 representing the title of information to be transmitted and character strings 202 and 203 representing the main points of the information are displayed. The presenter 9 conveys predetermined information to the audience by speaking while referring to the image 8. On the other hand, in the support mode that is applied instead of the normal mode when the silence state or the immobility state is detected, the character strings 301 and 302 representing the detailed contents regarding the character strings 202 and 203 are used as supplementary information. 203 is displayed. By displaying the supplemental information, the presenter 9 can read the supplemental information and proceed with the presentation. Even if the speech of the presenter 9 remains interrupted, the audience can understand to some extent the information that the presenter 9 wants to convey by displaying supplementary information.

  A document in which the amount of information displayed for each page is different between the normal mode and the support mode is a document in which the display level of the character string can be set for each line by using a technique such as a pull-down list 501 shown in FIG. It can be created by using a creation application. In the pull-down list 501 of FIG. 4, it is possible to select level 1 (document level) displayed in both the normal mode and the support mode, and level 2 (supplementary explanation level) displayed only in the support mode.

  FIG. 5 shows a second example of display contents in each of the normal mode and the support mode. In the normal mode, as in the example of FIG. 3, a character string 201 representing the title of information to be transmitted and character strings 202 and 203 representing the main points of the information are displayed. On the other hand, in the support mode, balloons 301b and 302b including character strings representing detailed contents related to the character strings 202 and 203 are displayed together with the character strings 201, 202, and 203 as supplementary information.

  FIG. 6 shows a third example of display contents in each of the normal mode and the support mode. As shown in FIG. 6A, the image 8b is displayed on the screen 7 in the normal mode, and in the support mode, the material 81 associated with the image 8b by, for example, a hyperlink is displayed side by side with the image 8b as supplementary information. FIG. 6B is a modification of FIG. As shown in FIG. 6B, in the support mode, the image 8b and the material 81 may be displayed in a 2-in-1 format that is combined into one page.

  FIG. 7 shows a fourth example of display contents in the support mode. In this example, a material consisting of a plurality of pages 81, 82, 83 is associated as supplementary information with the image 8b displayed in both the normal mode and the support mode. In the support mode, one page of the supplementary information is displayed together with the image 8b, and page switching buttons 91 and 92 are further displayed. When the presenter 9 points to one of the page switching buttons 91 and 92 with a pointer or a laser pointer, the page of supplemental information displayed is switched as shown in the figure.

  FIG. 8 shows a display example of marks for instructing display processing mode switching. In FIG. 8, a circular mark 95 is displayed so as to overlap the periphery of the image 8. When the presenter 9 points to the mark 95, the normal mode is switched to the support mode as described above, and supplementary information is displayed. If the presenter 9 determines that display of supplementary information is unnecessary (the supplementary information may remain hidden), the presenter 9 may advance the presentation without pointing to the mark 95. The display position of the mark 95 is specified by coordinates in a coordinate system having the origin at the lower left corner of the image 8 as shown in FIG. 9, for example, and is notified to the image analysis unit 17. The unit of coordinates can be the resolution (pixel) of the projected image, the document size (mm), and the like.

  In parallel with the display of the mark 95 on the screen 7, the mark 95b is also displayed on the monitor display 27 as shown in FIG. As described above, the normal mode is switched to the support mode when the mouse operation (referred to as mark pressing) pointing to the mark 95b is performed.

  FIG. 11 shows an operation flow of the MFP 1 in the presentation support system.

  The MFP 1 takes in and analyzes the audio data and the shooting data (S11, S12). If it is a silence state or a stationary state (YES in S13), a supplementary information display instruction is sent to the personal computer 2 (S14), and the presence or absence of notification of mark coordinates is checked (S15). After receiving the notification of the mark coordinates (YES in S16), the digital camera 5 is instructed to photograph the screen 7 periodically (for example, every second), receives photographing data, and indicates whether or not there is an operation indicating the mark 95. Image analysis determined by comparing time-series data is performed (S17, S18, S19). If there is an operation for pointing the mark 95 (YES in S20), a mark pointing notification is sent to the personal computer 2 (S21).

  FIG. 12 shows the operation flow of the personal computer 2 in the presentation support system.

  When the personal computer 2 receives an instruction to display supplementary information from the MFP 1 (S31, S32), whether or not supplemental information is associated with the currently displayed image (page), that is, the displayed image is supplementary information. It is checked whether the image is a variable image that can be hidden / displayed (S33, S34). If it is a variable image, the marks 95 and 95b are displayed on the projector 3 and the display 27, respectively (S35), and the mark coordinates are notified to the MFP 1 (S36). If there is a mark press indicating the mark 95b on the display 27 or a mark pointing notification from the MFP 1 (S38, S39, S40), the display processing mode is switched from the normal mode to the support mode, and supplementary information is displayed (S41). Thereafter, the fact that the mode has been switched to the support mode is stored, and even if the display target is switched to another image in accordance with the display order instead of the currently displayed image, if the switched image is a variable image, the presenter 9 is in the state of change. First, supplementary information is displayed (S42). That is, once the normal mode is switched to the support mode, the support mode is maintained thereafter.

  In the above embodiment, the presenter 9 exemplifies the operation of determining whether or not the display processing mode should be switched by displaying the marks 95 and 95b. However, the silencer or the immobile state can be detected without displaying the marks 95 and 95b. In response, the normal mode may be switched to the support mode. Moreover, you may make it detect only one of a silence state and an immobility state. As an example, FIG. 13 shows a flow of operations of the MFP 1 and the personal computer 2 when the display processing mode is switched when a stationary state is detected.

  In FIG. 13, the MFP 1 instructs the digital camera 5 to perform shooting (S51), and captures and analyzes the input shooting data (S52, S53). If it is in an immobile state (YES in S54), a supplementary information display instruction is sent to the personal computer 2. If it is not in a stationary state (NO in S54), the process returns to step S51 to continue monitoring the presenter 9 based on the photographing data. On the other hand, when receiving an instruction to display supplementary information from the MFP 1 (S56, S57), the personal computer 2 checks whether the currently displayed image (page) is a variable image (S58). If it is a variable image, the display processing mode is switched from the normal mode to the support mode, and supplementary information is displayed on the projector 3 (S59, S60).

  FIG. 14 shows a functional configuration of the presentation support apparatus 2b. In the above example, the functions of the presentation support system 100 distributed between the MFP 1 and the personal computer 2 can be integrated into one information device. In FIG. 14, the functions for supporting the presenter 9 are integrated into one personal computer, and this personal computer operates as the presentation support apparatus 2b.

  In the presentation support apparatus 2b, the communication processing unit 18 is omitted, and the display processing control unit 22b makes a shooting request to the camera control unit 15. Except for this point, the function of the display processing control unit 22b is the same as the function of the display processing control unit 22 of FIG. In the configuration of FIG. 14, a large display 3b is used instead of the projector 3 as a display device for display to the audience.

  In the above embodiment, the reference time for detecting the silence state and the immobility state is not limited to the above 3 to 5 seconds, and may be appropriately selected according to the actual situation. Different times may be set for the silence state and the immobility state.

  According to the above-described embodiment, since the marks 95 and 95b are displayed, the presenter 9 can adjust the display information amount by switching between non-display / display of supplementary information based on his / her own judgment.

100 Presentation support system 1 MFP (image forming apparatus)
2 Personal computer (information equipment)
2b Presentation support device 8, 8b Image (variable image)
3 Projector (display device)
3b Large display (display device)
21 Display processing unit 4 Microphone 16 Audio analysis unit 22, 22b Display processing control unit 5 Digital camera 17 Image analysis unit 95, 95b mark

Claims (9)

As a display processing mode for a variable image, which is an image associated with supplemental information, and responsible for display processing that sequentially displays a plurality of presentation images one by one on the display device one by one, normal mode that does not display the supplementary information and supplementary information A display processing unit having a support mode for displaying the image together with the variable image;
A voice analysis unit for detecting a silence state in which the magnitude of the voice input by the microphone is equal to or less than a threshold value over a certain period of time;
A display processing control unit for causing the display processing unit to switch from the normal mode to the support mode when the silence state is detected during a period in which the display in the normal mode is performed for the variable image; A presentation support system comprising: Based on the captured image input by the camera, further comprising an image analysis unit for detecting a stationary state in which the presenter points to the same part in the image displayed by the display device over a certain period of time,
The display processing control unit causes the display processing unit to switch from the normal mode to the support mode when the immobile state is detected during a period in which the display of the variable image is performed in the normal mode. The presentation support system according to claim 1 to be executed. The image analysis unit determines whether or not there is an operation indicating the mark in a period in which a mark that is an operation button for the presenter to instruct switching of the display processing mode is displayed together with the variable image,
The display processing control unit switches the display processing mode to the display processing unit when the silence state or the immobility state is detected during a period in which the variable image is displayed in the normal mode. The display processing unit is caused to execute a process of displaying the mark prior to performing the display process, and when it is determined that there has been an operation pointing to the mark, the display process is performed to switch from the normal mode to the support mode. The presentation support system according to claim 2. An image forming apparatus capable of network communication, having a communication processing unit capable of voice input by the microphone and notifying the display processing control unit that the voice analysis unit and the silence state have been detected;
The presentation support system according to claim 1, further comprising: an information device capable of connecting to the display device and having the display processing unit and the display processing control unit and capable of network communication. Voice input by the microphone and input of the captured image by the camera are possible, and the voice analysis unit, the image analysis unit, and the display processing control unit are notified that the silence state or the immobility state has been detected. An image forming apparatus capable of network communication, having a communication processing unit for
The presentation support system according to claim 2, further comprising an information device capable of connecting to the display device and having the display processing unit and the display processing control unit and capable of network communication. As a display processing mode for a variable image, which is an image associated with supplemental information, and responsible for display processing that sequentially displays a plurality of presentation images one by one on the display device one by one, normal mode that does not display the supplementary information and supplementary information From the normal mode when the silent state is detected during a period in which the variable image is displayed in the normal mode. An image forming apparatus that is communicably connected to an information device including a display processing control unit that causes the display processing unit to switch to the support mode via a network,
Voice input with a microphone and input of a shot image with a camera are possible.
A voice analysis unit for detecting a silence state in which the magnitude of the voice input by the microphone is equal to or less than a threshold value over a certain period of time;
Based on the photographed image, an image analysis unit that detects a stationary state in which the presenter indicates the same part in the image displayed by the external display device over a certain period of time;
An image forming apparatus comprising: a communication processing unit that notifies the display processing control unit that the silence state or the immobile state is detected. A presentation support device,
As a display processing mode for a variable image, which is an image associated with supplemental information, and responsible for display processing that sequentially displays a plurality of presentation images one by one on the display device one by one, normal mode that does not display the supplementary information and supplementary information A display processing unit having a support mode for displaying the image together with the variable image;
A voice analysis unit for detecting a silence state in which the magnitude of the voice input by the microphone is equal to or less than a threshold value over a certain period of time;
An image analysis unit that detects a stationary state in which a presenter indicates the same portion in an image displayed by the display device over a certain period of time based on a captured image input by a camera;
Display for causing the display processing unit to switch from the normal mode to the support mode when the silence state or the immobility state is detected during the period in which the display in the normal mode is performed for the variable image. And a processing control unit. The image analysis unit determines presence / absence of an operation indicating the mark in a period in which the presenter displays a mark together with the variable image as an operation button for instructing switching of a display processing mode,
The display processing control unit switches the display processing mode to the display processing unit when the silence state or the immobility state is detected during a period in which the variable image is displayed in the normal mode. The display processing unit is caused to execute a process of displaying the mark prior to performing the display process, and when it is determined that there has been an operation pointing to the mark, the display process is performed to switch from the normal mode to the support mode. The presentation support apparatus according to claim 7, wherein the presentation support apparatus is executed by a section. A computer program for operating an information device provided with a computer as a presentation support device,
When executed by the computer,
As a display processing mode for a variable image, which is an image associated with supplemental information, and responsible for display processing that sequentially displays a plurality of presentation images one by one on the display device one by one, normal mode that does not display the supplementary information and supplementary information A display processing unit having a support mode for displaying the image together with the variable image;
A voice analysis unit for detecting a silence state in which the magnitude of the voice input by the microphone is equal to or less than a threshold value over a certain period of time;
An image analysis unit that detects a stationary state in which a presenter indicates the same portion in an image displayed by the display device over a certain period of time based on a captured image input by a camera;
Display for causing the display processing unit to switch from the normal mode to the support mode when the silence state or the immobility state is detected during the period in which the display in the normal mode is performed for the variable image. A computer program for causing a computer to implement a processing control unit.

Images