JP6399221B2 - Presentation support device, presentation support method, and presentation support program - Google Patents

Description

  The present invention relates to a presentation support apparatus, a presentation support method, and a presentation support program.

  As an example of a technology for supporting a presentation, there is a technique for presenting a presenter or a listener with a part that the presenter is explaining. For example, a display device has been proposed that aims to display a manuscript read by a speaker in an easy-to-understand manner. This display device recognizes a phrase uttered by a speaker, identifies a read-out portion of a document being displayed on a display panel based on the recognized phrase, and displays a display state of the identified portion as a first state. The display state is changed to a second display state different from the first display state, for example, highlight display such as blinking.

JP 2009-271814 A

  However, the longer the distance to which the highlight display is switched before and after the highlight display is switched, the lower the possibility that the location where the highlight display is performed after the transition will be noticed. This is because in a state where the focus is focused on the portion where the highlight display has been performed before the transition, the visual recognition power decreases as the distance from the focus increases. This is not to say that the degree of emphasis in highlight display should be increased uniformly. This is because when the degree of emphasis is increased uniformly, the burden on the eyes is increased, which may lead to eye strain.

  An object of one aspect is to provide a presentation support device, a presentation support method, and a presentation support program that can easily recognize a place where highlight display is switched.

  A presentation support device according to an aspect includes a determination unit that determines a region to be subjected to highlight display among pages displayed on a screen by a display device, and a calculation unit that calculates a distance between regions where the highlight display is switched. When the distance between the regions is greater than or equal to a predetermined threshold, the second highlight having a higher highlight intensity than the first highlight display executed when the distance between the regions is less than the threshold. A display control unit that applies display.

  This makes it easy to recognize where the highlight display changes.

FIG. 1 is a diagram illustrating the configuration of the presentation support system according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the presentation support apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of a method for calculating a distance between regions. FIG. 4 is a diagram illustrating an example of an angle. FIG. 5 is a diagram illustrating an example of an angle. FIG. 6 is a flowchart illustrating a procedure of extracted word data generation processing according to the first embodiment. FIG. 7 is a flowchart illustrating the procedure of the speech recognition process according to the first embodiment. FIG. 8 is a flowchart illustrating a procedure of highlight display control processing according to the first embodiment. FIG. 9 is a diagram illustrating a hardware configuration example of a computer that executes the presentation support program according to the first embodiment and the second embodiment.

  Hereinafter, a presentation support apparatus, a presentation support method, and a presentation support program according to the present application will be described with reference to the accompanying drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[System configuration]
FIG. 1 is a diagram illustrating the configuration of the presentation support system according to the first embodiment. The presentation support system 1 shown in FIG. 1 is a presentation support service that highlights an area corresponding to a result of speech recognition in a page screen included in a document file, for example, a page displayed on the display device 5 of a slide. I will provide a. Hereinafter, as an example, a case where an area on a slide to be highlighted is determined by using voice recognition is illustrated, but the voice recognition may not necessarily be used for such determination.

  As part of such a presentation support service, the presentation support system 1 increases the highlight intensity of the transition destination area when the distance to which the highlight display area in the displayed slide moves is greater than or equal to the threshold. Highlight display. Thus, as one aspect, the possibility of losing sight of the highlight display after the transition is reduced even when the distance of movement before and after the highlight display is switched is long. Even if the highlight display after the transition is lost, the period until it is recognized again is shortened. Therefore, it is possible to easily recognize the place where the highlight display is switched. As another aspect, the burden on the eye can be reduced as compared with the case where the highlight intensity is uniformly increased.

  Here, in the following, as an example, assuming that the above-described highlight display function is added to the presentation software, one or more slides included in a document file created using the presentation software are displayed on the display device 5. Assume that the presentation is progressed by displaying it on the screen. Such slides can be imported with text and graphics as well as content created by other application programs. For example, you can import documents created with word processing software, tables and graphs created with spreadsheet software, images and movies captured with an imaging device, and images and movies edited with image editing software. And can be imported.

  As shown in FIG. 1, the presentation support system 1 accommodates a microphone 3, a display device 5, an input device 7, and a presentation support device 10. The peripheral devices such as the microphone 3, the display device 5 and the input device 7 and the presentation support device 10 are connected by wire or wirelessly.

  The microphone 3 is a device that converts sound into an electrical signal, and is sometimes called a microphone. For example, the microphone 3 can be attached to a presenter who performs a presentation. In this case, a headset-type or tie-pin type microphone can be attached to a predetermined position of the presenter's body or clothes, or a hand-type microphone can be carried by the presenter. The microphone 3 can also be installed at a predetermined position in a range where the utterance of the presenter can be collected. In this case, the microphone 3 may be an attachment type or a stationary type microphone. In any of these cases, a microphone having any type of directivity can be adopted as the microphone 3, but sounds other than the presenter's utterance, for example, the utterance of the listener and the noise such as noise are collected. In order to suppress this, the sensitivity of the microphone can be limited to the speaking direction of the presenter. The microphone 3 can employ any conversion method such as a dynamic type, an electret capacitor type, or a capacitor type.

  The analog signal obtained by collecting the sound in the microphone 3 is converted into a digital signal and then input to the presentation support apparatus 10.

  The display device 5 is a device that displays various types of information. For example, the display device 5 may be a liquid crystal display or an organic EL (electroluminescence) display that realizes display by light emission, or a projector that realizes display by projection. Further, the number of installed display devices 5 is not necessarily limited to one, and a plurality of display devices 5 may be provided. In the following, as an example, it is assumed that a projector and a screen that displays an image projected by the projector are mounted as a shared display device that is viewed by both presenters and listeners who are participants of the presentation.

  For example, the display device 5 displays a presentation screen according to an instruction from the presentation support device 10. For example, the display device 5 displays a slide of a document file opened by presentation software that operates on the presentation support device 10. In this case, the display device 5 can display any slide specified by the presenter via the input device 7 among the slides included in the document file, and the slide show function of the presentation software is set to the ON state. In this case, the slides included in the document file can be switched and displayed in the order in which the slides are created.

  The input device 7 is a device that receives instruction inputs for various types of information. For example, when the display device 5 is mounted as a projector, a laser pointer indicating the position on the slide projected on the screen can be mounted as the input device 7. That is, among laser pointers, there is also a laser pointer with a remote control function provided with an operation unit such as various buttons for advancing and returning a slide page. The operation unit of the laser pointer with a remote control function can be used as the input device 7. In addition, a predetermined part of the screen or presenter was imaged in order to adopt a mouse or keyboard as the input device 7, or to sense the position of the pointer pointed by the laser pointer, detect the gaze of the presenter, or recognize the gesture. An image sensor that inputs an image may be employed as the input device 7. When the display device 5 is mounted as a liquid crystal display, the input device 7 may be a touch sensor bonded on the liquid crystal display.

  As an example, the input device 7 accepts designation of a document file to be executed by the presentation software on the presentation support device 10, an operation of advancing a slide page, an operation of returning a slide page, and the like. The operation accepted through the input device 7 in this way is output to the presentation support device 10.

  The presentation support apparatus 10 is a computer on which presentation software is executed.

  As an embodiment, the presentation support apparatus 10 may employ an information processing apparatus such as a desktop or notebook personal computer. In addition, the presentation support apparatus 10 can employ not only a stationary terminal such as the personal computer but also various portable terminal apparatuses. For example, as an example of a mobile terminal device, a mobile communication terminal such as a smart phone, a mobile phone or a PHS (Personal Handyphone System), and a slate terminal such as a PDA (Personal Digital Assistants) are included in the category.

  In the present embodiment, as an example, it is assumed that the presentation support apparatus 10 provides the above-described presentation support service in a stand-alone manner that independently executes the above-described presentation software without depending on external resources. Although the details will be described later, the presentation support service is not limited to the implementation provided in a stand-alone manner. For example, it is possible to construct a client server system or a thin client system by providing a server that provides the presentation support service for a client that executes presentation software.

[Configuration of Presentation Support Device 10]
Next, the functional configuration of the presentation support apparatus 10 according to the present embodiment will be described. FIG. 2 is a block diagram illustrating a functional configuration of the presentation support apparatus 10 according to the first embodiment. As shown in FIG. 2, the presentation support apparatus 10 includes an input / output I / F (InterFace) unit 11, a storage unit 13, and a control unit 15. In FIG. 2, a solid line representing the input / output relationship of data is shown, but for the convenience of explanation, only a minimum portion is shown. That is, the input / output of data related to each processing unit is not limited to the illustrated example, and the input / output of data other than those illustrated, for example, between the processing unit and the processing unit, between the processing unit and data, and between the processing unit and the external Data input / output between devices may be performed.

  The input / output I / F unit 11 is an interface that performs input / output with peripheral devices such as the microphone 3, the display device 5, and the input device 7.

  As one embodiment, the input / output I / F unit 11 outputs various operations input from the input device 7 to the control unit 15. Further, the input / output I / F unit 11 outputs the slide image data output from the control unit 15 to the display device 5, and outputs a highlight instruction or an instruction to cancel the area included in the slide to the display device 5. To do. Further, the input / output I / F unit 11 outputs the audio data input from the microphone 3 to the control unit 15.

  The storage unit 13 is a device that stores data used for various programs such as an OS (Operating System) and presentation software executed by the control unit 15 and application programs.

  As an embodiment, the storage unit 13 is implemented as a main storage device in the presentation support apparatus 10. For example, various semiconductor memory elements such as a RAM (Random Access Memory) and a flash memory can be employed for the storage unit 13. The storage unit 13 can also be implemented as an auxiliary storage device. In this case, an HDD (Hard Disk Drive), an optical disk, an SSD (Solid State Drive), or the like can be employed.

  The storage unit 13 stores document data 13a, extracted word data 13b, and recognized word data 13c as an example of data used in a program executed by the control unit 15. In addition to these data, the storage unit 13 can also store other electronic data such as definition data related to highlight display. The extracted word data 13b and the recognized word data 13c other than the document data 13a will be described in accordance with the description of the processing unit that registers or references each data.

  The document data 13a is data related to a document.

  As an embodiment, a document file in which one or more slides are created using presentation software can be adopted as the document data 13a. Such slides can be imported with text and graphics as well as content created by other application programs. For example, you can import documents created with word processing software, tables and graphs created with spreadsheet software, images and movies taken with an imaging device, and images and movies edited with image editing software. And can be imported. In this way, in order to realize a keyword search by voice recognition, meta information including a character string such as an explanatory word or a description of the content is added to the content other than the text before the presentation is started. Can do.

  The control unit 15 has an internal memory for storing various programs and control data, and executes various processes using these.

  As an embodiment, the control unit 15 is implemented as a central processing unit, a so-called CPU (Central Processing Unit). The control unit 15 is not necessarily implemented as a central processing unit, and may be implemented as an MPU (Micro Processing Unit) or a DSP (Digital Signal Processor). Thus, the control part 15 should just be mounted as a processor, and it is not ask | required whether the classification is a general purpose type or a specialization type. The control unit 15 can also be realized by a hard wired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  The control unit 15 virtually implements the following processing unit by executing various programs. For example, as illustrated in FIG. 2, the control unit 15 includes an extraction unit 15a, a recognition unit 15b, a determination unit 15c, a calculation unit 15d, and a display control unit 15e.

  The extraction unit 15a is a processing unit that extracts, as extracted word data 13b, words registered in dictionary data used for speech recognition from slides included in a document file.

  As an embodiment, the extraction unit 15a can automatically start the process of extracting the extracted word data 13b or can be started manually. For example, when starting automatically, when the presentation software saves the document file in the storage unit 13 and then closes, or when the document file is saved in the storage unit 13 while editing the document file via the presentation, Can be activated. In addition, when starting with manual setting, the processing can be activated when an instruction to execute presentation pre-processing is received via the input device 7. In any case, the processing is started by reading out the document file corresponding to the save or execution instruction from the document files included in the document data 13a stored in the storage unit 13.

The generation of the extracted word data 13b will be described. The extraction unit 15a has received an instruction to execute a preprocessing of a document file that has been saved or a presentation among document files included in the document data 13a stored in the storage unit 13. Read a document file. Here, as an example, the case where the extraction unit 15a reads a document file from the storage unit 13 is illustrated, but the acquisition path of the document file is not limited to this. For example, the extraction unit 15a may be an auxiliary storage device such as a hard disk or an optical disk, a memory card, or a USB (Universal
Serial Bus) Document files can also be obtained from removable media such as memory. The extraction unit 15a can also acquire a document file by receiving it from an external device via a network.

  Subsequently, the extraction unit 15a divides the slide included in the previously read document file into a plurality of areas. For example, the extraction unit 15a divides the slide in units of one sentence, line, paragraph, and the like. In this case, the extraction unit 15a scans a character string included in the slide, detects a delimiter character corresponding to a space, a punctuation point, or a line feed, and sets the delimiter character as a boundary of the region. The extraction unit 15a divides the character string included in the slide before and after such a boundary. Thus, the slide is divided into a plurality of areas for each delimiter character. After that, the extraction unit 15a assigns an index for identifying the area to the area obtained by dividing the slide. Here, the case where the slide is automatically divided is illustrated, but the slide may be divided by manual setting by designating the boundary of the area via the input device 7 or the like.

  After dividing the slide, the extraction unit 15a selects one of the plurality of regions included in the slide. Subsequently, the extraction unit 15a extracts a word by executing natural language processing on the character string included in the previously selected region. For example, the extraction unit 15a extracts a word whose part of speech is a noun, a word forming a phrase, or the like from morphemes obtained by executing morphological analysis or the like on a character string in the region. Then, the extraction unit 15a assigns an index assigned to a region including the word to each previously extracted word. Thereafter, the extraction unit 15a repeatedly executes the extraction of the word and the assignment of the index until all the regions included in the slide are selected.

  After the words are extracted from all the areas in this way, the extraction unit 15a calculates the number of appearances of the word k in the document for each word included in the slide. For example, the number of appearances in the document is calculated by counting the number of times the word k appears in the document. Then, the extraction unit 15 a registers the extracted word data 13 b in which the word k, the index idx, and the number of appearances of the word k in the document are associated with the storage unit 13.

  The recognition unit 15b is a processing unit that performs voice recognition.

  As an embodiment, the recognition unit 15b is activated when the presentation software receives a presentation start instruction with the document file opened, and waits until an audio signal having a predetermined length of time is input from the microphone 3. For example, it waits for an audio signal having a time length of at least one frame, for example, 10 msec. The recognizing unit 15b performs voice recognition such as word spotting on the voice signal every time a voice signal having a predetermined time length is input from the microphone 3. At this time, the recognizing unit 15b extracts the extracted word data relating to the slide that is included in the document file that is being executed by the presentation software among the extracted word data 13b stored in the storage unit 13 and that is being displayed on the display device 5. 13b applies to word spotting. Thereby, the recognition unit 15b recognizes whether or not a word extracted from each region included in the slide being displayed exists in the utterance of the presenter. And when the reading of a word is recognized from an audio | voice signal, the recognition part 15b registers the recognition word data 13c with which the said word and the time when the word was recognized were matched to the memory | storage part 13. FIG. When the same word is recognized a plurality of times as time passes, the last, that is, the latest recognized time is registered in the storage unit 13.

  Thereafter, the recognizing unit 15b determines whether or not there is a word for which a predetermined period has elapsed since it was registered in the storage unit 13 among the recognized word data 13c stored in the storage unit 13. For example, for each word included in the recognition word data 13c, the recognition unit 15b has a difference between the time registered in association with the word and the time when the recognition unit 15b refers to the recognition word data 13c, that is, the current time. It is determined whether or not a predetermined threshold is exceeded. At this time, the recognizing unit 15b can change the threshold value used for the above determination according to a unit in which the slide is divided, for example, one sentence, line, paragraph, or the like. For example, when the slide is divided in units of lines, it can be assumed that the number of characters read out in one area is approximately 20 to 30 characters. In this case, as an example of the above threshold, 3 seconds can be used by calculating the time required for reading from the average reading speed of the explanation voice, which is 7 beats / second to 8 beats / second. Further, when the slide is divided in units of paragraphs, it can be assumed that a time longer than the line unit is devoted to reading. In this case, the number of rows × 3 seconds can be used as an example of the above threshold value.

  Here, when there is a word for which a predetermined period, for example, 3 seconds has elapsed since registration in the storage unit 13, there is a high possibility that the explanation regarding the slide region including the word has ended. If such a word is left, the possibility that the area where the explanation has ended will be highlighted. Therefore, the recognition unit 15b deletes the record related to the word from the recognized word data 13c stored in the storage unit 13. On the other hand, when there is no word for which a predetermined period has elapsed since registration in the storage unit 13, there is a high possibility that the description regarding the slide area where the word included in the recognized word data 13c appears has not ended. Therefore, the recognition unit 15b leaves the word included in the recognized word data 13c stored in the storage unit 13 without deleting it.

  The recognizing unit 15b determines whether or not the slide page displayed on the display device 5 has been changed. For example, the recognizing unit 15b determines whether a slide is switched by a slide show, or an operation for advancing a slide page or an operation for returning a slide page is received via the input device 7. At this time, when the slide page displayed on the display device 5 is changed, it is highly possible that the description of the presenter is switched from the slide of the page before the change to the slide of the page after the change. In this case, the recognition unit 15b deletes the recognized word data 13c stored in the storage unit 13. On the other hand, when the slide page displayed on the display device 5 is not changed, there is a high possibility that the page explained by the presenter will not change. In this case, the recognition unit 15b leaves the word included in the recognized word data 13c stored in the storage unit 13 as it is without being deleted.

  Through these series of operations, the recognition unit 15b recognizes a word that is highly likely to be explained by the presenter in the displayed slide. In the following, a word included in the extracted word data 13b is described as an “extracted word”, and a word included in the recognized word data 13c is described as a “recognized word” to distinguish the labels from each other. is there.

  The determination unit 15c is a processing unit that determines an area for performing highlight display.

  As one embodiment, the determination unit 15c calculates a score from the recognized word included in the area for each area in the slide being displayed. For example, the determination unit 15 c selects one index among the indexes of the area included in the slide that is being displayed on the display device 5. Subsequently, the determination unit 15c extracts a recognition word included in the previously selected index area among the recognition words included in the recognition word data 13c. At this time, the number of appearances of the word k in the document can be acquired by referring to the extracted word data 13b. Then, the determination unit 15c uses the number of appearances f (x) of the recognized word x in the document, the mora number m (x) of the recognized word x, and the probability c (x) of the recognition result to recognize the recognized word x. The word score s (x) is calculated. The “probability” here refers to how similar the recognition result is to the spectrum of each phoneme included in the standard model of the word. The value is 1.0.

  More specifically, the determination unit 15c adds the number of appearances f (x) of the recognition word x in the document, the mora number m (x) of x, and the probability c of the recognition result to the following formula (1). The word score s (x) of the recognized word x is calculated by substituting parameters such as x). Here, min (a, b) indicates a function that outputs the smaller value of a and b, and M is a constant, for example, 6 is adopted. Thereafter, the determination unit 15c calculates a highlight score S (d) by calculating a word score s (x) for each recognized word included in the region d and then summing the word scores of all the recognized words. . In the following equation (1), the word score s (x) of the recognized word x is calculated using the number of appearances f (x) of the recognized word x in the document and the number of mora m (x) of the recognized word x. In addition, although an example using three parameters of the probability c (x) of the recognition result has been described, only an arbitrary parameter may be used, and the method of obtaining is not limited to this.

  s (x) = 1 / f (x) × min (1.0, m (x) / M) × c (x) (1)

  After the highlight score is calculated in this way, the determination unit 15c determines whether or not the highest score among the highlight scores calculated for each region is equal to or greater than a predetermined threshold Th1, for example, “2”. At this time, when the highest score is equal to or greater than the threshold Th1, the determination unit 15c further determines whether highlight display is being performed. When the highlight display is not being executed, the determination unit 15c determines to execute the highlight display regarding the region having the highest score. On the other hand, when highlight display is being performed, the determination unit 15c determines whether the index of the area for which the highest score is calculated matches the index of the area for which highlight display is being performed, in other words, the highest. It is further determined whether or not the area where the score is calculated is the same as the area where the highlight display is being executed. At this time, if the area where the highest score is calculated and the area where the highlight display is being executed are the same, the determination unit 15c maintains the highlight display being executed. On the other hand, when the area where the highest score is calculated is different from the area where the highlight display is being executed, the determination unit 15c highlights the area from which the highlight display is being executed to the area where the highest score is calculated. To migrate.

  The calculation unit 15d is a processing unit that calculates the distance between regions included in the slide being displayed.

  As one embodiment, the calculation unit 15d determines whether the area where the highest score is calculated by the determination unit 15c is different from the area where the highlight display is being executed, that is, when the highlight display is switched, between the two areas. To calculate the distance between each other. If the highest score is less than the threshold value Th1, highlighting is not being executed, or if there is no change in the area having the highest score between the previous time and the current time, highlighting is performed in the first place. There is no need to calculate the distance between the regions because there is no transition even if there is no highlight display.

  FIG. 3 is a diagram illustrating an example of a method for calculating a distance between regions. FIG. 3 illustrates a horizontal slide 200. The slide 200 includes a heading item “technical point”, an area E1 including a bulleted item “◯◯” of (1) and a bulleted item “ΔΔ” of (2), and a heading item “assumed service”, (b) And the area E2 including the bulleted item “XX”.

  For example, the calculation unit 15d can calculate the distance d1 between the coordinates of the upper left vertex LT1 of the region E1 and the coordinates of the upper left vertex LT2 of the region E2. The calculating unit 15d can also calculate the distance d2 between the coordinates of the center of gravity G1 of the region E1 and the coordinates of the center of gravity G2 of the region E2. Furthermore, the calculation unit 15d can also calculate a vertical difference between the lower end of the region E1 and the upper end of the region E2 as the distance d3. Further, the calculation unit 15d can also calculate a distance in the vertical direction between the upper end of the region E1 and the lower end of the region E2 as the distance d4. In addition, although not shown, the calculation unit 15d can calculate the length of the shortest straight line connecting the region E1 and the region E2 as the distance d5. A statistical value of one or a plurality of distances among these d1 to d5 is calculated as the distance d between the regions. Here, as an example, even if the highlight display moves in the direction from the top to the bottom of the slide or from the bottom to the top of the slide, the distance between the absolute values is calculated with the sign of each other being positive. Assuming the case, the sign may be changed depending on the direction.

  After the distance between the areas is calculated in this way, the calculation unit 15d converts the distance between the areas on the slide into the display distance between the areas on the screen according to the size of the screen of the display device 5. In other words, since the distance between the previously obtained areas is the distance on the slide created by the presentation software, the distance between the areas displayed on the screen of the display device 5 may be different. For example, when the slide size is set to A4, the distance between areas displayed within a rectangle of 210 mm long × 297 mm wide is calculated. Therefore, if the screen size is not A4, the area on the slide is calculated. And the distance between the areas actually displayed on the screen is different. From this, the calculation unit 15d converts the distance between the areas on the slide into the display distance between the areas on the screen according to the size of the screen.

  More specifically, the calculation unit 15d, for example, is either a document file that has been saved by presentation software, a document file that has received an instruction to execute presentation pre-processing, or a document file that has received an instruction to start a presentation. For example, identification information of a venue where a presentation relating to the conference is held, for example, a conference room or a hall input by the user is acquired. By using such venue identification information as a search key, the screen size can be automatically specified by searching or the like. For example, the venue identification information described above may be managed as venue information in a state in which the venue identification information is associated with equipment available at the venue, such as a product number on a screen. When the search for the product number of the screen of the display device 5 installed in the venue is successful from the venue information, the calculation unit 15d displays screen product information (not shown), for example, information in which the product number and the screen size are associated with each other. The size of the screen associated with the product number is retrieved by referring to. Thereby, the size of the screen on which the display device 5 projects the slide can be specified. Thereafter, the calculation unit 15d multiplies the distance between the areas on the slide by a magnification of a size obtained from the screen size and the slide size thus identified, for example, a magnification of one side or a magnification of an area. The display distance between areas on the screen is calculated.

  In this example, the screen size is searched for each document file. However, when the screen and slide dimensions are fixed, it is not necessary to search the screen size, and the ratio of each size is a fixed value. It is also possible to calculate the display distance between the areas on the screen by setting a work area on the memory referred to by the calculation unit 15d and referring to the work area.

  After the display distance between the regions is calculated in this way, the calculation unit 15d calculates the presentation participant before and after the highlight display shifts based on the display distance between the regions and the presentation participant and the screen distance. An angle at which the line of sight changes can be further calculated.

  For example, assume that the distance from the listener to the screen is used as an example of the distance between the participants in the presentation and the screen. The above venue identification information is managed as venue information in a state associated with seat arrangements at the venue, such as the distance from the screen where the projector displays the slide to the presenter's place, the distance from the screen to the auditorium, etc. Therefore, the distance from the screen to the auditorium can be searched by using the venue identification information as a search key. In this case, as an example, the distance from the screen to the nearest auditorium is used as the distance between the listener and the screen, and as another example, a predetermined statistical value of the distance from the screen to each auditorium, for example, the mode The value, median or average value can be used as the distance between the listener and the screen.

  Here, in a place deviating from a focal point of the viewer's line of sight on the screen by a predetermined angle, for example, 5 degrees (about 1/4 of the screen), the visual acuity of the listener is about 30% compared to the focal point. To fall. In other words, when the highlight display is shifted to a place that is separated by 5 degrees or more, the possibility that the highlight display is performed after the shift is reduced. Therefore, as an example, the calculation unit 15d uses a listener before and after the highlight display shifts to change the highlight intensity depending on whether or not the highlight display shifts to a position where the focus of the listener shifts by 5 degrees or more. Find the angle at which the line of sight changes.

  For example, the calculation unit 15d uses a display distance between regions defined in a plane direction parallel to the display surface of the screen and a distance to a predetermined fixed point defined in the normal direction of the screen display surface, By taking arctan which is an inverse function of tan, an angle θ at which the audience's line of sight changes before and after the highlight display shifts is obtained. That is, since the relationship of tan θ≈ (display distance between areas on the screen / distance from the listener to the screen) is established, the calculation unit 15d determines arctan (display distance between areas on the screen / listener to screen). The angle θ is obtained by calculating (distance to) = θ.

  The display control unit 15 e is a processing unit that executes display control for the display device 5.

  As one embodiment, when a document file is opened by presentation software, the display control unit 15e causes the display device 5 to display a slide included in the document file. At this time, the display control unit 15e may display the slide of the first page among the slides included in the document file, or may display the slide of the page edited last. Thereafter, when receiving a page switching instruction via the input device 7, the display control unit 15 e changes the slide displayed on the display device 5. For example, when an operation for advancing a page is received, the display control unit 15e causes the display device 5 to display a slide on the next page of the slide being displayed. Further, when the operation of returning the page is received, the display control unit 15e causes the display device 5 to display the slide of the previous page of the displayed slide.

  In addition, the display control unit 15e performs highlight display relating to the region determined by the determination unit 15c in accordance with an instruction from the determination unit 15c. Here, “highlight display” means not only narrow highlight display, that is, display control in a broad sense, but also display control that brightens or reverses the background color. For example, highlighting in general can be arbitrarily executed, such as emphasizing area filling and emphasizing fonts (font size, underline and italic).

  At this time, the display control unit 15e has the highest score between the previous time and the current time when the highest score is equal to or higher than the threshold Th1 and highlight display is not being executed, or the highest score is equal to or higher than the threshold Th1. When there is no change in the area, the first highlight display having a lower highlight intensity than the second highlight display described later is executed. For example, when emphasizing the fill of an area having the highest score or emphasizing a font (font size, underline or italic), the first highlight display is made by lowering the contrast from the second highlight display described later. Is realized. In addition, when the highlight intensity is increased by blinking display in the second highlight display, the first highlight display is turned on to realize a lower highlight intensity than the second highlight display. You can also

  The display control unit 15e determines the angle calculated by the calculation unit 15d when the region where the highest score is calculated by the determination unit 15c is different from the region where the highlight display is being executed, that is, when the highlight display is switched. It is determined whether θ is greater than or equal to a predetermined threshold Th2. At this time, if the angle θ calculated by the calculation unit 15d is less than the threshold Th2, the display control unit 15e performs the first highlight display on the region having the highest score.

  Here, when the angle θ calculated by the calculation unit 15d is equal to or larger than the threshold Th2, the display control unit 15e executes the second highlight display having a higher highlight intensity than the first highlight display. . For example, the second highlight display can be realized by raising the contrast to be higher than the first highlight display. When the first highlight display is realized by lighting display, the second highlight display can be realized. By using a blinking display, a second highlight display having a higher highlight intensity than the first highlight display can be realized.

  FIG. 4 is a diagram illustrating an example of an angle. FIG. 4 shows a case where the listener browses the slide on the screen of the projector. Further, FIG. 4 assumes a case where a screen of about 100 inches having a height of 1.5 m is arranged at a distance of 5 m from the listener. As shown in FIG. 4, when the distance from the listener to the projector screen is 5 m, the highlight display is a distance of “0.44 m” or more from the center of the upper and lower ends of the screen to the screen surface direction, for example, the vertical direction. When moving to a distant position, the angle θ is 5 ° or more. This is because, by calculating arctan (0.44 / 5) = θ, the angle θ≈5 degrees can be obtained when the display distance between regions on the screen is 0.44 m. For example, if the material is made up of 10 lines with the same character size and line spacing, when the highlight display moves over 4 lines or more, the audience's line of sight is shifted by 5 degrees or more. In such a case, by executing the second highlight display whose highlight intensity is higher than that of the first highlight display, even if the highlight display moves away from the place more than 5 degrees, The possibility of losing sight of the light display can be reduced, and even if the highlight display after the transition is lost, the period until it is recognized again can be shortened.

  FIG. 5 is a diagram illustrating an example of an angle. FIG. 5 shows a case where the listener browses the slide on the monitor. Further, FIG. 5 assumes a case where a screen of about 24 inches having a height of 40 cm is arranged at a distance of 100 cm from the listener. As shown in FIG. 5, when the distance from the listener to the monitor screen is 100 cm, the highlight display is a distance of “8.7 cm” or more from the center of the upper and lower ends of the screen to the screen surface, for example, the vertical direction. When moving to a distant position, the angle θ is 5 ° or more. This is because, by calculating arctan (8.7 / 100) = θ, when the display distance between the regions on the screen is 8.7 cm, the angle θ≈5 degrees can be obtained. For example, if the material is made up of 10 lines with the same character size and line spacing, the viewer's line of sight will be shifted by 5 degrees or more when the highlight display moves over 3 lines or more. In such a case, by executing the second highlight display whose highlight intensity is higher than that of the first highlight display, even if the highlight display moves away from the place more than 5 degrees, The possibility of losing sight of the light display can be reduced, and even if the highlight display after the transition is lost, the period until it is recognized again can be shortened.

  Thus, using the distance to a predetermined fixed point defined in the normal direction of the screen display surface and the distance between the areas defined in the plane direction parallel to the screen display surface, highlighting from the fixed point is performed. An angle is obtained when the line of sight to the area before the display transition changes to the line of sight to the area after the highlight display transition. Then, when the angle is a predetermined threshold value, which is 5 degrees or more in the above example, the second highlight display having a higher highlight intensity than the first highlight display is executed. As a result, it is possible to easily recognize the place where the highlight display is switched, and it is possible to reduce the burden on the eyes as compared with the case where the highlight intensity is uniformly increased.

[Process flow]
Next, a processing flow of the presentation support apparatus 10 according to the present embodiment will be described. Here, description will be made in the order of (1) extracted word data generation processing, (2) speech recognition processing, and (3) highlight display control processing executed by the presentation support apparatus 10.

(1) Extracted Word Data Generation Processing FIG. 6 is a flowchart illustrating a procedure of extracted word data generation processing according to the first embodiment. This process can be started automatically or manually. For example, when starting automatically, when the presentation software saves the document file in the storage unit 13 and then closes, or when the document file is saved in the storage unit 13 while editing the document file via the presentation, Can be activated. In addition, when starting with manual setting, the processing can be activated when an instruction to execute presentation pre-processing is received via the input device 7. In any case, the processing is started by reading out the document file corresponding to the execution instruction of the storage or preprocessing among the document files included in the document data 13a stored in the storage unit 13.

  As illustrated in FIG. 6, the extraction unit 15a divides a slide included in the document file into a plurality of regions in units of one sentence, line, paragraph, or the like (step S101). Subsequently, the extraction unit 15a assigns an index for identifying each region to the region obtained in step S101 (step S102).

  Then, the extraction unit 15a selects one index from the indexes assigned in step S102 (step S103). Subsequently, the extraction unit 15a extracts a word whose part of speech is a noun from morphemes obtained by performing morphological analysis or the like on the character string in the index area selected in step S103 (step S104). Thereafter, the extraction unit 15a gives each word extracted in step S104 an index assigned to the area including the word (step S105).

  Then, the extraction unit 15a repeatedly executes the processes from step S103 to step S105 until all the indexes assigned in step S102 are selected (No in step S106).

Thereafter, when all the indexes assigned in step S102 are selected (step S106 Yes), the extraction unit 15a calculates the appearance frequency f _k of the word k for each word included in the slide (step S107). Then, the extraction unit 15a registers the extracted word data 13b in which the word k, the index idx, and the appearance frequency _fk are associated with each other in the storage unit 13 (Step S108), and ends the process.

(2) Voice Recognition Processing FIG. 7 is a flowchart illustrating the procedure of voice recognition processing according to the first embodiment. This process is started when the presentation software receives a presentation start instruction with the document file opened, and is repeatedly executed until a presentation end instruction is received.

  As shown in FIG. 7, the recognition unit 15b waits until an audio signal having a predetermined time length is input from the microphone 3 until an audio signal having a time length of, for example, at least one frame, for example, 10 msec is input (Step S15). S301).

  When an audio signal having a predetermined time length is input from the microphone 3 (Yes at Step S301), the recognition unit 15b performs voice recognition such as word spotting on the audio signal (Step S302). When word spotting is executed in step S302, the slide is included in the document file being executed by the presentation software among the extracted word data 13b stored in the storage unit 13 and is being displayed on the display device 5. Extracted word data relating to a certain slide is applied as dictionary data for speech recognition.

  At this time, when a word is recognized from the voice signal (Yes at Step S303), the recognition unit 15b stores the recognized word data 13c in which the word recognized at Step S302 and the time when the word is recognized are associated with each other. (Step S304), and the process proceeds to step S305.

  On the other hand, when a voice signal having a predetermined time length is not input from the microphone 3 or when a word is not recognized from the voice signal (No in Step S301 or Step S303), the subsequent processing is skipped and the processing proceeds to Step S305. .

  Here, the recognizing unit 15b determines whether or not there is a word for which a predetermined period has elapsed since registration in the storage unit 13 among the recognized word data 13c stored in the storage unit 13 (step S305). If there is a word for which a predetermined period has elapsed since registration in the storage unit 13 (Yes in step S305), the recognition unit 15b deletes the record related to the word from the recognized word data 13c stored in the storage unit 13. (Step S306). If there is no word for which a predetermined period has elapsed since registration in the storage unit 13 (No in step S305), the process of step S306 is skipped and the process proceeds to step S307.

  Thereafter, the recognition unit 15b determines whether or not the slide page displayed on the display device 5 has been changed (step S307). At this time, when the slide page displayed on the display device 5 is changed (Yes in step S307), the recognizing unit 15b deletes the recognized word data 13c stored in the storage unit 13 (step S308), and in step S301. Returning to the processing, the processing after step S301 is repeatedly executed. If the slide page displayed on the display device 5 has not been changed (No at Step S307), the process returns to Step S301 without executing Step S308.

(3) Highlight Display Control Processing FIG. 8 is a flowchart illustrating a procedure of highlight display control processing according to the first embodiment. This process is executed in parallel with the speech recognition process shown in FIG. 7, and is started when the presentation software receives a presentation start instruction with the document file opened, and gives a presentation end instruction. It is repeatedly executed until it is accepted. The cycle in which the execution of the process is repeated may be the same as or different from the voice recognition process shown in FIG. 7 and is executed in synchronization with the voice recognition process shown in FIG. It can also be executed asynchronously.

  As illustrated in FIG. 8, the determination unit 15c calculates a highlight score of each area included in the slide being displayed (step S501). Thereafter, the determination unit 15c determines whether or not the highest score among the highlight scores calculated for each region in Step S501 is equal to or greater than a predetermined threshold Th1 (Step S502). If the highest score is less than the threshold value Th1 (No in step S502), the process is terminated as it is.

  On the other hand, when the highest score is equal to or higher than the threshold Th1 (step S502 Yes), the determination unit 15c further determines whether highlight display is being executed (step S503). If the highlight display is not being executed (No in step S503), the display control unit 15e executes the first highlight display for the region having the highest score (step S510), and the process is terminated.

  Here, when the highlight display is being executed (Yes in step S503), the determination unit 15c differs from the index of the area where the highest score is calculated in step S501 and the index of the area where the highlight display is being executed. It is determined whether or not (step S504). If the area where the highest score is calculated in step S501 is the same as the area where highlight display is being executed (No in step S504), the display control unit 15e maintains the highlight display being executed (step S504). S510), the process is terminated.

  At this time, when the area where the highest score is calculated in step S501 is different from the area where the highlight display is being executed (Yes in step S504), the calculation unit 15c calculates the distance between the two areas on the slide. Calculate (step S505).

  Subsequently, the calculation unit 15d converts the distance between the areas on the slide into the display distance between the areas on the screen according to the size of the screen of the display device 5 (step S506). Then, the calculation unit 15d changes the line of sight of the presentation participant before and after the highlight display shifts based on the display distance between the areas on the screen calculated in step S506 and the presentation participant and the screen distance. The angle θ is calculated (step S507).

  Then, the display control unit 15e determines whether or not the angle θ calculated in step S507 is greater than or equal to the threshold Th2 (step S508). At this time, when the angle θ calculated in step S507 is less than the threshold Th2 (No in step S508), the display control unit 15e executes the first highlight display for the region having the highest score (step S508). S510), the process is terminated.

  On the other hand, when the angle θ calculated in step S507 is equal to or larger than the threshold Th2 (step S508 Yes), the display control unit 15e displays the second highlight display having a higher highlight intensity than the first highlight display. This is executed (step S509), and the process is terminated.

[One aspect of effect]
As described above, the presentation support apparatus 10 according to the present embodiment, when the distance to which the highlight display area in the slide being displayed moves is equal to or greater than the threshold, the highlight intensity of the transfer destination area. To highlight it. Therefore, as one aspect, it is possible to reduce the possibility of losing sight of the highlight display after the transition even when the distance of movement before and after the highlight display is switched is long. Further, even when the highlight display after the transition is lost, the period until it is recognized again can be shortened. Therefore, according to the presentation support apparatus 10 according to the present embodiment, it is possible to easily recognize the place where the highlight display is switched. As another aspect, the burden on the eye can be reduced as compared with the case where the highlight intensity is uniformly increased.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[How to determine the area to highlight]
In the first embodiment, the case where the area on the slide where highlight display is performed is determined by using the voice recognition is exemplified, but the voice recognition may not necessarily be used for the determination. For example, it is possible to designate an area for performing highlight display from the presenter via the input device 7. In addition, refer to the schedule data in which the time elapsed from the start of the presentation or the time based on the calendar and the area where the highlight is scheduled to be performed at that time, and the time data is correlated with the time The predetermined scheduled area may be determined as an area to be highlighted. Even in these cases, the same display control can be realized by executing the processing of steps S505 to S510 shown in FIG.

[Cancel highlighting]
In the first embodiment, if there is no region having a highlight score that is equal to or higher than the threshold Th1 in the highlight score calculated by the determination unit 15c, the processing is terminated without performing any processing. However, at this time, if there is an area in which highlight display is being executed, the highlight display may be deleted, or the highlight score of the area in which highlight display is being executed becomes less than the threshold Th1. The highlight display may be erased when a certain time (for example, 10 seconds) or more has elapsed. As a result, for example, when a transition is made to an explanation of a topic that is not written in the document in the explanation, the highlight display is erased after a certain period of time, so that the topic of the highlighted display portion continues to the listener. Suppresses misleading situations.

[Highlight intensity switching]
In the first embodiment, the case where the second highlight display of the area is continued as long as the highlight score of the area having the highest score is equal to or higher than the threshold Th1 is described. Even if the score is equal to or higher than the threshold Th1, when a certain time, for example, 10 seconds or more has elapsed since the second highlight display is started, the second highlight display can be changed to the first highlight display. . As a result, when it can be assumed that the highlight display after the transition has been recognized, the highlight intensity is lowered, and the burden on the eyes can be further reduced.

[Application examples of document files]
In the first embodiment, the case where the document file created by the presentation software is used has been exemplified. However, the document file created by another application program can be used. That is, if the document file includes a page that is displayed on a screen basis at the time of display, the page included in the document file of the word processing software is replaced with a slide, or the sheet included in the document file of the spreadsheet software is replaced with a slide. The process shown in FIG. 8 can be similarly applied.

[Other implementation examples]
In the first embodiment, the case where the presentation support apparatus 10 provides the presentation support service in a stand-alone manner in which the presentation software 10 is independently executed without depending on an external resource is exemplified. Can also be adopted. For example, a client server system can be constructed by providing a server that provides the presentation support service to a client that executes presentation software. In this case, the server device can be implemented by installing a presentation support program for realizing the above presentation support service as package software or online software. For example, the server device may be implemented as a Web server that provides the presentation support service, or may be implemented as a cloud that provides the presentation support service by outsourcing. In this case, the client starts the presentation after uploading the document file used for the presentation and the identification information of the venue where the presentation is performed to the server device. When the presentation is started, the client uploads the audio signal collected from the microphone 3 in real time, and uploads the slide page information every time the slide page being displayed on the display device 5 is switched. As a result, the server apparatus can execute the processes shown in FIGS. Furthermore, the client can also be constructed as a thin client system by transmitting operation information related to the input device 7 to the server and causing the display device 5 to display only the processing result transmitted from the server. In this case, various resources such as document data are also held by the server, and the presentation software is also implemented as a virtual machine on the server. In the first embodiment, it is assumed that the presentation support program is added to the presentation software. However, when a request for referring to the presentation support program as a library is received from a client having a license authority, the presentation support program is It can also be plugged in.

[Application to remote conference system]
In the first embodiment, the case where a document displayed on one display device is shared and viewed by a plurality of people is illustrated, but a plurality of terminals are connected via a remote conference system or the like, and a plurality of terminals are connected. The same applies to the case where one of the terminals designates a shared document and highlight and transmits information to another terminal.

[Presentation support program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. Therefore, in the following, an example of a computer that executes a presentation support program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 9 is a diagram illustrating a hardware configuration example of a computer that executes the presentation support program according to the first embodiment and the second embodiment. As illustrated in FIG. 9, the computer 100 includes an operation unit 110 a, a speaker 110 b, a camera 110 c, a display 120, and a communication unit 130. Further, the computer 100 includes a CPU 150, a ROM 160, an HDD 170, and a RAM 180. These units 110 to 180 are connected via a bus 140.

  As shown in FIG. 9, the HDD 170 stores a presentation support program 170a that exhibits the same functions as the determination unit 15c, the calculation unit 15d, and the display control unit 15e described in the first embodiment. This presentation support program 170a may be integrated or separated, as is the case with the components of the determination unit 15c, calculation unit 15d, and display control unit 15e shown in FIG. That is, the HDD 170 does not necessarily have to store all the data shown in the first embodiment, and data used for processing may be stored in the HDD 170.

  Under such an environment, the CPU 150 reads the presentation support program 170 a from the HDD 170 and expands it on the RAM 180. As a result, the presentation support program 170a functions as a presentation support process 180a as shown in FIG. The presentation support process 180a expands various data read from the HDD 170 in an area allocated to the presentation support process 180a in the storage area of the RAM 180, and executes various processes using the expanded data. For example, examples of processing executed by the presentation support process 180a include the processing illustrated in FIGS. Note that the CPU 150 does not necessarily operate all the processing units described in the first embodiment, and the processing unit corresponding to the process to be executed may be virtually realized.

  Note that the presentation support program 170a is not necessarily stored in the HDD 170 or the ROM 160 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 100 may acquire and execute each program from these portable physical media. In addition, each program is stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires and executes each program from these. It may be.

DESCRIPTION OF SYMBOLS 1 Presentation support system 3 Microphone 5 Display apparatus 7 Input device 10 Presentation support apparatus 11 Input / output I / F part 13 Memory | storage part 13a Document data 13b Extraction word data 13c Recognition word data 15 Control part 15a Extraction part 15b Recognition part 15c Determination part 15d Calculation unit 15e Display control unit

Claims (8)

A determining unit that determines a region to be highlighted in a page displayed on the screen by the display device;
A calculation unit for calculating a distance between regions where the highlight display is switched;
When the distance between the regions is equal to or greater than a predetermined threshold, the second highlight display having a higher highlight intensity than the first highlight display executed when the distance between the regions is less than the threshold. And a display control unit that applies the presentation support device.   The presentation support apparatus according to claim 1, wherein the calculation unit calculates a distance between regions on the page.   The presentation support apparatus according to claim 2, wherein the calculation unit converts a distance between areas on the page into a display distance between areas on the screen according to a screen size of the display apparatus. The calculation unit determines whether the display from the fixed point before and after the highlight display is switched from the display distance between the areas on the screen and the distance to a predetermined fixed point set in the normal direction of the display surface of the screen. Calculate the angle at which the line of sight changes,
The presentation support apparatus according to claim 3, wherein the display control unit applies the second highlight display when the angle is equal to or greater than a predetermined threshold.   2. The presentation according to claim 1, wherein the display control unit applies the first highlight display when a predetermined period elapses after the application of the second highlight display is started. Support device. A recognition unit that performs speech recognition;
A calculation unit for calculating a degree of association from the first word extracted from the region and the second word obtained as a result of the voice recognition for each region in the page being displayed on the display device; And
The said determination part determines the area | region where the said degree of association is the highest among the area | regions in the page currently displayed on the said display apparatus as an area | region made into the execution object of the said highlight. Presentation support device. Computer
A process of determining an area to be highlighted in a page displayed on the screen by the display device;
A process of calculating a distance between areas where the highlight display is switched;
When the distance between the regions is equal to or greater than a predetermined threshold, the second highlight display having a higher highlight intensity than the first highlight display executed when the distance between the regions is less than the threshold. A presentation support method, comprising: On the computer,
A process of determining an area to be highlighted in a page displayed on the screen by the display device;
A process of calculating a distance between areas where the highlight display is switched;
When the distance between the regions is equal to or greater than a predetermined threshold, the second highlight display having a higher highlight intensity than the first highlight display executed when the distance between the regions is less than the threshold. A presentation support program characterized by causing a process to be applied to be executed.

Images