JP2019124750A - Method of displaying presentation material - Google Patents

Abstract

To effectively display a presentation material according to progress of presentation.SOLUTION: Registered word information represents information associated with a registered word and a slide within a presentation material including the word. A system display the slide in the presentation material. The system analyzes voice data of spoken voice to identify a speech word, and determines whether the speech word is included in the words within the slide registered in the registered word information, referring to the registered word information. When the speech word is included in the words within the slide registered in the registered word information, the system highlights the speech word in the image of the displayed slide.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a method of displaying presentation material.

  As background art of this indication, Unexamined-Japanese-Patent No. 2004-288008 (patent document 1) is known, for example. JP-A-2004-288008 discloses an apparatus for preventing omission of explanation in a presentation. Specifically, while causing the projector to display a page based on the original data on the screen, the computer converts audio data acquired from the microphone into text data and records the text data in the speech table. Also, when the computer receives an instruction to switch the display to the next page at the input device, the computer searches the text data using the keyword corresponding to this page as the search condition, and if any keyword can not be detected, displaying Output a warning on the manuscript screen of (see summary).

Unexamined-Japanese-Patent No. 2004-288008

  However, in a presentation performed while displaying presentation material, it is desirable to effectively change the display of the presentation material in accordance with the progress of the presentation.

  A representative example of the present disclosure is a method in which a system displays presentation material, and the system stores a processor, a program executed by the processor, the presentation material, and registered word information. A storage device, wherein the registered word information indicates information about a registered word and a slide in the presentation material in which the word is included, and the method causes the processor to display the slide in the presentation material The voice data of the displayed voice is analyzed, the voice word of the voice is specified, the reference word information is referred to, and the voice word is included in the words in the slide registered in the registration word information. In the slide in which the uttered word is registered in the registered word information When included in a word, to highlight the speech words in an image of the slide being displayed includes.

  According to one aspect of the present disclosure, presentation materials can be effectively displayed as the presentation progresses.

An exemplary configuration of a presentation system is shown. An example software configuration of a presentation system is shown. The data structure example of a presentation file is shown. The structural example of word information DB is shown. 3 shows a flowchart of a word registration process. Fig. 6 shows a flowchart of display processing of presentation material according to an utterance. The flowchart of the highlighting process by expansion is shown. Fig. 6 shows a flowchart of highlighting processing by highlighting.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that the present embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. The same reference numerals are given to the same configuration in each drawing.

  The system of the present disclosure analyzes audio data of a presentation that uses the presentation material. When a word in a pre-registered presentation material is uttered, the system highlights the word in the display image of the presentation material. This makes it possible to effectively display the presentation material as the presentation progresses.

  FIG. 1 shows an exemplary configuration of a presentation system 100. As shown in FIG. The presentation system 100 is, for example, a computer system that executes a program for displaying presentation material. FIG. 1 shows an example of the hardware configuration of a presentation system 100 which is a computer system. The presentation system 100 includes a processor 110, a main storage 120, an auxiliary storage 130, and an interface (I / F) 140. These are connected to the internal bus and can communicate with each other.

  The presentation system 100 further includes a display device 150, a microphone 161, a touch input device 162, a pen input device 163, and a mouse 164. These are connected to the internal bus via the I / F 140. The display device 150 is an output device, and is, for example, an LCD display or a projector. The microphone 161, the touch input device 162, the pen input device 163, and the mouse 164 are input devices.

  The processor 110 operates in accordance with a program stored in the main storage device 120 to implement predetermined functions of the presentation system 100. The main storage device 120 is, for example, a volatile storage device, and stores a program executed by the processor 110 and data to be referred to. The auxiliary storage device 130 is, for example, a non-volatile storage device, and stores data to be loaded into the main storage device 120. The main storage device 120, the auxiliary storage device 130, and the combination thereof are storage devices.

  FIG. 2 shows an example of the software configuration of the presentation system 100. The main storage device 120 stores a speech recognition program 201, a presentation program 202, a speech focusing program 203, and a word registration program 204. These programs are loaded to the main storage 120 from, for example, the auxiliary storage 130 or from an external device (including a device on a network) via the I / F 140.

  As described above, the processor 110 operates as a specific functional unit in accordance with the above program. Specifically, the processor 110 functions as a voice recognition unit, a presentation unit, a voice focusing unit, and a word registration unit by operating according to the above program. At least a part of these functions may be implemented by logic circuits other than the processor 110.

  The main storage 120 further stores a basic dictionary 221, a word information DB 222 (registered word information), and a presentation file 224 (presentation material). Basic Dictionary 221 and Presentation File Before the start of presentation, the presentation system 100 is pre-installed. The basic dictionary 221 and the presentation file 224 are loaded to the main storage 120 from, for example, the auxiliary storage 130 or from an external device (including a device on a network) via the I / F 140.

  As will be described later, the word information DB 222 is generated from the presentation file 224 by the word registration program 204. The word registration program 204 may store the generated word information DB 222 in the auxiliary storage device 130. The data stored in the auxiliary storage device 130 is loaded into the main storage device 120 and used by the processor 110. The auxiliary storage device 130 may be omitted.

  The presentation program 202 displays the presentation file 224 on the display device 150 in response to a user operation. In the example of FIG. 2, the mouse 164 is operated by the user, but other input devices may be used.

  The voice recognition program 201 analyzes voice data input from the microphone 161 and converts it into text data. The speech recognition program 201 refers to the basic dictionary 221 and the word information DB 222 in the analysis of speech data. The basic dictionary 302 is a dictionary referred to for speech recognition of any speech, and the word information DB 222 includes a dictionary dedicated to a presentation to be executed.

  When a word registered in the word information 22 is uttered during the presentation, the voice focus program 203 instructs the presentation program 202 to highlight the word. The presentation program 202 highlights the designated word in the slide being displayed. This makes it possible to display presentation materials effectively as the presentation progresses.

  FIG. 3 shows an example of the data structure of the presentation file 224. The presentation file contains one or more slides (images). The slide is one whole image displayed on the screen of the display device 150 in the presentation material. When a plurality of slides exist, the plurality of slides are sequentially displayed on the display device 150.

  As shown in FIG. 3, for each slide, a unique slide ID is given in the presentation file 224. The presentation file 224 contains information on the size (width, height) of each slide.

  The slide contains one or more display elements (objects). The object has a position (X, Y) and a size (width, height) in the slide. Each object is given a unique object ID in the presentation file 224.

  Objects can have additional information, depending on the display content. Specifically, an object (text object) representing a character string has information on the color of the character string and the character, and the color of the character background. An object representing a table (table object) has one or more rows and columns, and each cell contains a text object.

  An object representing a graph (graph object) includes a plurality of text objects displayed in the graph and a diagram. When a text object exists in a table or graph, the text object has a table or graph object ID as a parent object ID. Table objects and graph objects may be referred to simply as tables and graphs, respectively.

  FIG. 4 shows a configuration example of the word information DB 222. The word information DB 222 includes a noun DB 401 and a table graph DB 402. The noun DB 401 stores information on nouns extracted from the presentation file 224. The table graph DB 403 stores table and graph information extracted from the presentation file 224. As described later, the table graph DB 403 stores information on a table or graph including nouns registered in the noun DB 401.

  As shown in FIG. 4, the noun DB 401 includes a noun ID column 411, a slide ID column 412, an object ID column 413, a noun column 414, a position column 415, a size column 416, a table / graph ID column 417, and a reading column. Including 418.

  The noun ID column 411 indicates a unique ID in the noun DB 401, which is assigned to each noun registered in the noun DB 401. The slide ID column 412 indicates the slide ID of the slide containing the noun. An object ID column 413 indicates an object ID of a text object including a noun.

  The noun column 414 shows registered nouns. The position column 415 indicates the position in the slide of the text object including the noun, for example, the position of the center of gravity. The size column 415 indicates the size of the text object containing the noun. The table / graph ID column 417 indicates the object ID of a table object or graph object (parent object) in which a noun is included. If the noun is not included in either the table object or the graph object, the table / graph ID column 417 indicates, for example, NULL. The reading column 418 shows the reading (sound) of the noun.

  The table graph DB 403 includes a table / graph ID column 431, a slide ID column 432, an object ID column 433, a position column 434, and a size column 435. The table / graph ID column 431 indicates a unique ID in the table graph DB 403 assigned to each of the table objects and graph objects registered in the table graph DB 403.

  The slide ID column 432 indicates the slide ID of the slide in which the table object or graph object is included. An object ID column 433 indicates an object ID of a table object or a graph object. The position column 434 indicates the position of the table or graph object in the slide, for example, the position of the center of gravity. The size column 435 shows the size of the table object or graph object.

  The processing of each program will be described below. First, word registration processing by the word registration program 204 will be described with reference to FIG. FIG. 5 shows a flowchart of the word registration process. The word registration process is started, for example, in response to a presentation start instruction from the input device, or is executed in advance before the presentation starts.

  As shown in FIG. 5, the word registration program 204 reads the presentation file 224 (S101), and acquires an unprocessed slide from the presentation file 224 (S102).

  Next, the word registration program 204 acquires a slide ID and a list of text objects from the slide (S103). The word registration program 204 acquires an unprocessed text object from the list, and acquires data of an object ID, a character string, a position, and a size from the text object (S104).

  Next, the word registration program 204 refers to the presentation file 224 and determines whether the text object is included in the table object or the graph object. If the parent object of the text object is a table object or a graph object, the text object is included in the table object or the graph object. The parent object may be an object immediately above it or an object above it.

  When the text object is included in the table object or the graph object, the word registration program 204 acquires the object ID from the presentation file 224 (S105).

  The word registration program 204 uses the basic dictionary 221 to perform morphological analysis of the character string of the text object and identify nouns in the character string. The word registration program 204 assigns a noun ID to each noun (S106). Subtypes of nouns registered in the noun DB 401 are set in advance in the word registration program 204. For example, common nouns, proper nouns, and sa-change nouns are set as subtypes of nouns registered in the noun DB 401.

  The word registration program 204 registers the information of each noun acquired from the text object in the noun DB 401 (S107). Specifically, the word registration program 204 registers the noun ID, the slide ID, the text object ID, the noun, the position of the text object, and the size of the text object in the columns 411 to 416 of the word registration program 204, respectively.

  When the text object is included in the table object or graph object, the word registration program 204 assigns a table / graph ID to the table object or graph object, and registers it in the table / graph ID column 417. The word registration program 204 converts nouns into (kana) and registers them in the reading column 418.

  When the text object is included in the table object or graph object, the word registration program 204 stores the information in the table graph DB 403 (S108). Specifically, the word registration program 204 assigns the table / graph ID assigned, the slide ID of the current slide, the object ID of the table object or graph object, and the position and size of the table object or graph object, respectively. Register in columns 431-435.

  The word registration program 204 refers to the list and determines whether all text objects of the current slide have been processed (S109). If an unprocessed text object remains (S109: NO), the word registration program 204 returns to step S104 and selects the next text object.

  If all text objects have been processed (S109: YES), the word registration program 204 determines whether all slides of the presentation file 224 have been processed (S110). If an unprocessed slide remains (S110: NO), the word registration program 204 returns to step S102 and selects the next slide. If all the slides have been processed (S110: YES), the word registration process ends.

  In the above example, nouns in the presentation file 224 are registered in the word information DB 222. Unlike this, the word registration program 204 may register words of other parts of speech. The word registration program 204 can register one or more types of part-of-speech set in advance.

  In the above example, the word registration program 204 automatically selects a registered word and registers it in the word information DB 222. Unlike this, a word registered in word information DB 222 may be selected by the user. For example, the word registration program 204 acquires the information of the word input from the input device from the presentation file 224 and registers it in the word information DB 222. Alternatively, the word registration program 204 may acquire a word of a defined part of speech from the presentation file 224, and register information of the word selected by the user from the word in the word information DB 222.

  In the following, display processing of presentation material according to the utterance in the presentation will be described. FIG. 6 shows a flowchart of display processing of presentation material according to the utterance. The flowchart mainly shows the processing of the speech recognition program 201 and the speech focusing program 203, and does not include the display processing of the presentation file 224 by the presentation program 202.

  When a presentation using the presentation file 224 by the user is started, the speech recognition program 201 executes speech recognition processing of speech data input from the microphone 161 (S201). The speech recognition program 201 analyzes speech data using the word information DB 222 and the basic dictionary 221, and sequentially generates sentences (text sentences) consisting of characters. The generated sentence (the current sentence) is stored in the main storage device 120.

  Speech recognition processing is a widely known technology, and the detailed description is omitted. The voice recognition program 201 analyzes voice data, extracts acoustic features, and selects a word having a similar acoustic feature from the words registered in the word information DB 222 or the basic dictionary 221. The speech recognition program 201 refers to the word information DB 222 prior to the basic dictionary 221.

  For example, when the homonym exists in both the word information DB 222 and the basic dictionary 302, the speech recognition program 201 selects a word in the word information DB 222. The voice recognition program 201 may search for a corresponding word in the basic dictionary 302 when it can not find a word corresponding to (determined to be) voice data in the word information DB 222.

  The speech focusing program 203 executes morphological analysis of the character string (sentence) generated by the speech recognition program 201 using the basic dictionary 302, and selects a noun of a preset subtype (S203). The audio focus program 203 acquires the slide ID of the slide currently displayed from the presentation program 202 (S24).

  The speech focus program 203 refers to the noun DB 401 and acquires a noun ID having a noun acquired from the current slide ID and speech recognition result. If the current sentence does not include the noun registered in the noun DB 401 (S205: NO), the speech focus program 203 returns to step S202.

  If the current sentence includes a noun registered in the noun DB 401 (S 205: YES), the voice focus program 203 registers all entries included in the current sentence from the position column 415 and the size column 416 of the noun DB 401. Information on the position and size of the noun is acquired (S206).

  When the noun is included in the table object or the graph object, the speech focusing program 203 acquires information on the position and size of the table object or the graph object from the table graph DB 403 (S207). Specifically, in the noun DB 401, when the record of the noun has a value in the table / graph ID column 417, the audio focus program 203 determines the position column 434 and the size of the record of the table / graph ID in the table graph DB 403. A value is obtained from column 435.

  Next, the speech focusing program 203 executes highlighting processing of the identified noun (S208). Details of the highlighting process will be described later. When the voice focus program 203 and the voice recognition program 201 receive a presentation end instruction from the presentation program 202 (S210: YES), the flow ends. If the presentation continues (S201: NO), the audio focusing program 203 returns to step S202.

  For example, upon receiving an input indicating the end of presentation, the presentation program 202 notifies the speech recognition program 201 and the speech focus program 203 of the end of presentation by the presentation file 224, and ends the present presentation.

  The process of highlighting the registered noun will be described below. There are several ways to highlight registered nouns. In the following, a method of highlighting by expanding a noun and a method of highlighting by highlighting a noun will be described.

  FIG. 7A shows a flowchart of highlight display processing by enlargement. The example described below defines a rectangle that contains all the objects to be enlarged, and moves the rectangle to the center of the display area and expands it.

  The speech focusing program 203 calculates a rectangle A1 including all of the text object including the noun indicated by the acquired noun ID and the table / graph object indicated by the table / graph ID (S231). Specifically, the audio focus program 203 acquires, from the word information DB 222, information on the position and size of each of the text object, the table object, and the graph object to be enlarged, and includes all the occupied areas in those slides. Calculate the rectangle A1.

The voice focusing program 203 calculates a rectangle A2 obtained by multiplying the rectangle A1 by the specified enlargement ratio z1. Further, the audio focusing program 203 obtains the ratio of the rectangle A2 to the vertical and horizontal of the current slide, and obtains the larger rmax. rmax is expressed by the following equation.
rmax = max (A2 width / slide width, A2 height / slide height)

  If rmax does not exceed 1, the audio focus program 203 determines the enlargement ratio z2 as z1. If rmax exceeds 1, the audio focusing program 203 determines the enlargement ratio z2 as z1 / rmax. As a result, it is possible to avoid that a part of the target image is not displayed after enlargement, and it is possible to expand at the maximum displayable ratio when enlargement can not be performed at a prescribed ratio.

The audio focusing program 203 calculates the center (cx, cy) of the rectangle A1 according to the following equation.
cx = position of rectangle A1 X width of rectangle A1 / 2
cy = position Y of rectangle A1 height of rectangle A / 2

  The audio focus program 203 instructs the presentation program 202 to enlarge and move the current slide by specifying the center coordinates (cx, cy) and the enlargement ratio z 2 (S 235). The presentation program 202 generates an enlarged image so that the designated center coordinate is at the center of the slide image displayed in the display area of the display device 150.

  The outer shape of the slide image to be displayed (the display area in the display device 150) is maintained, and a part of the enlarged slide is displayed in the outer shape (within the display area). The center in the display area coincides with the center coordinates (cx, cy). The audio focusing program 203 may move the center of the rectangle A1 to a position different from the slide center when approaching the stop of the slide.

  The voice focus program 203 counts the elapsed time since the enlargement display instruction is issued, and instructs the presentation program 202 to cancel the enlargement display when the elapsed time reaches a prescribed number of seconds (S236). As a result, the highlighting process S208 based on enlargement ends.

  Next, highlight display processing S208 by highlighting will be described. FIG. 7B shows a flowchart of highlight display processing by highlighting. The speech focus program 203 refers to the object ID column 413 of the noun DB 401 for all noun IDs found from the noun DB 401, and creates a list of text object IDs (S251).

  The audio focus program 203 executes steps S252 to S255 for each of the text object IDs in the list. First, the audio focus program 203 takes out one unselected text object ID from the list (S 252). The audio focus program 203 specifies the text object ID from the presentation program 202, and acquires the text object of the presentation file 224 (S253).

  The speech focus program 203 compares the acquired character string of the text object with the character string of the noun of the speech recognition result, and specifies the start position of the noun in the text object (S254). The start position can be represented by the number of characters from the start of the text object.

  The voice focus program 203 designates the text object ID, the start position of the noun, and the number of characters of the noun for the presentation program 202, and sets the background color of the designated character string (noun) to the designated color in the text object. Command (object attribute change command) (S255).

  The presentation program 202 changes the background of the designated string to the designated color. When one text object includes a plurality of registered nouns, the audio focus program 203 instructs to change the background color of each registered noun.

  The audio focus program 203 counts the elapsed time after instructing the background color setting, and instructs the presentation program 202 to return the background color set when the elapsed time reaches a prescribed number of seconds (S 256). . As a result, the highlight display processing by highlight S208 ends.

  The method of emphasizing nouns may be different from the above two methods. For example, nouns can be emphasized by underlining or changing them to bold. The speech focusing program 203 may select the emphasizing method according to the position of the noun. For example, when the noun is included in the table / graph object, the speech focusing program 203 executes highlighting by the above-mentioned enlargement. If the noun is not included in the table / graph object, the audio focus program 203 performs highlighting with the above-mentioned highlight.

  The word information DB 222 may specify a method of emphasizing nouns. The speech focusing program 203 highlights each noun according to a designated method (aspect). The highlighting method (aspect) may be set by the user in the word information DB 222, for example.

  The present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. In addition, with respect to a part of the configuration of each embodiment, it is possible to add, delete, and replace other configurations.

  In addition, each of the above configurations, functions, processing units, etc. may be realized by hardware, for example, by designing part or all of them with an integrated circuit. Further, each configuration, function, etc. described above may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as a program, a table, and a file for realizing each function can be stored in a memory, a hard disk, a recording device such as a solid state drive (SSD), or a recording medium such as an IC card or an SD card. Further, control lines and information lines indicate what is considered to be necessary for the description, and not all control lines and information lines in the product are necessarily shown. In practice, almost all the configurations may be considered to be mutually connected.

DESCRIPTION OF SYMBOLS 100 Presentation system, 110 processor, 120 main memory, 130 auxiliary memory, 150 display, 161 microphone, 162 touch input device, 163 pen input device, 164 mouse, 201 speech recognition program, 202 presentation program, 203 speech focus program , 204 word registration program, 221 basic dictionary, 222 word information DB, 224 presentation file

Claims (8)

The system is a way to display presentation material,
The system
A processor,
A program executed by the processor, the presentation material, and a storage device storing registration word information;
The registration word information indicates information on a registered word and a slide in the presentation material in which the word is included,
In the method, the processor displays slides in the presentation material;
Analyze the voice data of the uttered voice to identify a uttered word;
With reference to the registered word information, it is determined whether the uttered word is included in a word in the slide registered in the registered word information;
Highlighting the spoken word in the image of the slide being displayed when the spoken word is included in a word in the slide that is registered in the registered word information. The method according to claim 1, wherein
The registered word information is
Indicates whether each of the registered words is a word in a table or graph,
The method comprises:
With reference to the registered word information, it is determined whether the spoken word is included in a table or a graph in the slide,
When the uttered word is included in the table or graph, the slide is expanded in the display area to the extent that the entire table or graph is included in the display area on the display device of the slide. The way it involves. The method according to claim 2, wherein
Said processor
Determining an image area to be enlargedly displayed on the slide so as to include the utterance word and a table or graph including the utterance word;
Displaying an enlarged image of the slide in the display area such that the center of the display area and the center of the image area coincide. The method according to claim 1, wherein
Said processor
Analyze the presentation material,
Select a predetermined part of speech word from the presentation material,
Registering the selected word in the registration word information. A program for causing a computer system to display presentation material, the computer system comprising:
Display slides in the presentation material,
Analyze the voice data of the uttered voice to identify a uttered word;
Refer to a registered word, a slide in the presentation material including the word, a position of the word on the slide, and registered word information indicating information.
It is determined whether the uttered word is included in the word in the slide registered in the registered word information,
The program is executed to emphasize the spoken word in the slide being displayed, when the spoken word is included in a word in the slide registered in the registered word information. The program according to claim 5, wherein
The registered word information indicates whether each of the registered words is a word in a table or a graph;
The program is executed by the computer system
With reference to the registered word information, it is determined whether the spoken word is included in a table or a graph in the slide,
When the uttered word is included in the table or graph, the slide is expanded in the display area to the extent that the entire table or graph is included in the display area on the display device of the slide. A program that lets you do things. The program according to claim 6, wherein the computer system
Determining an image area to be enlargedly displayed on the slide so as to include the utterance word and a table or graph including the utterance word;
A program that displays an enlarged image of the slide in the display area such that the center of the display area coincides with the center of the image area. The program according to claim 5, wherein the computer system
Analyze the presentation material,
Select a predetermined part of speech word from the presentation material,
A program that causes the selected word to be registered in the registered word information.

Images