JP2006208483A - Device, method, and program for assisting survey of interesting matter of listener, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily make a survey of how much a listener is actually interested in matters that an explainer side wants the listener to be interested in by applying speech recognition and voiceprint authentication after a new article or new system is somewhat explained to the listener. <P>SOLUTION: An inputted keyword is registered. The registered keyword is extracted through speech recognition from voice data generated by recording utterances of the listener. A GUI picture including a figure 41 representing relative time positions from the start of sound recording of the voice data and marks 42 to 44 for identifying keyword-extracted parts of the voice data by keywords in time series on the figure 41 is displayed by a display means. The voice data are reproduced through operation to specify a production position on the GUI picture. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an apparatus, a method, and the like for supporting an operation of investigating the degree of interest of a listener after giving some explanation such as explanation of a new product in front of the listener.

  When companies develop new products and new systems, it is often the case that general consumers and the media are invited to hold presentations. In such a presentation, it is important not only to fully explain the new product and the new system itself, but also to conduct marketing based on the questions and reactions of the listener (the actual concept of the new product and the new system). It is important to investigate the degree of interest of the listener and use it for the sales strategy of the new product or new system, or for the development of the next product or system).

  However, in reality, due to the limited number of staff on the organizer's side, the presentation venues often concentrate on explanations and are often placed in a situation where they cannot pay close attention to the listener's reaction. . In such a situation, only the voices of loud listeners and listeners with high actions remain in the impression, and when collected, only extremely biased survey results are often obtained.

  Also, if you record the audience's remarks at the presentation venue and later conduct marketing based on the recorded content, you will also need to create a tape or organize information from documented data. It takes a lot of time and effort.

  Here, as a system that can be used by the presenter at the presentation venue, information on the interests of a specific inviter (important customer) is sent from the center to the event venue, and this information is utilized by the presenter. In the past, there has been proposed a system that can explain (so as to stimulate purchase intentions of important customers) (for example, see Patent Document 1).

  However, in this system, the listener's interests have been investigated, and the explanation content is determined accordingly (that is, the interests are investigated first and the explanation is later). Therefore, it is not useful for investigating the listener's interest from the response at the time of explanation.

  On the other hand, voice recognition and voiceprint authentication exist as techniques for processing voice data. Speech recognition is a technology for recognizing a specific word from speech, and is mainly used for command input by voice (operation of a specific device, etc.) (unpublished at the time of filing this patent application, Patent application No. 2003-286255).

  Voiceprint authentication is a technique for identifying a speaker from voice, and is mainly used as a kind of biometric authentication (personal authentication based on characteristics of a human body) (not disclosed at the time of filing this patent application). However, for example, the patent application of Japanese Patent Application No. 2004-143952).

JP 2002-109200 A (paragraph numbers 0006 to 9, FIGS. 1 and 2)

  However, no technology has yet been proposed for applying such speech recognition and voiceprint authentication to research such as marketing.

  In view of the above-mentioned points, the present invention applies voice recognition and voiceprint authentication, after the explanation is given in front of the listener, such as explanation of a new product, and the presenter is interested in the listener. The task was to make it easier to investigate the actual level of interest of the listener about the desired matter.

  In order to solve this problem, the investigation support device for the interests of the listener according to the present invention uses the registration means for registering the input keyword and the speech data from which the listener's remarks are recorded. Extraction means for extracting a keyword registered by the registration means, a figure expressing a relative time position from the start of recording of the voice data, and a portion of the voice data from which the keyword has been extracted by the extraction means is classified by keyword. Based on the display processing means stage for displaying on the display means a GUI screen provided with marks to be identified in time series on the figure, and the operation of specifying the playback position on the figure on the GUI screen, And a reproduction processing means for starting reproduction of the audio data from the designated reproduction position.

  In addition, the method for supporting the investigation of the interests of the listener according to the present invention includes a first step of registering the input keyword, and the first step by voice recognition from the voice data recording the listener's speech. The second step of extracting the keyword registered in step 1, the figure representing the relative time position from the start of recording of the voice data, and the portion of the voice data from which the keyword was extracted in the second step are classified by keyword. Based on the third step of displaying on the display means a GUI screen provided with marks for time-series identification on this graphic, and the operation of specifying the playback position on this graphic on this GUI screen, And a fourth step of starting reproduction of the audio data from the designated reproduction position.

  Also, the program according to the present invention includes a computer for registering an input keyword, and an extraction unit for extracting a keyword registered by the registration unit from voice data recording a listener's speech by voice recognition. , A graphic representing the relative time position from the start of recording of the voice data, and a mark for identifying the part of the voice data from which the keyword has been extracted by the extracting means in time series on the graphic for each keyword. Display processing means for displaying a GUI screen to be displayed on the display means, and on the basis of the operation of designating the reproduction position on the graphic on the GUI screen, reproduction of the audio data is started from the designated reproduction position. It functions as a reproduction processing means.

  Further, the computer-readable recording medium according to the present invention includes a registration means for registering an input keyword to a computer, a keyword registered by the registration means by voice recognition from voice data recording a listener's speech. Extraction means for extracting the voice data, a figure expressing the relative time position from the start of recording of the voice data, and a part of the voice data in which the keyword is extracted by the extraction means is identified in time series on the figure for each keyword. Display processing means for displaying a GUI screen provided with a mark to be displayed on the display means, and the audio data from the designated reproduction position based on the operation of designating the reproduction position on the graphic on the GUI screen. It is characterized by recording a program for functioning as a playback processing means for starting playback of .

  In these inventions, when any explanation is given in front of the listener, the listener's questions and so forth are recorded, and the explanation side of the explanation contents wants the listener to be interested. Alternatively, if a plurality of items are input as keywords, the keywords are extracted by voice recognition from voice data in which the listener's speech is recorded.

  Then, there is a graphic representing the relative time position from the start of recording of the audio data, and a mark for identifying the part of the audio data from which the keyword has been extracted by the extracting means in time series on the graphic for each keyword. The provided GUI screen is displayed on the display means.

  The user (for example, the presenter himself / herself or another investigator) determines whether the presenter wants the listener to be interested from the number of marks for each keyword provided on the GUI screen graphic. The number of utterances can be grasped intuitively.

  Further, from the position of each mark on the figure, it is possible to intuitively confirm the portion of the voice data in which the listener's speech is recorded, where the listener has spoken the keyword. When the operation of specifying the position of this mark as a guide (for example, a position slightly before this mark) is performed as the playback position, the audio data recording the listener's speech is expressed by the explainer's interest to the listener. It starts to play from the part of the remarks about the matter you want to have.

  Therefore, the user can listen easily and quickly to the content of the listener's remarks on the matter that the explainer wants the listener to be interested in.

  Thereby, after giving some explanation in front of the listener, it is possible to easily investigate the actual degree of interest of the listener about the item that the explainer wants the listener to be interested in.

  As an example, this GUI screen is further provided with an icon indicating the current reproduction position of the audio data on this graphic and an operation button for designating a desired range on this graphic. It is preferable to display on the display means information obtained by summing the lengths of the respective ranges for each mark included in the range.

  While the audio data is being played back using the mark position as a guide, the user can use this icon to start and end the utterance about the item that the explainer wants the listener to be interested in. 2 positions) can be confirmed.

  For each mark, if the range from the start position to the end position of this statement is specified with this operation button, the total length of this range (this The total length corresponds to the listener's total speaking time length for that matter).

  Thereby, the user can grasp | ascertain intuitively the length of the listener's speech time about the matter for every matter which the instructor wants a listener to be interested in. Therefore, it becomes possible to investigate the degree of interest of the listener in more detail.

  Further, as an example, for the portion of the voice data from which the keyword is extracted, the speaker is specified using the voiceprint data of the listener registered in advance, and the attribute information that classifies the listener for each attribute is used. It is preferable to associate the identified speaker attribute with the keyword and display the result of the association on the display means.

  Accordingly, the user can intuitively understand what attribute the listener has a high degree of interest in the matter for each matter that the explainer wants the listener to be interested in. Therefore, it becomes possible to investigate the degree of interest of the listener in more detail.

  Further, as an example, the keyword extraction processing from the voice data recording the listener's speech is performed by shifting a predetermined predetermined amount of voice data while shifting the position of the voice data to be captured from the recording means recording the voice data. Are sequentially compared and the acoustic characteristics of each of the acquired predetermined amount of voice data and the voice data of the registered keyword are compared and collated. As a result of this comparison and collation, the voice data of this keyword is matched. It is preferable to carry out by extracting a predetermined amount of voice data as a keyword.

  As a result, a predetermined amount of audio data sequentially captured while shifting the position from the audio data (keyword extraction target audio data) recorded from the listener's speech, and the acoustic characteristics of the audio data of the registered keyword audio data Keywords are extracted based on the result of comparing and collating. That is, the keywords are extracted only by the result of comparison and collation between the voice data without converting the voice data into text data.

  Therefore, for example, compared to the case where the voice data in which the listener's speech is recorded and the voice data of the keyword are both converted into text data by voice recognition processing and the text data are compared and collated. Keyword can be extracted with high accuracy. Further, unlike the case where the entire voice data in which the listener's speech is recorded is converted to text data, the burden of the keyword extraction process is reduced even in the case of voice data over a long period of time.

  According to the present invention, after giving some explanation in front of the listener, the explainer side can intuitively grasp the number of times the listener speaks about matters that the listener wants to be interested in, and The presenter can easily and quickly re-listen to the listener's remarks about matters that the listener wants to be interested in.

  Therefore, after giving some explanation in front of the listener, it is possible to easily investigate the degree of interest of the actual listener on the matter that the explainer wants the listener to be interested in.

  In addition, for each item that the explainer wants the listener to be interested in, the length of the listener's speaking time for that item can be intuitively grasped, so the degree of interest of the listener can be further detailed. The effect that it can be investigated is acquired.

  In addition, for each item that the explainer wants the listener to be interested in, it is possible to intuitively understand what kind of attribute the listener is highly interested in that item. The effect of being able to investigate in more detail is obtained.

  Further, it is possible to extract a keyword with high accuracy from voice data in which a listener's speech is recorded, and to reduce the burden of keyword extraction processing even in the case of long-time voice data.

  Hereinafter, an example in which the present invention is applied to a case where a listener's remarks at a new product announcement venue is recorded and marketing is performed based on the recorded contents will be specifically described with reference to the drawings.

  FIG. 1 shows a system configuration example for carrying out the present invention. From the start to the end of the explanation of the product, the explanation staff A on the organizer's side, at the new product announcement site, will give questions such as questions from the listeners B1 to Bn (invited general consumers, media representatives, etc.) Recording is performed on the personal computer 2 via the sound collector 1.

  In order to record only the statements of the listeners B1 to Bn as much as possible (so that the voice of the staff A etc. is not recorded), for example, a microphone having a strong directivity is used as the sound collector 1, and the microphone is used as a listener. The listener who asks questions is directed near the microphone while facing the B1 to Bn side. As another example, a headset or a pin microphone may be prepared for each listener B1 to Bn as the sound collecting device 1.

  Before and after the explanation, the staff A obtains the consent of the listeners B1 to Bn and records the voices of the listeners B1 to Bn to the personal computer 2 via the sound collector 1 for voiceprint authentication. At the same time, the profiles (attributes such as name, sex, and age group) of each listener B1 to Bn are confirmed.

  FIG. 2 is a block diagram showing the configuration of the personal computer 2 in a range necessary for explaining the present invention. A CPU 11, a display device (for example, a liquid crystal display) 12, an external storage device (HDD) 13, a memory 14, a sound interface (sound card) 15, a pointing device (for example, a mouse) 16, and a keyboard 17 are connected to a bus. 18 is connected.

  The external storage device 13 records various device drivers for peripheral devices (sound drivers, drivers for pointing devices, etc.) and recording software (sound data from a microphone) on the hard disk of the computer, similar to a normal personal computer. Software) is installed.

  Various device drivers are loaded onto the memory 14 when the personal computer 2 is powered on. Audio data from the sound collecting device 1 is input into the personal computer 2 from the sound interface 15 by a sound driver.

  The recording software is activated by a user operation and loaded onto the memory 14. When the explainer A records the voices of the listeners B1 to Bn and the voiceprint authentication voice, the voice data input from the sound interface 15 into the personal computer 2 is externally stored in the WAVE file format by the recording software. It is recorded in the device 13.

  The external storage device 13 is further installed with a keyword extraction application, a voiceprint authentication engine, and a marketing information extraction application. These programs may be provided as a recording medium such as a CD-ROM, or may be downloaded from a website.

  The keyword extraction application is application software for extracting keywords from a WAVE file. The keyword extraction application is activated by the marketing information extraction application and loaded onto the memory 14. Designation of a WAVE file as a keyword extraction target and registration of a keyword are also performed by processing of a marketing information extraction application.

  FIG. 3 is a flowchart showing the processing contents of the keyword extraction application. First, character data of each registered keyword is converted into voice data, and data (voice model) in which features are extracted from the voice data by acoustic analysis (spectrum analysis) is generated. This acoustic analysis is the same as that performed as a process in a general speech recognition engine (step S1).

  Subsequently, a recognition window having a predetermined time length (for example, 1 second), which is a time length usually required for speaking a keyword that is the basis of the voice model, is set (step S2).

  Subsequently, continuous audio data of the amount corresponding to the recognition window is taken from the head portion of the data area of the keyword-extracted WAVE file (step S3).

  FIG. 4A is a diagram conceptually showing the state of taking in the audio data in step S3. A recognition window W is provided at the beginning of the audio data in the data area of the WAVE file, and continuous audio data for a time length N is captured.

  As shown in FIG. 3, following step S3, features are extracted from the captured voice data by acoustic analysis (acoustic analysis performed by the voice recognition engine as described above) (step S4). Then, the extracted feature is compared with the voice model of each search keyword (step S5), and it is determined whether or not it matches the voice data of any search keyword (step S6).

  If yes, the acquired voice data is extracted as a keyword (step S7). Then, information indicating the position of the voice data extracted as the keyword in the WAVE file (address from which sampling to what sampling is the voice data) is registered (step S8). Then, the process proceeds to step S9. On the other hand, if no in step S6, the process proceeds directly to step S9.

  In step S9, it is determined whether or not the recognition window has finished capturing the last audio data in the data area of the WAVE file. If no, the capture position is shifted toward the end of the data area by a predetermined time length (which is shorter than the recognition window, for example 0.2 seconds) than when the capture was performed immediately before. Audio data corresponding to the amount of the window is captured (step S10). And it returns to step S4 and repeats step S4 and subsequent steps. If the answer is yes in step S9, the process is terminated.

  FIGS. 4B to 4D are diagrams conceptually showing how audio data is captured by repeating step S10. As shown in FIG. 4A, after continuous audio data for a time length N is taken from the head portion of the audio data in the data area of the WAVE file, the position of the recognition window W is set to the time length P (time length). (Sequential time length shorter than N), continuous audio data of time length N minutes are sequentially captured until the end of the data area is reached.

  FIG. 5 is a diagram conceptually illustrating an example of a keyword extraction result by the keyword extraction application. Of the audio data in the data area of the WAVE file, continuous audio data D1 for a time length N at a certain position is extracted as a keyword “small and light”, for example, and is continuous for a time length N at another position. The audio data D2 is extracted as a keyword “stylish”, for example.

  As described above, in this keyword extraction application, the acoustic characteristics between the audio data of the predetermined amount (recognition window) each sequentially fetched while shifting the position from the pre-recorded audio data and the audio data of the search keyword audio data. A keyword is extracted based on the result of comparing and collating. That is, these words are extracted as keywords based only on the result of comparison and collation between voice data without converting the voice data into text data.

  As a result, for example, keywords are extracted through two steps: a step of converting previously recorded voice data and search keyword voice data into text data by voice recognition processing, and a step of comparing and collating the text data. The keywords can be extracted with higher accuracy than in the case of doing so. Further, unlike the case where the entire voice data recorded in advance is converted into text data, the processing load is reduced even in the case of voice data recorded over a long period of time.

  The voiceprint authentication engine in the external storage device 13 of FIG. 2 performs speaker identification (voiceprint authentication processing) by voiceprint authentication and registration of the voiceprint of the speaker to be authenticated (voiceprint registration processing) as a previous stage. Application software.

The voiceprint registration process includes the following processes (1) and (2).
(1) Generation of voiceprint data The voice data of each utterance target subject to voiceprint authentication is sequentially subjected to spectrum analysis for each voice data for a predetermined voiceprint authentication unit time (for example, 3 seconds) to extract the characteristics of the voiceprint. Then, voice print data is generated. That is, the voiceprint data expresses a collection of various sounds included in the voice in a three-dimensional pattern of time, frequency, and sound intensity.

(2) Registration of voiceprint data A screen (not shown) for inputting the ID of each speaker with the keyboard 17 is displayed on the display device 14. Then, the ID of each speaker input on this screen and the voiceprint data of the speaker generated in the process (1) are registered in association with each other (stored in the external storage device 13).

The voiceprint authentication process includes the following processes (1) to (3).
(1) Generation of voiceprint data Voiceprint data is generated from the supplied voice data of the speaker in the same manner as in the voiceprint registration process.

(2) Voiceprint data collation The generated voiceprint data and voiceprint data registered in advance by the voiceprint registration process are compared and collated, and a matching score SCR corresponding to the degree to which the model feature amount approximates ( Large). As a result, the ID corresponding to the voiceprint data that most closely approximates the voiceprint data of the input voice is specified from the registered voiceprint data.

(3) Comparison between collation score and threshold value The collation score SCR calculated in the above process (2) is compared with the threshold value THD, and if the collation score SCR exceeds the threshold value THD, it is specified in the above process (2). The determined ID is determined to be valid.

  In this voiceprint authentication engine, a routine for voiceprint registration processing is started by a user process and loaded onto the memory 14, while a routine for voiceprint authentication processing is started by a marketing information extraction application and loaded onto the memory 14. As the voiceprint authentication technique, a known technique may be applied to this voiceprint authentication engine.

  The marketing information extraction application in the external storage device 13 in FIG. 2 is application software for extracting and generating marketing information from voice data in which the utterances of listeners such as the listeners B1 to Bn in FIG. 1 are recorded. . This marketing information extraction application is activated by the user's processing and loaded onto the memory 14, and first displays the menu screen on the display device 12. FIG. 6 shows this menu screen. A keyword registration button 21, a profile registration button 22, a marketing information generation button 23, and an end button 24 are provided.

  When the keyword registration button 21 is designated on the menu screen by the pointing device 16 (FIG. 2), a keyword registration screen for the user to register a desired keyword is displayed. FIG. 7 shows this keyword registration screen. As keywords 1, keywords 2, and keywords 3, there are provided input fields 31 to 33 for inputting desired keywords using the keyboard 17 (FIG. 2) and a registration button 34 for registering the keywords input to the input fields 31 to 33. ing.

  When the registration button 34 is designated by the pointing device 16 after inputting the keywords in the input fields 31, 32, and 33 on the keyword registration screen, the input keywords are registered as keywords 1, keywords 2, and keywords 3, respectively (the external storage device in FIG. 2). 13).

  When the profile registration button 22 is designated by the pointing device 16 on the menu screen of FIG. 6, a profile registration screen for registering the listener's profile is displayed. Although illustration is omitted, on this profile registration screen, an input field for inputting a profile (attribute such as name, gender, age group) and ID for each listener and an input result of the input field are displayed. A confirmation button for confirmation is provided. When a confirmation button is designated after inputting a profile and ID in the input field on this profile registration screen, the profile information of each listener is stored in the external storage device 13 in association with the ID of the listener.

  When the marketing information generation button 23 is designated by the pointing device 16 on the menu screen of FIG. 6, a marketing information generation process is executed. FIG. 8 is a flowchart showing the contents of the marketing information generation process. First, a list of WAVE files recorded in the external storage device 13 is displayed, and a file selection screen (not shown) for selecting a WAVE file as a keyword extraction target is displayed on the display device 12 (FIG. 2). (Step S11).

  Subsequently, the keyword extraction application (FIG. 3) described above is activated. Then, keywords registered on the keyword registration screen in FIG. 7 are extracted from the keyword extraction target WAVE file selected on the file selection screen in step S11 (step S12).

  Subsequently, position information of each keyword registered by the keyword extraction application (step S8 in FIG. 3) is acquired (step S13). Then, a graphic representing the relative time position from the start of recording of the voice data of the WAVE file to be extracted as a keyword, and a portion of the voice data from which the keyword is extracted by the keyword extraction application are time-sequentially displayed on this graphic by keyword. A GUI screen provided with a mark to be identified is displayed on the display device 12 (step S14).

  FIG. 9 shows a display example of this GUI screen. The relative time position from the beginning of recording to the end of recording of the audio data of the WAVE file that is the keyword extraction target is a straight line segment with the start of recording (the beginning of data) as the left end and the end of recording (the end of data) as the right end. 41. Then, on this line segment 41, the portion of the voice data from which the keyword is extracted is a mark 42 that is a mark having a different shape for each keyword registered as Keyword1, Keyword2, Keyword3 on the keyword registration screen of FIG. (Black circles), 43 (black triangles), and 44 (black squares) are identified and displayed in time series.

  In FIG. 9, the characters “Keyword 1”, “Keyword 2”, and “Keyword 3” are used, but actually the registered keywords themselves (for example, “small and light”, “stylish”, etc.) are displayed. . (The same applies to FIGS. 12, 14, and 15 below.)

  The GUI screen is provided with a Start button 45, a Stop button 46, a Remove button 47, a profile association button 48, and an end button 49.

  As shown in FIG. 8, following step S14, WAVE file playback and marketing information generation are performed based on operations on the GUI screen (except for specifying the profile association button 48 and end button 49) (step S15). ).

  In this step S15, when any position on the line segment 41 in FIG. 9 is designated by the pointing device 16 (FIG. 2), the data position corresponding to the designated position (for example, the mark at the left end in FIG. 9). When a position slightly to the left of 42 is specified, playback of the WAVE file to be extracted from the keyword is started from the sound data of the keyword registered as Keyword 1 (a data position slightly before the data position first extracted). Then, audio is output from the built-in speaker of the personal computer 2 (or an external speaker connected to the personal computer 2). At the same time, the reproduction position display arrow 50 is displayed in the vicinity of the line segment 41 as illustrated in FIG. 10, and the arrow 50 is moved to the right along the line segment 41 according to the change in the reproduction position.

  When the arrow 50 is designated by the pointing device 16 during the reproduction of the WAVE file, the reproduction is stopped and the movement of the arrow 50 is stopped. The arrow 50 can be dragged and dropped along the line segment 41. When the arrow 50 is designated again by the pointing device 16 at the drop destination, the voice data at the data position corresponding to the drop destination is used. The reproduction of the WAVE file is started and the arrow 50 is moved rightward from the drop destination.

  As illustrated in FIG. 11, the start button 45 is designated by the pointing device 16 with the arrow 50 stopped at the point P1 on the left side of any of the marks 42 to 44, and then the arrow 50 When the Stop button 46 is designated by the pointing device 16 while stopped at the right point P2, the range from the point P1 to the point P2 on the line segment 41 (the range including the mark) is identified and displayed in the frame 51. To do. Then, the audio data in the data range corresponding to the frame 51 is saved as another WAVE file.

  Further, when the remove button 47 is designated by the pointing device 16 while the arrow 50 is stopped at any of the marks 42 to 44, the mark is deleted (a frame 51 is displayed in a range including the mark). If it is, the frame 51 is also deleted). FIG. 11 illustrates the state after the second mark 43 from the left end, the second mark 44 from the right end, and the right end mark 44 among the marks 42 to 44 shown in FIG. 9 are erased. Yes. The Remove button 47 is a button to be operated when the voice data before and after the data position from which the keyword is extracted is reproduced but the keyword is not spoken.

  Further, each time the Start button 45 and the Stop button 46 are operated to increase the number of new frames 51, the length of each frame 51 on the line segment 41 is indicated by marks 42 to 44 included in the frame 51. Total by shape (ie by keyword). Then, the total value of the length of the frame 51 for each keyword is displayed as a percentage of the total length of the line segment 41 next to the characters of Keyword 1, Keyword 2, and Keyword 3 on the GUI screen as illustrated in FIG.

  When the profile association button 48 is designated by the pointing device 16, as shown in steps S16 and S17 in FIG. 8, the voice print authentication routine of the voice print authentication engine described above is activated. Then, the audio data in the data range corresponding to each frame 51 on the GUI screen of FIG. 9 is reproduced, and the audio data is subjected to voiceprint authentication.

  Subsequently, the ID of the speaker identified by voiceprint authentication (information indicating that it is unknown if a valid ID is not identified) is acquired (step S18). Then, referring to the profile information (male / gender / age) of the speaker with the ID registered on the profile registration screen described above, the keyword is associated with the profile, and the result of the association is saved (external storage). (Stored in the device 13) (step S19).

  FIG. 13 is a diagram conceptually showing how the keyword and the profile are associated with each other. The voice data in the data range of each frame 51 includes keywords registered as Keyword1, Keyword2, or Keyword3. From the voice data in the data range of these frames 51, voiceprint data is generated by voiceprint authentication processing, and the ID of the speaker is specified.

  Since the profile of the speaker is registered in association with the ID on the profile registration screen, the profile of the speaker is derived from the ID of the speaker identified by the voiceprint authentication process. Thereby, a keyword and a profile can be associated.

  As shown in FIG. 8, following step S19, the stored association result is displayed on the display device 12 (step S20).

  FIG. 14 shows an example of a display screen of the result of this association. For example, among the listeners (listeners B1 to Bn in FIG. 1), Suzuki (25-year-old woman), Sato-san (41-year-old man), etc., who spoke the keyword registered as Keyword1 ( However, one case indicates that the result of voiceprint authentication was unknown). The display screen also includes a keyword selection button 61, an age group button 62, a sex button 63, and an end button 64.

  On this display screen, when any keyword is selected with the keyword selection button 61 and the age group button 62 or the gender button 63 is designated, the age group or gender of the listener who has spoken the selected keyword. A graph showing the percentage of. FIG. 15 exemplifies a graph displayed when keyword 1 is selected by keyword selection button 61 and age group button 62 is specified (a graph showing the ratio of listeners who have made keyword 1 by age group). Yes.

  When the return button (button 71 in FIG. 15) is designated on this graph display screen, the screen returns to the screen of FIG. When the end button 64 is designated on the screen of FIG. 14 or the end button 49 is designated on the GUI screen of FIG. 9, the marketing information generation process is terminated as shown in steps S21 and S22 in FIG. Return to the menu screen of FIG. When the end button 24 is designated on the menu screen, the marketing information extraction application is ended.

  Next, in the system shown in FIG. 1, how the organizer performs marketing after the end of the new product presentation will be described.

As described above, the staff A on the organizer side performs the following operations (1) and (2) at the presentation site.
(1) During the explanation of the product, the utterances of the listeners B1 to Bn are recorded in the personal computer 2 via the sound collecting device 1.

  (2) Before and after the description, the voices of the listeners B1 to Bn are recorded in the personal computer 2 via the sound collecting device 1 for voiceprint authentication, and the profiles of the listeners B1 to Bn are confirmed.

In addition to this, the staff A or another staff (hereinafter simply referred to as a staff) performs the following operations (3) to (5).
(3) After completion of the operation (2), the voice print registration routine of the voice print authentication engine is started, and the voice print authentication voice data of each of the listeners B1 to Bn recorded in the personal computer 2 is obtained. The voice print registration process described above is executed. (However, at the time of the operation (2), this voice print registration process routine is started, and the voice print registration process is executed in real time for the voice print authentication data of the listeners B1 to Bn from the sound collecting device 1. May be.)

  (4) After completion of the above operation (2) (or simultaneously with this operation), the marketing information extraction application is started, the profile registration button 22 is designated on the menu screen of FIG. The profiles and IDs of the users B1 to Bn are registered. (The ID of each listener is the same as the ID input during the voiceprint registration process.)

  (5) Before and after the explanation, the marketing information extraction application is started, the keyword registration button 21 is designated on the menu screen of FIG. 6, and the listener is interested in the new product on the keyword registration screen of FIG. The items (here, “small and light”, “stylish”, and (easy to operate) “easy”), which are the concepts of the new product, are registered as keywords 1, keywords 2, keywords 3, respectively.

  When the above operations are completed, the staff activates the marketing information extraction application, designates the marketing information generation button 23 on the menu screen of FIG. 6, and explains the product on the file selection screen (step S11 of FIG. 8). Sometimes a WAVE file that records the statements of the listeners B1-Bn is selected. Then, keywords “small and light”, “stylish”, and “simple” are extracted from the audio data of the WAVE file by the keyword extraction application (step S12 in FIG. 8, FIGS. 3 to 5).

  Then, the location information of each keyword in the WAVE file registered by the keyword extraction application is acquired by the marketing information extraction application (step S13 in FIG. 8), and a GUI screen as shown in FIG. 9 is displayed (here) Then, the characters “Small and Light”, “Stylish”, and “Easy” appear in the “Keyword 1”, “Keyword 2”, and “Keyword 3” portions, respectively (step S14 in FIG. 8).

  From the number of marks 42 to 44 on the line segment 41 of the GUI screen, the staff actually determines the listeners B1 to Bn for the new product concepts of “small and light”, “stylish”, and “easy”. You can intuitively understand how much you are interested. That is, for example, if the number of the marks 42 is small, it is understood that the “small and light” is not so much interested, and if the number of the marks 43 is large, the “stylish” is quite interested. I understand that.

  In addition, from the positions of the marks 42 to 44 on the line segment 41, it is possible to intuitively confirm the portion where the listeners B1 to Bn uttered the keyword in the voice data recording the utterances of the listeners B1 to Bn. it can. Then, when an operation for designating the position of the marks 42 to 44 as a guide (for example, a position slightly before the marks 42 to 44) as a playback position is performed, audio data recording the listener's speech is given to the listener. It begins to be reproduced from the part of the remarks about the item you want to be interested in (concept of new product).

  Therefore, the staff can easily and quickly listen to the contents of the comments of the listeners B1 to Bn about the concept of the new product. That is, if the reproduction is started from a position slightly before the mark 44, it is possible to listen to a remark such as “This product is easier to operate and easier to use for the elderly”.

  Thereby, after the end of the presentation of the new product, it is possible to easily investigate the degree of interest of the actual listeners B1 to Bn about the concept of the new product.

  Further, as shown in FIG. 10, the current playback position is displayed by an arrow 50 during playback of audio data and when playback is stopped. Therefore, when the staff listens to the remarks of the listeners B1 to Bn about the concept of the new product, the staff starts and ends the remarks with the mark in between (for example, “This product is more than the previous one. If the content of the remark is “it is compact and lightweight and convenient to carry”, the position immediately before “this product” and the position immediately after “convenient”) are respectively confirmed from the arrows 50. Then, the arrow 50 is stopped at the start position and the Start button 45 is designated, and then the arrow 50 is stopped at the end position and the Stop button 46 is designated.

  The confirmation of the start / end positions of the content of the message and the operation of the Start button 45 and the Stop button 46 are performed for each of the marks 42 to 44 on the line segment 41.

  Then, as illustrated in FIG. 11 and FIG. 12, among the voice data in which the comments of the listeners B1 to Bn are recorded at the time of the description of the product, the range of the content of the speech about the concept of the new product is identified and displayed in the frame 51. The As shown in FIG. 12 (here, “Keyword 1”, “Keyword 2”, “Keyword 3” characters “small and light”, “stylish”, and “easy” appear), the total length of the line segment 41 is shown. The value of% of the length of the frame 51 for each keyword is displayed (this value is proportional to the total speech time length of the listener for the keyword).

  As a result, the staff can intuitively grasp the length of the speaking time of the listeners B1 to Bn for each new product concept of “small and light”, “stylish”, and “easy”. . That is, in the example of FIG. 12, it can be seen that “Keyword 1” (here, the concept of “compact and lightweight”) has been considerably interested because the total speech time of the listeners B1 to Bn is long, and “Keyword 2” ( Here, “Stylish” (concept) is not very interested because the total speaking time of the listeners B1 to Bn is short, “Keyword3” (here (operation) “easy” concept) It can be seen that the listener B1 to Bn had no interest because the talk time of the listeners B1 to Bn was zero.

  Next, the staff designates the profile association button 48 on this GUI screen. Then, as illustrated in FIG. 14 and FIG. 15 (here, the characters “Small and Light”, “Stylish”, and “Easy” appear in the “Keyword 1”, “Keyword 2”, and “Keyword 3” parts, respectively) A result of associating each concept with the profile of the listener who has spoken about the concept among the listeners B1 to Bn is displayed.

  As a result, the staff can intuitively understand what attributes the listener is highly interested in for each new product concept of “small and light”, “stylish”, and “easy”. . That is, in the example of FIG. 15, it can be seen that “Keyword 1” (here, the concept of “compact and lightweight”) was particularly interested in listeners in their 40s.

  As described above, the staff can intuitively grasp the number of remarks of the listeners B1 to Bn about each concept of the new product after explaining the new product in front of the listeners B1 to Bn. In addition, it is possible to easily and quickly listen to the contents of the comments of the listeners B1 to Bn about the concept of the new product.

  Accordingly, after explaining the new product in front of the listeners B1 to Bn, it is possible to easily investigate the degree of interest of the actual listeners B1 to Bn about the concept of the new product.

  In addition, for each new product concept, it is possible to intuitively grasp the length of the speaking time of the listeners B1 to Bn regarding the concept. Therefore, the degree of interest of the listeners B1 to Bn regarding the concept of the new product can be investigated in more detail.

  In addition, for each concept of a new product, it is possible to intuitively understand what kind of attribute (age group or gender) of listeners B1 to Bn is highly interested in the concept. Therefore, the degree of interest of the listeners B1 to Bn regarding the concept of the new product can be investigated in more detail.

  In the above example, as shown in FIG. 9, the relative time position from the start of recording of the voice data in which the listener's speech is recorded is expressed by a straight line segment 41. However, this relative time position may be expressed by a figure having a shape other than a straight line segment (for example, a circular figure like a dial of an analog clock).

  In the above example, the present invention is applied to conduct marketing after explaining a new product. However, as another example, the present invention may be applied to investigate the degree of actual student interest in matters that the teacher wants the student to be interested in after the teacher conducts a class in front of the student. Good. More generally, after giving some explanation in front of the listener, the present invention should be applied to any case where the presenter needs to investigate the actual degree of interest of the listener about what the listener wants to be interested in. May apply.

  In the above example, the recording software, the keyword extraction application, the voiceprint authentication engine, and the marketing information extraction application are separate software, but one piece of software having all these software functions is created to create a personal computer. You may install it on

  In the above example, a personal computer in which recording software, a keyword extraction application, a voice print authentication engine, and a marketing information extraction application are installed is provided. However, as another example, a device having a dedicated processor for executing firmware having the same processing contents as the recording software, the keyword extraction application, the voiceprint authentication engine, and the marketing information extraction application may be provided instead of the personal computer. Good.

It is a figure which shows the example of a system configuration for implementing this invention. It is a block diagram which shows the structure of the personal computer of FIG. It is a flowchart which shows the processing content of a keyword extraction application. It is a figure which shows notionally the mode of taking in audio | voice data in the process of FIG. It is a figure which illustrates the extraction result of the keyword in the process of FIG. It is a figure which shows the menu screen of a marketing information extraction application. It is a figure which shows a keyword registration screen. It is a flowchart which shows the processing content of a marketing information generation process. It is a figure which shows the GUI screen displayed by a marketing information generation process. FIG. 10 is a diagram showing one display state of the GUI screen of FIG. 9. FIG. 10 is a diagram showing one display state of the GUI screen of FIG. 9. FIG. 10 is a diagram showing one display state of the GUI screen of FIG. 9. It is a figure which shows notionally the mode of the correlation of a keyword and a profile. It is a figure which illustrates the display screen of the result of correlation with a keyword and a profile. It is a figure which illustrates the display screen of the result of correlation with a keyword and a profile.

Explanation of symbols

  1 sound collecting device, 2 personal computer, 11 CPU, 12 display device, 13 external storage device, 14 memory, 15 sound interface, 16 pointing device, 17 keyboard, 18 bus

Claims (7)

A registration means for registering the input keyword;
An extraction means for extracting a keyword registered by the registration means by voice recognition from voice data recording a listener's speech;
A graphic representing a relative time position from the start of recording of the audio data and a mark for identifying a portion of the audio data, from which the keyword has been extracted by the extracting means, on the graphic in time series are provided for each keyword. Display processing means for displaying a GUI screen on the display means;
Reproduction processing means for starting reproduction of the audio data from the designated reproduction position based on an operation of designating a reproduction position on the graphic on the GUI screen. Survey support device for interested parties. In the investigation support device for the interests of the listener according to claim 1,
The display processing means further includes an icon indicating the current reproduction position of the audio data on the graphic and an operation button for designating a desired range on the graphic on the GUI screen. An apparatus for assisting investigation of a matter of interest of a listener, characterized in that information indicating the total length of each designated range for each mark included in the range is displayed on the display means. In the investigation support device for the interests of the listener according to claim 1,
Voiceprint authentication means for identifying a speaker using the voiceprint data of the listener registered in advance for a portion of the voice data from which the keyword has been extracted by the extraction means;
Using attribute information that classifies the listener for each attribute, and further comprising processing means for associating the attributes of the speaker identified by the voiceprint authentication means with the keywords,
The display processing means causes the display means to display a result of associating the attribute with the keyword by the processing means. In the investigation support device for the interests of the listener according to claim 1,
The extraction means includes
Capture means for sequentially capturing a predetermined amount of sound data while shifting the position of the sound data to be captured from the recording means for recording the sound data;
A comparison / collation unit that compares and collates acoustic features of each of the predetermined amount of voice data captured by the capture unit and the keyword voice data registered by the registration unit; An investigation supporting apparatus for a matter of interest of a listener, wherein the predetermined amount of voice data that matches the voice data of the keyword as a result of comparison and collation is extracted as a keyword. A first step of registering the entered keyword;
A second step of extracting the keyword registered in the first step by voice recognition from voice data recording a listener's speech;
A graphic representing a relative time position from the start of recording of the audio data and a mark for identifying a portion of the audio data from which the keyword has been extracted in the second step in time series on the graphic for each keyword are provided. A third step of causing the display means to display a GUI screen to be displayed;
And a fourth step of starting reproduction of the audio data from the designated reproduction position based on an operation of designating a reproduction position on the graphic on the GUI screen. Support method for investigating the interests of employees. Computer
Registration means for registering the entered keywords,
Extraction means for extracting a keyword registered by the registration means by voice recognition from voice data recording a listener's speech;
A graphic representing a relative time position from the start of recording of the audio data and a mark for identifying a portion of the audio data, from which the keyword has been extracted by the extracting means, on the graphic in time series are provided for each keyword. Display processing means for displaying a GUI screen on the display means;
A reproduction processing means for starting reproduction of the audio data from the designated reproduction position based on an operation of designating a reproduction position on the graphic on the GUI screen;
Program to function as. Computer
Registration means for registering the entered keywords,
Extraction means for extracting a keyword registered by the registration means by voice recognition from voice data recording a listener's speech;
A graphic representing a relative time position from the start of recording of the audio data and a mark for identifying a portion of the audio data, from which the keyword has been extracted by the extracting means, on the graphic in time series are provided for each keyword. Display processing means for displaying a GUI screen on the display means;
A reproduction processing means for starting reproduction of the audio data from the designated reproduction position based on an operation of designating a reproduction position on the graphic on the GUI screen;
A computer-readable recording medium in which a program for functioning as a computer is recorded.

Images