JP2008102538A - Storage/reproduction device and control method of storing/reproducing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly find out a target part in voice data to be processed and use it, without causing troubles to user. <P>SOLUTION: A position of a speaker is identified by a voice signal processing part 136 from a voice signal part detected from signals captured through microphones 131(1) and 131(2), and a changing point of a voice signal to be processed is detected by a voice characteristic analysis part 143 for taking into consideration the speaker's location. Signals, captured through the microphones 131(1) and 131(2), are converted into digital signals and are recorded to a data storage device 111, together with the information on the detected switching point of the speaker. When the information, stored in the data storage device 111, is reproduced, display information relating to the stored speaker is displayed on an LCD 135, in synchronization with the reproduction of the digital signals, based on the information on the switching point of the speaker. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to various apparatuses for processing audio signals such as an IC (Integrated Circuit) recorder, an MD (Mini Disc (registered trademark)) recorder, and a personal computer, and a method used in the apparatus.

  For example, as disclosed in Patent Document 1 to be described later, the minutes for performing voice recognition on the recorded voice data, converting the voice data into text data, and automatically creating the minutes A creation device has been proposed. By using such a technique, it is possible to quickly create the minutes of the meeting without human intervention. However, there is a case where it is not necessary to create a minutes based on all the recorded audio data, and it is desirable to create a minutes of only the important part. For this reason, it is necessary to find a target portion from the recorded audio data.

For example, when a situation such as a long-time meeting is recorded using an IC recorder or an MD recorder, in order to find a place to be heard from the recorded audio data, the audio data is reproduced and the reproduced audio is listened to. Must be. Of course, it is possible to search for a target portion by using a function such as fast-forward or fast-reverse, but it often takes time and effort. For this reason, there is provided a recording apparatus having a function capable of embedding (adding) a “mark for facilitating search” in recorded audio data. For example, an MD recorder or the like is realized as a function for adding a track mark.
JP-A-2-206825

  However, as described above, the function of adding “a mark for easy search” to data can be used by a user's manual operation. If there is no user's operation, the mark is added. Cannot be added. Therefore, even if you want to mark an important part during recording, you might forget the mark-marking operation if you are concentrating on a meeting. Conceivable.

  Further, even if a mark is added to a noticeable speech part, the mark embedding operation is performed when a noteworthy comment is heard, so that the mark is recorded after the noticeable comment. Therefore, in order for the user to listen to the remarks of interest, the user has to move the playback position to the mark and then move it back a little. It is a very cumbersome and stressful operation for the user to repeat this operation by going too far beyond the desired place or returning too much.

  Also, I don't know until I ask what the place is marked with. If it is not the target location after listening, the operation of moving to the next landmark must be repeated until the target location is reached, which is also a time-consuming work. As described above, the function of adding “a mark for facilitating search” to the data is convenient. However, in the case where the user's operation is unclear, it is possible to add to the target portion of the audio data. The function of attaching a mark cannot be sufficiently functioned.

  In view of the above, the present invention provides an apparatus and method that can quickly find and use a target portion in an audio signal to be processed without bothering the user. With the goal.

In order to solve the above-mentioned problem, a storage / reproduction device according to claim 1 is provided.
Multiple microphones,
Digital conversion means for converting a signal input from the microphone into a digital signal;
Audio signal detection means for detecting an audio signal portion from the signal converted into the digital signal;
Speaker detection means for specifying the position of the speaker based on the audio signal parts collected by the plurality of microphones detected by the audio signal detection means and detecting a change point of the speaker;
Speaker information storage means for associating data for identifying a speaker and speaker-related display information related to the speaker for each speaker;
Storage control means for storing information on the digital signal and the change point of the speaker in a storage medium;
Display means for displaying information;
Reproducing means for reproducing the digital signal stored in the storage medium and the information of the change point of the speaker;
And control means for displaying speaker-related display information on the display means in synchronism with the reproduction of the digital signal based on information on the change point of the speaker.

  According to the recording / reproducing apparatus of the first aspect of the present invention, a plurality of microphones are provided, and a signal input through each of the plurality of microphones is converted into a digital signal by the digital conversion means. The audio signal portion is detected from the signal by the audio signal detecting means. Then, based on the audio signal portion detected from the signal collected by each microphone, the speaker detecting means specifies the position of the speaker and detects the change point of the speaker. Then, the digital signal converted by the digital conversion means and the information on the change point of the speaker detected by the speaker detection means are stored in a predetermined recording medium under the control of the storage control means.

  The digital signal recorded on the predetermined recording medium and the information about the speaker change point are reproduced by the reproducing means. The speaker information storage means includes data for identifying the speaker and the speaker. The speaker related display information is stored for each speaker in association with each other. Based on the conversion point information of the speaker, the speaker related display information is synchronized with the reproduction of the digital signal by the control means. It is displayed on the display means.

  Thus, by using a plurality of microphones, the position of the speaker is also detected, and the change point of the speaker can be accurately detected in consideration of the detected position of the speaker. The And, for each change point of the speaker, speaker related display information is displayed on the display means so that each time the speaker changes, it is possible to surely and quickly know who the speaker is. To be.

  According to the present invention, even when a long meeting is recorded, a switching mark (mark) is automatically added every time a speaker is switched. This makes it possible to easily and quickly replay the speech portion of the target speaker.

  In addition, it is possible to identify the speaker at the change point and manage the information indicating the identified speaker and the change point of the voice data in association with each other, so that a specific speaker can be managed without reproducing the voice data. Can be easily and quickly located.

  In addition, it is possible to eliminate the portion that has been relied on the memory of the minutes creator so far, and to improve the efficiency of the minutes creation work, which took time and effort. In addition, it is possible to omit the creation of the minutes itself and use the recorded data as minutes of the audio data format with high searchability.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the embodiments described below, the present invention will be described by taking as an example a case where the present invention is applied to an IC recorder which is a recording / reproducing apparatus for audio signals.

[First Embodiment]
[Overview of IC recorder configuration and operation]
FIG. 1 is a block diagram for explaining an IC recorder which is a recording / reproducing apparatus according to the first embodiment. As shown in FIG. 1, an IC recorder of this embodiment includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102 storing programs and various data, and a RAM mainly used as a work area. A (Random Access Memory) 103 is connected via a CPU bus 104 and includes a control unit 100 configured as a microcomputer. Note that the RAM 103 is provided with a compressed data area 103 (1) and a PCM (Pulse Code Modulation) area 103 (2), as will be described later.

  A data storage device 111 is connected to the control unit 100 through a file processing unit 110, and a key operation unit 121 is connected through an input processing unit 120. Further, a microphone 131 is connected to the control unit 100 through an analog / digital converter (hereinafter abbreviated as A / D converter) 132 and through a digital / analog converter (hereinafter abbreviated as D / A converter) 134. A speaker 133 is connected. Further, an LCD (Liquid Crystal Display) 135 is connected to the control unit 100. In this embodiment, the LCD 135 also has an LCD controller function.

  Further, a data compression processing unit 141, a data expansion processing unit 142, an audio feature analysis unit 143, and a communication interface (hereinafter abbreviated as communication I / F) 144 are connected to the control unit 100. In FIG. 1, the data compression processing unit 141, the data expansion processing unit 142, and the audio feature analysis unit 143 indicated by double lines also realize their functions by software (programs) executed by the CPU 101 of the control unit 100. It is something that can be done.

  In this embodiment, the communication interface 144 is a digital interface such as USB (Universal Serial Bus) or IEEE (Institute of Electrical and Electronics Engineers) 1394, and is a personal computer connected to the connection terminal 145, digital Data can be exchanged with various electronic devices such as cameras.

  In the IC recorder of the first embodiment, when the REC key (recording key) 211 of the key operation unit 121 is pressed, the CPU 101 controls each unit to perform recording processing. In this case, a sound signal collected by the microphone 131 and digitally converted by the A / D converter 132 and data-compressed by the function of the data compression processing unit 141 is transmitted to the predetermined data storage device 111 through the file conversion unit 110. Recorded in the storage area.

  The data storage device 111 according to the first embodiment is a flash memory or a memory card using a flash memory, and a database area 111 (1) and an audio file 111 (2) are provided as will be described later. It is what was done.

  During the recording process, the IC recorder according to the first embodiment performs a feature analysis for each predetermined processing unit on the sound signal to be collected and recorded (recorded) by the function of the sound feature analysis unit 143. When it is detected that a feature has changed, a mark (mark) is added at the time when the feature has changed. By using this mark, the target audio signal portion can be quickly searched from the recorded audio signal.

  FIG. 2 is a diagram for explaining the outline of the process for marking a change point of an audio signal to be collected and recorded. In the IC recorder according to the first embodiment, as described above, the feature analysis is performed for each predetermined processing unit on the sound signal collected by the microphone 131.

  Then, by comparing with the previous feature analysis result, the change point changed from the silent part or the noise part to the spoken voice part, or the change point where the speaker changed even in the spoken voice part, The position (time) of the change point on the audio signal is specified. Then, the specified position is stored in the data storage device 111 as change point information (mark information). Thus, holding the change point information indicating the position of the change point on the audio signal marks the audio signal to be collected and recorded.

  Specifically, as shown in FIG. 2, when recording the state of the conference, it is assumed that Mr. A starts speaking 10 seconds after the start of recording. In this case, before Mr. A's speech starts, silent or non-sensed sounds such as noise that is different from clear speech, such as a noise or a noise that pulls a chair, or a sound that hits a table are collected. Therefore, when Mr. A starts speaking and the speech is collected, the characteristic analysis result of the collected voice signal is clearly different from that before Mr. A starts speaking.

  The voice feature analysis unit 143 detects a change point of the voice signal to be collected and recorded, specifies (acquires) the position of the change point on the voice signal, and specifies the specified change point information (on the voice signal). The specified position information) is stored and held in the data storage device 111 as the mark MK1 in FIG. FIG. 2 shows an example in which the elapsed time from the start of recording is stored and held as change point information.

  Then, suppose that Mr. B started speaking after a while after Mr. A's speech ended. Immediately before Mr. B starts speaking, it is silent or noisy. Also in this case, when Mr. B starts speaking and the speech is collected, the characteristic analysis result of the collected voice signal is clearly different from that before Mr. B started speaking. In FIG. 2, as indicated by the mark MK2, the change point information (mark MK2) is stored and held in the data storage device 111 so that the start portion of Mr. B's speech is marked.

  Furthermore, a case may occur where Mr. C breaks in the middle of Mr. B's statement. In this case, since the voice of Mr. B is different from the voice of Mr. C, the analysis result of the collected voice signal is also different. As shown by mark MK3 in FIG. The change point information (mark MK3) is stored and held in the data storage device 111 so that a mark is attached to the start part of the remark.

  As described above, the IC recorder according to this embodiment performs the feature analysis of the collected sound signal during the recording process, and stores and holds the position on the sound signal in which the feature has been changed. The mark can be attached at the time of change.

  In FIG. 2, as indicated by the column “other” in the marks MK1, MK2, and MK3, for example, by recognizing a speech portion and converting it into text data, the text data is associated and stored. Or other related information can be stored and held together.

  In the IC recorder according to the first embodiment, when the PLAY key (reproduction key) 212 of the key operation unit 121 is pressed, the CPU 101 controls each unit to perform reproduction processing. That is, a recorded audio signal (digital audio signal) that has been compressed and stored in a predetermined storage area of the data storage device 111 is read through the file processing unit 110, and is decompressed by the function of the data expansion processing unit 142. It is processed and restored to the original digital audio signal before data compression. The restored digital audio signal is converted into an analog audio signal by the D / A converter 134, and this is supplied to the speaker 133, and the audio corresponding to the audio signal recorded and reproduced is emitted.

  At the time of this reproduction processing, in the first IC recorder, the NEXT key (key for instructing positioning to the next mark) 214 and the PREV key (key for instructing positioning to the previous mark) 215 of the key operation unit 121. In response to this, the playback position is quickly positioned at the position where the mark is given, and playback can be performed from there.

  FIG. 3 is a diagram for explaining the positioning operation to the position on the audio signal indicated by the mark performed at the time of reproduction of the recorded audio signal, and is a diagram showing the change in the display information of the LCD 135 that changes according to the operation. is there. As shown in FIG. 3, when the PLAY key 211 is pressed, as described above, the CPU 101 controls each unit and starts playback from the head of the instructed recorded audio signal.

  In the remark part of Mr. A, as described with reference to FIG. 2, based on the mark MK1 (stored and held) at the time of the recording process, as shown in FIG. A start time is displayed, and SEQ-No. Indicating that this is the first mark added from the start of recording. 1 is displayed.

  When playback is continued and playback of Mr. B's speech portion is started, the start time of Mr. B's speech is displayed as shown in FIG. 3B, and this is the second mark from the start of recording. SEQ-No. 2 is displayed. Thereafter, when the PREV key 215 is pressed, as shown in FIG. 3C, the CPU 101 starts the part of Mr. A's remark indicated by the mark MK1 whose start time is 10 seconds after the beginning (0 minutes 10 seconds later). Position the playback position at, and resume playback from there.

  Thereafter, when the NEXT key is pressed, as shown in FIG. 3D, the CPU 101 positions the playback position at the start portion of Mr. B's remark indicated by the mark MK2 whose start time is 1 minute 25 seconds after the beginning, Playback resumes from there. Further, when the NEXT key is pressed, the CPU 101 positions the playback position at the start portion of Mr. C's remark indicated by the mark MK3 whose start time is 2 minutes and 30 seconds after the beginning, as shown in FIG. 3E. Resume playback from.

  As described above, the IC recorder of this embodiment automatically analyzes the characteristics of the collected audio signal during the recording process, and marks the change points of the characteristics, and at the time of the reproduction process. By operating the NEXT key 214 and PREV key 215, the playback position can be quickly positioned at the position on the recorded audio signal indicated by the marked mark, and playback can be performed from there. .

  As a result, the user can quickly position the playback position on the speech portion of the target speaker (speaker) and play and listen to the recorded audio signal, so that the minutes of the target speech portion can be recorded. Can be created quickly.

  Here, for the sake of simplicity, the time information from the recording start time is used as the change point information. However, the present invention is not limited to this, and recording of the recorded audio signal in the data storage device 111 is not limited thereto. Addresses on the medium can also be used as change point information.

[Details of IC recorder operation]
Next, the recording process and the reproduction process in the IC recorder of the first embodiment will be described in detail with reference to the flowcharts of FIGS.

[Recording process]
First, the recording process will be described. FIG. 4 is a flowchart for explaining a recording process performed in the IC recorder according to the first embodiment. The process illustrated in FIG. 4 is a process performed by the CPU 101 controlling each unit.

  When the IC recorder according to the first embodiment is in a state where the power is turned on and is not operating, it waits for an operation input from the user (step S101). When the user presses an operation key on the operation unit 121, the input processing unit 120 detects this and notifies the CPU 101, so the CPU 101 determines whether or not the received operation input is a pressing operation of the REC key 211. (Step S102).

  If it is determined in step S102 that the received operation input is not a pressing operation of the REC key 211, the CPU 101 performs processing corresponding to the key operated by the user, for example, reproduction processing corresponding to the PLAY key 212, A positioning process for the next mark according to the NEXT key 124, a positioning process for the previous mark according to the PREV key 215, and the like are performed (step S103). Of course, it is also possible to perform fast forward processing, fast reverse processing, and the like.

  If it is determined in step S102 that the REC key has been pressed, the CPU 101 instructs the file processing unit 110 to perform file recording processing, and in response to this, the file processing unit 110 performs data recording. An audio file 111 (2) is created in the recording device 111 (step S104).

  Then, the CPU 101 determines whether or not the STOP key (stop key) 213 of the key operation unit 121 has been pressed (step S105). If it is determined in step S105 that the STOP key 213 has been operated, a predetermined end process is performed (step S114), as will be described later, and the process shown in FIG. 4 ends.

  If it is determined in step S105 that the STOP key 213 is not operated, the CPU 101 instructs the A / D converter 132 to convert an analog audio signal input through the microphone 131 into a digital audio signal. Digital conversion of the collected sound is performed (step S106).

  As a result, the A / D converter 132 writes the digital audio signal obtained by converting the analog audio signal input through the microphone 131 at regular intervals (predetermined processing units) into the PCM data area 103 (2) of the RAM 103, The CPU 101 is notified of the writing (step S107).

  In response to this, the CPU 101 instructs the data compression processing unit 141 to compress the data of the digital audio signal (PCM data) stored in the PCM data area 103 (2) of the RAM 104 (step S108). In response to this, the data compression processing unit 141 compresses the digital audio signal in the PCM data area 103 (2) of the RAM 103, and writes the compressed digital audio signal in the compressed data area 103 (1) of the RAM 103 (step S109). ).

  Then, the CPU 101 instructs the file processing unit 110 to write the compressed digital audio signal in the compressed data area 103 (1) of the RAM 103 into the audio file 111 (2) created in the data storage device 111. As a result, the file processing unit 110 writes the compressed digital audio signal in the compressed data area of the RAM 103 into the audio file 111 (2) of the data storage device 111 (step S110).

  When the file processing unit 110 finishes writing the compressed digital audio signal to the audio file 111 (2), the file processing unit 110 notifies the CPU 101 of this, so the CPU 101 notifies the audio feature analysis unit 143 of the PCM data in the RAM 103. The feature analysis of the digital audio signal recorded in the area 103 (2) is instructed, and the audio feature analysis unit 143 extracts the feature of the digital audio signal in the PCM data area 103 (2) of the RAM 103 (step S111). .

  The feature analysis (feature extraction) processing of the digital voice signal performed in the voice feature analysis unit 143 uses various methods such as voice print analysis, speech speed analysis, interval analysis, and voice strength analysis. Is possible. Here, in order to simplify the description, the audio feature analysis unit 143 of the IC recorder according to the first embodiment will be described as extracting features of a digital audio signal to be analyzed by performing voiceprint analysis.

  Then, the voice feature analysis unit 143 compares the voice feature (voice print data) extracted this time with the voice print data of the voice extracted in the past, and the features extracted from the input voice signal are It is determined whether or not the feature has changed, and the determination result is notified to the CPU 101. Based on this, the CPU 101 determines whether or not the feature of the collected sound has changed (step S112).

  When it is determined in step S112 that there is no change, the CPU 101 repeats the processing from step S105, and the audio signal of the next cycle (next processing unit) is also processed from the above-described steps S105 to S112. Do processing.

  When it is determined in step S112 that there has been a change, the CPU 101 determines that “the speaker has been switched”, and causes the file processing unit 110 to change the feature of the sound on the sound signal to be processed. Is instructed to be marked (step S113). As a result, the file processing unit 110 uses the audio file as information indicating the location where the audio characteristics have changed as information related to the audio file 111 (2) in the database area 111 (1) on the data recording device 111. The time information from the head of 111 (2) or the address information corresponding to the recording position is written. In this case, the audio file and information indicating the location where the audio feature has changed are stored in association with each other.

  After the process of step S113, the CPU 101 repeats the process from step S105, and performs the process from step S105 to step S112 on the audio signal of the next cycle (next process unit).

  When it is determined in the determination process in step S105 that the user has pressed the STOP key 213, the CPU 101 writes data to the audio file 111 (2) of the data storage device 111 to the file processing unit 110. A predetermined end process is performed such as stopping the data compression processing unit 141 to stop the compression process and instructing the A / D converter 132 to stop the conversion to a digital signal (step S114). The process shown in FIG.

  Note that the voice feature analysis unit 143 determines whether or not the voice feature has been changed by retaining previously extracted voice feature data (voice print data) and newly extracted feature data ( Voice print data). In this case, if it is only necessary to compare with the immediately preceding feature data, only the immediately preceding feature data needs to be held. However, in order to improve accuracy, when comparing with two or more feature data in the past and judging that the feature has changed when two or more differences occur, two or more past data are used. It is necessary to store the feature data.

  As described above, the IC recorder according to the first embodiment performs feature analysis of a sound signal to be collected and recorded, detects a change point of the feature of the collected sound signal, and corresponds to the change point. The mark can be attached to the position on the collected sound signal.

[About playback processing]
Next, the reproduction process will be described. FIG. 5 is a flowchart for explaining the reproduction process performed in the IC recorder according to the first embodiment. The processing illustrated in FIG. 5 is processing performed by the CPU 101 controlling each unit.

  In the playback process of the IC recorder of the first embodiment, as described with reference to FIG. 4, the characteristic change point of the collected voice (sound collected and recorded) added during the recording process is used. By using the attached mark, a target audio signal portion can be quickly detected from the recorded audio signal.

  The IC recorder according to the first embodiment is in a state where power is turned on, and when not operating, the IC recorder waits for an operation input from the user (step S201). When the user presses an operation key on the operation unit 121, the input processing unit 120 detects this and notifies the CPU 101. Therefore, the CPU 101 determines whether or not the received operation input is a pressing operation of the PLAY key 212. (Step S202).

  If it is determined in step S202 that the received operation input is not a pressing operation of the PLAY key 212, the CPU 101 performs processing corresponding to the key operated by the user, for example, recording processing corresponding to the REC key 212, A positioning process for the next mark according to the NEXT key 124, a positioning process for the previous mark according to the PREV key 215, and the like are performed (step S203). Of course, it is also possible to perform fast forward processing, fast reverse processing, and the like.

  In the determination process of step S202, when it is determined that the received operation input is a PLAY key pressing operation, the CPU 101 instructs the file processing unit 110 to read the audio file 111 (2) on the data recording device 111. (Step S204). Then, the CPU 101 determines whether or not the STOP key (stop key) 213 of the key operation unit 121 has been pressed (step S205).

  If it is determined in step S205 that the STOP key 213 has been operated, a predetermined end process is performed (step S219) as will be described later, and the process shown in FIG. 5 ends.

  If it is determined in step S205 that the STOP key 213 has not been operated, the CPU 101 controls the file processing unit 110 to store the compressed digital data stored in the audio file 111 (2) of the data storage device 111. The audio signal is read out in an amount corresponding to a predetermined processing unit defined by the system, and is written into the compressed data area 103 (1) of the RAM 103 (step S206).

  When the writing is completed, the CPU 101 is notified of this, so the CPU 101 instructs the data expansion processing unit 142 to perform the expansion processing of the compressed digital audio signal in the compressed data area 103 (1) of the RAM 103. Then, the digital audio signal compressed by the data expansion processing unit 142 is expanded and written in the PCM data area 103 (2) of the RAM 103 (step S207).

  When the writing is completed, this is notified to the CPU 101, and the CPU 101 analogizes the digital audio signal (expanded digital audio signal) stored in the PCM data area 103 (2) of the RAM 103 to the D / A converter 134. Control is performed so that the sound signal is converted and supplied to the speaker 133.

  As a result, sound corresponding to the digital sound signal stored and held in the sound file 111 (2) of the data storage device 111 is emitted from the speaker 133. Then, since the D / A converter 134 notifies the CPU 101 that the analog audio signal after D / A conversion has been output, the CPU 101 determines whether or not the operation key of the key operation unit 121 has been operated ( Step S209).

  If it is determined in step S209 that the operation key has not been operated, the processing from step S205 is repeated, and the reproduction of the digital audio signal of the audio file 111 (2) in the data storage device 111 is continued.

  When determining in step S209 that the operation key is operated, the CPU 101 determines whether the operated key is the PREV key 215 (step S210). If it is determined in step S210 that the PREV key 215 has been operated, the CPU 101 instructs the file processing unit 110 to stop reading the digital audio signal from the audio file 111 (2), and the data expansion processing is performed. The decompression process is instructed to the unit 142, and the conversion to the analog signal is instructed to the D / A converter 134 (step S211).

  Next, the CPU 101 instructs the file processing unit 110 to read the information (change point information) immediately before the position that is currently reproduced from the database area 111 (1) of the data storage device 111, and the reading is performed. The playback position is positioned at the position on the audio signal indicated by the mark information, and playback is started from that position (step S212). As described with reference to FIG. 3, the mark information used for positioning is added. The corresponding reproduction position information is displayed (step S213), and the processing from step S205 is repeated.

  When it is determined in step S210 that the operated key is not the PREV key 215, the CPU 101 determines whether or not the operated key is the NEXT key 214 (step S214). If it is determined in step S214 that the NEXT key 214 has been operated, the CPU 101 controls the file processing unit 110 to stop reading the digital audio signal from the audio file 111 (2). 142 is instructed to stop the decompression process, and the D / A converter 134 is instructed to stop the conversion to an analog signal (step S215).

  Next, the CPU 101 instructs the file processing unit 110 to read the information (change point information) immediately after the position that is currently reproduced from the database area 111 (1) of the data storage device 111, and the reading is performed. The playback position is positioned at the position on the audio signal indicated by the mark information, and playback is started from that position (step S216). As described with reference to FIG. 3, the mark information used for positioning is added to the mark information. The corresponding reproduction position information is displayed (step S217), and the processing from step S205 is repeated.

  When it is determined in step S214 that the operated key is not the NEXT key 214, the CPU 101 performs processing corresponding to the operated key, for example, according to the operated key such as fast forward or fast reverse. The process from step S205 is repeated.

  As described above, when the IC recorder determines that the voice characteristics have changed during recording, it is determined that the speaker is switched, and a mark is automatically added at that position, so that the user can use the PREV key 215 and the NEXT key 214 during playback. It is possible to easily call the head position of each utterance by pressing the button, and when creating a minutes, it is possible to greatly reduce the trouble of repeatedly reproducing a certain utterance or finding an important utterance. That is, the target audio signal portion can be quickly searched from the recorded audio signal.

  In addition, the change point of the characteristics of the collected sound is automatically detected, and the mark is automatically assigned to the change point, so that the user's hand is not troubled about the mark assignment to the change point. .

[Modification of the first embodiment]
By the way, when recording the state of a meeting and creating a minutes based on this recording, it is more convenient if it is possible to know who has spoken and where without replaying the recorded sound. In view of this, the IC recorder of this modification example stores voiceprint data, which is a voice characteristic analysis result of attendees attending a conference, in association with symbols for identifying each attendee, thereby allowing a speaker to be stored. A mark that can be identified is attached.

  The IC recorder of this modification is configured similarly to the IC recorder of the first embodiment shown in FIG. However, an audio feature database for the attendees of the conference is formed in the storage area of, for example, the external storage device 111 or the RAM 103 of the IC recorder of this modification. In the following description, the voice feature database is described as being formed in the external storage device 111.

  FIG. 6 is a diagram for explaining an example of a voice database formed in the storage area of the external storage device 111 of the IC recorder of this modification. As shown in FIG. 6, the voice database of this example includes an identifier (for example, a sequence number corresponding to the registration order) for identifying a conference attendee, the name of the conference attendee, and the voice of the conference attendee. This is composed of voice print data, which is a result of feature analysis, image data such as face photos of meeting attendees, icon data assigned to each meeting attendee, and text data.

  Each of the voiceprint data, image data, icon data, and other data is stored in the external storage device 111 in the form of a file, which stores and holds the identifier of each attendee of the meeting as key information (association information). Has been. Note that the voiceprint data, which is the result of feature analysis, is obtained in advance by collecting voices of the attendees of the conference and performing feature analysis prior to the conference.

  That is, the IC recorder of this example has a voice database creation mode. When the voice database creation mode is selected, the voices of the attendees of the conference are collected, and the voice analysis is performed by the voice feature analysis unit 143 to obtain voice print data. Can be stored in the storage area of the external storage device 111 in association with an identifier such as a sequence number.

  Information other than the identifier and voiceprint data, such as name, image data, and icon data, is supplied to the IC recorder of this example through, for example, a personal computer connected to the connection terminal 145, as shown in FIG. And stored in association with the corresponding identifier and voiceprint data. Of course, a name or the like can be input by operating an operation key provided in the key operation unit 121 of the IC recorder. The image data can also be captured from a digital camera connected to the connection terminal 145.

  As described with reference to FIGS. 1, 2, and 4, the IC recorder of this example also performs a feature analysis of the collected voice, detects a change point of voiceprint data, and responds to the change point. When a change point is detected, the voiceprint data of the latest collected voice is matched with the voiceprint data of the voice database when a change point is detected. It is included in the mark that gives the identifier of the meeting attendee whose data matches.

  FIG. 7 is a diagram for explaining the outline of the process of adding marks to the audio signal to be collected and recorded in the IC recorder of this modification. The process of adding marks is basically performed in the same manner as described with reference to FIG. However, the speaker identifier is added to the mark.

  As shown in FIG. 7, when recording the state of the meeting, it is assumed that Mr. A starts speaking after 10 seconds from the start of recording. In this case, before Mr. A's speech starts, silent or non-sensed sounds such as noise that is different from clear speech, such as a noise or a noise that pulls a chair, or a sound that hits a table are collected. Therefore, the characteristic analysis result of the collected audio signal is clearly different from that before Mr. A started speaking. The position of the change point on the audio signal is specified (acquired), and the specified change point information is stored and held as a mark MK1 in FIG.

  In this case, the latest voiceprint data and voiceprint data in the voice database are matched, and the identifier of the speaker (conference attendee) corresponding to the matching voiceprint data is included in the mark MK1. FIG. 7 also shows a case where the elapsed time from the start of recording is stored and held as change point information.

  Then, suppose that Mr. B started speaking after a while after Mr. A's speech ended. It is assumed that there was no sound or noise immediately before Mr. B's remark. Also in this case, when Mr. B starts speaking and is collected, the characteristic analysis result of the collected voice signal is clearly different from that before Mr. B starts speaking. As shown by the mark MK2, the change point information (mark MK2) is stored and held so as to mark the start part of Mr. B's speech.

  Also in this case, matching is performed between the latest voiceprint data and the voiceprint data of the voice database, and the identifier of the speaker (conference attendee) corresponding to the matching voiceprint data is included in the mark MK2.

  Furthermore, there may be a case where Mr. C breaks in the middle of Mr. B's speech. In this case, the voice of the collected sound is different because Mr. B's voice is different from Mr. C's voice. The signal analysis results are also different. In FIG. 7, as indicated by the mark MK3, the change point information (mark MK3) is stored and held so that a mark is attached to the start portion of Mr. C's speech.

  Also in this case, the latest voiceprint data is matched with the voiceprint data of the voice database, and the identifier of the speaker (conference attendee) corresponding to the matched voiceprint data is included in the mark MK3.

  By doing so, it becomes possible to specify which part of the recorded audio signal is who's utterance. For example, only Mr. A's utterance is reproduced and Mr. A's utterance is reproduced. It becomes easy to summarize the summary of.

  The other information of each mark of this modification is, for example, by performing voice recognition of the collected voice, converting the collected voice into text data, and using this text data as other information in a file format (text data file ) Is stored and retained. By using this text data, minutes and summaries of statements can be quickly created.

  Also in this modified example of the IC recorder, the recorded voice can be reproduced in the same manner as described with reference to FIGS. And in the case of the IC recorder of this modification, it is possible to specify without reproducing the recorded voice of the utterance part of each speaker in the recorded voice.

  FIG. 8 is a diagram for explaining an operation of positioning a mark performed when a recorded audio signal is reproduced, and is a diagram showing a change in display information on the LCD 135 that changes in accordance with an operation. As shown in FIG. 8, when the PLAY key 211 is pressed, as described above, the CPU 101 controls each unit to start playback from the head of the instructed recorded audio signal.

  In Mr. A's remarks part, as described with reference to FIG. 7, based on the mark MK1 (stored and held) during the recording process, as shown in FIG. A speech start time D (1), a face photo D (2) corresponding to the speaker's image data, a speaker's name D (3), and text data D (4) of the first part of the speech are displayed. During playback, display D (5) is displayed.

  Then, when the reproduction is continued and the reproduction of Mr. B's speech part is started, the message start time D (1) for Mr. B is shown in FIG. 8B based on the mark MK2 attached at the time of recording. ), A face photograph D (2) corresponding to the image data of the speaker, a name D (3) of the speaker, and text data D (4) of the first part of the utterance, and a display D (5) during playback ) Is displayed.

  Thereafter, when the PREV key 215 is pressed, as shown in FIG. 8C, the CPU 101 starts the part of Mr. A's speech indicated by the mark MK1 whose start time is 10 seconds after the beginning (0 minutes and 10 seconds later). The playback position is positioned at, and playback starts from there. In this case, as in the case of FIG. 8A, the speech start time D (1), face photo D (2) corresponding to the speaker image data, and speaker name D (3) The text data D (4) of the first part of the utterance is displayed and the display D (5) during playback is displayed.

  Thereafter, when the NEXT key is pressed, as shown in FIG. 8D, the CPU 101 positions the playback position at the start portion of Mr. B's remark indicated by the mark MK2 whose start time is 1 minute 25 seconds after the beginning, Start playback from there. In this case, as in the case of FIG. 8B, the start time D (1) of the utterance about Mr. B, the face photo D (2) corresponding to the image data of the speaker, and the name D (3) of the speaker The text data D (4) of the first part of the utterance is displayed and the display D (5) during playback is displayed.

  Further, when the NEXT key is pressed, the CPU 101 positions the playback position at the start of Mr. C's remark indicated by the mark MK3 whose start time is 2 minutes and 30 seconds after the beginning, as shown in FIG. 8E. Start playback from. In this case, Mr. C's speech start time D (1), face photo D (2) corresponding to the speaker's image data, speaker's name D (3), and text of the first part of the speech The data D (4) is displayed and the playback display D (5) is displayed.

  In this modification, for example, when the NEXT key or PREV key is pressed twice quickly while Mr. A's speech part is being played back, the part where Mr. A's speech part appears next or before that, A mode may be added in which a playback position is positioned at an appearing portion and playback is started from there. In other words, by repeating this operation, only Mr. A's remark part can be traced or reproduced retroactively. Of course, instead of the NEXT key or PREV key, an operation key that explicitly indicates this mode may be provided. In this case, the remark portion of Mr. A is automatically reproduced one after another.

  As described above, the IC recorder of this modified example automatically performs the feature analysis of the collected audio signal at the time of the recording process, adds a mark to the change point of the feature, and at the time of the reproduction process. By operating the NEXT key 214 and PREV key 215, the playback position can be quickly positioned at the position on the recorded audio signal indicated by the assigned mark, and playback can be performed from there. .

  Moreover, at the change point of the recorded audio signal, it is possible to clearly indicate who is speaking by displaying the name of the speaker or displaying a face photo. Can be searched quickly, and only the utterance portion of a specific speaker can be reproduced. Of course, as information for specifying a speaker, an icon corresponding to icon data unique to each speaker may be displayed. In addition, since the text data of the first part of the utterance can be displayed, it can be used when determining whether or not the utterance part is the target.

  Then, the user of the IC recorder of this modified example uses the display information at the time of reproduction to quickly position the reproduction position in the remarked part of the intended person and reproduce and listen to the recorded audio signal. Therefore, it is possible to quickly create the minutes of the target remark part.

  In other words, without replaying the recorded audio signal after recording, it is possible to visually grasp who is speaking, and it is possible to easily find out the speech of a specific speaker. As the symbols, in addition to character strings and symbols, information that makes it easier to specify the speaker, such as a photograph of the speaker's face, can be used, so searchability is improved.

  In addition, by associating a utterance of a speaker whose voice characteristics are unregistered (when the IC recorder cannot be identified even though it is registered) with a symbol indicating that the speaker is an unregistered speaker, You can easily find that part. In this case, the minutes maker may reproduce the utterance part of the unregistered speaker and determine who the person is.

  When it is known who the unregistered speaker is, if it is a registered speaker, a symbol associated with the speaker can be re-marked. If the speaker is unregistered, the speaker can be newly registered. The features of the sound are extracted from the recorded sound, and the symbol to be associated is input with a symbol or character string registered in advance in the IC recorder, a photographed image if the IC recorder has a camera photographing function, or image data captured from an external device. .

  The recording process of the IC recorder of this modification is performed in the same manner as the recording process described with reference to FIG. 4, but in the process of assigning speaker switching marks MK1, MK2, MK3,. Then, matching with the voice print data of the voice database is performed, and the identifier of the corresponding speaker is added. Further, when there is no corresponding voiceprint data, a mark indicating no corresponding is given.

  Further, the reproduction process of the IC recorder of this modification is performed in the same manner as the reproduction process described with reference to FIG. 5, but in the reproduction position information display process in steps S213 and S217, the face photograph of the speaker and the name The text data of the content of the utterance will be displayed.

  Also in the case of the IC recorder of this modification, the time from the recording start time is used as the change point information. However, the present invention is not limited to this, and recording of the recorded audio signal in the data storage device 111 is not limited thereto. An address on the medium may be used as change point information.

[Mark execution timing]
In the IC recorder of the first embodiment described above and the IC recorder of the modification of the first embodiment, a change point of the collected sound is detected during the recording process, and an audio signal corresponding to the change point is detected. Although the mark is attached to the position, it is not limited to this. A mark can be added after the recording process is completed. That is, it is possible to add a mark at the time of reproduction processing, or to perform only the mark addition processing.

  FIG. 9 is a flowchart for explaining a process of marking a change point of a recorded audio signal after the recording process is completed. That is, the process shown in FIG. 9 is performed when a mark is added to the change point of the recorded sound at the time of the reproduction process, or when only the mark providing process is performed independently for the change point of the recorded sound. It is. The processing shown in FIG. 9 is also processing performed by the CPU 101 of the IC recorder controlling each unit.

  First, the CPU 101 controls the file processing unit 104 to read out the recorded audio signals stored in the audio file of the data storage device 111 after being compressed in predetermined units (step S301), and to record all the recorded audio signals. It is determined whether or not the reading is finished (step S302).

  If it is determined in step S302 that all the recorded audio signals have not been read, the CPU 101 controls the data expansion processing unit 142 to perform the expansion processing of the data-compressed recorded audio signals ( Step S303). Thereafter, the CPU 101 controls the voice feature analysis unit 143 to perform the feature analysis of the expanded voice signal, obtain voice print data, and compare it with the previously obtained voice print data. It is determined whether or not has changed (step S305).

  If it is determined in step S305 that the characteristics of the recorded audio signal have not changed, the processing from step S301 is repeated. If it is determined in the determination process in step S305 that the characteristics of the recorded voice signal have changed, the CPU 101 determines that “the speaker has been switched” and the file processing unit 110 has changed the voice characteristics. Is instructed to add a mark (step S306).

  As a result, the file processing unit 110 uses the database area 111 (1) on the data recording device 111 as information related to the audio file 111 (2) as information indicating the location where the audio characteristics have changed. The time information from or the address information corresponding to the recording position is written. In this case, the audio file and information indicating the location where the audio feature has changed are stored in association with each other.

  After the processing in step S306, the CPU 101 repeats the processing from step S301, and performs the same processing for the audio signal in the next cycle (next processing unit). If it is determined in step S302 that reading has been completed for all the recorded audio signals, a predetermined termination process is performed (step S307), and the process shown in FIG. 9 is terminated.

  As a result, after the recording process, the change point of the recorded voice is detected during the reproduction process, and a mark is given to the recorded voice signal, or only the mark giving process is independently performed on the recorded voice. And so on. When adding marks during reproduction processing, the audio signal expanded in step S 303 shown in FIG. 9 is D / A converted, and the analog audio signal after D / A conversion is supplied to the speaker 133. What should I do?

  As described above, it is expected that the processing load and power consumption during recording can be reduced by adding marks to the changing points of the characteristics of the recorded audio signal after recording. Also, the user may not want automatic markup for all recordings. It may be possible to turn on / off the automatic marking function during recording. If the user has recorded with the recording set to OFF, and marking is required later, the recorded audio signal is marked even after the recording process as described above. Is very convenient.

  Further, as described above, since the recorded audio signal can be marked, it can be applied to a device that does not have a recording function but has a signal processing function. For example, the present invention can be applied to application software of a personal computer. That is, an audio signal recorded by an audio recording device can be transferred to a personal computer and marked by the above-described signal processing application software operating on the personal computer.

  Further, by sharing data created by a device to which the present invention is applied via a network or the like, it is possible to use the data itself as a minutes without writing the minutes from the data.

  Therefore, the present invention can be applied not only to a recording device but also to various electronic devices capable of signal processing, and even an already recorded audio signal is processed by the electronic device by applying the present invention. Thus, the same result can be obtained. That is, the minutes can be created efficiently.

  As described above, the IC recorder according to the first embodiment described with reference to FIG. 1 includes the communication I / F 144 and can be connected to an electronic device such as a personal computer. Therefore, if an audio signal (digital audio signal) recorded by the IC recorder of the first embodiment described above and marked with a change point is transferred to the personal computer, the personal computer's Through the display device having a large display screen, it is possible to display more detailed information and to quickly search for a speech portion of a target speaker.

  10 and 11 show the display connected to the personal computer based on the recorded audio signal transferred from the IC recorder of the first embodiment described above to the personal computer and the given change point information (mark information). 6 is a diagram for explaining a display example of change point information on a display screen of the apparatus 200. FIG.

  In the case of FIG. 10, the time zone display 201 corresponding to the recorded audio signal and the mark display (change point display) MK1, MK2, MK3, MK4,... To do. In this way, it is possible to recognize the positions of a plurality of change points at a glance. Then, for example, by using a pointing device such as a mouse to position the cursor on the target mark display and clicking, the recorded sound can be reproduced from that position.

  In the case of FIG. 11, a plurality of the displays shown in FIG. 8 are displayed on the display image of the display device 200 all at once, and the speaker's face photographs 211 (1), 211 (2 ), 211 (3),... And text data 212 (1), 212 (2), 212 (3),... Corresponding to the content of the utterance are displayed to quickly search the utterance portion of the target speaker. And so on. In addition, the title display 210 can be performed using a function of a personal computer.

  In the display example of FIG. 11, “00”, “01”, “02”, “03”,... On the left side indicate the time from the beginning of the recorded voice. Of course, it is possible to realize various display modes such as performing a plurality of displays as shown in FIG.

  If the data in which the utterance (recorded voice) is associated with the information (symbol) identifying the utterer is transferred to a device such as a personal computer that has a large display unit, the agenda can be used without writing the sentence from the voice data. A record can be created. That is, the data itself recorded by the IC recorder to which the present invention is applied is the minutes.

  In addition, if software such as a plug-in that makes the data public on a web page and can be viewed on a web browser is prepared, the minutes can be shared through the network. As a result, sharing of information, that is, time and effort until the information is disclosed can be greatly reduced by using the present invention.

[Second Embodiment]
[Outline of configuration and operation of IC recorder]
FIG. 12 is a block diagram for explaining an IC recorder which is a recording / reproducing apparatus according to the second embodiment. The IC recorder of the second embodiment includes two microphones 131 (1) and 131 (2) and an audio signal processing unit that processes audio signals from the two microphones 131 (1) and 131 (2). Except for the point provided with 136, it is configured in the same manner as the IC recorder of the first embodiment shown in FIG. For this reason, in the IC recorder of the second embodiment, the same reference numerals are given to the same components as those of the IC recorder of the first embodiment shown in FIG. The description will be omitted.

  In the IC recorder according to the second embodiment, the voice signal processing unit 136 processes the collected voice signals from the two microphones 131 (1) and 131 (2), so that the speaker's The position (sound source position) is specified, and the change point (speaker change point) of the collected sound signal can be specified taking this into consideration. That is, as auxiliary information for detecting a change point of a collected voice signal using voiceprint data obtained as a result of voice analysis, the position of the speaker based on the collected voices of two microphones is also used to be more accurate. In addition, change points and speakers can be specified.

  FIG. 13 is a diagram for explaining a configuration example of the microphones 131 (1) and 131 (2) and the audio signal processing unit 136. In the case of the example shown in FIG. 13, each of the two microphones 131 (1) and 131 (2) is unidirectional as shown in FIG. The microphones 131 (1) and 131 (2) are arranged close to each other so that the main directing direction is opposite. Thereby, the microphone 131 (1) can collect the voice of the speaker A satisfactorily, and the microphone 131 (2) can collect the voice of the speaker B satisfactorily.

  The audio signal processing unit 136 includes an adder 1361, a comparator (comparator) 1362, and an A / D converter 1363 as shown in FIG. The sound signals collected by the microphones 131 (1) and 131 (2) are supplied to an adder 1361 and a comparator 1362.

The adder 1361 adds the collected sound signal from the microphone 131 (1) and the collected sound signal from the microphone 131 (2), and supplies the added sound signal to the A / D converter 1363. The sum signal of the collected sound from the microphone 131 (1) and the collected sound from the microphone 131 (2) can be expressed as the following (Equation 1), and is collected by the omnidirectional microphone. You can see that they are the same.
((1 + cos θ) / 2) + ((1-cos θ) / 2) = 1 (Expression 1)
Further, the comparator 1362 compares the collected sound signal from the microphone 131 (1) with the collected sound signal from the microphone 131 (2). The comparator 1362 determines that the speaker A is mainly speaking if the level of the collected sound signal from the microphone 131 (1) is larger, and the value becomes “1 (high level)”. A speaker discrimination signal is supplied to the control unit 100. The comparator 1362 determines that the speaker B is mainly speaking if the level of the collected sound signal from the microphone 131 (2) is larger, and the value becomes “0 (low level)”. A speaker discrimination signal is supplied to the control unit 100.

  Thus, the position of the speaker is specified based on the collected sound signal from the microphone 131 (1) and the collected sound signal from the microphone 131 (2), and the speaker A speaks or speaks. It is possible to determine whether the message is B.

  Note that the third speaker C sees the speaker A and the speaker B obliquely forward in the direction crossing the main direction of the microphones 131 (1) and 131 (2) (FIG. 13). )), The output levels of the collected sound from the microphones 131 (1) and 131 (2) are substantially equal.

  When the speaker C at such a position is also supported, two threshold values in the comparator 1362 are provided, and if the level difference is within ± Vth, it is determined that the speaker C is in the horizontal direction, If the difference is larger than + Vth, the speaker A may be determined, and if the level difference is smaller than −Vth, the speaker B may be determined.

  Then, a speaker located in the directivity direction of the microphone 131 (1), a speaker located in the directivity direction of the microphone 131 (2), and a direction intersecting the directivity direction of the microphones 131 (1) and 131 (2). By knowing who each of the speakers is, it is possible to identify who the speaker (speaker) is. Therefore, in addition to detection of change points based on voiceprint data obtained as a result of collected voice feature analysis, it is possible to more accurately identify the speaker by taking into account the level of the collected voice of the microphone. it can.

[Other examples of microphone and audio signal processor]
Further, the microphones 131 (1) and 131 (2) and the audio signal processing unit 136 can be configured as shown in FIG. That is, FIG. 14 is a diagram for explaining another configuration example of the microphones 131 (1) and 131 (2) and the audio signal processing unit 136. In the case of the example shown in FIG. 14, each of the two microphones 131 (1) and 131 (2) is omnidirectional as shown in FIG. The microphones 131 (1) and 131 (2) are arranged close to each other with a spacing of about 1 cm, for example.

  As shown in FIG. 14, the audio signal processing unit 136 of this example includes an adder 1361, an A / D converter 1363, a subtractor 1364, and a phase comparator 1365. The collected sound signals from the microphones 131 (1) and 131 (2) are supplied to the adder 1361 and the subtractor 1364, respectively.

  Here, the added output signal from the adder 1361 is equivalent to the omnidirectional microphone output, and the subtracted output from the subtractor 1364 is equivalent to the bidirectional (eight-shaped directivity) microphone output. In the bi-directional microphone, the phase of the output is normal phase or reverse phase depending on the incident direction of the sound wave. Therefore, the polarity of the subtracted output from the subtractor 1364 is determined by performing phase comparison between the added output (omnidirectional output) from the adder 1361 and the subtracted output from the subtractor 1364 by the phase comparator 1365. By doing so, the speaker can be identified.

  That is, when the polarity of the subtraction output from the subtractor 1364 is normal phase, the speech of the speaker A is collected, and when the polarity of the subtraction output from the subtractor 1364 is reverse phase, the speaker is collected. It can be determined that the utterance of B is collected.

  Similarly to the case described with reference to FIG. 13, it also tries to determine the utterance of the utterer C located at the position where the utterer A and the utterer B are viewed obliquely forward (in the horizontal direction in FIG. 14). In this case, the level of the subtraction output of the audio signal obtained by collecting the utterance of the utterer C becomes small. Therefore, by checking the levels of the addition output from the adder 1361 and the subtraction output from the subtractor 1364, it is possible to recognize the speech of the speaker C.

  In the case of the audio signal processing unit 136 shown in FIG. 14, an adder 1361 is used. However, the adder 1361 is not an essential component. For example, the output signal of either the microphone 131 (1) or 131 (2) may be supplied to the A / D converter 1363 and the phase comparator 1365.

  As described above, in FIGS. 13 and 14, the position of the speaker can be specified using the levels and polarities of the collected voices of the two microphones 131 (1) and 131 (2) during the recording process. I am doing so. By taking this identification result into consideration, it is possible to accurately detect the change point of the collected sound and to identify the speaker.

  The method using FIG. 13 and FIG. 14 is used not only at the time of the recording process but also when a mark is given to the recorded sound at the time of the reproduction process, or when only the mark giving process is performed on the recorded sound independently. Can also be used.

  For example, when the system described with reference to FIG. 13 is to be used after recording processing, as shown in FIG. 15A, each of the unidirectional microphones 131 (1) and 131 (2) is used. 2 channel stereo recording of the collected audio signal. Then, as shown in FIG. 15B, during reproduction or when the mark providing process is performed independently, each of the compressed two-channel audio signals compressed from the external storage device 111 is decompressed, and after the decompression process. Are input to a comparator having the same function as the comparator 1362 shown in FIG.

  This makes it possible to determine whether the collected sound signal of the microphone 131 (1) is mainly used or whether the collected sound signal of the microphone 131 (2) is mainly used. The speaker can be specified based on the position of the speaker with respect to each microphone.

  Similarly, when the method described with reference to FIG. 14 is to be used after the recording process, the output signals from the microphones 131 (1) and 131 (2) are recorded in two channels in stereo, and are reproduced or marked. In the case where the assigning process is performed independently, the speaker can be specified by performing the same process as the audio signal processing unit 136 shown in FIG.

  The position information of the speaker with respect to each of the microphones 131 (1) and 131 (2) prepared in advance when performing speaker specifying processing using output signals from the microphones 131 (1) and 131 (2) is as follows. For example, it may be stored and held in the IC recorder as in the speaker position database shown in FIG.

  FIG. 16 is a diagram for explaining an example of the speaker position database. The speaker position database in this example mainly uses the speaker identification signal corresponding to the identification result from the audio signal processing unit 136 of the IC recorder, the microphone identification information corresponding to each speaker identification signal, and each microphone. It consists of an identifier (speaker identifier) of a speaker candidate. Further, as shown in FIG. 16, a plurality of speaker identifiers can be registered for one microphone.

  The speaker location database as shown in FIG. 16 is preferably created before the start of the conference. In general, there are many cases where attendees to a conference and the seating order of each attendee are determined in advance, so that it is possible to create a speaker location database before the start of the conference in consideration of the installation position of the IC recorder. .

  Also, if the attendees suddenly change or the seat changes during the meeting, for example, the speaker is not recognized according to the sound collected by the microphone, and voice analysis processing is performed. Only the change point is detected based on the voiceprint data obtained from the above, or after the recording process, the speaker position database is adjusted to be accurate and the recorded voice is remarked. It can also be made.

  By using the speaker position database as shown in FIG. 16, the speaker position can be specified, and the speaker itself at that position can also be specified.

  In the second embodiment, two microphones 131 (1) and 131 (2) are used and two or three speakers are used as an example. However, the present invention is not limited to this. is not. By using more microphones, it is possible to identify more speakers.

  Further, the method of specifying the speaker himself / herself by specifying the position of the speaker according to the output signal from the microphone is not limited to the method described with reference to FIGS. For example, a proximity 4-point method or a proximity 3-point method can be used.

  As shown in FIG. 17A, the proximity four-point method is a time structure of audio signals collected by four microphones M0, M1, M2, and M3 that are arranged in close proximity so that one microphone is not necessarily in the same plane. In this method, spatial information such as the position and size of the sound source is calculated by a technique such as short-time correlation or acoustic intensity. In this way, by using at least four microphones, it is possible to accurately specify the position of the speaker and to specify the speaker according to the position (seat position) of the speaker.

  Further, in the case where it is possible to limit the speaker to be located substantially in the horizontal plane, the arrangement relationship of the microphones arranged close to each other may be three in the horizontal plane as shown in FIG. 17B.

  As shown in FIGS. 17A and 17B, the arrangement relationship of the microphones does not have to be an orthogonal relationship. In the case of the proximity three-point method shown in FIG. 17B, the three microphones may have a positional relationship such as being arranged at the apex of an equilateral triangle, for example.

[Modification of Second Embodiment]
In the above-described IC recorder of the second embodiment, when detecting a change point of a collected sound signal using voiceprint data obtained as a result of sound analysis, mainly based on the collected sound of two microphones. By taking into account the discrimination results of the microphones used, the change point of the audio signal is detected with higher accuracy. However, it is not limited to this.

  For example, as shown in FIG. 18, an IC recorder may be configured in which two microphones 131 (1) and 131 (2) and an audio signal processing unit 136 are provided, but an audio feature analysis unit 143 is not provided. Is possible. That is, the IC recorder of FIG. 18 is configured similarly to the IC recorder of the second embodiment shown in FIG. 12 except that the audio feature analysis unit 143 is not provided.

  Then, based on the collected sound of the two microphones 131 (1) and 131 (2), the change point of the speaker is detected based only on the discrimination result of the microphone that is mainly used, and according to the change point. It is also possible to add a mark to the corresponding position on the audio signal. In this case, since it is not necessary to perform the voice feature analysis process, the load on the CPU 101 can be reduced.

  In the above-described embodiment, a mark is attached to the change point of the processing target audio signal. However, even if it is a change point, the mark is attached only to the change point to the spoken voice. , You can make the search more efficient. For example, based on the signal level or voiceprint data of the processing target speech signal, the speech and other unnecessary parts such as noise are clearly recognized, and only the start point of the speech is marked. You can also

  It is also possible to determine whether the speaker is male or female based on voiceprint data, frequency characteristic data of the audio signal, etc., and to notify the gender of the speaker at the changing point.

  In addition, based on the mark information attached as described above, a search mode for performing only a search, a mark edit mode for changing, deleting, or adding a position of the attached mark, It is also possible to provide a special reproduction mode for reproducing only the utterance portion of the speaker that can be specified according to the mark, for example, only the utterance portion of Mr. A. Each of these modes can be realized relatively easily only by adding to the program executed by the CPU 101.

  In addition, the voice print data in the voice feature database shown in FIG. 6 may be updated with the voice print data used for detecting the change point to obtain a highly accurate voice feature database. . For example, even if there is a mismatch in the voice print data comparison process, if the voice feature database of the speaker is actually present, the voice print data of the speaker database is changed to the newly acquired voice print data. Can be.

  In addition, if the voiceprint data is matched in the voiceprint data comparison process but actually matches the voiceprint data of a different speaker, the voiceprint data of the different speaker is set not to be used in the comparison process. You can also.

  In addition, when the voiceprint data matches the voiceprint data of a plurality of speakers, the voiceprint data to be used may be prioritized so as to match only the correct speaker.

  In addition, the position where the mark is added may be added not only to the start point of the speech but also to the end point, and may be changed in consideration of the convenience of each user such as several seconds or several seconds before the start point. It is also possible to make it possible.

  In addition, as described above, the feature analysis of the audio signal can obtain highly accurate analysis data by using not only the voice print analysis but also one or more of various methods.

  In the above-described second embodiment, the case where two microphones are mainly used has been described as an example, but the present invention is not limited to this. As long as the number of microphones is two or more, any number of microphones may be used. By using various parameters such as the signal level, polarity, and delay time until sound collection of each of the plurality of microphones. The position of the speaker can be specified, and the speaker himself can be specified according to the position.

  In the first and second embodiments described above, the case where the present invention is applied to an IC recorder that is a recording / reproducing apparatus for audio signals has been described as an example. However, the present invention is not limited to this. For example, the present invention can be applied to a recording apparatus, a reproducing apparatus, and a recording / reproducing apparatus that use a recording medium such as a hard disk drive or a magneto-optical disk such as an MD or an optical disk such as a DVD. That is, the present invention can be applied to a recording apparatus, a reproducing apparatus, and a recording / reproducing apparatus that use various recording media.

[Realization by software]
Also, a program for realizing the functions of each processing unit such as the audio feature analysis unit 143 and the audio signal processing unit 136 of the IC recorder according to the above-described embodiment and organically linking the functions is created. However, the present invention can also be realized by causing the CPU 101 to execute this program. That is, the present invention can be realized by creating a program for performing the processing shown in the flowcharts of FIGS. 4 and 5 and causing the CPU 101 to execute the program.

  Similarly to the above-described embodiment, for example, the personal computer installed with the program that implements the function of the voice feature analysis unit 143 is loaded with the voice data recorded by the recorder, and the switching of the speaker is detected. Is also possible.

It is a block diagram for demonstrating an example of the recording / reproducing apparatus with which this invention was applied. FIG. 2 is a diagram for explaining an outline of processing for marking a change point of an audio signal to be collected and recorded in the recording / reproducing apparatus shown in FIG. 1. It is a figure for demonstrating the positioning operation | movement to the mark performed at the time of reproduction | regeneration of the recorded audio | voice signal, and is a figure which shows the change of the display information of LCD135 which changes according to operation. 4 is a flowchart for explaining a recording process in the recording / reproducing apparatus shown in FIG. 1. 3 is a flowchart for explaining a reproduction process in the recording / reproducing apparatus shown in FIG. 1. It is a figure for demonstrating an example of the audio | voice database formed in the storage area of the external storage device 111 of the recording / reproducing apparatus which has the structure shown in FIG. It is a figure for demonstrating the outline | summary of the process which attaches a mark to the collected audio | voice signal performed in the recording / reproducing apparatus which has the structure shown in FIG. It is a figure for demonstrating the positioning operation | movement to the mark performed at the time of reproduction | regeneration of the recorded audio | voice signal, and is a figure which shows the change of the display information of LCD135 which changes according to operation. It is a flowchart for demonstrating the process in the case of performing the process which attaches a mark to the change point of the recorded audio | voice signal after completion | finish of a recording process. It is a figure for demonstrating the example of a display of the change point information on the display screen of a display apparatus according to the data transferred to the personal computer from the recording / reproducing apparatus shown in FIG. It is a figure for demonstrating the example of a display of the change point information on the display screen of a display apparatus according to the data transferred to the personal computer from the recording / reproducing apparatus shown in FIG. It is a block diagram for demonstrating the other example of the recording / reproducing apparatus with which this invention was applied. It is a figure for demonstrating an example of microphone 131 (1), 131 (2) and the audio | voice signal processing part 136. FIG. FIG. 11 is a diagram for explaining another example of microphones 131 (1) and 131 (2) and an audio signal processing unit 136. It is a figure for demonstrating the process in the case of performing the process which attaches a mark to the change point of the recorded audio | voice signal after completion | finish of a recording process. It is a figure for demonstrating an example of a speaker position database. It is a figure for demonstrating the other example of the system which specifies a speaker himself / herself by specifying the position of a speaker according to the output signal from a microphone. It is a block diagram for demonstrating the other example of the recording / reproducing apparatus with which this invention was applied.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... CPU, 102 ... ROM, 103 ... RAM, 104 ... CPU bus, 110 ... File processing unit, 111 ... Data storage device, 120 ... Input processing unit, 121 ... Key operation unit, 132 ... A / D converter, 131 ... Microphone, 134 ... D / A converter, 133 ... Speaker, 135 ... LCD, 141 ... Data compression processing unit, 142 ... Data expansion processing unit, 143 ... Audio feature analysis unit, 144 ... Communication I / F, 145 ... Connection terminal, 131 (1), 131 (2) ... microphone, 136 ... audio signal processing unit

Claims (11)

Multiple microphones,
Digital conversion means for converting a signal input from the microphone into a digital signal;
Audio signal detection means for detecting an audio signal portion from the signal converted into the digital signal;
Speaker detection means for specifying the position of the speaker based on the audio signal parts collected by the plurality of microphones detected by the audio signal detection means and detecting a change point of the speaker;
Speaker information storage means for associating data for identifying a speaker and speaker-related display information related to the speaker for each speaker;
Storage control means for storing information on the digital signal and the change point of the speaker in a storage medium;
Display means for displaying information;
Reproducing means for reproducing the digital signal stored in the storage medium and the information of the change point of the speaker;
A storage / reproducing apparatus comprising: control means for displaying speaker-related display information on the display means in synchronism with reproduction of the digital signal based on information on a change point of the speaker.   Speaker position input means for inputting speaker position information for specifying a speaker at the specified speaker position is further provided, and the input speaker position is stored in the speaker information storage means. The storage / reproducing apparatus according to claim 1. Information compression means for compressing information on the signal converted into the digital signal;
Compression information expansion means for expanding the compressed digital signal stored in the storage means,
The storage / reproducing apparatus according to claim 2, wherein the storage medium stores a digital signal compressed with information.   4. The storage / reproducing apparatus according to claim 3, further comprising input means for inputting the speaker-related display information.   5. The storage / reproducing apparatus according to claim 4, wherein the speaker-related display information is a face photograph. Information compression means for compressing information on the signal converted into the digital signal;
Compression information expansion means for expanding the compressed digital signal stored in the storage means,
The storage / reproducing apparatus according to claim 1, wherein the storage medium stores a digital signal compressed with information.   The storage / reproducing apparatus according to claim 1, further comprising an input means for inputting the speaker-related display information.   The storage / reproducing apparatus according to claim 1, wherein the speaker-related display information is a face photograph. In a control method of a storage / reproduction device that can record signals input from a plurality of microphones on a recording medium and reproduce a signal stored in the recording medium,
Control for storing the signal input from the microphone in the recording medium is as follows.
Converting a signal input from a microphone into a digital signal;
Detecting an audio signal portion from the signal converted into the digital signal;
Identifying the position of the speaker based on the audio signal portions collected by the detected plurality of microphones and detecting a change point of the speaker;
Storing the digital signal and information on the change point of the speaker in a recording medium;
With
The control for reproducing the signal recorded on the recording medium is as follows:
Reproducing the digital signal stored in the storage medium and the information of the change point of the speaker;
Reading speaker related display information from speaker information storage means for storing speaker-specific data and speaker-related display information related to the speaker for each speaker and storing them for each speaker. When,
And a step of controlling to display the speaker-related display information read from the speaker information storage means on the display means in synchronism with the reproduction of the digital signal. Control method. Accepting input of speaker location information for identifying a speaker at the specified speaker location;
The storage / reproducing apparatus control method according to claim 9, further comprising: storing the received speaker position information in the speaker information storage unit.   The control method for a storage / reproducing apparatus according to claim 10, further comprising a step of receiving input of the speaker-related display information.

Images