JP5279028B2 - Audio processing apparatus, audio processing method, and program - Google Patents

Description

  The present invention includes a plurality of microphones (array microphones), extracts and records subject sound (target sound) for recording purposes, and reproduces the recorded target sound, a sound processing method, and a computer. The present invention relates to a program that functions as a voice processing device.

  In the field of audio recording and video shooting, multiple voices are input from individual microphones, the voices input from the microphones are recorded together with time information, and the recorded user is used to play the equipment. There are technologies that improve usability. For example, as one of these, a first audio input unit that records subject audio and a second audio input unit that records photographer audio are provided, and the photographer's speech and subject audio are recorded separately. In addition, a technology has been disclosed in which the voice analysis unit converts the photographer's speech into text and manages it as metadata along with time information, and displays the photographer's speech corresponding to the video as a telop along with the video during playback. (See Patent Document 1). Patent Document 1 also discloses a technique in which a device user freely selects and reproduces a subject's voice and a photographer's voice.

  In addition, regarding the recording of subject audio, in the recording of a target sound in a distance or a specific target sound in a conference, by processing multiple input sounds and suppressing ambient noise other than the target sound, the target sound is clearly displayed. There are many techniques to improve the performance. For example, Patent Document 2 includes an array microphone, and from a plurality of sounds acquired by the array microphone, the direction of each sound source sound is determined based on the phase difference of the same sound source sound input to each microphone, and a specific direction is determined. For example, a sound source separation technique for extracting only a target sound such as a subject voice is disclosed. With these technologies, it is possible to extract only the sound of a specific sound source expected by the device user, such as subject sound, from ambient noise.

JP 2007-104405 A JP 2006-227328 A

  However, in the configurations disclosed in Patent Documents 1 and 2, in order to use the recorded sound at the time of reproduction, ambient noise and other sounds simultaneously captured from the input sound of the sound input unit are removed as noise and recorded. It is necessary to maintain clarity of the sound source. Also, in order to constantly record a plurality of input sounds at the time of shooting, it is necessary to always operate a plurality of sound codecs. Furthermore, if the sound source separation technique for extracting the sound source sound is not used and the clarity of the sound source sound is not ensured, it may be difficult to convert to text.

  The present invention has been made in view of the above problem, and extracts target sound such as subject sound emitted from a specific direction and photographer's sound with high clarity from input sounds from a plurality of sound input units. Also, by limiting the recording period of the photographer's voice by the recording period determination means, the chapter information (scene break information) including the photographer's voice and the time information of the determined recording period is recorded, and the chapter information is recorded during playback. An object of the present invention is to provide a voice processing device or the like that uses the.

In order to achieve the above object, a speech processing apparatus according to the first aspect of the present invention provides:
A voice acquisition unit for acquiring a plurality of voices;
An imaging unit that captures a moving image together with the plurality of sounds;
A voice extraction unit that extracts a subject's voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition unit ;
A determination unit that determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction unit ;
A generating unit that generates information indicating a delimiter associated with each of the sound of the subject and the sound of the photographer based on the delimiter determined by the determination unit ;
A display unit for displaying information indicating a break generated by the generation unit ;
A timekeeping unit that measures the time at which the plurality of sounds are acquired;
A character conversion unit that converts the sound of the subject and the sound of the photographer into a character string corresponding to the sound;
The generation unit generates a predetermined frame constituting the moving image associated with the break as information indicating the break,
The display unit displays the time measured by the time measuring unit in association with the information indicating the separation, and displays the character string in association with the information indicating the separation .

It is also possible to further include a movement detection unit that detects the movement amount of the voice processing device.
The determination unit may determine the separation between the sound of the subject and the sound of the photographer when the movement amount detected by the movement detection unit is equal to or greater than a predetermined threshold amount.

In order to achieve the above object, an audio processing method according to another aspect of the present invention includes an audio acquisition unit, an imaging unit, an audio extraction unit, a determination unit, a generation unit, a display unit, and a timing unit. A voice processing method executed by a voice processing device having a character conversion unit ,
The voice acquisition unit acquires a plurality of voices;
The imaging unit captures a moving image together with the plurality of sounds; and
The voice extraction unit extracts a subject voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition step ;
The determination unit determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction step ;
The generating unit generates information indicating a delimiter associated with each of the subject's voice and the photographer's voice based on the delimiter determined in the determination step ;
The display unit displays information indicating a break generated by the generation step ;
The timekeeping unit is a timekeeping step of measuring times when the plurality of sounds are acquired;
The character conversion unit includes a character conversion step of converting the sound of the subject and the sound of the photographer into a character string corresponding to the sound,
In the generating step, a predetermined frame constituting the moving image associated with the break is generated as information indicating the break,
In the display step, the time measured by the timing step and information indicating the break are displayed in association with each other, and the character string is displayed in association with information indicating the break .

In order to achieve the above object, a program according to another aspect of the present invention provides:
An audio acquisition unit that acquires multiple audio from the computer ;
An imaging unit that captures a moving image together with the plurality of sounds,
A voice extraction unit that extracts a subject's voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition unit ;
A determination unit that determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction unit ;
A generating unit that generates information indicating a delimiter associated with each of the sound of the subject and the sound of the photographer based on the delimiter determined by the determination unit ;
A display unit for displaying information indicating a break generated by the generation unit ;
A timekeeping unit that times the time at which the plurality of sounds are acquired;
Functioning as a character conversion unit that converts the sound of the subject and the sound of the photographer into a character string corresponding to the sound;
The generation unit generates a predetermined frame constituting the moving image associated with the break as information indicating the break,
The display unit displays the time measured by the time measuring unit in association with the information indicating the separation, and displays the character string in association with the information indicating the separation.

  According to the present invention, it is possible to create a chapter indicating a break of a recorded scene, and to perform a scene search that is highly convenient for the user. In addition, it is possible to save power in the voice processing device.

It is a figure which shows the block configuration of the moving image imaging device in Embodiment 1. It is a figure which shows an example of the block structure containing a level measurement part. It is a figure which shows an example of the block structure containing the movement detection part of an apparatus. It is a figure which shows an example of the chapter information in Embodiment 1. It is a figure which shows an example of a chapter information table. It is a figure which shows the example of a list of the photography scene in one imaging | photography. It is a figure which shows the example of a menu screen for reproducing | regenerating the audio | voice in Embodiment 1. FIG. It is a figure which shows the block configuration of the moving image imaging device in Embodiment 2. It is a figure which shows an example of the chapter information in Embodiment 2. It is a figure which shows the example of a menu screen for reproducing | regenerating the audio | voice in Embodiment 2. FIG.

(Embodiment 1)
The present invention is applied to a device having an audio recording / playback function in which a plurality of microphones (array microphones) are mounted as an audio input unit. The invention according to Embodiment 1 will be described with reference to FIGS. FIG. 1 is a block configuration diagram of a terminal device in which a sound processing device according to the present invention is applied to a moving image photographing device 100.

  In the following embodiment, in order to describe the present invention as the moving image photographing apparatus 100, the term “chapter” indicating a video segmentation or a group of images is used.

  Note that the present invention is not limited to moving image shooting, and can also be applied to, for example, a micro recorder having a recording / playback function for only sound. It can also be applied to mobile phones, digital still cameras, and the like having equivalent functions.

  In addition, when used in an audio recording / playback apparatus such as a micro recorder, since there is no information related to video, it can be used by replacing it with scene break information.

  The moving image shooting apparatus 100 measures a predetermined time by a video / audio processing unit 20 that performs overall processing of video and audio, a control unit 21 that controls each unit, an operation unit 22 that receives operations from a user, and a periodic clock. And a timer unit 23.

  The video / audio processing unit 20 includes an array microphone 1, an ADC 2, an audio extraction unit 3, a recording period determination unit 4, a camera 5, an image processing unit 6, a codec unit 7, a chapter information generation unit 8, a recording unit 9, and a chapter information reading unit. 10, an OSD creation unit 11, an OSD composition unit 12, a display unit 13, a DAC 14, and a speaker 15.

  The array microphone 1 accepts a plurality of voice inputs. The array microphone 1 is configured by arranging a plurality of individual microphones connected to each other. The array of microphones is arranged, for example, in one dimension, two dimensions, or three dimensions.

  The ADC 2 converts the analog audio signal input from the array microphone 1 into a digital audio signal.

  The sound extraction unit 3 individually extracts a plurality of sounds (for example, subject sounds and photographer sounds) emitted from a specific direction from a plurality of sounds input from the array microphone 1.

  The recording period determination unit 4 determines the recording period of the photographer's voice. The recording period determination unit 4 can determine the recording period based on an arbitrary method.

  The camera 5 captures an image. The camera 5 can capture an arbitrary image such as a moving image or a still image.

  The image processing unit 6 performs signal processing such as image quality adjustment and resizing on the output signal from the camera 5.

  The codec unit 7 performs compression processing on the audio signal output from the audio extraction unit 3 and the video signal output from the image processing unit 6 during shooting.

  In the present embodiment, the video signal output from the image processing unit 6 at the time of shooting and the audio signal output from the audio extraction unit 3 are combined with the audio signal and video expanded in the codec unit 7 at the time of reproduction. Although it is configured to have a function of switching to a signal, a configuration provided outside the codec unit 7 is also possible.

  The chapter information generation unit 8 generates chapter information indicating a break (break) in the video.

  The recording unit 9 records the video / audio data compressed by the codec unit 7 and the chapter information created by the chapter information generation unit 8.

  The chapter information reading unit 10 reads the chapter information recorded in the recording unit 9 during video reproduction.

  The OSD creation unit 11 performs an OSD (On Display) such as a menu or information display based on a user operation received by the operation unit 22 or timing information from the timer 23 or chapter information read by the chapter information reading unit 10. Screen display).

  The OSD synthesis unit 12 synthesizes the OSD created by the OSD creation unit 11 and the video signal output from the codec unit 7.

  The display unit 13 displays the video signal output from the OSD synthesis unit 12.

  The DAC 14 converts the digital audio signal output from the codec unit 7 into an analog audio signal.

  The speaker 15 outputs an analog audio signal converted by the DAC 14.

  The flow of processing at the time of shooting will be described using the block diagram of FIG.

  The plurality of audio signals captured by the array microphone 1 are converted into digital signals by the ADC 2, and then the subject audio and the photographer audio are extracted by the audio extraction unit 3 and input to the codec unit 7.

  The video signal captured by the camera 5 is subjected to signal processing such as image quality adjustment and resizing by the image processing unit 6 and then input to the codec unit 7.

  The subject sound output from the sound extraction unit 3 is sound output from the speaker 15 during reproduction. The subject audio is recorded in the recording unit 9 after being compressed by the codec unit 7 during shooting and recording, together with the video signal output from the image processing unit 6 during video recording.

  The photographer's voice, which is an output from the voice extraction unit 3, is a target voice used as chapter information. For the photographer's voice, only the photographer's voice at the same playback position as that of the chapter during shooting and recording is output to the codec unit 7 and recorded in the recording unit 9.

  Here, the chapter information is a segment of a moving image (scene break), and is information used for easily jumping to a scene intended by the user during reproduction.

  The recording period of the photographer's voice is determined by the recording period determination unit 4. The control unit 21 controls the voice extraction unit 3 and the codec unit 7 according to the determination period, and records only the photographer's voice in the recording period in the recording unit 9.

  Next, a method in which the recording period determination unit 4 determines the recording period will be described with reference to FIG.

  As a first example, the recording period determination unit 4 determines the recording period in accordance with the shooting start operation from the photographer (user) received by the operation unit 22. Based on the determination by the recording period determination unit 4, the chapter information generation unit 8 captures the photographer's voice for a certain period from the start and generates chapter information. Here, the measurement for a certain period is executed by the timer unit 23.

  As a second example, the recording period determination unit 4 determines a recording period in which a photographer's voice is captured when a photographer (user) operates a chapter button provided on the operation unit 22. The chapter information generation unit 8 generates chapter information based on the determination by the recording period determination unit 4. The method for setting the recording period by the photographer's operation is determined by an arbitrary method such as a toggle button, ON / OFF, in a depressed state, or a certain period after the button is depressed. Both the first example and the second example can be realized by a photographer's button operation without newly adding a processing unit to the block configuration diagram of FIG.

  Next, another method for determining the recording period by the recording period determination unit 4 will be described with reference to FIG.

  FIG. 2 is a block configuration diagram for determining the recording period based on the measurement result of the level measurement unit. FIG. 2 shows a configuration in which a level measuring unit 30 for measuring the photographer's voice level of the voice extracting unit 3 is added to FIG. A third example in which the recording period determination unit 4 determines the recording period is shown in FIG. Hereinafter, the same processing units as those in FIG.

  The level measurement unit 30 notifies the recording period determination unit 4 whether or not the sound level of the photographer's voice has reached a specific level set in advance. The recording period determination unit 4 determines that the period reaching the specific level is the recording period. Based on the determination result of the recording period determination unit 4, the chapter information generation unit 8 generates chapter information including the position information of the photographer's voice with respect to the captured video, and records the chapter information in the recording unit 9. The voice extraction unit 3 outputs a photographer voice related to one chapter information generated based on the determination of the recording period determination unit 4 to the codec unit 7. The codec unit 7 compresses the photographer's voice output from the voice extraction unit 3 and records the photographer's voice in the recording unit 9.

  The specific level is an arbitrary audio level such as an audio level above a certain volume, an audio level below a certain volume, an audio level within a range of 60 dB to 80 dB, and the like.

  Next, another method for determining the recording period by the recording period determination unit 4 will be described with reference to FIG. FIG. 3 is a block diagram for determining the recording period based on the detection result of the movement detection unit 40. FIG. 3 shows a configuration in which a movement detection unit 40 for detecting the movement state of the device is added to FIG. FIG. 3 shows a fourth example in which the recording period determination unit 4 determines the recording period. Hereinafter, the same processing units as those in FIG.

  The movement detection unit 40 includes, for example, a general movement detection sensor that can detect the movement and movement amount of an object, detects the movement start and movement stop of the device being photographed, and notifies the recording period determination unit 4. The recording period determination unit 4 determines that a certain period from the movement detection notification notified from the movement detection unit 40 is a recording period. The control unit 21 causes the chapter information generation unit 8 to generate chapter information according to the recording period determined by the recording period determination unit 4 and instructs the codec unit 7 to compress the photographer's voice. The chapter information generation unit 8 generates chapter information including position information of the photographer's voice with respect to the captured video and records the chapter information in the recording unit 9. The codec unit 7 records the compressed photographer voice in the recording unit 9.

  Examples of the movement detection unit 40 include a GPS (Global Positioning System) and an acceleration sensor, but are not limited to the GPS and the acceleration sensor. The movement detection unit 40 has a function of acquiring position information, and can be considered as an arbitrary device that can use movement information calculated from the position information.

  Note that the first to fourth examples described above do not limit the method for determining the recording period of the photographer's voice, and the examples can be used in combination.

  Next, a specific example of chapter information is shown in FIG. 4, and the contents of items included therein will be described.

  The chapter information 400 includes at least one item of the photographer sound storage location and time information for each chapter. The photographer voice storage location indicates a place where the photographer voice recorded in the recording unit 9 is stored. For example, when the photographer stores the photographer's voice at the zoo, the photographer's voice storage location is the zoo. The photographer's voice is used to determine the scene contents as scene related information. The time information indicates the time when the utterance of the photographer voice recorded in the recording unit 9 is started. For example, when the photographer starts speaking at 10:12:12, the time information is 10:12:12. This time information includes date information and chapter position information. The time information is used to indicate reproduction position information in the captured image, and can be a frame number.

  Next, an example of the chapter information table is shown in FIG. In the chapter information table 500, at least one chapter information is recorded for one shooting moving image (from the start to the stop of one shooting). For example, as illustrated in FIG. 5, the captured image 1, the captured image 2, and the captured image 3 have four, three, and five chapter information, respectively. In one chapter, a chapter name, a shooting sound storage location, and time information are recorded in association with each other. The chapter information table 500 is recorded in the recording unit 9.

  Next, a method of using chapter information at the time of reproduction and the operation of each processing unit at that time will be described with reference to FIG. 1, FIG. 6, and FIG. Taking a case where various animals are photographed in a zoo as an example, a technique will be described in which a photographer uses a moving image in which the type of animal is recorded as a photographer's voice.

  FIG. 6 is a diagram schematically showing a list of shooting scenes in one shooting. FIG. 6 shows an example in which a zoo entrance, an elephant, a lion, and a monkey are photographed in one shot. The recorded image shooting scene is one scene of a shot moving image or one still image. The subject sound is a sound (for example, an animal cry) uttered by a subject collected by the array microphone 1 when shooting a moving image. The photographer uttered voice is a voice of the photographer collected by the array microphone 1 when shooting a moving image. The time information indicates the time when the utterance of the photographer's voice is started.

  The chapter information generation unit 8 generates chapter information for each scene such as a zoo entrance, and records the chapter information in the recording unit 9. The chapter information generation unit 8 can also use a part of the recorded image shooting scene as chapter information.

  When displaying the menu screen for reproducing the chapter information, the chapter information reading unit 10 reads the chapter information recorded in the recording unit 9 and passes the chapter information to the OSD creation unit 11. The OSD creation unit 11 creates an OSD (On Screen Display) based on the chapter information and passes the OSD to the OSD composition unit 12. The OSD is combined with the video signal in the OSD combining unit 12 and displayed on the display unit 13.

  FIG. 7 is an example of a menu screen for reproducing audio corresponding to chapter information. A selection button 701 corresponding to the number of captured moving images recorded in the recording unit 9 is displayed on the menu image 700. When the selection button 701 is selected, the sound associated with the selected selection button 701 is reproduced. For example, a shooting moving image including four recorded image shooting scenes shown in FIG. 6 corresponds to the shooting moving image 1 shown in FIG. Therefore, when the selection button 701 corresponding to the photographed moving image 1 is selected, the four photographing scenes shown in FIG. 6 are reproduced.

  The menu image 700 also displays a progress bar 702 indicating the temporal position of chapter information. The progress bar 702 displays time information corresponding to the captured moving image (recorded image shooting scene).

  When the selection button 701 corresponding to the photographed moving image 1 is selected by the photographer (user), the selection button 701 for the photographed moving image 1 is highlighted, for example, based on the time information in the chapter information for the photographed moving image 1 (1 ) Time information such as 10:12:12 is displayed on the progress bar 702. The photographer's utterance voice associated with each chapter information is sequentially reproduced, and time information corresponding to the voice being reproduced is highlighted.

  The sound reproduced at this time is a photographer's utterance voice such as “XX Zoo”, “Elephant”, “Lion”, “Monkey”. When the photographer's speech corresponding to the recorded image shooting scene is being played back, the photographer (user) plays back a desired moving image scene by instructing playback using a playback button or the like provided in the operation unit 22. be able to. For example, when a photographer's utterance voice “Elephant” is being played, if a predetermined playback button is operated, the recorded image shooting scene is played back from the time 10:15:31 when the photographer's utterance voice was recorded. Is done.

  When all the selection buttons 701 indicating the shooting videos 1 to 3 or the “all” selection button 701 for reproducing the shooting videos 1 to 3 are selected, the audio corresponding to the chapter information for the shooting videos 1 to 3 is selected. Are played sequentially. For example, the photographer can recognize which captured image is being reproduced by highlighting the selection button 701 corresponding to the captured moving image being reproduced.

  As shown in FIG. 7, the selection buttons 701 are displayed as shooting videos 1 to 3, but time information including date information acquired from the first chapter information of each shooting video is displayed in the selection button 701. May be. In addition, in the menu image 700, a characteristic subject sound can be reproduced when a captured moving image is reproduced.

  As described above, as shown in the present embodiment, when audio recording of a moving image is used for playback, chapter information can be created in the shooting scene intended by the photographer, and the chapter information recorded during playback can be easily used. Skip, scene search, etc. can be executed.

  Also, the recording period of the photographer's voice is limited by the recording period determination unit, which can reduce the processing load of encoding, contributing to the low power consumption required especially for portable devices with limited battery capacity it can.

  Furthermore, by limiting the recording period, when chapter information is used for scene search or the like, it is possible to suppress as much as possible the possibility of erroneous search due to voice that the user does not recognize as chapter voice.

(Embodiment 2)
The invention according to Embodiment 2 will be described with reference to FIGS. FIG. 8 is a block configuration diagram of a terminal device in which the audio processing device according to the present invention is applied to the moving image photographing device 100. 8 is a configuration in which a speech analysis text converting unit 50 is added between the speech extraction unit 3 and the chapter information generation unit 8 with respect to the block configuration diagram of FIG. Hereinafter, the same processing units as those in FIG.

  The voice analysis text conversion unit 50 converts the photographer's utterance voice output from the voice extraction unit 3 into text, and passes the text to the chapter information generation unit 8. In addition, the voice analysis text converting unit 50 can perform the entire or a part of the photographer's utterance voice including predetermined contents based on the volume of the photographer's utterance voice, the instruction from the photographer, or the photographing time. Can also be converted into text.

  The chapter information generation unit 8 records the content of the photographer's utterance voice converted into text together with the chapter information in the recording unit 9.

  A specific example of the chapter information according to the present embodiment recorded in the recording unit 9 is shown in FIG. Similar to the first embodiment, the photographer's voice storage location and time information are recorded as chapter information. In this embodiment, the photographer's utterance content converted into text is included as chapter information.

  When displaying the menu screen for reproducing the chapter information, the chapter information reading unit 10 reads the chapter information recorded in the recording unit 9 and displays the chapter information in the OSD creation unit 11 as in the first embodiment. hand over. The OSD creation unit 11 creates an OSD based on the chapter information and passes the OSD to the OSD synthesis unit 12. The OSD is combined with the video signal in the OSD combining unit 12 and displayed on the display unit 13.

  FIG. 10 is an example of a menu screen for reproducing the audio corresponding to the chapter information. The menu image 1000 is displayed as a selection button 1001 in which a photographer's utterance voice corresponding to a chapter of a captured moving image recorded in the recording unit 9 is converted into text. For example, if the moving image including the shooting scene shown in FIG. 6 is the shooting moving image 1, since the voice of the photographer is recorded as “XX Zoo”, “Elephant”, “Lion”, “Monkey”, the audio information “XX Zoo” is converted into text, and “XX Zoo” as character information is displayed on the selection button 1001. Then, when the photographer selects the selection button 1001 displayed as “Lion” in the captured moving image 1, the scene is reproduced from the lion scene in the captured moving image 1.

  In FIG. 10, the captured moving image 1 and the captured moving image 2 are described for each captured moving image. However, time information including date information acquired from the first chapter information of each captured moving image may be displayed.

  As described above, as shown in the present embodiment, it is possible to easily recognize a scene to be reproduced after a chapter from text information displayed on a selection button on a menu screen on which chapter information is displayed. The convenience of reproduction such as reproduction from a scene can be enhanced.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation and application are possible.

  The chapter information generation unit 8 can also generate a plurality of shooting scenes for one shooting moving image. The display unit 13 can also display a plurality of generated shooting scenes so as to be continuous. For example, the recorded image shooting scene shown in FIG. 6 is displayed as a plurality of continuous scenes or as a moving image.

DESCRIPTION OF SYMBOLS 100 ... Movie imaging device, 1 ... Array microphone, 2 ... ADC, 3 ... Sound extraction part, 4 ... Recording period determination part, 5 ... Camera, 6 ... Image processing part, 7 ... Codec part, 8 ... Chapter information generation part, DESCRIPTION OF SYMBOLS 9 ... Recording part, 10 ... Chapter information reading part, 11 ... OSD preparation part, 12 ... OSD synthetic | combination part, 13 ... Display part, 14 ... DAC, 15 ... Speaker, 20 ... Video / audio processing part, 21 ... Control part, 22 ... Operation part 23 ... Timer part 30 ... Level measurement part 40 ... Movement detection part

Claims (4)

A voice acquisition unit for acquiring a plurality of voices;
An imaging unit that captures a moving image together with the plurality of sounds;
A voice extraction unit that extracts a subject's voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition unit ;
A determination unit that determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction unit ;
A generating unit that generates information indicating a delimiter associated with each of the sound of the subject and the sound of the photographer based on the delimiter determined by the determination unit ;
A display unit for displaying information indicating a break generated by the generation unit ;
A timekeeping unit that measures the time at which the plurality of sounds are acquired;
A character conversion unit that converts the sound of the subject and the sound of the photographer into a character string corresponding to the sound;
The generation unit generates a predetermined frame constituting the moving image associated with the break as information indicating the break,
The display unit displays the time measured by the time measuring unit in association with the information indicating the separation, and displays the character string in association with the information indicating the separation;
A speech processing apparatus characterized by that. A movement detection unit for detecting a movement amount of the voice processing device;
The determination unit determines a separation between the sound of the subject and the sound of the photographer when the amount of movement detected by the movement detection unit is equal to or greater than a predetermined threshold amount;
The speech processing apparatus according to claim 1 . A voice processing method executed by a voice processing device having a voice acquisition unit, an imaging unit, a voice extraction unit, a determination unit, a generation unit, a display unit, a timing unit, and a character conversion unit. And
The voice acquisition unit acquires a plurality of voices;
The imaging unit captures a moving image together with the plurality of sounds; and
The voice extraction unit extracts a subject voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition step ;
The determination unit determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction step ;
The generating unit generates information indicating a delimiter associated with each of the subject's voice and the photographer's voice based on the delimiter determined in the determination step ;
The display unit displays information indicating a break generated by the generation step ;
The timekeeping unit is a timekeeping step of measuring times when the plurality of sounds are acquired;
The character conversion unit includes a character conversion step of converting the sound of the subject and the sound of the photographer into a character string corresponding to the sound,
In the generating step, a predetermined frame constituting the moving image associated with the break is generated as information indicating the break,
In the display step, the time measured by the timing step is displayed in association with the information indicating the break, and the character string is displayed in association with the information indicating the break.
And a voice processing method. An audio acquisition unit that acquires multiple audio from the computer ;
An imaging unit that captures a moving image together with the plurality of sounds,
A voice extraction unit that extracts a subject's voice and a photographer's voice emitted from a predetermined direction from a plurality of voices acquired by the voice acquisition unit ;
A determination unit that determines a separation between the sound of the subject and the sound of the photographer based on the volume of the sound of the subject and the sound of the photographer extracted by the sound extraction unit ;
A generating unit that generates information indicating a delimiter associated with each of the sound of the subject and the sound of the photographer based on the delimiter determined by the determination unit ;
A display unit for displaying information indicating a break generated by the generation unit ;
A timekeeping unit that times the time at which the plurality of sounds are acquired;
Functioning as a character conversion unit that converts the sound of the subject and the sound of the photographer into a character string corresponding to the sound;
The generation unit generates a predetermined frame constituting the moving image associated with the break as information indicating the break,
The display unit displays the time measured by the time measuring unit in association with the information indicating the separation, and displays the character string in association with the information indicating the separation;
A program characterized by that .

Images