JP5392827B2 - Sound data processing device - Google Patents

Description

  The present invention relates to a sound data processing apparatus including a plurality of microphones.

A sound source separation technique is known in which sound is captured by a plurality of microphones and specific sound is extracted or suppressed (for example, Non-Patent Document 1). In recent years, this sound source separation technique has been widely applied to portable devices such as video cameras, mobile phones, and IC recorders.
For example, Patent Document 1 discloses a recording device for a video camera having a sound source separation function. This recording device synchronizes with the subject of the video camera to the subject, focuses the directional characteristic of the microphone on the subject, extracts the audio signal from the subject, and enables recording with high presence.
Patent Document 2 discloses an audio data editing apparatus that records sound of a plurality of microphones together with an image during moving image shooting, and performs sound source separation on the sound of the sound source displayed in the image after shooting.

  In order to maintain the sound source separation performance and resolution as described above, a certain number of microphones are required. In addition, there may be a need for a certain interval between the microphones. Furthermore, in order to expand the range of the analysis direction of the sound source, it is necessary to arrange the microphones not only in a plane but also in a three-dimensional manner. However, in a portable terminal such as a video camera or a mobile phone, it is difficult to install a plurality of microphones at ideal positions because there are only limited parts where microphones can be installed.

  Therefore, in a portable terminal having a movable part (for example, a case having a monitor part of a video camera, a case having a monitor part of a folding type or slide type mobile phone), a microphone is also arranged in the movable part. doing.

  When the microphone is arranged in the movable part, the microphone is displaced from the planned position due to the movement of the movable part, so that the sound source process cannot be appropriately performed, and the quality of the sound data after the sound source separation process may be deteriorated. Furthermore, it is assumed that the original voice is degraded. However, the techniques disclosed in Patent Documents 1 and 2 described above do not take into consideration that the position of the microphone (microphone position) changes during shooting.

Japanese Patent Laid-Open No. 5-308553 JP 2008-131168 A

Masato Togami, Akio Amano, Hiroshi Shinjo, Ryota Kamoshida, “Hearing Function of Human Symbiotic Robot EMIEW”, Japanese Society for Artificial Intelligence, pp. 59-64, 2005/10/14

The present invention has been made in view of the above-described problems, and an object of the present invention is to prevent deterioration of sound data after sound source separation even when the position of a microphone at the time of recording is different from a preset position.
Another object of the present invention is to enable sound source separation processing appropriately in an electronic device in which a microphone is installed in a movable part.

In order to achieve the above object, a sound data processing apparatus according to the first aspect of the present invention provides:
A first housing;
A second housing capable of changing its position relative to the first housing;
At least one microphone provided in the first housing;
At least one microphone provided in the second housing;
Microphone position specifying information acquiring means for acquiring information for specifying the positional relationship of the microphone;
Storage means for recording sound data captured from the microphone and microphone position specifying information acquired by the microphone position specifying information acquiring means on a recording medium;
It is characterized by providing.

The sound data processing device according to the second aspect of the present invention is:
A first housing;
A second housing capable of changing its position relative to the first housing;
At least two or more microphones provided in the first housing;
At least one microphone provided in the second housing;
Position determining means for determining whether or not a positional relationship between the first casing and the second casing is a predetermined positional relationship;
If the positional relationship between the first housing and the second housing is the same as the predetermined positional relationship, a microphone provided in the first and second housings is selected, and the first housing A microphone selecting means for selecting a microphone provided in the first housing when the positional relationship between the housing and the second housing is different from the predetermined positional relationship;
Microphone position specifying information acquiring means for acquiring information for specifying the positional relationship of the microphone selected by the microphone selecting means;
Storage means for recording sound data captured from the microphone selected by the microphone selecting means and microphone position specifying information acquired by the microphone position specifying information acquiring means on a recording medium;
It is characterized by providing.

Preferably,
The microphone position specifying information acquisition means further includes position detection means for detecting the positional relationship between the first casing and the second casing,
The microphone position specifying information includes information indicating the positional relationship between the first casing and the second casing.
It is characterized by that.

Also preferably,
Position detecting means for detecting the positional relationship between the first casing and the second casing;
Based on the positional relationship detected by the position detection means, a plurality of microphones including at least two or more microphones provided in the first casing and at least one or more microphones provided in the second casing. Microphone position acquisition means for obtaining the positional relationship of the microphone;
Further comprising
The microphone position specifying information includes information for specifying the positional relationship of the plurality of microphones acquired by the microphone position acquiring unit.
It is characterized by that.

More preferably,
The microphone position specifying information includes information for specifying timing when the positional relationship of the microphone has changed, or information for specifying the positional relationship of the microphone at the time of recording,
It is characterized by that.

More preferably,
The apparatus further comprises warning means for giving a warning when the positional relationship between the first housing and the second housing is different from the predetermined positional relationship.

More preferably,
Reading means for reading out sound data recorded on the recording medium by the sound data processing device;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Reproducing means for reproducing the sound data subjected to the sound source separation processing;
It is characterized by providing.

More preferably,
Reading means for reading out sound data recorded on the recording medium by the sound data processing device;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Reproducing means for reproducing the sound data subjected to the sound source separation processing;
With
On the basis of the information specifying the timing when the positional relationship of the microphone is changed on the recording medium by the sound data processing device, the sound data read by the reading means reaches the timing when the positional relationship of the microphone changes. , A microphone position acquisition means for obtaining a positional relationship of the plurality of microphones from the positional relationship;
Is further provided.

More preferably,
Reading means for reading out sound data recorded on the recording medium by the sound data processing device;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Sound source separation data storage means for storing sound data subjected to sound source separation processing by the sound source separation means;
It is characterized by providing.

More preferably,
Reading means for reading out sound data recorded on the recording medium by the sound data processing device;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Sound source separation data storage means for storing sound data subjected to sound source separation processing by the sound source separation means;
With
On the basis of the information specifying the timing when the positional relationship of the microphone is changed on the recording medium by the sound data processing device, the sound data read by the reading means reaches the timing when the positional relationship of the microphone changes. , A microphone position acquisition means for obtaining a positional relationship of the plurality of microphones from the positional relationship;
Is further provided.

More preferably,
It further comprises sound editing means for performing sound editing processing on sound data recorded on the recording medium or sound data subjected to sound source separation processing by the sound source separation means.

  ADVANTAGE OF THE INVENTION According to this invention, even if a movable part is moved during imaging | photography or recording in the portable apparatus which has the microphone installed in the movable part, it becomes possible to prevent deterioration of the sound data after sound source separation.

It is a block diagram which shows an example of a function structure of the portable apparatus which concerns on Embodiment 1 of this invention. (A) And (b) is a figure which shows the structural example on the external appearance of the portable apparatus which concerns on Embodiment 1 of this invention. It is a flowchart figure which shows the flow of the video recording process of the portable apparatus which concerns on Embodiment 1 of this invention. (A) is an example of the table which shows the reference position of a microphone, (b) is an example of the table which shows the relationship between the change of hinge angle (theta) and (phi), and the position of a microphone. It is a block diagram which shows an example of a function structure of the portable apparatus which concerns on Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the video recording process of the portable apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the example of the video recording data which concern on Embodiment 2 of this invention. It is a flowchart figure which shows the flow of the reproduction | regeneration edit process of the portable apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows an example of a function structure of the portable apparatus which concerns on Embodiment 3 of this invention. (A) And (b) is a figure which shows the structural example on the external appearance of the portable apparatus which concerns on Embodiment 3 of this invention. It is a flowchart figure which shows the flow of the video recording process of the portable apparatus which concerns on Embodiment 3 of this invention. (A) is a figure which shows the example of the correspondence of the switch discrimination | determination signal and the position of a microphone in the portable apparatus 901a, (b) is the correspondence of the switch discrimination | determination signal and the position of a microphone in the portable apparatus 901b. It is a figure which shows the example of. It is a block diagram which shows an example of a function structure of the portable apparatus which concerns on Embodiment 4 of this invention. It is a flowchart figure which shows the flow of the video recording process of the portable apparatus which concerns on Embodiment 4 of this invention. It is a figure which shows the example of the video recording data which concern on Embodiment 4 of this invention. It is a flowchart figure which shows the flow of the reproduction | regeneration edit process of the portable apparatus which concerns on Embodiment 4 of this invention.

  Hereinafter, an embodiment in which the present invention is applied to a portable device such as a folding cellular phone having a sound data processing function and a moving image photographing function will be described. Further, techniques for extracting or suppressing only specific sound sources are collectively referred to as sound source separation processing.

(Embodiment 1)
<Example of discriminating the attitude of a portable device having a movable part during recording and appropriately performing sound source separation processing according to the discriminated attitude>
In the present embodiment, in a portable device 101 including a main body and a movable portion, sound source separation processing is appropriately performed according to an angle formed by the main body portion and the movable portion, and sound data subjected to the sound source separation processing is displayed together with an image. The portable device 101 to be stored will be described.

  As shown in FIG. 1, the portable device 101 according to the present embodiment includes microphones 102 to 105, an ADC (Analog to Digital Converter) 106, a photographing unit 107, an input unit 108, an angle sensor 109, and a control unit 110. A storage unit 113, a display unit 114, a DAC (Digital to Analog Converter) 115, and a speaker 116.

  The microphones 102 to 105 collect sound and convert the collected sound into an analog signal. As will be described later, the microphones 102 to 104 are disposed in the main body, and the microphone 105 is disposed in the movable portion.

  The ADC 106 converts an analog signal of sound data input from the microphones 102 to 105 into a digital signal, and sends the digital signal to the control unit 110.

  The photographing unit 107 is installed in the main body, and is composed of a CCD (Charge Coupled Device) camera, a CMOS (Complimentary MOS) sensor, and the like, converts an image into an electric signal, and sends the image signal to the control unit 110.

  The input unit 108 includes a power switch, a recording or recording button, a button for designating an image zoom magnification, and a sound source separation process effect (sound source separation effect) such as sound source extraction and suppression. An instruction of various control processes such as zoom control and sound source separation process is received from the user, and the instruction information is sent to the control unit 110.

  The angle sensor 109 is composed of a rotary encoder, a gyro sensor, and the like, and is installed in a movable part including the display unit 114. The angle sensor 109 detects an angle formed by the main body unit and the movable unit (an angle of a hinge unit (hinge angle) described later) and sends the angle information to the control unit 110.

  The control unit 110 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The control unit 110 executes a program stored in the storage unit 113, executes the original function of the portable device 101, and also executes an operation as a sound data processing device. For example, the processing shown in the flowchart of FIG. Run.

  The control unit 110 functionally includes a sound source separation processing unit 111 and a microphone position acquisition unit 112.

The sound source separation processing unit 111 performs sound source separation processing on a plurality of sound data sent from the ADC 106 based on an instruction sent from the input unit 108. The sound source separation processing unit 111 performs sound source separation based on the actual values of the microphones 102 to 105 at that time based on the detection value of the angle sensor 109.
Note that the sound source separation performed by the sound source separation processing unit 111 can arbitrarily apply processing based on predetermined digital signal processing or other algorithms.

  The microphone position acquisition unit 112 is based on the hinge angle detected by the angle sensor 109 when the position of the movable unit is not the reference position (for example, θ = 90 °, φ = 0 ° in FIG. 2B). The positions of the microphones 102 to 105 are obtained by geometric calculation, and the information is sent to the sound source separation processing unit 111. Note that the method for obtaining the microphone position itself is arbitrary.

The storage unit 113 includes a ROM, a flash memory, an HDD (Hard Disk Drive), and the like. The image data captured from the imaging unit 107, the sound data after the sound source separation process, and the microphone reference position (the movable unit is at the reference position). Various information (sound source separation parameters) used in sound source separation processing such as information on the position of the microphone at the time, information obtained from the sound source separation processing unit 111, and the like are stored. In addition, a program or the like related to processing performed by the control unit 110 is stored.
The sound source separation parameter stored in advance in the storage unit 113 includes information on the reference position of the microphone, information on the sound source separation effect (sound source enhancement amount, ambient sound suppression amount, etc.), and sound source arrival direction range (within that range). The sound from the sound source located at (1) is emphasized or attenuated). For example, the range of the sound source arrival direction may be stored in advance in association with the stage of image zoom at the time of shooting.

  The display unit 114 includes an LCD (Liquid Crystal Display) or an organic EL display (organic Electro-Luminescence display), a driver, and the like, and displays various images such as an image captured by the imaging unit 107 and an imaging mode.

  When the DAC 115 outputs sound data, the DAC 115 converts the digital signal into an analog signal and sends the analog signal to the sound output unit 116.

  The sound output unit 116 includes an amplifier, a speaker, and the like, and outputs sound based on a signal from the DA converter 115.

Next, the mechanical configuration of the mobile device 101 will be described.
The portable apparatus 101 is comprised from the main-body part 201, the movable part 202, and the hinge part which connects these so that it may illustrate to Fig.2 (a), (b).
Microphones 102 to 105 and the photographing unit 107 are fixed to the main body 201, and the microphone 105 is disposed on the movable unit 202.
2A shows a portable device 101a in which the movable unit 202 opens and closes in one direction, and FIG. 2B shows a portable device 101b in which the movable unit 202 moves around two axes.

  In the portable device 101a shown in FIG. 2A, the hinge portion is configured by a uniaxial hinge mechanism, the angle sensor 109 is installed on the hinge portion, and measures an angle θ formed by the main body portion 201 and the movable portion 202. . The position of the microphone 105 installed on the movable unit 202 is obtained from the hinge angle θ.

  On the other hand, in the portable device 101b shown in FIG. 2 (b), the hinge portion is composed of a biaxial hinge mechanism. The biaxial hinge mechanism has axes (rotation axis x, rotation axis y) orthogonal to each other, and the movable portion 202 can rotate about the rotation axis x and can rotate about the rotation axis y. The angle sensor 109 detects an angle θ formed between the main body 201 and the movable part 202 around the rotation axis x and an angle φ formed between the main body 201 and the movable part 202 around the rotation axis y. The position of the microphone 105 installed on the movable part 202 is obtained from the hinge angles θ and φ.

  In the following description, the portable device 101b will be described for easy understanding.

  The microphones 102 to 104 are arranged in the main body unit 201, and the microphone 105 is arranged in the movable unit 202. Here, as shown in FIG. 2B, the optical axis of the photographing unit 107 is set as the z axis, and an orthogonal coordinate system including the x axis, the y axis, and the z axis perpendicular to each other is set. The positions of the microphones 102 to 105 in the three-dimensional space are defined by this orthogonal coordinate system. The position of the microphone 105 and the positional relationship between the microphones 102 to 105 change depending on the posture of the portable device 101b.

  Hereinafter, the recording process executed in the portable device 101b will be described. Note that operations as a general image capturing apparatus such as image capturing / storing are the same as those in the past, and thus the description will focus on sound source separation processing unique to the present embodiment.

  When the user inputs an instruction to start recording processing and sound source separation processing to the input unit 108, the input unit 108 sends the instructions to the control unit 110. In response to the instruction to start the recording process and the sound source separation process, the control unit 110 starts the recording process shown in the flowchart of FIG.

  First, the control unit 110 causes the photographing unit 107 to start capturing an image and causes the microphones 102 to 105 to start capturing sound. Further, the control unit 110 temporarily stores the data acquired from the imaging unit 107 and the microphones 102 to 105 in the RAM or the like of the control unit 110 (step S301).

  Subsequently, the control unit 110 reads the sound source separation parameter stored in advance in the storage unit 113 into a RAM or the like (step S302).

  Next, the controller 110 determines whether or not the hinge angles θ and φ sent from the angle sensor 109 are the same as the reference angle, that is, whether θ = 90 ° and φ = 0 ° (step). S303).

If the hinge angles θ and φ are both the same as the reference angle (step S303; No), the control unit 110 sends the information on the reference position of the microphone read in step S302 to the sound source separation processing unit 111, and the sound source separation processing. The unit 111 sets information on the reference position of the initial microphone as a sound source separation parameter (step S310).
Subsequently, the sound source separation processing unit 111 performs sound source separation processing on the sound data from the microphones 102 to 105 supplied via the ADC 106 based on the sound source separation parameter (step S307). For example, if the separation target sound source instructed by the user is the sound source at the focus position of the photographing unit 107 and the effect of the sound source separation processing is “extraction (emphasis)”, the control unit 110 transmits the microphones 102 to 105 via the ACDC 106. A sound source separation process for extracting sound from the focus position is performed on the supplied sound data.
The control unit 110 writes the image data and the sound data after the sound source separation process in the storage unit 113 (step S308).

Next, the control unit 110 determines whether there is an instruction to end the recording process (step S309). When there is no instruction to end the recording process (step S309; No), the control unit 110 determines whether or not the hinge angles θ and φ are changed (step 311).
When the hinge angles θ and φ are changed (step S311; Yes), the control unit 110 proceeds to step S303. When the hinge angles θ and φ are not changed (step S311; No), the control unit 110 performs the processing on the sound data. Subsequently, the same sound source separation process is performed (step S307).

  On the other hand, if at least one of the hinge angles θ and φ is not the reference angle at the start of recording (step S303; Yes), or after the start of recording, the movable unit 202 is operated to reduce at least the hinge angles θ and φ. When one side is no longer the reference angle (step S311; Yes, step S303; Yes), the control unit 110 displays on the display unit 114 that the movable unit 202 is not the reference position (step S304). For example, a message such as “The microphone position has been moved” is displayed on the display unit 114 during shooting.

Next, the microphone position acquisition unit 112 newly acquires the position of the microphone 105 at that time based on the values of the hinge angles θ and φ (step S305). For example, in the coordinate system of FIG. 2B, the reference position of the movable part is defined as θ = 90 ° and φ = 0 °, and the coordinates of the microphone 105 at the reference position are defined as (x0, y0, z0). Here, when the movable part is in the closed state, θ = 0 °, and when the movable part is in the state of FIG. 2B, θ = 90 °. The difference of the hinge angle θ when the movable part moves around the x axis while φ = 0 ° is expressed as Δθ = θ′−θ (θ: hinge angle before movement, θ ′: hinge angle after movement) ), The coordinates (x1, y1, z1) of the microphone 105 after movement are obtained as (x0, y0 · cos Δθ, y0 · sin Δθ). Further, the difference of the hinge angle φ when the movable part moves around the y axis while θ = 90 ° is expressed as Δφ = φ′−φ (φ: hinge angle before movement, φ ′: after movement) Assuming that (hinge angle), the coordinates (x2, y2, z2) of the microphone 105 after movement are obtained as (x0 · cosΔφ, y0, x0 · sinΔφ).
Thereafter, the microphone position acquisition unit 112 sends the acquired information on the new microphone position to the sound source separation processing unit 111, and the sound source separation processing unit 111 sets the received new microphone position as a sound source separation parameter (step S306).

The sound source separation processing unit 111 performs sound source separation processing based on the sound source separation parameter including the set microphone position (step S307). In the example described above, the sound from the focus position of the photographing unit 107 is extracted using the new positional relationship of the microphones 102 to 105.
The control unit 110 writes the image data and the sound data after the sound source separation process in the storage unit 113 (step S308).

  If an instruction to end the recording process is received (step S309: Yes), the recording process is ended.

  According to the present embodiment, in the portable device 101 having the movable part 202, when storing the sound data after the sound source separation process, even if the movable part 202 provided with the microphone moves from the reference position during recording or recording. By acquiring the movable part and the microphone position from the information of the angle sensor 109 and applying the microphone position to the sound source separation process, recording can be performed while maintaining the quality of the data after the sound source separation process. In addition, even if the movable part in which the microphone is installed moves during photographing, the photographer can be notified that the position of the movable part 202 is not the reference position.

  Although the embodiment has been described by taking the configuration shown in FIG. 2B as an example, in the configuration shown in FIG. 2A, φ = 0 may be set.

  In the above description, when the hinge angle θ or φ is other than the reference angle, the position of the microphone 105 is obtained by calculation processing in step S305. For example, the hinge angles θ and φ are determined with a predetermined resolution. , The position of the microphone 105 is obtained in advance, converted into a table as shown in FIG. 4B and stored in the storage unit 113. In step S302, this table is expanded on the RAM. In step S305, The position of the microphone 105 may be obtained by searching this table using the hinge angles θ and φ as keys.

  Note that the present invention is not limited to a foldable mobile phone, and can be applied to a recording apparatus having a movable portion or a general recording apparatus. For example, the present invention can be applied to a slide-type mobile phone, a mobile phone capable of selecting a plurality of usage styles, a video camera, an electronic camera, a movie, a PDA, a notebook computer, a wearable personal computer, a calculator, an electronic dictionary, and the like. In addition, the application is not limited to moving image shooting and recording, but can be applied to a TV phone, a telephone call, a voice recognition application, and the like.

  In the present embodiment, the portable unit 202 has been described with respect to the portable device including the two-axis hinge that can move around the two axes of the rotation axis x and the rotation axis y. However, the movable range is not limited to this. . Further, when the movable unit 202 is not at the reference position, the warning method is not limited to the message display on the monitor, but the blinking of the lamp installed in the casing, the vibration of the casing, A warning sound or the like may be used.

  In the present embodiment, four microphones are arranged as shown in FIGS. 2A and 2B. However, the number of microphones is two or more, and the microphones are the main body 201 and the movable part 202, respectively. If this is the case, the present invention is applicable.

(Embodiment 2)
<Example of sound source separation after shooting and during playback editing>
In the first embodiment, sound source separation processing is performed during recording / recording, and sound data subjected to sound source separation processing is recorded in the storage unit 113. However, during recording / recording, raw sound data is recorded and played back / edited. Sometimes sound source separation processing can be applied. The portable device 501 having such a configuration will be described below.

As illustrated in FIG. 5, the portable device 501 includes a microphone 102 to 105, an ADC 106, an imaging unit 107, an input unit 108, an angle sensor 109, a control unit 110, a storage unit 113, and a display unit 114. , DAC 115 and speaker 116.
Except for the control unit 110, it has the same functional configuration as the portable device 101 of the first embodiment. In addition, the configuration example of the appearance of the portable device 501 in the second embodiment is the same as that shown in FIG. Hereinafter, the control unit 110 having a functional configuration different from that of the first embodiment will be described.

The control unit 110 functionally includes a sound source separation processing unit 111, a microphone position acquisition unit 112, a reproduction / editing processing unit 511, and a timer unit 512. The sound source separation processing unit 111 and the microphone position acquisition unit 112 have the same functions as those in the first embodiment.
The controller 110 further determines whether or not the hinge angles θ and φ measured by the angle sensor 109 have been changed. Then, the information indicating the changed timing and the hinge angles θ and φ are associated with each other and stored in the storage unit 113.

  The reproduction editing processing unit 511 performs reproduction editing of image data and sound data stored in the storage unit 113, reproduction or writing of the sound data processed by the sound source separation processing unit 111, and the like. For example, the reproduction / edit processing unit 511 performs enhancement of a specific frequency band, volume conversion, and the like on the sound data stored in the storage unit 113 and the sound data processed by the sound source separation processing unit 111.

  The timer unit 512 measures an elapsed time from the start of the recording process. When the hinge angles θ and φ are changed, the control unit 110 causes the storage unit 113 to store the elapsed time measured by the time measuring unit 512 in association with information on the new hinge angles θ and φ.

  An example of the data structure of the recording data stored in the storage unit 113 is shown in FIG. The recorded data includes image data captured from the photographing unit 107, sound data captured from the microphones 102 to 105, sound source separation parameters such as a reference position of the microphone, hinge angle of the movable unit at the start of the recording process, φ (movable Part angle 1), the time when the hinge angles θ and φ are changed, and the changed hinge angle θ and φ (the angle 2 of the movable part,...).

  Hereinafter, an operation of a recording process and an operation of a reproduction / editing process performed by the portable device 501 will be described.

  When the user inputs an instruction to start recording processing to the input unit 108, the input unit 108 sends an instruction to start recording processing to the control unit 110. When receiving an instruction to start the recording process, control unit 110 starts the recording process shown in the flowchart of FIG.

  The control unit 110 causes the image capturing unit 107 to start capturing an image and the microphones 102 to 105 to start capturing sound, and temporarily stores the data captured from the image capturing unit 107 and the microphones 102 to 105 into the RAM of the control unit 110 or the like. Store (step S301).

  Subsequently, the control unit 110 writes the number of microphones and their reference positions in the three-dimensional space in the header area of the recording data stored in the storage unit 113 as shown in FIG. 7 (step S602).

  Next, the control unit 110 determines whether or not the hinge angles θ and φ sent from the angle sensor 109 are the same as the reference angle (step S303).

  If the hinge angles θ and φ are the same as the reference angle (step S303; No), the control unit 110 records the elapsed time measured by the time measuring unit 512 and the values of the hinge angles θ and φ detected by the angle sensor 109. Write in the header area of the data (step 605). Here, at the start of recording, the hinge angles θ and φ at the start are written (in the example of FIG. 7, elapsed time = 0 [sec], θ = 90 °, φ = 0 °).

  On the other hand, when the hinge angle θ or φ is not the reference angle (step S303; Yes), the control unit 110 displays on the display unit 114 that the movable unit 202 is not the reference position (step S304). Then, the control unit 110 writes the elapsed time measured by the time measuring unit 512 and the values of the hinge angles θ and φ detected by the angle sensor 109 in the header area of the recording data (step S605). For example, if the movable unit 202 is moved from the reference position 8 seconds after the start of photographing and the hinge angle θ is 45 °, the control unit 110 changes the position of the movable unit 202 as shown in FIG. The elapsed time (change time) and the new hinge angles θ and φ are written in the header of the recording data. Thereafter, when the position of the movable portion 202 is changed, “change of the movable portion position 3”, “position of the movable portion 4”, and the elapsed time (change time) and the hinge angles θ, φ are set in each case. Remember.

  When the writing to the header is completed, the control unit 110 writes the image data and the sound data in the storage unit 113 (step S606).

  Next, the control unit 110 determines whether there is an instruction to end the recording process (step S309). If there is no instruction to end the recording process (S309; No), the control unit 110 determines whether or not the hinge angles θ and φ are changed (step 311). When there is a change in the hinge angles θ and φ (step S311; Yes), the control unit 110 repeats the processing after step S303. If the hinge angles θ and φ are not changed (step S311; No), the image data and the sound data are continuously written in the storage unit 113 (step S606).

  When an instruction to end the recording process is received (step S309; Yes), the recording process ends.

  In this manner, the recording unit 113 stores image data and sound data at each time point after recording / recording start, and information that can specify the position of each microphone.

  Next, operations for reproducing and editing data stored in the storage unit 113 will be described with reference to FIG.

  When the user inputs instructions for selecting recording data, starting reproduction editing and starting sound source separation processing to the input unit 108, the input unit 108 sends the instructions to the control unit 110. When control unit 110 accepts the instruction, control unit 110 starts the processing shown in the flowchart of FIG.

  First, the control unit 110 selects recorded data stored in the storage unit 113 and starts reading necessary information (step S801). Subsequently, the control unit 110 calls parameters such as the number of microphones at the time of recording and their reference positions in the three-dimensional space from the header area of the recording data (step S802). Next, it is determined whether or not an instruction to perform sound source separation processing is given by the user (step S803).

  If no sound source separation instruction is given from the user (step S803; No), the stored sound data is called (step S812), and the sound data is reproduced as it is together with the image data (step S810). For example, when the user wants to play the recorded sound data as it is for a while after playing the data, in the case of monaural playback, one of a plurality of sound data stored in the storage unit 113 is played back, and in the case of stereo playback Plays two sound data. For example, sound data captured from the microphones 102 to 105 is reproduced.

  When an instruction to perform sound source separation processing is given from the user (step S803; Yes), the control unit 110 performs sound source separation processing using a plurality of sound data stored in the storage unit 113. First, the control unit 110 acquires sound source separation parameters corresponding to the sound source separation effect designated by the user (step S804). For example, when the user touches and designates a sound source existing in the screen with a touch panel or the like and instructs to extract only the sound source, the control unit 110 should extract from the position of the sound source existing in the screen. Get the range of arrival direction of the sound source.

  Next, the control unit 110 determines whether or not the elapsed time of the recording data being reproduced and edited has reached the change time when the position of the movable unit 202 has been changed (step S805). If the elapsed time reaches the change time (step S805; Yes), the microphone position is acquired based on the hinge angles θ203 and φ204 at that timing (step S806). At this time, when the hinge angle θ203 is 90 ° and φ204 is 0 ° as the reference position, if the hinge angle θ203 is 90 ° and φ204 is 0 °, a new microphone position needs to be acquired. There is no need to read the initial microphone position written in the header area of the recorded data. As in the first embodiment, the microphone position acquisition method may be obtained by geometric calculation based on the values of θ203 and φ204, or may be obtained from a table as shown in FIG. Then, the control unit 110 sends the obtained microphone position to the sound source separation processing unit 111, and the sound source separation processing unit 111 sets the received microphone position as a sound source separation parameter (step 807).

  On the other hand, if the elapsed time of the recording data being reproduced / edited has not reached the change time (step S805; No), the already determined microphone position is set as the sound source separation parameter (step S813).

  The sound source separation processing unit 111 reads all sound data captured from the microphones 102 to 105 stored in the storage unit 113 (step S808). Thereafter, the sound source separation processing unit 111 performs sound source separation on a plurality of sound data based on the sound source separation parameter (step S809).

  The reproduction editing processing unit 511 reproduces the sound data after the sound source separation process together with the stored image data, or stores the sound data after the sound source separation process in the storage unit 113 (step S810).

  Next, the control unit 110 determines whether or not there is an instruction to end reproduction editing by the user (step S811). If an instruction to end reproduction / editing is received (step S811; Yes), the reproduction / editing process is terminated. When the control unit 110 does not accept an instruction to end reproduction / editing (step S811; No), the processing from step S803 is repeated.

  According to the present embodiment, in the portable device 101 having the movable unit 202, when performing sound source separation processing at the time of playback and editing, even if the position of the movable unit 202 is different from the reference position, the stored elapsed time and microphone position By using, it becomes possible to prevent the quality deterioration of the sound data after the sound source separation processing. Further, at the time of shooting, the photographer can be notified that the position of the movable portion 202 is not the reference position.

  In this embodiment, when the movable unit 202 is moved during recording and recording, an example of storing the elapsed time and the values of the hinge angles θ203 and φ204 is given. However, the hinge angle θ203, The value of φ204 may be stored. Further, instead of storing the values of θ and φ at the time of recording and recording, the microphone position may be acquired from the values of θ and φ at the time of recording and recording, and the microphone position itself may be stored in the storage unit 113. This increases the file capacity when storing data, but it can save the microphone position acquisition processing when performing sound source separation processing during playback and editing, so it is effective when the calculation resources are limited and real-time performance is required. It is.

  In the second embodiment, the timing unit 512 measures the elapsed time from the start of the recording process. However, since it is only necessary to know when the hinge angle is changed in the stored sound data, for example, the date and time are measured. Also good.

  In the second embodiment, the example of the sound source separation effect is extraction of an arbitrary sound source. However, the present invention can be similarly implemented when suppressing an arbitrary sound source. Furthermore, the sound source separation effect of extraction and suppression can be designated not only for one arbitrary sound source but also for a plurality of sound sources.

  In addition, when editing the stored data after shooting, the edited data can be overwritten with a plurality of original audio data, or stored in a separate data area. Sometimes the user can decide. If the original audio data is left without being overwritten, the sound source separation effect can be edited any number of times. The saved sound data can be played back on a general video playback device (for example, a video format AVI (Audio Video Interleave) that can be played back on a PC, an AVCHD (Advanced Video Codec High Definition) that is a video camera recording format, etc. ) Can be stored together with the image data. If stored in such a general moving image recording format, the recorded data can be played back regardless of the playback device.

  As in the first embodiment, if there are two or more microphones, this embodiment can be applied, and the movable direction of the movable portion is the axis of the rotation axis x and the rotation axis y in FIG. The means for notifying that the movable part is not at the predetermined position at the time of shooting is not limited to the present embodiment.

(Embodiment 3)
<Example of recording simultaneously with sound source separation in a portable device having a switch discriminating mechanism that can discriminate only whether or not the movable part is at a predetermined position>
In the first and second embodiments, the portable device having the angle sensor has been described. However, it is assumed that the angle sensor cannot be installed in terms of cost, design, and the like.
In the third embodiment, a portable device 901 having a switch discrimination mechanism that only knows whether the position of the movable part is the reference position, performing sound source separation processing during recording or recording, and storing sound data subjected to sound source separation processing will be described. To do.

  As illustrated in FIG. 9, the portable device 901 includes microphones 102 to 105, an ADC 106, an imaging unit 107, an input unit 108, a switch determination mechanism 902, a control unit 110, a storage unit 113, and a display unit 114. And a DAC 115 and a speaker 116.

  In addition, the mobile device 901 of the present embodiment can take two configurations of the mobile device 901a and the mobile device 901b in appearance. The portable device 901a is a form used for a general video camera, a mobile phone with a moving image photographing function, or the like. The portable device 901a includes a main body unit 201 and a movable unit 202, and the portable device 901b includes a main body unit 205 and a movable unit 206. The movable units 202 and 205 are installed in the main body units 201 and 205 via a hinge mechanism. In the portable device 901 a, the microphones 102, 103, and 104 are installed in the main body unit 201, and the microphone 105 is installed in the movable unit 202. On the other hand, in the portable device 901b, the microphone 102 is installed in the main body 205, and the microphones 103, 104, and 105 are installed in the movable unit 206. As in the first embodiment, an orthogonal coordinate system is set based on the photographing unit 107, and the microphone position is determined based on this coordinate system.

  Hereinafter, a switch discrimination mechanism 902 that provides a characteristic function of the portable device 901 will be described.

  The switch discriminating mechanism 902 is a mechanism that can discriminate whether or not the position of the movable parts 202 and 206 is the reference position, and is configured by a micro switch or a magnet switch. The switch determination mechanism 902 sends a determination result (switch determination signal) to the control unit 110. For example, when the movable units 202 and 206 are at the reference position, the switch determination mechanism 902 sends a signal “1” to the control unit 110, and sends a signal “0” otherwise.

  In the portable device 901 having the switch determination mechanism 902, if the posture of the portable device 901 is changed, the positional relationship between the main body portions 201 and 205 and the movable portions 202 and 206 cannot be detected. Therefore, it is impossible to detect the microphone position of a microphone installed in a housing different from the reference imaging unit 107. In such a case, if sound source separation processing is performed using sound data acquired from a microphone whose microphone position cannot be detected, the quality of sound data after the sound source separation processing may be degraded. Hereinafter, a recording process in which the sound data captured from the microphone is not used for the sound source separation process will be described.

  When the user inputs an instruction to start recording processing and sound source separation processing to the input unit 108, the input unit 108 sends the instructions to the control unit 110. Upon receiving instructions for starting the recording process and starting the sound source separation process, the control unit 110 starts the recording process shown in the flowchart of FIG.

  First, the control unit 110 causes the photographing unit 107 to start capturing images and the microphones 102 to 105 to start capturing sound, and temporarily stores them in the RAM or the like of the control unit 110 (step S301).

  Subsequently, the control unit 110 calls the sound source separation parameter stored in advance in the storage unit 113 to the RAM or the like (step S302).

  Next, the control unit 110 determines the switch determination signal sent from the switch determination mechanism 902 (step S1103).

  When the switch determination signal is “1” (the position of the movable part is the reference position) (S1103; No), the control unit 110 sends the initial microphone position to the sound source separation processing unit 111, and the sound source separation processing unit 111 is initialized. The microphone position is set as a sound source separation parameter (step S310).

  On the other hand, when the switch determination signal is “0” (the position of the movable part is not the reference position) (step S1103; Yes), the control unit 110 displays on the display unit 114 that the movable parts 202 and 206 are not the reference position. (Step S304). For example, a message such as “Please set the microphone position to the reference position” is displayed on the display unit 114 during shooting.

  Next, the control unit 110 selects a microphone that captures sound data used for sound source separation processing based on the result of the switch determination mechanism 902 (step S1105). In the portable device 901a, the microphones 102 to 104 installed in the main body 201 including the photographing unit 107 do not change in the positional relationship with the photographing unit 107 and remain at the reference position of the microphone. On the other hand, when the movable unit 202 is moved from the reference position, the position of the microphone 105 is shifted from the initial microphone position. Therefore, when the switch determination signal = “1”, the control unit 110 selects the microphones 102 to 105, and when the switch determination signal = “0”, the control unit 110 selects the microphones 102 to 104.

  In the case of the portable device 901b, even if the movable unit 206 is moved, the positional relationship between the microphones 103 to 105 installed in the movable unit 206 and the photographing unit 107 does not change. On the other hand, the microphone 102 installed in the main body 205 is displaced from the reference position of the microphone because the positional relationship with the imaging unit 107 changes. Therefore, when the switch determination signal = “1”, the control unit 110 selects the microphones 102 to 105, and when the switch determination signal = “0”, the control unit 110 selects the microphones 103 to 105.

  Further, the control unit 110 obtains the microphone position of the selected microphone. A microphone for capturing sound data for sound source separation processing is obtained from the attitude of the housing. Accordingly, the correspondence relationship between the switch discrimination signal and the microphone position is set in a table as shown in FIG. 12A (in the case of the portable device 901a) or FIG. 12B (in the case of the portable device 901b), and the sound source separation process is performed in advance. It can be stored in the unit 111 or the storage unit 113. Control unit 110 obtains microphone position information using the value of the switch determination signal as a key.

  The control unit 110 sends the selected microphone and microphone position information to the sound source separation processing unit 111, and the sound source separation processing unit 111 sets the microphone position of the selected microphone as a sound source separation parameter (step S1106).

  The sound source separation processing unit 111 performs sound source separation on the sound data acquired from the microphone selected based on the sound source separation parameter (step S1107). That is, in the portable device 901a, sound data captured from other than the microphone 105 deviated from the initial microphone position is used, and in the portable device 901b, sound data acquired from other than the microphone 102 deviated from the initial microphone position is subjected to sound source separation processing. Use. As a result, quality deterioration of the sound data after the sound source separation can be prevented. The control unit 110 writes the image data and the sound data after the sound source separation process in the storage unit 113 (step S308).

  Next, the control unit 110 determines whether there is an instruction to end the recording process (step S309). When there is no instruction to end the recording process (step S309; No), the control unit 110 determines whether there is a change in the switch determination signal (step S1111). If there is a change in the switch determination signal (step S1111; Yes), the processing after step S1103 is repeated. If there is no change in the switch determination signal (step S1111; No), the sound source separation process is continuously performed on the sound data (step S1107), and the processed sound data is written in the storage unit 113 (step S308).

  When an instruction to end the recording process is received (step S309; Yes), the recording process ends.

  According to this embodiment, even if the movable parts 202 and 206 are moved during photographing, the sound source separation is performed by using an inexpensive switch discriminating mechanism that can discriminate whether or not the positions of the movable parts 202 and 206 are the reference positions. It is possible to prevent quality degradation of the processed sound data. Further, a warning or the like can be displayed on the monitor to prompt the user to return to the reference position.

  Note that the invention described in the present embodiment is not limited to the configuration example in appearance as shown in FIG. 10, and an information processing apparatus in which a plurality of microphones are installed in each of the main body unit and the movable unit. Applicable.

(Embodiment 4)
<Example of performing sound source separation at the time of playback and editing after shooting in a portable device having a switch discriminating mechanism that can discriminate only whether or not the movable part is at a predetermined position>
In the fourth embodiment, the portable device 1301 has a switch determination mechanism that can know only whether the position of the movable portion is the reference position, stores sound data captured by a plurality of microphones in the storage portion, and performs sound source separation processing during reproduction editing. Will be described.

As illustrated in FIG. 13, the portable device 1301 includes a microphone 102 to 105, an ADC 106, an imaging unit 107, an input unit 108, a switch determination mechanism 902, a control unit 110, a storage unit 113, and a display unit 114. And a DAC 115 and a speaker 116.
The control unit 110 has the same functional configuration as that of the second embodiment.
Except for the control unit 110, it has the same functional configuration as the portable device 901 of the third embodiment.
The external configuration of the portable device 1301 is the same as that shown in FIG.

  Hereinafter, recording processing performed by the portable device 1301 including the switch determination mechanism 902 will be described.

  When the user inputs an instruction to start recording processing to the input unit 108, the input unit 108 sends an instruction to start recording processing to the control unit 110. When control unit 110 receives an instruction to start the recording process, control unit 110 starts the process shown in the flowchart of FIG.

  The control unit 110 causes the image capturing unit 107 to start capturing an image and the microphones 102 to 105 to start capturing sound, and temporarily stores the data captured from the image capturing unit 107 and the microphones 102 to 105 into the RAM of the control unit 110 or the like. Store (step S301).

  Subsequently, the control unit 110 writes the number of microphones and their reference positions in the three-dimensional space in the header area of the recording data stored in the storage unit 113 (step S602).

  Next, the control unit 110 determines the switch determination signal sent from the switch determination mechanism 902 (step S1103). When the switch discrimination signal is “1” (step S1103; No), the time measured by the timer unit 512 and the switch discrimination signal value “1” are written in the header area of the recorded data as shown in FIG. 15 (step S1405). ).

  On the other hand, when the switch determination signal is “0” (step S1103; Yes), the control unit 110 causes the display unit 114 to display that the movable units 202 and 206 are not the reference position (step S304). Then, the control unit 110 writes the elapsed time measured by the time measuring unit 512 and the value “0” of the switch determination signal in the header area of the recording data (step S1405). For example, assuming that the movable unit 202 is moved from the reference position 8 seconds after the start of imaging, the control unit 110 calculates the elapsed time (change time) when the position of the movable unit 202 is changed as shown in FIG. The switch discrimination signal “0” is written in the header of the recording data.

  When the writing to the header is completed, the control unit 110 selects the microphone that has captured the sound data to be stored based on the value of the switch determination signal, similarly to step S1105 (step S1406).

  Then, the control unit 110 writes the image data and the sound data captured from the selected microphone in the storage unit 113 (step S1407).

  Next, the control unit 110 determines whether there is an instruction to end the recording process (step S309). When there is no instruction to end the recording process (S309; No), the control unit 110 determines whether or not there is a change in the switch determination signal (step S1111). If there is a change in the switch determination signal (step S1111; Yes), the processing after step S1103 is repeated. If there is no change in the switch determination signal (step S1111; No), the image data and the sound data are continuously written in the storage unit 113 (step S1407).

  When an instruction to end the recording process is received (step S309; Yes), the recording process ends.

  Next, operations for reproducing and editing data stored in the storage unit 113 will be described with reference to FIG.

  When the user inputs instructions for selecting recording data, starting reproduction editing and starting sound source separation processing to the input unit 108, the input unit 108 sends the instructions to the control unit 110. When control unit 110 accepts the instruction, control unit 110 starts the processing of the flowchart shown in FIG.

  First, the control unit 110 selects recorded data stored in the storage unit 113 and starts reading necessary information (step S801). Subsequently, the control unit 110 calls parameters such as the number of microphones at the time of recording and their reference positions in the three-dimensional space from the header area of the recording data (step S802). Next, it is determined whether or not an instruction to perform sound source separation processing is given by the user (step S803).

  If no sound source separation instruction is given from the user (step S803; No), the stored sound data is called (step S812), and the sound data is reproduced as it is together with the image data (step S810).

  When an instruction to perform sound source separation processing is given from the user (step S803; Yes), the control unit 110 performs sound source separation processing using a plurality of sound data stored in the storage unit 113. First, a sound source separation parameter corresponding to the type of sound source separation processing instructed by the user is acquired (step S804).

  Next, the control unit 110 determines whether or not the elapsed time of the recording data being reproduced and edited has reached the change time when the position of the movable unit 202 has been changed (step S805). When the elapsed time reaches the change time (step S805; Yes), the microphone position of the microphone that does not adversely affect the sound source separation, the switch determination signal recorded in the header area, and FIG. 12 (a) or FIG. 12 (b). (S1606). Then, the sound source separation processing unit 111 sets the obtained microphone position information as a sound source separation parameter (S1607).

  On the other hand, if the elapsed time of the recording data being reproduced / edited has not reached the change time (step S805; No), the already determined microphone position is set as the sound source separation parameter (step S813).

  Next, the sound data of the selected microphone is called from the storage unit 113 (S1608). Thereafter, the sound source separation processing unit 111 performs sound source separation on a plurality of sound data based on the sound source separation parameter (step S809).

  Thereafter, the same processing as that after step 810 in the second embodiment is performed.

  According to the present embodiment, in the portable device 1301 having the movable unit 202 and an inexpensive switch discriminating mechanism, when the sound source separation process is performed at the time of reproduction editing, the movable unit 202 is stored even if it is different from the reference position. By using the elapsed time and the microphone position, it is possible to prevent the quality deterioration of the sound data after the sound source separation process.

  In this embodiment, in the operation at the time of recording, all the sound data of the four microphones are always stored regardless of the position of the movable part. However, if the position of the movable part is not a predetermined position, the sound source separation is adversely affected. Since the input data of the given microphone is not used for sound source separation, it need not be stored. As a result, the capacity of stored data can be reduced.

101, 101a, 101b, 501, 901, 1301 ... portable device, 102, 103, 104, 105 ... microphone, 106 ... ADC, 107 ... shooting unit, 108 ... input unit, 109 ... angle sensor, 110 ... control unit, 111 ... sound source separation processing unit, 112 ... microphone position acquisition unit, 113 ... storage unit, 114 ... display unit, 115 ... DAC, 116 ... speaker, 201, 205 ... main body unit, 202, 206 ... movable unit, 203 ... rotation axis x Hinge angle θ formed by the main body part and the movable part around, 204... Hinge angle φ formed by the main body part and the movable part around the rotation axis y, 511... Playback editing processing part, 512.

Claims (11)

A first housing;
A second housing capable of changing its position relative to the first housing;
At least one microphone provided in the first housing;
At least one microphone provided in the second housing;
Microphone position specifying information acquiring means for acquiring information for specifying the positional relationship of the microphone;
Storage means for recording sound data captured from the microphone and microphone position specifying information acquired by the microphone position specifying information acquiring means on a recording medium;
A sound data processing apparatus comprising: A first housing;
A second housing capable of changing its position relative to the first housing;
At least two or more microphones provided in the first housing;
At least one microphone provided in the second housing;
Position determining means for determining whether or not a positional relationship between the first casing and the second casing is a predetermined positional relationship;
If the positional relationship between the first housing and the second housing is the same as the predetermined positional relationship, a microphone provided in the first and second housings is selected, and the first housing A microphone selecting means for selecting a microphone provided in the first housing when the positional relationship between the housing and the second housing is different from the predetermined positional relationship;
Microphone position specifying information acquiring means for acquiring information for specifying the positional relationship of the microphone selected by the microphone selecting means;
Storage means for recording sound data captured from the microphone selected by the microphone selecting means and microphone position specifying information acquired by the microphone position specifying information acquiring means on a recording medium;
A sound data processing apparatus comprising: The microphone position specifying information acquisition means further includes position detection means for detecting the positional relationship between the first casing and the second casing,
The microphone position specifying information includes information indicating the positional relationship between the first casing and the second casing.
The sound data processing device according to claim 1 or 2, characterized in that. The microphone position specifying information acquisition means includes:
Position detecting means for detecting the positional relationship between the first casing and the second casing;
Based on the positional relationship detected by the position detection means, a plurality of microphones including at least two or more microphones provided in the first casing and at least one or more microphones provided in the second casing. A microphone position obtaining means for obtaining a positional relationship of the microphone, and
The microphone position specifying information includes information for specifying the positional relationship of the plurality of microphones acquired by the microphone position acquiring unit.
The sound data processing device according to claim 1 or 2, characterized in that. The microphone position specifying information includes information for specifying timing when the positional relationship of the microphone has changed, or information for specifying the positional relationship of the microphone at the time of recording,
The sound data processing device according to any one of claims 1 to 4, characterized in that. Either one of claims 1 to 5, characterized in that the positional relationship between the first casing and the second casing further comprises a warning means for performing warning when different from the predetermined positional relationship 1 The sound data processing device according to the item. Reading means for reading out sound data recorded on a recording medium by the sound data processing device according to any one of claims 1 to 5 ,
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Reproducing means for reproducing the sound data subjected to the sound source separation processing;
A sound data processing apparatus comprising: Reading means for reading sound data recorded on a recording medium by the sound data processing device according to claim 5 ;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Reproducing means for reproducing the sound data subjected to the sound source separation processing;
With
The timing at which the sound data read by the reading means changes in the positional relationship of the microphone based on the information specifying the timing at which the positional relationship of the microphone has changed on the recording medium by the sound data processing device according to claim 5. Trigger position acquisition means for obtaining the positional relationship of the plurality of microphones from the positional relationship,
The sound data processing device further comprising: Reading means for reading out sound data recorded on a recording medium by the sound data processing device according to any one of claims 1 to 5 ,
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Sound source separation data storage means for storing sound data subjected to sound source separation processing by the sound source separation means;
A sound data processing apparatus comprising: Reading means for reading sound data recorded on a recording medium by the sound data processing device according to claim 5 ;
Microphone position specifying means for specifying a microphone positional relationship during recording based on microphone position specifying information recorded on the recording medium;
Sound source separation means for performing sound source separation processing on the sound data read by the reading means based on the positional relationship of the microphones specified by the microphone position specifying means;
Sound source separation data storage means for storing sound data subjected to sound source separation processing by the sound source separation means;
With
The timing at which the sound data read by the reading means changes in the positional relationship of the microphone based on the information specifying the timing at which the positional relationship of the microphone has changed on the recording medium by the sound data processing device according to claim 5. Trigger position acquisition means for obtaining the positional relationship of the plurality of microphones from the positional relationship,
The sound data processing device further comprising: Sound data recorded on the recording medium, or to claim 9 or 10, further comprising a sound editing means for performing a sound editing processing on sound data source separation process is performed by the sound source separation unit The sound data processing device described.

Images