JP2016092772A - Signal processor and signal processing method and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a signal processor capable of using a sound source acquired by a simple device as a sound source for wavefront synthesis.SOLUTION: The signal processor includes: an acquisition part that acquires a piece of audio information and plural multiplexed data multiplexed with at least a piece of time information; a sampling rate adjusting section that adjusts the sampling rate of the audio information on each of the plural multiplexed data; and an output section that outputs plural pieces of audio information after adjusting the sampling rate to the speakers different from each other depending on the audio information. This disclosure is applicable, for example, to signal processors etc.SELECTED DRAWING: Figure 6

Description

  The present disclosure relates to a signal processing device, a signal processing method thereof, and a program, and in particular, a signal processing device that enables a sound source acquired by a simple device to be used as a sound source for wavefront synthesis, and The present invention relates to the signal processing method and program.

  A wavefront synthesis technique is known as a sound field reproduction technique for collecting a sound wavefront of sound in a sound field with a plurality of microphones and reproducing the sound field based on the obtained sound pickup signal. One of the wavefront synthesis techniques is boundary sound field control (see, for example, Patent Document 1).

  In general, in the sound field reproduction method, a special microphone and a large number of microphones are required as a plurality of microphones for collecting a sound source.

JP 2012-10011 A

  In recent years, smartphones have been carried one by one, and many smartphones have a GPS (Global Positioning System) function for receiving their position and a function for recording video and audio.

  Therefore, it is desirable if the sound source necessary for wavefront synthesis can be easily recorded with a smartphone. At this time, the problem is the clock shift of each sound source obtained by a plurality of devices. In a smartphone, for example, video can be recorded at a sampling frequency of 30 fps and audio can be recorded at a sampling frequency of 48 kHz, but strictly speaking, the sampling frequency is slightly different for each device.

  In a dedicated machine for reproducing the sound field, sampling is performed based on an audio reference signal called Wordclock and a video reference signal called Genlock, so synchronization is not lost. However, in general consumer equipment other than dedicated machines, there is no such mechanism, and thus a clock shift occurs in signals obtained between the respective equipments.

  This indication is made in view of such a situation, and makes it possible to use the sound source acquired with simple equipment as the sound source of wave front composition.

  A signal processing device according to an aspect of the present disclosure includes an acquisition unit that acquires a plurality of multiplexed data in which audio information and time information are at least multiplexed, and a sampling rate of the audio information for each of the plurality of multiplexed data. A sampling rate adjusting unit to adjust; and an output unit that outputs the plurality of audio information after the sampling rate adjustment to different speakers for each audio information.

  In the signal processing method according to an aspect of the present disclosure, the signal processing device acquires a plurality of multiplexed data in which audio information and time information are multiplexed at least, and the audio information is sampled for each of the plurality of multiplexed data. The rate is adjusted, and the plurality of audio information after the sampling rate adjustment is output to different speakers for each audio information.

  According to one aspect of the present disclosure, a program includes: an acquisition unit configured to acquire a plurality of multiplexed data in which audio information and time information are at least multiplexed; and a sampling rate of the audio information for each of the plurality of multiplexed data And a sampling rate adjusting unit that adjusts the sampling rate and a plurality of the audio information after the sampling rate adjustment are functioned as an output unit that outputs to different speakers for each audio information.

  In one aspect of the present disclosure, a plurality of multiplexed data in which at least audio information and time information are multiplexed is acquired, and the sampling rate of the audio information is adjusted for each of the plurality of multiplexed data, and the sampling rate adjustment is performed. The plurality of later audio information is output to different speakers for each audio information.

  The program can be provided by being transmitted through a transmission medium or by being recorded on a recording medium.

  The signal processing device may be an independent device or an internal block constituting one device.

  According to one aspect of the present disclosure, a sound source acquired by a simple device can be used as a sound source for wavefront synthesis.

  Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

It is a block diagram showing an example of composition of an embodiment of an acoustic system concerning this indication. It is a figure explaining the boundary sound field control method. It is a block diagram which shows the detailed structural example of a recording device. It is a flowchart explaining the multiplexed data creation process by a recording device. It is a figure which shows the data arrangement example of multiplexed data. It is a block diagram which shows the detailed structural example of a reproducing | regenerating apparatus. It is a figure which shows the correspondence of audio information and time information. It is a flowchart explaining the multiplexed data reproduction | regeneration processing by a reproducing | regenerating apparatus. It is a figure explaining the acquisition method of the sound source data in this Embodiment. It is a figure which shows the example which utilizes multiplexed data for the speaker system of 5.1 channels. It is a block diagram showing an example of composition of a 1 embodiment of a computer concerning this indication.

<Configuration example of acoustic system>
FIG. 1 shows a configuration example of an embodiment of an acoustic system according to the present disclosure.

The acoustic system 1 shown in FIG. 1 is a system that reproduces the realistic sensation of sound using wavefront synthesis, and includes K (K> 1) speakers 10 _{1 to} 10 _K and M (M> 1) recording devices 11. _{1 to} 11 _M , a data storage device 12, M playback devices 13 _{1 to} 13 _M , and M speakers 14 _{1 to} 14 _M.

In the following, when it is not necessary to distinguish each of K loudspeakers 10 ₁ to 10 _K, simply described as the speaker 10, if it is not necessary to distinguish the M recording device 11 ₁ to 11 _M, respectively Is simply referred to as the recording device 11. The same applies to the _M playback devices 13 _{1 to} 13 _M and the M speakers 14 _{1 to} 14 _M.

  The K speakers 10 and the M recording devices 11 constitute a sound collection system 15 and are installed in a predetermined venue when collecting sound sources. On the other hand, the M playback devices 13 and the M speakers 14 constitute a playback system 16 and are installed at a predetermined venue when playing back a collected sound source. The data storage device 12 can be incorporated as a part of the sound collection system 15 or the reproduction system 16.

  With reference to FIG. 2, a boundary sound field control method, which is one of wavefront synthesis techniques for reproducing the realistic sensation of sound, will be described.

  In the boundary sound field control method, a microphone is placed in the reproduction space that reproduces the sound field in the same way as the original space where the sound source is collected, and the microphone observed when the signal observed by the microphone is collected in the original space. In this method, an input signal is given to a plurality of speakers installed around the reproduction space so as to be the same as the signal.

  Specifically, as shown in FIG. 2, a large number of recording devices 11 in which sound output from a large number of speakers 10 arranged in the vicinity of the venue A with the venue A as an original space are arranged in the venue. Sound is picked up.

  Then, in the venue B as the reproduction space, as shown in FIG. 2, when a large number of recording devices 11 are installed in the same arrangement as the original space, the audio signals observed in the respective recording devices 11 are When the playback device 13 outputs from the speaker 14 a sound that is the same as the signal collected in the original space, the sound field in the original space is reproduced in the playback space. The position of each recording device 11 in the original space and the reproduction space is called a control point in the boundary sound field control method.

In order to make the audio signal observed at the control point in the reproduction space the same as the signal collected in the original space, the transfer function (transfer characteristic) G from each speaker 14 to each recording device 11 in the reproduction space is set to The measurement is actually performed, and the playback device 13 calculates an inverse function filter H = G ⁻¹ that is the inverse characteristic of the transfer function G, and outputs the audio signal collected in the original space from the speaker 14 through the inverse function filter H.

  Returning to the description of FIG. 1, the sound collection system 15 is used when collecting a sound source in the original space or actually measuring the transfer function G in the reproduction space.

  Each of the K speakers 10 outputs a predetermined sound. Each of the M recording devices 11 collects sound output from the K speakers 10 and supplies an audio signal obtained as a result to the data storage device 12.

  The reproduction system 16 is used when reproducing a sound source collected in the original space in the reproduction space.

The reproducing device 13 calculates an inverse function filter H = G ⁻¹ that is an inverse characteristic of the transfer function G. Then, the playback device 13 performs a filtering process using the inverse function filter H on the audio signal acquired from the data storage device 12, and outputs the audio signal after the filtering process. The speaker 14 outputs a sound based on the audio signal supplied from the connected playback device 13.

For the calculation of the transfer function G of wavefront synthesis and its inverse function G- ¹ , in addition to the principle of boundary sound field control described above, an arbitrary calculation known as a wavefront synthesis technique such as the Kirchhoff-Holmhertz integral equation should be used. Can do. Hereinafter, the inverse function filter H of the transfer function G executed by the playback device 13 on the audio signal collected in the original space is referred to as a wavefront synthesis filter.

  The acoustic system 1 is configured as described above.

  Note that, in the sound collection system 15, the processing performed by the plurality of recording devices 11 may be performed by one signal processing device. Similarly, in the reproduction system 16, it is possible to adopt a configuration in which a single signal processing device performs processing performed by a plurality of reproduction devices 13. Further, the recording device 11 and the playback device 13 may have the function of the data storage device 12, and the data storage device 12 may be omitted.

  In general, a dedicated device designed for sound field reproduction is used as the recording device 11 of the acoustic system 1 that reproduces the sound field. However, in this embodiment, as described later, the recording device 11 is a smartphone. Consumer devices owned by such a large number of users can be used. When a consumer device is used as the recording device 11, a problem is a synchronization shift between the plurality of recording devices 11. Therefore, as will be described later, the playback device 13 of the acoustic system 1 according to the present disclosure has a function of correcting and reproducing a synchronization shift between the plurality of recording devices 11.

  Details of the recording device 11 and the playback device 13 will be described below.

<Configuration example of recording device>
FIG. 3 is a block diagram illustrating a detailed configuration example of the recording device 11.

  The recording device 11 includes a sound collection unit 31, an imaging unit 32, a GPS reception unit 33, a control unit 34, a data storage unit 35, and a communication unit 36.

  The sound collection unit 31 includes, for example, a microphone, acquires ambient sounds, and supplies an audio signal obtained as a result to the control unit 34.

  The imaging unit 32 is configured by, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like, captures a predetermined video, and supplies a video signal obtained as a result to the control unit 34.

  The GPS receiving unit 33 receives a GPS signal transmitted from a GPS satellite, and supplies position information and time information included in the received GPS signal to the control unit 34. The time information included in the GPS signal is time information based on an atomic clock and is highly accurate time information.

  The control unit 34 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and controls various operations of the recording apparatus 11 by executing various programs. Further, the control unit 34 multiplexes the audio signal supplied from the sound pickup unit 31, the video signal supplied from the imaging unit 32, the position information and time information supplied from the GPS receiving unit 33 in a predetermined format. Multiplexed data is generated and stored in the data storage unit 35. Further, the control unit 34 causes the generated multiplexed data to be output to the data storage device 12 (FIG. 1) via the communication unit 36.

  The data storage unit 35 is configured by a nonvolatile memory such as an EEPROM (Electronically Erasable and Programmable Read Only Memory), and stores a predetermined program and data. The data storage unit 35 stores, for example, a program executed by the control unit 34 and data supplied from the control unit 34.

  The communication unit 36 includes a communication module for connecting to a local area network (LAN) by wire or wireless, for example, and connects to the network to communicate with other devices. The communication unit 36 may perform communication by other communication methods regardless of wired or wireless, such as USB I / F (Inter Face) or Bluetooth (registered trademark) based on the USB (Universal Serial Bus) standard.

  The recording apparatus 11 having the above configuration can be configured by a portable information terminal such as a smartphone, for example.

<Multiplexed data creation process>
Next, the multiplexed data creation process by the recording device 11 will be described with reference to the flowchart of FIG. This process is executed, for example, when each of the M recording devices 11 is instructed to start recording a sound source.

  First, in step S <b> 11, the GPS receiving unit 33 receives a GPS signal, acquires time information included in the received GPS signal, and supplies the time information to the control unit 34. The control unit 34 encodes the supplied time information as start time information by a predetermined method and packetizes it.

  In step S <b> 12, the GPS receiving unit 33 receives a GPS signal, acquires position information and time information included in the received GPS signal, and supplies them to the control unit 34. The control unit 34 encodes the position information and time information supplied from the GPS reception unit 33 as GPS position information and GPS time information by a predetermined method and packetizes them.

  In step S <b> 13, the sound collection unit 31 acquires ambient sound and supplies an audio signal obtained as a result to the control unit 34. The control unit 34 acquires the audio signal supplied from the sound collection unit 31, encodes it as audio information by a predetermined method, and packetizes it.

  In step S <b> 14, the imaging unit 32 captures a predetermined video and supplies a video signal obtained as a result to the control unit 34. The control unit 34 acquires the video signal supplied from the imaging unit 32, encodes it as video information by a predetermined method, and packetizes it.

  For convenience of explanation, the processes in steps S12 to S14 are described as being performed in order, but in actuality, they are executed in parallel (simultaneously).

  In step S15, the control unit 34 determines whether or not the data acquisition is completed.

  If it is determined in step S15 that the data acquisition has not yet been completed, the process returns to step S12, and the above-described steps S12 to S15 are repeatedly executed.

  On the other hand, if it is determined in step S15 that the data acquisition is completed, the process proceeds to step S16, and the GPS receiving unit 33 receives the GPS signal and acquires time information included in the received GPS signal. It supplies to the control part 34. The control unit 34 encodes the supplied time information as end time information by a predetermined method and packetizes it.

  In step S17, the control unit 34 creates multiplexed data in which the start time information, the GPS position information, the GPS time information, the audio information, the video information, and the end time information packetized in each of the above steps are multiplexed. And stored in the data storage unit 35. Alternatively, instead of storing the created multiplexed data in the data storage unit 35, the control unit 34 outputs the multiplexed data to the data storage device 12 (FIG. 1) via the communication unit 36 and stores it in the data storage device 12. Also good.

<Data arrangement example of multiplexed data>
FIG. 5 shows a data arrangement example of multiplexed data created by the multiplexed data creation process of FIG.

  For example, start time information indicating the time when recording of the multiplexed data is started is arranged at the beginning of the multiplexed data, and end time information indicating the time when recording is ended is arranged at the end of the multiplexed data. A set of GPS time information, GPS position information, audio information, and video information is repeatedly arranged between the start time information and the end time information.

  Each information is composed of one or more packets corresponding to the data length. Here, one set of audio information and video information is in units of N samples (N sheets) determined in advance.

  The storage positions and order of GPS time information, GPS position information, audio information, and video information are merely examples, and the present invention is not limited to this example. As the file format of the multiplexed data, for example, a general-purpose file format such as MP4 or MPEG2-TS (Moving Picture Experts Group 2 Transport Stream) can be appropriately selected and adopted.

  In the example described above, for the time information, both the start time information and the end time information and the time information periodically acquired during signal acquisition are stored as recorded data. It is only necessary to store at least one of the start time information and the end time information, or the time information acquired periodically.

<Configuration example of playback device>
FIG. 6 is a block diagram illustrating a detailed configuration example of the playback device 13.

  The playback device 13 includes a data acquisition unit 51, a demultiplexer (DEMUX) 52, a sample rate converter 53, a wavefront synthesis filter 54, a D / A converter 55, an amplifier (AMP) 56, and a reference clock generation unit 57.

  The data acquisition unit 51 acquires multiplexed data of a predetermined file format recorded by the recording device 11 from the data storage device 12 and supplies it to the demultiplexer 52.

  The demultiplexer 52 separates the multiplexed data of a predetermined file format supplied from the data acquisition unit 51 for each data type (track), and decodes it by a decoding method corresponding to the encoding method at the time of saving the file. The demultiplexer 52 supplies GPS time information and audio information obtained as a result of decoding to the sampling rate converter 53, and supplies GPS position information obtained as a result of decoding to the wavefront synthesis filter 54.

  The sampling rate converter 53 functions as a sampling rate adjustment unit that adjusts the sampling rate of the audio information supplied from the demultiplexer 52. Specifically, the sampling rate converter 53 converts the sampling rate of the audio information supplied from the demultiplexer 52 into a reproduction rate fs supplied from the reference clock generation unit 57. Then, the sampling rate converter 53 supplies the audio information converted to the reproduction rate fs to the wavefront synthesis filter 54.

  GPS time information and audio information are supplied to the sampling rate converter 53. The GPS time information is associated with N samples of audio information as shown in FIG.

  For example, GPS time information T1 is associated with N-sample audio information 71, GPS time information T2 is associated with N-sample audio information 72, and N-sample audio information 73 is GPS time information T3 is associated.

  Here, it is assumed that the reproduction rate fs of the reproduction device 13 supplied from the reference clock generation unit 57 is 48 kHz, for example, and the sampling rate when the recording device 11 records the audio signal is also the same 48 kHz. However, when the recording device 11 is a consumer device such as a smartphone, the accuracy of the clock signal is not as high as that of a dedicated device, so the sampling rate at the time of recording is, for example, 47.998 kHz or 48.001 kHz. , Have some deviation.

When the sampling rate of the N-sample audio information 71 completely matches the playback rate fs, the N-sample audio information 71 and the playback rate fs include
(T2-T1) = N / fs
The relationship holds.

On the other hand, when the sampling rate of the recording device 11 is slower than 48 kHz,
(T2-T1)> N / fs
When the sampling rate of the recording device 11 is faster than 48 kHz,
(T2-T1) <N / fs
It becomes.

The sampling rate converter 53
fs_est = N / (T2-T1)
And the sampling rate fs_est at the time of recording the audio information 71 of N samples is calculated. Then, the sampling rate converter 53 converts the sampling rate fs_est of the N-sample audio information 71 into the reproduction rate fs supplied from the reference clock generation unit 57 and supplies it to the wavefront synthesis filter 54.

  The sampling rate converter 53 sequentially repeats the above processing for audio information in units of N samples. As a result, even when the sampling rate at the time of recording does not exactly match the reproduction rate fs, the sampling rate can be corrected to the reproduction rate fs and reproduced. Also, the sampling rate at the time of reproduction can be made to coincide with the reproduction rate fs among the plurality of reproducing devices 13.

  When the time information included in the multiplexed data is only the start time information and the end time information, the sampling rate converter 53 calculates the sampling rate fs_est at the time of recording from the start time information and the end time information. , Convert to playback rate fs.

  Returning to the description of FIG. 6, the sampling rate converter 53 supplies the audio information after converting the sampling rate to the reproduction rate fs to the wavefront synthesis filter 54. Further, the GPS position information corresponding to the audio information is also supplied from the demultiplexer 52 to the wavefront synthesis filter 54.

  The wavefront synthesis filter 54 calculates a transfer function G when the audio information supplied from the sampling rate converter 53 is recorded based on GPS position information corresponding to the audio information. Then, the wavefront synthesis filter 54 performs wavefront synthesis filter processing that is the inverse characteristic of the transfer function G on the audio information supplied from the sampling rate converter 53. The wavefront synthesis filter 54 supplies the filtered audio signal to the D / A converter 55.

  The D / A converter 55 converts the digital audio signal supplied from the wavefront synthesis filter 54 into an analog signal and supplies the analog signal to the amplifier 56. The wavefront synthesis filter 54 and the D / A converter 55 are also supplied with a clock signal having a reproduction rate fs from the reference clock generator 57. The wavefront synthesis filter 54 and the D / A converter 55 operate at the reproduction rate fs. I do.

  The amplifier 56 amplifies the analog audio signal supplied from the D / A converter 55 and outputs it to the speaker 14.

  The reference clock generation unit 57 generates a clock signal corresponding to the reproduction rate fs and supplies the clock signal to the sampling rate converter 53, the wavefront synthesis filter 54, and the D / A converter 55.

<Multiplexed data playback processing>
With reference to the flowchart of FIG. 8, the multiplexed data reproduction process by the reproduction apparatus 13 will be described. This process is executed, for example, when the reproduction apparatus 13 is instructed to reproduce multiplexed data.

  First, in step S <b> 41, the data acquisition unit 51 acquires multiplexed data of a predetermined file format from the data storage device 12 and supplies it to the demultiplexer 52.

  In step S42, the demultiplexer 52 separates the multiplexed data of a predetermined file format supplied from the data acquisition unit 51 for each data type (track) and decodes it by a decoding method corresponding to the encoding method at the time of file storage. To do. The GPS time information and audio information obtained as a result of decoding are supplied to the sampling rate converter 53, and the GPS position information is supplied to the wavefront synthesis filter 54.

  In step S43, the sampling rate converter 53 converts the sampling rate of the audio information supplied from the demultiplexer 52 into the reproduction rate fs supplied from the reference clock generation unit 57, and the audio information after the sampling rate conversion is wavefronted. This is supplied to the synthesis filter 54.

  In step S <b> 44, the wavefront synthesis filter 54 calculates the transfer function G when the audio information supplied from the demultiplexer 52 is recorded based on the GPS position information corresponding to the audio information supplied from the demultiplexer 52. .

  In step S <b> 45, the wavefront synthesis filter 54 performs a wavefront synthesis filter process with the inverse characteristic of the transfer function G on the audio information supplied from the sampling rate converter 53. The filtered audio signal is supplied to the D / A converter 55.

  In step S 46, the D / A converter 55 converts the digital audio signal supplied from the wavefront synthesis filter 54 into an analog signal and supplies the analog signal to the amplifier 56.

  In step S 47, the amplifier 56 amplifies the analog audio signal supplied from the D / A converter 55 and outputs it to the speaker 14. The speaker 14 outputs a sound corresponding to the analog audio signal supplied from the amplifier 56.

  The multiplexed data reproduction process is executed as described above.

  In the acoustic system 1 in FIG. 1, as described above, a smartphone or the like owned by many users can be used as the recording device 11. Each user's smartphone as the recording device 11 multiplexes the audio signal and the video signal together with the position information and time information acquired from the GPS signal, and records the multiplexed data in a predetermined file format.

  The playback device 13 acquires multiplexed data of a predetermined file format created by the recording device 11 and separates it into GPS time information, GPS position information, audio information, and video information. Then, the playback device 13 converts the sampling rate of the audio signal from the sampling rate fs_est at the time of recording to a predetermined playback rate fs based on the GPS time information included in the multiplexed data, and wavefronts the converted audio signal. Synthetic filter processing is performed and output from the speaker 14.

  Here, the time information included in the multiplexed data is not based on the counter clock in the recording device 11, but is time information acquired from the received GPS signal, and is highly accurate time information based on the atomic clock.

  The playback device 13 detects a deviation in the sampling rate of the recorded audio signal based on the highly accurate GPS time information included in the multiplexed data, converts the sampling rate to a predetermined playback rate fs, Output sound. Thus, even when consumer devices such as smartphones owned by a large number of users are used as the recording device 11, it is possible to synchronize the sound output between the plurality of playback devices 13, and the sound at the time of recording The field can be reproduced.

  That is, a sound source acquired with a simple device can be used as a sound source for wavefront synthesis.

  The multiplexed data of a predetermined file format created by the recording device 11 includes GPS position information indicating the position where the recording device 11 has acquired the sound source.

  The reproducing device 13 can calculate the transfer function G based on the GPS position information included in the multiplexed data, and can execute wavefront synthesis filter processing given by the inverse characteristic of the transfer function G.

  Thus, normally, when acquiring sound source data in the reproduction space, as described with reference to FIG. 2, the recording devices 11 are arranged in the same manner as in the original space, or the recording devices 11 whose characteristics have been measured are arranged. It was necessary to bring it to the site of the reclaimed space.

  However, according to the present embodiment, when the recording device 11 acquires the sound source data in the reproduction space, the GPS position information indicating the position of the recording device 11 can also be acquired. Therefore, as shown in FIG. Even if the sound collection positions (control points) are unevenly arranged, the transfer function G is calculated based on the distance to the plurality of speakers 14 based on the GPS position information, and the inverse characteristic is obtained. be able to. That is, it is possible to more easily calculate the transfer function G of the reproduction space and reproduce the sound field.

  In the above description, the example in which the multiplexed data created by the multiplexed data creation process is used for the wavefront synthesis process to reproduce the sound field has been described, but it may be used other than the wavefront synthesis process. For example, only a part of a plurality of multiplexed data created by the multiplexed data creation process can be used for a 5.1 channel speaker system.

  FIG. 10 shows an example in which multiplexed data created by multiplexed data creation processing is used in a 5.1 channel speaker system.

  Among the multiplexed data of the multiple sound collection positions created by the multiplexed data creation process, for example, as shown in FIG. 10, the multiplexed data acquired by the recording device 11A is used for the center speaker 91A. The multiplexed data acquired by the recording device 11B is allocated to the right front speaker 91B, the multiplexed data acquired by the recording device 11C is allocated to the right rear speaker 91C, and the multiplexed data acquired by the recording device 11D is Multiplexed data assigned to the left rear speaker 91D and acquired by the recording device 11E can be assigned to the left front speaker 91E for output.

  In the above description, the process of synchronizing the audio information between the plurality of playback devices 13 has been described. Similarly, for the video information, the sampling rate is controlled based on the time information included in the multiplexed data, so that the audio information Synchronized video can be displayed, and viewpoint interpolation of video acquired at a plurality of recording positions can also be performed.

  In the example described above, the recording device 11 recognizes its own accurate position information from the position information included in the GPS signal, but high-accuracy time information may be acquired by other methods. For example, even if the control unit 34 of the recording device 11 obtains highly accurate time information by accessing a NTP (Network Time Protocol) server that distributes accurate time information via the communication unit 36. Good.

  GPS satellites are positioning satellites operated by the United States, but transmitted from positioning satellites of other satellite positioning systems (GNSS) such as GLONASS (Global Orbiting Navigation Satellite System) by Russia and Galileo by EU. The received transmission signal may be received to acquire time information and position information.

  Each of the recording device 11 and the playback device 13 as signal processing devices for processing audio information and video information can be created as dedicated hardware, or causes a computer to execute a program for performing the above-described series of processing. It can also be realized. When the above-described series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing various programs by installing a computer incorporated in dedicated hardware.

  FIG. 11 is a block diagram illustrating a hardware configuration example of a computer that executes the above-described series of processing by a program.

  In a computer, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103 are connected to each other via a bus 104.

  An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

  The input unit 106 includes a keyboard, a mouse, a microphone, a camera, and the like. The output unit 107 includes a display, a speaker, and the like. The storage unit 108 includes a hard disk, a nonvolatile memory, and the like. The communication unit 109 includes a network interface or the like. The drive 110 drives a removable recording medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  In the computer configured as described above, the CPU 101 loads, for example, the program stored in the storage unit 108 to the RAM 103 via the input / output interface 105 and the bus 104 and executes the program. Is performed.

  In the computer, the program can be installed in the storage unit 108 via the input / output interface 105 by attaching the removable recording medium 111 to the drive 110. Further, the program can be received by the communication unit 109 and installed in the storage unit 108 via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In addition, the program can be installed in the ROM 102 or the storage unit 108 in advance.

  In the present specification, the steps described in the flowcharts are performed in parallel or in a call even if they are not necessarily processed in chronological order, as well as performed in chronological order according to the described order. It may be executed at a necessary timing such as when.

  In this specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .

  Embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure.

  For example, the present disclosure can take a configuration of cloud computing in which one function is shared by a plurality of apparatuses via a network and is jointly processed.

  In addition, each step described in the above flowchart can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Note that the effects described in this specification are merely examples and are not limited, and there may be effects other than those described in this specification.

In addition, this indication can also take the following structures.
(1)
An acquisition unit for acquiring a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, a sampling rate adjusting unit that adjusts a sampling rate of the audio information;
A signal processing apparatus comprising: an output unit that outputs the plurality of audio information after adjusting the sampling rate to a different speaker for each audio information.
(2)
The signal processing apparatus according to (1), wherein the sampling rate adjustment unit adjusts a sampling rate of the audio information based on the time information.
(3)
The multiplexed data includes position information,
(1) or (2) further comprising a filter unit that calculates a transfer function based on the position information and filters the audio signal after adjusting the sampling rate using a filter corresponding to the calculated transfer function. The signal processing apparatus as described.
(4)
The signal processing device according to any one of (1) to (3), wherein the time information is information acquired from a GPS signal.
(5)
The signal processing device according to any one of (1) to (3), wherein the time information is information acquired from an NTP server.
(6)
The signal processing apparatus according to any one of (1) to (5), wherein the position information is information acquired from a GPS signal.
(7)
The signal processing device according to any one of (1) to (6), wherein the time information is included for each audio information of a predetermined number of samples.
(8)
The signal processing device according to any one of (1) to (7), wherein the time information is start time information at which recording of the multiplexed data is started and end time information at which recording of the multiplexed data is ended. .
(9)
The sampling rate adjustment unit adjusts the sampling rate only for some of the multiplexed data acquired by the acquisition unit,
The signal processing apparatus according to any one of (1) to (8), wherein the output unit outputs the plurality of audio information after adjusting the sampling rate to different speakers for each audio information.
(10)
Obtaining a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, adjust the sampling rate of the audio information,
A signal processing method of a signal processing device that outputs the plurality of audio information after adjusting a sampling rate to a different speaker for each audio information.
(11)
Computer
An acquisition unit for acquiring a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, a sampling rate adjusting unit that adjusts a sampling rate of the audio information;
A program for causing a plurality of audio information after sampling rate adjustment to function as an output unit for outputting to a different speaker for each audio information.

  DESCRIPTION OF SYMBOLS 1 Audio system, 10 Speaker, 11 Recording apparatus, 12 Data storage apparatus, 13 Playback apparatus, 14 Speaker, 51 Data acquisition part, 52 Demultiplexer, 53 Sampling rate converter, 54 Wavefront synthesis filter, 56 Amplifier, 81 Speaker, 101 CPU, 102 ROM, 103 RAM, 106 input unit, 107 output unit, 108 storage unit, 109 communication unit

Claims (11)

An acquisition unit for acquiring a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, a sampling rate adjusting unit that adjusts a sampling rate of the audio information;
A signal processing apparatus comprising: an output unit that outputs the plurality of audio information after adjusting the sampling rate to a different speaker for each audio information. The signal processing apparatus according to claim 1, wherein the sampling rate adjustment unit adjusts a sampling rate of the audio information based on the time information. The multiplexed data includes position information,
The signal processing apparatus according to claim 1, further comprising: a filter unit that calculates a transfer function based on the position information and filters the audio signal after adjusting the sampling rate using a filter corresponding to the calculated transfer function. . The signal processing apparatus according to claim 1, wherein the time information is information acquired from a GPS signal. The signal processing device according to claim 1, wherein the time information is information acquired from an NTP server. The signal processing apparatus according to claim 1, wherein the position information is information acquired from a GPS signal. The signal processing device according to claim 1, wherein the time information is included for each audio information of a predetermined number of samples. The signal processing device according to claim 1, wherein the time information is start time information at which recording of the multiplexed data is started and end time information at which recording of the multiplexed data is ended. The sampling rate adjustment unit adjusts the sampling rate only for some of the multiplexed data acquired by the acquisition unit,
The signal processing apparatus according to claim 1, wherein the output unit outputs the plurality of audio information after adjusting the sampling rate to different speakers for each audio information. The signal processor
Obtaining a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, adjust the sampling rate of the audio information,
A signal processing method of outputting a plurality of the audio information after adjusting a sampling rate to a different speaker for each audio information. Computer
An acquisition unit for acquiring a plurality of multiplexed data in which at least audio information and time information are multiplexed;
For each of the plurality of multiplexed data, a sampling rate adjusting unit that adjusts a sampling rate of the audio information;
A program for causing a plurality of audio information after sampling rate adjustment to function as an output unit for outputting to a different speaker for each audio information.

Images