WO2022168559A1 - 情報処理装置、情報処理方法及び情報処理プログラム - Google Patents
情報処理装置、情報処理方法及び情報処理プログラム Download PDFInfo
- Publication number
- WO2022168559A1 WO2022168559A1 PCT/JP2022/000919 JP2022000919W WO2022168559A1 WO 2022168559 A1 WO2022168559 A1 WO 2022168559A1 JP 2022000919 W JP2022000919 W JP 2022000919W WO 2022168559 A1 WO2022168559 A1 WO 2022168559A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information processing
- section
- point
- processing
- output control
- Prior art date
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 141
- 238000003672 processing method Methods 0.000 title claims description 6
- 238000001514 detection method Methods 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 216
- 230000005540 biological transmission Effects 0.000 claims description 22
- 230000003287 optical effect Effects 0.000 claims description 8
- 230000005236 sound signal Effects 0.000 claims description 4
- 238000000034 method Methods 0.000 description 75
- 230000008569 process Effects 0.000 description 53
- 238000004891 communication Methods 0.000 description 46
- 238000012544 monitoring process Methods 0.000 description 35
- 238000010586 diagram Methods 0.000 description 18
- 239000000872 buffer Substances 0.000 description 17
- 230000015654 memory Effects 0.000 description 16
- 238000005457 optimization Methods 0.000 description 14
- 230000006866 deterioration Effects 0.000 description 12
- 230000008859 change Effects 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 241001342895 Chorus Species 0.000 description 3
- HAORKNGNJCEJBX-UHFFFAOYSA-N cyprodinil Chemical group N=1C(C)=CC(C2CC2)=NC=1NC1=CC=CC=C1 HAORKNGNJCEJBX-UHFFFAOYSA-N 0.000 description 3
- 238000005562 fading Methods 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 240000007594 Oryza sativa Species 0.000 description 1
- 235000007164 Oryza sativa Nutrition 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 235000009566 rice Nutrition 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L2019/0001—Codebooks
- G10L2019/0007—Codebook element generation
- G10L2019/001—Interpolation of codebook vectors
Definitions
- the present disclosure relates to an information processing device, an information processing method, and an information processing program.
- the audio data to be reproduced has a discontinuous point with a different audio level, the discontinuous point becomes noise, which degrades the reproduction quality, such as outputting an offensive sound.
- discontinuities may occur at the data junctions. rice field.
- a technique for suppressing deterioration in reproduction quality at a discontinuity point by performing a fade process on audio data near the discontinuity point is known.
- continuous audio data includes silent intervals, such as when part of the audio data is lost during transmission.
- all the data may not be acquired and some of the audio data may be lost.
- Discontinuity points are generated at both ends of a silent section in which audio data is lost.
- the present disclosure proposes an information processing device, an information processing method, and an information processing program capable of suppressing deterioration in reproduction quality due to data loss during transmission.
- the information processing device includes a detection unit and a control execution unit.
- the detection unit detects a discontinuity point where the signal level of the input signal is discontinuous.
- the control execution unit performs predetermined control on a loss interval that is an interval between the first discontinuity point and the second discontinuity point detected by the detection unit.
- the predetermined control has a control start position at a point in time before the first discontinuous point by a first period, and a control end position at a point in time after the second discontinuous point by a second period.
- FIG. 1 is a diagram illustrating a configuration example of an information processing device according to a first embodiment of the present disclosure
- FIG. FIG. 3 is a diagram showing an overview of processing according to the first embodiment of the present disclosure
- FIG. 4 is a flowchart showing an example of processing according to the first embodiment of the present disclosure
- FIG. 7 is a diagram showing an outline of processing according to the second embodiment of the present disclosure
- FIG. FIG. 11 is a flow chart showing an example of processing according to the second embodiment of the present disclosure
- FIG. FIG. 11 is a diagram showing an overview of processing according to a third embodiment of the present disclosure
- FIG. FIG. 11 is a flow chart showing an example of processing according to the third embodiment of the present disclosure
- FIG. 12 is a diagram illustrating a configuration example of an information processing device according to a fourth embodiment of the present disclosure
- FIG. 12 is a diagram showing an overview of processing according to the fourth embodiment of the present disclosure
- FIG. FIG. 12 is a flowchart showing an example of processing according to the fourth embodiment of the present disclosure
- FIG. 11 is a diagram illustrating a configuration example of an information processing device according to a fifth embodiment of the present disclosure
- FIG. 12 is a diagram showing an outline of processing according to the fifth embodiment of the present disclosure
- FIG. FIG. 16 is a flowchart showing an example of processing according to the fifth embodiment of the present disclosure
- FIG. FIG. 13 is a diagram showing an outline of processing according to the sixth embodiment of the present disclosure
- FIG. FIG. 21 is a flow chart showing an example of processing according to the sixth embodiment of the present disclosure
- First Embodiment 1-1 Configuration of Information Processing Apparatus According to First Embodiment 1-2. Outline of processing according to first embodiment 1-3. Procedure of processing according to the first embodiment2. Second embodiment 2-1. Outline of processing according to second embodiment 2-2. Procedure of processing according to the second embodiment3. Third Embodiment 3-1. Outline of processing according to third embodiment 3-2. Procedure of processing according to the third embodiment;4. Fourth Embodiment 4-1. Configuration of Information Processing Apparatus According to Fourth Embodiment 4-2. Outline of processing according to fourth embodiment 4-3. Procedure of Processing According to Fourth Embodiment 4-4. Modified example of the fourth embodiment5. Fifth Embodiment 5-1.
- FIG. 1 is a diagram showing a configuration example of an information processing device 1 according to the first embodiment of the present disclosure.
- the information processing device 1 is a device that reproduces audio data acquired from an external device, such as headphones or TWS (True Wireless Stereo) earphones.
- the TWS earphone is an earphone in which left and right earphones are connected by various wireless communication methods.
- the information processing device 1 acquires audio data from an external device, for example, by wireless communication.
- various communication standards such as Bluetooth (registered trademark), BLE (Bluetooth (registered trademark) Low Energy), Wi-Fi (registered trademark), 3G, 4G, and 5G can be used as appropriate. .
- the external device is, for example, a device that wirelessly transmits various data such as music and video audio data.
- Devices such as smart phones, tablet terminals, PCs (Personal Computers), mobile phones, and PDA (Personal Digital Assistants) can be used as the external device.
- the external device performs signal processing such as encoding processing and modulation processing on the audio data, and transmits the processed audio data to the information processing device 1 .
- Audio data is transmitted from an external device to the information processing apparatus 1 for each frame (packet) containing a predetermined number of samples.
- the information processing device 1 may acquire audio data from an external device through wired communication. Further, the information processing device 1 may be configured integrally with an external device.
- the information processing device 1 includes a communication unit 2, a buffer 3, a signal processing unit 4, a buffer 5, a DA conversion unit 6 and a control unit 7.
- the communication unit 2 performs wireless communication with an external device and receives audio data from the external device.
- the communication section 2 outputs the received audio data to the buffer 3 .
- the communication unit 2 has, as a hardware configuration, a communication circuit conforming to a corresponding communication standard for wireless transmission.
- the communication unit 2 has a communication circuit compatible with the Bluetooth standard.
- the buffer 3 is a buffer memory that temporarily stores the audio data output from the communication unit 2.
- the signal processing unit 4 demodulates (decodes) the audio data temporarily stored in the buffer 3 for each frame containing a predetermined number of samples.
- the signal processing unit 4 also decodes the encoded data (audio data) in units of frames using a predetermined decoder.
- the signal processing unit 4 outputs the decoded frame-by-frame audio data to the buffer 5 .
- the signal processing unit 4 has, as a hardware configuration, a processor such as a DSP (Digital Signal Processor) and memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory).
- the processor loads the program stored in the ROM into the RAM and executes the loaded program (application), thereby realizing each function of the signal processing unit 4 .
- the signal processing unit 4 includes a CPU (Central Processing Unit), MPU (Micro-Processing Unit), FPGA (Field Programmable Gate Array) and other PLD (Programmable Logic Device), ASIC (Application Specific Integrated Circuit), and other processors.
- CPU Central Processing Unit
- MPU Micro-Processing Unit
- FPGA Field Programmable Gate Array
- PLD Programmable Logic Device
- ASIC Application Specific Integrated Circuit
- the buffer 5 is a buffer memory that temporarily stores the frame-by-frame audio data output from the signal processing unit 4 .
- the DA converter 6 is a circuit that converts the audio data (digital signal) temporarily stored in the buffer 5 into an analog signal and supplies the converted analog signal to an output device such as a speaker.
- the DA converter 6 also includes a circuit that changes the amplitude (signal level) of the analog signal supplied to an output device such as a speaker under the control of the controller 7 .
- changing the amplitude of the analog signal includes at least mute processing and fade processing of the analog signal (audio signal). Fade processing includes fade-in processing and fade-out processing.
- the control unit 7 controls operations of the information processing device 1 such as the communication unit 2, the signal processing unit 4, and the DA conversion unit 6.
- the control unit 7 has a processor such as a CPU and memories such as a RAM and a ROM as a hardware configuration.
- the processor loads the program stored in the ROM into the RAM, and executes the loaded program (application) to implement each function of the control unit 7 (sound skip monitoring unit 71 and output control unit 72). .
- the sound skipping monitoring unit 71 refers to the frame-by-frame audio data stored in the buffer 5, and performs sound skipping detection processing for monitoring the presence or absence of sound skipping due to audio data loss (packet loss).
- the skipping monitor 71 is an example of a detector.
- the output control unit 72 performs output control processing for changing the signal level of the output signal (analog signal) by the DA conversion unit 6 in response to the sound skipping detection by the sound skipping monitoring unit 71 .
- Output control processing includes fade-out processing, fade-in processing and mute processing.
- the fade-out process is a process of gradually lowering the signal level of the output signal from the DA converter 6 .
- the fade-in process is a process of gradually increasing the signal level of the output signal from the DA converter 6 .
- the muting process is a process of setting the signal level of the output signal from the DA converter 6 to zero.
- the output control section 72 is an example of a control execution section.
- the output control processing is not limited to fade-out processing, fade-in processing, and mute processing. For example, processing such as gradually fading out the volume and maintaining the volume after reaching a certain volume that is not zero may be performed.
- control unit 7 may have a processor such as an MPU, a DSP, a PLD such as an FPGA, or an ASIC in place of or in addition to the CPU as a hardware configuration.
- a processor such as an MPU, a DSP, a PLD such as an FPGA, or an ASIC in place of or in addition to the CPU as a hardware configuration.
- At least two of the buffer 3, the buffer 5, the memory of the signal processing unit 4, and the memory of the control unit 7 may be integrated.
- the buffers 3, 5, the memory of the signal processing unit 4, and the memory of the control unit 7 may each be composed of two or more memories.
- processor of the signal processing unit 4 and the processor of the control unit 7 may be integrated. Moreover, the processor of the signal processing unit 4 and the processor of the control unit 7 may each be composed of two or more processors.
- Information processing devices 1 such as headphones and TWS earphones that reproduce audio data acquired from an external device are required to reduce the size of the main body from the viewpoint of improving portability and reducing the burden on the user by reducing the weight and size. there is Therefore, such an information processing apparatus 1 is greatly restricted in terms of the size and number of circuit configurations such as a CPU, power consumption, antenna performance, and the like.
- part of the audio data may be lost due to the communication environment when acquiring the audio data from the external device, the processing speed of the audio data in the information processing device 1, and the like.
- the information processing apparatus 1 when the information processing apparatus 1 is configured as a mobile device and audio data is acquired from an external device by wireless audio transmission, the communication environment may suddenly deteriorate.
- part of the audio data acquired by the information processing apparatus 1 may be lost due to the processing speed of audio data transmission in the external device.
- the processing speed of audio data transmission may be reduced when a read error occurs in audio data to be transmitted in an external device, or due to delays in signal processing such as encoding processing and modulation processing.
- discontinuities occur at both ends of silent intervals where audio data is lost. If the audio data to be reproduced has discontinuous points with different audio levels, the discontinuous points become noise, resulting in deterioration of reproduction quality, such as output of harsh sounds.
- the present disclosure proposes an information processing device 1 capable of suppressing deterioration in reproduction quality due to data loss during transmission.
- FIG. 2 is a diagram showing an overview of processing according to the first embodiment of the present disclosure.
- the horizontal axis indicates time.
- a region hatched with slanting lines sloping to the right indicates a section in which no loss occurs in the input signal 801 (audio data) to the information processing apparatus 1 .
- areas (loss sections TL1 and TL2) not hatched with diagonal lines sloping to the right indicate sections in which a loss occurs in the input signal 801 to the information processing apparatus 1 .
- the height of the area hatched with diagonal lines sloping downward to the right schematically indicates the signal level of the input signal to the information processing apparatus 1 .
- each loss section TL1 and TL2 are discontinuous points where the signal level of the input signal is discontinuous.
- each loss interval TL1, TL2 is the interval between two discontinuities.
- the area hatched with dots schematically shows the output control 803 according to the embodiment.
- the output control unit 72 changes the signal level of the output signal with respect to the discontinuous points at both ends of the loss section TL1.
- predetermined control Specifically, as shown in FIG. 2, the output control unit 72 sets the control start position A11 at a point in time before the start position of the loss interval TL1 by a predetermined period (first period). Also, as shown in FIG. 2, the output control unit 72 sets the control end position A22 at a point in time after a predetermined period (second period) from the end position of the loss interval TL1. As shown in FIG. 2, the output control unit 72 performs output control 803 from the control start position A11 to the control end position A22.
- the output control unit 72 performs fade-out processing from the control start position A11 to the fade-out processing end position A12 by the DA conversion unit 6, as shown in FIG.
- the output control unit 72 preferably sets the control start position A11 such that the end position A12 of the fade-out process is the start position of the loss section TL1 or a time point before the start position.
- the section from the control start position A11 to the fade-out processing end position A12 is preferably shorter than or equal to the first period. Note that the fade-out process may end after the start position of the loss section TL1.
- the output control unit 72 performs fade-in processing from the fade-in processing start position A21 to the control end position A22 by the DA conversion unit 6, as shown in FIG.
- the output control unit 72 preferably sets the control end position A22 such that the fade-in process start position A21 is at the end position of the loss section TL1 or at a point after the end position.
- the section from the fade-in process start position A21 to the control end position A22 is preferably shorter than or equal to the second period. Note that the fade-in process may be started before the end position of the loss section TL1.
- the output control section 72 performs muting processing from the end position A12 of the fade-out processing to the start position A21 of the fade-in processing by the DA conversion section 6.
- the output control unit 72 controls the first period (control start position A11) is set. In addition, the output control unit 72 adjusts the second period (control end position A22 ).
- the output control 803 for the loss interval TL1 has been described with reference to FIG. 2, the output control unit 72 similarly performs the output control 803 for the loss interval TL2.
- the speed of signal level decrease in the fade-out process and the speed of signal level increase in the fade-in process are determined in advance and stored in the memory of the control unit 7, for example.
- FIG. 2 exemplifies a case where the rate of change in signal level is constant, the present invention is not limited to this.
- the change speed of the signal level may change during at least one of fade-out processing and fade-in processing. Also, it may be appropriately set by the user.
- FIG. 3 is a flowchart illustrating an example of processing according to the first embodiment of the present disclosure.
- the flow of FIG. 3 is started, for example, when audio data is received from an external device. Further, the flow of FIG. 3 ends, for example, when the reproduction of the audio data received from the external device ends, or when the power of the information processing device 1 is turned off.
- the sound skipping monitoring unit 71 determines whether or not sound skipping has been detected (S101). When it is not determined that the sound skipping is detected (S101: No), the sound skipping monitoring unit 71 repeats the processing of S101.
- the output control unit 72 performs fade-out processing on the discontinuous point at the start position of the lost section (sound skipping section) in which the skipping has been detected. (S102). Further, after the fade-out processing is completed, the output control unit 72 performs mute processing on the sound skip section.
- the output control unit 72 determines whether or not sound skipping is detected, that is, whether or not the sound skipping section (loss section) has ended (S103).
- the sound skip section is in units of packets (frames).
- the length of the sound skip section can be calculated in advance according to, for example, the audio data wireless transmission system or codec. Therefore, in this determination, whether or not skipping is detected may be determined in the same manner as in the processing of S101, or may be determined based on whether the calculated length has passed from the start position of the sound skipping section. may be When it is not determined that the sound skip section has ended (S103: No), the output control unit 72 continues the mute processing for the sound skip section.
- the output control unit 72 performs fade-in processing on the discontinuous point at the end position of the sound skip section. After that, the flow in FIG. 3 returns to the process of S101.
- the information processing apparatus 1 changes the signal level for the discontinuous points at both ends of the sound skip section (silent section) when it is determined that the sound skip has been detected. Perform output control processing.
- harsh sound skipping at discontinuous points caused by loss of audio data can be changed to mellow sound skipping with improved listening comfort.
- the information processing apparatus 1 according to the first embodiment it is possible to suppress deterioration in reproduction quality due to data loss during transmission.
- the information processing apparatus 1 performs fade processing (output control processing) on discontinuous points at both ends of each sound skip section (loss section TL1, TL2) for each sound skip section (loss section TL1, TL2).
- fade processing output control processing
- the information processing apparatus 1 can also perform a series of output control processes for continuously occurring sound skip intervals.
- the information processing device 1 according to the second embodiment has the same configuration as the information processing device 1 according to the first embodiment described with reference to FIG.
- FIG. 4 is a diagram illustrating an overview of processing according to the second embodiment of the present disclosure.
- the output control unit 72 sets the control start position A1 at a point in time before the start position of the loss interval TL1 by a predetermined period (first period), as in the first embodiment. . That is, when the loss interval TL1 is detected, the output control unit 72 sets the control start position A1 based on the start position of the detected loss interval TL1.
- the output control unit 72 performs Then, a series of output control 803 is performed.
- the mute period TM is determined in advance and stored in the memory of the control section 7 or the like.
- the time width of the mute section TM is 200ms.
- the mute period TM may be set to a desired period based on, for example, the type of codec, sampling rate, and the like.
- the output control unit 72 sets the control end position A2 at a point after the mute section TM1 (second period) from the end position of the loss section TL1, as indicated by the dashed arrow in FIG.
- loss section TL2 is detected from the end position of loss section TL1 until mute section TM1 (mute section TM) passes.
- the output control unit 72 sets a mute section TM2 (mute section TM) from the end position of the loss section TL2.
- the output control section 72 resets the mute section TM with the end position of the detected loss section as the starting point.
- the output control unit 72 resets the control end position A2 at a point after the mute section TM2 (second period) from the end position of the loss section TL2, as indicated by the solid arrow in FIG.
- the output control unit 72 detects the end of the loss section TL1.
- a control end position A2 is set at a point after the mute section TM1 (second period) from the position.
- mute sections TM1 and TM2 (mute sections TM) start from the end positions of the loss sections TL1 and TL2
- the mute intervals TM1, TM2 may start from the starting positions of the loss intervals TL1, TL2. In this way, when the loss interval TL1 is detected, the output control section 72 can also use the start position of the detected loss interval TL1 as the reference timing for various kinds of output control.
- the control end position A2 may be set at a point in time after a predetermined period (second period) from the end position of , that is, at a point in time after the mute section TM1.
- the output control unit 72 performs a series of output controls 803 (predetermined control) on the continuous loss sections TL1 and TL2 between the control start position A1 and the control end position A2.
- the output control 803 according to the second embodiment does not include fade processing. Therefore, the first period and the second period according to the second embodiment can be set shorter than the first period and the second period according to the first embodiment, respectively.
- the output control unit 72 performs mute processing (output control 803) at the control start position A1 by the DA conversion unit 6, as shown in FIG. Further, the output control unit 72 performs unmute processing (output control 803) at the control end position A2 by the DA conversion unit 6.
- FIG. Note that, as shown in FIG. 4, the output control unit 72 does not perform unmuting processing for the end position of the loss section TL1 when the loss section TL2 is detected in the mute section TM1. Similarly, when the loss section TL2 is detected in the mute section TM, the output control section 72 does not perform the unmuting process for the start position of the loss section TL2.
- FIG. 5 is a flowchart illustrating an example of processing according to the second embodiment of the present disclosure. Note that differences from the flow of processing according to the first embodiment shown in FIG. 3 will be mainly described here.
- the sound skipping monitoring unit 71 determines whether or not sound skipping has been detected (S201) in the same manner as in the process of S101 in FIG.
- the output control unit 72 performs mute processing on the discontinuous point at the start position of the loss interval (sound skipping interval) in which the sound skipping has been detected ( S202).
- the output control unit 72 determines whether or not the sound skip section has ended (S203) in the same manner as in the process of S103 in FIG.
- the output control section 72 determines whether or not the mute section has ended (S204).
- the flow of FIG. 5 returns to the process of S203.
- the output control unit 72 performs unmuting processing on the discontinuous point at the end position of the last sound skipping period included in the mute period. (S205). After that, the flow of FIG. 5 returns to the process of S201.
- the information processing apparatus 1 newly detects the next sound skip section when the next sound skip section is detected between the end position of the detected sound skip section and the end of the mute section.
- a series of output control processing is also performed on the skipped interval.
- FIG. 4 illustrates a case where a series of output controls are performed for two sound skipping intervals
- the information processing apparatus 1 performs three or more sound skipping intervals for a sound skipping interval included in a mute interval.
- the jump section is also subject to a series of output control processes. As a result, it is possible to reduce the number of discontinuous points, thereby suppressing deterioration in reproduction quality due to data loss during transmission.
- the output control according to the second embodiment does not include fade processing, which is generally more computationally expensive than mute processing. Therefore, according to the information processing apparatus 1 according to the second embodiment, in addition to the effects obtained in the first embodiment, it is possible to reduce the calculation cost related to the output control process. A reduction in calculation cost contributes to a reduction in the size and number of mounted circuit configurations and a reduction in power consumption.
- the information processing apparatus 1 that performs a series of muting processes (output control processes) for a plurality of consecutively occurring sound skip intervals was exemplified, but the present invention is not limited to this.
- the information processing apparatus 1 can also perform a series of fading processes (output control processes) for consecutively occurring multiple sound skip intervals in the same manner as the output control process of the first embodiment.
- the information processing apparatus 1 according to the third embodiment has the same configuration as the information processing apparatus 1 according to the first and second embodiments described with reference to FIG.
- FIG. 6 is a diagram showing an overview of processing according to the third embodiment of the present disclosure.
- the output control unit 72 sets a control start position A11 and a fade-out process end position A12 in the same manner as in the first embodiment.
- the output control unit 72 fades the fade section TL2 according to the loss section TL2 detected from the end position of the loss section TL1 until the mute section TM1 elapses, as in the second embodiment.
- a start position A21 of the IN process and a control end position A22 are set.
- the output control unit 72 sets the start position A21 (broken line) of the fade-in process at a point after the muting section TM1 (mute section TM) from the end position of the loss section TL1.
- the output control unit 72 starts the fade-in process at a point after the mute section TM2 (mute section TM) from the end position of the loss section TL2.
- Position A21 solid line
- the output control unit 72 performs a series of output controls 803 (predetermined control) for the continuous loss sections TL1 and TL2 between the control start position A11 and the control end position A22.
- FIG. 7 is a flowchart illustrating an example of processing according to the third embodiment of the present disclosure. Note that differences from the flow of processing according to the second embodiment shown in FIG. 5 will be mainly described here.
- the output control unit 72 changes the starting position of the lost section (sound skipping section) in which the skipping is detected.
- a fade-out process is performed on the discontinuous point (S302).
- the output control unit 72 determines whether or not the sound skip section has ended (S303) and whether or not the mute section has ended (S304) in the same manner as in the processes of S203 and S204 in FIG. do.
- the output control unit 72 performs fade-in processing on the discontinuous point at the end position of the last sound skipping period included in the mute period (S305). .
- the flow of FIG. 7 returns to the processing of S301.
- the information processing apparatus 1 according to the third embodiment performs the mute process in the information processing apparatus 1 according to the second embodiment, and also performs the muting process in the information processing apparatus 1 according to the first embodiment. Fade processing is performed in the same manner. As a result, the calculation cost can be reduced as compared with the information processing apparatus 1 according to the first embodiment, and the sound skipping can be more mellow and more comfortable than the information processing apparatus 1 according to the second embodiment.
- FIG. 8 is a diagram showing a configuration example of the information processing device 1 according to the fourth embodiment of the present disclosure. Here, differences from the configuration of FIG. 1 will be mainly described.
- the information processing apparatus 1 acquires metadata of the audio data from an external device in addition to the audio data. Further, when the signal processing unit 4 decodes the audio data, the information processing apparatus 1 may add metadata to the audio data.
- the metadata is, for example, type information of the audio data, importance information of the audio data, and the like.
- the audio data type information is, for example, information indicating whether the audio data is music data or video data. Also, the importance of audio data is, for example, information indicating whether or not it is a chorus of music. Also, the importance of audio data is not limited to the chorus, and may be information indicating a musical part, for example.
- the musical parts are, for example, an introduction, an A melody, a B melody, a chorus, an outro, and the like.
- the importance of audio data is, for example, information indicating the type of music, such as classical music or jazz.
- the importance of audio data is, for example, information indicating whether or not a moving image is a climax scene.
- the importance of audio data may be, for example, information indicating a part in a moving image.
- the part in the moving image indicates, for example, whether or not it is the dialogue of the main character.
- a part in a moving image indicates, for example, whether it is an environmental sound.
- the importance of audio data is assumed to be included in the metadata attached to the audio data, but this is not the only option.
- the importance of audio data may be obtained by searching the information processing apparatus 1 using the Internet or the like based on the type and name of the audio data, or the information processing apparatus 1 may obtain a reference regarding the importance in advance.
- the data may be stored in a table format, for example, and assigned by referring to the table.
- the reference data may be stored not in the information processing device 1 but in the cloud.
- the user may be allowed to set as appropriate.
- the processor of the control unit 7 further implements the metadata monitoring unit 73 by loading the program stored in the ROM into the RAM and executing the loaded program (application).
- the metadata monitoring section 73 is an example of an adjustment section.
- the metadata monitoring unit 73 acquires the type and importance of audio data from the signal processing unit 4.
- the metadata monitoring unit 73 determines the details of the output control 803 for the target loss section (skipping section) based on the type and importance of the acquired audio data.
- the metadata monitoring unit 73 supplies the content of the determined output control 803 to the output control unit 72 .
- the output control unit 72 performs output control processing according to the contents of the output control 803 supplied from the metadata monitoring unit 73.
- the type and importance of audio data are acquired from the signal processing unit 4 here, they may be acquired from a server outside the information processing apparatus 1, the cloud, or the like.
- FIG. 9 is a diagram showing an overview of processing according to the fourth embodiment of the present disclosure.
- FIG. 9 illustrates a case where output control 803a applying fade processing and output control 803b applying mute processing are executed.
- the metadata monitoring unit 73 determines the content of the output control 803 for the target loss section (skipping section) based on the acquired metadata of the audio data.
- the user can arbitrarily set which process is applied to which metadata (for example, type, importance), and is stored in advance in the memory of the control unit 7 or the like. do.
- the metadata monitoring unit 73 determines to apply fade processing to music and mute processing to dialogue. In this case, it is possible to realize output control processing such as improving the reproduction quality of music and reducing the loss of information amount of dialogue.
- FIG. 10 is a flowchart illustrating an example of processing according to the fourth embodiment of the present disclosure.
- differences from the processing flow according to the second embodiment shown in FIG. 5 or the processing flow according to the third embodiment shown in FIG. 7 will be mainly described.
- the metadata monitoring unit 73 receives the type and importance of the audio data from the signal processing unit 4. Get degrees.
- the metadata monitoring unit 73 determines the content of output control for the target loss section (skipping section) based on the type and importance of the acquired audio data.
- the metadata monitoring unit 73 supplies the content of the determined output control to the output control unit 72 (S402).
- the output control unit 72 performs output control processing according to the details of the output control supplied from the metadata monitoring unit 73 (S403).
- the processing of S403 is the same as the processing of S202 in FIG. 5 when the mute processing is applied. Further, the processing of S403 is the same as the processing of S302 in FIG. 7 when fade processing is performed.
- the output control unit 72 determines whether the sound skipping section has ended (S404), and determines whether the mute section has ended. It is determined whether or not (S405). When it is determined that the mute section has ended (S405: Yes), the output control section 72 controls the metadata supplied from the monitoring section 73 to the discontinuous point at the end position of the last sound skipping section included in the mute section. Output control processing is performed in accordance with the contents of the output control (S406). After that, the flow of FIG. 10 returns to the processing of S401.
- the information processing apparatus 1 that determines the content of the output control 803 for the target loss section (sound skipping section) based on the metadata of the audio data was exemplified, but the present invention is not limited to this.
- the metadata monitoring unit 73 can determine the rate of change (inclination angle) of the signal level in fade processing based on the metadata of the audio data.
- the change speed in the fade-out process and the change speed in the fade-in process may be the same or different.
- the user can arbitrarily set which rate of change is applied to which metadata, and is stored in advance in the memory of the control unit 7 or the like.
- the metadata monitoring unit 73 sets a large change speed for lines from the viewpoint of reducing the loss of the amount of information.
- the metadata monitoring unit 73 sets a small change speed from the viewpoint of reproduction quality.
- the output control unit 72 may perform fade-in processing in the processing of S406 when only one sound skip section is detected. can.
- the information processing apparatus 1 determines the details of the output control 803 for the target loss section (skipping section) based on the type and importance of the audio data.
- FIG. 11 is a diagram showing a configuration example of the information processing device 1 according to the fifth embodiment of the present disclosure. Here, differences from the configuration of FIG. 1 will be mainly described.
- the sound skipping monitoring unit 71 refers to the audio data output from the communication unit 2 stored in the buffer 3, and detects sound caused by loss of audio data (packet loss). Received packet monitoring processing for monitoring the presence or absence of skipping is further performed.
- the processor of the control unit 7 further implements the communication control unit 74 by loading the program stored in the ROM into the RAM and executing the loaded program (application).
- the communication control section 74 is an example of a control execution section.
- the communication control unit 74 sets a communication optimization period (third period).
- the communication control unit 74 executes a communication optimization process that controls so that lost audio data is not retransmitted during a communication optimization interval.
- FIG. 12 is a diagram illustrating an overview of processing according to the fifth embodiment of the present disclosure.
- the communication control section 74 acquires the control start position A1 and the control end position A2 set by the output control section 72 .
- the communication control unit 74 sets a communication optimization section TO (third period) shorter than the interval between the control start position A1 and the control end position A2.
- the communication control unit 74 executes communication optimization processing for controlling so that audio data is not retransmitted in the communication optimization section TO.
- a transmission method may be used in which the information processing device 1 transmits a request to resend the audio data in the lost section to an external device.
- the communication control unit 74 does not transmit a retransmission request for audio data to the external device even if the audio data is lost in the set communication optimization section TO.
- the communication control unit 74 transmits to the external device a request to stop the remaining number of transmissions for the communication optimization section TO.
- the communication control unit 74 may transmit to the external device that data from the current time to a predetermined time ahead is not required, depending on the length of the optimization interval TO.
- the predetermined time is determined in advance and stored in the memory of the control unit 7, for example.
- the predetermined time may be determined based on the metadata of the audio data (for example, the type and importance), as in the information processing apparatus according to the fourth embodiment, and may be appropriately set by the user.
- FIG. 13 is a flowchart illustrating an example of processing according to the fifth embodiment of the present disclosure. Note that differences from the flow of processing according to the second embodiment shown in FIG. 5 will be mainly described here.
- the output control unit 72 performs mute processing in the same manner as the process of S202 of FIG. ).
- the communication control unit 74 starts communication optimization processing (S503). 5
- the output control unit 72 determines whether or not the sound skip section has ended (S504) and whether or not the mute section has ended (S505). do.
- the communication control unit 74 ends the communication optimization process (S506).
- the output control unit 72 performs unmute processing (S406) in the same manner as in S205 of FIG. After that, the flow of FIG. 13 returns to the processing of S501.
- the information processing apparatus 1 performs communication optimization processing that does not retransmit lost audio data in output control for a target loss section (sound skip section).
- a target loss section sound skip section
- a process (PLC: Packet Loss Concealment) of interpolating the audio data of the lost section from the audio data before and after the skipped section may be executed.
- PLC Packet Loss Concealment
- the information processing device 1 according to the sixth embodiment has the same configuration as the information processing device 1 according to the fifth embodiment described with reference to FIG.
- the output control unit 72 detects the loss interval (sound skipping interval). ) is subjected to PLC by the signal processing unit 4 .
- the interval width for PLC is determined in advance and stored in the memory of the control unit 7, for example.
- the output control unit 72 performs output control processing on a section that could not be interpolated by PLC among the loss sections.
- FIG. 14 is a diagram illustrating an overview of processing according to the sixth embodiment of the present disclosure.
- the area hatched with diagonal lines rising to the right indicates the input signal 805 (audio data) interpolated by the PLC.
- the loss section TL1 includes a section TL1a interpolated by PLC and a section TL1b that could not be interpolated by PLC.
- the lost section TL2 includes a section TL2a interpolated by PLC and a section TL2b not completely interpolated by PLC.
- the output control unit 72 controls the control start position A11 and fade-out processing for the section TL1b of the loss section TL1 that could not be interpolated by the PLC in the same manner as in the third embodiment.
- the start position of the section TL1b according to the sixth embodiment corresponds to the start position of the loss section TL1 according to the third embodiment.
- the output control unit 72 sets the start position of the fade-in processing to the section TL2b of the loss section TL2 that could not be interpolated by the PLC in the same manner as in the third embodiment. A21 and control end position A22 are set.
- the output control unit 72 replaces the sections TL1b and TL2b of the loss sections TL1 and TL2, which could not be interpolated by the PLC, with the loss sections TL1 and TL2 according to the third embodiment. , and a series of output control 803 (predetermined control) is performed for the continuous sections TL1b and TL2b.
- FIG. 15 is a flowchart illustrating an example of processing according to the sixth embodiment of the present disclosure. Note that differences from the flow of processing according to the third embodiment shown in FIG. 7 will be mainly described here.
- the output control unit 72 determines whether or not the target sound skipping section is in a range that cannot be interpolated. Determine (S602). When it is not determined that the target sound skip section is in a range where interpolation is not possible (S602: No), the output control unit 72 performs PLC by the signal processing unit 4, and interpolates the audio data of the sound skip section. (S603). After that, the flow of FIG. 15 returns to the processing of S601.
- the output control unit 72 performs PLC by the signal processing unit 4, The audio data is interpolated for the range, that is, the interpolable range (S604). Further, the output control unit 72 performs fade-out processing on the discontinuous point at the start position of the section that could not be interpolated by the PLC, among the loss sections (sound skip sections) in which skipping was detected (S605).
- the output control unit 72 determines whether or not the sound skip section has ended (S606) and whether or not the mute section has ended (S607) in the same manner as in the processes of S303 and S304 in FIG. do.
- the output control unit 72 determines the end position of the last sound skip section that cannot be interpolated by the PLC, among the sound skip sections included in the mute section.
- a fade-in process is performed on the discontinuous point (S608). After that, the flow of FIG. 15 returns to the processing of S601.
- the information processing apparatus 1 interpolates the audio data by PLC for the interpolable range of the sound skip section when it is determined that the sound skip has been detected. Then, for the interpolable range of the sound skip section, output control processing is performed in the same manner as in each of the above-described embodiments.
- discontinuous points can be eliminated in sound skip sections that can be interpolated by PLC.
- the technology according to the sixth embodiment can be arbitrarily combined with the technology according to each of the above-described embodiments.
- the above lighting device may be configured to be able to reproduce audio data. In this case, it is not necessary to perform output control processing for both audio data and optical signals. It is also possible to perform the output control processing according to the above-described embodiments only on audio data, and to perform the output of the optical signal in conjunction with the output control of the audio data. As a result, an increase in processing cost can be suppressed even when output control is further performed on the optical signal.
- the output control processing according to each of the above-described embodiments is not limited to the lighting device, and may be applied to display control of, for example, an HMD (Head Mounted Display).
- HMD Head Mounted Display
- the output control process may be performed on at least one of the two discontinuous points that define the loss interval. In other words, the output control process does not have to be performed for either one of the discontinuous points of the start position and the end position of the loss interval.
- the information processing apparatus 1 includes a skipping monitoring unit 71 (detection unit) and an output control unit 72 (control execution unit).
- the sound skipping monitor 71 detects a discontinuity point where the signal level of the input signal 801 is discontinuous.
- the output control unit 72 performs output control 803 (predetermined control) on the loss interval TL1, which is the interval between the first discontinuity point and the second discontinuity point detected by the sound skipping monitoring unit 71.
- the information processing method executed in the information processing apparatus 1 detects a discontinuity point where the signal level of the input signal 801 is discontinuous, and detects the detected first discontinuity point and the detected second discontinuity point.
- the information processing program executed by the information processing apparatus 1 detects discontinuous points where the signal level of the input signal 801 is discontinuous, and detects the detected first discontinuous point and second discontinuous point.
- the computer is made to implement the output control 803 (predetermined control) for the loss interval TL1, which is the interval between .
- the output control 803 has a control start position A11 at the time point before the first discontinuity point by the first period, and the control end point at the time point after the second discontinuity point by the second period. It has a position A22.
- the information processing apparatus 1 can convert harsh sound skipping at discontinuous points caused by loss of audio data (input signal) into mellow sound skipping with improved listening comfort. In other words, according to the information processing device 1, it is possible to suppress deterioration in reproduction quality due to data loss during transmission.
- the output control 803 (predetermined control) is at least one of fade processing and mute processing.
- the information processing device 1 can suppress deterioration in reproduction quality due to data loss during transmission.
- the output control 803 (predetermined control) further includes non-retransmission processing of the input signal 801 (communication optimization processing).
- the information processing device 1 can suppress a decrease in data transfer efficiency due to retransmission of the input signal 801 from the external device.
- the input signal 801 includes metadata.
- the output control 803 (predetermined control) is at least one of fade processing and mute processing.
- the output control unit 72 performs at least one of fade processing and mute processing according to the metadata.
- the information processing apparatus 1 can implement appropriate control according to the data to be reproduced.
- the output control 803 (predetermined control) is fade processing.
- the information processing apparatus 1 further includes a metadata monitoring unit 73 (adjusting unit) that adjusts the lengths of the first period and the second period.
- the information processing apparatus 1 can realize control according to each of the viewpoint of reducing the loss of information amount and the viewpoint of reproduction quality, according to the data to be reproduced.
- the input signal 801 includes metadata. Also, the metadata monitoring unit 73 (adjusting unit) adjusts the lengths of the first period and the second period according to the metadata.
- the information processing apparatus 1 can realize control according to each of the viewpoint of reducing the loss of information amount and the viewpoint of reproduction quality, according to the data to be reproduced.
- the metadata includes at least type information and importance information of the input signal 801.
- the information processing apparatus 1 can implement appropriate control according to the data to be reproduced.
- the output control unit 72 (control execution unit) generates an input signal 805 of the interpolation interval TC, which is at least part of the loss interval TL1, based on the input signal 801 before and after the loss interval TL1. to interpolate.
- the information processing device 1 can eliminate discontinuous points in sound skipping sections that can be interpolated by PLC.
- the silent section caused by the output control 803 can be shortened even for the skipped section that cannot be completely interpolated by the PLC.
- control start position A11 is the end position of the interpolation interval TC.
- the information processing apparatus 1 can shorten the silent section caused by the output control 803 even for the skipped section that cannot be interpolated by the PLC.
- the input signal 801 is at least one of an audio signal and an optical signal.
- the input signal is audio data
- sound quality playback quality
- optical signal it is possible to suppress deterioration of illumination quality (reproduction quality) such as visual flickering due to data loss during transmission.
- the loss section TL1 is a section in which the input signal 805 is lost during wireless transmission.
- the present technology can also take the following configuration.
- a detection unit that detects a discontinuity point where the signal level of the input signal is discontinuous;
- a control execution unit that performs a predetermined control on a loss interval that is an interval between the first discontinuity point and the second discontinuity point detected by the detection unit;
- the predetermined control has a control start position at a point in time before the first discontinuous point by a first period, and a control end position at a point in time after the second discontinuous point by a second period.
- the predetermined control is at least one of fade processing and mute processing.
- the predetermined control further includes non-retransmission processing of the input signal.
- the input signal includes metadata; the predetermined control is at least one of fade processing and mute processing; The information processing apparatus according to any one of (1) to (3), wherein the control execution unit performs at least one of the fading process and the muting process according to the metadata.
- the predetermined control is fade processing, Further comprising an adjustment unit that adjusts the length of the first period and the second period, The information processing apparatus according to (1).
- the input signal includes metadata; The adjustment unit adjusts the lengths of the first period and the second period according to the metadata.
- the information processing device according to (5) above.
- (7) The information processing apparatus according to (4) or (6), wherein the metadata includes at least type information and importance information of the input signal.
- control execution unit interpolates the input signal in an interpolation interval that is at least a part of the loss interval, based on the input signal before and after the loss interval.
- the information processing device according to any one of .
- the information processing device wherein the control start position is the end position of the interpolation interval.
- the information processing apparatus according to any one of (1) to (9), wherein the input signal is at least one of an audio signal and an optical signal.
- the loss section is a section in which the input signal is lost in wireless transmission.
- the predetermined control has a control start position at a point in time before the first discontinuous point by a first period, and a control end position at a point in time after the second discontinuous point by a second period.
- the predetermined control has a control start position at a point in time before the first discontinuous point by a first period, and a control end position at a point in time after the second discontinuous point by a second period.
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Circuit For Audible Band Transducer (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
1.第1の実施形態
1-1.第1の実施形態に係る情報処理装置の構成
1-2.第1の実施形態に係る処理の概要
1-3.第1の実施形態に係る処理の手順
2.第2の実施形態
2-1.第2の実施形態に係る処理の概要
2-2.第2の実施形態に係る処理の手順
3.第3の実施形態
3-1.第3の実施形態に係る処理の概要
3-2.第3の実施形態に係る処理の手順
4.第4の実施形態
4-1.第4の実施形態に係る情報処理装置の構成
4-2.第4の実施形態に係る処理の概要
4-3.第4の実施形態に係る処理の手順
4-4.第4の実施形態の変形例
5.第5の実施形態
5-1.第5の実施形態に係る情報処理装置の構成
5-2.第5の実施形態に係る処理の概要
5-3.第5の実施形態に係る処理の手順
6.第6の実施形態
6-1.第6の実施形態に係る処理の概要
6-2.第6の実施形態に係る処理の手順
7.その他の実施形態
8.本開示に係る情報処理装置による効果
[1-1.第1の実施形態に係る情報処理装置の構成]
図1を用いて、第1の実施形態に係る情報処理装置1の構成例を説明する。図1は、本開示の第1の実施形態に係る情報処理装置1の構成例を示す図である。
外部装置から取得したオーディオデータを再生するヘッドホンやTWSイヤホンなどの情報処理装置1においては、携帯性の向上や軽量化及び小型化によるユーザへの負担低減の観点から本体サイズの抑制が要求されている。したがって、このような情報処理装置1には、CPUなどの搭載される回路構成の大きさや数、消費電力、アンテナ性能などの制約が大きい。
次に、図3を用いて、実施形態に係る処理の手順について説明する。図3は、本開示の第1の実施形態に係る処理の一例を示すフローチャートである。図3の流れは、例えば外部装置からオーディオデータを受信したときに開始される。また、図3の流れは、例えば、外部装置から受信されたオーディオデータの再生が終了したとき、あるいは情報処理装置1の電源がオフ状態とされたときに終了する。
第1の実施形態では、各音飛び区間(損失区間TL1,TL2)ごとに、各音飛び区間の両端部の不連続点に対してフェード処理(出力制御処理)を行う情報処理装置1を例示したが、これに限らない。情報処理装置1は、連続して発生する音飛び区間に対して、一連の出力制御処理を行うこともできる。
図4は、本開示の第2の実施形態に係る処理の概要を示す図である。
次に、図5を用いて、実施形態に係る処理の手順について説明する。図5は、本開示の第2の実施形態に係る処理の一例を示すフローチャートである。なお、ここでは、主として図3の第1の実施形態に係る処理の流れとの相違点を説明する。
第2の実施形態では、連続して発生する複数回の音飛び区間に対して、一連のミュート処理(出力制御処理)を行う情報処理装置1を例示したが、これに限らない。情報処理装置1は、連続して発生する複数回の音飛び区間に対して、第1の実施形態の出力制御処理と同様にして、一連のフェード処理(出力制御処理)を行うこともできる。
図6は、本開示の第3の実施形態に係る処理の概要を示す図である。
次に、図7を用いて、実施形態に係る処理の手順について説明する。図7は、本開示の第3の実施形態に係る処理の一例を示すフローチャートである。なお、ここでは、主として図5の第2の実施形態に係る処理の流れとの相違点を説明する。
上述の各実施形態では、出力制御処理において、フェード処理及びミュート処理のいずれかを行う情報処理装置1を例示したが、これに限らない。出力制御処理においては、フェード処理及びミュート処理のうちの適切な処理を、オーディオデータの内容に応じて適用することもできる。
図8を用いて、第4の実施形態に係る情報処理装置1の構成例を説明する。図8は、本開示の第4の実施形態に係る情報処理装置1の構成例を示す図である。なお、ここでは、主として図1の構成との相違点を説明する。
図9は、本開示の第4の実施形態に係る処理の概要を示す図である。図9は、フェード処理を適用する出力制御803aと、ミュート処理を適用する出力制御803bとが実行される場合を例示する。上述したように、メタデータ監視部73は、取得されたオーディオデータのメタデータに基づいて、対象の損失区間(音飛び区間)に対する出力制御803の内容を決定する。
次に、図10を用いて、実施形態に係る処理の手順について説明する。図10は、本開示の第4の実施形態に係る処理の一例を示すフローチャートである。なお、ここでは、主として図5の第2の実施形態に係る処理又は図7の第3の実施形態に係る処理の流れとの相違点を説明する。
なお、第4の実施形態では、オーディオデータのメタデータに基づいて、対象の損失区間(音飛び区間)に対する出力制御803の内容を決定する情報処理装置1を例示したが、これに限らない。メタデータ監視部73は、オーディオデータのメタデータに基づいて、フェード処理における信号レベルの変化速度(傾斜角度)を決定することができる。このとき、フェードアウト処理における変化速度と、フェードイン処理における変化速度とは、同じであってもよいし、異なっていても構わない。ここで、いずれのメタデータに対していずれの変化速度を適用するかは、ユーザが任意に設定可能であり、予め定められて制御部7のメモリなどに記憶されているとする。一例として、メタデータ監視部73は、オーディオデータの重要度がミュージカルを示す情報であるとき、セリフに対しては、情報量の損失を低減する観点から大きな変化速度を設定する。一例として、メタデータ監視部73は、オーディオデータの重要度が音楽であるとき、再生品質の観点から小さな変化速度を設定する。
上述の各実施形態では、出力制御処理を行う間においても外部装置から情報処理装置1にオーディオデータが送信され続ける場合を例示したが、これに限らない。出力制御処理により音飛び区間(損失区間)が存在しても再生品質の低下を抑制することができるため、損失したオーディオデータは使用されていない。そこで、本実施形態では、出力制御処理とともに、通信の最適化処理を行う情報処理装置1を説明する。
図11を用いて、第5の実施形態に係る情報処理装置1の構成例を説明する。図11は、本開示の第5の実施形態に係る情報処理装置1の構成例を示す図である。なお、ここでは、主として図1の構成との相違点を説明する。
図12は、本開示の第5の実施形態に係る処理の概要を示す図である。通信制御部74は、出力制御部72から設定された制御開始位置A1及び制御終了位置A2を取得する。通信制御部74は、図12に示すように、制御開始位置A1及び制御終了位置A2の間より短い通信の最適化区間TO(第3の期間)を設定する。通信制御部74は、通信の最適化区間TOに関するオーディオデータの再送が行われないように制御する通信の最適化処理を実行する。
次に、図13を用いて、実施形態に係る処理の手順について説明する。図13は、本開示の第5の実施形態に係る処理の一例を示すフローチャートである。なお、ここでは、主として図5の第2の実施形態に係る処理の流れとの相違点を説明する。
上述の各実施形態に係る情報処理装置1において、音飛び区間に対してその前後のオーディオデータから損失区間のオーディオデータを補間する処理(PLC:Packet Loss Concealment)が実行されてもよい。
図14は、本開示の第6の実施形態に係る処理の概要を示す図である。図14に示す例において、右上がりの斜線のハッチングが付された領域は、PLCにより補間された入力信号805(オーディオデータ)を示す。また、図14に示す例において、損失区間TL1は、PLCにより補間された区間TL1aと、PLCにより補間しきれなかった区間TL1bとを含む。同様に、損失区間TL2は、PLCにより補間された区間TL2aと、PLCにより補間しきれなかった区間TL2bとを含む。
次に、図15を用いて、実施形態に係る処理の手順について説明する。図15は、本開示の第6の実施形態に係る処理の一例を示すフローチャートである。なお、ここでは、主として図7の第3の実施形態に係る処理の流れとの相違点を説明する。
なお、上述の各実施形態では、入力信号がオーディオデータである場合を例示したが、これに限らない。上述の各実施形態に係る出力制御処理は、例えば照明装置などの光源の明暗処理に適用することもできる。つまり、入力信号としては、光源からの光信号を使用することもできる。この場合、視覚へのちらつきなど、照明品質(再生品質)の低下を抑制できるという効果が得られる。
情報処理装置1は、音飛び監視部71(検出部)と、出力制御部72(制御実行部)とを備える。音飛び監視部71は、入力信号801の信号レベルが不連続である不連続点を検出する。出力制御部72は、音飛び監視部71により検出された第1の不連続点及び第2の不連続点の間の区間である損失区間TL1に対して出力制御803(所定の制御)を行う。また、例えば情報処理装置1において実行される情報処理方法は、入力信号801の信号レベルが不連続である不連続点を検出し、検出された第1の不連続点及び第2の不連続点の間の区間である損失区間TL1に対して出力制御803(所定の制御)を行うことを含む。また、例えば情報処理装置1により実行される情報処理プログラムは、入力信号801の信号レベルが不連続である不連続点を検出し、検出された第1の不連続点及び第2の不連続点の間の区間である損失区間TL1に対して出力制御803(所定の制御)を行うことをコンピュータに実現させる。ここで、出力制御803は、第1の不連続点から第1の期間だけ前の時点に制御開始位置A11を有し、第2の不連続点から第2の期間だけ後の時点に制御終了位置A22を有する。
(1)
入力信号の信号レベルが不連続である不連続点を検出する検出部と、
前記検出部により検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行う制御実行部と
を備え、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理装置。
(2)
前記所定の制御は、フェード処理及びミュート処理のうち少なくとも一方である、前記(1)に記載の情報処理装置。
(3)
前記所定の制御は、前記入力信号の非再送処理をさらに含む、前記(2)に記載の情報処理装置。
(4)
前記入力信号は、メタデータを含み、
前記所定の制御は、フェード処理及びミュート処理のうち少なくとも一方であり、
前記制御実行部は、前記メタデータに応じて前記フェード処理及び前記ミュート処理のうち少なくとも一方を行う、前記(1)~(3)のいずれかに記載の情報処理装置。
(5)
前記所定の制御は、フェード処理であり、
前記第1の期間及び前記第2の期間の長さを調整する調整部をさらに備える、
前記(1)に記載の情報処理装置。
(6)
前記入力信号は、メタデータを含み、
前記調整部は、前記メタデータに応じて前記第1の期間及び前記第2の期間の長さを調整する、
前記(5)に記載の情報処理装置。
(7)
前記メタデータは、前記入力信号の種別情報及び重要度情報を少なくとも含む、前記(4)又は(6)に記載の情報処理装置。
(8)
前記制御実行部は、前記損失区間の前後の前記入力信号に基づいて、前記損失区間のうちの少なくとも一部の区間である補間区間の前記入力信号を補間する、前記(1)~(7)のいずれかに記載の情報処理装置。
(9)
前記制御開始位置は、前記補間区間の終了位置である、前記(8)に記載の情報処理装置。
(10)
前記入力信号は、オーディオ信号及び光信号のうちの少なくとも一方である、前記(1)~(9)のいずれかに記載の情報処理装置。
(11)
前記損失区間は、前記入力信号が無線伝送において損失した区間である、前記(1)~(10)のいずれかに記載の情報処理装置。
(12)
入力信号の信号レベルが不連続である不連続点を検出することと、
検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行うことと
を含み、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理方法。
(13)
コンピュータに、
入力信号の信号レベルが不連続である不連続点を検出することと、
検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行うことと
を実現させ、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理プログラム。
2 通信部
3 バッファ
4 信号処理部
5 バッファ
6 DA変換部
7 制御部
71 音飛び監視部(検出部)
72 出力制御部(制御実行部)
73 メタデータ監視部(調整部)
74 通信制御部(制御実行部)
Claims (13)
- 入力信号の信号レベルが不連続である不連続点を検出する検出部と、
前記検出部により検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行う制御実行部と
を具備し、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理装置。 - 前記所定の制御は、フェード処理及びミュート処理のうち少なくとも一方である、請求項1に記載の情報処理装置。
- 前記所定の制御は、前記入力信号の非再送処理をさらに含む、請求項2に記載の情報処理装置。
- 前記入力信号は、メタデータを含み、
前記所定の制御は、フェード処理及びミュート処理のうち少なくとも一方であり、
前記制御実行部は、前記メタデータに応じて前記フェード処理及び前記ミュート処理のうち少なくとも一方を行う、請求項1に記載の情報処理装置。 - 前記所定の制御は、フェード処理であり、
前記第1の期間及び前記第2の期間の長さを調整する調整部をさらに備える、
請求項1に記載の情報処理装置。 - 前記入力信号は、メタデータを含み、
前記調整部は、前記メタデータに応じて前記第1の期間及び前記第2の期間の長さを調整する、
請求項5に記載の情報処理装置。 - 前記メタデータは、前記入力信号の種別情報及び重要度情報を少なくとも含む、請求項4に記載の情報処理装置。
- 前記制御実行部は、前記損失区間の前後の前記入力信号に基づいて、前記損失区間のうちの少なくとも一部の区間である補間区間の前記入力信号を補間する、請求項1に記載の情報処理装置。
- 前記制御開始位置は、前記補間区間の終了位置である、請求項8に記載の情報処理装置。
- 前記入力信号は、オーディオ信号及び光信号のうちの少なくとも一方である、請求項1に記載の情報処理装置。
- 前記損失区間は、前記入力信号が無線伝送において損失した区間である、請求項1に記載の情報処理装置。
- 入力信号の信号レベルが不連続である不連続点を検出することと、
検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行うことと
を含み、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理方法。 - コンピュータに、
入力信号の信号レベルが不連続である不連続点を検出することと、
検出された第1の不連続点及び第2の不連続点の間の区間である損失区間に対して所定の制御を行うことと
を実現させ、
前記所定の制御は、前記第1の不連続点から第1の期間だけ前の時点に制御開始位置を有し、前記第2の不連続点から第2の期間だけ後の時点に制御終了位置を有する、
情報処理プログラム。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202280011959.8A CN116888667A (zh) | 2021-02-03 | 2022-01-13 | 信息处理设备、信息处理方法和信息处理程序 |
US18/262,838 US20240096333A1 (en) | 2021-02-03 | 2022-01-13 | Information processing device, information processing method, and information processing program |
JP2022579413A JPWO2022168559A1 (ja) | 2021-02-03 | 2022-01-13 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021015786 | 2021-02-03 | ||
JP2021-015786 | 2021-02-03 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022168559A1 true WO2022168559A1 (ja) | 2022-08-11 |
Family
ID=82741283
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2022/000919 WO2022168559A1 (ja) | 2021-02-03 | 2022-01-13 | 情報処理装置、情報処理方法及び情報処理プログラム |
Country Status (4)
Country | Link |
---|---|
US (1) | US20240096333A1 (ja) |
JP (1) | JPWO2022168559A1 (ja) |
CN (1) | CN116888667A (ja) |
WO (1) | WO2022168559A1 (ja) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10135935A (ja) * | 1996-10-31 | 1998-05-22 | Sharp Corp | データ通信装置 |
JP2004064390A (ja) * | 2002-07-29 | 2004-02-26 | Matsushita Electric Ind Co Ltd | パケット補間装置 |
JP2006042210A (ja) * | 2004-07-29 | 2006-02-09 | Victor Co Of Japan Ltd | 光無線受信装置 |
JP2006135872A (ja) * | 2004-11-09 | 2006-05-25 | Sharp Corp | 受信装置及び無線通信システム |
JP2010282699A (ja) * | 2009-06-05 | 2010-12-16 | Renesas Electronics Corp | 外部音声入力装置及びそのミュート制御方法 |
JP2017513072A (ja) * | 2014-03-19 | 2017-05-25 | フラウンホーファー−ゲゼルシャフト・ツール・フェルデルング・デル・アンゲヴァンテン・フォルシュング・アインゲトラーゲネル・フェライン | 適応型ノイズ推定を使用してエラー隠し信号を生成する装置及び方法 |
-
2022
- 2022-01-13 US US18/262,838 patent/US20240096333A1/en active Pending
- 2022-01-13 WO PCT/JP2022/000919 patent/WO2022168559A1/ja active Application Filing
- 2022-01-13 CN CN202280011959.8A patent/CN116888667A/zh active Pending
- 2022-01-13 JP JP2022579413A patent/JPWO2022168559A1/ja active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10135935A (ja) * | 1996-10-31 | 1998-05-22 | Sharp Corp | データ通信装置 |
JP2004064390A (ja) * | 2002-07-29 | 2004-02-26 | Matsushita Electric Ind Co Ltd | パケット補間装置 |
JP2006042210A (ja) * | 2004-07-29 | 2006-02-09 | Victor Co Of Japan Ltd | 光無線受信装置 |
JP2006135872A (ja) * | 2004-11-09 | 2006-05-25 | Sharp Corp | 受信装置及び無線通信システム |
JP2010282699A (ja) * | 2009-06-05 | 2010-12-16 | Renesas Electronics Corp | 外部音声入力装置及びそのミュート制御方法 |
JP2017513072A (ja) * | 2014-03-19 | 2017-05-25 | フラウンホーファー−ゲゼルシャフト・ツール・フェルデルング・デル・アンゲヴァンテン・フォルシュング・アインゲトラーゲネル・フェライン | 適応型ノイズ推定を使用してエラー隠し信号を生成する装置及び方法 |
Also Published As
Publication number | Publication date |
---|---|
JPWO2022168559A1 (ja) | 2022-08-11 |
CN116888667A (zh) | 2023-10-13 |
US20240096333A1 (en) | 2024-03-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
ES2738494T3 (es) | Metadatos para control de sonoridad y rango dinámico | |
US8750536B2 (en) | Output control apparatus, output controlling method, program, and output control system | |
US8989884B2 (en) | Automatic audio configuration based on an audio output device | |
US8242942B2 (en) | System and method to modify a metadata parameter | |
US8407565B2 (en) | System and method for producing importance rate-based rich media, and server applied to the same | |
JP2006313310A (ja) | 音声遅延を低減するための方法およびシステム | |
US20060080109A1 (en) | Audio decoding apparatus | |
US10440475B2 (en) | Signal processing device, signal processing method, and program | |
US11830512B2 (en) | Encoded output data stream transmission | |
US10080207B2 (en) | Audio device, audio system, and synchronous reproduction method | |
CN114006890A (zh) | 一种数据传输方法、设备及存储介质和终端设备 | |
CN112562638A (zh) | 语音预览的方法、装置及电子设备 | |
WO2022168559A1 (ja) | 情報処理装置、情報処理方法及び情報処理プログラム | |
JP2021093578A (ja) | 音声処理装置 | |
JP2004221951A (ja) | 伝送データのジッター補正方法 | |
JP2008067279A (ja) | 音声再生装置、映像音声再生装置、及びその音場モード切り替え方法 | |
JP2004328513A (ja) | 音声データ処理装置、音声データ処理方法、そのプログラム、および、そのプログラムを記録した記録媒体 | |
JP5489900B2 (ja) | 音響データ通信装置 | |
JP6575684B2 (ja) | 同期設定装置、配信システム、同期設定方法、及びプログラム | |
JP2013201651A (ja) | Avアンプ | |
WO2013146602A1 (ja) | Avアンプ | |
US20230244267A1 (en) | Multimedia system and multimedia operation method | |
WO2021002135A1 (ja) | データ送信装置、データ送信システム、及び、データ送信方法 | |
EP4336343A1 (en) | Device control | |
KR20060029486A (ko) | 피디에이 및 이를 이용한 디지털 오디오 방송 서비스의스트리밍 방법 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22749436 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022579413 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18262838 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202280011959.8 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 22749436 Country of ref document: EP Kind code of ref document: A1 |