CN113099373A

CN113099373A - Sound field width expansion method, device, terminal and storage medium

Info

Publication number: CN113099373A
Application number: CN202110334420.3A
Authority: CN
Inventors: 芮元庆
Original assignee: Tencent Music Entertainment Technology Shenzhen Co Ltd
Current assignee: Tencent Music Entertainment Technology Shenzhen Co Ltd
Priority date: 2021-03-29
Filing date: 2021-03-29
Publication date: 2021-07-09
Anticipated expiration: 2041-03-29
Also published as: CN113099373B

Abstract

The application discloses a method, a device, a terminal and a storage medium for expanding sound field width, and belongs to the technical field of audio adjustment. The method comprises the following steps: acquiring a left channel signal and a right channel signal corresponding to target music, and a first angle and a second angle; determining a first signal and a second signal based on a signal difference of the left channel signal and the right channel signal; performing sound field width expansion processing on the first signal and the second signal based on the first angle and the second angle to obtain a third signal and a fourth signal; determining an updated left channel signal based on the third signal and the left channel signal, determining an updated right channel signal based on the fourth signal and the right channel signal, and playing the updated left channel signal and the updated right channel signal. The method provided by the embodiment of the application can solve the problems that in the prior art, when the sound field width of music is expanded, human voice in the music after the sound field width is expanded is virtual, and the texture is poor.

Description

Sound field width expansion method, device, terminal and storage medium

Technical Field

The present application relates to the field of audio adjustment technologies, and in particular, to a method, an apparatus, a terminal, and a storage medium for expanding a sound field width.

Background

With the advancement of technology, listeners often place left and right speakers of a dual speaker system for better immersion in a song, wherein the dual speaker system is arranged such that the left and right speakers are respectively disposed right in front of the listener, and an angle formed by the dual speaker system and the listener is equal to 60 degrees, i.e., the listener is located on a center line of the left speaker system and the right speaker system, an angle formed by a line connecting the listener and the left speaker system and the center line is 30 degrees, and an angle formed by a line connecting the listener and the right speaker system and the center line is 30 degrees, as shown in fig. 1. However, in practice, due to space and scene limitations, a non-standard arrangement is often used to place the left and right speakers of a dual speaker. The stereo effect of the songs played by the double loudspeakers in the nonstandard arrangement is poor, and the listening experience of a listener is seriously influenced, wherein the left loudspeaker and the right loudspeaker are respectively arranged right in front of the listener in the nonstandard arrangement, and the angle formed by the double loudspeakers and the listener is less than 60 degrees.

In the related art, in order to improve the stereo effect of a song played by using dual speakers arranged in a non-standard manner, a left channel signal and a right channel signal may be transformed by a filter function, and a sound field width between the left channel signal and the right channel signal is widened, where the sound field width is a distance between sound sources corresponding to the two sound signals respectively determined by a listener according to the two sound signals. The specific steps of widening the width of the sound field are as follows: left and right channel signals corresponding to the target music are obtained, and an angle of a line between the left speaker and the listener with respect to a direction directly in front of the listener (i.e., a first angle) and an angle of a line between the right speaker and the listener with respect to a direction directly in front of the listener (i.e., a second angle). Based on the first angle, the second angle and the sound field expansion algorithm, sound field width expansion processing is carried out on the left sound channel signal and the right sound channel signal, so that a left sound channel signal and a right sound channel signal to be played are obtained, the left loudspeaker is enabled to play the left sound channel signal to be played, and the right loudspeaker is enabled to play the right sound channel signal to be played. In this case, although the actual distance between the left speaker and the right speaker is not changed, the virtual distance (or referred to as listening distance) between the left speaker and the right speaker determined by the sound of the listener is wider than the actual distance, so that the sound field width is widened, and the listening experience of the listener is improved.

In the course of implementing the present application, the inventors found that the related art has at least the following problems:

in the process of expanding the sound field width of the left channel signal and the right channel signal, the sound field width expansion processing is not only performed on the accompaniment of the target music, but also performed on the voice of the target music, so that when the target music is played by the double loudspeakers, the voice in the target music is weakened, and the texture is poor.

Disclosure of Invention

The embodiment of the application provides a method, a device, a terminal and a storage medium for expanding the width of a sound field, and can solve the problems that in the prior art, when the width of the sound field is expanded, human voice in music after the width of the sound field is expanded is virtual, and the texture is poor. The technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a method for sound field width expansion, where the method includes:

acquiring a left channel signal and a right channel signal corresponding to target music, and a first angle and a second angle, wherein the first angle is an angle between a connecting line of a left loudspeaker and a listener in the double loudspeakers and a perpendicular line from the listener to the connecting line of the double loudspeakers, and the second angle is an angle between a connecting line of a right loudspeaker and the listener in the double loudspeakers and a perpendicular line from the listener to the connecting line of the double loudspeakers;

determining a signal difference between the left channel signal and the right channel signal, and respectively performing first decorrelation processing and second decorrelation processing on the signal difference to respectively obtain a first signal and a second signal;

performing sound field width expansion processing on the first signal and the second signal based on the first angle and the second angle to obtain a third signal and a fourth signal;

determining an updated left channel signal based on the third signal and the left channel signal, determining an updated right channel signal based on the fourth signal and the right channel signal, and playing the updated left channel signal and the updated right channel signal.

Optionally, the performing, based on the first angle and the second angle, sound field width expansion processing on the first signal and the second signal to obtain a third signal and a fourth signal includes:

the first angle theta is adjusted₁The second angle theta₂Said first signal Ls and said second signal Rs are substituted into the formula

Obtaining a third signal Ls 'and a fourth signal Rs';

wherein H_LLA head-related transfer function, H, representing the transfer of said left channel signal from said left loudspeaker to the left ear of said listener_LRA head-related transfer function, H, representing the transfer of said right channel signal from said right loudspeaker to the left ear of said listener_RLA head-related transfer function, H, representing the transfer of said left channel signal from said left loudspeaker to the right ear of said listener_RRA head-related transfer function representing the transfer of said right channel signal from said right loudspeaker to the right ear of said listener.

Optionally, the performing a first decorrelation process and a second decorrelation process on the signal difference respectively to determine a first signal and a second signal includes:

if the signal difference is time domain data, respectively performing first delay processing and second delay processing on the signal difference to respectively obtain a first signal and a second signal;

and if the signal difference is frequency domain data, performing first phase shifting processing and second phase shifting processing on the signal difference to respectively obtain a first signal and a second signal.

Optionally, before the obtaining the left channel signal and the right channel signal corresponding to the target music and the first angle and the second angle, the method further includes:

when a first operation instruction is received, acquiring a first direction of a preset axis of a terminal, when a second operation instruction is received, acquiring a second direction of the preset axis, when a third operation instruction is received, acquiring a third direction of the preset axis, wherein the first operation instruction is used for indicating that the first direction is superposed with a vertical line direction of a connecting line between a listener and the two speakers, the second operation instruction is used for indicating that the second direction is superposed with a connecting line direction between the left speaker and the listener, and the third operation instruction is used for indicating that the third direction is superposed with a connecting line direction between the right speaker and the listener;

and determining an included angle between the second direction and the first direction as a first angle, and determining an included angle between the third direction and the first direction as a second angle.

Optionally, the method further includes:

when a first operation instruction is received, acquiring a first direction of a preset axis of a terminal, when the left loudspeaker is detected to be positioned on a longitudinal symmetry axis of an image collected by a camera, acquiring a second direction of the preset axis, and when the right loudspeaker is detected to be positioned on the longitudinal symmetry axis of the image collected by the camera, acquiring a third direction of the preset axis, wherein the first operation instruction is used for indicating that the first direction is superposed with a vertical line direction of a connecting line between a listener and the two loudspeakers;

In a second aspect, an embodiment of the present application provides an apparatus for expanding a sound field width, where the apparatus includes:

the acquisition module is configured to acquire a left channel signal and a right channel signal corresponding to target music, and a first angle and a second angle, wherein the first angle is an angle between a connecting line of a left loudspeaker and a listener in the two loudspeakers and a perpendicular line from the listener to the connecting line of the two loudspeakers, and the second angle is an angle between a connecting line of a right loudspeaker and the listener in the two loudspeakers and a perpendicular line from the listener to the connecting line of the two loudspeakers;

a determining module configured to determine a signal difference of the left channel signal and the right channel signal, and perform a first decorrelation process and a second decorrelation process on the signal difference respectively to obtain a first signal and a second signal respectively;

a sound field width expansion module configured to perform sound field width expansion processing on the first signal and the second signal based on the first angle and the second angle to obtain a third signal and a fourth signal;

an update module configured to determine an updated left channel signal based on the third signal and the left channel signal, determine an updated right channel signal based on the fourth signal and the right channel signal, and play the updated left channel signal and the updated right channel signal.

Optionally, the sound field width expanding module is configured to:

Obtaining a third signal Ls 'and a fourth signal Rs';

Optionally, the determining module is configured to:

Optionally, the apparatus further comprises a first angle determining module configured to:

Optionally, the apparatus further comprises a second angle determination module configured to:

In a third aspect, the present application provides a terminal, which includes a processor and a memory, where the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned sound field width extension method.

In a fourth aspect, the present application provides a computer-readable storage medium, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the above-mentioned sound field width extension method.

In the technical solution provided in the embodiment of the present application, the human voice in the left channel signal and the human voice in the right channel signal are generally the same, and then the first signal and the second signal obtained according to the signal difference between the left channel signal and the right channel signal do not contain human voice. The first signal and the second signal which do not contain the voice are subjected to sound field width expansion processing, so that the problems of voice blurring and poor texture in music are avoided.

Meanwhile, the updated right channel signal in the embodiment of the application includes a third signal obtained by expanding the sound field width of the first signal, and the updated left channel signal includes a fourth signal obtained by expanding the sound field width of the second signal, so that when the updated right channel signal is played by the right speaker and the updated left channel signal is played by the left speaker, the stereo effect is maintained, and the problem of sound deficiency of people in the prior art is solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of an extended sound field width provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of an implementation environment with an extended sound field width provided by an embodiment of the present application;

FIG. 3 is a flow chart of a method for expanding the width of a sound field according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an expanded sound field width provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of an expanded sound field width provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an apparatus for expanding a sound field width according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

Fig. 2 is a schematic diagram of an implementation environment with an extended sound field width provided by an embodiment of the present application. Referring to fig. 2, the implementation environment includes: the terminal 201.

The terminal 201 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, a virtual reality terminal, an augmented reality terminal, a wireless terminal, a laptop computer, and the like, the terminal 201 may run various different types of applications, such as a video application, a social application, a live application, and the like, the terminal 201 has a communication function and can access the internet, the terminal 201 may refer to one of a plurality of terminals, and this embodiment is illustrated only by the terminal 201. Those skilled in the art will appreciate that the number of terminals described above may be greater or fewer.

Fig. 3 is a flowchart of a method for expanding a sound field width according to an embodiment of the present application. The embodiment is described by taking a terminal as an execution subject, and referring to fig. 3, the embodiment includes:

step 301, obtaining a left channel signal and a right channel signal corresponding to the target music, and a first angle and a second angle.

Wherein, the target music comprises human voice and accompanying voice. The first angle is the angle between the line connecting the left loudspeaker of the dual loudspeakers to the listener and the perpendicular to the line connecting the listener to the dual loudspeakers, and the second angle is the angle between the line connecting the right loudspeaker of the dual loudspeakers to the listener and the perpendicular to the line connecting the listener to the dual loudspeakers. In order to ensure the listening experience of the listener, the difference value between the first angle and the second angle is smaller than the preset value under the ideal state. When the first angle and the second angle are equal, namely the listener is positioned on the perpendicular bisector of the connecting line of the two loudspeakers, the stereo effect of the music played by the two loudspeakers is the best, and the listener has the best experience of listening to the songs.

In an implementation, the terminal is connected to the dual speaker, for example, by a bluetooth signal. After the terminal and the dual speakers are connected, the terminal may display a main page of the music application in response to a start instruction of the music application. The terminal displays a search page containing at least one music in response to a search instruction triggered by the listener. When the terminal responds to a playing instruction of the target music on the search page, the terminal sends an acquisition request to the server, wherein the acquisition request carries the music identification of the target music. The server receives an acquisition request sent by the terminal, acquires the music identifier in the acquisition request, searches the audio file corresponding to the music identifier in a pre-stored music library according to the music identifier, and sends the music file to the terminal. After the terminal receives the audio file of the target music, whether the audio file comprises a left channel file and a right channel file is determined. And if the audio file comprises a left channel file and a right channel file, acquiring a left channel signal corresponding to the left channel file and a right channel signal corresponding to the right channel file. If the audio file does not contain the left channel file and the right channel file, the target music is a single-channel audio, and the audio file of the target music can be directly sent to the double speakers, so that the double speakers can play the target music at the same time.

Optionally, the listener can determine the first angle and the second angle through his own experience, and can use the terminal to detect the first angle and the second angle. In the embodiment of the present application, the specific steps of detecting the first angle and the second angle by using the terminal are as follows: when a first operation instruction is received, a first direction of a preset axis of the terminal is obtained, when a second operation instruction is received, a second direction of the preset axis is obtained, when a third operation instruction is received, a third direction of the preset axis is obtained, wherein the first operation instruction is used for indicating that the first direction is overlapped with a vertical line direction of a connecting line from a listener to the two speakers, the second operation instruction is used for indicating that the second direction is overlapped with a connecting line direction of the left speaker and the listener, and the third operation instruction is used for indicating that the third direction is overlapped with a connecting line direction of the right speaker and the listener. And determining the included angle between the second direction and the first direction as a first angle, and determining the included angle between the third direction and the first direction as a second angle.

The terminal is provided with an angular velocity movement device in advance, and the angular velocity movement device is used for detecting the rotation angle of the terminal. The angular velocity movement device in the embodiment of the present application may be a gyroscope, and may also be another device for detecting an angle. It should be noted that, when the terminal is used to detect the first angle and the second angle, the listener cannot accurately determine the perpendicular line from the listener to the two speakers and the line between the speakers and the listener, and thus the detected direction has a certain error.

In implementation, before a listener clicks a play button of target music, the terminal responds to a trigger instruction of a sound effect button in a music play page to display the sound effect page, wherein the sound effect page comprises buttons of panoramic surround, 5.1 surround, play surround and the like. The terminal responds to a trigger instruction of the play surrounding button, a setting page is displayed and comprises a mobile phone button, a sound box button and an automobile button, wherein the mobile phone button is used for indicating the loudspeaker in the mobile phone to play music, the sound box button is used for indicating the external double loudspeakers and controlling the double loudspeakers to play music, and the automobile button is used for indicating the loudspeakers on the external automobile and controlling the loudspeakers on the automobile to play music. The terminal responds to a trigger instruction of a sound box button, displays an angle measurement page and starts an angular speed movement device on the terminal, wherein a determination button is displayed on the angle measurement page. As shown in fig. 4, a listener may hold the terminal horizontally and empirically determine whether the terminal pointing direction coincides with the listener's perpendicular to the two-speaker line. When a listener determines that the pointing direction of the terminal is overlapped with the perpendicular line from the listener to the connection line of the two speakers according to experience, the terminal responds to a first operation instruction corresponding to a determination button on the angle measurement page, and obtains a first direction of a preset axis of the terminal. After the terminal determines the first direction, the listener slowly rotates the pointing direction of the terminal until the pointing direction of the terminal is judged to be coincident with a connecting line between the left loudspeaker and the listener based on experience of people, and at the moment, the terminal responds to a second operation instruction corresponding to a determination button on the angle measurement page to acquire a second direction of a preset axis of the terminal. After the second direction is determined, the listener slowly rotates the pointing direction of the terminal in the opposite direction until the pointing direction of the terminal is judged to be coincident with the connecting line between the right loudspeaker and the listener based on the experience of the listener, and the terminal responds to a third operation instruction corresponding to a determination button on the angle measurement page to obtain a third direction of a preset axis of the terminal. The terminal calculates the angle between the second direction and the first direction, and stores the angle as the first angle in the terminal. And the terminal calculates the angle between the third direction and the first direction and stores the angle as a second angle.

Alternatively, in the process of detecting the angle, the second direction may be detected first, then the first direction may be detected, and finally the third direction may be detected. Of course, the third direction may be detected first, then the first direction may be detected, and finally the second direction may be detected.

When the terminal displays the goniometric page, it may first be determined whether the listener detected the first direction or the other direction first. For example, a first button corresponding to a first direction, a second button corresponding to a second direction and a third button corresponding to a third direction are displayed on the goniometric page. When the listener clicks the first button, a first direction is acquired, and after the first direction is acquired, the second button and the third button are displayed on the goniometric page. And when the listener clicks the second button, acquiring the second direction, and directly acquiring the third direction after acquiring the second direction. The above order of acquiring the three directions is not unique. Or, the angle measuring page displays the acquisition sequence options of the first direction, the second direction and the third direction, and after the listener selects a certain option, the listener can acquire the directions according to the acquisition sequence corresponding to the option.

It should be noted that, in the above process, the second direction and the third direction are both directions in which the speaker is located, so the second button and the third button may be set as the same button, and whether the direction is the second direction or the third direction may be further determined according to the directions corresponding to the two directions corresponding to the speaker.

When the terminal detects the second direction and the third direction, a camera on the terminal is started to assist the terminal in determining the direction of the loudspeaker. The specific process is as follows: when a first operation instruction is received, a first direction of a preset axis of the terminal is obtained, when the left loudspeaker is detected to be located on a longitudinal symmetry axis of an image collected by the camera, a second direction of the preset axis is obtained, when the right loudspeaker is detected to be located on the longitudinal symmetry axis of the image collected by the camera, a third direction of the preset axis is obtained, wherein the first operation instruction is used for indicating that the first direction coincides with the direction of a perpendicular line connecting a listener to the two loudspeakers. And determining the included angle between the second direction and the first direction as a first angle, and determining the included angle between the third direction and the first direction as a second angle.

Whether the image acquired by the camera has the loudspeaker or not can be detected through the terminal, and whether the center of the loudspeaker is in the preset area or not is detected to determine whether the left loudspeaker is located on the longitudinal symmetry axis of the image acquired by the camera or not. For example, when the terminal detects that a speaker is present in the image and the center of the speaker is within a preset area, it is determined that the left speaker is located on the longitudinal symmetry axis of the image.

In implementation, when a listener empirically determines that the pointing direction of the terminal coincides with a perpendicular line from the listener to a connection line of the two speakers, the terminal responds to a first operation instruction corresponding to a determination button on the goniometric page to acquire a first direction of a preset axis of the terminal. And when the terminal detects the second direction and the third direction, starting a camera on the terminal. The terminal detects whether the image collected by the camera has a loudspeaker, and whether the center of the loudspeaker is in a preset area. If the center of the left speaker in the image is within the preset area, determining that the pointing direction of the terminal is coincident with the connecting line between the left speaker and the listener, and acquiring a second direction of the preset axis. And if the center of the right loudspeaker in the image is in the preset area, determining that the pointing direction of the terminal is superposed with the connecting line between the right loudspeaker and the listener, and acquiring a third direction of the preset axis. And determining the included angle between the second direction and the first direction as a first angle, and determining the included angle between the third direction and the first direction as a second angle.

Thus, the angle page may not have a confirmation button. And when the center of the loudspeaker is detected to be positioned in the preset area of the image, directly acquiring the direction of the preset axis of the terminal. And prompting a listener to successfully acquire the direction after acquiring the direction of the preset axis of the terminal. For example, the prompt for successful acquisition of direction may be a voice prompt for "successfully acquire direction, please continue operation".

Of course, the determination button may be displayed on the goniometric page, and the determination button may be further configured to acquire the second direction and the third direction. The specific steps are that if the center of the right loudspeaker in the image is in a preset area, the pointing direction of the terminal is determined to be superposed with a connecting line between the right loudspeaker and a listener, at the moment, the state of a determination button displayed by the terminal is an operable state, and the terminal can respond to an operation instruction corresponding to the determination button on the angle measurement page to acquire the direction of a preset axis of the terminal. If the center of the speaker in the image is not located within the preset area of the image or the speaker is not present in the image, the pointing direction of the terminal is not coincident with the connecting line between the speaker and the listener, and the terminal is prompted to continue to rotate until the center of the speaker in the image is detected to be located within the preset area of the image. The preset area is an area which takes the center of the image as the center of a circle and takes the preset numerical value as the radius, and the preset numerical value can be preset by a technician.

It should be noted that, when the speaker is not detected in the image, or the center of the speaker in the image is not located in the preset area, the state of the determination button on the terminal is an inoperable state. Therefore, the problem that the listener triggers the confirming button by mistake and acquires wrong direction can be avoided.

In the embodiment of the application, the center of the loudspeaker can be automatically detected, whether the center is located in the preset area of the image or not is judged, whether the pointing direction of the terminal is overlapped with the connecting line between the loudspeaker and the listener or not is automatically determined, whether the pointing direction of the terminal is overlapped with the connecting line between the loudspeaker and the listener or not is not required to be artificially judged, the operation steps of the listener are saved, and the experience of the listener is improved.

In the embodiment of the application, whether the loudspeaker exists in the image or not and the center of the loudspeaker is in the preset area are detected through a machine learning method. The method comprises the following specific steps: a large number of sample images are acquired in advance. The technician makes a judgment on each sample image to determine a reference category for the sample image. And inputting the sample image into the machine learning model, outputting the type of the sample image, and inputting the type of the sample image and the reference type into a loss function to obtain loss information. And adjusting the machine learning model based on the loss information so as to complete a training process. And selecting one sample image from other sample images to train the machine learning model until the preset training process is completed, so as to obtain the machine learning model trained in advance. In the actual process, the image acquired by the camera is input into a machine learning model trained in advance to obtain the category of the image, and then whether the speaker exists in the image or not and whether the center of the speaker is in a preset area or not is determined.

The type of the sample image includes two types, one type is that a loudspeaker exists in the image, and the center of the loudspeaker is located in a preset area. Another type is that there is no speaker in the image or that there is a speaker in the image but the center of the speaker is not located within the preset area.

In most of the speakers actually used, they are regularly shaped, and it is relatively easy to determine the center point. For irregular shaped speakers, the prior art centering method can be used and is not described in detail here.

Optionally, when the terminal cannot detect the first angle and the second angle, the terminal responds to a click operation of the play-out surrounding button, and displays an angle selection page, where the angle selection page includes a plurality of angle options corresponding to the first angle and a plurality of angle options corresponding to the second angle, and one angle option corresponds to one angle value. Or displaying a text box corresponding to the first angle and a text box corresponding to the second angle on the angle selection page, wherein the text boxes are used for the listener to fill in the angle value of the first angle or the angle value of the second angle.

Step 302, determining a signal difference between the left channel signal and the right channel signal, and performing a first decorrelation process and a second decorrelation process on the signal difference respectively to obtain a first signal and a second signal respectively.

In the embodiment of the present application, the right channel signal, the left channel signal, and the signal difference are time domain data, the first signal and the second signal are frequency domain data, and the first signal and the second signal are uncorrelated.

In an implementation, the left channel signal is subtracted from the right channel signal to obtain a signal difference between the left channel signal and the right channel signal. Or subtracting the left channel signal from the right channel signal to obtain the signal difference between the left channel signal and the right channel signal. And respectively carrying out first decorrelation processing and second decorrelation processing on the obtained signal difference to respectively obtain a first signal and a second signal. The first decorrelation process and the second decorrelation process are different.

Optionally, if the signal difference is time domain data, performing first delay processing and second delay processing on the signal difference respectively to obtain a first signal and a second signal respectively; and if the signal difference is frequency domain data, performing first phase shift processing and second phase shift processing on the signal difference to respectively obtain a first signal and a second signal.

And the delay time of the first delay processing is different from the delay time of the second delay processing. The shifted phase of the first phase shift processing and the shifted phase of the second phase shift processing are different. The decorrelation operation in the embodiments of the present application may be a time-delay process and a phase shift process. When the decorrelation operation is a delay processing, a first delay processing is performed on the signal difference to determine a first signal, a second delay processing is performed on the signal difference to determine a second signal. When the decorrelation operation is a phase shift process, a first phase shift process is performed on the signal difference to determine a first signal, and a second phase shift process is performed on the signal difference to determine a second signal.

It should be noted that the delay time in the delay processing and the phase of the shift in the phase shift processing may be set by a skilled person in advance. And when the signal difference is time domain data, respectively carrying out time delay processing on the signal difference, carrying out Fourier transform on the signal subjected to the first time delay processing to obtain a first signal, and carrying out Fourier transform on the signal subjected to the second time delay processing to obtain a second signal. When the signal difference is frequency domain data, the phase shift processing is directly carried out on the signal difference.

The delay processing or the phase shift processing in the embodiment of the present application may be implemented by a time-varying filter in the prior art.

It should be noted that, the data types corresponding to the signal differences are different, the decorrelation methods used are different, the frequency domain data is processed by phase shifting, and the time domain data is processed by time delay.

Optionally, the decorrelation process may also be performed in a more complex decorrelation method. The decorrelation process is implemented, for example, by an all-pass lattice filter, or by a more complex time-varying filter.

And step 303, performing sound field width expansion processing on the first signal and the second signal based on the first angle and the second angle to obtain a third signal and a fourth signal.

Optionally, the first angle theta is adjusted₁A second angle theta₂The first signal Ls and the second signal Rs are substituted into the formula

To obtain a third signal Ls 'and a fourth signal Rs'.

Wherein H_LLRepresenting the head-related transfer function, H, of the left channel signal from the left loudspeaker to the left ear of the listener_LRA head-related transfer function, H, representing the transfer of the right channel signal from the right loudspeaker to the left ear of the listener_RLRepresenting the head-related transfer function, H, of the left channel signal from the left loudspeaker to the listener's right ear_RRRepresenting the head-related transfer function of the right channel signal from the right speaker to the listener's right ear.

It should be noted that, in the above process, the third signal Ls ', the fourth signal Rs', the first signal Ls, and the second signal Rs are all frequency domain data. Of course, when the third signal ls' and the fourth signal r are in the same states', the first signal ls and the second signal rs are time domain data, the first angle theta is adjusted₁A second angle theta₂The first signal ls and the second signal rs are substituted into the formula

A third signal ls 'and a fourth signal rs' are obtained. Wherein, is convolution operation, h_llRepresenting the head-related impulse response, h, of the left channel signal transmitted from the left loudspeaker to the left ear of the listener_lrHead-related impulse response, h, representing the transmission of a right channel signal from a right loudspeaker to the left ear of a listener_rlRepresenting the head-related impulse response, h, of the left channel signal transmitted from the left loudspeaker to the listener's right ear_rrRepresenting the head-related impulse response of the right channel signal transmitted from the right speaker to the listener's right ear. And performing inverse Fourier transform on the head-related transmission function to obtain head-related impulse response.

In the process of sound transmission, direct sound waves emitted by a sound source reach two ears after being scattered and reflected by physiological structures such as a head, an auricle, a trunk and the like, and binaural sound pressure is formed, wherein the binaural sound pressure comprises various sound source positioning factors (such as binaural time difference, binaural sound level difference, spectrum factors and the like). The auditory system uses these factors in combination to localize the sound source. The head related transfer function may simulate the sound source to binaural transmission process, which includes all factors of sound source localization.

It should be noted that the preset angle corresponding to the head-related transfer function in the above formula is 30, so as to extend the sound field width of the two speakers to the standard width (i.e. 60 degrees). In implementation, the value of the preset angle can be set according to actual expansion requirements, and different preset angles can achieve different sound field expansion effects.

Step 304, determining an updated left channel signal based on the third signal and the left channel signal, determining an updated right channel signal based on the fourth signal and the right channel signal, and playing the updated left channel signal and the updated right channel signal.

In implementation, the terminal superimposes the third signal on the left channel signal to obtain an updated left channel signal, and sends the updated left channel signal to a left speaker of the dual speakers. And after receiving the updated left channel signal, the left loudspeaker plays the updated left channel signal. And the terminal superimposes the fourth signal on the right channel signal to obtain an updated right channel signal and sends the updated right channel signal to a right loudspeaker in the double loudspeakers. And the right loudspeaker plays the updated right channel signal after receiving the updated right channel signal.

It should be noted that, when the third signal and the fourth signal are both frequency domain data, inverse fourier transform needs to be performed on the third signal and the fourth signal to obtain a transformed third signal and a transformed fourth signal. And superposing the transformed third signal to the left channel signal to obtain an updated left channel signal. And superposing the transformed fourth signal to the right channel signal to obtain an updated right channel signal. And when the third signal and the fourth signal are both time domain data, directly superposing the third signal on the left channel signal to obtain an updated left channel signal, and directly superposing the fourth signal on the right channel signal to obtain an updated right channel signal.

In the embodiment of the present application, the left speaker and the right speaker simultaneously play the updated left channel signal and the updated right channel signal.

When the target music is played by the two speakers, as shown in fig. 5, there are a virtual left speaker playing the first signal, an actual left speaker playing the left channel signal, a virtual right speaker playing the second signal, and an actual right speaker playing the right channel signal (the first signal played by the virtual left speaker is equivalent to the third signal played by the actual left speaker, and the second signal played by the virtual right speaker is equivalent to the fourth signal played by the actual right speaker). It can be seen that in the embodiment of the present application, there are actually four paths of signals, so that the stereo effect of the target music is maintained, and the problem of the artificial sound in the target music is avoided.

In the embodiment of the present application, the human voice in the left channel signal and the human voice in the right channel signal are generally the same, and thus the first signal and the second signal obtained according to the signal difference between the left channel signal and the right channel signal do not contain the human voice. The first signal and the second signal which do not contain the voice are subjected to sound field width expansion processing, so that the problems of voice blurring and poor texture in music are avoided.

Meanwhile, the updated right channel signal in the embodiment of the application includes a third signal obtained by expanding the sound field width of the first signal, and the updated left channel signal includes a fourth signal obtained by expanding the sound field width of the second signal, so that when the updated right channel signal is played by the right speaker and the updated left channel signal is played by the left speaker, the stereo effect is maintained, and the problem of voice deficiency in the prior art is solved.

Optionally, the embodiment of the present application further includes a dual speaker system, where the dual speaker system is configured to receive the updated left channel signal and the updated right channel signal sent by the terminal, send the updated left channel signal to the left speaker, and send the updated right channel signal to the right speaker. The method comprises the following specific steps: after the terminal obtains the updated left channel signal and the updated right channel signal, the terminal may send the updated left channel signal and the updated right channel signal to the dual speaker system through a bluetooth signal or other transmission signal. After the dual-speaker system receives the updated left channel signal and the updated right channel signal, the updated left channel signal is sent to the left speaker and the updated right channel signal is sent to the right speaker.

The dual-speaker system in the above process may be set in the left speaker or the right speaker, or may be set in other devices.

In some application scenarios, for a listener, the location of the listener is generally fixed, and therefore, after the terminal detects the first angle and the second angle, the terminal can send the first angle and the second angle to the dual-speaker system, and the dual-speaker system stores the first angle and the second angle, thereby avoiding the terminal from updating the music signal and saving the terminal operating memory. The method comprises the following specific steps: the terminal transmits the first angle and the second angle to the dual speaker system. The dual speaker system receives and stores the first angle and the second angle. The dual-speaker system updates and stores all music stored in the dual-speaker system based on the method provided by the embodiment of the application, so that the left channel signal updated by each piece of music replaces the original left channel signal of the music, and the updated right channel signal replaces the original right channel signal.

After the dual-speaker system receives the left channel signal and the right channel signal sent by the terminal, the dual-speaker system updates the received left channel signal and the received right channel signal based on the method provided by the embodiment of the application, and plays and stores the updated left channel signal and the updated right channel signal.

Optionally, the terminal may further update the first angle and the second angle stored in the dual speaker system, and the specific steps are as follows: and the terminal sends an angle updating notice to the dual-loudspeaker system, wherein the angle updating notice carries the updated first angle and the second angle. And the double-loudspeaker system receives the angle updating notice and updates the first angle and the second angle stored in the double-loudspeaker system.

Optionally, the target music includes a left channel accompaniment signal, a right channel accompaniment signal and a vocal signal. The terminal carries out sound field width expansion processing on the left channel accompaniment signal and the right channel accompaniment signal to obtain an updated left channel accompaniment signal and an updated right channel accompaniment signal, sends the updated left channel accompaniment signal and the updated vocal signal to the left loudspeaker, and sends the updated right channel accompaniment signal and the updated vocal signal to the right loudspeaker. And the left loudspeaker receives the updated left channel accompaniment signal and the vocal signal and plays the updated left channel accompaniment signal and the vocal signal. And the right loudspeaker receives the updated right channel accompaniment signal and the voice signal and plays the updated right channel accompaniment signal and the voice signal.

In the method, only the accompaniment signal is subjected to sound field width expansion processing, so that the problems of artificial voice in the audio signal subjected to the sound field width expansion processing, and poor texture are avoided.

Fig. 6 is a schematic structural diagram of an apparatus for expanding a sound field width, which may be applied in a terminal, according to an embodiment of the present application, and referring to fig. 6, the apparatus includes:

an obtaining module 610 configured to obtain a left channel signal and a right channel signal corresponding to a target music, and a first angle and a second angle, wherein the first angle is an angle between a line connecting a left speaker of the two speakers and a listener and a perpendicular line connecting the listener to the two speakers, and the second angle is an angle between a line connecting a right speaker of the two speakers and the listener and a perpendicular line connecting the listener to the two speakers;

a determining module 620 configured to determine a signal difference between the left channel signal and the right channel signal, and perform a first decorrelation process and a second decorrelation process on the signal difference respectively to obtain a first signal and a second signal respectively;

a sound field width expansion module 630 configured to perform sound field width expansion processing on the first signal and the second signal based on the first angle and the second angle, so as to obtain a third signal and a fourth signal;

an update module 640 configured to determine an updated left channel signal based on the third signal and the left channel signal, determine an updated right channel signal based on the fourth signal and the right channel signal, and play the updated left channel signal and the updated right channel signal.

Optionally, the sound field width expanding module 630 is configured to:

Obtaining a third signal Ls 'and a fourth signal Rs';

wherein H_LLRepresenting the head-related transfer function, H, of the left channel signal from the left loudspeaker to the left ear of the listener_LRA head-related transfer function, H, representing the transfer of the right channel signal from the right loudspeaker to the left ear of the listener_RLRepresenting the transmission of left channel signals from a left loudspeaker toHead-related transfer function of the listener's right ear, H_RRRepresenting the head-related transfer function of the right channel signal from the right speaker to the listener's right ear.

Optionally, the determining module 620 is configured to:

It should be noted that: the sound field width expanding apparatus provided in the above embodiment is only illustrated by dividing the above functional modules when the sound field width is expanded, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the sound field width expansion device provided by the above embodiment and the sound field width expansion method embodiment belong to the same concept, and the specific implementation process thereof is described in detail in the method embodiment and is not described herein again.

Fig. 7 shows a block diagram of a terminal 700 according to an exemplary embodiment of the present application. The terminal 700 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 700 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and so on.

In general, terminal 700 includes: a processor 701 and a memory 702.

The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 701 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit) which is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 701 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 702 may include one or more computer-readable storage media, which may be non-transitory. Memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 702 is used to store at least one program code for execution by processor 701 to implement the method of sound field width extension provided by method embodiments herein.

In some embodiments, the terminal 700 may further optionally include: a peripheral interface 703 and at least one peripheral. The processor 701, the memory 702, and the peripheral interface 703 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning component 708, and a power source 709.

The peripheral interface 703 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 701 and the memory 702. In some embodiments, processor 701, memory 702, and peripheral interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 704 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 704 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 704 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 704 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: metropolitan area networks, various generation mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 704 may also include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to capture touch signals on or over the surface of the display screen 705. The touch signal may be input to the processor 701 as a control signal for processing. At this point, the display 705 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 705 may be one, disposed on a front panel of the terminal 700; in other embodiments, the display 705 can be at least two, respectively disposed on different surfaces of the terminal 700 or in a folded design; in other embodiments, the display 705 may be a flexible display disposed on a curved surface or on a folded surface of the terminal 700. Even more, the display 705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display 705 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or the like.

The camera assembly 706 is used to capture images or video. Optionally, camera assembly 706 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 706 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuitry 707 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 701 for processing or inputting the electric signals to the radio frequency circuit 704 to realize voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different portions of the terminal 700. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 707 may also include a headphone jack.

The positioning component 708 is used to locate the current geographic Location of the terminal 700 for navigation or LBS (Location Based Service). The Positioning component 708 can be a Positioning component based on the GPS (Global Positioning System) in the united states, the beidou System in china, the graves System in russia, or the galileo System in the european union.

Power supply 709 is provided to supply power to various components of terminal 700. The power source 709 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When power source 709 includes a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 700 also includes one or more sensors 710. The one or more sensors 710 include, but are not limited to: acceleration sensor 711, gyro sensor 712, pressure sensor 713, fingerprint sensor 714, optical sensor 715, and proximity sensor 716.

The acceleration sensor 711 can detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the terminal 700. For example, the acceleration sensor 711 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 701 may control the display screen 705 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 712 may detect a body direction and a rotation angle of the terminal 700, and the gyro sensor 712 may cooperate with the acceleration sensor 711 to acquire a 3D motion of the terminal 700 by the user. From the data collected by the gyro sensor 712, the processor 701 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 713 may be disposed on a side frame of terminal 700 and/or underneath display 705. When the pressure sensor 713 is disposed on a side frame of the terminal 700, a user's grip signal on the terminal 700 may be detected, and the processor 701 performs right-left hand recognition or shortcut operation according to the grip signal collected by the pressure sensor 713. When the pressure sensor 713 is disposed at a lower layer of the display screen 705, the processor 701 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 705. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 714 is used for collecting a fingerprint of a user, and the processor 701 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the identity of the user according to the collected fingerprint. When the user identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, including unlocking a screen, viewing encrypted information, downloading software, paying, changing settings, and the like. The fingerprint sensor 714 may be disposed on the front, back, or side of the terminal 700. When a physical button or a vendor Logo is provided on the terminal 700, the fingerprint sensor 714 may be integrated with the physical button or the vendor Logo.

The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 may control the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is adjusted down. In another embodiment, processor 701 may also dynamically adjust the shooting parameters of camera assembly 706 based on the ambient light intensity collected by optical sensor 715.

A proximity sensor 716, also referred to as a distance sensor, is typically disposed on a front panel of the terminal 700. The proximity sensor 716 is used to collect the distance between the user and the front surface of the terminal 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front surface of the terminal 700 gradually decreases, the processor 701 controls the display 705 to switch from the bright screen state to the dark screen state; when the proximity sensor 716 detects that the distance between the user and the front surface of the terminal 700 is gradually increased, the processor 701 controls the display 705 to switch from the breath-screen state to the bright-screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 7 is not intended to be limiting of terminal 700 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

The computer device provided by the embodiment of the application can be provided as a server. Fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application, where the server 800 may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 801 and one or more memories 802, where the memory 802 stores at least one program code, and the at least one program code is loaded and executed by the processors 801 to implement the method for sound field width extension provided by the above-mentioned method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the server may also include other components for implementing the functions of the device, which are not described herein again.

In an exemplary embodiment, there is also provided a computer-readable storage medium, such as a memory, including program code, which is executable by a processor in a terminal or a server to perform the method of sound field width extension in the above-described embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a Random Access Memory (RAM), a compact-disc read-only memory (cd-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by hardware associated with program code, and the program may be stored in a computer readable storage medium, and the above mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of sound field width extension, the method comprising:

2. The method of claim 1, wherein performing sound field width extension processing on the first signal and the second signal based on the first angle and the second angle to obtain a third signal and a fourth signal comprises:

Obtaining a third signal Ls 'and a fourth signal Rs';

wherein H_LLA head-related transfer function, H, representing the transfer of said left channel signal from said left loudspeaker to the left ear of said listener_LRA head-related transfer function, H, representing the transfer of said right channel signal from said right loudspeaker to the left ear of said listener_RLRepresenting the left channel informationHead-related transfer function, H, of signals transmitted from said left loudspeaker to the right ear of said listener_RRA head-related transfer function representing the transfer of said right channel signal from said right loudspeaker to the right ear of said listener.

3. The method of claim 1, wherein the performing a first decorrelation process and a second decorrelation process on the signal difference to determine a first signal and a second signal comprises:

4. The method of claim 1, wherein the obtaining the left channel signal and the right channel signal corresponding to the target music and the first angle and the second angle are preceded by:

5. The method of claim 1, further comprising:

6. An apparatus for sound field width expansion, the apparatus comprising:

a determining module configured to obtain a signal difference between the left channel signal and the right channel signal, and perform first decorrelation processing and second decorrelation processing on the signal difference respectively to obtain a first signal and a second signal respectively;

7. The apparatus of claim 6, wherein the sound field width expanding module comprises:

Obtaining a third signal Ls 'and a fourth signal Rs';

8. The apparatus of claim 6, wherein the determination module is configured to:

9. A terminal, characterized in that the terminal comprises a processor and a memory, wherein the memory has stored therein at least one program code, which is loaded and executed by the processor to implement the operations performed by the method for sound field width extension according to any one of claims 1 to 5.

10. A computer-readable storage medium having stored therein at least one program code, the at least one program code being loaded into and executed by a processor to perform operations performed by a method of sound field width extension as claimed in any one of claims 1 to 5.