JP2019220973A - Reproduction device, reproduction method, and program - Google Patents

Abstract

To allow a user to easily hear lyrics voices for singing music when the music containing lyrics is being reproduced.SOLUTION: A reproduction device outputs a music signal and a lyrics voice signal expressing lyrics of the music signal. The reproduction device lowers the level of a lyrics voice of the music signal in an overlapping section where the output of the lyrics voice signal overlaps with a lyrics section containing the lyrics in the music signal.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a technique for outputting lyric information as music is played.

  2. Description of the Related Art A karaoke apparatus that synthesizes and outputs lyrics data prior to a karaoke music piece is known (for example, Patent Documents 1 and 2).

JP-A-4-67467 JP 10-63274 A

  In the case of a karaoke apparatus, since the reproduced music does not include the lyrics, the lyrics voice output according to the prior art does not become difficult to hear. However, in the case where the user listens to normal music instead of karaoke, if the lyrics are output by the prior art method, the output lyrics sound overlaps with the lyrics included in the original music. It may be difficult to hear. In addition, for example, when listening to music while driving a vehicle, the lyric sound output by the prior art method may be difficult to hear because it overlaps with a voice message for guidance provided by the in-vehicle navigation device. .

  The problems to be solved by the present invention include, for example, those described above. SUMMARY OF THE INVENTION It is an object of the present invention to provide a user who can easily hear a lyric voice for singing a song while playing music including the lyrics.

  The invention according to claim 1 is a playback device that outputs a music signal and a lyrics audio signal representing lyrics of the music signal, wherein a lyrics section that outputs lyrics of the music signal and an output of the lyrics audio signal are output. It is characterized in that a level adjusting means for lowering the level of the lyric sound of the music signal is provided for an overlapping section.

  The invention according to claim 4 is a reproducing method executed by a reproducing device that outputs a music signal and a lyric sound signal representing the lyric of the music signal, wherein the lyric section for outputting the lyric of the music signal and the A level adjusting step is provided for lowering the level of the lyric sound of the music signal in an overlapping section where the output of the lyric sound signal overlaps.

  The invention according to claim 5 is a program that is executed by a playback device that includes a computer and outputs a music signal and a lyric sound signal representing the lyrics of the music signal, wherein the lyric outputs the lyrics of the music signal. The computer is caused to function as level adjusting means for lowering the level of the lyric sound of the music signal in an overlapping section in which the output of the lyric sound signal overlaps with the section.

It is a figure showing the concept of an assist vocal. It is a flowchart of an assist vocal process. It is a flowchart of a speech information generation process. The outline of the speech information generation process will be described. An example of lyrics blocking will be described. An example of a speech insertion method will be described. An example of a speech enhancement process will be described. 13 shows a configuration according to another example of the speech enhancement process. 13 shows a configuration according to another example of the speech enhancement process. It is a block diagram showing the whole composition of a music reproduction system. FIG. 3 is a block diagram illustrating an example of an internal configuration of a terminal device. 6 is a flowchart of an assist vocal process by the music reproduction system according to the first embodiment. It is a flowchart of the assist vocal process by the music reproduction system of the second embodiment. It is a flowchart of an assist vocal process for reproducing only speech. FIG. 6 is a diagram illustrating a method for specifying a music piece being played back by an external source.

  In a preferred embodiment of the present invention, in the playback device that outputs a music signal and a lyrics audio signal representing the lyrics of the music signal, the output of the lyrics audio signal overlaps with the lyrics section that outputs the lyrics of the music signal. A level adjusting unit is provided for lowering the level of the lyric sound of the music signal for the overlapping section.

  The playback device outputs a music signal and a lyrics audio signal indicating the lyrics of the music signal. Here, in the overlap section in which the output of the lyrics signal overlaps with the lyrics section in which the music signal includes the lyrics, the playback device lowers the level of the lyrics voice of the music signal. This makes it easier for the user to hear the lyric sound reproduced from the lyric sound signal.

  In one aspect of the playback device, the level adjustment unit fades out the level of the lyric sound of the music signal in the overlapping section. As a result, the level of the lyric sound of the music signal can be reduced without discomfort.

  In another aspect of the playback device, the level adjustment unit sets the level of the lyric sound of the music signal to zero in the overlapping section. As a result, only the lyrics voice signal is reproduced in the overlapping section, so that the user can surely hear the lyrics voice.

  In another preferred embodiment of the present invention, the reproducing method executed by the reproducing device that outputs a music signal and a lyric sound signal representing the lyric of the music signal includes a lyric section that outputs the lyric of the music signal, A level adjusting step is provided for lowering the level of the lyric sound of the music signal in an overlapping section where the output of the lyric sound signal overlaps. This method also makes it easier for the user to hear the lyric sound reproduced from the lyric sound signal.

  In another preferred embodiment of the present invention, a program that is executed by a playback device that includes a computer and outputs a music signal and a lyric sound signal representing the lyrics of the music signal includes a lyric that outputs the lyrics of the music signal. The computer is caused to function as level adjusting means for lowering the level of the lyric sound of the music signal in an overlapping section in which the output of the lyric sound signal overlaps with the section. By executing this program on a computer, the above-described reproducing apparatus can be realized. This program can be stored in a storage medium and handled.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[1] Assist Vocal [1.1] Concept of Assist Vocal When a user driving a vehicle plays and listens to music in the vehicle, the user may want to sing the song being listened to. However, since the user cannot see the lyrics information while driving, the user cannot sing unless the lyrics of the song are stored.

  In the present embodiment, when a song including lyrics is being reproduced, the lyrics included in the song are output as an audio signal to inform the user. Specifically, when a song stored in a memory or the like of a terminal device is being played, lyrics included in the song are output as audio before the lyrics are played in the song. Tell the user. Thereby, the user can sing the song being reproduced even while driving. Also, a user other than the driver can sing a song without looking at a lyrics collection or the like.

  The function of outputting the contents of the lyrics as voice and transmitting them to the user prior to the timing at which the lyrics are reproduced in the music is called "assist vocal". In this embodiment, it is assumed that the reproduced music is not a karaoke but a normal music including lyrics.

  FIG. 1 illustrates the concept of assist vocals. FIG. 1 schematically shows one music piece. The horizontal axis in FIG. 1 indicates time. One music piece includes a lyrics portion divided into a plurality of blocks. The part of the lyrics included in the reproduced music is called “vocal”. In the music, a portion other than the vocals is called an “interlude”. Therefore, one music is usually composed of a plurality of interludes and a plurality of vocals.

  In the example of FIG. 1, the music is composed of three vocals 1 to 3 and a plurality of interludes. The content (lyrics) of vocal 1 is "aiueo", the content of vocal 2 is "kakikukeko", and the content of vocal 3 is "sashisuseso".

  In a situation where such a song is being played, in the present embodiment, the lyrics “Aioeo” corresponding to the vocal 1 are output as sound prior to the timing at which the vocal 1 in the song is played. In the present specification, the lyrics voice output by the assist vocal is called "speech" and is distinguished from "vocal" included in the music.

  In the example of FIG. 1, the speech 1 corresponding to the vocal 1 is output prior to the vocal 1. Similarly, speech 2 is output prior to vocal 2, and speech 3 is output prior to vocal 3.

  The speech outputs only the vocal lyrics included in the song as an audio signal, and basically does not include elements such as pitch and rhythm. Also, as will be described later, the speech is basically inserted into the interlude before the corresponding vocal, so its length is adjusted as necessary, and it is usually easier to play the song as a vocal during playback. Is also short. In a typical example, the speech is a voice that speaks the lyrics of the corresponding vocal in a buzz.

[1.2] Assist Vocal Processing Next, assist vocal processing for outputting speech will be described. FIG. 2 is a flowchart of the assist vocal process. Note that this process is executed by a terminal device mounted on the vehicle, typically, a mobile terminal such as a smartphone, and the details thereof will be described later. In the following description, it is assumed that the terminal device executes the processing.

  First, the terminal device determines whether or not the assist vocal is turned on (step S1). Here, the assist vocal is turned on / off manually by the user or automatically. In the case of performing manually, when the user wants to reproduce the speech by the assist vocal, the user operates a predetermined button or the like to turn on the assist vocal, and the terminal device detects this. On the other hand, when performing automatically, the terminal device determines the voice of the user using, for example, a microphone, and automatically turns on the assist vocal when the user is singing or performing an act similar to the singing. Set. The automatic setting method of the assist vocal will be further described later.

  If the assist vocal is not set to ON (step S1: No), the process ends. On the other hand, when the assist vocal is set to ON (step S1: Yes), the terminal device specifies the music being reproduced (step S2). In this case, the music played in the car is downloaded from a server and stored in the terminal device, a music stored in a storage medium such as a CD or a memory of an on-board unit, a radio, or the like. Includes music that is being played back from When music stored in the terminal device is being played, the terminal device can easily specify the music being played. On the other hand, when music stored in a storage medium such as a CD is being played or music is being played from the radio, the terminal device collects the music played from the speaker in the car with a microphone. Then, the audio data is transmitted to an external music search server. The music search server stores a large number of music data in a database and stores the data. The music search server specifies a music that matches the audio data received from the terminal device and indicates the music (for example, a song name, an artist name, etc. (Referred to as “song identification information”) to the terminal device. In this way, the terminal device acquires the music identification information of the music currently being played.

  When the music being reproduced is specified in this way, the terminal device executes a speech information generation process (step S3). FIG. 3 is a flowchart of the speech information generation process. FIG. 4 shows an outline of the speech information generation process.

  In FIG. 3, the terminal device acquires the lyrics data of the music piece specified in step S2 from an external server or the like (step S31). Here, “lyric data” is information that defines what lyrics are reproduced at which timing in the music, and specifically, lyrics text data indicating the lyrics included in the music, This is information associated with playback time data indicating the playback time (elapsed time from the start time of the song) at which the lyrics are played.

  Next, the terminal device acquires music analysis data (step S32). The music analysis data is information indicating musical features such as beat positions and bar positions in the music, and is generated based on audio data of the reproduced music. Specifically, the terminal device has a music analysis application built therein, collects the music reproduced from the speaker of the vehicle with a microphone, obtains audio data, and analyzes the audio data to obtain a beat position. Get music analysis data such as Instead of incorporating the music analysis application in the terminal device, music analysis data may be acquired using an external music analysis device or server.

  Next, the terminal device performs lyrics block formation (step S33). The lyrics blocking is a process of blocking lyrics text data included in the lyrics data acquired in step S31, and one block corresponds to one speech. That is, the lyrics block is a process of dividing lyrics text data into speech units.

  In the example of FIG. 4, the terminal device has acquired “Aioe Okaki Koke Kosa Sisseso” as lyric text data, and the terminal device has obtained the three blocks of “Ai Ue O”, “Kaki Koke Seso”, and “Sashi Sesso”. To generate block lyrics data.

  FIG. 5 shows an example of lyrics block. FIG. 5A shows the first method. In this method, one block is set between the interludes included in the music. The “interlude” is a part of the music other than “vocal”. Specifically, when the length It of the section other than the vocal (non-vocal section) is longer than the predetermined length t1, the terminal device determines that section as an interlude.

  However, exceptionally, there are cases where a plurality of blocks are combined into one block depending on the length of the interlude. As in the example shown in FIG. 5B, when the length It2 of the interlude 2 immediately before the length Vt3 of the vocal 3 is very short (It2 <α1 · Vt3; α1 is an arbitrary coefficient), It is difficult to output the speech of vocal 3 during interlude 2. In such a case, if the length It1 of the preceding interlude 1 is longer than a predetermined length, the terminal device sets the vocals 2 and 3 as one block. Thereby, the speech corresponding to vocal 2 and vocal 3 is performed in interlude 1.

  FIG. 5C shows the second method. In this method, the terminal device determines each block based on a break included in the lyrics data. In other words, when the lyrics text data included in the lyrics data includes the information of the segment in advance, the terminal device can block the lyrics text data according to the segment.

  Next, the terminal device performs lyrics speech (step S34). The block lyrics data obtained by the lyrics block conversion is text data indicating lyrics only, and the lyrics speech conversion is a process of converting the block lyrics data into audio data. Specifically, the terminal device incorporates text-to-speech conversion (TTS: TextToSpeech) software, and converts each block lyrics data obtained in step S33 into speech data. As a result, as shown in FIG. 4, speech data 1 to 3 are generated from the block lyrics data. Instead of incorporating the TTS software in the terminal device, TTS conversion by an external server or the like may be used.

  Next, the terminal device changes the speech length (step S35). The speech length change is a process of shortening the temporal length of each speech obtained by lyric speech conversion so that the speech can be reproduced in a short time. As described above, each speech is reproduced in the interlude preceding the corresponding vocal. However, since the length of the interlude is limited, it is necessary to reproduce the speech in a short time. Therefore, the speech length is changed.

Basically, the reproduction time of each speech is shortened (the reproduction speed is increased) within a range that can be heard by a human. For example, if the temporal length of each speech (referred to as “original speech length”) obtained in step S34 is “St” and the speech length conversion coefficient is “α2”, the speech length after the change by changing the speech length is assumed. The length "Stv" is
Stv = St · α2 (α2 <1.0) (1)
Given by For example, if α2 = 0.7, each speech is reproduced at 30% of the original speed by changing the speech length.

  In addition to the above-described batch change, the playback time may be further shortened according to the interlude time corresponding to each speech. In this case, even if the speech has the same number of characters or the words of the same lyrics, the playback time differs depending on the position in the music (length of the preceding interlude).

  Next, the terminal device calculates speech insertion timing (step S36). The terminal device inserts a speech corresponding to a certain vocal prior to the reproduction timing of the vocal. In the example shown in FIG. 4, the speech 1 corresponding to the vocal 1 is inserted before the vocal reproduction timing. Similarly, speech 2 corresponding to vocal 2 is inserted before the reproduction timing of vocal 2, and speech 3 corresponding to vocal 3 is inserted before the reproduction timing of vocal 3.

  FIG. 6 shows a specific example of a method for inserting speech. FIG. 6 shows an example of the timing at which the speech 2 corresponding to the vocal 2 is inserted.

  In method 1, the speech ends a fixed time before the start timing of the corresponding vocal. Specifically, as shown in FIG. 6, the speech 2 is inserted so as to end a predetermined time T2 before the reproduction start timing of the vocal 2. That is, the speech 2 ends a predetermined time T2 before the reproduction of the vocal 2 starts. In this case, the reproduction start timing of speech 2 is determined according to the length of speech 2. According to the method 1, since a certain time is secured from the end of the speech reproduction to the reproduction of the corresponding vocal, the user can sing the vocal part with a margin.

  In the method 2, the end timing of the speech is matched with the beat position of the music. Specifically, in the example of FIG. 6, the speech 2 is inserted so as to end N beats before the vocal 2 reproduction start timing (N is an arbitrary integer; N = 1 in this example). In this case, the reproduction start timing of speech 2 is determined according to the length of speech 2. The beat position of the music is obtained from the music analysis data described above.

  In method 3, both the playback start timing and the playback end timing of the speech coincide with the beat position of the music. Specifically, in the example of FIG. 6, the reproduction start timing and the reproduction end timing of the speech 2 are made to coincide with the third beat of the quadruple.

  When the speech end timing or both the start / end timings are matched with the beat position of the music as in the methods 2 and 3, the speech is linked to the music, so that the user can easily sing the music.

  As described above, the terminal device determines the speech insertion timing. Specifically, the reproduction start timing and the reproduction end timing of each speech are defined by the elapsed time from the beginning of the music. The reproduction start timing and the reproduction end timing of each speech are stored as a part of the speech information. That is, the speech information includes an audio signal (hereinafter, also referred to as “speech signal”) corresponding to each speech, and reproduction start timing / reproduction end timing of each speech.

  Next, the process returns to the main routine shown in FIG. 2, and the terminal device acquires the current playback position of the music being played (step S4). Specifically, the terminal device acquires the current playback position by counting the elapsed time from the playback start time of the music being played back.

  Next, the terminal device performs a speech enhancement process (step S5). The speech emphasizing process is a process of distinguishing vocals included in the music from speech and making it easier to hear, and details thereof will be described later.

  Next, the terminal device reproduces the speech based on the reproduction start timing / reproduction end timing of each speech included in the speech information and the current reproduction position (step S6). Specifically, the speech playback is started at the speech playback start timing, and the speech playback is ended at the speech playback end timing. As a result, the corresponding speech is reproduced prior to the vocal in the music.

  Next, the terminal device determines whether or not to end the speech reproduction (Step S7). The case where the speech reproduction should be ended includes a case where the speech information has been lost, a case where the music reproduction itself has ended, a case where the assist vocal has been turned off by a user operation, and the like. If the speech reproduction should not be ended (step S7: No), the process returns to step S4, and continues the speech reproduction. On the other hand, if the speech reproduction should be ended (Step S7: Yes), the assist vocal process ends.

[1.3] Method of Setting Assist Vocal Automatically On Next, a method of automatically setting the assist vocal on in step S1 of the assist vocal process shown in FIG. 2 will be described.

  As a basic method, the terminal device collects a voice uttered by the user with a microphone, and the user sings (songs) to the music or performs an action similar to the singing. Automatically turns on the assist vocal if it is determined. For example, if the voice data collected by the microphone is analyzed and it is determined that the user is humming, singing a piece of music, or humming, the assist vocal is turned on. On the other hand, if the voice data is not singing but a conversation with a fellow passenger, the assist vocal is not turned on. The assist vocal is not turned on even when the voice data includes a humming portion or when most of the speech is a conversation.

  Whether or not the user's voice included in the voice data is a singing can be determined based on the presence or absence of a rhythm or a pitch included in the voice data. For example, if the rhythm is regular or the pitch change is large, it is determined that the song is singing, and if the rhythm is irregular, the pitch is small and the song is not singing (it is a conversation). be able to. Also, using the above-described music analysis application, it may be determined that the song is a singing when a beat or a bar can be extracted from the audio data, and it may be determined that the song is not a singing when it cannot be extracted. In addition, using the above-described music search server or music search function, it may be determined that the song is singing when the song can be identified from the voice data, and it may be determined that the song is not singing when the song cannot be identified.

  Further, the terminal device calculates a correlation between the collected audio data and the music being played, and when there is a correlation equal to or more than a certain value, determines that the user is singing and turns on the assist vocal. Is also good. Further, when the terminal device has already acquired the lyrics data of the song being reproduced, if the correlation between the voice data collected by the microphone and the lyrics data is equal to or greater than a certain value, the user may sing. You may decide. Further, if the user's voice is output even at the interlude position of a song where no lyrics should exist, based on the lyrics data, it may be determined that it is a conversation.

  Also, rhythm information collected by a microphone may be used. For example, if it is determined that the user is hitting the steering wheel or the like with the hand or finger in accordance with the rhythm of the music, or if the user is stepping on the floor and taking the rhythm, the user performs an act similar to singing May be determined and assist vocals may be turned on. In this case, the correlation between the rhythm collected by the microphone and the rhythm of the music being reproduced may be calculated, and if the correlation is equal to or greater than a certain value, the assist vocal may be turned on. Also, even if the correlation with the rhythm of the music being played is not calculated, if the rhythm collected by the microphone repeats a certain rhythm, the assist vocal may be turned on. .

  Furthermore, the assist vocal may be turned on when the user captures the state of the user with a camera that captures the inside of the vehicle and shakes his / her head in time with the music. Also, the camera that captures the inside of the car detects whether or not there is a passenger in the passenger seat or the rear seat, and depending on the presence or absence of the passenger, it is possible to change the determination standard whether the user is singing or talking. Good.

  Also, in the above example, an example in which the assist vocal is turned on when it is determined that the user is singing has been described. However, even if the user sings, the user knows the lyrics and plays the assist vocal. If it is determined that there is no need to perform, the assist vocal need not be turned on. Specifically, for example, when the correlation between the collected audio data and the music being reproduced is equal to or greater than a certain value and the correlation with the lyrics data is equal to or greater than a certain value, the user knows the lyrics. Assist vocals are not turned on even when singing.

  However, in this case, since the user may not be able to understand the lyrics in the middle, preparations may be made to generate and output the speech information. Thereafter, if the correlation between the collected audio data and the music being reproduced is less than a certain value or the correlation with the lyrics data is less than a certain value, it is determined that the user does not know the lyrics. And output assist vocals.

  Further, in the above example, the method of automatically turning on the assist vocal has been described, but the automatic turning off of the assist vocal can also be performed. While the assist vocal is turned on, the user does not sing to the music (not singing a song) or acts in accordance with the singing (singing humming, singing a piece of music, humming) Assist Vocal may be automatically turned off when it is determined that the vocal is not performed. Similarly, if a conversation is detected, the assist vocal may be automatically turned off, or if it is determined that the rhythm is not being taken or the user is not shaking his head to the music, the assist vocal may be automatically turned off. Vocals may be turned off automatically.

  Further, in the above example, the automatic vocal setting of the assist vocal or the automatic off setting of the assist vocal is described based on whether the user is singing or performing an act similar to the singing. The automatic on setting or the automatic off setting may be performed based on the configuration described above. For example, for a user who wants to sing only the chorus part of the music, when playing the chorus part of the music, the assist vocal is automatically set to on and when playing the parts other than the chorus of the music, The assist vocal may be automatically turned off. Conversely, for a user who knows the rust part and wants to practice the part other than the rust, the assist vocal is automatically turned on when playing the part other than the rust, and When playing a portion, the assist vocal may be automatically turned off.

[1.4] Speech emphasis processing Next, the speech emphasis processing executed in step S5 of the assist vocal processing shown in FIG. 2 will be described. The speech emphasis process is a method for the user to distinguish between speech and vocal to make it easier to hear, and shows the following several methods.

[1.4.1] Processing when speech and vocal overlap The speech is basically reproduced during the interlude immediately before the corresponding vocal, and preferably does not overlap with the vocal in time. For this purpose, the above-described speech length changing process (step S35) is performed. Depending on the length of the speech and the length of the interlude, even if the speech length is reduced, the speech may not be reproduced during the interlude. That is, when the length of the speech is longer than the length of the interlude, the speech and the vocal are partially overlapped and reproduced. Instead of reproducing the speech and the vocal in such a manner, any of the following processes may be performed.

  (1) Adjust the vocal level.

  If the speech and the vocal overlap, there is a method of lowering the volume level of the vocal. FIG. 7A shows a case where the rear part of the speech and the beginning of the vocal overlap and an overlapping part X occurs. In this case, the volume of the vocal is adjusted in the overlapping portion X. Specifically, the volume of the vocal is reduced to a level at which speech can be heard, or set to zero. Thereby, in the overlapping portion X, the reproduction of the speech is prioritized, and the speech is easily heard.

  FIG. 7B shows a case where the beginning of the speech and the back of the previous vocal overlap, and an overlap X occurs. Also in this case, the volume of the vocal is adjusted in the overlapping portion X. Specifically, the volume of the vocal is reduced to a level at which speech can be heard, or set to zero. Also, in the overlapping portion X, instead of suddenly lowering the volume level of the vocal, the vocal may be faded out to gradually lower the volume level. Thereby, in the overlapping portion X, the reproduction of the speech is prioritized, and the speech is easily heard.

  Specifically, in the above-described level adjustment, when the vocal component and the performance component of an instrument or the like are separated in the music signal, the volume level of the vocal component may be reduced. On the other hand, when the vocal portion is synthesized with the performance portion of the musical instrument or the like and the volume of only the vocal cannot be adjusted, the volume level of the entire music signal may be lowered, or the general music signal may be used. Alternatively, the volume level of only the frequency band component corresponding to vocals (human voice) may be reduced.

  (2) Adjust the level of speech.

  If the speech and the vocal overlap, there is a converse way to lower the volume level of the speech. FIG. 7C shows a case where the rear part of the speech and the head part of the vocal overlap and an overlapping part X occurs. In this case, in the overlapping portion X, the volume of the speech is adjusted. Specifically, the volume of the speech is reduced or set to zero. Instead of suddenly lowering the volume of the speech, the volume of the speech may be gradually lowered by fading out the speech. In this case, the speech cannot be heard at the overlapping portion X, but generally, when listening to a song that the user knows to some extent, the user does not remember all of the lyrics, but once the head of the lyrics is known, In many cases, you can sing while remembering the lyrics. Therefore, as shown in FIG. 7 (C), if the head of the speech can be heard, the rear part of the speech may not be easily heard. This method is effective in such a case.

[1.4.2] Processing to hear speech and vocals from different directions Humans have the ability to distinguish sounds coming from different directions at the same time (so-called cocktail party effect). Using this, a method of allowing the user to distinguish between speech and vocal can be considered. This method is executed regardless of whether the speech and the vocal overlap in time.

(1) Method of Adjusting Phase with Left and Right Speakers FIG. 8A shows a configuration in which the phases of speech output from the left and right speakers are inverted. The music signal of the left (L) channel is supplied to an adder 32, and the music signal of the right (R) channel is supplied to an adder 33. On the other hand, the speech signal is supplied to the adder 33 as it is, and the phase is inverted by the phase inverter 31 and supplied to the adder 32. The output of the adder 32 is supplied to the left speaker 30L, and the output of the adder 33 is supplied to the right speaker 30R.

  According to this configuration, the sound image of the music including the vocals is localized between the left and right speakers, whereas the sound image of the speech is localized around the user's ear, and the user can distinguish between the speech and the vocals in the music. It will be easier. In the example of FIG. 8A, only the phase of the speech signal supplied to the left speaker 30L is inverted by the phase inverter 31, but only the phase of the speech signal supplied to the right speaker 30R is inverted. It may be reversed. In addition, if there is a certain phase difference between the speech signals supplied to the left and right speakers, the sound image position of the speech and the sound image position of the music can be made different. There is no need to invert (change 180 °). That is, a fixed phase difference may be given between the speech signal supplied to one speaker and the speech signal supplied to the other speaker.

(2) Method of Controlling Localization of Sound Image FIG. 8B shows a configuration in which a sound image of a speech can be set at an arbitrary position. The music signal of the left (L) channel is supplied to an adder 32, and the music signal of the right (R) channel is supplied to an adder 33. On the other hand, the speech signal is supplied to adders 32 and 33 via a sound image localization control calculation unit 34 and a crosstalk cancellation unit 35. The sound image localization control calculation unit 34 convolves the transfer function between the target speaker position and the listening position (the position of the user) into the speech signal, and the crosstalk canceling unit 35 determines the difference between the speaker outputting the music and the listening position. A process for canceling the transfer function between them is performed. Thereby, the sound image of the music can be localized between the left and right speakers 30L and 30R, and the sound image of the speech can be localized at the target speaker position, so that the user can easily distinguish between the speech and the vocal.

(3) Method of Using Headrest Speaker When a headrest speaker is mounted on a vehicle seat in addition to the vehicle speaker, music including vocals can be output from the vehicle speaker, and speech can be output from the headrest speaker. . FIG. 9 shows a configuration example in this case.

  The music signals of the left and right channels are supplied to the speakers 30L and 30R of the vehicle, respectively. The speech signal is supplied to the right headrest speaker 35R as it is, and the phase is inverted by the phase inverter 31 and supplied to the left headrest speaker 35L. Also in this case, since the phase difference is given to the speech signals supplied to the two headrest speakers 35L and 35R, the sound image of the speech is localized at a position different from the sound image of the music, and the user can hear the speech and the vocals in the music. Can be easily distinguished. In this example, as in the example of FIG. 8A, a fixed phase difference is given between the speech signal supplied to one headrest speaker and the speech signal supplied to the other headrest speaker. Just do it.

  When a headrest speaker is used, speech may be reproduced using a headrest speaker in a passenger seat instead of a headrest speaker in a driver seat. Further, when headrest speakers are mounted on a plurality of seats of the vehicle, the necessity of speech reproduction may be selected and set for each seat. By doing so, it is possible to set so that the speech is reproduced only from the headrest speaker at the seat of the passenger who wants to listen to the speech and sing the music.

  Also, instead of giving a phase difference, similarly to the processing described with reference to FIG. 8B, by using the sound image localization control calculating unit 34 and the crosstalk canceling unit 35, the sound image of the speech can be shifted to an arbitrary position. May be localized. This makes it easier for the user to distinguish between speech and vocals.

[2] System Configuration Next, a description will be given of a configuration example of a music reproduction system that realizes the above-described assist vocal.

[2.1] First Embodiment In the first embodiment, the assist vocal process is mainly executed on the terminal device side. FIG. 10 shows the overall configuration of the music reproduction system according to the first embodiment. In the music reproduction system according to the first embodiment, a plurality of vehicles 1, a content provider 2, and a gate server 3 can communicate with each other via a network 4. Note that the plurality of vehicles 1 can communicate with the content server 2 and the gate server 3 via the network 4 by wireless communication.

  The content provider 2 is a server such as a music distributor, and provides music data, music metadata, lyrics data, and the like. The gate server 3 is a server that functions to realize the assist vocal according to the present embodiment. The gate server 3 acquires music data, metadata, lyrics data, and the like of necessary music from the content provider 2 and stores the data in a database (not shown). ing.

  FIG. 11A shows an example of the internal configuration of the vehicle 1. The vehicle 1 includes a terminal device 10, a music playback device 20, and a speaker 30.

  The terminal device 10 is typically a mobile terminal such as a smartphone, and includes a communication unit 11, a control unit 12, a storage unit 13, a microphone 14, and an operation unit 15. The communication unit 11 communicates with the gate server 3 via the network 4. The control unit 12 includes a CPU or the like, and controls the entire terminal device 10.

  The storage unit 13 is a memory such as a ROM and a RAM, and stores programs for the control unit 12 to execute various processes, and also functions as a work memory. When the control unit 12 executes the program stored in the storage unit 13, processing including assist vocal processing is performed. The storage unit 13 may store music data of music stored by the user.

  The microphone 14 generates sound data by collecting sounds reproduced in the car, voices of a user singing, talking, and the like. The operation unit 15 is typically a touch panel or the like, and receives an operation and a selection input by a user.

  The music reproducing device 20 is, for example, a car audio, and includes an amplifier. The speaker 30 is a speaker mounted on a vehicle. The music playback device 20 plays music from the speaker 30 based on music data supplied from the terminal device 10.

  Another example of the internal configuration of the vehicle 1 is shown in FIG. In this example, the vehicle 1 includes a terminal device 10x. The terminal device 10x is a device having both functions of the terminal device 10 such as the portable terminal shown in FIG. 11A and the music reproducing device 20 such as car audio. The terminal device 10 x includes a communication unit 11, a control unit 12, a storage unit 13, a microphone 14, and an operation unit 15, similarly to the terminal device 10, and includes a music playback unit 16 corresponding to the music playback device 20. The terminal device 10x is connected to the speaker 30, and reproduces music from the speaker 30 based on music data.

  Next, an assist vocal process performed by the music reproduction system according to the first embodiment will be described. FIG. 12 is a flowchart of the assist vocal process according to the first embodiment. In this process, the assist vocal process is mainly executed by the terminal device 10 or 10x (hereinafter, simply referred to as “terminal device 10”).

  First, the gate server 3 connects to the content provider 2 via the network 4, acquires song data and lyrics data for a plurality of songs, and stores them in an internal database (step S101).

  The terminal device 10 receives the designation of the music to be reproduced by the operation of the operation unit 15 by the user (Step S102), and transmits the music designation information for designating the music to the gate server 3 (Step S103). The gate server 3 acquires the song data and the lyrics data of the song corresponding to the received song designation information from the database, and transmits them to the terminal device 10 (Step S104).

  Next, the terminal device 10 performs the processing of steps S105 to S109 using the received music data and lyrics data. Here, the processing of steps S105 to S109 is the same as that of steps S3 to S7 in FIG.

  Thus, in the music reproduction system of the first embodiment, the terminal device 10 mounted on the vehicle 1 mainly executes the assist vocal process.

  In the above example, the gate server 3 acquires the music data from the content provider in step S101. However, when the music data is stored in the terminal device 10, the gate server 3 transmits the music data from the terminal device 10. May be acquired. When music data is stored in the database in the gate server 3, music data may be acquired therefrom.

[2.2] Second Embodiment In the second embodiment, a part of the assist vocal process is executed on the gate server 3 side. The overall configuration of the music reproduction system according to the second embodiment is the same as that of the first embodiment shown in FIG.

  Next, an assist vocal process by the music reproduction system according to the second embodiment will be described. FIG. 13 is a flowchart of the assist vocal process according to the second embodiment. In this processing, the gate server 3 generates speech information, generates music data with speech, and transmits the generated music data to the terminal device 10. The terminal device 10 receives and reproduces the music data with speech. The details will be described below.

  First, the gate server 3 connects to the content provider 2 via the network 4, acquires song data and lyrics data for a plurality of songs, and stores them in an internal database (step S201). Then, the gate server 3 generates speech information for each song based on the acquired song data and lyrics data (step S202). This speech information generation process is the same as step S3 in FIG.

  After generating the speech information, the gate server 3 generates speech-added music data by adding speech to the music data (step S203). Specifically, based on the generated speech information, the gate server 3 combines the speech signal corresponding to each speech with the music data at the timing calculated by the process of step S36 in FIG. 3 to generate music data with speech. And store it in the database. That is, the music data with speech is data in which speech is reproduced in addition to the music by being reproduced as it is.

  The terminal device 10 receives the designation of the music to be reproduced by the operation of the operation unit 15 by the user (step S204), and transmits the music designation information designating the music to the gate server 3 (step S205). The gate server 3 transmits the music data with speech of the music corresponding to the received music designation information to the terminal device 10 (Step S206).

  Next, the terminal device 10 reproduces the received music data with speech (step S207). Thereby, the speech is reproduced at an appropriate timing during the reproduction of the music. Next, the terminal device 10 determines whether or not to end the reproduction of the music (Step S208). When the music has been played to the end, or when the user has stopped the playing, or when the playing should be ended (Step S208: Yes), the terminal device 10 ends the playing. On the other hand, when the reproduction of the music should not be ended (step S208: No), the process returns to step S207, and the reproduction of the music data with speech is continued.

  Thus, in the music reproduction system of the second embodiment, the music data with speech is generated on the gate server 3 side and provided to the terminal device 10. The terminal device 10 can listen to music including speech by reproducing the received music data with speech.

  In the above example, the gate server 3 acquires the music data from the content provider in step S201. However, when the music data is stored in the terminal device 10, the gate server 3 transmits the music data from the terminal device 10. May be acquired. When music data is stored in the database in the gate server 3, music data may be acquired therefrom.

[3] Assist Vocal Playing Only Speech In the assist vocal process described above, the music being played by the terminal device 10 is played with speech added. However, it is convenient if speech can be added to music played from a source other than the terminal device 10, for example, an in-car radio, a CD, or the like (hereinafter, referred to as an “external source”). In this case, the terminal device 10 basically generates the speech information by the above-described method, and only needs to reproduce the speech at a timing corresponding to the reproduction position of the music being reproduced from the external source.

  FIG. 14 shows a flowchart of the assist vocal process in this case. First, the terminal device 10 collects music reproduced from an external source by the microphone 14 to obtain reproduced music data (step S151), and transmits the data to the gate server 3 (step S152).

  The gate server 153 receives the reproduced music data from the terminal device 10, and specifies the corresponding music and its reproduction position (Step S153). Specifically, the gate server 3 includes a music search unit having the function of the above-described music search server, specifies the music based on the reproduced music data, and sets a reproduction position corresponding to the reproduced music data portion. To identify. Then, the gate server 3 transmits the lyrics data and the reproduction position information to the terminal device 10 together with the song name and the artist name of the identified song (step S154).

  The terminal device 10 generates speech information using the received lyrics data (step S155). The generation of the speech information is performed in the same manner as described with reference to FIG. Note that the terminal device 10 can acquire music analysis data by analyzing the reproduced music data acquired by the microphone 14 (the process of step S32 in FIG. 3).

  Next, the terminal device 10 calculates the current playback position in the music based on the playback position information acquired from the gate server 3 (Step S156). This method will be described later. Next, the terminal device 10 performs a speech emphasizing process (step S157), and reproduces the speech at an appropriate timing in accordance with the music being reproduced by the external source (step S158). As a result, the speech is reproduced according to the music being reproduced from the external source.

  Then, the terminal device 10 determines whether or not to end the speech reproduction (step S159). If not, the terminal device 10 returns to step S156 to continue the processing. On the other hand, when the reproduction of the speech from the external source is completed, when the music being reproduced is changed to another music, when there is no more speech to be reproduced, or when the reproduction of the speech should be completed (step S159: Yes), the process ends.

  Next, a method of specifying the current playback position of the music in step S156 will be described with reference to FIG. The playback music data transmitted from the terminal device 10 to the gate server 3 is actually data of a plurality of audio frames. That is, the terminal device 10 collects the music played by the external source with the microphone 14 and sequentially transmits the music to the gate server 3 as a plurality of audio frames.

  In the example of FIG. 15, the terminal device 10 outputs audio frames n, (n + 1), (n + 2),. . . Are sequentially transmitted to the gate server 3 as reproduction music data. At this time, the terminal device 10 stores the time when the reproduced music data is transmitted first, in the example of FIG. 15, the time when the audio frame n is transmitted (hereinafter, referred to as “reference time t0”).

  The music search unit of the gate server 3 refers to information of a large number of music pieces stored in the database, and specifies a music piece based on a plurality of received audio frames. In the example of FIG. 15, it is assumed that the music search unit of the gate server 3 has been able to specify a music piece based on audio frames n to (n + 4). In this case, the gate server 3 uses the playback time (tn) from the beginning of the tune of the audio frame n first received from the terminal device 10 as the playback position information in addition to the tune name and the artist name as the tune determination result. Transmit to the device 10. In other words, the reproduction position information transmitted from the gate server 3 to the terminal device 10 in step S154 of FIG. 14 includes the elapsed time from the beginning of the music of the audio frame n first transmitted by the terminal device 10 to the gate server 3. Has become. Therefore, in step S156, the terminal device 10 calculates the elapsed time Δt of the elapsed time from the reference time t0 to the present, which is stored in advance, and adds this to the reproduction time tn. That is, the reproduction time tn transmitted from the gate server 3 is the time from the beginning of the music to the audio frame n, and the elapsed time Δt is the time from the audio frame n to the present. Therefore, the current reproduction position (reproduction time) Tc is calculated by the following equation.

Tc = tn + Δt (2)
As described above, by providing the music search function in the gate server 3 and specifying the music and its reproduction position based on the reproduced music data, the speech can be reproduced in accordance with the music reproduced from the external source. . Instead of providing the gate server 3 with the music search function, an external music search server may be used.

  In step S159, the reproduction may be ended when one music ends, but if another music is being reproduced after one music ends, the processing is continued. Is also good. That is, while the transmission of the music reproduction data from the terminal device 10 to the gate server 3 is continued, the reproduction of the speech may be continued. As a result, even if the music reproduced from the external source changes, it is possible to continue the reproduction of the speech following the change.

Reference Signs List 1 vehicle 2 content provider 3 gate server 4 network 10, 10x terminal device 12 control unit 13 storage unit 14 microphone 20 music playback device 30 speaker

Claims (1)

A playback device that outputs a music signal and a lyrics audio signal representing lyrics of the music signal,
A reproducing apparatus, comprising: a level adjusting means for lowering the level of the lyric sound of the music signal in an overlap section in which the output of the lyric sound signal overlaps with the lyric section in which the lyrics of the music signal are output.