JP3854263B2 - Karaoke device, karaoke method, and program - Google Patents

Description

  The present invention relates to a karaoke apparatus, a karaoke method, and a program for realizing these with a computer.

Conventionally, various karaoke apparatuses have been provided. In the karaoke machine, music data prepared as MIDI (Musical Instrument Digital Interface) data and voice waveform data prepared as PCM (Pulse Code Modulation) data are prepared in advance, and this is output as accompaniment data. At the same time, the lyrics are displayed on the screen of a television device or the like in accordance with the timing of the audio output. In addition, the input of the singer's voice waveform data is received from a microphone connected to the karaoke apparatus, and this is mixed with the accompaniment data as appropriate, and the voice is output from the speaker. The accompaniment data may be recorded on an information recording medium such as a CD-ROM (Compact Disk Read Only Memory) or a DVD-ROM (Digital Versatile Disk ROM), or other storage device via a computer communication network. In some cases, it can be obtained from The technology of such a karaoke apparatus is disclosed in the following documents, for example.
Japanese Patent No. 2925759

  [Patent Document 1] discloses a technique for scoring a singer's singing ability in such a karaoke apparatus.

  Conventionally, such a karaoke apparatus is used in a karaoke-dedicated facility called a karaoke box and various restaurants, and these facilities generally output accompaniment and singing at a loud volume from a speaker. It was the target.

  Furthermore, with the development of computer technology, an environment in which karaoke can be enjoyed by using game devices in each home is being prepared. In karaoke using such a game device, the video output is connected to a television device or the like and used to display lyrics and the like. The microphone is directly connected to the game device, and the sound input from the microphone is mixed with the accompaniment data stored in the DVD-ROM for the game device or the accompaniment data downloaded via the computer communication network, and the television device or The sound is output from the speaker via the audio input of the stereo device.

  However, it is not always the case that a speaker outputs sound. For example, if you want to enjoy karaoke at midnight or want to practice alone, you may hear the sound output from the game device using headphones or earphones. In such an environment, there are many cases where it is desired to sing a song with a small voice, instead of speaking with a loud voice.

  On the other hand, it is too complicated for the singer to appropriately set the sound output from the speaker or the sound output from the headphones or earphones by operating the game device.

Therefore, there is a strong demand for a technique for appropriately estimating and selecting under what circumstances karaoke is being performed.
The present invention was made in order to solve such a problem, and appropriately estimates an environment in which karaoke is performed, and a karaoke apparatus, a karaoke method, and the like suitable for performing karaoke in accordance with the environment, An object of the present invention is to provide a program for realizing these by a computer.

In order to achieve the above object, the following invention is disclosed in accordance with the principle of the present invention.
The karaoke apparatus according to the first aspect of the present invention includes an input receiving unit, a storage unit, a mixing unit, an output unit, and a control unit, and is configured as follows.

  That is, the input receiving unit receives input of the first audio data. For example, a microphone or the like functions as an input reception unit, and voice data (singing data) of a song sung by a singer becomes the first voice data. The first voice data is an environment where karaoke is performed. In addition, various kinds of noise and voice information not directly related to karaoke are also included. Such voice input from the microphone other than the singing data is referred to as “environmental sound”.

  On the other hand, the storage unit stores the second audio data in advance. Typically, the second audio data is recorded on an information recording medium such as a CD-ROM or DVD-ROM, and this is accompaniment data for karaoke. When accompaniment data is temporarily downloaded to the hard disk or the like of the game device via the network, the hard disk or the like corresponds to the storage unit.

  Furthermore, the mixing unit mixes the first audio data that has been accepted for input with the second audio data that is stored in advance. When the second audio data (accompaniment data) is MIDI data, MIDI data is converted into PCM data which is audio waveform data with reference to separately prepared sound source data, and the first audio data. Mixing is performed by performing addition or weighted addition with PCM data which is (singing data), and adopting a saturation operation as necessary.

  And an output part outputs the audio | voice data of the mixed result. That is, accompaniment data and singing data are output from headphones, earphones, etc., in addition to speakers of a stereo device or a television device connected to the game device, and the accompaniment and the song can be heard together.

  On the other hand, the control unit controls the mixing ratio of the first audio data to be mixed according to the intensity of the component of the second audio data stored in advance included in the first audio data whose input has been accepted. . When sound is output from the speaker, it is expected that the user enjoys normal karaoke. When sound is output from the headphones or earphones, for example, karaoke is performed at midnight, It is expected that people are practicing.

  When sound is output from the speaker, the sound of the accompaniment data is input as the environmental sound. On the other hand, when using headphones or the like, the sound of the accompaniment data does not become an environmental sound (unless there is sound leakage from the headphones). Therefore, it is examined how much the component of the second sound data (accompaniment data) is included in the first sound data (the sum of the singing data and the environmental sound) input from the microphone. Karaoke is performed using a speaker. If it is small, it is assumed that headphones are used. Then, when using headphones or the like, the input receiving unit and the mixing unit are controlled so that the ratio of voice input from the microphone is increased so that it is not necessary to sing with a loud voice.

  According to the present invention, a karaoke apparatus that appropriately estimates whether the sound output of karaoke is enjoyed with a speaker or with headphones or the like, and adjusts the intensity of the singing data when mixing the singing data accordingly. An apparatus can be provided.

Further, in the karaoke apparatus of the present invention, the control unit is configured so that the intensity of the first audio data to be mixed becomes an intensity that is previously associated with the intensity of the component of the second audio data. It can be configured to control the mixing ratio of one audio data. For example, the intensity of the song data is divided into several stages in advance, and the degree of the output intensity of the song data is associated with each stage, and a mixing ratio corresponding to this is determined. Then, the singing data is mixed and output with the corresponding output intensity.
The present invention is one of the preferred embodiments of the above-described invention. The method of performing karaoke under a typical environment is examined by experiment and the volume of singing data from the microphone is appropriately controlled based on the result. It can be applied when you want to.

Moreover, in the karaoke apparatus of the present invention, the control unit includes the first audio data so that the intensity of the first audio data to be mixed monotonously increases with respect to the intensity of the component of the second audio data. The mixing ratio can be controlled.
That is, as the accompaniment sound entering the microphone increases, the mixing ratio is controlled so that the singing sound increases.
The present invention is one of the preferred embodiments of the present invention, and according to the present invention, it is possible to provide a karaoke apparatus that automatically adjusts and mixes the accompaniment volume and the singing volume.

In the karaoke apparatus of the present invention, the control unit controls the mixing ratio of the first audio data to be mixed by changing the sensitivity of the input receiving unit.
Not only microphones used in game devices but also general microphones, there are many that can adjust directivity and sensitivity. Therefore, in the present invention, the output intensity of the singing data is adjusted by lowering the sensitivity of the microphone when using a speaker and increasing the sensitivity of the microphone when using headphones or the like.
The present invention is one of the preferred embodiments of the present invention, and according to the present invention, it is possible to provide a karaoke apparatus that automatically adjusts the sensitivity of a microphone and mixes accompaniment and singing.

In the karaoke apparatus of the present invention, the control unit is configured to control the mixing ratio of the first audio data to be mixed by changing the amplification factor of the first audio data to be mixed. be able to.
As described above, at the time of mixing, weighted addition is performed, and the weight of the accompaniment and singing can be adjusted by changing the weight. Therefore, in the present invention, the output intensity of singing data is reduced by lowering the amplification factor of the microphone side input when using a speaker, and by increasing the amplification factor of the microphone side input when using headphones or the like. Adjust.
The present invention is one of the preferred embodiments of the present invention. According to the present invention, there is provided a karaoke apparatus that automatically adjusts the weight of a microphone input during mixing and mixes accompaniment and singing. can do.

  In the karaoke apparatus according to the present invention, the control unit mixes and outputs the predetermined test audio data in the second audio data and outputs the predetermined audio data included in the first audio data. It is possible to obtain the intensity of the component of the test audio data and control the subsequent mixing ratio of the first audio data according to the intensity of the acquired component.

In other words, instead of determining whether the accompaniment itself is a speaker or headphones, special data called test audio data is output at a predetermined timing before the karaoke performance starts or during the karaoke performance. The output intensity of the singing data is determined by how much this data comes from the microphone.
In the present invention, it is possible to more accurately determine whether a speaker is used or headphones are used by adopting dedicated inspection audio data.

In the karaoke apparatus of the present invention, the storage unit can be configured to store in advance audio data outside the human audible area as the predetermined test audio data.
That is, it is determined whether a speaker or headphones are used by using sound that cannot be heard by humans.
In the present invention, for example, even in the middle of a karaoke performance, it is possible to discriminate whether a speaker or headphones are used accurately so as not to disturb the singer's singing. Become.

  Moreover, the karaoke apparatus of the present invention includes an acquisition unit that acquires the second audio data via a computer communication network instead of the storage unit, and replaces the “second audio data stored in advance” with “ The second audio data obtained via the computer communication network ”can be used.

The present invention relates to a modified embodiment of the above-described invention. Accompaniment data is streamed and distributed via a computer communication network, and the distribution and audio output are performed in a pipeline, so that the hard disk or the like of a game device can be used. This is applied when accompaniment data is not downloaded.
According to the present invention, even when accompaniment data for karaoke is provided by streaming delivery, it is determined whether a speaker is used or headphones are used, and the intensity of input from the microphone is appropriately determined. Will be able to adjust.

The karaoke method according to the second aspect of the present invention includes an input reception process, a mixing process, an output process, and a control process, and is configured as follows.
That is, in the input receiving step, input of the first audio data is received.
On the other hand, in the mixing step, the first audio data whose input has been accepted is mixed with the second audio data stored in advance.
Further, in the output step, the mixed audio data is output.

In the control step, the mixing ratio of the first audio data to be mixed is controlled according to the intensity of the second audio data component stored in advance in the first audio data that has been accepted.
A program according to another aspect of the present invention is configured to cause a computer to function as each unit of the karaoke device or to cause the computer to execute each step of the karaoke method.

  The program of the present invention can be recorded on a computer-readable information recording medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory. The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information recording medium can be distributed and sold independently of the computer.

  According to the present invention, there is provided a karaoke apparatus, a karaoke method, and a program that realizes these by a computer, which is suitable for appropriately estimating an environment in which karaoke is performed and performing karaoke in accordance with the environment. be able to.

  Embodiments of the present invention will be described below. In the following, for ease of understanding, an embodiment in which the present invention is applied to a game device will be described. However, the present invention can be similarly applied to information processing devices such as various computers, PDAs, and mobile phones. it can. That is, the embodiment described below is for explanation, and does not limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  FIG. 1 is a schematic diagram showing a schematic configuration of a typical game device in which a karaoke device according to one embodiment of the present invention is realized. Hereinafter, a description will be given with reference to FIG.

  The game apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, an external memory 106, an image processing unit 107, and the like. A DVD (Digital Versatile Disk) -ROM drive 108 and a NIC (Network Interface Card) 109.

  A DVD-ROM storing a game program and data is loaded into the DVD-ROM drive 108 and the game apparatus 100 is turned on to execute the program, thereby realizing the karaoke apparatus of the present embodiment. .

  The CPU 101 controls the overall operation of the game apparatus 100 and is connected to each component to exchange control signals and data.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire game apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication.

  The controller 105 connected via the interface 104 receives an operation input performed by the user when executing a game such as karaoke.

  The external memory 106 detachably connected via the interface 104 stores data indicating the karaoke play status (songs sung in the past, scoring results, etc.), data indicating the progress of the game, and chat communication logs (records). ) Is stored in a rewritable manner. The user can record these data in the external memory 106 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 107 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 107, and then processes the processed data on a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to a monitor (not shown) connected to the image processing unit 107. Thereby, various image displays are possible.

The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.
In addition, the polygon information arranged in the virtual three-dimensional space and added with various texture information is rendered by the Z buffer method, and a rendering image obtained by overlooking the polygon arranged in the virtual three-dimensional space from a predetermined viewpoint position is obtained. High speed execution of the obtained operation is also possible.

  Further, the CPU 101 and the image arithmetic processor operate in a coordinated manner, so that a character string can be drawn as a two-dimensional image in a frame memory or drawn on the surface of each polygon according to font information that defines the character shape. is there. The font information is recorded in the ROM 103, but it is also possible to use dedicated font information recorded in the DVD-ROM.

  The NIC 109 is for connecting the game apparatus 100 to a computer communication network (not shown) such as the Internet, and conforms to the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). Therefore, analog modems for connecting to the Internet using telephone lines, ISDN (Integrated Services Digital Network) modems, ADSL (Asymmetric Digital Subscriber Line) modems, cables for connecting to the Internet using cable television lines A modem or the like, and an interface (not shown) that mediates between them and the CPU 101 are configured.

  The audio processing unit 110 converts audio data read from the DVD-ROM into an analog audio signal and outputs the analog audio signal from a speaker (not shown) connected thereto. Further, under the control of the CPU 101, sound effects and music data to be generated during the progress of the game are generated, and sound corresponding to this is output from the speaker.

  When the audio data recorded on the DVD-ROM is MIDI data, the audio processing unit 110 refers to the sound source data included in the audio data and converts the MIDI data into PCM data. If the compressed audio data is in ADPCM format or Ogg Vorbis format, it is expanded and converted to PCM data. The PCM data can be output by performing D / A (Digital / Analog) conversion at a timing corresponding to the sampling frequency and outputting it to a speaker.

  Furthermore, a microphone 111 can be connected to the game apparatus 100 via the interface 104. In this case, the analog signal from the microphone 111 is subjected to A / D conversion at an appropriate sampling frequency so that processing such as mixing in the audio processing unit 110 can be performed as a PCM format digital signal.

  When the game apparatus 100 is used as a karaoke apparatus, audio data read from a DVD-ROM or audio data acquired from a computer communication network via the NIC 109 is used as accompaniment data, and audio data input from a microphone is used. The audio processing unit 110 mixes accompaniment data and singing data as singing data, and outputs them from the speaker. Moreover, it is possible to output sound using headphones (not shown) or earphones (not shown) instead of the speakers.

  In addition, the game apparatus 100 uses a large-capacity external storage device such as a hard disk so as to perform the same functions as the ROM 102, the RAM 103, the external memory 106, the DVD-ROM attached to the DVD-ROM drive 108, and the like. It may be configured.

    (Configuration of Karaoke Device) FIG. 2 is an explanatory diagram showing a schematic configuration of a karaoke device realized on the game device 100 or the like. FIG. 3 is a flowchart showing a flow of control of the karaoke method executed by the karaoke apparatus. Hereinafter, a description will be given with reference to FIG.

  The karaoke apparatus 201 according to the present embodiment includes an input receiving unit 202, a storage unit 203, a mixing unit 204, an output unit 205, and a control unit 206.

  Here, the storage unit 203 stores the second audio data in advance. In the present embodiment, a DVD-ROM mounted on the DVD-ROM drive 108, a hard disk of another computer connected via the NIC 109, a RAM 103 that temporarily stores downloaded music data, and the like. Functions as the storage unit 203. The second audio data corresponds to accompaniment data for karaoke.

  When accompaniment data is acquired by stream distribution via a computer communication network, an acquisition unit (not shown) that accepts stream distribution is provided instead of the storage unit 203, and the acquisition unit A form in which the stream-accompanied accompaniment data is given to the mixing unit 204 as the second audio data may be employed.

  First, the input receiving unit 202 receives input of first audio data having a predetermined time length (step S301). In the present embodiment, the microphone 111 functions as the input receiving unit 202. The first sound data includes singing data (sound data of a song sung by a singer) and environmental sounds (various noises, external sounds not related to karaoke, etc.).

  In addition, as the microphone 111 for karaoke, a microphone having strong directivity is often used, and when the microphone 111 is placed in front of the singer's mouth using a headset or the like, the average of the singing data is Comparing the intensity with the average intensity of the environmental sound, the former is generally much larger than the latter.

  The input receiving unit 202 stores data obtained by A / D converting the first audio data input from the microphone 111 in a buffer of a predetermined size prepared in the RAM 103 or the like. Therefore, the subsequent processing is repeated in units of time corresponding to the buffer of the predetermined size.

  Note that the subsequent processing is not started until the input from the microphone 111 is accumulated for the buffer length. On the other hand, since the input from the microphone 111 is A / D converted and stored at an appropriate sampling rate as described above, it is typically performed using an interrupt or parallel processing technique.

  Therefore, in step S301, until the buffer length fragment is accumulated for the first audio data, the CPU 101 is in a standby state for this process and can execute other processes in parallel.

  Next, the mixing unit 204 acquires the weights stored in the variable areas a and b prepared in advance in the RAM 103 or the like (step S302).

  Next, the second audio data (accompaniment data) is acquired in order from the top, and the fragments having the predetermined time length are acquired (step S303). As described above, since the audio data input from the microphone 111 is converted into PCM data by appropriately performing A / D conversion, accompaniment data having the same length as the PCM data length is sequentially acquired.

  Of the first audio data (singing data + environmental sound) that has been accepted and the second audio data (accompaniment data) stored in advance with the weight mixing ratio a: b, The audio data fragment corresponding to one audio data is mixed (step S304). That is, the CPU 101 functions as the mixing unit 204 in cooperation with the audio processing unit 110.

  Specifically, when the accompaniment data is MIDI data, the MIDI data is converted into PCM data that is voice waveform data by referring to separately prepared sound source data, and when the data is compressed data, This is expanded and converted into PCM data.

  Then, weighted saturation addition is performed on the PCM data corresponding to the first audio data and the PCM data corresponding to the second audio data. Weighted saturation addition is a technique used for mixing audio data.

  PCM data is a string of data obtained by digitizing an analog signal sampled at an appropriate sampling frequency with a predetermined accuracy. Typically, the sampling frequency is 44100 Hz, 48000 Hz, or a value obtained by dividing this by an integer. The precision is 16 bits, 24 bits, 32 bits, or the like.

Therefore, in order to facilitate understanding, the sequence of PCM data corresponding to the first audio data is changed in order from the top.
v [0], v [1], v [2], ...
Similarly, the PCM data column corresponding to the second audio data
s [0], s [1], s [2], ...
far.

In the simplest mixing method, the mixing result is
v [0] + s [0], v [1] + s [1], v [2] + s [2], ...
Is output.

In the case of weighted mixing, since the weight for the first audio data is a and the weight for the second audio data is b, as a mixing result,
a ・ v [0] + b ・ s [0], a ・ v [1] + b ・ s [1], a ・ v [2] + b ・ s [2], ...
Is output. In the case of weighted mixing, in karaoke, if a is increased, a human voice is reproduced greatly, and if b is increased, an accompaniment is reproduced greatly.

  Saturation calculation is calculation considering the accuracy of sampling. For example, when the sampling accuracy is a 16-bit signed integer, the values included in the PCM data column are included in the range of −32768 to 32767. Since A / D conversion and D / A conversion are performed with a 16-bit signed integer, the result of mixing must be included in this range.

Therefore,
f (x) = 32767 (x>32767);
= -32768 (x <-32768);
= x (other than above)
In the saturation operation, the result of mixing is
f (v [0] + s [0]), f (v [1] + s [1]), f (v [2] + s [2]), ...
Or
f (a ・ v [0] + b ・ s [0]), f (a ・ v [1] + b ・ s [1]), f (a ・ v [2] + b ・ s [2]) , ...
And

  And the output part 205 outputs the audio | voice data of the mixed result (step S305). In this embodiment, the audio processing unit 110 functions as the output unit 205 in cooperation with a speaker, headphones, earphones, and the like connected thereto.

  Specifically, the PCM data obtained as a result of mixing as described above is D / A converted at the sampling frequency into an analog signal, which is applied to a speaker or the like to output sound.

  In general, when there is mixed audio data in the RAM 103, the audio processing unit 110 sequentially processes the mixed audio data using an interrupt or the like and outputs the mixed audio data from a speaker or the like. In this case, the processing in step S305 is immediately terminated, and the above processing in the audio processing unit 110 is performed in parallel with the subsequent processing.

In the present embodiment, the control unit 206 receives the first audio data that has been accepted.
v [0], v [1], v [2], ...
And second audio data
s [0], s [1], s [2], ...
And the weights a and b are adjusted based on the result (step S306). Therefore, the CPU 101 functions as the control unit 206 in cooperation with the voice processing unit 110.

  Hereinafter, a specific comparison method and an example of a weight change method using the method will be described.

  First, consider a situation in which the second audio data is output from a speaker or the like via the audio processing unit 110 and the sound is input to the microphone 111 as an environmental sound. In such a case, a delay occurs, and the maximum delay time is set as Tc (seconds).

  On the other hand, assuming that the sampling frequency of various audio data is f (Hz), the maximum delay offset T when the second audio data component appears in the first audio data is T = Tc · f. Can express.

  Then, let W be a width for checking the degree of appearance of the second audio data component. This is called a “window” in the field of wireless communication synchronization technology. In this embodiment, the “window” is slid to estimate the intensity of the second audio data component.

First, a subscript of data to be processed at a certain point in time in the second audio data is set as i. The second audio data for comparison is
s [i], s [i + 1], ..., s [i + W-1]
W number sequence.

On the other hand, the first audio data in the range of t (0 <t ≦ T)
v [i + t], v [i + t + 1], ..., v [i + t + W-1]
Contrast with.

Here, a numerical sequence composed of these W elements is considered as a vector. That is,
S (i) = (s [i], s [i + 1], ..., s [i + W-1]);
V (i, t) = (v [i + t], v [i + t + 1], ..., v [i + t + W-1])
For these, the following evaluation value e is calculated.
e (i, t) = (S (i) ・ V (i, t)) / (| S (i) | | V (i, t) |)

  The evaluation value e (t) is obtained by dividing the inner product of the vector S (i) and the vector V (i, t) by the size of each vector. If the angle formed by these vectors is θ, e (i, t) is a value corresponding to cos θ. Therefore, the evaluation value e (t) takes a value between −1 and 1, and in the case of 1, the vector S (i) and the vector V (i, t) are in the same direction.

  When considered as audio data, when e (i, t) = 1, it corresponds to the phase of the two audio data being the same, and one is an amplified version of the other, e (i, t) = A case of -1 corresponds to an antiphase.

  Therefore, t is changed in the range of 0 <t ≦ T, and the evaluation value e (i, t) is maximized. The value of t at this time is set to τ (i).

  It can be inferred that τ (i) corresponds to the delay time required for accompaniment data to be returned through the microphone 111 after the sound is output.

  Now, at time t = τ (i), the proportion of the second audio data (accompaniment data) component included in the first audio data (singing data + environmental sound) is e (i, τ (i)). e (i, τ (i)) is a numerical value indicating the “correlation” between the two.

  Therefore, the mixing weight of the first audio data and the second audio data is changed according to the magnitude of this value.

  The simplest method is to obtain a threshold value by investigating beforehand how much e (i, τ (i)) is different between using a speaker in a typical environment and using headphones. , E (i, τ (i)) If it is equal to or greater than the threshold value, it is assumed that the output is a speaker, and if it is less than the threshold value, it is estimated that the output is headphones or the like.

  As for the weights a and b, a value for speaker output and a headphone output are prepared in advance, and are switched according to the value of e (i, τ (i)).

In addition, using appropriate positive constants A and C, the value of weight a is
a = A-C e (i, τ (i))
There is also a method of changing like this. In the case of an output from headphones or the like, the evaluation value e (i, τ (i)) is considered to be almost zero, so the weight a = A. On the other hand, in the case of speaker output, the evaluation value e (i, τ (i)) takes a certain positive value, and therefore the weight is smaller than in the case of output from headphones or the like.

  As a result, the volume of the voice sung by the singer is smaller in the case of output from headphones or the like than in the case of speaker output.

  The values of these constants can be changed as appropriate according to the environment to which the present invention is applied.

In a general karaoke environment, it is difficult to imagine a situation in which the speaker output and headphones output are frequently changed. Therefore, the sequence τ (0), τ (1), τ (2), ...
Or a sequence representing how much accompaniment data is input from the microphone 111 again
e (0, τ (0)), e (1, τ (1)), e (2, τ (2)), ...
Should be almost a constant sequence, and it is thought that this abrupt change is due to some chance such as noise.

Therefore,
e (0, τ (0)), e (1, τ (1)), e (2, τ (2)), ...
Instead of determining the weight by using the value as it is, a method of adopting an average of a certain section may be taken.

For example, using a positive integer N of an appropriate size,
μ (i) = (1 / N) Σ _{j = 0} ^N-1 τ (i + j)
Average delay time as
e (0, μ (0)), e (1, μ (1)), e (2, μ (2)), ...
Alternatively, using an appropriate positive integer M
e (M, μ (0)), e (M + 1, μ (1)), e (M + 2, μ (2)), ...
In this way, the ratio (time change) of the second audio data component contained in the first audio data can be obtained.

other than this,
E (i) = (1 / N) Σ _{j = 0} ^N-1 e (i + j, τ (i + j))
In the meantime, as a ratio that the component of the second audio data is included in the first audio data,
E (0), E (1), E (2), ...
May be adopted.

  While these methods can eliminate the influence of noise and the like, it is also possible to cope with switching between speaker output and headphone output during karaoke performance.

  Now, when the process of the predetermined time length is finished in this way, it is determined whether or not the karaoke play is finished (step S307). When the process is finished (step S307; Yes), this process is finished. To do. On the other hand, when it has not ended (Step S307; No), it returns to Step S301.

  In the present embodiment, the method of changing the weights a and b for determining the mixing ratio according to the component ratio is adopted. However, depending on the configuration of the microphone 111, the sensitivity of the microphone 111 may be changed. Some can change the amplification factor of the amplifier included in the microphone 111. Therefore, instead of changing the weights a and b, a mode of controlling the sensitivity of the microphone 111 and the amplification factor of the amplifier may be adopted.

  Thus, according to the present embodiment, when enjoying karaoke, it is appropriately estimated whether the audio output is from a speaker or from headphones, and the mixing ratio of the singing data is accordingly determined. Can be automatically changed, and in the former case, a karaoke device can be realized that urges the singer to sing with a loud voice, and in the former case, urges the singer to sing with a low voice.

  In the above embodiment, the intensity of outputting the first audio data is controlled by obtaining the correlation between the first audio data and the second audio data. However, a simpler method can be adopted. It is.

  In other words, the accompaniment data is a method in which “speech data that the singer does not consider to be part of the song”, that is, “speech data that is output during a period in which the singer does not speak”. This is used as inspection voice data.

  As a typical example of the test audio data, a method of adopting audio data of a rhythm portion that is often placed at the head of accompaniment data, or audio data of a so-called intro portion can be considered.

  While the voice data for inspection is being output, the voice data coming from the microphone 111 is considered to contain almost no singer's voice component, so the input from the microphone 111 is output from the speaker. Is the sum of the component of the speaker output and the noise, and it is considered that only the noise is output when the output of headphones or the like is output.

  Further, before the performance of the section (before the intro), it is considered that the input from the microphone 111 is only noise, whether it is output from a speaker or headphone. This is because the accompaniment data is not output from the speaker before the intro and should be silent.

  Therefore, the intensity of the input signal from the microphone 111 before the reproduction of the test audio data is compared with the intensity of the input signal from the microphone 111 during the reproduction of the test audio data, and if there is a significant difference (predetermined) This is a method for determining that the output is a speaker output, and otherwise determining that the output is headphones or the like.

  Since the present embodiment requires very little processing for determination as compared with the above-described embodiment, the present invention can be applied even to a game apparatus 100 that is inferior in calculation ability.

  This embodiment is a method suitable for the case where the game apparatus 100 has a function of frequency component decomposition such as Fourier transform implemented by hardware.

  That is, a sound component having a predetermined frequency and a predetermined intensity is put in the accompaniment data, and this is used as inspection audio data.

  On the other hand, the input from the microphone 111 is subjected to fast Fourier transform to obtain the intensity of the predetermined frequency component.

  Then, by comparing the intensity of the frequency component with a threshold value, it is inspected how much the test voice data has returned to the microphone 111, and comparing this with a predetermined threshold value, whether the output is speaker output. It is determined whether the output is headphones or the like.

  In the present embodiment, it is desirable that the inspection audio data has a constant volume. In particular, when sound of non-audible frequency is used as the sound data for inspection, it is not disturbed to enjoy karaoke, but more accurately whether the speaker is used or headphones are used. It becomes possible to discriminate.

  As described above, according to the present invention, the karaoke apparatus 201, the karaoke method, and the karaoke apparatus suitable for performing the karaoke according to the environment appropriately estimated by the environment in which the karaoke is performed are performed by the computer. The present invention can be applied not only to the karaoke apparatus 201 that can provide a program to be realized and used on a dedicated facility such as a karaoke box, but also to a karaoke apparatus 201 that is realized on a general-purpose game apparatus or a general-purpose computer. it can.

It is a mimetic diagram showing an outline composition of a typical game device by which a karaoke device concerning an embodiment of the present invention is realized. It is explanatory drawing which shows schematic structure of the karaoke apparatus of this embodiment. It is a flowchart which shows the flow of control of the karaoke method performed with the karaoke apparatus of this embodiment.

Explanation of symbols

100 game machine 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 External Memory 107 Image Processing Unit 108 DVD-ROM Drive 109 NIC
DESCRIPTION OF SYMBOLS 110 Sound processing part 111 Microphone 201 Karaoke apparatus 202 Input reception part 203 Storage part 204 Mixing part 205 Output part 206 Control part

Claims (10)

A karaoke apparatus that receives an input of a singing sound uttered by a user with respect to an accompaniment sound, mixes at least the singing sound and the accompaniment sound, and outputs the mixed sound to the user,
An input receiving unit that receives input of first audio data including at least the voice of the singing sound ;
A storage unit for storing second sound data representing the accompaniment sound in advance;
The intensity of the component of the second audio data stored in advance included in the first audio data that has received the input is acquired, and the first audio data with respect to the second audio data is acquired based on the intensity. A control unit that controls the mixing ratio, and when the strength increases, the control unit controls the mixing ratio to decrease.
A mixing unit that mixes the first audio data received with the input and the second audio data stored in advance at the controlled mixing ratio ; and
An output unit that outputs the mixed audio data to the user
Karaoke apparatus comprising: a. The karaoke apparatus according to claim 1,
The control unit controls the mixing ratio so that the intensity of the first audio data included in the mixed result becomes an intensity preliminarily associated with the intensity of the component of the second audio data. A karaoke machine characterized by that . The karaoke apparatus according to claim 1,
The controller determines in advance a mixing ratio when the intensity is equal to or greater than a predetermined threshold and a mixing ratio when the intensity is less than the predetermined threshold, and whether the intensity is equal to or greater than the predetermined threshold. The karaoke apparatus characterized by controlling the mixing ratio depending on whether or not . The karaoke apparatus according to any one of claims 1 to 3,
The said control part controls the said mixture ratio by changing the sensitivity of the said input reception part . The karaoke apparatus characterized by the above-mentioned . The karaoke apparatus according to any one of claims 1 to 3,
The control unit, by changing the amplification factor of the first voice data to be mixed in the mixing section, the karaoke apparatus characterized by controlling the mixing ratio. A karaoke apparatus according to any one of claims 1 to 5,
Each time the predetermined test audio data in the second audio data is mixed and output, the control unit increases the intensity of the component of the predetermined test audio data included in the first audio data. And the subsequent mixing ratio is controlled by the intensity of the acquired component . The karaoke apparatus according to claim 6,
The said memory | storage part memorize | stores the audio | voice data outside a human audible area | region beforehand as the said predetermined | prescribed test audio | voice data . The karaoke apparatus characterized by the above-mentioned . The karaoke apparatus according to any one of claims 1 to 6,
An acquisition unit that acquires the second audio data via a computer communication network instead of the storage unit;
A karaoke apparatus using “second voice data acquired via the computer communication network” instead of “second voice data stored in advance” . The input of the singing sound uttered by the user with respect to the accompaniment sound is received, and at least the singing sound and the accompaniment sound are mixed and output to the user. The input receiving unit, the control unit, the mixing unit, and the output unit A karaoke method executed by a karaoke apparatus comprising:
An input receiving step in which the input receiving unit receives an input of first audio data including at least the voice of the singing sound ;
The control unit obtains the intensity of the component of the second audio data stored in advance in the first audio data that has received the input, and based on the intensity, the control unit obtains the intensity of the second audio data. A control step for controlling the mixing ratio of the first audio data, wherein when the intensity increases, the control step controls so that the mixing ratio decreases.
A mixing step in which the mixing unit mixes the first audio data that has received the input and the second audio data representing the accompaniment sound stored in advance at the controlled mixing ratio ; and
The output step in which the output unit outputs the audio data resulting from the mixing
Karaoke method characterized by comprising a. A program that allows a computer to receive an input of a singing sound uttered by a user with respect to an accompaniment sound, and to function to mix and output at least the singing sound and the accompaniment sound to the user,
An input receiving unit that receives input of first audio data including at least the voice of the singing sound ;
A storage unit for storing second sound data representing the accompaniment sound in advance;
The intensity of the component of the second audio data stored in advance included in the first audio data that has received the input is acquired, and the first audio data with respect to the second audio data is acquired based on the intensity. A control unit that controls the mixing ratio, and when the strength increases, the control unit controls the mixing ratio to decrease.
A mixing unit that mixes the first audio data received with the input and the second audio data stored in advance at the controlled mixing ratio ; and
An output unit that outputs the mixed audio data to the user
Program for causing to function as.

Images