WO2012115417A2

WO2012115417A2 - Method, system, and computer-readable recording medium for enabling a user to play a digital musical instrument based on his/her voice

Info

Publication number: WO2012115417A2
Application number: PCT/KR2012/001284
Authority: WO
Inventors: 이정석; 김병오; 한동숭
Original assignee: 전주대학교 산학협력단
Priority date: 2011-02-23
Filing date: 2012-02-21
Publication date: 2012-08-30
Also published as: WO2012115417A3

Abstract

The present invention relates to a method, system, and computer-readable recording medium for enabling a user to play a digital musical instrument based on his/her voice. According to one embodiment of the present invention, a method for enabling a user to play a digital musical instrument based on his/her voice comprises the following steps: receiving information on a specific digital musical instrument selected by the user from among multiple digital musical instruments; receiving the voice of the user and transmitting a voice signal; extracting information on the intensity of the tune and voice of the voice signal as time passes, and generating a MIDI signal based on the extracted information on the intensity of the tune and voice; extracting or generating a MIDE sound source corresponding to the specific digital musical instrument based on the generated MIDI signal; and outputting the MIDI sound source.

Description

Methods, systems, and computer readable recording media for enabling a user to play a digital instrument based on his or her voice

The present invention relates to a method, system and computer readable recording medium for enabling a user to play a digital instrument based on his or her voice.

MIDI (Music Instrument Digital Interface) is a protocol developed to be widely used among all electronic instruments in order to solve the problem that the compatibility between the electronic instruments is not guaranteed because the way of generating and storing the sound is different for each electronic instrument. Originally standardized as an international standard in 1981, MIDI was not originally intended for computer music, but is now widely used for computer music.

Since MIDI is a standardized format for the creation and storage of electronic musical instrument data, MIDI files contain not only the pitch but also information about musical characteristics such as the strength and speed of the sound, and information about the instrument such as the type of instrument. In spite of the fact that it does not store waveform information unlike wave files, it is widely used in the field of computer music because of its relatively small file size and easy operation such as adding and deleting musical instruments.

Conventional systems for processing such MIDI files generally consist of a keyboard as a MIDI input device and a synthesizer as a sound synthesizer. In this case, when pitch information and sound intensity information are obtained from a sensor attached to each keyboard of the keyboard, such information is usually converted into a MIDI signal by the synthesizer's MIDI processor and output. However, since a conventional keyboard inevitably has a large number of keyboards to express various pitches, it is true that it is difficult to play MIDI music as desired by the user unless the user can skillfully play a keyboard instrument. In addition, since the conventional keyboard can express only discontinuous sound basically, there is a differential sound input unit to compensate for such a disadvantage, but it is true that a complicated operation for musically unnatural or natural expression is required.

Therefore, in order to solve the problems of the prior art, a device for composing music based on the user's voice has been proposed through Korean Patent Application Publication No. 2007-76942, etc., but such devices have their own musical limits. It is true.

The present invention aims to solve all of the above-mentioned problems of the prior art.

Another object of the present invention is to enable a user to play a digital musical instrument based on his or her own voice.

In addition, another object of the present invention is to enable a user to easily play music by sounding a desired digital instrument based on his or her own voice.

In addition, another object of the present invention is to allow a user to perform a performance suitable for a specific musical instrument desired by the user so that the user can feel as if he is actually playing the musical instrument.

In addition, another object of the present invention is to generate an additional signal corresponding to a specific voice pronounced by a user using a voice recognition technology.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, there is provided a method for enabling a user to play a digital instrument based on his or her own voice, the method comprising: receiving information regarding a user's selection of a specific digital instrument among various digital instruments; Receiving a voice to transmit a voice signal, and extracting information about the pitch and the loudness of the voice signal according to time, and generating a midi signal using the extracted pitch and the information about the loudness; And extracting or generating a MIDI sound source corresponding to the specific digital instrument with reference to the generated MIDI signal, and outputting the MIDI sound source, wherein the voice signal is used by the user using a controller. Information related to the performance control performed is applied, and the MIDI signal indicates the beginning of the audio signal. Key indicates a note on message, a pitch wheel message indicating an irregular pitch change of the speech signal, a modulation message indicating a regular pitch change of the speech signal, and a volume indicating the volume of the speech signal A method is provided that includes at least one of a volume message, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal.

According to another aspect of the present invention, there is provided a system for allowing a user to play a digital instrument based on his or her own voice, comprising: a user interface unit for receiving information regarding a user's selection of a specific digital instrument among various digital instruments; A voice player for receiving a voice of a user and transmitting a voice signal, wherein the voice player includes a controller; a voice signal processor for applying information on a performance control performed by the user to the voice signal; A MIDI signal generator for extracting information on the pitch and sound intensity of the voice signal according to time, and generating a MIDI signal by using the extracted information about the pitch and sound intensity, and the generated MIDI signal With reference to the extraction or generation of a MIDI sound source corresponding to the specific digital instrument, A MIDI sound source management unit for outputting a MIDI sound source, wherein the MIDI signal includes a note on message indicating a start of the voice signal, a pitch wheel message indicating an irregular pitch change of the voice signal, A modulation message indicating a regular pitch change of the voice signal, a volume message indicating a volume of the voice signal, an expression message indicating an irregular volume change of the voice signal, and an end of the voice signal. A system is provided that includes at least one of a note off message that points to it.

According to yet another aspect of the present invention, there is provided a method for enabling a user to play a digital instrument based on his or her own voice, the method comprising: receiving information regarding a user's selection of a specific digital instrument among various digital instruments, the user Receiving a voice of the voice signal and transmitting a voice signal, according to time, information about the pitch and the intensity of the voice signal is extracted, and a MIDI signal is generated using the information about the extracted pitch and the intensity of the voice; Analyzing the voice signal using a voice recognition technology, and if it is determined that the voice signal includes preset voice information, generating an additional signal corresponding to the preset voice information; And generating a digital sound source with reference to the generated MIDI signal and the generated additional signal. The voice signal includes information on a performance control performed by the user using a controller, and the MIDI signal includes a note on message indicating the start of the voice signal and an irregularity of the voice signal. A pitch wheel message indicating a pitch change, a modulation message indicating a regular pitch change of the voice signal, a volume message indicating a volume of the voice signal, and an irregular volume change of the voice signal A method is provided that includes at least one of an expression message and a note off message indicating the end of the voice signal.

According to another aspect of the present invention, a system for allowing a user to play a digital instrument based on his or her own voice, the user interface unit for receiving information about the user's selection of a particular digital instrument of the various digital instruments A voice player which receives a user's voice and transmits a voice signal, extracts information on the pitch and intensity of the voice signal according to time, and uses the extracted pitch and information on the intensity A MIDI signal generation unit for generating a MIDI signal and analyzing the voice signal using a voice recognition technology, and if it is determined that the voice signal includes preset voice information, the voice signal corresponding to the preset voice information An additional signal generator which generates an additional signal, and the generated MIDI signal and the generated additional signal A sound source generation and management unit for generating a digital sound source with reference to a call, wherein the sound signal is applied with information about a performance control performed by the user using a controller, and the MIDI signal is a signal of the sound signal. A note on message indicating a start, a pitch wheel message indicating an irregular pitch change of the voice signal, a modulation message indicating a regular pitch change of the voice signal, and a volume of the voice signal A system is provided that includes at least one of a pointing volume message, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal.

In addition, there is further provided a computer readable recording medium for recording another method for implementing the present invention, another system, and a computer program for executing the method.

According to the present invention, a user can play a digital musical instrument based on his or her own voice.

According to the present invention, a user can easily play music by making a sound of a digital musical instrument he / she wants based on his or her own voice.

According to the present invention, the user can perform a performance suitable for a specific musical instrument that he / she wants, so that the user can feel as if he is actually playing the musical instrument.

According to the present invention, an additional signal corresponding to a specific voice pronounced by a user can be generated using a speech recognition technology. Since the additional signal includes signals other than the MIDI standard, both the MIDI signal and the additional signal are used. This makes it possible to create a richer and more diverse digital sound source.

1 is a diagram illustrating an internal configuration of a voice digital musical instrument system 100 according to an embodiment of the present invention.

2 is a diagram exemplarily illustrating a voice player 120 employing a hand controller as the controller 124 according to one embodiment of the present invention.

3 is a diagram illustrating various types of MIDI messages that can be included in a MIDI signal according to an embodiment of the present invention.

4 is a diagram illustrating an internal configuration of the voice digital instrument system 400 according to another embodiment of the present invention.

110: user interface unit

120: voice player

130: voice signal processing unit

140: MIDI signal generator

150: MIDI sound management unit

160: communication unit

170: control unit

410: user interface unit

420: voice player

430: voice signal processing unit

440: MIDI signal generator

450: additional signal generator

460: sound generation and management unit

470: communication unit

480: control unit

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the invention. In addition, it is to be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be taken as encompassing the scope of the claims of the claims and all equivalents thereto. Like reference numerals in the drawings indicate the same or similar elements throughout the several aspects.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention.

Configuration of Voice Digital Instrument System According to an Embodiment of the Present Invention

Some of the components of the voice digital musical instrument system 100 according to an embodiment of the present invention may be similar to the conventional components included in the conventional digital musical instrument system, but some of the components may be novel. The voice digital instrument system 100 may be a set of dedicated devices or may be combined with a user mobile device.

As shown in FIG. 1, the voice digital instrument system 100 includes a user interface unit 110, a voice player 120, a voice signal processor 130, a MIDI signal generator 140, and a MIDI sound source manager ( 150, the communication unit 160 and the control unit 170 may be configured. According to an embodiment of the present invention, the user interface unit 110, the voice player 120, the voice signal processor 130, the MIDI signal generator 140, the MIDI sound source manager 150, the communication unit 160 and The controller 170 may be a program module in which at least some of them communicate with an external device. Such program modules may be included in the voice digital instrument system 100 in the form of operating systems, application modules or other program modules, and may be physically stored in various known storage devices. Such program modules may also be stored in a remote storage device capable of communicating with the voice digital instrument system 100. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, the user interface unit 110 according to an embodiment of the present invention allows information about various digital instruments to be provided to a user through a predetermined display panel (not shown), and relates to a user's selection of a specific digital instrument. It can perform a function of receiving information. The user interface unit 110 may be linked with a predetermined digital instrument database (not shown) that stores information on various types of digital instruments and information on a playing method corresponding to each digital instrument.

Next, the voice player 120 according to an embodiment of the present invention may perform a function of receiving the voice when the user plays the ball. The voice player 120 may be configured of only a general microphone, but more preferably, may include a means for preventing input of ambient noise together, that is, a voice input means 122 in the form of a mouthpiece. have. The user can perform the oral performance with the voice input means 122 in the mouth. Meanwhile, the voice player 120 may include a separate controller 124 to further perform a function of receiving information about a control that is performed so that the user corresponds to the digital instrument selected by the user while performing the sound of the ball. .

As an example of the controller 124 that can be applied to the present invention, a hand controller having various buttons for playing can be assumed. The hand controller may be provided with various buttons 126, wheels 127, triggers 128, and the like, which allow the user to perform hand control to correspond to the digital musical instrument. Although the voice input unit 122 and the hand controller 124 may be connected to each other by a known cable 129 or the like, the voice input unit 122 and the hand controller 124 may be wirelessly connected or integrated with each other according to a choice of a person skilled in the art. 2 is a diagram exemplarily illustrating a configuration of a voice player 120 employing a hand controller as the controller 124 according to an embodiment of the present invention.

In addition, as another example of the controller 124 that may be applied to the present invention, a hand held device based on a touch interface may be assumed. According to an embodiment of the present invention, a touch interface for allowing a user to touch and operate a finger on a touch interface-based portable device may be variably implemented for various musical instruments. The touch interface may be installed in a smartphone. It may be implemented in the form of an application that can be executed. According to an embodiment of the present invention, the communication between the touch interface-based portable device and the voice digital instrument system 100 is Wi-Fi, Bluetooth, Remote Frequency IDentification, or Near Field Communication. ) May be performed based on a predetermined wireless communication protocol.

Although only an embodiment in which a hand controller is employed as the controller 124 is shown in FIG. 2, the controller according to the present invention is not necessarily limited to that shown in FIG. 2, but may be within a range capable of achieving the object of the present invention. It will be appreciated that various modifications can be employed, including the aforementioned touch interface based portable devices.

Next, the voice signal processor 130 according to an embodiment of the present invention is a MIDI signal generation unit 140 which will be described later to the voice signal of the user received by the voice player 120 through the voice input unit 122, etc. Or a change in the voice signal of the user according to the operation of the controller 124 of the user may be transmitted to the MIDI signal generator 140. In the latter case, the voice signal processing unit 130 performs the performance control for the specific digital instrument using the controller 124 while the user inputs the voice through the voice input means 122 or the like, so that the performance control information is displayed. The applied voice signal may be transmitted. For example, when the gayageum is selected as the digital musical instrument, the user inputs a voice to the voice input unit 122 with a humming or the like to express the melody, and bounces the trigger 128 of the hand controller employed as the controller 124 to express the melody. In this case, the voice signal processor 130 may transmit the voice signal, in which the sound intensity information is partially changed, to the MIDI signal generator 140 according to the trigger control.

Meanwhile, the voice signal processor 130 may perform a function of amplifying and transmitting the filtered voice signal after performing noise filtering on the voice signal received from the voice player 120. To this end, the voice signal processor 130 may include an audio filter (not shown) such as a white noise filter and an audio amplifier (not shown) for amplifying a voice signal as an internal component thereof. Techniques for filtering or amplifying voice signals for the present invention can be freely adopted and applied by those skilled in the art from various known techniques.

Next, the MIDI signal generator 140 according to an embodiment of the present invention extracts information regarding pitch and sound intensity from the voice signal received from the voice signal processor 130 according to time, and extracts the extracted information. The information may be generated using the information. More specifically, the MIDI signal generation unit 140 quantizes the frequency of the voice signal and extracts information about the pitch according to time, and extracts information about the sound intensity by quantizing the amplitude of the voice signal. In addition, information about the digitized pitch and sound intensity can be generated as a MIDI signal conforming to the MIDI protocol.

More specifically, according to an embodiment of the present invention, the MIDI signal generated from the voice signal received from the voice signal processor 130 may be composed of various kinds of MIDI messages conforming to the MIDI protocol. For example, a MIDI signal generated from a voice signal may be composed of a note on / off message, a pitch wheel message, a modulation message, a volume message, an expression message, or the like. Can be. 3 shows a more detailed description of the various types of MIDI messages that may be included in the MIDI signal.

Referring to FIG. 3, when a voice signal is started, a note on message and a volume message may be generated, where the note on message has a note number and a velocity indicating an initial pitch and an initial volume of the corresponding voice signal, respectively. It can have a value. In addition, a modulation message may be generated when a regular pitch change occurs after the voice signal is started, and a pitch wheel message may be generated when an irregular pitch change of the voice signal occurs, and an irregularity of the voice signal may be generated. When the volume change occurs, an expression message may be generated. In addition, when the voice signal ends, a note off message may be generated or a note on message having a velocity value of 0 may be generated.

Next, the MIDI sound source management unit 150 according to an embodiment of the present invention refers to the MIDI signal generated by the MIDI signal generation unit 140 and displays the MIDI sound source corresponding to the digital instrument selected by the user in the sound source database (not shown). Extract or generate a), such that the MIDI sound source is output through a predetermined audio output channel (not shown).

Next, the communication unit 160 according to an embodiment of the present invention, if necessary, the user interface unit 110, voice player 120, voice signal processor 130, MIDI signal generator 140 and A function of enabling data transmission / reception from / to the MIDI sound source manager 150 may be performed. That is, the voice digital instrument system 100 may perform data transmission and reception with an external device through a predetermined communication network (for example, the Internet or a LAN).

Finally, the control unit 170 according to an embodiment of the present invention is a user interface unit 110, voice player 120, voice signal processing unit 130, MIDI signal generator 140, MIDI sound source management unit 150 ) And the communication unit 160 may control a flow of data. That is, the controller 170 according to the present invention controls the data flow from / to the outside of the voice digital musical instrument system 100 or the data flow between the components of the voice digital musical instrument system 100, thereby providing a user interface unit. 110, the voice player 120, the voice signal processor 130, the MIDI signal generator 140, the MIDI sound source manager 150, and the communicator 160 may control to perform unique functions.

Configuration of Voice Digital Instrument System According to Another Embodiment of the Present Invention

Some of the components of the voice digital musical instrument system 400 according to another embodiment of the present invention may be similar to the conventional components included in the conventional digital musical instrument system, but some of the components may be novel. The voice digital instrument system 400 may be a set of dedicated devices or may be combined with a user mobile device.

As shown in FIG. 4, the voice digital instrument system 400 includes a user interface unit 410, a voice player 420, a voice signal processor 430, a MIDI signal generator 440, and an additional signal generator. 450, a sound source generation and management unit 460, a communication unit 470, and a controller 480 may be configured. According to another exemplary embodiment of the present invention, the user interface unit 410, the voice player 420, the voice signal processor 430, the MIDI signal generator 440, the additional signal generator 450, the sound source generation and management unit 460, the communication unit 470, and the control unit 480 may be program modules in which at least some of them communicate with an external device. Such program modules may be included in the voice digital instrument system 400 in the form of an operating system, an application program module or other program modules, and may be physically stored in various known storage devices. Such program modules may also be stored in a remote storage device capable of communicating with the voice digital instrument system 400. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, the user interface 410, the voice player 420, the voice signal processor 430, and the MIDI signal generator 440 according to another embodiment of the present invention are the user according to the embodiment of the present invention. Since the interface 110, the voice player 120, the voice signal processor 130, and the MIDI signal generator 140 perform the same functions, a detailed description thereof will be omitted.

Next, according to another exemplary embodiment of the present invention, the additional signal generator 450 may perform a function of recognizing a user's voice signal received from the voice player 420 and generating the additional signal. More specifically, the additional signal generator 450 according to another embodiment of the present invention is configured to store the predetermined voice information and the predetermined additional signal by mapping each other. When the voice signal is recognized and it is determined that the voice signal of the user includes preset voice information, an additional signal corresponding to the preset voice information may be generated.

According to another embodiment of the present invention, the speech recognition technology applied by the additional signal generator 450 may include various speech recognition technologies such as phoneme recognition technology, phonological recognition technology, and syllable recognition technology. For example, the additional signal generator 450 according to another exemplary embodiment of the present invention may use a known keyword spotting technique to extract meaningful voice information from a user's voice signal.

In addition, according to another embodiment of the present invention, the additional signal generated by the additional signal generation unit 450 corresponding to the predetermined voice information may include not only a MIDI standard signal but also a signal other than the MIDI standard, that is, the MIDI standard. Signals embodying various musical elements that are difficult to express may be further included.

In particular, according to another embodiment of the present invention, the additional signals other than the MIDI standard generated by the additional signal generator 450 may control various elements, items, values, etc. of the sampler or synthesizer. When the sound source is generated by using the additional signal, it is possible to control a plurality of layers that can be configured by the sampler or synthesizer to realize a richer musical effect. That is, when a sound source is generated by using an additional signal other than the MIDI standard together with the MIDI signal, it is possible to realize various musical effects such as a tone and a performance technique as well as basic musical effects such as pitch and strength.

For example, if a user pronounces a voice including a sound such as “car”, the additional signal generator 450 according to another embodiment of the present invention may correspond to a specific group corresponding to the sound sound among a plurality of layer samples of the sampler. An additional signal for selecting may be generated. In another example, the additional signal generator 450 according to another embodiment of the present invention may pitch as a vibrato element to be input to a sound source, depending on whether the user pronounces a voice including a sound such as "bee". An additional signal that determines whether to generate and use a MIDI message called a pitch bend or to generate and use a low frequency of a synthesizer can be generated.

Next, according to another embodiment of the present invention, the sound source generation and management unit 460 uses the MIDI signal generated by the MIDI signal generator 440 and the additional signal generated by the additional signal generator 460. A function of generating, synthesizing, transforming or managing digital sound sources can be performed. Here, the sound source generation and management unit 460 according to another embodiment of the present invention may include a predetermined sampler (not shown) or a synthesizer (not shown) that makes it possible to generate a digital sound source using additional signals. Can be.

Next, the communication unit 470 according to another embodiment of the present invention, if necessary, the user interface unit 410, voice player 420, voice signal processor 430, MIDI signal generator 440, The additional signal generator 450, the sound source generator and manager 460, the communicator 470, and the controller 480 may perform a function of enabling / receiving data to / from the receiver. That is, the voice digital instrument system 400 may perform data transmission / reception with an external device through a predetermined communication network (for example, the Internet or a LAN).

Finally, the control unit 480 according to another embodiment of the present invention is a user interface unit 410, voice player 420, voice signal processor 430, MIDI signal generator 440, additional signal generator ( 450, a function of controlling the flow of data between the sound source generation and management unit 460 and the communication unit 470 may be performed. That is, the controller 480 according to the present invention controls the data flow from / to the outside of the voice digital musical instrument system 400 or the data flow between the components of the voice digital musical instrument system 400, thereby providing a user interface unit. 410, the voice player 420, the voice signal processor 430, the MIDI signal generator 440, the additional signal generator 450, the sound source generator and manager 460, and the communicator 470 respectively. It can be controlled to perform.

Embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified with one or more software modules to perform the processing according to the present invention, and vice versa.

Although the present invention has been described by specific matters such as specific components and limited embodiments and drawings, it is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art may make various modifications and changes from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is defined not only in the claims below, but also in the ranges equivalent to or equivalent to those of the claims. Will belong to.

Claims

As a way to allow a user to play a digital instrument based on his or her voice,

Receiving information regarding the user's selection of a particular digital instrument among the various digital instruments,

Receiving a voice of a user and transmitting a voice signal,

Extracting information about the pitch and the intensity of the voice signal according to time, and generating a MIDI signal using the extracted pitch and the information about the intensity of the sound,

Extracting or generating a MIDI sound source corresponding to the specific digital instrument with reference to the generated MIDI signal, and

Outputting the MIDI sound source

Including,

The voice signal is applied to the information on the performance control performed by the user using the controller,

The MIDI signal includes a note on message indicating the start of the voice signal, a pitch wheel message indicating an irregular pitch change of the voice signal, and a modulation indicating a regular pitch change of the voice signal. ) A message, a volume message indicating the volume of the voice signal, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal. How to.
The method of claim 1,

And the speech signal is subjected to noise filtering and amplification on the speech signal input by the user.
The method of claim 1,

In the MIDI signal generating step, according to time, the frequency of the speech signal is quantized to extract information about a pitch, the amplitude of the speech signal is quantized to extract information about a sound intensity, and then the digitized pitch And generating information about the loudness as a MIDI signal conforming to the MIDI protocol.
A system for allowing a user to play a digital instrument based on his or her voice,

A user interface unit for receiving information regarding a user's selection of a specific digital instrument among various digital instruments,

A voice player for receiving a voice of a user and transmitting a voice signal, wherein the voice player includes a controller;

A voice signal processing unit for applying information on a performance control performed by the user using the controller to the voice signal;

A MIDI signal generator for extracting information on the pitch and sound intensity of the voice signal according to time, and generating a MIDI signal using the extracted information on the pitch and sound intensity; and

A MIDI sound source management unit which extracts or generates a MIDI sound source corresponding to the specific digital instrument with reference to the generated MIDI signal, and outputs the MIDI sound source.

Including,

The MIDI signal includes a note on message indicating the start of the voice signal, a pitch wheel message indicating an irregular pitch change of the voice signal, and a modulation indicating a regular pitch change of the voice signal. ) A message, a volume message indicating the volume of the voice signal, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal. System.
The method of claim 4, wherein

And the voice signal processor further performs noise filtering and amplification on the voice signal.
The method of claim 4, wherein

The MIDI signal generation unit, over time, quantizes the frequency of the voice signal to extract information about a pitch, and quantizes the amplitude of the voice signal to extract information about a sound intensity, and then the digitized pitch and A system that generates information about loudness as a MIDI signal conforming to MIDI conventions.
As a way to allow a user to play a digital instrument based on his or her voice,

Receiving information regarding the user's selection of a particular digital instrument among the various digital instruments,

Receiving a voice of a user and transmitting a voice signal,

Extracting information about the pitch and the intensity of the voice signal according to time, and generating a MIDI signal using the extracted pitch and the information about the intensity of the sound,

Analyzing the voice signal using a voice recognition technology, and if it is determined that the voice signal includes preset voice information, generating an additional signal corresponding to the preset voice information; and

Generating a digital sound source with reference to the generated MIDI signal and the generated additional signal;

Including,

The voice signal is to apply the information about the performance control performed by the user using the controller,

The MIDI signal includes a note on message indicating the start of the voice signal, a pitch wheel message indicating an irregular pitch change of the voice signal, and a modulation indicating a regular pitch change of the voice signal. ) A message, a volume message indicating the volume of the voice signal, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal. How to.
The method of claim 7, wherein

The preset voice information includes at least one of a phoneme, a phoneme, and a syllable.
The method of claim 7, wherein

The additional signal includes a signal capable of controlling at least one of an element, an item, and a value of a sampler or synthesizer.
A system for allowing a user to play a digital instrument based on his or her voice,

A user interface unit for receiving information regarding a user's selection of a specific digital instrument among various digital instruments,

A voice player for receiving a voice of a user and transmitting a voice signal,

A MIDI signal generator for extracting information on the pitch and sound intensity of the voice signal according to time, and generating a MIDI signal using the extracted information on the pitch and sound intensity;

An additional signal generator configured to analyze the voice signal using a voice recognition technology, and generate an additional signal corresponding to the preset voice information when it is determined that the voice signal includes preset voice information; And

A sound source generation and management unit generating a digital sound source with reference to the generated MIDI signal and the generated additional signal.

Including,

The voice signal is to apply the information about the performance control performed by the user using the controller,

The MIDI signal includes a note on message indicating the start of the voice signal, a pitch wheel message indicating an irregular pitch change of the voice signal, and a modulation indicating a regular pitch change of the voice signal. ) A message, a volume message indicating the volume of the voice signal, an expression message indicating an irregular volume change of the voice signal, and a note off message indicating the end of the voice signal. System.
The method of claim 10,

The preset voice information includes at least one of a phoneme, a phoneme, and a syllable.
The method of claim 10,

The additional signal includes a signal capable of controlling at least one of an element, an item, and a value of a sampler or synthesizer.
A computer-readable recording medium for recording a computer program for executing the method according to any one of claims 1 to 3 and 7 to 9.