KR101166822B1 - Apparatus and method for digital music mixing by multi touch - Google Patents

Abstract

The present invention provides a digital music mixing apparatus using the multi-touch, and a method thereof, comprising: a processor (10) for controlling digital music mixing operations using multi-touch by transmitting information necessary for multi-touch using a TUIO protocol; A BLU 21, an infrared LED 22, a fixed frame 23, and a flash application display unit 24 are provided to receive multi-touch information of a user using an infrared light source, and under the control of the processor 10. A display unit 20 for outputting digital music mixing information; A sub lamp 31 for outputting an infrared light source to the display unit 20 and a touch recognition camera 32 made of an infrared camera to recognize the multi-touch information of the display unit 20, the touch of the display unit 20 A touch recognition unit 30 for recognizing and transferring the same to the processor 10; It includes a microphone 41 and the speaker 42, the input and output unit 40 for performing input and output processing under the control of the processor 10; comprising a multi-touch using an infrared LED and a touch recognition camera It implements a screen, transmits the information needed for multi-touch using the TUIO protocol, applies it to a flash application, and displays the existing sound controller and instrument functions on the LCD screen so that the user can simultaneously respond to touch of multiple fingers or fans. You will be able to mix various sound sources in real time.

Description

Apparatus and method for digital music mixing by multi touch}

The present invention relates to a digital music mixing device and a method thereof, and more particularly, to implement a multi-touch screen using an infrared light emitting diode (LED) and a touch recognition camera, and to a multi-touch using a TUIO (Tangible User Interface Object) protocol. It transmits the necessary information, applies it to the flash application, and displays the existing sound controller and instrument functions on the LCD screen so that the user can mix various sound sources in real time using touch response of multiple fingers or fans at the same time. A digital music mixing device using multi-touch and a method thereof.

In general, digital music mixing refers to mixing and outputting electronic digital music. Also, touch means to recognize only one finger at a time, and multi-touch means to recognize two or more fingers.

In the related art, a technique of "implementing a piano using a multi-touch screen" (Publication No. 2010-0028988) has been disclosed.

This conventional technique is similar to the present invention in that the music is played through the piano using a multi-touch screen, but similar to the present invention, using the TUIO protocol as in the present invention transmits information necessary for multi-touch and flash application There is no function that acts as a digital music mixer, such as applied to there is a big difference from the present invention.

Accordingly, the present invention has been proposed to solve the above-mentioned conventional problems, and an object of the present invention is to implement a multi-touch screen using an infrared LED and a touch recognition camera, and use information required for multi-touch using the TUIO protocol. Multi-touch, which can be applied to flash applications, displays existing sound controllers and instrument functions on the LCD screen, and allows users to mix various sound sources in real time using multiple finger or fan touch responses. The present invention provides a digital music mixing device and a method thereof.

2 is a block diagram of a digital music mixing apparatus using multi-touch according to an embodiment of the present invention.

As shown therein, the processor 10 transmits information necessary for multi-touch using the TUIO protocol to control digital music mixing operations using the multi-touch; A BLU (Back Light Unit) 21, an infrared LED 22, a fixed frame 23, and a flash application display unit 24 are provided to receive multi-touch information of a user using an infrared light source. A display unit 20 for outputting digital music mixing information under the control of the processor 10; A sub lamp 31 for outputting an infrared light source to the display unit 20 and a touch recognition camera 32 made of an infrared camera to recognize the multi-touch information of the display unit 20, the touch of the display unit 20 A touch recognition unit 30 for recognizing and transferring the same to the processor 10; And an input / output unit 40 including a microphone 41 and a speaker 42 and performing input / output processing under the control of the processor 10.

The processor 10 includes: a touch recognition processor 11 for performing image processing by using the multi-touch information recognized by the touch recognition unit 30; The touch recognition processor 11 receives the processing information, converts the white touch points displayed on the screen recognized by the touch recognition camera 32 into x, y coordinates proportional to the screen size, and converts the plurality of points. A touch point application 12 for simultaneously switching each of the same effects as a mouse click; Receives the information of the touch point application 12, reproduces a sound source or a sound source pre-designated in the processor 10 to provide a screen to the display unit 20, and the sound source through the speaker 42 Mixing application 13 for outputting a; characterized in that comprises a.

The processor 10 performs a turntable and a mixer function to simultaneously process a plurality of sound sources, provides a fader function to convert volume and pan attributes, and provides a cue function to play from a predetermined part of a sound source. It is characterized by processing to provide a sample of the instrument, including drums, synthesizers, bass to control a variety of sound sources.

The processor 10 provides a drum machine and a keyboard function, simultaneously outputs various drum sounds through nine predetermined drum buttons in the case of a drum machine, and uses a sound source such as playing a piano in the case of a keyboard. Characterized in that the processing to play through the touch.

The touch recognition camera 32 is characterized in that the visible light is filtered on the webcam and the exposed film is attached so that only infrared rays are recognized.

5 is a flowchart illustrating a digital music mixing method using multi-touch according to an embodiment of the present invention.

As shown in the drawing, the infrared LED is emitted so that infrared rays are distributed on the surface of the display unit 20, and when the user touches the proximal surface of the display unit 20, the infrared rays of the touch point are detected by the touch recognition unit 30. A touch input step ST10 to ST30 for total reflection in the direction in which the touch recognition camera 32 is located; An input processing step (ST40 to ST50) of recognizing the totally reflected infrared signal and receiving and processing the screen of the touch recognition camera 32 by the processor 10 after the touch input step; After the input processing step, the touch point application in the processor 10 receives touch recognition information, executes a mixer function specified in coordinates, and then outputs mixed digital music through the display unit 20 and the speaker 42. It characterized in that it comprises a; application steps (ST60 ~ ST80).

FIG. 6 is a detailed flowchart of the input processing step of FIG. 5.

As shown therein, the input processing step includes: binning, by the processor 10, the image of the touch recognition camera 32 into black and white pixels (ST51); The processor (10) includes the steps of limiting the minimum and maximum size of points (ST52); The processor (10) removing the noise and the background on the screen to be output to the display unit (ST53); The processor (10) expanding the recognition screen by the screen magnification of the display unit (ST54); The processor 10 converts the touch point into x, y coordinate values on the screen (ST55); The processor 10 may include converting the touch point into a corresponding coordinate click effect (ST56).

FIG. 7 is a detailed flowchart of an application step in FIG. 5.

As shown in the drawing, the application step may include: arranging an acoustic controller and connecting the processor 10 with the touch event to connect the execution of a function with the touch event (ST71, ST72); The processor 10 performs a Fourier transform on the output sound and visualizes and displays the converted values (ST73, ST74).

Digital music mixing apparatus and method using a multi-touch according to the present invention implements a multi-touch screen using an infrared LED and a touch recognition camera, transmits information necessary for multi-touch using the TUIO protocol, and apply to flash applications In addition, by displaying the existing sound controller and instrument functions on the LCD screen, the user can mix various sound sources in real time using a touch response such as multiple fingers or fans at the same time.

In addition, the present invention has the advantage that can be used in a single device having a variety of existing expensive expensive sound equipment.

In addition, the present invention can be arranged and changed in a user-oriented interface rather than a unilateral design.

Furthermore, the present invention can be used to control various functions at the same time using a multi-touch screen from the limitation that only one point can be recognized in the existing single touch screen.

In addition, the present invention can be made less space constraint than when using a plurality of real sound equipment and musical instruments.

In addition, the present invention may present a new industrial development possibility as a touch controller deviating from the existing sound device control method.

In addition, the present invention can enable the development of various multimedia controllers at low cost in a short time.

1 is a conceptual diagram illustrating an entire system to which a digital music mixing device using the multi-touch of the present invention is applied.
2 is a block diagram of a digital music mixing apparatus using multi-touch according to an embodiment of the present invention.
FIG. 3 is a perspective view illustrating an example of implementing a digital music mixing device using the multi-touch of FIG. 2.
4 is a conceptual diagram illustrating a basic operation example of the digital music mixing apparatus using the multi-touch of FIG. 2.
5 is a flowchart illustrating a digital music mixing method using multi-touch according to an embodiment of the present invention.
FIG. 6 is a detailed flowchart of the input processing step of FIG. 5.
FIG. 7 is a detailed flowchart of an application step in FIG. 5.
8 is a view showing an example screen configuration by the flash application display unit in the present invention.
9 is a diagram showing an example of the screen configuration by the flash application display unit when the piano keyboard function is included in the present invention.
FIG. 10 is a diagram showing an example of a screen configuration by a flash application display unit in the case of including other code functions in the present invention.
11 shows an example of a device implemented by the present invention.
12 is a view showing the overall appearance of FIG.
FIG. 13 is a view illustrating the inside of FIG. 12.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of a digital music mixing apparatus and a method using multi-touch according to the present invention configured as described above will be described in detail with reference to the accompanying drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. It is to be understood that the following terms are defined in consideration of the functions of the present invention, and may be changed according to the intention of the user, the operator, or the precedent, and the meaning of each term should be interpreted based on the contents will be.

First of all, the present invention implements a multi-touch screen using an infrared LED and a touch recognition camera, transmits information necessary for multi-touch using the TUIO protocol, applies it to a flash application, and applies existing sound controller and instrument functions to a liquid crystal display. It is intended to allow users to mix various sound sources in real time using touch reactions such as multiple fingers or fans at the same time.

Here, TUIO Protocol stands for Tangible User Interface Object Protocol. It is a communication protocol between various multi-touch tracking hardware and applications, and it can be used in various languages and environments by converting tracked information into an abstract concept.

The necessity of the present invention can be summarized as follows.

1) New interface for music equipment

-Realize digital music mixer using multi-touch based on major subjects learned so far from existing sound equipment with limited functions made of analog circuit

-It is possible to move with the minimum of space restrictions, and users can select and use the desired function.

-Implement the functions of several music devices on one monitor screen and present a new interface that can be operated simultaneously using multi-touch

2) New Multi-Touch Control Method

-The multi-touch function for freely manipulating the screen through the fingertips is expected to be applied to the overall IT devices with the recent application of iPhone and Microsoft's Windows 7

-Most devices can be operated by controlling the movement of the pointer more intuitive and natural than tapping the keyboard and clicking the mouse.

-Multi-touch function is required according to the necessity of simultaneous mixing and playing to perform the function of existing hardware equipment

In the present invention, without the purchase of an expensive infrared camera is removed to remove the infrared filter attached to the webcam and attached to the exposed film to block visible light retrofit to recognize only infrared

3) Challenge to the creative field

-Challenges to creative fields based on what they have learned in their majors, and attempts to be useful in many ways

1 is a conceptual diagram illustrating an entire system to which a digital music mixing device using the multi-touch of the present invention is applied.

Thus, the main features of the present invention are as follows.

-Simultaneous output of two or more sound sources

-Adjustable playback status and properties for each sound source

-User can select various sound sources

-You can specify the desired playback point (Que point) of each sound source

-Auto fader function between playback source channels

-Virtual drum machine

-Selectable basic drum loops

-Built-in sound effects playback, virtual scratch effect

-Play the basic guitar chords (C, F, G, Am, E7, Dm7, etc.) with just a touch

-1 octave virtual piano

-Equalizer display for the sound being played

-Full playback sound can be stopped at the same time

-Multi-touch screen that can recognize infinite touch points

-Ball type wheels are attached for convenience of movement

2 is a block diagram of a digital music mixing apparatus using multi-touch according to an embodiment of the present invention.

First, the processor 10 transmits information required for multi-touch using the TUIO protocol to control digital music mixing operations using the multi-touch. The processor 10 may use a PC equipped with a quad-core CPU, a graphics card, and a sound card for the speed of simultaneous multiple processing.

The processor 10 may include a touch recognition processor 11, a touch point application 12, and a mixing application 13.

The touch recognition processor 11 performs image processing by using the multi-touch information recognized by the touch recognition unit 30.

The touch point application 12 receives the processing information from the touch recognition processor 11, converts the white touch point displayed on the screen recognized by the touch recognition camera 32 into x, y coordinates proportional to the screen size, Each of the converted points is simultaneously converted into an effect such as a mouse click. The touch point application 12 performs image processing using a C # program on a PC.

The mixing application 13 receives the information of the touch point application 12, reproduces a sound source pre-stored in the processor 10 or a sound source designated on a program, provides a screen to the display unit 20, and provides a speaker 42. Output sound source through And the mixing application 13 can be produced using Adobe Flash and AS (Action Script) 3.0 language, and operates in the form of pre-stored internal / playback of the sound source specified in the program, and the data provided by the library of the production language. You can use processing methods.

In addition, the processor 10 performs the following main functions.

First, it performs turntable and mixer functions to provide simultaneous playback of multiple sound sources, provides a fader function to convert volume and pan properties, provides a cue function to play back from a specific part of the sound source, and to control various sound sources. Process to provide instrument samples, including drums, synthesizers, and bass.

Secondly, it provides drum machine and keyboard function, and in case of drum machine, it can output various drum sounds at the same time through nine predetermined drum buttons, and in case of keyboard, it can be played by touch using a sound source such as playing piano. Do it.

In addition, the display unit 20 includes a BLU 21, an infrared LED 22, a fixed frame 23, and a flash application display unit 24. The display unit 20 receives multi-touch information of a user using an infrared light source, and receives a processor 10. Digital music mixing information is output according to the control.

Therefore, the display unit 20 is connected to a PC using only the LCD panel and the AD board (analog digital conversion circuit), and as a light source, two fluorescent light bulb type BLU (back light units) and two small high-brightness LED bulbs Can be used. In addition, an acrylic plate is attached to the panel surface of the display unit 20 to protect the panel surface during touch input.

In addition, the display unit 20 includes an infrared LED 22 and a fixed frame 23. An infrared light source is used to check a touch recognition point, and the infrared LED 22 is positioned at four sides of the display screen at intervals of about 2 cm. Place it towards the center. In addition, when the infrared surface generated on the screen of the display unit 20 is pressed, the light at that point is totally reflected downward (the direction in which the touch recognition camera 32 is located). In addition, the display unit 20 may supply power through an independently connected 5V 1mA adapter.

The touch recognition unit 30 includes a sub lamp 31 for outputting an infrared light source to the display unit 20 and a touch recognition camera 32 made of an infrared camera to recognize multi-touch information of the display unit 20. The touch of 20 is recognized and transmitted to the processor 10.

In this case, the touch recognition camera 32 is disposed at the bottom of the screen by modifying the webcam so that visible light is filtered and only infrared rays are recognized. The touch recognition camera 32 may be configured by removing an infrared cut filter (glass) inside a typical webcam. In addition, it is configured by attaching an exposed film to prevent visible light from passing outside the lens of the webcam. Therefore, the touch recognition camera 32 transmits 30 frames per second to the PC by receiving the infrared rays totally reflected upon touch.

The input / output unit 40 includes a microphone 41 and a speaker 42, and performs input / output processing under the control of the processor 10.

FIG. 3 is a perspective view illustrating an example of implementing a digital music mixing device using the multi-touch of FIG. 2. So in consideration of the convenience of movement can be configured to include a wheel 50 for moving the ball-type wheels.

4 is a conceptual diagram illustrating a basic operation example of the digital music mixing apparatus using the multi-touch of FIG. 2.

Therefore, the digital music mixing device using the multi-touch operates in the order of touch input-> infrared recognition-> processor processing-> screen driving.

5 is a flowchart illustrating a digital music mixing method using multi-touch according to an embodiment of the present invention.

First, the infrared LED is emitted so that infrared rays are distributed on the surface of the display unit 20 (ST10), and when the user touches the proximal surface of the display unit 20 (ST20), the infrared rays of the touch point are the touch recognition unit 30. To the total reflection (ST30) in the direction of the touch recognition camera 32.

The touch recognition camera 32 recognizes the totally reflected infrared signal (ST10), and the processor 10 receives and processes the screen of the touch recognition camera 32 (ST50).

In addition, the touch point application in the processor 10 receives touch recognition information (ST60), executes the mixer function specified in the coordinates (ST70), and outputs the mixed digital music through the display unit 20 and the speaker 42. Do it.

FIG. 6 is a detailed flowchart of the input processing step of FIG. 5.

Thus, the processor 10 binarizes the image of the touch recognition camera 32 into black and white pixels (ST51). In addition, the processor 10 limits the minimum and maximum size of the point (ST52). The processor 10 removes the noise and the background on the screen to be output to the display unit 20 (ST53). In addition, the processor 10 expands the recognition screen by the screen magnification of the display unit 20 (ST54). Then, the touch point is converted into x, y coordinate values on the screen (ST55). The touch point is converted into the corresponding coordinate click effect (ST56).

FIG. 7 is a detailed flowchart of an application step in FIG. 5.

First, the sound device controller is disposed and connected to the processor 10 (ST71), and then the touch event and function execution are connected (ST72). The processor 10 performs a Fourier transform on the output sound (ST73) and visualizes and displays the converted value (ST74).

As such, the present invention implements a multi-touch screen using an infrared LED and a touch recognition camera, transmits information required for multi-touch using the TUIO protocol, applies it to a flash application, and applies existing sound controller and instrument functions to an LCD screen. In this case, the user can mix various sound sources in real time using touch response of multiple fingers or fans at the same time.

Although the present invention has been described in more detail with reference to the examples, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but are intended to illustrate and not limit the scope of the technical spirit of the present invention. The protection scope of the present invention should be interpreted by the claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

10: processor
11: touch recognition processing unit
12: touch point application
13: Mixing Application
20: display unit
21: BLU
22: infrared LED
23: fixed frame
24: flash application display unit
30: touch recognition unit
31: sub lamp
32: touch recognition camera
40: input / output unit
41: microphone
42: speaker

Claims (8)

A processor 10 for controlling digital music mixing operations using multi-touch by transmitting information required for multi-touch using a TUIO protocol;
A BLU 21, an infrared LED 22, a fixed frame 23, and a flash application display unit 24 are provided to receive multi-touch information of a user using an infrared light source, and under the control of the processor 10. A display unit 20 for outputting digital music mixing information;
A sub-lamp 31 which outputs an infrared light source to the display unit 20, an infrared camera that recognizes the multi-touch information of the display unit 20, and which is exposed to the webcam to filter visible light and to recognize only infrared light. A touch recognition unit 30 including a touch recognition camera 32 configured to recognize the touch of the display unit 20 and transmit the touch recognition unit 30 to the processor 10;
An input / output unit (40) having a microphone (41) and a speaker (42), and performing input / output processing under the control of the processor (10);
Digital music mixing device using a multi-touch, characterized in that configured to include.
The method according to claim 1,
The processor 10,
A touch recognition processor 11 for performing image processing using the multi-touch information recognized by the touch recognition unit 30;
The touch recognition processor 11 receives the processing information, converts the white touch points displayed on the screen recognized by the touch recognition camera 32 into x, y coordinates proportional to the screen size, and converts the plurality of points. A touch point application 12 for simultaneously switching each of the same effects as a mouse click;
Receives the information of the touch point application 12, reproduces a sound source or a sound source pre-designated in the processor 10 to provide a screen to the display unit 20, and the sound source through the speaker 42 A mixing application 13 for outputting the mixing;
Digital music mixing device using a multi-touch, characterized in that configured to include.
The method according to claim 1,
The processor 10,
Performs turntable and mixer functions to provide simultaneous playback of multiple sound sources, provides fader functions to convert volume and pan properties, provides cue function to play from a specific part of the sound source, and controls various sound sources. A digital music mixing device using multi-touch, characterized in that the processing to provide a sample of the instrument, including drums, synthesizers, bass.
The method according to claim 1,
The processor 10,
Provides drum machine and keyboard function, drum machine can output several drum sounds at the same time through nine predefined drum buttons, and keyboard can play through touch using sound source such as playing piano Digital music mixing device using a multi-touch, characterized in that.
delete Infrared LEDs are made to emit light so that infrared rays are distributed on the surface of the display unit 20, and when the user touches the proximal surface of the display unit 20, the infrared rays of the touch point are filtered by the webcam of the touch recognition unit 30. And the touch input step (ST10 ~ ST30) to be totally reflected in the direction of the touch recognition camera 32 attached to the exposed film so that only infrared light is recognized;
An input processing step (ST40 to ST50) of recognizing the totally reflected infrared signal and receiving and processing the screen of the touch recognition camera 32 by the processor 10 after the touch input step;
After the input processing step, the touch point application in the processor 10 receives touch recognition information, executes a mixer function specified in coordinates, and then outputs mixed digital music through the display unit 20 and the speaker 42. An application step ST60 to ST80;
Digital music mixing method using a multi-touch, characterized in that including a.
The method of claim 6,
The input processing step,
Binning, by the processor (10), an image of the touch recognition camera (32) into black and white pixels (ST51);
The processor (10) includes the steps of limiting the minimum and maximum size of points (ST52);
The processor (10) removing the noise and the background on the screen to be output to the display unit (ST53);
The processor (10) expanding the recognition screen by the screen magnification of the display unit (ST54);
The processor 10 converts the touch point into x, y coordinate values on the screen (ST55);
The processor 10 converts the touch point into a corresponding coordinate click effect (ST56);
Digital music mixing method using a multi-touch, characterized in that including a.
The method according to claim 6 or 7,
The application step,
Arranging an acoustic controller and connecting the processor 10 to a touch event and executing a function execution (ST71 and ST72); The processor 10 performs a Fourier transform of the output sound and visualizes and displays the converted value (ST73, ST74);
Digital music mixing method using a multi-touch, characterized in that including a.

Images