KR102119701B1 - Music interaction robot - Google Patents

Abstract

A music interaction robot is provided. The robot includes a microphone that receives a music signal or a song voice signal, a speaker that outputs a music signal, an operation unit including a shape memory alloy that is transformed by the supply of a power signal, and restored to the blocking of the power signal, the power A driver for generating a signal, outputting the power signal to the shape memory alloy, and extracting beat information from the music signal or the song voice signal, and determining at least one motion matching the extracted beat information, and the operation It includes a control unit for controlling the power signal output of the driving unit to operate in at least one motion determined additionally, and the at least one motion is formed by deformation and restoration of the shape memory alloy.

Description

Music interaction robot {MUSIC INTERACTION ROBOT}

The present invention relates to a music interaction robot.

The AI (Artificial Intelligence) craze is hot in the world market. In addition to making real robots, we provide customized services for numerous customers by integrating various AI technologies such as big data processing and deep learning technology.In addition, private homes other than robots used by companies such as smart speakers and home robots The market for home home robots that provide services tailored to the needs is also growing.

The robot is emerging as a new partner, not just a functional secretary, and the form of interaction is established by processing user's personal data and processing natural language. In particular, music is being used as a new element used to enhance emotional interaction between robots and people.

However, the interaction of robots and music activated in the market with smart speakers, headphones, etc. is optimized for personalized music recommendations through artificial intelligence services, and it is optimized to listen to music such as automatic volume control, or robots play music directly. It is limited to doing.

The problem to be solved by the present invention is to provide a robot that is designed with a design with high user intimacy by grafting an origami technique and interacting with a person through music.

According to one feature of the invention, the music interaction robot is a microphone that receives a music signal or a song voice signal, a speaker that outputs a music signal, a shape memory that is transformed by the supply of a power signal, and is restored to the blocking of the power signal. An operation unit including an alloy, a driving unit that generates the power signal and outputs the power signal to the shape memory alloy, and extracts time information from the music signal or the song voice signal, and at least matches the extracted time information It includes a control unit for determining one motion and controlling the power signal output of the driving unit to operate with at least one motion determined by the operation unit, and the at least one motion is formed by deformation and restoration of the shape memory alloy. .

The music interaction robot further includes an image sensor that captures a surrounding image, generates an image signal, and a communication unit that is connected to a server device that provides recommended music through a network to transmit and receive data. Analyzes the audio signal transmitted from the video signal or the video signal received from the image sensor, transmits the analysis result to the server device, receives recommended music according to the analysis result from the server device, and sends the recommended music to the speaker Simultaneously with output, the driving unit may be controlled to operate in at least one motion matching the beat information of the recommended music.

The control unit may analyze the input signal received from the microphone or the image sensor, and receive the recommended music from the server device when the analysis result satisfies a preset automatic music recommendation condition.

The controller may control the driving unit to recognize a voice signal transmitted from the microphone and operate the at least one motion according to a result of the voice signal recognition.

The control unit converts the music signal input from the microphone or output to the speaker or a song voice signal input from the microphone into an acoustic frequency signal having a pitch height and a length, and analyzes the acoustic frequency signal to generate sound The beat can be extracted through time information between notes, and the driving unit can be controlled to repeat supply and cut of the power signal according to the beat.

The control unit may select at least one motion according to the beat from a first motion expressing a bouncing motion by expanding and restoring the body, a second motion in which the body moves in the left or right direction, and a third motion in which the body rotates. Can be.

The operation unit includes an origami structure having a protruding portion including an expanded shape memory alloy wire and a folding portion including a shrinking shape memory alloy wire, forming a predetermined shape,

The expanded shape memory alloy wire is expanded when the temperature rises due to the supply of the power signal and exceeds a threshold temperature, and is restored when the supply of the power signal is stopped,

The shrink shape memory alloy wire, when the temperature rises due to the supply of the power signal exceeds the threshold temperature and is restored when the supply of the power signal is stopped,

The driving unit is connected to the expanded shape memory alloy wire and the contracted shape memory alloy wire, and is connected to a plurality of switch circuits that are turned on and off according to a control signal output from the controller, and the plurality of switch circuits, It may include a power signal generating unit for generating and outputting a power signal according to the control signal output from the control unit.

The control unit turns off a switch circuit connected to the contracted shape memory alloy wire, and repeatedly turns on or off a switch circuit connected to the expanded shape memory alloy wire to control the bouncing operation by expanding and restoring the body. The switch circuit connected to the expanded shape memory alloy wire is turned off, and the switch circuit connected to the contracted shape memory alloy wire is repeatedly turned on or off to control the motion of the body moving in the left and right or up and down directions and the motion of the body rotating. have.

According to another feature of the present invention, the music interaction robot includes a shape memory alloy that is deformed by the supply of a power signal and is restored to the blockage of the power signal, and to the beat information of the music through deformation and restoration of the shape memory alloy. A main robot operating with at least one matched motion, and detached from the main robot, having a shape of a small size that can be carried, and when separated from the main robot, music output, light emission driving, and vibration according to speech recognition And a sub-robot that performs at least one of the occurrences.

The sub-robot may perform a short-range communication connection with the main robot, transmit the input voice signal or voice recognition result to the main robot, and receive and output recommended music according to the voice recognition result from the main robot. .

The sub-robot may detect whether or not it is in contact with the main robot, control light emission driving according to whether or not it has been touched, or output a conversation voice.

According to another feature of the present invention, the music interaction robot is a body of an origami structure including a shape memory alloy that is transformed by supply of a power signal and restored to blocking of the power signal, mounted on the body, a music signal or singing voice A microphone that receives a signal, a speaker mounted on the main body, a speaker for outputting a music signal, mounted on the main body, an image sensor that captures surrounding images, and generates a video signal, mounted on the main body, detects the user's motion A motion sensor mounted on the main body, a light emitting unit including a plurality of light emitting elements that receive power and emit light, a driving unit that generates the power signal and outputs the power signal to the shape memory alloy, and a user interaction program. It includes a storage unit for storing, and at least one processor for executing the program, wherein the program analyzes an input signal input through at least one of the microphone, the image sensor, and the motion sensor, and according to the analysis result. Driving at least one of the speaker, the light emitting unit, and the driving unit, and when the input signal is a music signal or a song voice signal, extracts time signature from the input signal and performs at least one motion matching the extracted time signature information And instructions for controlling the output of the power signal of the driver so that the at least one motion is produced by the deformation and restoration of the shape memory alloy.

The music interaction robot further includes a communication unit that transmits and receives data to and from the server device through a wired/wireless network, and the program transmits an analysis result of the audio signal transmitted from the microphone or the video signal received from the image sensor to the server device. And, receiving the recommended music according to the analysis result from the server device, and outputs the recommended music to the speaker, and at the same time, the power signal of the driving unit so that at least one motion matching the beat information of the recommended music is produced It can include commands that control the output.

In the main body, a protruding portion including an expanded shape memory alloy wire and a folding portion including a shrinking shape memory alloy wire are continuously connected to form a predetermined shape, and the expanded shape memory alloy wire is supplied with the power signal. When the temperature rises and exceeds the critical temperature, it expands and is restored when the supply of the power signal is stopped, and the shrink shape memory alloy wire contracts when the temperature rises due to the supply of the power signal and exceeds the critical temperature and contracts the power. When the supply of the signal is stopped, it is restored, and the control unit may control the driving unit to selectively supply and cut off power to the shrink shape memory alloy wire and the expanded shape memory alloy wire.

The music interaction robot further includes a power supply unit that supplies operating power, and the program can include instructions for monitoring the operating power supply and driving at least one of the speaker, the light emitting unit, and the driving unit according to the monitoring result. have.

According to an embodiment of the present invention, by combining the origami technique, the rigid and inorganic design of the existing robot design is deviated, and the outer skin is designed with a soft material such as paper to provide a friendly and user-friendly robot.

In addition, unlike conventional music, which is a music recommendation service that simply listens to music for the user, it provides not only emotional comfort and psychological stability through music, but also a robot that actively responds to various pieces of music to operate. It provides various interactions.

In addition, it can be configured as a main robot and a sub-robot to provide differentiated user services as a main user and a secondary user.

1 is a block diagram showing the hardware configuration of a music interaction robot according to an embodiment of the present invention.
2 is a perspective view showing the external configuration of a music interaction robot according to an embodiment of the present invention.
3 is a flow chart showing the operation of the music interaction robot according to an embodiment of the present invention.
4 is a flow chart showing the operation of the music interaction robot according to another embodiment of the present invention.
5 is a flow chart showing the operation of the music interaction robot according to another embodiment of the present invention.
6 is a view showing an origami structure of a music interaction robot according to an embodiment of the present invention.
7 and 8 are views for explaining the principle of the bounce operation of the origami structure according to an embodiment of the present invention.
9 is a view showing a bounce operation of a music interaction robot according to an embodiment of the present invention.
10 is an exemplary view showing a main body of a music interaction robot according to an embodiment of the present invention.
11 is a view showing a section in which the power supply is controlled in order to move the music interaction robot in the left and right directions according to an embodiment of the present invention.
12 is a diagram illustrating an operation in which a music interaction robot according to an embodiment of the present invention moves in the right direction.
13 is a diagram illustrating an operation in which a music interaction robot according to an embodiment of the present invention moves in the left direction.
14 is a diagram illustrating an operation in which the music interaction robot according to the embodiment of the present invention moves in the left and right directions.
15 is a view showing a section in which the power supply is controlled in order to move the music interaction robot forward and backward according to an embodiment of the present invention.
16 illustrates an operation in which a music interaction robot according to an embodiment of the present invention moves forward and backward.
Figure 17 shows the movement of the music interaction robot according to an embodiment of the present invention continuously moving left, right, front and back.
18 is a diagram illustrating an operation in which a music interaction robot rotates according to an embodiment of the present invention.
19 is a block diagram showing a detailed configuration of a driving unit and an operating unit according to an embodiment of the present invention.
20 is a block diagram of a music interaction robot according to another embodiment of the present invention.
21 is an exemplary view illustrating separation of a main robot and a sub-robot according to an embodiment of the present invention.
22 is a perspective view showing an external configuration of a sub-robot according to an embodiment of the present invention.
23 is a block diagram showing the hardware configuration of a sub-robot according to an embodiment of the present invention.
24 is a flowchart illustrating the operation of a sub-robot according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part “includes” a certain component, this means that other components may be further included rather than excluding other components, unless specifically stated otherwise.

In this specification, expressions expressed in singular may be interpreted as singular or plural unless explicit expressions such as “one” or “single” are used.

A robot generally refers to a machine having a shape and function similar to a human, but in this specification, the robot does not limit the shape to a shape similar to a human, but is a machine that automatically handles or operates a given task by its own ability. Is defined as

The music interaction robot according to the embodiment of the present invention provides an interaction operation through music between a robot and a person. Music is an art that expresses thoughts and emotions by synthesizing beats, melodies, harmonies, and tones with certain laws and forms, using the sound and noise of the human audible area as a subject matter. Music is generally expressed through pitch and time.

The music interaction robot according to the embodiment of the present invention is an origami structure that folds or combines to form a predetermined shape in three dimensions. The music interaction robot responds to the temperature of a shape-memory alloy (SMA) to produce motion defined by the principle of stretching and restoring, and/or expanding and restoring. Here, motion is defined as three basic operations, as shown in Table 1 below.

Motion classification Motion definition First motion Bouncing action that repeats the action of inflating and restoring the body Second motion Body moving left and right, back and forth 3rd motion Body rotation in clockwise or counterclockwise direction

The music interaction robot can express dance movements in accordance with music using the three motions in Table 1. The dance movement is expressed in rhythm. The basic unit of rhythm is beat. A beat is a basic musical time consisting of a fixed number of fixed unit hours called a beat.

For example, the unit time at the time of three quarters means one quarter note length, the constant number is three, and the basic musical time corresponds to the length of three quarter notes. And periodically, every three beats accent. Assuming that the dance motion is expressed as the first motion in Table 1, the body swells whenever the three beats arrive, and restores the original shape between the previous three beats and the next three beats, thereby expressing the bouncing action that fits the three beat cycle. Can be.

The configuration and operation of the music interaction robot will be described below with reference to the drawings.

1 is a block diagram showing a hardware configuration of a music interaction robot according to an embodiment of the present invention, FIG. 2 is a perspective view showing an external configuration of a music interaction robot according to an embodiment of the present invention, and FIG. 3 is the present invention Flowchart showing the operation of the music interaction robot according to an embodiment of the present invention, Figure 4 is a flow chart showing the operation of a music interaction robot according to another embodiment of the present invention, Figure 5 is music according to another embodiment of the present invention Flowchart showing the operation of the interaction robot, Figure 6 is a view showing the origami structure of the music interaction robot according to an embodiment of the present invention, Figures 7 and 8 is the principle of the bounce operation of the origami structure according to an embodiment of the present invention 9 is a view illustrating a bounce operation of a music interaction robot according to an embodiment of the present invention, and FIG. 10 is an exemplary view of a main body compartment of a music interaction robot according to an embodiment of the present invention, and FIG. 11 Is a diagram showing a section in which the power supply is controlled in order to move the music interaction robot in the left and right directions according to an embodiment of the present invention, and FIG. 12 shows an operation in which the music interaction robot according to an embodiment of the present invention moves in the right direction 13 is a view showing the movement of the music interaction robot according to an embodiment of the present invention in the left direction, and FIG. 14 is a view showing the movement of the music interaction robot according to the embodiment of the present invention in a left and right direction. 15 is a diagram showing a section in which the music interaction robot according to an embodiment of the present invention is supplied with power to move forward and backward, and FIG. 16 is a music interaction robot according to an embodiment of the present invention, 17 shows the motion of the music interaction robot according to an embodiment of the present invention, and moves continuously to the left, right, front, and back. FIG. 18 shows that the music interaction robot according to the embodiment of the present invention rotates. FIG. 19 is a block diagram showing a detailed configuration of a driving unit and an operation unit according to an embodiment of the present invention. All.

1, the music interaction robot 100 includes an input unit 101, a microphone 103, an image sensor 105, a motion sensor 107, a communication unit 109, a light emitting unit 111, a speaker 113 , A driving unit 115, an operation unit 117, a storage unit 119, a control unit 121, and a power unit 123.

Referring to FIG. 2, the main body 110 of the music interaction robot 100 is a three-dimensional origami structure having a predetermined shape, and may have a cylindrical shape, but is not limited to such a shape.

Referring again to FIG. 1, the input unit 101 is a user input means, and generates an input signal and outputs it to the control unit 121. The input unit 101 may be configured as a key input unit, a touch unit, or the like. The input unit 101 may be located outside the main body 110, for example, may be located on the upper surface of the main body 110.

According to one embodiment, the input unit 101 may generate an input signal according to the pressure applied from the outside, for example, may be implemented in the form of a key button.

According to another embodiment, the input unit 101 may detect a user touch and generate an input signal, for example, may be implemented in the form of a touch button or a touch pad.

The microphone 103 detects a user voice, various sounds, music, and the like, generates a voice detection signal, and outputs it to the control unit 121. The microphone 103 may be at least one directional microphone, and may be mounted at various points of the body 110.

The image sensor 105 is mounted on the main body 110, photographs surrounding images, generates an image signal corresponding to the image, and outputs the image signal to the control unit 121. The image sensor 105 may be at least one camera, for example, a CCD (Charge Coupled Device) camera.

The motion sensor 107 is mounted on the main body 110, detects user motion and outputs a motion detection signal to the controller 121. For example, a motion of tapping the main body 110 by hand may be detected. The motion sensor 107 may include a gyro sensor, a touch sensor, and the like.

The communication unit 109 is connected to the music platform server 200 through a wired/wireless network (not shown), and transmits and receives data to and from the music platform server 200.

The communication unit 109 transmits the analysis result of the control unit 121 to the music platform server 200, receives music recommendation information according to the analysis result of the control unit 121 from the music platform server 200, and then transmits the analysis result to the control unit 121 To deliver.

The light emitting unit 111 receives power and emits light according to a control signal received from the control unit 121. The light emitting unit 111 includes a plurality of light emitting elements and a driving circuit that selectively drives some or all of the plurality of light emitting elements on and off. Here, the light emitting device includes a light emitting diode (LED), and in addition, a surface mount devices (SMD), an organic light emitting diode (OLED), an active matrix organic light emitting diode (AMOLED), or a passive matrix organic light emitting diode (PMOLED), etc. It can be used, but is not limited to this.

The light emitting unit 111 may emit light in a plurality of different colors as well as a single color. The light emitting unit 111 is mounted on the main body 110, and the main body 110 is made of a material capable of transmitting light from the light emitting unit 111. At least one of light emission, light emission time, light emission color, and light emission is controlled by the control unit 121 in the light emitting unit 111.

The speaker 113 is included in the main body 110, and outputs voice, music, and effect sounds output by the control unit 121.

The driving unit 115 generates a driving signal for driving the operation unit 117 under the control of the control unit 121. The operation unit 117 is driven according to the driving signal of the driving unit 115, and operates as described in Table 1. Description of the driving unit 115 and the operation unit 117 will be described later with reference to FIGS. 4 to 17.

The storage unit 119 may be referred to as a memory or a storage device. The storage unit 119 stores a program including instructions for performing an operation of the music interaction robot 100. The storage unit 119 may store data and programs required for the music interaction robot 100 to operate. Such a program implements the present invention in combination with hardware such as the storage unit 119 and at least one processor.

The storage unit 119 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory, etc.). , Magnetic Memory, Magnetic Disk, Optical Disk, Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Programmable Read Only Memory (PROM), Electrically Erasable Programmable Read-EEPROM Only Memory) may include at least one storage medium.

The control unit 121 is at least one processor, and executes a program stored in the storage unit 119, so that each component of the music interaction robot 100, that is, the input unit 101, the microphone 103, the image sensor 105 ), a motion sensor 107, a communication unit 109, a light emitting unit 111, a speaker 113, a driver 115 and a signal transmitted between the operation unit 117 processes and controls the operation. Through this, the control unit 121 controls the interaction operation with the user, and the interaction operation can be divided into a basic interaction operation and a music interaction operation using music as a medium.

First, the basic interaction operation will be described in various embodiments.

Referring to FIG. 3, the control unit 121 analyzes the input signal received from at least one of the input unit 101, the microphone 103, the image sensor 105, and the motion sensor 107 (S103), A user interaction control including at least one of light emission control, sound control, and motion control is determined according to the analysis result (S105). Then, a control signal for driving at least one of the light emitting unit 111, the speaker 113, and the driving unit 115 is generated according to the determination and output to the corresponding components 111, 113, and 115. The user interaction control of the controller 121 may be in various embodiments, and some embodiments are as follows.

According to an embodiment, the input unit 101 may be a power button. When the user presses the power button, an input signal according to a button press is output from the input unit 101 to the control unit 121. At this time, when the input signal continues for a predetermined period of time in an inactive state, the controller 121 recognizes it as a start signal, outputs a control signal requesting the driving of the light emitting unit 111 to the light emitting unit 111, and displays a speaker ( 113) to output the starting sound effect, starting voice, etc. The light emitting unit 111 emits light with a color and a light emission amount according to a control signal.

According to another embodiment, when an input signal according to a button press from the input unit 101 is continuously input for a predetermined time in an operating state, the control signal is recognized as an end signal and stops driving of the light emitting unit 111. It outputs to (111), and outputs an end effect sound, an end voice, etc. to the speaker 113. The light emitting unit 111 stops the light emitting operation according to the control signal.

According to another embodiment, the controller 121 checks the power supply amount of the power supply unit 123, generates a control signal requesting to emit light in a color set in a section to which the power supply amount belongs, and outputs the control signal to the light emitting unit 111. The power unit 123 may be a battery equipped with an external power supply terminal, a charging terminal, and the like, and each component (101, 103, 105, 107, 109, 111, 113, 115, 117, of the music interaction robot 100) 119, 121).

According to another embodiment, the control unit 121 does not input a signal from at least one of the input unit 101, the microphone 103, the image sensor 105, and the motion sensor 107, that is, a predetermined time If a command and an operation do not occur during (eg, minute), a control signal requesting to implement a preset default motion is generated and output to the driving unit 115, and a control signal requesting light emission with a specific color is emitted from the light emitting unit ( 111). At this time, the default motion may be the first motion in Table 1. The control unit 121 may drive the light emitting unit 111 to emit light while sequentially changing in a plurality of colors.

According to another embodiment, the control unit 121 recognizes an external voice input through the microphone 103. Since such a speech recognition technology is well known, a detailed description is omitted here. The controller 121 may drive the light emitting unit 111 to emit light while recognizing an external voice. When the external speech recognition fails, the controller 121 may generate a dialogue sentence informing the user of the external speech recognition failure and output the generated text to the speaker 113. Here, since the technology for generating a conversation sentence is well known, a detailed description is omitted here.

According to another embodiment, the control unit 121 checks the location where the external voice is input through the directional microphone, generates a control signal to change or move the direction to the identified location, and outputs the control signal to the driver 115. In addition, the control unit 121 may track a location where a voice signal is generated from outside using a time difference of arrival (TDOA) method. The arrival time difference method is a positioning method that determines the position of a sound source by measuring a relative difference in time of arrival of radio waves from two signal sources.

According to another embodiment, the control unit 121 analyzes the audio signal input through the microphone 103 and the video signal input through the image sensor 105 to check the location where the external audio is input, and confirms A control signal may be generated and output to the driving unit 115 to change or move the direction to the position.

According to another embodiment, when a motion detection signal is input from the motion sensor 107, the controller 121 may analyze the motion detection signal and perform an operation matching the analysis result. For example, the motion detection signal may be two claps, two taps, and so on. If a specific motion is being implemented, the controller 121 may control the driver 115 to stop driving the specific motion.

In addition, the control unit 121 may stop the operation of the driving unit 115 in the case of a voice stopping the operation as a result of the recognition of the external voice input from the microphone 103.

As such, the music interaction robot 100 provides an interaction operation with a user in various ways.

Next, the music interaction operation will be described in various embodiments.

4 is an example of a music interaction operation, and it is not necessary to operate in this order. Referring to FIG. 4, the control unit 121 receives an input signal from at least one of the input unit 101, the microphone 103, the image sensor 105, and the motion sensor 107 (S201).

The control unit 121 determines whether the input signal is a voice signal received from the microphone 103 (S203). If it is determined in step S203 that the voice signal is received, the controller 121 analyzes the voice signal based on the voice recognition technology to determine whether it is an input such as a dance command, for example,'dance' (S205). ).

If it is determined in step S205 that the dance motion command is input, the controller 121 performs a preset default motion control (S207). For example, the default motion may be set to sequentially execute the first motion, the second motion, and the third motion in Table 1.

If it is determined in step S205 that the dance motion command is not input, it is determined whether the input signal is a song voice signal received from the microphone 103 (S209).

If it is determined in step S209 that the song is not a voice signal, the control unit 121 determines whether a music recommendation command is input, for example, a command such as'Please recommend an exciting song' (S211).

If it is determined in step S211 that the music recommendation command is not input, the controller 121 analyzes the voice signal and performs operation control according to the analysis result (S213).

If it is determined in step S211 that the music recommendation command is input, the controller 121 requests the music platform server 200 according to the music recommendation command to receive the recommended music (S215). The controller 121 checks the meta data of the recommended music, and checks the time signature of the recommended music (S217).

The controller 121 matches at least one motion selected from the three motions of Table 1 to the beat information checked (S217) (S219). Then, the driving unit 115 is controlled to operate with at least one matched motion (S221).

On the other hand, if the input signal is determined to be a song voice signal in step S209, the controller 121 analyzes the voice waveform to extract time signature information (S223). The controller 121 matches at least one motion selected from the three motions of Table 1 to the beat information extracted (S223) (S219). Then, the driving unit 115 is controlled to operate with at least one matched motion (S221).

In addition, if it is determined in step S203 that the input signal is not a voice reception, the controller 121 determines whether the input signal is a music signal (S225).

If it is determined in step S225 that the input signal is a music signal, the controller 121 analyzes the music signal waveform and extracts time signature (S227). The controller 121 matches at least one motion selected from the three motions of Table 1 to the beat information extracted (S227) (S219). Then, the driving unit 115 is controlled to operate with at least one matched motion (S221).

Here, various techniques are already known as a method of extracting a beat from a song voice signal or a music signal. Briefly, for example, the controller 121 converts a music signal or a song voice signal into a digital signal, and then converts the music signal or song voice signal into a digital signal. It can be generated as an acoustic frequency signal having a length. If the interval between the sound and the sound in the sound frequency signal is 1 second, it can be seen that one beat is expressed between 1 second, so that the controller 121 can repeat the inflated motion in the first motion at 1 second intervals. have. However, it is not limited to this method, and various known techniques may be used.

In addition, if it is determined in step S225 that the input signal is not a music signal, the controller 121 determines whether the input signal is an image signal received from the image sensor 105 or a motion detection signal received from the motion sensor 107. (S229).

If it is determined in step S229 that the input signal is an image signal, the controller 121 analyzes the image signal (S231), transmits the analysis result to the music platform server 200, and recommends music according to the analysis result to the music platform server ( 200) (S233).

For example, if it is determined that the user recognizes the storybook as a result of analyzing the video signal, the controller 121 determines text from the storybook image and requests the music platform server 200 to recommend music matching the determined text. Can be received.

The controller 121 checks the time signature information through the metadata of the received music (S233) (S235). The controller 121 matches at least one motion with the time signature (S219).

The controller 121 outputs the recommended music through the speaker 113 and controls the driver 115 to operate with at least one motion matched in step S219 at the same time (S221).

In addition, when the input signal is determined to be a motion signal in step S229, the controller 121 analyzes the motion signal (S237), and operates the driver 115 to operate in at least one of three motions of Table 1 according to the analysis result. It is controlled (S239).

Referring to FIG. 5, the control unit 121 sets a period after the activation of the image sensor 105 (S301) (S303). Then, when the cycle arrives (S305), it is determined whether an input signal is generated from the microphone 103 (S307). When an input signal is generated, the input signal is analyzed (S309), and an interaction operation control according to the analysis result is performed (S311).

On the other hand, if the input signal does not occur,'It is quiet. Let's play the music?' and generate a dialogue sentence, and output it to the speaker 113 (S313). Then, the music platform server 200 is requested to receive the recommended music (S315).

The controller 121 controls the driver 115 to operate in at least one motion matching the beat information of the recommended music while outputting the received recommended music to the speaker 113 (S317) (S317 ).

As described above, the music interaction robot 100 may provide an interaction action with the user through music by implementing various dancing actions that match the beat of the music signal or the recommended music signal requested by the user. Of course, in addition to the above-described embodiment, it is possible to control various music interaction operations according to input.

In the above embodiment of the present invention, the music interaction robot 100 implements the motion of Table 1 based on the characteristics of the origami structure. The motion implementation example of the music interaction robot 100 is as follows.

Referring to Figure 6, the main body 110 of the music interaction robot 100 is made of a good elastic material, for example, paper may be used. The main body 110 is an origami structure in which a protruding portion P1 and a recessed portion P3 are continuously connected to form a three-dimensional shape. At this time, the protruding portion P1 includes an expansion wire, and the folding portion P3 includes a shrinking wire.

When a current flows by supplying power to the expansion wire of the projecting portion P1, the temperature rises, and when the temperature exceeds the critical temperature, the expansion wire of the projecting portion P1 expands in the direction ①. When the power supplied to the expansion wire is cut off, the projecting portion P1 is restored to its original shape.

Here, the expansion wire is made of a shape-memory alloy (SMA) material, and in particular, is made of a material having a characteristic of expanding when the temperature rises and exceeds a critical temperature. For example, Nitinol (Ni-Ti) wire may be used, but is not limited thereto.

When current flows by supplying power to the shrinking wire of the folding portion P3, the temperature rises, and when the temperature exceeds the critical temperature, the shrinking wire of the folding portion P3 shrinks in the direction ②. When the power supplied to the shrink wire is cut off, the folding portion P3 is restored to its original shape.

Here, the shrink wire is made of a shape memory alloy material, and in particular, is made of a material having a characteristic of shrinking when the temperature rises and exceeds a critical temperature. For example, copper wire may be used, but is not limited thereto.

The body 110 may produce the motions of Table 1 according to the expansion and restoration, and the contraction and restoration of the origami structure.

Referring to (a) of FIG. 7, when a current flows by supplying power to the expansion wire, the temperature rises, and when the temperature exceeds a critical temperature, expands in the direction (1) and folds as shown in FIG. 7(b) (P3) ) Area is reduced.

8 shows the protruding portion P1 and the foldable portion P3 from the side. 8(a) shows a state before power is supplied to the expansion wire. When power is supplied to the expansion wire, the region of the protruding portion P1 rises and the area of the folding portion P3 decreases as shown in FIG. This expansion causes an operation of protruding the surface of the body 110 toward the front.

The power supply to the expansion wire is cut off, and the temperature decreases, and when the temperature falls below a critical temperature, the protruding portion P1 and the folding portion P3 are restored to their original shape.

In this way, as the expansion and restoration are repeated operations, as shown in FIG. 9, the main body 110 is inflated toward the front side and then restored, and inflated and restored in the vertical direction to implement a bounce operation. This bounce motion is produced similar to breathing motion.

Referring to FIG. 10, the main body 110 is divided into an A axis and a B axis. The A-axis divides the main body 110 in the horizontal direction facing the ground. The B axis divides the main body 110 in the vertical direction.

Referring to FIG. 11, when power is supplied to the shrink wire included in the first section P5 which is the circumferential surface of the body 110 that divides the body 110 into A and B axes, current flows, and the temperature rises. Then, when the temperature exceeds the critical temperature, it contracts in the ② direction (FIG. 2), and the body 110 extends to the right with respect to the B axis as shown in FIG. In addition, when the power supply to the shrink wire of the first section P5 is cut off, the temperature decreases and becomes below a critical temperature, and the main body 110 is restored to its original shape. With such repetition of contraction and restoration, the body 110 directs a movement moving to the right with respect to the B axis.

Referring to FIG. 11 again, when power is supplied to the shrink wire of the second section P7, which is the circumferential surface facing the first section P5, based on the B axis, when current flows, the temperature rises and the temperature is critical When the temperature is exceeded, it contracts in the ② direction, and the main body 110 extends to the left with respect to the B axis as shown in FIG. 13. In addition, when the power supply to the shrink wire of the second section P7 is cut off, the temperature falls and becomes below a critical temperature, and the main body 110 is restored to its original shape. With such repetition of contraction and restoration, the main body 110 directs the movement to the left with respect to the B axis.

When the power supply is alternately made to the contraction wire of the first section P5 and the contraction wire of the second section P7, the main body 110 directs the movement in the left and right directions based on the B axis as shown in FIG. 14. do.

In addition, referring to FIG. 15, the body 110 is further divided into an A-axis and a B-axis in addition to a C-axis. The C axis partitions the body 110 in a vertical direction that intersects the B axis at a right angle.

When power is supplied to the shrink wire of the third section P9, which is the circumferential surface of the body 110 partitioned by the A-axis and the C-axis, when the current flows, the temperature rises, and when the temperature exceeds the critical temperature, ② Shrinking in the direction, as shown in FIG. 16, the front body, ie, the front body 110, is extended with respect to the C axis. And when the power supply to the shrink wire of the third section (P9) is cut off, the temperature falls and becomes below the critical temperature, the main body 110 is restored to its original shape. With such repetition of contraction and restoration, the main body 110 directs a movement moving forward based on the C axis.

In addition, although not shown in the drawing, when power is supplied to the shrink wire of the rear section (not shown), which is the circumferential surface facing the third section P9 based on the C axis, when current flows, the temperature rises and the temperature is the critical temperature. If it exceeds, it contracts in the direction of ② in FIG. 2, and the rear body, that is, the body 110 is extended, based on the C axis. And when the power supply to the shrink wire of the rear section (not shown) is cut off and the temperature falls and becomes below a critical temperature, the main body 110 is restored to its original shape. With such repetition of contraction and restoration, the main body 110 directs a motion to move backward with respect to the C axis.

When the power supply is alternately made to the shrink wire of the third section P9 and the shrink wire of the rear section (not shown), the body 110 directs the movement in the forward and backward directions.

In addition, when power is supplied to each of the shrink wires of the first section P5, the second section P7, the third section P9, and the rear section, the main body 110 is left and right as shown in FIG. 17. Directs movements that move continuously in the forward and backward directions.

10 to 17, the first section (P5), the second section (P7), the third section (P9), the back section (not shown), the power supply to each of the shrink wire is made at the same time, a specific section Compared to this other section, the amount of power supply is higher, so that the temperature rises more quickly, so that when contracted more in the direction ② in FIG. 2, the main body 110 rotates in the section.

Referring to FIG. 18, when the amount of power supply increases in the order of the first section P5, the rear section (not shown), the second section P7, and the third section P9, if the contraction amount of the shrink wire is different, counterclockwise The body 110 rotates in the direction. Conversely, if the amount of power supply increases in the order of the first section (P5), the third section (P9), the second section (P7), and the rear section (not shown), the shrinkage of the shrink wire is different, the body 110 in the clockwise direction ) Rotates.

The configurations of the driving unit 115 and the operation unit 117 that implement the motions described in FIGS. 6 to 18 are the same as in FIG. 19.

Referring to FIG. 19, the operation unit 117 includes expansion wires 117-1 and shrinking wires 117-3 partitioned on the basis of the A-axis, B-axis, and C-axis, respectively. The driver 115 includes a plurality of switch circuits 115-1 and at least one power signal generator 115-3.

The expansion wires 117-1 and the shrinking wires 117-3, respectively partitioned based on the A-axis, the B-axis, and the C-axis are connected to different switch circuits 115-1. For example, in FIG. 11, the shrink wire of the second section P7 and the shrink wire of the third section P9 are connected to different switch circuits 115-1. The plurality of switch circuits 115-1 selectively output the supply of the power signal to the connected expansion wire 117-1 or contraction wire 117-3 according to the control signal output from the control unit 121.

When the plurality of switch circuits 115-1 are switched on according to a control signal received from the control unit 121, an expansion wire 117-1 or a contraction connecting the power signal supplied from the power signal generation unit 115-3 It outputs with the wire 117-3. When the plurality of switch circuits 115-1 are switched off according to the control signal received from the control unit 121, the supply of the power signal to the connected expansion wire 117-1 or contraction wire 117-3 is stopped.

The power signal generation unit 115-3 generates a power signal according to the control signal received from the control unit 121 and outputs it to each switch circuit 115-1. At this time, the power signal generation unit 115-3 may vary the power supply amount of the power signal output for each switch circuit 115-1.

At this time, the storage unit 119 of FIG. 1 includes power supply control information for driving the motions described with reference to FIGS. 6 to 18. The power supply control information includes section partition information of the main body 110, for example, which expansion wire 117-1 or contraction wire 117-3 is controlled and cut off to supply power to produce motion. Is set. When the motion to be executed is determined, the control unit 121 extracts control information for driving the motion from the storage unit 119 and outputs it to the power signal generation unit 115-3.

The moving unit 125 may include an operating member moving in a preset direction according to a control signal received from the control unit 121. Here, the operating member may be a motor and a wheel.

When an input signal is generated from the microphone 103, the control unit 121 may drive the moving unit 125 so that the main body 110 moves toward a position where the input signal is generated.

According to an embodiment of the present invention, the music interaction robot 100 may be composed of a main robot and a sub-robot.

20 is a configuration diagram of a music interaction robot according to another embodiment of the present invention, FIG. 21 is an exemplary view of separation of a main robot and a sub-robot according to an embodiment of the present invention, and FIG. 22 is a diagram of an embodiment of the present invention It is a perspective view showing the external configuration of the sub-robot, Figure 23 is a block diagram showing the hardware configuration of the sub-robot according to an embodiment of the present invention, Figure 24 is a flow chart showing the operation of the sub-robot according to an embodiment of the present invention.

Referring to FIG. 20, the music interaction robot includes a main robot 100 and a sub-robot 300 coupled to the main robot 100 to be detachable. Here, the main robot means the music interaction robot 100 described with reference to FIGS. 1 to 19, and detailed description is omitted.

Referring to FIG. 21, the sub-robot 300 may be fixed to or coupled to the coupling portion 120 provided on the upper surface of the main robot 100, or may be separated from the coupling portion 120. At this time, the sub-robot 300 may be coupled to the coupling portion 120 using various known coupling methods, for example, a fitting coupling method, a magnet attachment method, etc. may be used, but is not limited thereto.

Referring to FIG. 22, the sub-robot 300 is designed in a tri-pod shape having a good grip feeling of a waterproof material, and is configured to be easy to handle and easy to hold by a user, and to operate in water.

Referring to FIG. 23, the sub robot 300 includes an input unit 301, a microphone 303, a speaker 305, a light emitting unit 307, a communication unit 309, a vibration generating unit 311, and a storage unit 313 , A power supply unit 315 and a control unit 317.

The input unit 301, the microphone 303, the speaker 305, the light emitting unit 307, the storage unit 313, and the power supply unit 315 are the input unit 101, the light emitting unit 111, and the speaker 113 described in FIG. ), the same functions as the storage unit 119 and the power supply unit 123 are omitted.

The input unit 301 is configured in the form of a button, and as illustrated in FIG. 22, may be provided outside the sub-robot 300.

The light emitting unit 307 may be arranged in a light emitting device so as to emit light throughout the body of the sub-robot 300, but is not limited to this embodiment, and various implementations are possible.

The communication unit 309 may connect to the music platform server 200 through a wired or wireless network (not shown), and transmit and receive data to and from the music platform server 200. Alternatively, the communication unit 309 may be connected to the main robot 100 through short-range communication, and may communicate with the music platform server 200 through the main robot 100.

The vibration generating unit 311 generates vibration under the control of the control unit 317.

The storage unit 313 stores a program including commands for performing an operation of the sub-robot 300. The storage unit 313 may store data and programs required for the sub-robot 300 to operate. Such a program implements the present invention in combination with hardware such as a storage unit 313 and at least one processor.

The control unit 317 is at least one processor, and executes a program stored in the storage unit 313, so that the input unit 301, a microphone 303, a speaker 305, a light emitting unit 307, a communication unit 309, vibration The generator 311, the storage 313, and the power unit 315 is driven.

Referring to FIG. 24, the control unit 317 analyzes (S403) the input signal received from the input unit 301 and/or the microphone 303 (S403), and performs user interaction, sound control, and light emission control according to the analysis. The speaker 305, the light emitting unit 307, and the vibration generating unit 311 are controlled to perform at least one of vibration control (S405). The user interaction operation of the sub-robot 300 may be in various embodiments, but some embodiments are described below.

According to an embodiment of the present invention, the control unit 317 drives the light emitting unit 307 to allow the user to check whether the power unit 123 is being charged. For example, when charging is completed, the light emitting unit 307 is controlled to emit green light, and when charging is failed, the light emitting unit 307 is controlled to emit red, and when charging is performed, the light emitting unit 307 emits orange light. Can be controlled.

According to another embodiment of the present invention, the control unit 317 outputs a predetermined sound to the speaker 305 when an input signal is received from the input unit 301.

According to another embodiment of the present invention, when a voice signal is input from the microphone 303, the control unit 317 controls the light emitting unit 307 to emit light in a specific color, for example, blue or green, and is simultaneously determined. By outputting the sound to the speaker 305, it is possible to inform the user that the voice is being recognized.

According to another embodiment of the present invention, if the voice recognition fails, the control unit 317 controls the light emitting unit 307 to emit light in a specific color, for example, red, and at the same time, a sound effect such as a warning sound ( 305).

According to another embodiment of the present invention, the control unit 317 controls the light emitting unit 307 to produce flickering during tethering with the main robot 100, and if tethering fails, for example, red The color is controlled to emit light, and at the same time, an effect sound such as a warning sound is output to the speaker 305.

According to another embodiment of the present invention, a configuration (not shown) that detects contact with the main robot 100 is added to the outer bottom of the sub-robot 300. As the touch sensing configuration (not shown), a current sensing method, a pressure sensing method, or the like may be used. That is, when the sub robot 300 is mounted on the main robot 100, the control unit 317 receives a touch detection signal. However, when the sub-robot 300 is separated from the main robot 100, since the touch detection signal is not received, the control unit 317 recognizes the separation state of the sub-robot 300. As described above, when the separation state is recognized, the light emitting unit 307 is controlled to emit light in a specific color, for example, purple.

According to another embodiment of the present invention, the controller 317 analyzes the voice signal received from the microphone 303, and if it is determined to be a music recommendation command, recommends music according to the music recommendation command from the music platform server 200 It receives and outputs to the speaker 305.

At this time, if the music recommendation command is determined to be a lullaby request, the lullaby sound composed of white noise stored in the storage unit 313 may be output to the speaker 305 without access to the music platform server 200.

According to another embodiment of the present invention, the controller 317 may recognize ambient noise through an input signal received from the microphone 303 while outputting recommended music to the speaker 305. At this time, if the ambient noise is not recognized, the music volume may be automatically reduced or the music output may be stopped.

The embodiment of the present invention described above is not implemented only through an apparatus and method, and may be implemented through a program that realizes a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (15)

A microphone that receives a music signal or a song voice signal,
Speaker that outputs music signals,
Operation unit including a shape memory alloy that is modified by the supply of the power signal, and restored to the blocking of the power signal,
A driver for generating the power signal and outputting the power signal to the shape memory alloy, and
A control unit for extracting time information from the music signal or the song voice signal, determining at least one motion matching the extracted time information, and controlling the power signal output of the driving unit to operate with the determined at least one motion Including,
The at least one motion,
It is formed by deformation and restoration of the shape memory alloy,
The control unit,
Music interaction that selects at least one motion according to the beat from the first motion expressing the bouncing motion after the body is expanded and restored, the second motion the body moves in the left or right or up and down motion, and the third motion the body rotates robot. In claim 1,
An image sensor that captures surrounding images and generates image signals, and
The server device providing the recommended music is connected through a network, and further includes a communication unit for transmitting and receiving data,
The control unit,
Analyze the audio signal transmitted from the microphone or the video signal received from the image sensor, transmit the analysis result to the server device, receive recommended music according to the analysis result from the server device,
A music interaction robot that outputs the recommended music to the speaker and controls the driving unit to operate in at least one motion matching the beat information of the recommended music. In claim 2,
The control unit,
A music interaction robot that analyzes an input signal received from the microphone or the image sensor and receives the recommended music from the server device when the analysis result satisfies a preset automatic music recommendation condition. In claim 2,
The control unit,
A music interaction robot that recognizes a voice signal transmitted from the microphone and controls the driving unit to operate in at least one motion according to a result of the voice signal recognition. In claim 2,
The control unit,
Converts the music signal input from the microphone or output to the speaker or the song voice signal input from the microphone into an acoustic frequency signal having a pitch height and a length, and analyzes the acoustic frequency signal to determine the time between the sound and the sound Extracting the beat from the information,
A music interaction robot that controls the driving unit to repeat supply and interruption of the power signal according to the beat. delete In claim 1,
The operation unit,
It includes a protruding portion comprising an expanded shape memory alloy wire and a folding portion comprising a shrinking shape memory alloy wire, and includes an origami structure forming a predetermined shape,
The expanded shape memory alloy wire,
When the temperature rises due to the supply of the power signal and exceeds a threshold temperature, it expands and is restored when the supply of the power signal is stopped.
The shrink shape memory alloy wire,
When the temperature rises due to the supply of the power signal and exceeds a threshold temperature, it contracts and is restored when the supply of the power signal is stopped.
The driving unit,
A plurality of switch circuits connected to the expanded shape memory alloy wire and the contracted shape memory alloy wire and turned on and off according to a control signal output from the controller; and
A power signal generator that is connected to the plurality of switch circuits and generates and outputs a power signal according to a control signal output from the controller
Including, music interaction robot. In claim 7,
The control unit,
The switch circuit connected to the contracted shape memory alloy wire is turned off, and the switch circuit connected to the expanded shape memory alloy wire is repeatedly turned on or off to expand and restore the body to control the bouncing operation,
The switch circuit connected to the expanded shape memory alloy wire is turned off, and the switch circuit connected to the contracted shape memory alloy wire is repeatedly turned on or off to control a motion in which the body moves in the left and right or up and down directions and a motion in which the body rotates. , Music interaction robot. A main robot that is transformed by supply of a power signal and restored to blocking of the power signal, and operates in at least one motion matched to the beat information of the music through deformation and restoration of the shape memory alloy, And
A sub-robot detached from the main robot and having a shape of a small size that can be carried, and separated from the main robot, performing at least one of music output, light emission driving, and vibration generation according to speech recognition
Including, music interaction robot. In claim 9,
The sub-robot,
A music interaction robot that performs a short-range communication connection with the main robot, transmits the input voice signal or voice recognition result to the main robot, and receives and outputs recommended music according to the voice recognition result from the main robot. In claim 9,
The sub-robot,
A music interaction robot that detects whether or not it is in contact with the main robot, controls light emission driving according to whether it is in contact, or outputs a conversational voice. The body of the origami structure including a shape memory alloy that is transformed by the supply of the power signal and restored to the blocking of the power signal,
A microphone mounted on the main body and receiving a music signal or a song voice signal,
A speaker mounted on the main body and outputting a music signal,
An image sensor mounted on the main body and capturing an image of the surroundings to generate an image signal,
It is mounted on the body, a motion sensor for detecting the user's motion,
A light emitting unit mounted on the main body and including a plurality of light emitting elements that emit light when supplied with power,
A driver for generating the power signal and outputting the power signal to the shape memory alloy,
A storage unit for storing user interaction programs, and
And at least one processor that executes the program,
The above program,
Analyzing an input signal input through at least one of the microphone, the image sensor and the motion sensor, and driving at least one of the speaker, the light emitting unit and the driving unit according to the analysis result,
When the input signal is a music signal or a song voice signal, time signature information is extracted from the input signal, at least one motion matching the extracted time signature information is determined, and the at least one motion is modified and restored by the shape memory alloy. Music interaction robot comprising instructions to control the power signal output of the driving unit so that the motion is directed. In claim 12,
Further comprising a communication unit for transmitting and receiving data with the server device through a wired or wireless network,
The above program,
The analysis result of the audio signal transmitted from the microphone or the video signal received from the image sensor is transmitted to the server device, and the recommended music according to the analysis result is received from the server device.
And a command for controlling the power signal output of the driving unit so as to output the recommended music to the speaker and at least one motion matching the beat information of the recommended music is produced. In claim 12,
The main body,
The protruding portion including the expanded shape memory alloy wire and the folded portion including the contracted shape memory alloy wire are continuously connected to form a predetermined shape,
The expanded shape memory alloy wire,
When the temperature rises due to the supply of the power signal and exceeds a threshold temperature, it expands and is restored when the supply of the power signal is stopped.
The shrink shape memory alloy wire,
When the temperature rises due to the supply of the power signal and exceeds a threshold temperature, it contracts and is restored when the supply of the power signal is stopped.
The above program,
And instructions for controlling the driving unit to selectively supply power to the contracted shape memory alloy wire and the expanded shape memory alloy wire and cut off the power supply. In claim 12,
Further comprising a power supply for supplying operating power,
The above program,
A music interaction robot including instructions for monitoring the operating power and driving at least one of the speaker, the light emitting unit, and the driving unit according to a monitoring result.

Images