KR20220113112A - Artificial intelligence robotics instrument playing technology and music education system using the same - Google Patents

Abstract

The present invention relates to an artificial intelligence robotics instrument playing technology and a music education system using the same that include a step of using real-time class data for a tutor generated by a tutor device and rendering an extended reality image of a learner device interacting with the tutor device on a display of the tutor device. The real-time class data includes voice data input in real time through the tutor device, motion data of a tutor, and state data of an instrument being played by the tutor.

Description

Artificial intelligence robotics instrument playing technology and music education system using the same

The present invention relates to an artificial intelligence robotic instrument performance technology and a music education and system using the same.

Recently, as interest in art culture increases and the demand for hobbies to enjoy such art culture increases, the demand for realizing music tutoring through public or private education is rapidly increasing.

However, in the case of music tutoring, which is currently generally implemented, there is a space limitation to provide music education to a large number of people at the same time, and there is a lack of physical infrastructure such as instruments and teaching aids, as well as providing professional education on various instruments. There is also a problem that human infrastructure is insufficient.

In addition, it is difficult for learners to find the best expert who can provide customized music tutoring in the way they want, and it is difficult for both learners and experts to find the most suitable learner for the music education that they can provide. There is a problem that it is not easy to provide a satisfactory tutoring experience.

On the other hand, with the development of extended reality technology encompassing virtual reality, augmented reality, and mixed reality, telepresence technology can be implemented. By applying such telepresence technology to various fields, non-face-to-face video solutions Attempts are being made to provide

The present invention has been devised to solve the above-mentioned conventional problems, and by providing a music tutoring service using an extended reality technology, it provides a non-face-to-face music tutoring service in an environment close to an actual face-to-face education environment while providing spatial constraints , the purpose is to solve the shortage of physical infrastructure and the shortage of human infrastructure.

In addition, the present invention provides a customized music tutoring service to learners and tutors by matching the optimal tutor who can provide the music tutoring desired by the learner and matching the optimal learner suitable for the music tutoring that the tutor can provide. has a purpose

According to some embodiments of the present invention, the artificial intelligence robotic instrument performance technology and music education using the artificial intelligence robotics instrument performance by a system including one or more learner devices, one or more tutor devices and a server, by the server, the one or more matching the learner device and the tutor device for performing augmented reality music tutoring based on the learner information transmitted from the learner devices and the tutor information transmitted from the one or more tutor devices; integrating, by the server, the learner environment information transmitted from the matched learner device and the tutor environment information transmitted from the matched tutor device; generating, by the server, real-time class data for a tutor based on real-time learner data transmitted from the learner device; rendering, by the tutor device, an extended reality image of the learner device interacting with the tutor device on the display of the tutor device using the generated real-time lesson data for the tutor; generating, by the server, real-time class data for learners based on real-time tutor data transmitted from the tutor device; and rendering, by the learner device, an extended reality image about the tutor device interacting with the learner device on the display of the learner device using the generated real-time class data for the learner. .

The present invention solves spatial constraints, lack of physical infrastructure, and lack of human infrastructure by providing music tutoring service non-face-to-face in an environment close to the actual face-to-face education environment by providing a music tutoring service using extended reality technology. At the same time, it allows learners and tutors to communicate in real time.

In addition, the present invention provides a customized music tutoring service to learners and tutors by matching the optimal tutor who can provide the music tutoring that the learner wants, and matching the optimal learner suitable for the music tutoring that the tutor can provide. make it possible

1 shows the concept of an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.
2 shows a detailed configuration of an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.
3 shows a flowchart for providing a music tutoring service by an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.
4 shows detailed flowcharts of an artificial intelligence algorithm by a system in accordance with some embodiments of the present invention.
5 illustrates a learner information input page and a tutor selection page provided to a learner device in an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.
6 illustrates a tutor information input page and a learner selection page provided to a tutor device in an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.
7 illustrates examples of extended reality images implemented by a learner device and a tutor device while performing extended reality music tutoring in an AI-based extended reality music tutoring service providing system according to some embodiments.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. The following description is only for specifying the embodiments, and is not intended to limit or limit the scope of rights according to the present invention. What a person of ordinary skill in the art can easily infer from the detailed description and embodiments of the present invention should be construed as belonging to the scope of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments according to the present invention will be described in detail with reference to the drawings. Detailed description of matters widely known to those of ordinary skill in the art of the present invention will be omitted.

First, in the present invention, extended reality (XR, eXtended Reality) is a hyper-realistic technology that encompasses virtual reality (VR, Virtual Reality), augmented reality (AR, Augmented Reality) and mixed reality (MR, Mixed Reality) technology and It can be defined as referring to a service.

In addition, in the present invention, a learner is defined as referring to all those who want to learn to play an instrument, and a tutor refers to all those who have playing ability to provide instrument lessons to the learner, such as a professional player, a music teacher, etc. can be defined as

1 illustrates an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.

The extended reality music tutoring service providing system according to some embodiments of the present invention is accessible to the network (N) through wired or wireless communication, stores information, and communicates with the learner device (A) and the tutor device (B) It is possible to connect to the network (N) through the server (S), wired or wireless communication to and may include a tutor device (B) that transmits and receives information to and from the server (S). At this time, if necessary, one or more learner devices (A) and one or more tutor devices (B) may be connected to the server (S).

2 shows a detailed configuration of an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention. Hereinafter used '… wealth', '… The term 'group' means a unit for processing at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

Referring to FIG. 2 , the server S includes a communication unit 91 , a control unit 92 , and a storage unit 93 . More specifically, the communication unit 91 may provide an interface for performing communication with other devices in the system, and may transmit and receive information with the learner device (A) and the tutor device (B). In addition, the control unit 92 controls the overall operations of the server S. For example, the control unit 92 matches the learner device (A) and the tutor device (B), generates avatar images of users of the learner device (A) and the tutor device (B), and the learner device (A) and Processes the voice to be transmitted to the tutor device (B), generates data that causes the learner device (A) and the tutor device (B) to render an extended reality image, and the learner’s playing pattern using the learner device (A) , controls the operations of the server (S), which will be described later in detail, as in analyzing. To this end, the control unit 92 may be implemented as an array of a plurality of logic gates or a general-purpose microprocessor. The control unit 92 may include at least one processor. In addition, the storage unit 93 stores data such as a basic program for the operation of the server S, an application program, and setting information. The storage unit 93 may be configured as a volatile memory, a non-volatile memory, or a combination of a volatile memory and a non-volatile memory. In addition, the storage unit 93 provides the stored data according to the request of the control unit 92 .

In addition, referring to FIG. 2 , the learner device (A) and the tutor device (B) are, respectively, a terminal, an extended reality implementation device (VR/AR/MR device, motion state detection device, audio, etc.) and an input device (microphone, musical instrument, etc.) etc.) is a concept that includes

In addition, the learner device (A) and the tutor device (B) may further include a body tracking device, and the motion detected through this is implemented as an image through a virtual avatar (student avatar, tutor avatar, etc.) can be

In addition, the terminal may mean including an integrated unit including a device for transmitting a signal through a wired or wireless Internet network. That is, it corresponds to a device for accessing the Internet, and broadly, input devices such as keyboard and mouse for inputting various data, output devices such as monitors and printers, transceivers such as modems or LANs, microprocessors or graphic cards, etc. It may be composed of a processing device and a storage device in which various programs for operating or controlling them are mounted. Examples of such a terminal include a Personal Communication System (PCS), a Global System for Mobile communications (GSM) terminal, a Personal Digital Cellular (PDC) terminal, a Personal Handyphone System (PHS) terminal, and a personal information terminal ( Personal Digital Assistant (PDA), smart phone, telematics, wireless data communication terminal and/or portable Internet terminal, personal computer and notebook computer may be exemplified.

3 shows a flowchart for providing a music tutoring service by an artificial intelligence-based extended reality music tutoring service providing system according to some embodiments of the present invention. Fig. 3 illustrates a flow chart relating to the exchange of signals between the learner device (A), the server (S) and the tutor device (B).

Referring to FIG. 3 , in step S11 , the learner device (A) may transmit learner information to the server (S). Further, in step S12, the tutor device B can transmit the tutor information to the server S. Further, in step S13, the server S configures the learner device A for performing extended reality music tutoring based on the learner information transmitted from the one or more learner devices and the tutor information transmitted from the one or more tutor devices, and It is possible to match the tutor device (B). In this case, as will be described later with reference to FIG. 4 , the matching may be performed using data analysis and modeling based on artificial intelligence (AI). In addition, although not shown in FIG. 3 , the server S may perform step S13 by receiving a matching request from the learner device A, or may perform step S13 by receiving a matching request from the tutor device B have.

At this time, the learner information includes music genre information for which the learner wants to receive tutoring (eg, classical, jazz, song, etc.), information on the instrument that the learner wants to receive tutoring (eg, piano, guitar, violin, etc.), and the learner's musical instrument performance level information (e.g., beginner, intermediate, advanced, etc.), information on the tutoring method preferred by the learner and tutoring time information (e.g., one-to-one learning, one-to-many learning, long classes, short classes, etc.), and information on the learner's tutoring purpose (e.g.: Mastering one song perfectly, building a solid foundation from the ground up, repeating past lessons, etc.) In addition, the tutor information includes music genre information (eg, classical, jazz, song, etc.) tutored by the tutor, information on the instrument tutored by the tutor (eg, piano, guitar, violin, etc.), and performance level information (eg, : Beginner, intermediate, advanced, etc.), past tutoring method information and tutoring time information (eg, one-on-one learning, one-to-many learning, long lessons, short lessons, etc.) things, building a solid foundation from the ground up, repeating past lessons, etc.). Additionally, the tutor information may include music genre information for which the tutor wants to provide tutoring, instrument information, performance level information of a tutoring target, tutoring method information, tutoring time information, and tutoring purpose information.

Through this, based on artificial intelligence, it is possible to match the optimal tutor considering various factors such as the learner's level and the learner's preferred music genre, tutoring method, purpose, etc., and taking the tutor's past tutoring history into consideration It is also possible to match learners who are most suitable for them. In addition, in some embodiments of the present invention, one-to-many tutoring may be performed by selecting the same tutor device by a plurality of learner devices.

In step S14, the server S may transmit to the learner device A a result of the tutor matching informing of the matched tutor device B. In addition, the learner device (A) determines to receive extended reality music tutoring from the tutor device (B) included in the tutor matching result, and in step S15, sends a tutor selection message to the server ( S) can be sent to In this case, the tutor matching result may include information on at least one tutor device.

In step S16, the server S may transmit the learner matching result informing the matched learner device A to the tutor device B. In addition, the tutor device (B) determines to provide extended reality music tutoring to the learner device (A) included in the learner matching result, and in step S17, a learner selection message notifying that the learner device (A) has been selected to the server It can be sent to (S). In this case, the learner matching result may include information on at least one learner device.

In this case, steps S14 to S15 and steps S16 to S17 may be selectively performed. Specifically, when the optimal tutor is proposed to the learner device (A) in step S13, steps S14 to S15 are performed, and when the optimal learner is proposed to the tutor device (B) in step S13, steps S16 to S17 are performed. can be performed.

In addition, in step S18, the server S transmits a matching completion message notifying that the matching is completed to the matched learner device A, and also in step S19, also sends a matching completion message to the matched tutor device B. can

In step S20, the learner device (A) may transmit the learner environment information to the server (S). In this case, the learner environment information may include background information of the first space where the learner is located, location information of the learner in the first space, and location and shape information of the instrument to be played by the learner in the first space. For example, if the learner is in his/her own room and wants to receive piano tutoring, the learner environment information includes background information on the size, shape, arranged furniture, etc. of the room, the learner's location information in the room, the location of the piano in the room, and It may include specific shape information such as a piano key.

In step S21, the tutor device B may transmit the tutor environment information to the server S. In this case, the tutor environment information may include background information of the second space where the tutor is located, location information of the tutor in the second space, and location and shape information of the instrument to be played by the tutor in the second space, This may be exemplified as the above-described learner environment information.

In step S22, the server S may integrate the learner environment information and the tutor environment information. In step S22, the server S may generate the initial integrated environment information for the learner and the initial integrated environment information for the tutor based on the integration of the learner environment information and the tutor environment information.

In this case, the initial integrated environment information for the learner may include the tutor avatar image and the first space-related information to which the tutor environment information is applied based on the learner environment information, and the initial integrated environment information for the tutor includes the learner avatar image and Second space-related information to which the learner environment information is applied based on the tutor environment information may be included. For example, the initial integrated environment information for the learner may be initial information obtained by mixing the learner environment information and the tutor environment information in order to express an image in which the tutor avatar interacts with the piano placed in the learner's room. Integrated environment information may be understood as a similar concept. Alternatively, the initial integrated environment information for learners may include learner environment information applied to the virtual studio, and the initial integrated environment information for tutors may include tutor environment information applied to the virtual studio. In this case, the following learner device (A) and the tutor device (B) may be implemented to display an extended reality image in which the learner avatar and the tutor avatar exist in the same virtual studio.

The server S may transmit the learner's initial integrated environment information to the learner device A in step S23, and may transmit the tutor initial integrated environment information to the tutor device B in step S24.

In step S25, the learner device A may render an extended reality image including the tutor avatar on the display of the learner device using the initial integrated environment information for the learner. For example, an image in which the tutor avatar exists in a space with the background of the room in which the learner is currently located may be implemented on the display of the extended reality implementation device worn by the learner.

In step S26, the tutor device B may render the extended reality image including the learner avatar on the display of the tutor device using the initial integrated environment information for the tutor.

In step S27, the learner device A may transmit real-time learner data to the server S. In this case, the real-time learner data may include voice data input in real time through the learner device, the learner's motion data, and the state data of the instrument the learner is playing.

In step S28, the server S may generate real-time class data for the tutor based on the real-time learner data. More specifically, the server (S) matches real-time learner data with the tutor environment information, so that the learner avatar interacts with the instrument played by the tutor in real time in the space where the tutor is located. can create

Also, in step S29, the server S may transmit real-time lesson data for a tutor to the tutor device B, and in step S30, the tutor device B uses the real-time lesson data for the tutor, and the tutor device B ) may be rendered on the display of the tutor device (B) of the extended reality image about the learner device (A) interacting with. More specifically, the tutor device B may render an extended reality image in which the learner avatar interacts with a musical instrument played by the tutor in real time on the display of the tutor device. Through this, the tutor can watch the learner avatar playing the tutor's piano in his or her room through the extended reality image implemented on the display of the tutor device.

Also, in step S31, the tutor device B can transmit real-time tutor data to the server S. In this case, the real-time tutor data may include voice data input in real time through the tutor device, operation data of the tutor, and state data of the instrument the tutor is playing.

In step S32, the server S may generate real-time class data for learners based on the real-time tutor data. More specifically, the server (S) matches the real-time tutor data with the learner environment information, so that the tutor avatar interacts with the instrument played by the learner in real time in the space where the learner is located. can create

In step S33, the server S may transmit real-time class data for the learner to the learner device A, and in step S34, the learner device A uses the real-time class data for the learner, the learner device A It is possible to render an extended reality image about the tutor device (B) interacting with the learner device (A) on the display. More specifically, the learner device A may render an extended reality image in which the tutor avatar interacts with a musical instrument played by the learner in real time on the display of the learner device. Through this, the learner can watch the tutor avatar performing a demonstration through the learner's piano in his or her room through the extended reality image implemented on the display of the learner's device.

Meanwhile, steps S27 to S34 have been described sequentially for convenience of explanation, and steps S27 to S30 and steps S31 to S34 are performed in parallel depending on the time at which the real-time learner data and the real-time tutor data arrive at the server S, respectively. can be performed.

Additionally, in step S35, the server S may analyze the learner's playing pattern based on the data in steps S27 to S34 described above. Specifically, deficiencies in the learner's performance ability may be analyzed, and the server S may additionally consider factors for supplementing the deficiencies when matching the learner and the tutor in step S13 later. Through this, a tutor capable of providing feedback to improve the skill of the learner can be matched.

Such a learner avatar and a tutor avatar may be implemented as a volumetric image by an extended reality implementation device (eg, at least one of VR glasses, MR glasses, or AR glasses) worn by the tutor and the learner, respectively. The extended reality implementation device may be included in the tutor device and the learner device as described above.

With the artificial intelligence robotics instrument performance technology as described above and music education using the same, real-time performance of a player (learner or tutor) who is remotely connected can be implemented by interworking with the instrument in the field where the counterpart is located. In addition, the performance of such a performer is realized in real time in the form of an extended reality image at the site where the other party is located, so that a non-face-to-face music tutoring service by telepresence technology can be provided. In some embodiments of the present invention, the musical instrument may include a musical instrument played directly by a player, a musical instrument playable by artificial intelligence, or a virtual instrument implemented in extended reality, but is not limited to the above examples.

4A and 4B show detailed flowcharts of step S13 of matching a learner and a tutor based on artificial intelligence by the system according to some embodiments of the present invention. FIG. 4A illustrates a case in which the server S receives a matching request from the learner device A, and FIG. 4B illustrates a case in which the server S receives a matching request from the tutor device B. Referring to FIG.

First, referring to FIG. 4A , in step S41 , the server S may input learner information of the learner device A that has transmitted the matching request into the tutor matching model. Also, in step S42, the server S may identify the characteristics of the music tutoring service requested by the learner of the learner device A from the learner information input to the tutor matching model. That is, the server S may identify characteristics such as the level of the learner and the music genre preferred by the learner, the tutoring method, and the purpose. Further, in step S43, the server S determines, based on the tutor matching model, among one or more tutor devices connected to the server S, the tutor device corresponding to the tutor information matching the characteristic of the identified music tutoring service ( B) can be extracted.

Referring to FIG. 4B , in step S44 , the server S may input the tutor information of the tutor device B that has transmitted the matching request to the tutor matching model. Also, in step S45, the server S may identify the characteristics of the music tutoring service that the tutor of the tutor device B can provide from the tutor information input to the tutor matching model. That is, the server S may identify at least one of the characteristics related to the tutor's past tutoring history or the characteristics related to the tutoring that the tutor wants to provide. Further, in step S46, the server S, based on the tutor matching model, among one or more learner devices connected to the server S, the learner device corresponding to the learner information matching the characteristic of the identified music tutoring service ( A) can be extracted.

In addition, matching between a learner and a tutor according to some embodiments of the present invention is performed by artificial intelligence, and the tutor matching model used for matching may be modeled by learning such as machine learning. More specifically, the server S receives information on the performance level of the learners for each instrument, analyzes the performance level of the learners, and models the characteristics of the tutor corresponding to the performance level of each learner, thereby providing a tutor A matching model may be generated, and the accuracy of the tutor matching model may be further improved through learning. In this case, the server S may generate, based on the tutor matching model, a model of the artificial intelligence tutor having the characteristics of the tutor corresponding to the performance level of each learner, and may store it.

In this case, the AI tutor's model divides established music that is open to the public by genre, inclination, harmony, rhythm, time signature, year, composer, lyricist, arranger, lyrics, that is, by attribute, and organizes it into a database. A data set generated by learning the database according to each attribute may be stored. In addition, the model of the AI tutor may perform music tutoring through an extended reality image by making voice utterances in the form of text to speech (TTS). In addition, the AI tutor model may provide learner-specific music tutoring by learning various educational courses according to the learner information of the matched learner.

On the other hand, although not shown in FIGS. 3 and 4A , when the learner device A transmits a matching request to the server S in step S11, matching with an AI tutor other than the tutor devices connected to the server S may request. More specifically, when matching with the artificial intelligence tutor is requested, the server S generates a model of the artificial intelligence tutor that matches the learner device A based on the tutor matching model, or uses the stored model of the artificial intelligence tutor can be called While music tutoring using such an artificial intelligence tutor is performed, the artificial intelligence tutor may be implemented in the form of an avatar by the extended reality implementation device of the learner device (A). That is, as described above, like music tutoring by a tutor using a tutor device, the AI tutor's avatar may also be implemented in conjunction with the space where the learner is located.

When the music tutoring is performed by the artificial intelligence tutor, real-time performance data of the learner of the learner device A is input, so that a part with a high frequency of mistakes by the learner can be automatically identified by the artificial intelligence tutor. Accordingly, the artificial intelligence tutor can analyze the learner's tendency to make mistakes and suggest corrections to the learner's performance method. This correction proposal may be provided as an image in which a virtual musical instrument is played in the extended reality implementation device of the learner device A, or as an image in which an avatar of an artificial intelligence tutor plays in the corrected method.

In addition, the artificial intelligence tutor may recommend a song to be played or practiced next to the learner by analyzing the level of growth through the learner's practice after suggesting to the learner the correction of the playing method. In addition, the AI tutor may recommend that the learner practice a song about a preferred music genre, which can be identified by the learner information. Furthermore, the artificial intelligence tutor identifies a learner similar to the level of the learner of the learner device A among learners using the learner devices connected to the server S, so that the learner device A communicates with the identified learner. You can also recommend an ensemble. In addition, the AI tutor may propose to the learner an ensemble with the AI tutor by generating music and phrases corresponding to the learner's performance level (eg, rhythm, accompaniment, harmony, etc.).

5 illustrates a learner information input page 500 and a tutor selection page 510 provided to a learner device in an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.

Specifically, the learner information input page 500 may be provided in step S11. The learner information input page 500 is, as described in FIG. 3 , an area 501 in which the learner inputs information on a music genre desired for tutoring, an area 502 in which instrument information is input, and information on the learner's musical instrument performance level. It may include an area 503 for inputting information on the learner's preferred tutoring method, an area 504 for inputting the learner's preferred tutoring method information, an area 505 for inputting the learner's preferred tutoring time information, and an area 506 for inputting the learner's tutoring purpose. .

Also, the tutor selection page 510 may be provided in step S14. For example, the tutor selection page 510 may include an area 511 for displaying information about the matched tutor, and the area 511 may include an area for displaying information on tutoring that can be provided by the tutor ( 511a), an area 511b for displaying an avatar or photo of the corresponding tutor, and a selection area 511c for selecting the corresponding tutor. Also, when a plurality of tutors are matched, the tutor selection page 510 may further include areas 512 and 513 for displaying information about the tutor.

6 illustrates a tutor information input page 600 and a learner selection page 610 provided to a tutor device in an AI-based extended reality music tutoring service providing system according to some embodiments of the present invention.

Specifically, the tutor information input page 600 may be provided in step S12 . The learner information input page 600 includes an area 601 for selecting whether or not the tutor directly inputs information, a music genre information area 602, an instrument information area 603, a learner level information area 604, a preferred tutoring method It may include an information area 605 , a preferred tutoring time information area 606 , and a tutoring purpose information area 607 . When 'Yes' is selected in the area 601, areas 602 to 607 can be directly set by the tutor, and when 'No' is selected in the area 601, areas 602 to 607 may be automatically set based on the tutor's past tutoring history stored in the server (S).

Also, the learner selection page 610 may be provided in step S16. For example, the learner selection page 610 may include an area 611 displaying information about the matched learner, and the area 611 may include an area 611a displaying information about tutoring desired by the learner; It may include an area 611b for displaying the avatar or picture of the learner and a selection area 611c for selecting the learner. Also, when a plurality of learners are matched, the learner selection page 610 may further include an area 612 or the like for displaying information about the learner.

7 shows an extended reality image 700 and a tutor device B implemented by the learner device (A) while the extended reality music tutoring is performed in the AI-based extended reality music tutoring service providing system according to some embodiments. ) shows examples of the extended reality image 710 implemented by . More specifically, the extended reality image 700 may be implemented by the extended reality implementation device of the learner device (A) and displayed on the display, and the extended reality image 700 is the tutor avatar ( 701) may be implemented to appear. In addition, the extended reality image 710 may be implemented by the extended reality implementation device of the tutor device B and displayed on the display, and the extended reality image 710 may be a learner avatar or a learner avatar in a space where the tutor actually exists. It can be implemented so that an expanded reality image 711 such as an enlarged image (eg, a hand playing a piano key, a foot stepping on a piano pedal, etc.) of a part that the tutor needs to look at in detail in the performance of .

Methods according to the embodiments described in the claims or specifications of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, various modifications are possible without departing from the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments and should be defined by the claims described below as well as the claims and equivalents.

A: learner device, B: tutor device, S: server, N: network, 91: communication unit, 92: control unit, 93: storage unit

Claims (10)

rendering an extended reality image about the learner device interacting with the tutor device on a display of the tutor device using real-time class data for a tutor generated by the tutor device;
The real-time class data includes voice data input in real time through the tutor device, operation data of the tutor, and state data of the instrument the tutor is playing. The method of claim 1,
The real-time learner data includes voice data input in real time through the learner device, the learner's motion data, and the state data of the instrument the learner is playing. The method of claim 1,
The tutor information, including music genre information tutored by the tutor, instrument information tutored by the tutor, performance level information of the past tutoring target, past tutoring method information, past tutoring time information, and purpose information of past tutoring, artificial intelligence robotics Musical instrument performance technology and music education system using the same. 3. The method of claim 2,
The step of matching the learner device and the tutor device by the server,
Based on the tutor matching model, from among the one or more learner devices connected to the server, extracting the learner device corresponding to learner information matching the characteristics of the identified music tutoring service, artificial intelligence Robotics instrument performance technology and music education system using it. 4. The method of claim 3,
extracting the tutor device corresponding to the tutor information matching the characteristics of the identified music tutoring service from among the one or more tutor devices connected to the server, based on the tutor matching model; and
Identifying the characteristics of the music tutoring service that the tutor of the tutor device can provide by inputting the tutor information of the tutor device into the tutor matching model; Artificial intelligence robotics instrument performance technology and a music education system using the same, further comprising; . 5. The method of claim 4,
and transmitting, by the tutor device, a learner selection message indicating that the matched learner device has been selected, to the server. The method of claim 1,
The tutor environment information includes background information of a second space in which the tutor is located, location information of the tutor in the second space, and location and shape information of an instrument to be played by the tutor in the second space, ,
The initial integrated environment information for the tutor includes a learner avatar image and second space-related information to which the learner environment information is applied based on the tutor environment information, artificial intelligence robotics instrument performance technology and a music education system using the same. 8. The method of claim 7,
receiving, by the learner device, the initial integrated environment information for the learner from the server; and
The initial integrated environment information for learners includes first space-related information to which the tutor environment information is applied based on the tutor avatar image and the learner environment information. The method of claim 1,
The learner information includes music genre information for which the learner wants to receive tutoring, information on the instrument that the learner wants to receive tutoring, information on the learner's musical instrument performance level, information on the tutoring method preferred by the learner, and tutoring time information. Technology and music education system using it. The method of claim 1,
matching, by the server, a learner device and a tutor device for performing augmented reality music tutoring based on the learner information transmitted from the one or more learner devices and the tutor information transmitted from the one or more tutor devices;
and generating, by the server, real-time class data for learners based on real-time tutor data transmitted from the tutor device.

Images