KR20230037092A - Apparatus and method for provide avatar on virtual stage for convergence education of art and technology - Google Patents

Abstract

According to an embodiment of the present invention, the apparatus for providing an avatar on a virtual stage for convergence education of art and technology comprises: a communication circuit configured to communicate with a user terminal; a memory; and a processor electrically connected to the communication circuit and the memory. The processor is able to receive a code expressing a plurality of objects corresponding to each of a plurality of body parts from a user terminal using the communication circuit, convert the received code into a designated format, acquire the converted code through an input window corresponding to each of the plurality of body parts, acquire a video image on a user photographed by the user terminal by using the communication circuit, acquire a plurality of coordinates corresponding to each of the plurality of body parts by analyzing the video image, and provide a virtual stage displaying a plurality of objects at locations corresponding to each of the plurality of coordinates by using the communication circuit to the user terminal. Therefore, immersion of education can be improved.

Description

Apparatus and method for providing an avatar on a virtual stage for convergence education of art and technology

Embodiments disclosed in this document relate to a device and method for implementing and outputting an avatar on a virtual stage according to a user's code input.

In 2015, the international community enacted the Sustainable Development Goals (SDGs). In particular, the education-related goals (SDG 4.7) include content on “sustainable development and global citizenship.” Sustainable Development and Global Citizenship covers environmental sustainability, a sustainable and inclusive economy, and inclusive social development. Among them, education for sustainable development (ESD), which has the theme of environmental sustainability, means education that aims for a society where people can learn the values, behaviors, and lifestyles necessary for a sustainable future and social transformation. Education related to this actually produces effects such as increased economic growth rate, increased environmental protection activities, and reduced crime rates. Therefore, there is a need to propose convergence education between arts and technology as part of education for sustainable development.

Through art and technology convergence education, participants can develop free imagination and creativity. Development and operation of various educational courses for art and technology convergence education are being attempted, but there is no educational method through an online platform. In particular, in order to respond to the untact era, the need to develop the above-mentioned online platform may be higher.

Embodiments of the present invention are to provide a device and method capable of performing education on computer programming and expression through body movements using an online platform.

An apparatus for providing an avatar on a virtual stage for convergence education of art and technology according to an embodiment disclosed in this document includes a communication circuit configured to communicate with a user terminal, a memory, and a processor electrically connected to the communication circuit and memory, , The processor receives a code representing a plurality of objects corresponding to each of a plurality of body parts from a user terminal using a communication circuit, converts the received code into a designated format, and inputs an input window corresponding to each of a plurality of body parts. Obtaining a code converted through, obtaining a video image of the user captured by the user terminal using a communication circuit, obtaining a plurality of coordinates corresponding to each of a plurality of body parts by analyzing the video image, and communicating A virtual stage displaying a plurality of objects at a position corresponding to each of a plurality of coordinates may be provided to a user terminal by using a circuit.

According to an embodiment, the processor acquires a plurality of 2-dimensional coordinates corresponding to each of a plurality of body parts by analyzing a video image using a machine learning model, and sets reference coordinates for the plurality of 2-dimensional coordinates at the bottom of the video image. It is changed to the central point, each of the changed plurality of 2D coordinates is normalized based on the height of the video image, each of the plurality of normalized 2D coordinates is converted into a plurality of 3D coordinates to correspond to the virtual stage, and the plurality of transformed 2D coordinates are converted. A virtual stage displaying a plurality of objects at positions corresponding to each of the 3D coordinates may be provided to the user terminal.

According to an embodiment, when one object is selected from among a plurality of objects on a virtual stage in a user terminal, the processor provides an input window corresponding to the selected object and changes the selected object from the user terminal using a communication circuit. A virtual stage displaying a modified object corresponding to the modified code by receiving the modified code and replacing the selected object may be provided to the user terminal.

According to an embodiment, the processor provides a QR code to the user terminal in response to a request from the user terminal, provides an additional object selection screen and an additional object control panel to another user terminal that has photographed the QR code, and provides an additional object selection screen displaying an additional object selected on the virtual stage, adjusting the position or size of the additional object on the virtual stage in response to an input to the additional object control panel, and adjoining at least some of the plurality of objects on the virtual stage and the additional object. , may provide a visual or auditory interaction corresponding to the additional object.

A method for providing an avatar on a virtual stage for art and technology convergence education performed by one or more computing devices according to an embodiment disclosed in this document expresses a plurality of objects corresponding to a plurality of body parts from a user terminal. Receiving a code to receive a code, converting the received code into a designated format, acquiring the converted code through an input window corresponding to each of a plurality of body parts, obtaining a video of the user captured by the user terminal obtaining a plurality of coordinates corresponding to each of a plurality of body parts by analyzing the video, and providing a virtual stage displaying a plurality of objects at positions corresponding to each of the plurality of coordinates to a user terminal. can do.

According to the embodiments disclosed in this document, by using a code input by a user and a video image of a user's motion, various objects corresponding to a plurality of user's motions are output on a virtual stage, thereby controlling the motion of the body. Convergence education for expression and computer programming can be performed.

In addition, by providing an additional object controlled by an education subject or an audience, it is possible to improve immersion in education by providing various stage decorations and enabling audience participation.

In addition to this, various effects identified directly or indirectly through this document may be provided.

1 illustrates an operating environment of an avatar providing device on a virtual stage according to an exemplary embodiment.
2 is a block diagram illustrating a configuration of an avatar providing device on a virtual stage according to an exemplary embodiment.
3 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
4 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
5 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
6 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
7 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
8 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
9 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.
10 is a flowchart illustrating a method of providing an avatar on a virtual stage according to an exemplary embodiment.
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar elements.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include various modifications, equivalents or substitutes of the embodiments of the present invention. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, when it is determined that a detailed description of a related known configuration or function in describing an embodiment of the present invention interferes with understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

1 illustrates an operating environment of an avatar providing device on a virtual stage according to an exemplary embodiment.

Referring to FIG. 1 , an apparatus for providing an avatar on a virtual stage according to an embodiment may be implemented in the form of a server 100 . The server 100 is a platform for art and technology convergence education, and a plurality of users (1, 2, 3) through communication with a plurality of user terminals (11, 12, 13) of the plurality of users (1, 2, 3). 3) It is possible to provide a virtual stage including an avatar that reacts according to the movement. The avatar may include a plurality of objects corresponding to each body part of the user.

For example, the server 100 may receive codes written by a plurality of users 1, 2, and 3 from a plurality of user terminals 11, 12, and 13. The server 100 may acquire video images for each of the plurality of users 1, 2, and 3 captured by the plurality of user terminals 11, 12, and 13. The server 100 may recognize the movement of each of the plurality of users 1, 2, and 3 by analyzing the video image. The server 100 may display avatars including a plurality of objects corresponding to codes written on the virtual stage according to the movements of each of the plurality of users 1, 2, and 3. A plurality of users (1, 2, 3) can learn coding by checking avatars implemented by code input, and can learn expression methods using their bodies using avatars that respond to their movements. .

While a plurality of users 1, 2, and 3 perform learning using a plurality of user terminals 11, 12, and 13, other users 4 (eg, spectators, observers, or guardians) use the user terminals 14 ) can be used to participate in learning. For example, the server 100 may provide a selection menu for selecting a prop object to the user terminal 14 and a control panel for controlling the prop object. The server 100 may display the prop object selected from the selection menu on the virtual stage, and may control the position and size of the prop object according to an input to the control panel. The server 100 may provide visual or auditory interactions between avatars corresponding to a plurality of users and prop objects. A sense of immersion in education can be improved by providing various stage decorations using prop objects and inducing active audience participation.

In FIG. 1 , the avatar providing device on the virtual stage is illustrated as being implemented by the server 100, but is not limited thereto, and the avatar providing device on the virtual stage may be implemented in the form of a user terminal, and two or more devices (e.g., A user terminal and a server, or two servers) may be implemented in a distributed environment.

2 is a block diagram illustrating a configuration of an avatar providing device on a virtual stage according to an exemplary embodiment.

Referring to FIG. 2 , an apparatus 200 for providing an avatar on a virtual stage according to an embodiment may include a communication circuit 210 , a memory 220 and a processor 230 . The avatar providing apparatus 200 on the virtual stage may be implemented, for example, in the form of a server or in the form of a user terminal such as a desktop, laptop, tablet, or smart phone. The avatar providing device 200 on the virtual stage may be physically located on premise or in the cloud. For another example, the avatar providing apparatus 200 on the virtual stage may be implemented with two or more computing devices in a distributed environment.

The communication circuit 210 may be configured to communicate with a plurality of user terminals 21, 22, 23, and 24 wirelessly or wired. The communication circuit 210 may transmit and receive data with a plurality of user terminals 21, 22, 23, and 24. For example, the communication circuit 210 may transmit and receive codes and video images.

Memory 220 may include volatile memory and/or non-volatile memory. The memory 220 may store various data handled by the avatar providing apparatus 200 on the virtual stage. For example, the memory 220 may store data processed inside the avatar providing apparatus 200 on the virtual stage or may store data received from the outside. For example, the memory 220 may temporarily or non-temporarily store codes and video images.

The processor 230 may be electrically connected to the communication circuit 210 and the memory 220 . The processor 230 may control the communication circuit 210 and the memory 220 and may perform various data processing and calculations. Although the processor 230 is shown as a single component in FIG. 2 , it may be implemented as being separated into a plurality of components. The processor 230 may perform the following operations by executing software or instructions stored in the memory 220 . Hereinafter, for convenience of description, an operation between the device 200 and the first user terminal 21 will be described, but the device 200 includes a plurality of user terminals (the first user terminal 21 and the second user terminal 21). (22) and the third user terminal 23) can perform the same operation at the same time.

According to an embodiment, the processor 230 may receive codes representing a plurality of objects corresponding to each of a plurality of body parts from the first user terminal 21 using the communication circuit 210 . The processor 230 may receive codes corresponding to body parts such as the head, shoulders, elbows, wrists, knees, and ankles from the first user terminal 21 . The code may be a code for setting the color, shape, size, position, and the like of an object corresponding to a body part. The processor 230 may provide an editor for inputting a code to the first user terminal 21 and may receive the code through the editor. The processor 230 may provide a preview corresponding to the input code to the first user terminal 21 . The user can check the object represented by the code written by the user, and can edit the code to create an object of a desired shape.

According to one embodiment, the processor 230 may convert the received code into a designated format. The processor 230 may convert the received code into a form suitable for rendering a dynamically moving object on a web page. For example, the processor 230 may convert the received code into JavaScript.

According to an embodiment, the processor 230 may obtain the converted code through an input window corresponding to each of a plurality of body parts. The processor 230 may provide the first user terminal 21 with a plurality of input windows corresponding to body parts such as the head, shoulders, elbows, wrists, knees, and ankles. The processor 230 may receive a converted code for a body part corresponding to an input window through each of a plurality of input windows from the first user terminal 21 .

According to an embodiment, the processor 230 may set a plurality of objects corresponding to each of a plurality of body parts according to a user's selection. The processor 230 may provide various types of object icons. The processor 230 may set an icon selected by the user among provided object icons as an object corresponding to a body part. The processor 230 may change the location coordinates of an object according to an input to a direction key or a GUI button.

According to an embodiment, the processor 230 may obtain a video image of a user captured by the first user terminal 21 using the communication circuit 210 . The first user terminal 21 may include a camera, and the user may take a video image of himself/herself using the camera. The processor 230 may receive a video image being photographed from the first user terminal 21 in real time.

According to an embodiment, the processor 230 may acquire a plurality of coordinates corresponding to each of a plurality of body parts by analyzing the video image. For example, the processor 230 may acquire a plurality of 2D coordinates corresponding to each of a plurality of body parts by analyzing a video image using a machine learning model. The processor 230 may track the positions of the head and joints by extracting feature points from the video image. For example, the processor 230 may obtain coordinates (340, 80) corresponding to the head in a 640×480 resolution image. The processor 230 may change the reference coordinates of the plurality of 2D coordinates from the top left corner of the video image to the bottom center point of the video image. For example, the processor 230 may change the coordinates (340, 80) whose origin is the top left corner of the video image to the coordinates (20, -400) whose origin is the bottom center point of the video image. The processor 230 may normalize each of the plurality of changed 2D coordinates based on the height of the video image. For example, the processor 230 may obtain the coordinates (0.0417, -0.8333) by normalizing the coordinates (20, -400). The processor 230 may convert each of the plurality of normalized 2D coordinates into a plurality of 3D coordinates to correspond to the virtual stage. For example, the processor 230 may convert 2D coordinates (0.0417, -0.8333) into 3D coordinates (0.4166, 8.3330, 0) to correspond to the format used in the virtual stage. After the Z coordinate is set as a basic value, it can be changed according to the user's input (eg, direction key input) on the virtual stage. The processor 230 may continuously acquire a plurality of coordinates that change according to the user's movement and track the user's movement by repeatedly analyzing the video image in real time.

According to an embodiment, the processor 230 uses the communication circuit 210 to create a virtual stage displaying a plurality of objects at positions corresponding to a plurality of coordinates (or converted 3D coordinates), respectively, in a first user terminal ( 21) can be provided. The processor 230 may obtain a plurality of coordinates corresponding to a virtual stage that is a 3D virtual space. The processor 230 may display an avatar including a plurality of objects represented by a code input to the input window at a position corresponding to each of a plurality of coordinates. A plurality of coordinates may change according to the user's movement, and accordingly, an avatar including a plurality of objects on the virtual stage may move to correspond to the user's movement. Through the first user terminal 21, the user can check an avatar that has a form corresponding to the code input by the user and includes a plurality of objects moving according to the user's movement.

According to an embodiment, the processor 230 may control the virtual stage according to the control of the manager terminal. The processor 230 may activate or stop at least some of the functions provided in the virtual stage according to an input from the manager terminal.

According to an embodiment, the processor 230 may provide a function of modifying and editing each of a plurality of objects on the virtual stage. For example, when one object is selected from among a plurality of objects on the virtual stage in the first user terminal 21, the processor 230 may provide an input window corresponding to the selected object. Selection of an object may be made by receiving a direct input for the object or receiving an input for a menu button corresponding to the object. The processor 230 may receive a modified code for changing the selected object from the first user terminal 21 using the communication circuit 210 . The processor 230 may convert the modified code and obtain coordinates of a body part corresponding to the modified code. The processor 230 may replace the selected object and provide a virtual stage displaying a modified object corresponding to the modified code to the first user terminal 21 .

According to an embodiment, the processor 230 may provide an additional object to the virtual stage according to an input of a user terminal of a spectator, observer, or guardian. For example, the processor 230 may provide a QR code to the first user terminal 21 in response to a request of the first user terminal 21 . The processor 230 may provide an additional object selection screen and an additional object control panel to the fourth user terminal 24 that has captured the QR code. The processor 230 may display the additional object selected on the additional object selection screen of the fourth user terminal 24 on the virtual stage. The processor 230 may adjust the position or size of the additional object on the virtual stage in response to an input to the additional object control panel of the fourth user terminal 24 . When the additional object is adjacent to at least some of the plurality of objects on the virtual stage, the processor 230 may provide a visual or auditory interaction corresponding to the additional object.

3 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 3 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide an editor 300 for code input to a user terminal through a web page, a web application, or a native application. The editor 300 may be configured based on the p5.js library. The editor 300 may include a code input area 310 and a preview area 320 .

The code input area 310 may receive a code corresponding to each part of the user's body. For example, the code input area 310 may receive a code corresponding to the user's head. The code may contain strings to set the color of each of one or more shapes (e.g. fill(“”, shape (e.g. ellipse)) and position/size (e.g. (0, 0, 300, 300)). The preview area 320 may display an image corresponding to the input code, and may visualize one or more figures according to the color, shape, and position/size indicated in the input code.

4 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 4 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide a code conversion window 400 for code conversion to a user terminal through a web page, a web application, or a native application. The code conversion window 400 may include a code input area 410 , a converted code output area 420 and a code copy button 430 .

The code input area 410 may receive a code written by a user. For example, the code input to the code input area 410 may be the same as the code input to the code input area 310 of FIG. 3 . A code input into the code input area 410 may be converted into a format suitable for rendering an object. The converted code output area 420 may display a converted code corresponding to the code input to the code input area 310 . The converted code may be composed of JavaScript. When a user input is received at the code copy button 430, the converted code may be copied.

The code conversion window 410 shown in FIG. 4 may be omitted when the editor is embedded in the website. In this case, the converted code may not be visually displayed to the user.

5 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 5 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide a login screen 500 to a user terminal through a web page, web application, or native application. The login screen 500 may include an avatar name input window 510, a basic color setting window 520, a code input button 530, a stage selection button 540, and a login button 550.

The name input window 510 may receive the avatar's name. In the basic color setting window 520, a basic color for an object for which no color is set can be selected. The code input button 530 may be a button for opening an input window for inputting a converted code corresponding to each of a plurality of body parts. An input window opened according to an input to the code input button 540 will be described in detail with reference to FIG. 6 . The stage selection button 540 may be a button for a user to select a stage to display an avatar among a plurality of stages. The login button 550 may be a button for initiating access to the virtual stage.

6 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 6 , the apparatus for providing an avatar on a virtual stage according to an embodiment may provide a code input window 600 to a user terminal through a web page, web application, or native application. The code input window 600 may be provided in the form of a pop-up window. The code input window 600 includes an object code input area 610, a body part selection menu 620, a first prop code input area 630, a second prop code input area 640, a prop add button 650, and A confirmation button 660 may be included.

The object code input area 610 may be an area for inputting the converted code shown in FIG. 4 . The object code input area 610 may receive codes corresponding to each of a plurality of body parts. The body part selection menu 620 may be a drop-down menu for selecting a code input area for which of a plurality of body parts to provide. An object code input area 610 for a body part selected in the body part selection menu 620 may be provided. The prop code input areas 630 and 640 may be areas for inputting codes for additional objects in addition to objects corresponding to body parts. The prop code input areas 630 and 640 may be activated by a user's selection (eg, a props add button). An additional prop code input area for receiving three or more prop codes may be implemented. The prop code input areas 630 and 640 may receive codes corresponding to various types of additional objects. The prop add button 650 may be a button for generating an additional prop code input area. The confirmation button 660 may be a button for completing input of a code.

7 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 7 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide a virtual stage output screen 700 to a user terminal through a web page, web application, or native application. The virtual stage output screen 700 includes a video image display area 710, an avatar 720 including a plurality of objects, a prop object 740, a body object selection button 750, a prop object selection button 760, and a QR A provision button 770 may be included.

The video image display area 710 may output a video image being captured by the user terminal. The video image display area 710 may display a video image including the user and feature points of the user's body part tracked by the machine learning model. The avatar 720 includes a first to eleventh object 721 corresponding to the head, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left knee, right knee, left ankle, and right ankle, respectively. 722, 723, 724, 725, 726, 727, 728, 729, 731, 732). Some of the body parts included in the avatar 720 may be omitted or replaced with other body parts. Each of the plurality of objects 721 to 732 included in the avatar 720 may correspond to the coordinates of feature points displayed in the video image display area. The color, shape, and size of each of the plurality of objects 721 to 732 may correspond to a code input to the object code input area 610 of FIG. 6 . The position of the avatar 720 on the virtual stage can be controlled by direction keys. The prop object 740 may include a first prop object 741, a second prop object 742, and a third prop object 743, and the prop code input areas 630 and 640 of FIG. You can respond to code.

The body object selection button 750 is a button for selecting one of a plurality of objects 721 to 732, and when an input for the body object selection button 750 is received, an input for modifying and editing the object corresponding to the button. A window may be provided. The prop object selection button 760 is a button for selecting one of the prop objects 760. When an input to the prop object selection button 760 is received, the position and size of the prop object corresponding to the button can be changed using the direction keys and the mouse. It can be adjusted using an input or the like, or a hide function can be performed. The QR code providing button 770 is a button for providing a QR code for audience participation, and when an input to the QR code providing button 770 is received, the QR code can be output on the virtual stage output screen. When the QR code is photographed by the spectator's user terminal, the additional object control screen shown in FIG. 8 may be provided to the spectator's user terminal.

8 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 8 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide an additional object control screen 800 to a user terminal of a spectator through a web page, a web application, or a native application. The additional object control screen 800 may include a control panel 810 and an additional object selection menu 820 .

The control panel 810 may include direction buttons for adjusting the position of the additional object and enlarge/reduce buttons for adjusting the size of the additional object. The additional object selection menu 820 may include a button for selecting one of various additional objects (eg, a car, cloud, tree, etc.). An additional object having a shape selected by the additional object selection menu 820 may be displayed on the virtual stage, and the position and size of the displayed additional object may be adjusted according to an input to the control panel 810 .

9 is a diagram for explaining an exemplary operation of an avatar providing device on a virtual stage according to an embodiment.

Referring to FIG. 9 , an apparatus for providing an avatar on a virtual stage according to an embodiment may provide a virtual stage output screen 900 to a user terminal through a web page, a web application, or a native application. An avatar 910 including a plurality of objects corresponding to each of a plurality of body parts and an additional object 920 may be displayed on the virtual stage.

By controlling the avatar 910 and/or the additional object 920, when the avatar 910 and the additional object 920 are adjacent to or in contact with each other, a visual or auditory communication between the avatar 910 and the additional object 920 occurs. Enemy interactions may be provided. For example, when the vehicle-shaped additional object 920 and the avatar 910 are adjacent to each other, a visual interaction in which the avatar 910 rides the additional object 920 or an auditory interaction in which a collision sound effect is output is provided. It can be. Different interactions may be provided depending on the type of additional object.

10 is a flowchart illustrating a method of providing an avatar on a virtual stage according to an exemplary embodiment.

Hereinafter, it is assumed that the avatar providing apparatus 200 in the virtual stage of FIG. 2 performs the process of FIG. 9 . Also, in the description of FIG. 10 , it may be understood that the operation described as being performed by the avatar providing device on the virtual stage is controlled by the processor 230 .

Referring to FIG. 10 , in operation 1010, the avatar providing apparatus on the virtual stage may receive codes representing a plurality of objects corresponding to a plurality of body parts, respectively, from a user terminal.

In step 1020, the avatar providing device on the virtual stage may convert the received code into a designated format.

In operation 1030, the avatar providing device on the virtual stage may obtain the converted code through an input window corresponding to each of a plurality of body parts.

In step 1040, the avatar providing device on the virtual stage may obtain a video of the user captured by the user terminal.

In operation 1050, the avatar providing device on the virtual stage may acquire a plurality of coordinates corresponding to each of a plurality of body parts by analyzing the video.

In operation 1060, the avatar providing apparatus on the virtual stage may provide a virtual stage displaying a plurality of objects at positions corresponding to each of a plurality of coordinates to the user terminal.

The embodiments of this document and the terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the embodiment. In connection with the description of the drawings, like reference numerals may be used for like elements. singular

An expression may include a plurality of expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B or C" or "at least one of A, B and/or C" refer to all of the items listed together. Possible combinations may be included. Expressions such as "first," "second," "first," or "second," may modify the corresponding components regardless of order or importance, and are used to distinguish one component from another. It is used only and does not limit the corresponding components. When an element is referred to as being "(functionally or communicatively) connected" or "connected" to another element, that element is either directly connected to the other element, or through another element. can be connected through

In this document, "adapted to or configured to" means "adapted to or configured to" depending on the situation, for example, hardware or software "adapted to," "having the ability to," "changed to," ""made to," "capable of," or "designed to" can be used interchangeably. In some contexts, the expression "device configured to" can mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor set up (or configured) to perform A, B, and C" may include a dedicated processor (e.g., embedded processor) to perform those operations, or by executing one or more programs stored in a memory device; It may refer to a general-purpose processor (eg, CPU) capable of performing corresponding operations.

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits, for example. can A “module” may be an integrally constructed component or a minimal unit or part thereof that performs one or more functions. A "module" may be implemented mechanically or electronically, for example, a known or future developed application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or A programmable logic device may be included.

At least a part of an apparatus (eg, modules or functions thereof) or a method (eg, operations) according to an embodiment may be implemented as instructions stored in a computer-readable storage medium in the form of a program module. When the command is executed by a processor, the processor may perform a function corresponding to the command.

Each component (eg, module or program module) according to an embodiment may be composed of one or more entities, and some sub-components among the aforementioned sub-components may be omitted or other sub-components may be used. can include more. Alternatively or additionally, some components (eg, modules or program modules) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. Operations performed by modules, program modules, or other components according to an embodiment are executed sequentially, in parallel, repetitively, or heuristically, or at least some operations are executed in a different order, omitted, or other operations. this may be added.

Claims (5)

An apparatus for providing an avatar on a virtual stage for convergence education of art and technology,
communication circuitry configured to communicate with a user terminal;
Memory; and
a processor electrically connected to the communication circuitry and the memory;
the processor,
Receiving a code representing a plurality of objects corresponding to each of a plurality of body parts from the user terminal using the communication circuit;
converting the received code into a designated format;
Obtaining the converted code through an input window corresponding to each of the plurality of body parts;
Obtaining a video image of a user captured by the user terminal using the communication circuit;
Obtaining a plurality of coordinates corresponding to each of the plurality of body parts by analyzing the video image;
and providing a virtual stage displaying the plurality of objects at a position corresponding to each of the plurality of coordinates to the user terminal using the communication circuit. According to claim 1,
the processor,
Obtaining a plurality of two-dimensional coordinates corresponding to each of the plurality of body parts by analyzing the video image using a machine learning model;
Changing the reference coordinates of the plurality of two-dimensional coordinates to a lower center point of the video image;
Normalizing each of the changed 2-dimensional coordinates based on the height of the video image;
Converting each of the plurality of normalized two-dimensional coordinates into a plurality of three-dimensional coordinates corresponding to the virtual stage;
and providing the virtual stage displaying the plurality of objects at positions corresponding to each of the converted plurality of three-dimensional coordinates to the user terminal. According to claim 1,
the processor,
When one of the plurality of objects on the virtual stage is selected in the user terminal, an input window corresponding to the selected object is provided;
Receiving a modified code for changing the selected object from the user terminal using the communication circuit;
and providing the virtual stage displaying a modified object corresponding to the modified code by replacing the selected object to the user terminal. According to claim 1,
the processor,
Responding to the request of the user terminal to provide a QR code to the user terminal,
Providing an additional object selection screen and an additional object control panel to another user terminal that has photographed the QR code,
displaying an additional object selected on the additional object selection screen on the virtual stage;
Adjusting the position or size of the additional object on the virtual stage in response to an input to the additional object control panel;
When at least some of the plurality of objects and the additional object are adjacent to each other on the virtual stage, a visual or auditory interaction corresponding to the additional object is provided. A method for providing an avatar on a virtual stage for convergence education of art and technology performed by one or more computing devices,
Receiving a code representing a plurality of objects corresponding to each of a plurality of body parts from a user terminal;
converting the received code into a designated format;
obtaining the converted code through an input window corresponding to each of the plurality of body parts;
obtaining a video of the user captured by the user terminal;
obtaining a plurality of coordinates corresponding to each of the plurality of body parts by analyzing the video; and
And providing a virtual stage displaying the plurality of objects at positions corresponding to each of the plurality of coordinates to the user terminal.

Images