KR102216312B1 - Method and apparus providng virtual simulation with vr - Google Patents

Abstract

The present invention relates to a method for providing a virtual scenario using virtual reality (VR) to provide a VR simulation related to various industrial sites and a device thereof. According to the present invention, a computer program stored in a computer-readable recording medium includes commands for allowing a processor of a first user terminal to execute the following steps: recognizing whether a camera of the first user terminal is activated; receiving, from an external server, at least one thumbnail image related to at least one variable image when it is recognized that the camera can be activated; when the at least one thumbnail image is received, transmitting the at least one thumbnail image to a second user terminal to allow the at least one thumbnail image to be displayed on the second user terminal; acquiring a first image including a user′s movement through the camera when the thumbnail image is transmitted to the second user terminal; transmitting the first image to the external server; receiving first motion recognition data related to the first image from the external server after transmitting the first image; recognizing a first thumbnail image selected by the user among the at least one thumbnail image based on the first motion recognition data; transmitting a second signal requesting a first variable image related to the first thumbnail image to the external server when the first thumbnail image is recognized; and allowing the first variable image to be displayed on the second user terminal as the second signal is transmitted.

Description

Virtual scenario provision method and device using VR {METHOD AND APPARUS PROVIDNG VIRTUAL SIMULATION WITH VR}

The present disclosure relates to a method and apparatus for providing a virtual scenario using VR.

Workers working at construction sites, plant industry sites or other industrial sites are always at risk of safety accidents. In order to prevent the occurrence of safety accidents, companies periodically simulate how to deal with machines, or perform simulations assuming when an accident occurs.

However, this method has many difficulties such as limited space, facility management problems, risk of safety accidents for beginners, and passive response to education.

In order to solve this difficulty, virtual scenarios for various sites may be performed using virtual reality such as VR (Virtual Reality).

VR (Virtual Reality) refers to a device or technology capable of providing a realistic experience by using human vision and hearing. These VRs may be classified into Virtual Environment (VE), Augmented Reality (AR), Augmented Virtuality (AV), Substitutional Reality (SR), and Mixed Reality (MR), depending on the implementation technology.

In the case of providing safety education or education on how to use various machines using VR, accidents that may occur due to workers' immaturity can be prevented. It can also reduce the time and cost required for workers to visit the site.

However, devices capable of performing such VR, for example, a head mounted display (HMD), are formed at a high price, so it may be practically difficult for all workers working in the field to perform such training.

Therefore, there is a demand for a method of providing a scenario related to an industrial site by implementing VR through a user terminal.

Republic of Korea Patent Publication 10-2019-0087912

The present disclosure is conceived in response to the above-described background technology, and is intended to provide a method of providing a virtual simulation using VR.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present disclosure for solving the above-described problem, a computer program stored in a computer-readable storage medium is disclosed.

A computer program stored in a computer-readable storage medium, the computer program comprising instructions for causing a game server to perform the following steps, the steps comprising: recognizing whether or not a camera of the first user terminal is activated; Receiving at least one thumbnail image related to at least one variable image from an external server when it is recognized that activation of the camera is possible; When the at least one thumbnail image is received, transmitting the at least one thumbnail image to a second user terminal, thereby causing the at least one thumbnail image to be displayed on the second user terminal; When the thumbnail image is transmitted to the second user terminal, acquiring a first image including the user's movement through the camera; Transmitting the first image to an external server; Receiving first motion recognition data related to the first image from the external server after transmitting the first image; Recognizing a first thumbnail image selected by a user from among the at least one thumbnail image based on the first motion recognition data; When the first thumbnail image is recognized, transmitting a second signal for requesting a first variable image related to the first thumbnail image to the external server; And causing the first variable image to be displayed on the second user terminal by transmitting the second signal. It may include.

In addition, the second user terminal may be a terminal paired with the first user terminal.

In addition, the second user terminal may be installed on a device that is worn on the user's head so that the display unit is positioned in front of the user's eyes.

In addition, when the first thumbnail image is recognized, transmitting a second signal for requesting a first variable image related to the first thumbnail image to the external server may include, when the first thumbnail image is recognized, at least Causing a first virtual screen including one virtual button to be displayed on the second user terminal; When the first virtual screen including the at least one virtual button is displayed on the second user terminal, obtaining a second image including the user's movement through the camera; Transmitting the second image to the external server; Receiving second motion recognition data related to the second image from the external server after transmitting the second image; And recognizing a first virtual button selected by the user from among the at least one virtual button based on the second motion recognition data. Including, In response to transmitting the second signal, causing the first variable image to be displayed on the second user terminal, the instruction information related to the first virtual button and a quiz related to the first virtual button Causing first information including at least one of the information to be displayed together with the first variable image; It may include.

In addition, as the second signal is transmitted, causing the first variable image to be displayed on the second user terminal may include, when the received first information is the instruction information, the user's movement through the camera. Acquiring a third image including this; Transmitting the third image to the external server; Receiving third motion recognition data related to the third image from the external server after transmitting the third image; Generating performance rate information for the indication information based on the third motion recognition data; And calculating a result of performing the first variable image based on the performance rate information. It may include.

Further, the step of recognizing a performance rate of performing at least one instruction included in the instruction information based on the third motion recognition data may include recognizing at least one instruction included in the instruction information; Determining at least one motion data corresponding to the at least one indication from among the third motion recognition data; Generating coincidence rate information by comparing the at least one motion data with predetermined expert motion data corresponding to the at least one indication; And calculating result information of performing the first variable image based on the matching rate information. It may include.

In addition, calculating result information of performing the first variable image based on the matching rate information may include: when the matching rate information is equal to or greater than a preset value, generating strength information based on the at least one motion data; It may include.

In addition, calculating result information of performing the first variable image based on the coincidence rate information may include: when the coincidence rate information is less than a preset value, generating weakness information based on the at least one motion data; It may include.

In addition, as the second signal is transmitted, causing the first variable image to be displayed on the second user terminal, and when the received first information is the quiz information, the user's movement through the camera Acquiring a fourth image including this; Transmitting the fourth image to the external server; Receiving fourth motion recognition data related to the fourth image from the external server after transmitting the fourth image; Generating correct answer rate information for the quiz information based on the fourth motion recognition data; And calculating a result of performing the first variable image including the quiz information based on the correct answer rate information. It may include.

In addition, causing a second variable image including a feedback panel image for the first variable image to be displayed on a second user terminal; And obtaining a feedback motion image corresponding to the feedback panel image. Transmitting the feedback motion image to the external server; Causing the external server to generate feedback information for the feedback motion image; It may include.

In addition, as a first user terminal, the communication unit; A camera that photographs the user's movement; A processor that recognizes whether or not the camera is activated; Including, wherein when it is recognized that the activation of the camera is possible, the processor receives at least one thumbnail image related to at least one variable image from an external server through the communication unit, and receives the at least one thumbnail image In one case, by transmitting the at least one thumbnail image to the second user terminal through the communication unit, the at least one thumbnail image is caused to be displayed on the second user terminal, and the thumbnail image is displayed on the second user terminal. In the case of transmission, the communication unit is controlled to obtain a first image including the user's movement through the camera, transmit the first image to an external server, and transmit the first image through the communication unit. Receives first motion recognition data related to the first image from an external server, recognizes a first thumbnail image selected by a user from among the at least one thumbnail image, based on the first motion recognition data, and the first thumbnail When an image is recognized, the communication unit is controlled to transmit a second signal for requesting a first variable image related to the first thumbnail image to the external server, and according to the transmission of the second signal, the first variable image This can cause it to be displayed on the second user terminal.

In addition, when the first thumbnail image is recognized, the processor causes a first virtual screen including at least one virtual button to be displayed on the second user terminal, and a second including the at least one virtual button 1 When a virtual screen is displayed on the second user terminal, controlling the communication unit to obtain a second image including the user's movement through the camera and to transmit the second image to the external server, After transmitting the second image, second motion recognition data related to the second image is received from the external server through the communication unit, and based on the second motion recognition data, the user of the at least one virtual button Recognizing the selected first virtual button and transmitting the second signal causes the first variable image to be displayed on the second user terminal, and instruction information related to the first virtual button and the first virtual button The first information including at least one of quiz information related to may be caused to be displayed together with the first variable image.

In addition, when the received first information is the instruction information, controlling the communication unit to obtain a third image including a user's movement through the camera, and to transmit the third image to the external server, and the After transmitting the third image, receive third motion recognition data related to the third image from the external server through the communication unit, and generate performance rate information for the instruction information based on the third motion recognition data. , A result of performing the first variable image may be calculated based on the performance rate information.

In addition, the processor recognizes at least one instruction included in the instruction information, determines at least one motion data corresponding to the at least one instruction from among the third motion recognition data, and the at least one motion data Concordance rate information may be generated by comparing the expert motion data corresponding to the at least one indication, and result information of performing the first variable image may be calculated based on the matching rate information.

Technical solutions obtainable in the present disclosure are not limited to the above-mentioned solutions, and other solutions that are not mentioned are clearly to those of ordinary skill in the technical field to which the present disclosure belongs from the following description. It will be understandable.

The present disclosure makes it possible to provide VR simulations related to various industrial sites using a user terminal.

The effects obtainable in the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description. .

Various aspects are now described with reference to the drawings, wherein like reference numbers are used collectively to refer to like elements. In the examples that follow, for illustrative purposes, a number of specific details are presented to provide a thorough understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects.
1 illustrates an exemplary virtual scenario providing system for performing a method of providing a virtual simulation according to some embodiments of the present disclosure.
FIG. 2 is a flowchart illustrating an example of a method of providing at least one variable image to a second user terminal by a processor according to some embodiments of the present disclosure.
3 is a flowchart illustrating an example of a method of causing a processor to display at least one piece of information together with a first variable image on a second user terminal according to some embodiments of the present disclosure.
4 is a flowchart illustrating an example of a method for calculating, by a processor, a result of performing a first variable image based on performance rate information, according to some embodiments of the present disclosure.
5 is a flowchart illustrating an example of a method of generating strength information or weakness information based on at least one motion data by a processor according to some embodiments of the present disclosure.
6 is a flowchart illustrating an example of a method of providing a first variable image including quiz information to a second user terminal by a processor according to some embodiments of the present disclosure.
7 is a diagram for describing an example of a third variable screen that can be displayed in a second user terminal according to some embodiments of the present disclosure.
FIG. 8 is a diagram for describing another example of a third variable screen that can be displayed in a second user terminal according to some embodiments of the present disclosure.
9 is a diagram for describing another example of a third variable screen that can be displayed on a second user terminal according to some embodiments of the present disclosure.
FIG. 10 is a diagram for describing another example of a third variable screen that can be displayed on a second user terminal according to some embodiments of the present disclosure.
11 shows a general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for illustrative purposes, a number of specific details are disclosed to aid in an overall understanding of one or more aspects. However, it will also be appreciated by those of ordinary skill in the art that this aspect(s) may be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in the principles of the various aspects may be used, and the descriptions described are intended to include all such aspects and their equivalents. Specifically, as used herein, "an embodiment", "example", "aspect", "example" and the like are not construed as having any aspect or design being better or advantageous than other aspects or designs. May not.

Hereinafter, the same or similar components are assigned the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted. In addition, in describing the embodiments disclosed in the present specification, when it is determined that detailed descriptions of related known technologies may obscure the subject matter of the embodiments disclosed in the present specification, detailed descriptions thereof will be omitted. In addition, the accompanying drawings are for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the present specification is not limited by the accompanying drawings.

Although the first, second, and the like are used to describe various devices or components, it goes without saying that these devices or components are not limited by these terms. These terms are only used to distinguish one device or component from another device or component. Therefore, it goes without saying that the first element or element mentioned below may be a second element or element within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless specified otherwise or is not clear from the context, "X employs A or B" is intended to mean one of the natural inclusive substitutions. That is, X uses A; X uses B; Or, when X uses both A and B, “X uses A or B” can be applied to either of these cases. In addition, the term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the listed related items.

In addition, the terms "comprising" and/or "comprising" mean that the corresponding feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. It should be understood as not. In addition, unless otherwise specified or when the context is not clear as indicating a singular form, the singular in the specification and claims should be interpreted as meaning "one or more" in general.

In addition, the terms “information” and “data” as used herein may often be used interchangeably with each other.

The suffixes "module" and "unit" for constituent elements used in the following description are given or used interchangeably in consideration of only the ease of preparation of the specification, and do not have a distinct meaning or role by themselves.

Objects and effects of the present disclosure, and technical configurations for achieving them will become apparent with reference to embodiments described in detail later with reference to the accompanying drawings. In describing the present disclosure, when it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the subject matter of the present disclosure, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present disclosure and may vary according to the intention or custom of users or operators.

The scope of rights to the method in the claims of the present disclosure is generated by the functions and features described in each step, and unless a precedence relationship of the order is specified in each step constituting the method, the claim It is not affected by the order of description of each step in the range. For example, in the claims described in a method including steps A and B, even if step A is described before step B, the scope of the rights is not limited to that step A must precede step B.

However, the present disclosure is not limited to the embodiments disclosed below, and may be implemented in various different forms. The present embodiments are provided only to make the present disclosure complete, and to completely inform the scope of the disclosure to those of ordinary skill in the art to which the present disclosure belongs, and the present disclosure is only defined by the scope of the claims. . Therefore, the definition should be made based on the contents throughout this specification.

In the present disclosure, the virtual scenario system may provide a variable image related to an industrial site through at least one user terminal. Also, the virtual scenario may be a scenario related to an industrial site. For example, it may be a scenario such as a construction site, a plant site, or a woodworking site. Meanwhile, the variable image may be an image whose screen can be changed according to a user's manipulation. For example, a user may be provided with a variable image using a device that makes a smartphone or the like assembled and positioned in front of the user's eyes. In this case, the movement accompanying the user by performing the instructions displayed through the variable image may be photographed by a first user terminal that may be provided in front or in all directions of the user. Then, the first user terminal may obtain motion recognition data by transmitting the captured image to an external server capable of motion recognition. Accordingly, the first user terminal may cause the variable image provided through the smartphone to change according to the user's movement. Hereinafter, a method of providing a variable image by the virtual scenario system according to the present disclosure will be described with reference to FIGS. 1 to 11.

1 illustrates an exemplary virtual scenario providing system for performing a method of providing a virtual simulation according to some embodiments of the present disclosure.

Referring to FIG. 1, a system for providing a virtual scenario may include a first user terminal 100, a second user terminal 200, an external server 300, and a network 400. However, the above-described components are not essential in implementing the virtual scenario providing system, and thus the virtual scenario providing system may have more or fewer components than the above-listed components.

The first user terminal 100, the second user terminal 200, and the external server 300 may mutually transmit and receive data for the virtual scenario providing system according to the present disclosure through the network 400.

The first user terminal 100 may comprise any type of computer system or computer device, such as, for example, a microprocessor, mainframe computer, digital processor, portable device and device controller, and the like.

In the present disclosure, the first user terminal 100 may cause at least one variable image to be displayed on the second user terminal 200 through the external server 300. Here, the at least one variable image may be an image of an industrial site related to a user. For example, the first user terminal 100 may transmit an image photographing a user of the first user terminal 100 to the external server 300. In this case, the external server 300 may recognize the user's motion through the image captured by the user. In addition, a variable image that changes according to a user's motion may be caused to be displayed on the second user terminal 200 or may be transmitted to the first user terminal 100. However, it is not limited thereto.

At least one variable image may represent various situations that may occur in an industrial site. In addition, at least one variable image may be used to learn actions for various situations that may occur in the industrial field. However, the present invention is not limited thereto, and at least one variable image may include various images.

Meanwhile, the first user terminal 100 may include a processor 110, a storage unit 120, a communication unit 130, and a camera 140. However, the above-described components are not essential for implementing the first user terminal 100, and the first user terminal 100 may have more or fewer components than the components listed above. Here, each component may be composed of a separate chip, module or device, or may be included in a single device.

The processor 110 may typically process the overall operation of the first user terminal 100. The processor 110 may provide or process appropriate information or functions to a user by processing signals, data, information, etc. that are input or output through the above-described elements or by driving an application program stored in the storage unit 120. .

Meanwhile, the storage unit 120 may include a memory and/or a permanent storage medium. Memory is a flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory), RAM (Random Access). Memory, RAM), SRAM (Static Random Access Memory), ROM (Read-Only Memory, ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, optical disk It may include at least one type of storage medium.

The storage unit 120 may store information in any form generated or determined by the processor 110 and information in any form received by the communication unit 130.

The communication unit 130 may be configured between the first user terminal 100 and the communication system, between the first user terminal 100 and the second user terminal 200, between the first user terminal 100 and the external server 300, or 1 It may include one or more modules that enable communication between the user terminal 100 and the network 400.

The communication unit 130 may include at least one of a mobile communication module, a wired Internet module, and a wireless Internet module.

The second user terminal 200 includes a personal computer (PC), a notebook (notebook), a mobile terminal, a smart phone, a tablet PC, and a virtual computer (VR) owned by the user. Reality) devices, head mounted displays, smart glasses, wearable devices, etc., and may include all types of terminals that can access wired/wireless networks. . However, it is not limited thereto.

In the present disclosure, the second user terminal 200 may be paired with the first user terminal 100 to display at least one variable image received from the first user terminal 100 or the external server 300. Here, the pairing may be performed by using short-range communication such as infrared (Infrared Data Association) or Bluetooth (Bluetooth). However, it is not limited thereto.

In addition, the second user terminal 200 in the present disclosure may be installed on a device (eg, a head mount device (HMD)) that is worn on the user's head so that the display unit is positioned in front of the user's eyes. However, it is not limited thereto.

Meanwhile, the external server 300 may include any type of computer system or computer device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a portable device and a device controller.

The external server 300 in the present disclosure may be a server that provides a platform to display at least one variable image related to an industrial site through the first user terminal 100 and the second user terminal 200. However, it is not limited thereto.

Meanwhile, the external server 300 may include a processor (not shown), a storage unit (not shown), and a communication unit (not shown). However, since it is not essential to implement the above-described configuration external server 300, the external server 300 may have more or less components than those listed above. Here, each component may be composed of a separate chip, module or device, or may be included in a single device.

In the present disclosure, the network 400 may be configured regardless of its communication mode such as wired and wireless, and may be configured with various communication networks such as a personal area network (PAN) and a wide area network (WAN). have. In addition, the network may be a known World Wide Web (WWW), and may use a wireless transmission technology used for short-range communication such as infrared (IrDA) or Bluetooth.

The techniques described herein may be used not only in the networks mentioned above, but also in other networks.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 of the first user terminal 100 provides at least one variable image related to the industrial site to the second user terminal 200 through the external server 300 can do. Hereinafter, a method of providing at least one variable image to the second user terminal 200 by the first user terminal 100 will be described with reference to FIG. 2.

FIG. 2 is a flowchart illustrating an example of a method of providing at least one variable image to a second user terminal by a processor according to some embodiments of the present disclosure.

Referring to FIG. 2, the processor 110 of the first user terminal 100 may recognize whether the camera 140 is activated (S110).

Specifically, the processor 110 may recognize whether the camera 140 is turned on or in a state in which a photograph is possible. In addition, the processor 110 may activate the camera 140 when it is recognized that the camera is turned off. However, it is not limited thereto.

On the other hand, when the processor 110 of the first user terminal 100 recognizes that the camera 140 can be activated, the at least one thumbnail image related to the at least one variable image is transmitted from the external server 300. Can receive (S120).

Here, the at least one thumbnail image may be a representative image of at least one variable image including a virtual scenario. Here, the virtual scenario may be a scenario for education related to an industrial site.

For example, the processor 110 may transfer at least one thumbnail image of at least one of a thumbnail image of a scenario related to a construction site, a thumbnail image of a scenario related to a carpentry site, or a thumbnail image of a scenario related to a plant site to an external server ( 300) can be requested. However, it is not limited thereto.

Meanwhile, when the processor 110 of the first user terminal 100 receives at least one thumbnail image, by controlling the communication unit 130 to transmit at least one thumbnail image to the second user terminal 200, at least It may cause one thumbnail image to be displayed on the second user terminal 200 (S130).

For example, the processor 110 of the first user terminal 100 is at least one of at least one of a thumbnail image of a scenario related to a construction site, a thumbnail image of a scenario related to a woodworking site, or a thumbnail image of a scenario related to a plant site. By transmitting one thumbnail image to the second user terminal 200, at least one thumbnail image may be caused to be displayed on the second user terminal 200.

However, the present invention is not limited thereto, and the processor 110 allows the external server 300 to transmit at least one thumbnail image to the second user terminal 200, so that at least one thumbnail image is converted to the second user terminal 200. May cause it to be displayed on.

Meanwhile, when the processor 110 of the first user terminal 100 transmits the thumbnail image to the second user terminal 200, the first image including the user's movement may be obtained through the camera 140. (S140).

Specifically, the processor 110 may control the camera 140 to photograph the user's movement. In this case, the processor 110 may acquire the first image including the user's movement. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit the first image to the external server 300 (S150).

In this case, the external server 300 may generate motion recognition data based on the first image.

Specifically, the external server 300 may image the first image to recognize a joint or a part of a body of the first user. In addition, the external server 300 may recognize movement by generating a skeleton related to the user's body through a joint or a part of the body of the first user recognized through at least one image. In addition, the external server 300 may generate motion recognition data based on the recognized movement of the first user. However, it is not limited thereto. Hereinafter, information related to motion recognition is a known technique and a detailed description thereof will be omitted.

Meanwhile, after transmitting the first image, the communication unit 130 of the first user terminal 100 may receive first motion recognition data related to the first image from the external server 300 (S160).

That is, the first motion recognition data may be a motion in which a user selects at least one thumbnail image. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may recognize a first thumbnail image selected by the user from among at least one thumbnail image based on the first motion recognition data (S170).

For example, the processor 110 may recognize that the user has selected the first thumbnail image related to the construction site based on the first motion recognition data. However, it is not limited thereto.

Meanwhile, when the first thumbnail image is recognized, the processor 110 of the first user terminal 100 transmits a second signal for requesting a first variable image related to the first thumbnail image through the communication unit 130 to an external server ( 300) can be transmitted (S180).

As an example, the second signal may be a signal requesting to transmit the first variable image to the first user terminal 100. As another example, the second signal may be a signal requesting to transmit the first variable image to the second user terminal 200. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may cause the first variable image to be displayed on the second user terminal 200 by transmitting the second signal (S190).

For example, when receiving the first variable image from the external server 300, the processor 110 controls the communication unit 130 to transmit the first variable image to the second user terminal 200, thereby controlling the first variable image. This may cause it to be displayed on the second user terminal 200.

In another example, the processor 110 transmits a second signal that causes the external server 300 to transmit the first variable image to the second user terminal 200, so that the first variable image is transmitted to the second user terminal 200. ) To be displayed. However, it is not limited thereto.

According to the above-described configuration, the processor 110 of the first user terminal 100 may cause at least one thumbnail image related to an industrial site to be displayed on the second user terminal 200. Further, the processor 110 may recognize an input for the user to select a first thumbnail image from among at least one thumbnail image. In this case, the processor 110 may cause the first variable image related to the first thumbnail image to be displayed on the second user terminal 200. Accordingly, the user may be provided with a first variable image related to an industrial site and including a virtual scenario through the second user terminal 200.

Meanwhile, according to some embodiments of the present disclosure, the first variable image may be an image for training a user or an image for testing a user. In this case, the user may determine whether to perform training or testing through the first variable image. Hereinafter, the processor 110 of the first user terminal 100 may display at least one of a first variable image including instruction information for training or a first variable image including quiz information for testing through FIGS. 3 to 5. Explain how to provide.

3 is a flowchart illustrating an example of a method of causing a processor to display at least one piece of information together with a first variable image on a second user terminal according to some embodiments of the present disclosure.

Referring to FIG. 3, when the first thumbnail image is recognized, the processor 110 of the first user terminal 100 displays a first virtual screen including at least one virtual button on the second user terminal 200. It can be caused to be (S210).

Here, the at least one virtual button may be a button for selecting at least one scenario related to the first variable screen.

For example, the at least one virtual button may be a button for selecting training through the first variable image. For another example, the at least one virtual button may be a button for selecting a test through the first variable image. However, it is not limited thereto.

On the other hand, when the first virtual screen including at least one virtual button is displayed on the second user terminal 200, the processor 110 of the first user terminal 100 detects the movement of the user through the camera 140. The included second image may be acquired (S220).

Specifically, the processor 110 may control the camera 140 to photograph the user's movement. In this case, the processor 110 may acquire a second image including the user's movement. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit the second image to the external server 300 (S230).

In this case, the external server 300 may generate second motion recognition data based on the second image.

Meanwhile, after transmitting the second image, the communication unit 130 of the first user terminal 100 may receive second motion recognition data related to the second image from the external server 300 (S240).

Here, the second motion recognition data may be data on motion in which the user selects at least one button. In this case, the processor 110 may recognize at least one button selected by the user based on the second motion recognition data. However, it is not limited thereto.

Meanwhile, when the processor 110 of the first user terminal 100 recognizes the first virtual button selected by the user among at least one virtual button based on the second motion recognition data (S250, Yes), the second signal By controlling the communication unit 130 to transmit the data to the external server 300, first information including at least one of instruction information related to the first virtual button and quiz information related to the first virtual button together with the first variable image It can be caused to be displayed (S260).

For example, when the first virtual button is a button for performing training through the first variable image, the processor 110 may cause the first variable image to contain and display instruction information. Here, the indication information may be information including content indicating an operation to be performed in the first variable screen. That is, the user can train a scenario related to the first variable image by following an instruction displayed in the first variable screen. However, it is not limited thereto.

For another example, when the first virtual button is a button for performing a test through the first variable image, the processor 110 may cause the first variable image to include quiz information and display it. Here, the quiz information may be information including a question related to an operation to be performed in the first variable screen. Accordingly, the user may perform a test related to the first variable image by performing an operation corresponding to the quiz displayed in the first variable screen. However, it is not limited thereto.

Meanwhile, when the processor 110 of the first user terminal 100 does not recognize the first virtual button selected by the user among at least one virtual button based on the second motion recognition data (S250, Yes), the second The signal may not be transmitted to the external server 300. In this case, the processor 110 may cause the first virtual screen including at least one virtual button to be displayed again on the second user terminal 200. However, it is not limited thereto.

According to the above-described configuration, the processor 110 of the first user terminal 100 may provide the user with a first variable image capable of performing training or testing.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 of the first user terminal 100 may display the result of the training or test performed by the user through the first variable image on the second user terminal 200. Can cause. Hereinafter, a method of causing the processor 110 to display the results of training or testing on the second user terminal 200 will be described with reference to FIGS. 4 to 6.

4 is a flowchart illustrating an example of a method for calculating, by a processor, a result of performing a first variable image based on performance rate information, according to some embodiments of the present disclosure.

Referring to FIG. 4, the processor 110 of the first user terminal 100 may acquire a third image including the user's movement through the camera 140 when the received first information is instruction information. (S310).

Here, the third image may be an image obtained by photographing an operation performed by the user based on the indication information being displayed on the first variable image. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit the third image to the external server 300 (S320).

In this case, the external server 300 may generate third motion recognition data based on the third image. That is, the third motion recognition data may be a motion in which at least one instruction is performed by the user according to instruction information being displayed on the first variable screen. However, it is not limited thereto.

Meanwhile, after transmitting the third image, the communication unit 130 of the first user terminal 100 may receive third motion recognition data related to the third image from the external server 300 (S330).

In this case, the processor 110 may recognize the user's movement based on the third motion recognition data.

For example, the first variable image displayed by the processor 110 through the second user terminal 200 may include instruction information indicating "operate the machine shown in front to inspect the wafer". have. In this case, the processor 110 may recognize whether the user has performed the instruction content displayed on the first variable screen based on the third motion recognition data. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may generate performance rate information for the instruction information based on the third motion recognition data (S340).

Here, the performance rate information may be information in which the user has performed at least one operation based on indication information displayed on the first variable screen. For example, the instruction information may include a total of five instructions. Further, the processor 110 may recognize that the user has performed 4 of the total 5 instructions based on the third motion recognition data. In this case, the processor 110 may generate performance rate information indicating that the user has performed an instruction of 80% from the instruction information displayed on the first variable screen. However, it is not limited thereto.

Meanwhile, a user of the first user terminal 100 may calculate a result of performing the first variable image based on the performance rate information (S350).

Specifically, the processor 110 may calculate a result including whether the user has successfully performed a scenario related to the first variable image through the first variable image based on the performance rate information. In this case, the processor 110 may cause the result to be displayed on the second user terminal 200.

For example, if the processor 110 recognizes that the user has performed 80% of the instructions included in the instruction information based on the instruction information and the third motion recognition data, the result of “80 points” is the second user terminal ( 200) can be caused to be displayed. However, it is not limited thereto.

According to the above configuration, the processor 110 of the first user terminal 100 provides a training scenario related to an industrial site by allowing instruction information to be included in the first variable image displayed on the second user terminal 200. can do. In addition, the processor 110 may provide a training result to the user by recognizing whether the user has successfully performed the training scenario provided through the first variable screen.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 of the first user terminal 100 determines whether the operation performed by the user at least one instruction is a strength or a weakness, based on the third motion recognition data. I can recognize it. Hereinafter, a method of generating strength information or weakness information by the processor 110 will be described with reference to FIG. 5.

5 is a flowchart illustrating an example of a method of generating strength information or weakness information based on at least one motion data by a processor according to some embodiments of the present disclosure.

Referring to FIG. 5, the processor 110 of the first user terminal 100 may recognize at least one instruction included in the instruction information (S341).

For example, the processor 110 may recognize at least one instruction "Move the operation unit of the machine inspecting the wafer to the right" included in the instruction information. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may determine at least one motion data corresponding to at least one instruction from among the third motion recognition data (S342).

Specifically, the processor 110 may recognize at least one instruction and determine at least one motion data that has performed the recognized at least one instruction.

For example, among the third motion recognition data, the processor 110 may recognize motion data in which a user moves a manipulation unit of a machine that inspects a wafer included in the first variable image to the right. In this case, the processor 110 may determine that the motion data is data related to an instruction indicating "Move the operation unit of the machine inspecting the wafer to the right". However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may generate coincidence rate information by comparing at least one motion data with predetermined expert motion data corresponding to the at least one instruction (S343).

Here, the predetermined expert motion data may be motion recognition data for a motion that the expert manipulates at least one machine. In addition, predetermined expert motion data may be received together with the first variable image. However, it is not limited thereto.

Meanwhile, the match rate information may be information obtained by comparing at least one motion data related to a user with predetermined expert motion data.

Specifically, the processor 110 may recognize at least one motion data for which the user has performed at least one instruction in the first variable image. Further, the processor 110 may compare the at least one motion data and the expert motion data to compare a match rate of motion. Further, the processor 110 may generate matching rate information based on the recognized matching rate.

For example, the processor 110 may recognize data that the expert has moved the manipulation unit 15 degrees to the right in order to manipulate the machine inspecting the wafer through the expert motion data. In addition, the processor 110 may recognize data indicating that the user has moved the manipulation unit 10 degrees to the right in order to manipulate the machine inspecting the wafer through at least one motion data related to the user. In this case, the processor 110 may generate coincidence rate information indicating that at least one motion data related to the user has a match rate of 66% with predetermined expert motion data corresponding to the at least one instruction. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may calculate result information of performing the first variable image based on the matching rate information.

Specifically, when the match rate information is greater than or equal to a preset value (S344, Yes), the processor 110 may generate result information as strength information based on at least one motion data (S345).

Here, the preset value may be pre-stored in the storage unit 120 of the first user terminal 100. However, it is not limited thereto.

Meanwhile, when the strength information is generated, the processor 110 may cause the strength information to be displayed on the second user terminal 200. However, it is not limited thereto.

Meanwhile, when the match rate information is less than a preset value (S344, No), the processor 110 of the first user terminal 100 may generate result information as weakness information based on at least one motion data (S346). .

In this case, the processor 110 may cause the weakness information to be displayed on the second user terminal 200. However, it is not limited thereto.

According to the above-described configuration, the processor 110 of the first user terminal 100 compares at least one motion data included in the third motion recognition data with predetermined expert motion data, thereby giving strength to at least one motion data. It is possible to generate information such as whether or not it represents a weakness or weakness. In addition, the processor 110 may provide information to the user by causing the generated strength information or weakness information to be displayed on the second user terminal 200.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 may provide a test-related scenario through the first variable image. Hereinafter, a method in which the processor 110 provides a test-related scenario will be described with reference to FIG. 6.

6 is a flowchart illustrating an example of a method of providing a first variable image including quiz information to a second user terminal by a processor according to some embodiments of the present disclosure.

Referring to FIG. 6, when the received first information is quiz information, the processor 110 of the first user terminal 100 may obtain a fourth image including the user's movement through the camera 140. (S410).

Here, the fourth image may be an image obtained by photographing an operation performed by the user based on quiz information displayed on the first variable image. However, it is not limited thereto.

Meanwhile, the communication unit 130 of the first user terminal 100 may transmit the fourth image to the external server 300 (S420).

In this case, the external server 300 may generate fourth motion recognition data based on the fourth image. That is, the fourth motion recognition data may be a motion in which a user performs at least one operation according to quiz information displayed on the first variable screen. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may receive the fourth motion recognition data related to the fourth image from the external server 300 through the communication unit 130 after transmitting the fourth image ( S430).

In this case, the processor 110 may recognize the user's movement based on the fourth motion recognition data.

For example, the first variable image displayed by the processor 110 through the second user terminal 200 may include quiz information “Operate a machine to be operated to inspect a wafer”. In this case, the processor 110 may recognize whether the user has performed an operation related to the quiz displayed on the first variable screen based on the fourth motion recognition data. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may generate correct answer rate information for the quiz information based on the fourth motion recognition data (S440).

Here, the correct answer rate information may be information generated based on whether the user has performed an operation corresponding to the correct answer to the quiz information displayed on the first variable screen.

For example, the first variable image including the quiz information that caused the processor 110 to be displayed on the second user terminal 200 may include a quiz that says "Operate a machine that must be operated to inspect a wafer". I can. Further, the processor 110 may recognize whether the user has operated the correct machine for the quiz based on the fourth motion recognition data. In this case, the processor 110 may recognize whether a correct answer or an incorrect answer has been made to at least one operation performed by the user. Further, the processor 110 may generate correct answer rate information by determining whether a correct answer or an incorrect answer for at least one operation is performed. For example, when the processor 110 recognizes that the user has corrected all five of five quizzes included in the quiz information, the processor 110 may generate correct answer rate information indicating that the correct answer rate is 100%. However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may calculate a result of performing the first variable image including quiz information based on the correct answer rate information (S450).

Specifically, the processor 110 may calculate a result including whether or not the user has successfully performed a scenario related to the first variable image through the first variable image, based on the correct answer rate information. In this case, the processor 110 may cause the result to be displayed on the second user terminal 200.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 of the first user terminal 100 includes a first variable image including instruction information being displayed by the second user terminal 100 or a first variable image including quiz information. 1 It is possible to recognize the end of the scenario related to the variable video. In this case, the processor 110 may cause the user to display feedback information requesting a feedback on the first variable image to be displayed on the second user terminal 200.

Specifically, the processor 110 may cause the second variable image including the feedback panel image for the first variable image to be displayed on the second user terminal 200. Here, the feedback panel image may be an image including at least one piece of information requesting feedback.

For example, the feedback panel image may be an image generated through text such as "good", "normal", or "bad". However, it is not limited thereto.

Meanwhile, the processor 110 of the first user terminal 100 may obtain a feedback motion image corresponding to the feedback panel image through the camera 140. In this case, the processor 110 may cause the external server 300 to generate feedback information for the feedback motion image by transmitting the feedback motion image to the external server 300. In this case, the external server 300 may correct at least one scenario based on the feedback information.

As described above, the processor 110 of the first user terminal 100 may provide a user with a first variable image including a test related to an industrial site.

Meanwhile, in a variable screen that causes the processor 110 of the first user terminal 100 to be displayed on the second user terminal 200 according to some embodiments of the present disclosure, various machines for performing scenarios related to the industrial site Fields, instructional information, or quiz information may be included. Hereinafter, an example of a variable screen according to the present disclosure will be described with reference to FIG. 7.

7 is a diagram for describing an example of a third variable screen that can be displayed in a second user terminal according to some embodiments of the present disclosure.

Referring to FIG. 7, the third variable screen 500 may include scenario information 510 and at least one first object 520. However, it is not limited thereto.

The third variable screen 500 may be a screen for executing at least one scenario related to an industrial site.

For example, when the processor 110 of the first user terminal 100 receives motion data for selecting at least one thumbnail image among thumbnail images of a scenario related to an industrial site, the third variable screen is displayed in the second user terminal. It can cause it to be displayed at 200.

The scenario information 510 may be information including a description related to an operation performed in the third variable screen 500. In addition, the scenario information 510 may include instruction information or quiz information. However, it is not limited thereto.

For example, the scenario information 510 may display information such as Check the defect wafer. Accordingly, the user can recognize that the operation currently being performed is an operation to check the defective wafer. However, it is not limited thereto.

Meanwhile, the at least one first object 520 may be a machine operated by a user. For example, the user may perform a movement of manipulating at least one first object 520 based on question information or instruction information included in the third variable screen 500. In this case, the processor 110 controls the communication unit 130 to capture the user's motion and transmit it to the external server 300, thereby causing at least one first object 520 to be displayed in response to the user's motion. I can. However, it is not limited thereto.

FIG. 8 is a diagram for describing another example of a third variable screen that can be displayed in a second user terminal according to some embodiments of the present disclosure.

Referring to FIG. 8, the third variable screen displayed on the second user terminal 200 may be implemented as a viewpoint similar to the screen actually viewed by the user.

Specifically, on the third variable screen, at least one first object corresponding to a part of the user's body, at least one second object related to the machine, and at least one third object interacting with the at least one first object are displayed. Can include. Here, the at least one first object may include a user's hand, wrist, back of hand, and arm. However, it is not limited thereto.

Here, the at least one third object may be an object related to a control panel, a controller, or a joy stick.

Meanwhile, a user may manipulate at least one second object through at least one third object.

Specifically, the processor 110 of the first user terminal 100 may capture a motion of a user manipulating at least one third object. In addition, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit a third image of the user's movement to the external server 200. In this case, the external server 300 may generate third motion recognition data based on the third image. In addition, when the third motion recognition data is generated, the processor 110 of the first user terminal 100 may receive the third motion recognition data from the external server 300 through the communication unit 130.

Meanwhile, the processor 110 may recognize a change in motion of at least one second object based on the third motion recognition data.

Specifically, the processor 110 may recognize a motion of a user manipulating at least one second object through at least one third object, based on the third motion recognition data. In this case, at least one second object displayed on the second user terminal 200 may operate according to the user's movement.

That is, the user may drive at least one second object by performing a movement to manipulate at least one third object being displayed on the third variable screen. However, it is not limited thereto.

Meanwhile, according to some embodiments of the present disclosure, the processor 110 of the first user terminal 100 may generate detailed feedback by comparing the manipulated motion of at least one second object and predetermined expert data.

Specifically, the processor 110 may recognize a first result value that is a result of an operation performed by the user through at least one second object based on the third motion recognition data.

For example, on the third variable screen, instruction information indicating "Detect a defective element included in the wafer" may be displayed. In this case, the processor 110 may recognize that 80% of defective elements have been detected by the user through manipulation of at least one second object based on the third motion recognition data. In this case, the processor 110 may recognize that the first result value is 80 points. However, it is not limited thereto.

Meanwhile, when the first result value is recognized, the processor 110 may generate detailed feedback by comparing the first result value with expert data.

For example, if the result value generated based on the expert data is a second result value of 90 points, the processor 110 provides detailed feedback on information that the first result value and the second result value differ by 10 points. It may be generated as information and displayed on a third variable screen. However, it is not limited thereto. In this case, the user may receive detailed feedback on an operation of manipulating at least one second object through at least one third object included in the third variable screen.

According to the above-described configuration, a virtual scenario related to at least one industrial site provided by the user through the second user terminal 200 may be displayed to be more similar to the actual situation.

Meanwhile, the position at which the instruction information or quiz information included in the third variable screen is displayed may be changed according to a user's manipulation or setting. Hereinafter, another example of the third variable screen will be described with reference to FIG. 9.

9 is a diagram for describing another example of a third variable screen that can be displayed on a second user terminal according to some embodiments of the present disclosure.

Referring to FIG. 9, a third variable screen displayed on the second user terminal 200 may include at least one second object, at least one third object, and at least one fourth object related to the entire user's body. I can. However, it is not limited thereto.

Meanwhile, the position of instruction information or quiz information included in the third variable screen may change according to a user's manipulation or a virtual scenario.

Specifically, the user may perform a movement of moving at least one of instruction information or quiz information included in the third variable screen. In this case, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit a fourth image of the user's movement to the external server 200. Meanwhile, the external server 300 may generate fourth motion recognition data based on the fourth image. In addition, when the fourth motion recognition data is generated, instruction information or quiz information being displayed on the second user terminal 200 may be displayed at the moved position based on the fourth motion recognition data.

That is, a position at which instruction information or quiz information being displayed on the third variable screen is arranged may be changed based on a user's movement. However, the present invention is not limited thereto, and the position at which the instruction information or quiz information is displayed in the third variable screen is preset according to the scenario, and the storage unit 120 of the first user terminal 100 or the external server 300 It may be stored in the storage.

According to the above-described configuration, even if the instruction information or quiz information displayed on the third variable screen is covered by at least one object, the position to be arranged may be changed according to a user's manipulation.

On the other hand, the viewpoint of viewing at least one scenario included in the variable screen displayed on the second user terminal 200 may be changed according to a user's manipulation. Hereinafter, another example of the variable screen displayed in the second user terminal 200 will be described with reference to FIG. 10.

FIG. 10 is a diagram for describing another example of a third variable screen that can be displayed on a second user terminal according to some embodiments of the present disclosure.

Referring to FIG. 10, a fourth variable screen may be displayed on the second user terminal 200. Here, the fourth variable screen may be a screen generated as a viewpoint of viewing the fourth object from the opposite side of the at least one fourth object. However, it is not limited thereto.

Specifically, the user may perform a movement to move the viewpoint of the fourth variable screen.

For example, a virtual joystick or the like for manipulating the viewpoint of the fourth variable screen may be displayed on the fourth variable screen. However, it is not limited thereto. In this case, the processor 110 of the first user terminal 100 may control the communication unit 130 to transmit a fifth image of the user's movement to the external server 200. Meanwhile, the external server 300 may generate fifth motion recognition data based on the fifth image. In addition, when the fifth motion recognition data is generated, the viewpoint of the fourth variable screen displayed on the second user terminal 200 may be changed. However, it is not limited thereto.

Meanwhile, in the present disclosure, at least one object, a viewpoint of viewing at least one variable screen, or a setting related to at least one variable screen may be changed based on a scenario related to at least one industrial site. However, it is not limited thereto.

According to the above-described configuration, the viewpoint of the fourth variable screen displayed by the second user terminal 200 may be changed by a user's manipulation. Accordingly, the user can experience scenarios related to at least one industrial site from various angles.

11 shows a general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

While the present disclosure has generally been described above with respect to computer-executable instructions that may be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may be implemented in combination with other program modules and/or as a combination of hardware and software. I will know.

In general, modules herein include routines, procedures, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Further, to those skilled in the art, the method of the present disclosure is not limited to single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable household appliances, and the like (each of which It will be appreciated that it may be implemented with other computer system configurations, including one or more associated devices).

The described embodiments of the present disclosure may also be practiced in a distributed computing environment where certain tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. A computer-accessible medium includes volatile and non-volatile media, transitory and non-transitory media, and removable and non-removable media. By way of example, and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media.

Computer-readable storage media include volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Includes the medium. Computer-readable storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage. Devices, or any other medium that can be accessed by a computer and used to store desired information.

Computer-readable transmission media typically implement computer-readable instructions, data structures, program modules or other data on a modulated data signal such as a carrier wave or other transport mechanism. Includes all information transmission media. The term modulated data signal refers to a signal in which one or more of the characteristics of the signal is set or changed to encode information in the signal. By way of example, and not limitation, computer-readable transmission media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above-described media are also intended to be included within the scope of computer-readable transmission media.

An exemplary environment 1100 is shown that implements various aspects of the present disclosure, including a computer 1102, which includes a processing device 1104, a system memory 1106, and a system bus 1108. do. System bus 1108 couples system components, including but not limited to, system memory 1106 to processing device 1104. The processing unit 1104 may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be used as processing unit 1104.

The system bus 1108 may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read-only memory (ROM) 1110 and random access memory (RAM) 1112. The basic input/output system (BIOS) is stored in non-volatile memory 1110 such as ROM, EPROM, EEPROM, etc. This BIOS is a basic input/output system that helps transfer information between components in the computer 1102, such as during startup Includes routines. RAM 1112 may also include high speed RAM such as static RAM for caching data.

The computer 1102 also includes an internal hard disk drive (HDD) 1114 (e.g., EIDE, SATA)-this internal hard disk drive 1114 can also be configured for external use within a suitable chassis (not shown). Yes-, magnetic floppy disk drive (FDD) 1116 (for example, to read from or write to removable diskette 1118), and optical disk drive 1120 (e.g., CD-ROM For reading the disk 1122 or reading from or writing to other high-capacity optical media such as DVD). The hard disk drive 1114, magnetic disk drive 1116, and optical disk drive 1120 are each connected to the system bus 1108 by a hard disk drive interface 1124, a magnetic disk drive interface 1126, and an optical drive interface 1128. ) Can be connected. The interface 1124 for implementing an external drive includes, for example, at least one or both of USB (Universal Serial Bus) and IEEE 1394 interface technologies.

These drives and their associated computer readable media provide non-volatile storage of data, data structures, computer executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of the computer-readable storage medium above refers to a removable optical medium such as a HDD, a removable magnetic disk, and a CD or DVD, those skilled in the art may include a zip drive, a magnetic cassette, a flash memory card, a cartridge, It will be appreciated that other types of computer-readable storage media, such as etc., may also be used in the exemplary operating environment and that any such media may contain computer-executable instructions for performing the methods of the present disclosure. .

A number of program modules, including the operating system 1130, one or more application programs 1132, other program modules 1134, and program data 1136, may be stored in the drive and RAM 1112. All or part of the operating system, applications, modules, and/or data may also be cached in RAM 1112. It will be appreciated that the present disclosure may be implemented on a number of commercially available operating systems or combinations of operating systems.

A user may input commands and information to the computer 1102 through one or more wired/wireless input devices, for example, a pointing device such as a keyboard 1138 and a mouse 1140. Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. These and other input devices are often connected to the processing unit 1104 through the input device interface 1142, which is connected to the system bus 1108, but the parallel port, IEEE 1394 serial port, game port, USB port, IR interface, It can be connected by other interfaces such as etc.

A monitor 1144 or other type of display device is also connected to the system bus 1108 through an interface such as a video adapter 1146. In addition to the monitor 1144, the computer generally includes other peripheral output devices (not shown) such as speakers, printers, etc.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communication. The remote computer(s) 1148 may be a workstation, server computer, router, personal computer, portable computer, microprocessor-based entertainment device, peer device, or other common network node, and is generally It includes many or all of the described components, but for brevity, only memory storage device 1150 is shown. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or to a larger network, eg, a wide area network (WAN) 1154. Such LAN and WAN networking environments are common in offices and companies, and facilitate an enterprise-wide computer network such as an intranet, all of which can be connected to a worldwide computer network, for example the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 via a wired and/or wireless communication network interface or adapter 1156. Adapter 1156 may facilitate wired or wireless communication to LAN 1152, which also includes a wireless access point installed therein to communicate with wireless adapter 1156. When used in a WAN networking environment, the computer 1102 may include a modem 1158, connected to a communication server on the WAN 1154, or through the Internet, etc. to establish communication through the WAN 1154 Have means. The modem 1158, which may be an internal or external and a wired or wireless device, is connected to the system bus 1108 through a serial port interface 1142. In a networked environment, program modules described for the computer 1102 or portions thereof may be stored in the remote memory/storage device 1150. It will be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers may be used.

Computer 1102 is associated with any wireless device or entity deployed and operated in wireless communication, e.g., a printer, scanner, desktop and/or portable computer, portable data assistant (PDA), communication satellite, wireless detectable tag. It operates to communicate with any equipment or place and phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Thus, the communication may be a predefined structure as in a conventional network or may simply be ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) allows you to connect to the Internet, etc. without wires. Wi-Fi is a wireless technology such as a cell phone that allows such devices, for example computers, to transmit and receive data indoors and outdoors, ie anywhere within the coverage area of a base station. Wi-Fi networks use a wireless technology called IEEE 802.11 (a, b, g, etc.) to provide a secure, reliable and high-speed wireless connection. Wi-Fi can be used to connect computers to each other, to the Internet, and to a wired network (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in unlicensed 2.4 and 5 GHz radio bands, for example at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). have.

A person of ordinary skill in the art of the present disclosure includes various exemplary logical blocks, modules, processors, means, circuits and algorithm steps described in connection with the embodiments disclosed herein, electronic hardware, (convenience). For the sake of clarity, it will be appreciated that it may be implemented by various forms of program or design code or a combination of both (referred to herein as "software"). To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. A person of ordinary skill in the art of the present disclosure may implement the described functions in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term “article of manufacture” includes a computer program or media accessible from any computer-readable device. For example, computer-readable storage media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CD, DVD, etc.), smart cards, and flash Memory devices (eg, EEPROM, cards, sticks, key drives, etc.), but are not limited thereto. The term “machine-readable medium” includes, but is not limited to, wireless channels and various other media capable of storing, holding, and/or transmitting instruction(s) and/or data.

It is to be understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on the design priorities, it is to be understood that within the scope of the present disclosure a specific order or hierarchy of steps in processes may be rearranged. The appended method claims provide elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be apparent to those of ordinary skill in the art, and general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments presented herein, but is to be interpreted in the widest scope consistent with the principles and novel features presented herein.

Claims (14)

A computer program stored on a computer-readable storage medium, comprising:
The computer program includes instructions for causing a processor of the first user terminal to perform the following steps, the steps:
Recognizing whether or not the camera of the first user terminal is activated;
Receiving at least one thumbnail image related to at least one variable image from an external server when it is recognized that activation of the camera is possible;
When the at least one thumbnail image is received, transmitting the at least one thumbnail image to a second user terminal, thereby causing the at least one thumbnail image to be displayed on the second user terminal;
When the thumbnail image is transmitted to the second user terminal, acquiring a first image including the user's movement through the camera;
Transmitting the first image to an external server;
Receiving first motion recognition data related to the first image from the external server after transmitting the first image;
Recognizing a first thumbnail image selected by a user from among the at least one thumbnail image based on the first motion recognition data;
When the first thumbnail image is recognized, transmitting a second signal for requesting a first variable image related to the first thumbnail image to the external server; And
Causing the first variable image to be displayed on the second user terminal as the second signal is transmitted;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The second user terminal,
A terminal paired with the first user terminal,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
The second user terminal,
Installed on a device worn on the user's head so that the display unit is positioned in front of the user's eyes,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
When the first thumbnail image is recognized, transmitting a second signal for requesting a first variable image related to the first thumbnail image to the external server,
Causing a first virtual screen including at least one virtual button to be displayed on the second user terminal when the first thumbnail image is recognized;
When the first virtual screen including the at least one virtual button is displayed on the second user terminal, obtaining a second image including the user's movement through the camera;
Transmitting the second image to the external server;
Receiving second motion recognition data related to the second image from the external server after transmitting the second image; And
Recognizing a first virtual button selected by the user from among the at least one virtual button based on the second motion recognition data;
Including,
In response to transmitting the second signal, causing the first variable image to be displayed on the second user terminal,
Causing first information including at least one of instruction information related to the first virtual button and quiz information related to the first virtual button to be displayed together with the first variable image;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 4,
In response to transmitting the second signal, causing the first variable image to be displayed on the second user terminal,
If the received first information is the indication information, obtaining a third image including a user's movement through the camera;
Transmitting the third image to the external server;
Receiving third motion recognition data related to the third image from the external server after transmitting the third image;
Generating performance rate information for the indication information based on the third motion recognition data; And
Calculating a result of performing the first variable image based on the performance rate information;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 5,
Recognizing a performance rate of performing at least one instruction included in the instruction information based on the third motion recognition data,
Recognizing at least one instruction included in the instruction information;
Determining at least one motion data corresponding to the at least one indication from among the third motion recognition data;
Generating coincidence rate information by comparing the at least one motion data with predetermined expert motion data corresponding to the at least one indication; And
Calculating result information of performing the first variable image based on the matching rate information;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 6,
Calculating result information of performing the first variable image based on the matching rate information,
Generating strength information based on the at least one motion data when the coincidence rate information is greater than or equal to a preset value;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 6,
Calculating result information of performing the first variable image based on the matching rate information,
Generating weakness information based on the at least one motion data when the match rate information is less than a preset value;
A computer program stored on a computer-readable storage medium.
The method of claim 4,
Causing the first variable image to be displayed on the second user terminal by transmitting the second signal,
When the received first information is the quiz information, obtaining a fourth image including the user's movement through the camera;
Transmitting the fourth image to the external server;
Receiving fourth motion recognition data related to the fourth image from the external server after transmitting the fourth image;
Generating correct answer rate information for the quiz information based on the fourth motion recognition data; And
Calculating a result of performing the first variable image including the quiz information based on the correct answer rate information;
Containing,
A computer program stored on a computer-readable storage medium.
The method of claim 1,
Causing a second variable image including a feedback panel image for the first variable image to be displayed on a second user terminal;
Obtaining a feedback motion image corresponding to the feedback panel image;
Transmitting the feedback motion image to the external server; And
Causing the external server to generate feedback information for the feedback motion image;
Containing,
A computer program stored on a computer-readable storage medium.
As a first user terminal,
Communication department;
A camera that photographs the user's movement;
A processor that recognizes whether or not the camera is activated;
Including,
The processor,
When it is recognized that activation of the camera is possible, at least one thumbnail image related to at least one variable image is received from an external server through the communication unit,
When the at least one thumbnail image is received, by transmitting the at least one thumbnail image to a second user terminal through the communication unit, the at least one thumbnail image is caused to be displayed on the second user terminal,
When the thumbnail image is transmitted to the second user terminal, a first image including the user's movement is obtained through the camera,
Controlling the communication unit to transmit the first image to an external server,
After transmitting the first image, receiving first motion recognition data related to the first image from the external server through the communication unit,
Recognizing a first thumbnail image selected by a user from among the at least one thumbnail image, based on the first motion recognition data,
When the first thumbnail image is recognized, controlling the communication unit to transmit a second signal requesting a first variable image related to the first thumbnail image to the external server,
Causing the first variable image to be displayed on the second user terminal by transmitting the second signal
The first user terminal.
The method of claim 11,
The processor,
When the first thumbnail image is recognized, causing a first virtual screen including at least one virtual button to be displayed on the second user terminal,
When the first virtual screen including the at least one virtual button is displayed on the second user terminal, a second image including the movement of the user is obtained through the camera,
Controlling the communication unit to transmit the second image to the external server,
After transmitting the second image, second motion recognition data related to the second image is received from the external server through the communication unit,
Recognizing a first virtual button selected by the user among the at least one virtual button based on the second motion recognition data,
By transmitting the second signal, causing the first variable image to be displayed on the second user terminal,
Causing first information including at least one of instruction information related to the first virtual button and quiz information related to the first virtual button to be displayed together with the first variable image,
The first user terminal.
The method of claim 12,
When the received first information is the indication information, a third image including a user's movement is acquired through the camera,
Controlling the communication unit to transmit the third image to the external server,
After transmitting the third image, receiving third motion recognition data related to the third image from the external server through the communication unit,
Generate performance rate information for the instruction information based on the third motion recognition data,
Calculating a result of performing the first variable image based on the performance rate information,
The first user terminal.
The method of claim 13,
The processor,
Recognizing at least one instruction included in the instruction information,
Determining at least one motion data corresponding to the at least one indication from among the third motion recognition data,
Generating coincidence rate information by comparing the at least one motion data with predetermined expert motion data corresponding to the at least one indication,
Calculating result information of performing the first variable image based on the matching rate information,
The first user terminal.

Images