KR102092290B1 - Providing system for a virtual manual of electronic device based on the agmented reality and method therefor - Google Patents

Abstract

The present invention relates to a system for providing a virtual manual for an electronic device based on augmented reality and a method for providing a virtual manual, and more specifically, to recognize and data an electronic device for which a manual is provided, and transmit it from a server to a corresponding electronic device through a communication network. The present invention relates to a system for providing a virtual manual for an electronic device and a method for providing a virtual manual, comprising a wearable computer that provides training manual data to a user based on augmented reality by receiving training manual data for Korea.

Description

Providing system for a virtual manual of electronic device based on the agmented reality and method therefor}

The present invention relates to a system for providing a virtual manual for an electronic device based on augmented reality and a method for providing a virtual manual, and more specifically, to recognize and data an electronic device for which a manual is provided, and transmit it from a server to a corresponding electronic device through a communication network. The present invention relates to a system for providing a virtual manual for an electronic device and a method for providing a virtual manual, comprising a wearable computer that provides training manual data to a user based on augmented reality by receiving training manual data for Korea.

In general, augmented reality (Agmented reality) refers to a technology that displays a virtual object superimposed on the real space that the user sees with the naked eye.

Augmented reality is also referred to as mixed reality because it displays virtual information that has additional information in real time in the real space, and is also referred to as mixed reality, and the related research continues to develop.

Augmented reality uses a virtual environment made of computer graphics, but computer graphics can provide better realism and additional information by providing additional information necessary for the real environment.

These augmented reality technologies are actively applied in information providing services of stores, telemedicine diagnosis, and games, and wearable computers have also been developed to realize augmented reality outdoors.

In particular, a wearable computer in the form of a head is capable of augmented reality by displaying computer graphics and characters in real time by superimposing on the real environment viewed by the user.

The wearable computer in the form of a head is called a head mounted display (HMD), and as a commercialized product, there is a product of the United States Microsoft (Micro Soft), a brand name, Hollens (Hololens).

The holo lens uses Windows 10 as an O / S, has a high-resolution hologram lens, and provides an API (application programming interface) that can recognize human eyes, gestures, voice, and surroundings. Supported, 3D scanning function is also available.

However, the level of technology used for practical and educational interactions with augmented reality by interacting with electronic devices used in various industrial sites has not been developed.

When augmented reality is used for information necessary for operation or maintenance of electronic devices, the user can immediately obtain information necessary for work while looking at actual equipment, and Boeing in the United States uses augmented reality technology to The system that assists with the assembly of electric wires has been developed and tested in the field, and the German BMW company has developed and tested an educational system based on augmented reality to assemble a car.

Therefore, there is an urgent need to develop an educational manual providing system for educational and practical application of electronic devices used in industrial sites by using augmented reality technology applied in industrial fields.

On the other hand, as a technique using augmented reality, a technique of a surgical robot system using the augmented reality of Korean Patent No. 957470 and a control method thereof are known.

The present invention was devised to solve the above-mentioned problems as described above, and provides education and practical application of electronic devices by providing manuals for electronic devices used in various industrial sites using augmented reality technology. It provides a virtual manual providing system and method for augmented reality based electronic devices.

The system for providing a virtual manual of the electronic device of the present invention and a method for providing the virtual manual for achieving the above technical problem are recognized and dataized by the electronic device for manual provision, and training manual data for the corresponding electronic device from a server through a communication network. It is characterized in that it comprises a wearable computer that provides the user with the corresponding training manual data based on augmented reality.

The virtual manual provision system and method of the augmented reality-based electronic device of the present invention configured as described above, utilizes sensors and functions provided by the wearable computer worn on the head, regardless of the size or function of the electronic device. It is easy to learn how to use the electronic device through a series of processes (referred to as calibration in the present invention) that recognizes and interacts with the hardware of the electronic device in the form, thereby performing education and practical application of the electronic device within a short time. I can do it.

In addition, the system and method for providing a virtual manual of the electronic device of the present invention can perform training and practical application regardless of the environment in which the electronic device is installed, thereby reducing the initial operation introduction period and training cost of practitioners of the device. It becomes possible.

In addition, since the present invention can download and use information of various electronic devices registered on the server for education, it is possible to learn how to use various electronic devices with one wearable computer and use it as a virtual manual for various devices. It has an effect.

In addition, if the educational system of the present invention is utilized by an educational institution of an electronic device, it can be trained as a manpower that can be immediately put into practice in an industrial field, which leads to an improvement in the employment rate of the educational institution and a socially economic ripple effect. It is a very advanced invention that expresses an effect that can bring about.

1 is a block diagram of a system for providing a virtual manual of an electronic device of the present invention,
2 is a view showing a training performance state using the virtual manual provision system of the electronic device of the present invention,
3 is a flowchart of a method for providing a virtual manual of an electronic device of the present invention,
4A and 4B are conceptual diagrams illustrating calibration steps of a virtual education method of a method for providing a virtual manual of an electronic device of the present invention;
5 is a flowchart of a detailed process of a calibration step of a method for providing a virtual manual of an electronic device of the present invention,
6A to 6I are diagrams of an example of an augmented reality screen displayed on a wearable computer in a virtual training process of an electronic device of the present invention.

Hereinafter, the configuration and operation of the virtual manual providing system and method of the augmented reality based electronic device of the present invention will be described in detail based on the accompanying drawings.

However, the illustrated drawings are provided as an example for sufficiently transmitting the spirit of the present invention to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other aspects.

In addition, unless otherwise defined in terms used in the specification of the present invention, those skilled in the art to which the present invention pertains have the meanings commonly understood, and the gist of the present invention in the following description and the accompanying drawings. Detailed descriptions of well-known functions and configurations that may be unnecessarily cloudy are omitted.

1 is a block diagram of a virtual manual provision system of an electronic device of the present invention.

The virtual manual provision system of the electronic device of the present invention (hereinafter abbreviated as 'virtual manual provision system') is a sensor and 3D scanning, high resolution hologram lens that recognizes the gaze, gesture, voice, and surrounding environment provided by the wearable computer. It is a system that performs education of the electronic device by recognizing and converting the electronic device to be trained using the function of.

To this end, the virtual manual providing system of the present invention recognizes and dataizes the electronic device 50 for education and receives training manual data for the electronic device 50 from the server 30 through the communication network 20. Characterized by a configuration comprising a wearable computer 10 provided to the user.

At this time, the wearable computer 10 of the present invention performs an operation to dataize the appearance of the electronic device 50, and hereinafter, this operation is referred to as calibration in the present specification.

The object to be calibrated by the wearable computer 10 of the present invention is the shape of the case of the electronic device 50, various types of terminals such as control terminals, switches, knobs, buttons, etc., placed on the case, and on the case. And a display window for a display attached to it.

On the other hand, the electronic device 50 that is the subject of education of the present invention, as shown, all the electronic devices such as machine tools, measuring instruments, test equipment, tester equipment, as well as sound control equipment (mixers) used to control the stage sound equipment Collectively.

FIG. 2 is a diagram showing a training performance state using the virtual manual provision system of the present invention, in which the electronic device 50 for training is worn by the user U while the user U wears the wearable computer 10. ) Indicates a state represented by a hologram.

The wearable computer 10 is capable of transmitting the training manual data of the electronic device 50 to the user U by recognizing and dataizing the electronic device 50 through calibration.

At this time, the wearable computer 10 generates spatial data around the electronic device 50 by performing spatial mapping by a built-in ambient environment recognition sensor (such as an image sensor), and generates the generated spatial data. It can be integrated with the data of the calibrated electronic device 50 to display menus and help data necessary for training according to conditions and situations. As an example, in the drawing shown in FIG. 2, help data such as 'turn the dial to adjust the frequency', which is one of the training data of the corresponding electronic device 50, is displayed.

The wearable computer 10 of the embodiment of the present invention employs a head mounted display (HMD), which is a wearable computer in the form of a head, and specifically, employs the product name Hollens, which is a product of Microsoft Soft of the United States. Hiring.

As described above, the holo lens uses Windows 10 as an O / S, a high-resolution hologram lens is embedded, and an API capable of recognizing human gaze, gesture, voice, and surrounding environment It supports (application programming interface), and it is a product that can perform 3D scanning function. In particular, it is a device that can basically perform the above-described spatial mapping function.

In addition, the server 30 for transmitting and receiving data through the wearable computer 10 and the communication network 20 stores training manual data for the corresponding electronic device 50 in the interlocked database 40, and the corresponding electronic device ( Update the training manual data of 50) to the latest data.

In addition, the server 30 provides identification information such as ID and password of a user who receives training manual data through the wearable computer 10 and password and personal information to provide a service through the virtual manual provision system of the present invention. Received and stored in database 40 and managed.

In addition, the communication network 20 is a known wired or wireless communication network such as the Internet capable of transmitting and receiving data between the wearable computer 10 and the server 30.

Accordingly, in the virtual manual providing system of the present invention configured as described above, when a user wears the wearable computer 10 and visually views the electronic device 50 to be trained, the communication network 20 from the server 30 Through this, the training manual data for the corresponding electronic device 50 is transmitted to the wearable computer 10 and displayed. The user can display the detailed name of each part / part of the corresponding electronic device 50 through the displayed training manual data. Function and usage can be trained sequentially through manual.

Hereinafter, the virtual education method of the electronic device of the present invention implemented by the virtual manual providing system of the present invention configured as described above will be described in detail. 3 is a flowchart of a method for providing a virtual manual of the present invention.

The virtual education method of the electronic device of the present invention utilizes the functions of the sensor, 3D scanning, and high-resolution hologram lens that recognizes the gaze, gesture, voice, and surrounding environment provided by the wearable computer, and recognizes and dataizes the electronic device of the education object. By doing so, it is a method of conducting education of the corresponding electronic device.

To this end, the virtual education method of the present invention recognizes and converts the electronic device 50 as an education target and receives the training manual data for the electronic device 50 from the server 30 through the communication network 20. It is performed by the wearable computer 10 provided to the.

Also, in order to perform virtual education of the electronic device of the present invention, it is preferable that an application program for education of an electronic device transmitted from the server 30 to the wearable computer 10 is executed and each step proceeds.

In addition, in order to perform the virtual education of the electronic device of the present invention, the training is performed while the wearable computer 10 sees the electronic device 50 as an education target, but the electronic device 10 is not physically present in the education field. Even if it does not exist, training can be performed with only training manual data transmitted from the server 30.

First, referring to FIG. 3, a method for providing a virtual manual of an electronic device of the present invention includes a step in which a wearable computer 10 worn on a user is logged in to the server 30 via a communication network 20 (S10) ) And the step (S20) of inputting identification information such as a user ID and password required for login into the server 30.

6A is a diagram illustrating an example of an augmented reality screen displayed as a hologram on the wearable computer 10 in the step of performing the login, and an input window of a user ID and password is displayed.

In addition, FIG. 6B is an example of an augmented reality screen for inputting identification information for login in step S20, and a character pad and a numeric keypad are displayed.

Next, the login of step S20 is performed so that the wearable computer 10 is connected to the server 30, and the server 30 is connected to the wearable computer 10 through the communication network 20 from the server 30. A list of electronic devices capable of providing manual data for education is delivered (S30).

6C shows an example of an augmented reality screen for selecting a list of electronic devices capable of providing manual data for education in step S30.

Then, when an electronic device desired to be trained is selected by the wearable computer 10, the server 30 determines whether the data for education of the selected electronic device is the latest training manual data and updates the corresponding data. Let (S40).

The update of the training manual data may be performed by searching the database 40 linked with the server 30, or notify that there is a patch data of training data to the user's wearable computer 10. After that, it may be made according to the user's request to perform the patch.

6D shows an example of an augmented reality screen showing a menu window for selecting a request to perform a patch of training manual data in step S40.

Subsequently, the wearable computer 10 performs calibration of the electronic device for education (S50).

Performing the above calibration is to make the appearance of data on the electronic device 50 to be educated, and in this case, the wearable computer 10 is spatial mapping by a built-in environment recognition sensor (such as an image sensor). To generate spatial data around the electronic device 50 and integrate the generated spatial data with the training manual data of the calibrated electronic device 50 to display menus and help data necessary for training according to conditions and circumstances. You can do it.

4A and 4B are conceptual diagrams illustrating calibration steps of a method for providing a virtual manual of an electronic device of the present invention.

First, as illustrated in FIG. 4A, the calibration by the wearable computer 10 of the present invention recognizes the entire outline of the electronic device 50 to be education, and recognizes the entire outline of the recognized electronic device 50. You will be tracking.

For this reason, considering that the wearable computer 10 is worn on the head, the movement and rotation of the X-axis, Y-axis, and Z-axis of the electronic device 50 change according to the movement of the user's head. Considering the case where it is difficult to discriminate the electronic device 50 due to the continuous 3D scanning when tracking the coordinates of the present invention, in the calibration step of the present invention, four or less points (P1 in FIG. 4A) for the entire outline of the electronic device 50. By designating, P2, P3, P4), it is possible to minimize the performance of 3D scanning that affects the speed of the frame by designating the shape of the object that is deformed according to the user's head movement.

Next, as shown in FIG. 4B, when the entire outer periphery of the electronic device 50 is recognized through four points P1, P2, P3, and P4 as described above, tracking becomes possible, and the present invention is worn. Various terminals such as control terminals, switches, and knobs on the case of the electronic device 50 recognized by the expression computer 10 as a whole, and a display window for a display on the case. Recognizes the outline of the small object of (B1, B2, B3, B4 in FIG. 4B), and tracking is performed on the outline of the recognized small object.

And, by using the position and size ratio information of the main terminals of the electronic device 50 stored in the server 30 to correct the viewing angle of the wearable computer 10 according to the user's head movement, the small size of the electronic device 50 The outer periphery of objects can be recognized more accurately.

Here, the correction of the viewing angle by the wearable computer 10 calculates the angle of inclination of the image of the electronic device currently displayed based on the external value of the electronic device on which the initial calibration is performed, and is preset in the server 30 The position and size of the small object of the electronic device 50 are corrected according to the information on the small object of the electronic device 50 and the angle of inclination of the calculated image of the currently displayed electronic device.

According to an embodiment of the present invention, it is preferable to recognize and track a small object having a size of at least 10 mm and 10 mm in length through calibration by the wearable computer 10 as described above.

5 is a flowchart of a detailed process of a calibration step of a method for providing a virtual manual of an electronic device of the present invention.

Referring to the drawings, the step of performing the above-described calibration (S50) includes: Spatial Mapping is performed by a surrounding environment recognition sensor such as an image sensor built into the wearable computer 10 (S51); The overall outline of the electronic device 50 actually displayed by the wearable computer 10 and the location of the entire outline of the electronic device 50 and the outline of the small objects by recognizing the outline of small objects such as terminals, buttons, and displays. Setting them (S52); When the entire outer edge of the electronic device 50 and the outer positions of the small objects are set, the entire outer edge of the electronic device 50 and the position of the main terminals of the electronic device 50 stored in the server 30 and size ratio information are used. It is made by performing a step (S53) of correcting the rotation value of the outline according to the position and size of the outline of the small objects and the angle of the tilt of the image described above.

FIG. 6E shows an example of the augmented reality screen displaying a menu of the steps of correcting the outline of the calibration, and adjusting the position value for each of the position, size, and rotation. Displays the menu (X-axis, Y-axis, UP-DOWN in Z-axis direction), menu (UP, DOWN) for adjusting the value of size, and menu (clockwise, counter-clockwise, UP / DlOWN) for adjusting the rotation value. have.

In addition, FIG. 6F shows an example of an augmented reality screen in which guidance for a basic button is displayed after calibration is performed.

In addition, in order to perform the above-described calibration, the calibration position according to the gaze data is adjusted, the position, size, and rotation values are finely adjusted, the calibration is terminated, and pop-ups of various menus are initialized or activated, and over time. Therefore, it is possible to disable the pop-up of the menu, create a virtual UI that accesses the home main, move the coordinates of the UI according to the gaze data, initialize the gesture manager, perform management events, effect effects, and recognize speech. It is preferable that the logics are performed by an electronic device education application program transmitted to the wearable computer 10, and examples of the logics will be described below in a modified manner. The contents of the logic refer to the fingerprint described in the contents of the algorithm.

1) Algorithm for adjusting calibration position according to gaze data SetScale.cs

private void LateUpdate ()
{
if (index == 0 && Application.platform! = RuntimePlatform.WindowsEditor)
{
transform.position = cursor.position; // 1. The coordinate value of the position you are looking at in the space created by spatial mapping

// Calculate the angle value of the main camera position and the position of 1
Vector3 directionToTarget = Camera.main.transform.position-transform.position;
//directionToTarget.x = 0.0f;
if (directionToTarget.sqrMagnitude <0.001f)
return;
// Modify cursor to the angle
transform.rotation = Quaternion.LookRotation (-directionToTarget);
transform.eulerAngles = new Vector3 (cursor.eulerAngles.x, transform.eulerAngles.y, transform.eulerAngles.z);
}

2) Alignment algorithm for position, size and rotation value CalibractionManager.cs

public void SetRotate (ButtonClick btn)
{
// Apply the value to rotate
targetObject.transform.Rotate (btn.posSend);
// Display the current panel's angle values x, y, z
rotX.text = string.Format ("{0}", (int) targetObject.transform.eulerAngles.x);
rotY.text = string.Format ("{0}", (int) targetObject.transform.eulerAngles.y);
rotZ.text = string.Format ("{0}", (int) targetObject.transform.eulerAngles.z);
}
public TextMesh posX, posY, posZ;
public void SetPosition (ButtonClick btn)
{
// Apply the value to move
targetObject.transform.Translate (btn.posSend);
// Display the current panel position values x, y, z
posX.text = string.Format ("{0: F2}", targetObject.transform.position.x);
posY.text = string.Format ("{0: F2}", targetObject.transform.position.y);
posZ.text = string.Format ("{0: F2}", targetObject.transform.position.z);
}
public TextMesh sclX, sclY, sclZ;
public void SetScale (ButtonClick btn)
{
// Apply the panel size value
targetObject.transform.localScale + = btn.posSend;
// Display the current panel size values x, y, z
sclX.text = string.Format ("{0: F2}", targetObject.transform.localScale.x);
sclY.text = string.Format ("{0: F2}", targetObject.transform.localScale.y);
sclZ.text = string.Format ("{0: F2}", targetObject.transform.localScale.z);

3) Calibration termination algorithm Setscale.cs

// delete parent transform
transform.parent = null;
p1.gameObject.SetActive (false);
p2.gameObject.SetActive (false);

// index initialization
index = 0;

// deactivate object
enabled = false;
// End spatial mapping
spatial.SetActive (false);

// View menu
MenuManager.Instance.FrontPanel ();

4) Popup initialization and instance creation and popup activation algorithm Popup.cs

void Start ()
{
Instance = this; // Instance registration
gameObject.SetActive (false); // disable on first run
}

Popup.cs

// Set the popup.
public void SetPopup (string desc, float time, bool useLimit, bool ViewProgress = false)
{
// Whether to use the content, the time to show, the function to disappear automatically, or to show the progress bar
isPlay = true;

tDesc.text = desc; // set content

timeSum = 0; // sum time
// Set maximum time when useLimit == true
if (useLimit)
{
HIDE = true;
timeLimit = time;
}

progress.gameObject.SetActive (ViewProgress); // show progress bar if ViewProgress == true
tPer.text = 0 + "%"; // ViewProgress == If true, display the progress bar%

gameObject.SetActive (true); // display popup
}

5) Pop-up deactivation algorithm over time Popup.cs

void Update () {
if (HIDE)
{
// Maximum time <Total time from start
if (timeLimit <(timeSum + = Time.deltaTime))
{
tPer.text = 100 + "%";
progress.value = 100;
isPlay = false;
gameObject.SetActive (false); // hide popup
}
}
}

6) Virtual UI generation algorithm approaching home main MenuManager.cs

// Return to the home menu.
public void Home ()
{
timeSum = 0;
transform.position = target.position; // specify the position where the menu is displayed
CHASE = false;

// Set the menu to hide when the reset position is not
for (int i = 0; i <notSettingHideList.Length; i ++)
notSettingHideList [i] .SetActive (SetPanel.SETTING);
// Hide menus other than the home menu
for (int i = 0; i <hideHomeList.Length; i ++)
hideHomeList [i] .SetActive (false);

mainMenuList.SetActive (true); // Show the home menu list.
menuSpectrumAnalyzer.SetActive (false); // hide the device's manual.
// Hide the panel.
if (ForotPanelManager.Instance! = null)
ForotPanelManager.Instance.HideObject ();
// MenuView ();

gameObject.SetActive (true); // Show the home menu.
}

7) UI Coordinate Movement Algorithm According to Gaze Data MenuManager.cs

void LateUpdate ()
{
if (anim! = null && anim.isPlaying)
return;

// If there is no menu in focus, it moves to focus after a certain time
if (GazeManager.Instance.focusedObject == null && CHASE)
{
// Calculate the distance between the focus and the current position
transform.position = Vector3.Lerp (toPos, target.position, (timeSum + = Time.deltaTime) / duration);

// End when the maximum movement time has elapsed and move to the focus position
if (timeSum> = duration)
{
timeSum = 0;
transform.position = target.position;
}
}
}

8) Gesture manager initialization and management event occurrence and effect effect algorithm GestureManager.cs

void Start ()
{
// Register gesture manager.
gestureRecognizer = new GestureRecognizer ();
gestureRecognizer.SetRecognizableGestures (GestureSettings.Tap);

// Tap event registration
gestureRecognizer.TappedEvent + = GestureRecognizer_TappedEvent;

// Start the gesture manager.
gestureRecognizer.StartCapturingGestures ();
}

public void TapEventEffect ()
{
if (effect == null)
return;

// Particle play
if (effect.isPlaying)
{
effect.Stop (); // stop
effect.Clear (); // initialize
}
effect.transform.position = transform.position;
// Move the particle position
effect.Play (); // Play
}
GestureManager.cs

// Called when a tab event occurs.
private void GestureRecognizer_TappedEvent (InteractionSourceKind source, int tapCount, Ray headRay)
{
// Used when setting the panel position
if (scale! = null)
{
// When the panel is not positioned
if (MOVE_PANEL)
{
// PanelDrop ();
MOVE_PANEL =! (Scale.SetPoint ()); // specify the location.

/ * if (! MOVE_PANEL)
{
panel.ViewLayOut ();
ForotPanelManager.Instance.SetTutorialMode (true);
} * /
}
}

// When the focus is on the object
if (focusedObject! = null)
{
focusedObject.SendMessage ("OnSelect"); // call message with that object
TapEventEffect (); // effect occurs at the location
}
}

9) Initialization and effect effect algorithm for speech recognition KeywardManager.cs

void Start ()
{
// Register keywords to be used for speech recognition
keywords.Add ("Reset Position", () =>
{
Debug.Log ("Key Words Start: Reset Position");
MenuManager.Instance.ResetPosition (); // Run this function when the keyword is recognized.
});
keywords.Add ("Front Panel", () =>
{
Debug.Log ("Key Words Start: Front Panel");
MenuManager.Instance.Features ();
});
keywords.Add ("View Manual", () =>
{
Debug.Log ("Key Words Start: View Manual");
MenuManager.Instance.SpectrumAnalyzer ();
});
keywords.Add ("Tutorial List", () =>
{
Debug.Log ("Key Words Start: Tutorial List");
MenuManager.Instance.listMenu [0] .ViewPlay ();
});
keywords.Add ("Select Machine", () =>
{
Debug.Log ("Key Words Start: Select Machine");
if (! MenuManager.Instance.mainMenuList.gameObject.active || SetPanel.SETTING)
return;
?
MenuManager.Instance.ViewSelectMachine ();
});
keywords.Add ("Step Back", () =>
{
Debug.Log ("Key Words Start: Tutorial Back");
Tutorial.BACK = true;
});
keywords.Add ("Step Next", () =>
{
Debug.Log ("Key Words Start: Tutorial Next");
Tutorial.NEXT = true;
});

// Register keyword and register keyword event and start.
keywordRecognizer = new KeywordRecognizer (keywords.Keys.ToArray ());
keywordRecognizer.OnPhraseRecognized + = KeywordRecognizer_OnPhraseRecognized;
keywordRecognizer.Start ();
}
?
private void KeywordRecognizer_OnPhraseRecognized (PhraseRecognizedEventArgs args)
{
// Called when a registered keyword is recognized.
System.Action keywordAction;
?
// Search using recognized keywords.
if (keywords.TryGetValue (args.text, out keywordAction))
{
// call the specified function
keywordAction.Invoke ();
}
}

Next, after the calibration step S40 is performed as described above, the main menu for educating the corresponding electronic device is displayed on the wearable computer 10 (S60).

Then, as any one of the main menus is selected, a step S70 of fine-adjusting the entire outline and the outlines of the small objects again on the screen of the calibration money electronic device 50 is performed or a button of the electronic device is pressed. A step (S80) of reading basic information of small objects such as terminals is performed, a step (S90) of reading a data sheet such as an operation manual of the electronic device is performed, or a list of educational manual data of the electronic device is displayed. By applying for (S100), a list of any one of the displayed training manual data list is selected (S110), and training for the corresponding electronic device is performed by the training manual data of the selected list (S120). When the training is finished (S130), the main menu of step S60 is displayed.

On the other hand, if a step S140 of selecting another electronic device is performed, the process returns to step S30 to select another electronic device.

FIG. 6G shows an example of the augmented reality screen displaying the main menu of step S60, wherein the 'Reset Position' is a menu for selecting the step of fine adjustment of step S60, 'Front Panel' ) 'Is a menu for selecting the step of viewing the basic information of the small objects in step S70,' View Manual 'is a menu for selecting the step of viewing the data sheet in step S80,' tutorial list ' (Tutorial List) 'is a menu for selecting a step of displaying a list of educational manual data in step S90, and' Select Machine 'represents a menu for selecting a step for selecting the other electronic device. have.

6H shows an example of an augmented reality screen of a menu for selecting a manual data list for education according to step S110. In the illustrated example, one of the training data of the 'MCU Pro' and the training data of the 'MCU Fader Alignment', which are training manual data of the corresponding electronic device, are selected.

FIG. 6I shows an example of a screen in which the overall appearance of the corresponding electronic device and the appearance of small objects such as buttons or terminals are displayed as augmented reality when education for the corresponding electronic device is performed in step S120.

That is, in the state in which the user wears the wearable computer 10 on the head and executes an application program for performing the virtual education method of the electronic device of the present invention, eye tracking (a function provided by the wearable computer 10) By using the eye-tracking function and the gesture recognition function, each menu of steps S10 to S140 described above is selected to perform the virtual education method of the electronic device of the present invention.

On the other hand, in the training process of the electronic device provided to the wearable computer 10 of the present invention, whether the user has fully understood the functions of the corresponding electronic device by analyzing the results calculated through the training process in real time with standard data. It is desirable to enable self-diagnosis.

In addition, the present invention can also be implemented as an aspect of a recording medium storing a computer program in which each step of the method for providing a virtual manual of the electronic device of the present invention is stored.

* Explanation of reference numerals for main parts of drawings *
10; Wearable computer
20; communications network
30; server
40; Database
50; Electronics

Claims (15)

delete delete delete delete delete It is a method for providing a virtual manual of the electronic device 50 performed by the wearable computer 10 performing data transmission and reception through the server 30 and the communication network 20,
A step in which the server 30 transmits a list of electronic devices capable of providing training manual data from the server 30 to the wearable computer 10 through the communication network 20 (S30);
When an electronic device desired to be educated by the wearable computer 10 is selected, the wearable computer 10 performs calibration by dataizing the appearance of the electronic device 50 to be trained (calibration) S50);
A step in which a main menu for educating the corresponding electronic device is displayed on the wearable computer 10 (S60);
Selecting a list of any one item from a list of training manual data of the electronic device by the wearable computer 10 (S110);
Step of performing education on the corresponding electronic device based on augmented reality by the training manual data of the selected list (S120); It comprises,
The wearable computer 10 is a step (S50) of performing an calibration by dataizing the appearance of the electronic device 50 to be educated (S50);
Spatial Mapping is performed by a surrounding environment recognition sensor embedded in the wearable computer 10 (S51);
Setting positions of the entire outer periphery of the electronic device 50 and the outer peripheries of the small objects by recognizing the outer perimeter and the outer perimeters of the small objects by the wearable computer 10 (S52);
When the entire outer outline of the electronic device 50 and the outer positions of the small objects are set, the entire outline and the small size of the electronic device 50 using the location and size ratio information of the small objects of the electronic device 50 stored in the server 30 A method of providing a virtual manual for an electronic device, characterized in that it is performed by correcting the rotation value of the outline according to the position and size of the outline of the objects and the angle of the inclination of the image.
7. The method of providing a virtual manual for an electronic device according to claim 6, wherein the electronic device education application program transmitted from the server (30) to the wearable computer (10) is executed and education is performed.
The method of claim 6,
After the step S30, if the electronic device desired to be educated by the wearable computer 10 is selected,
The server 30 determines whether the manual data for training of the selected electronic device is the latest training manual data and updates the corresponding data (S40); A method for providing a virtual manual of an electronic device, characterized in that further comprises a.
delete The method according to claim 6, wherein after the step (S60) of displaying the main menu for educating the corresponding electronic device on the wearable computer 10,
A method of providing a virtual manual for the electronic device, characterized in that the step (S70) of fine-adjusting the entire outline and the outlines of the small objects on the screen of the calibrated electronic device 50 is performed again.
The method according to claim 6, wherein after the step (S60) of displaying the main menu for educating the corresponding electronic device on the wearable computer 10,
A method of providing a virtual manual for an electronic device, characterized in that a step (S80) of reading basic information of small objects such as buttons or terminals of the electronic device is performed.
The method according to claim 6, wherein after the step (S60) of displaying the main menu for educating the corresponding electronic device on the wearable computer 10,
A method of providing a virtual manual for an electronic device, characterized in that the step (S90) of reading the data sheet of the electronic device is performed.
It is a method for providing a virtual manual of the electronic device 50 performed by the wearable computer 10 performing data transmission and reception through the server 30 and the communication network 20,
A step in which the server 30 transmits a list of electronic devices capable of providing training manual data from the server 30 to the wearable computer 10 through the communication network 20 (S30);
When an electronic device desired to be educated by the wearable computer 10 is selected, the wearable computer 10 performs calibration by dataizing the appearance of the electronic device 50 to be trained (calibration) S50);
A step in which a main menu for educating the corresponding electronic device is displayed on the wearable computer 10 (S60);
Selecting a list of any one item from a list of training manual data of the electronic device by the wearable computer 10 (S110);
Step of performing education on the corresponding electronic device based on augmented reality by the training manual data of the selected list (S120); It comprises,
In the step (S120) of training for the corresponding electronic device based on augmented reality according to the training manual data of the selected list,
A method of providing a virtual manual for an electronic device, characterized in that the user performs a self-diagnosis on whether to fully understand the function of the corresponding electronic device by analyzing the result value calculated through the training course in real time with standard data.
The method of claim 7, wherein the wearable computer 10 is worn on the user's head,
In the state that the application program for performing the virtual manual provision method of the electronic device is executed,
The electronic device is characterized in that the menus provided by the wearable computer 10 are selected using the eye-tracking function and the gesture recognition function provided by the wearable computer 10. How to provide a virtual manual.
Claims 6 to 8, 10 to 14, a recording medium storing a computer program containing each step of the electronic device virtual manual provision method of any one of claims.

Images