JP2024071251A - Ar display work scenario creation system, and ar display work scenario creation method - Google Patents

Abstract

To solve a problem that, in a manual work, it takes time to create a work scenario when an apparatus having a complicated configuration is an object in an arrangement work or there are many working procedures.SOLUTION: An AR display work scenario creation system creates, from a work procedure manual indicating an actual objective apparatus being an object of a work, and a work to be implemented following the work scenario of AR display as a time series, a work scenario while associating three-dimensional structure data of the objective apparatus with AR display data of a virtual object.SELECTED DRAWING: Figure 7

Description

Embodiments of the present invention relate to a system for creating a work scenario displayed in augmented reality (AR) and a method for creating a work scenario displayed in AR.

Augmented reality (AR) devices have been developed that overlay and display the work content according to a work procedure manual onto the visual information of the real target equipment that is the subject of the work. In currently known systems, when creating this work scenario, the creator of the scenario manually places virtual objects to be displayed as AR at each step of the work scenario.

For example, Patent Document 1 describes an authoring method in which a user interactively operates a general AR content creation program to edit AR content.

When creating a work scenario using the above-mentioned method, the creator adjusts the position of a virtual object while viewing an image in which an AR display, which is virtual visual information, is synthesized onto the target equipment that is the subject of the work and is visible to the viewer.

The positioning of this virtual object is achieved using hand tracking technology, which places the virtual object and fingers within the camera's field of view and moves the virtual object by grasping and moving it within the field of view.

JP 2012-168798 A

In the manual placement of AR displays as described above, the precision of the position coordinates of the virtual object depends on the precision of hand tracking, so the coordinate values cannot be controlled precisely, and it is easy for discrepancies to occur between the position you want to display and the position that is actually displayed. Therefore, if there is another similar part near the part to be worked on, there is an issue that it takes a lot of time to create a work scenario that targets equipment with a complex structure or has many work steps.

To solve the above-mentioned problems, the embodiment of the AR display work scenario creation system creates a work scenario by associating three-dimensional structural data of the target equipment and AR display data of the virtual object from a work procedure manual that shows the actual equipment to be worked on and the work to be performed in chronological order according to the AR display work scenario.

FIG. 13 is a diagram illustrating how to use a work scenario of an AR display according to an embodiment. 1 is a diagram showing an overall configuration of a work scenario creation system with AR display according to an embodiment; 3 is a functional configuration diagram of a preprocessing unit of the work scenario creation system for AR display shown in FIG. 2 . FIG. 1 is a diagram showing the operation target devices of this work scenario creation system, input system data, output system data, and data obtained when using an identification name. FIG. 13 is a flowchart illustrating a pre-processing according to an embodiment. FIG. 2 is a functional configuration diagram of a work scenario creation unit according to an embodiment. FIG. 7 is a diagram showing an outline of the operation of the configuration shown in FIG. 6 . FIG. 11 is a flowchart illustrating an operation of a work scenario creating unit according to an embodiment. FIG. 11 is a diagram for explaining coordinate calculation in preprocessing according to an embodiment.

(Embodiment 1)
Before describing an embodiment in detail, how to use a work scenario for AR display will be described with reference to FIG.
FIG. 1 shows an example of how to use a work scenario to instruct a task of removing a screw with a screwdriver.

When a worker performs a task, information about the task is usually presented visually on a wearable device or mobile terminal, such as a glasses-type display device 11.

In this case, if the display device 11 is an optical see-through type, the worker will recognize the visual information of the actual target device 12 being worked on as it is in the real field of view, while the information presented to the worker by the virtual display tool, for example the virtual driver 13, will be displayed on the display device 11 superimposed on the real visual information. As a result, the real visual information and the virtual visual information are combined and displayed as an augmented reality as in display example 14, and presented as a procedure manual for the work content.

The information in the work procedure manual presents information on what to do for which part of the equipment to be worked on in each work step. In this application, a work scenario displayed in AR is a series of information that processes the information in the work procedure manual using AR representation. To indicate which part is involved, information such as an icon indicating the work content is displayed superimposed in AR at the corresponding position on the equipment in each work step. AR information indicating the work content is synthesized and displayed with real-world visual information.

The scenario creator defines what AR will be displayed in what position for each work step. Then, this series of information is compiled in chronological order to create the work scenario.

In recent years, it has become common to set the shape of equipment using 3D CAD. This application combines design data that represents the three-dimensional structure of such equipment to be worked on with work procedures, creates information to be displayed in AR as work support from the work content described in the work procedures, and uses this to create a work scenario, thereby improving the accuracy of position information and generating a scenario in a short time.

Figure 2 shows an example of the overall functional configuration of the work scenario creation system of embodiment 1. This work scenario creation system 21 is broadly composed of a pre-processing unit 22 that receives input data, creates AR display data for a virtual display tool, and creates a revised work procedure manual, and a work scenario creation unit 23 that creates a work scenario based on the revised work procedure manual and outputs output data. An actual work scenario creation system includes an input device, an output device, an arithmetic processing unit, and a storage device, but this diagram shows them in a simplified manner for ease of understanding.

(Explanation of pre-processing)
As shown in FIG. 3 , the pre-processing unit 22 comprises an equipment shape data input unit 31 to which data of three-dimensional equipment shape is input, a work procedure manual input unit 32 to which a work procedure manual is input, an AR display data input unit 33 to which AR display data of a virtual display tool is input, a feature point coordinate extraction unit 34 that extracts coordinate data of feature points, a work area selection unit 35 that selects a work area from the list of feature point coordinates, an identification name assignment unit 36 that assigns a unique identification name (part tag) to the selected work area, a work content tag description unit 37 that writes an identification name (object tag) and an identification name (part tag) in the description of the work content, a modified equipment shape data creation unit 38 that creates modified equipment shape data, and a modified work procedure manual creation unit 39 that creates a modified work procedure manual.

Figure 4 (a) shows an overview of the operation target and the data handled in preprocessing. The target device is the work target device 1, which is the target device on which the worker actually performs operations. The work target device 1 has, for example, parts B1 and B2, which are the mounting positions for screws. The names used to identify these parts, such as part B1 and part B2, are called identification names.

The equipment shape data 2 is data that describes the external shape in three dimensions, created using three-dimensional structural data such as 3D CAD when designing the work target equipment 1. 3D CAD uses formats such as the STY format. It is data that is recognized by machines, and does not include identification names used by humans to understand parts, but is structured using three-dimensional coordinates. By using a semantically structured document such as the XML-based OpenDocument format, it is easier to process information mechanically, but the description uses expressions that humans can read and understand, and the operation target is described by name rather than coordinate value. The work procedure manual 3 is a document that describes the work content for the work target equipment 1. The equipment shape data 2 and the work procedure manual 3 are data that are prepared even when the work scenario displayed in AR is not used.

Equipment shape data 2, work procedure manual 3, and AR display data 4 (e.g., a driver) are input and pre-processed, after which a work scenario is created and a work scenario 5 is output as output data.

In one embodiment of the present application, preprocessing is performed using the identification name (FIG. 4(b)). Based on the equipment shape data 2, which is coordinate data without an identification name, a certain identification name and mapping information for that coordinate data are given as metadata, and the equipment shape data 2 is changed to modified equipment shape data 2+. In the explanation of attaching and removing the screw, the work procedure manual 3 maps part B1 as metadata to the description of the target screw X and part B2 as metadata to the description of screw Y, and adds elements to the XML data. Also, in order to display a screwdriver as the tool for manipulating the screw, the driver information of the AR display data 4 is mapped and an element is added to the XML data. In this way, the work procedure manual 3 is changed to modified work procedure manual 3+.

In the corrected work procedure manual 3+, the work content (operations on the target parts) in each step is described in chronological order, for example as follows:
Step 1: Do ×× of 〇〇 Step 2: Do ▽▽ of △△ At each step of this work procedure manual 3, information to be displayed as the work content for that step is selected from the data compiled as AR display data 4, and the work content is visually presented using AR display.

The AR display data 4 is a set of virtual objects to be displayed as AR. An identification name (object tag) is assigned to each virtual object in the AR display data. For example, the following shape data is prepared as the virtual object (virtual display tool).
- Arrows (used to indicate the object of work or the direction in which a part should be moved during work)
- Numbers (used to correspond with the description of the work and the parts to be worked on, or to indicate the order of work)
- Tools (used to indicate the tools, such as a screwdriver, used in that step)
Pre-processing is performed on the equipment configuration data 2 and the work procedure manual 3 to generate corrected equipment configuration data 2+ and corrected work procedure manual 3+, which are then used to create a work scenario.

The pre-processing operation will be described with reference to the flowchart shown in FIG.
In step S500, first, equipment shape data 2 of the three-dimensional structure (3D CAD) of the work target equipment 1 and a work procedure manual 3 are input to the equipment shape data input unit 31 and the work procedure manual input unit 32 in FIG.

The equipment shape data 2 is data describing the shape of the equipment surface. The surface curve is described by an equation, or it is expressed as a set of coordinate values sampled at a certain interval from the curve. When data is input to the equipment shape data input unit 31, candidate points are selected from this data to determine where the work area is located. Points that become work areas include screw holes and movement axes, so the coordinates of such points where the shape is discontinuous are extracted as feature points in the feature point coordinate extraction unit 34. This extraction is performed first, and the list is made into a candidate list of work areas, which is stored as work data in the corrected equipment shape data creation unit 38 (step S501). The candidate list is a list of the three-dimensional coordinates of the calculated candidate points.

In the next step S502, the revised work procedure manual creation unit 39 selects one part to be referenced in the work on the target equipment. In step S503, a unique identification name (part tag) is assigned to the selected part. For example, as shown in FIG. 4B, assume that there are virtual screws at parts B1 and B2 on the operation target equipment 1. Identification names such as screw X and screw Y are assigned to parts B1 and B2, respectively. In that case, they would be screw X (B1) and screw Y (B2). The unique identification name (part tag) is also associated with the equipment shape data 2 and the work procedure manual 3.
To match the equipment shape data, the equipment is modeled to its actual shape from the equipment shape data and displayed on the screen of the creation system. At that time, markers indicating candidate points are displayed at each coordinate position in the candidate list, and the creator selects the screw position that is the target of the work from among the displayed markers by controlling the creation system.

In step S503, the identification name assignment unit 36 assigns an identification name to the part tag that is unique to the part selected in step S502. Such an assignment of an identification name to a part tag is referred to as an identification name (part tag). The revised work procedure manual creation unit 39 uses the identification name to convert the description of work procedure manual 3 into work procedure manual 3+.

The creator of the work scenario searches for which feature point in the candidate list corresponds to the part of each identification name (part tag) (step S504). The modified work procedure manual creation unit 39 presents the candidates to the creator on the screen of the creation system, and the creator selects one to determine it. Then, in step S505, the modified equipment shape data creation unit 38 associates the coordinates of the feature point found in step S504 with the identification name (part tag) and updates the data.
In the next step S506, a search is made for a portion in the description of the work content in the work procedure manual 3 that refers to the relevant portion.

In the next step S507, the identification name (hereinafter referred to as the identification name (object tag)) assigned to the object tag to be displayed in AR and the identification name (body part tag) are specified in the work content tag description section 37. That is, in the work procedure manual 3, the information to be displayed in AR for the work content is also specified in this step S507.

For example, the description is "AR: Remove 'Screw X' with a 'Screwdriver'." 'Screw X' is the identification name (body part tag) described above (step S506), and is marked up as the body part to be displayed in AR. The virtual object to be displayed in AR is a 'Screwdriver,' and is the identification name (object tag) selected from the AR display data 4.

In step S508, it is checked whether there is a next part to be worked on. If there is still a part to be worked on, the process returns to step S502 and the above series of processes are repeated.
In step S508, if there are no remaining parts to be worked on, the process proceeds to the next step S509, in which the corrected device shape data creation unit 38 adds a coordinate list of the identification name (part tag) to create corrected device shape data 2+.

Then, in the next step S510, a revised work procedure manual 3+ is created based on the description of the work content updated by the revised work procedure manual creation unit 39. In this way, a revised work procedure manual 3+ is created that clarifies the work target by using the identification name.

(Explanation of Work Scenario Generation Process)
6, the work scenario creation unit 23 that executes the work scenario generation process includes a modified equipment shape data input unit 61 to which modified equipment shape data is input, a modified work procedure manual input unit 62 to which a modified work procedure manual is input, an AR display data input unit 63 to which AR display data is input, a work target part identification unit 64 to read out the work contents step by step from the modified work procedure manual and identify the part to be worked on, a work target part search unit 65 to search for the work target part from the modified equipment shape data, a part position coordinate orientation calculation unit 66 to calculate the position coordinates and orientation of the work part, an AR display virtual object creation unit 67 to create a virtual object for AR display, an object coordinate position orientation setting unit 68 to set the coordinate position and orientation of the virtual object, and a work scenario output unit 69 to output the created work scenario. The AR display data is input to the AR display data input unit 63 and used in the AR display virtual object creation unit 67.

As shown in Figure 7, the work scenario generation process involves reading the description of the work content in the revised work procedure manual 3+, step by step (for example, remove "screw X" with "driver Z") (71), calculating the position coordinates and orientation of the part on the surface of the equipment to be worked on (72), determining the display coordinates and orientation of the virtual object (driver) according to the position coordinates and orientation (73), and creating an AR display virtual object 7. Note that in Figure 7, 75 is a marker for the origin of the coordinates.

Next, the operation of the work scenario creation unit 23 shown in FIG. 6 will be described using the flowchart shown in FIG. 8. First, in step S801, the corrected equipment shape data 2+ and the corrected work procedure manual 3+ are input to the corrected equipment shape data input unit 61 and the corrected work procedure manual input unit 62, respectively. In the next step S802, the AR display data 4 is input to the AR display data input unit 63.

In step S803, an explanation of the work content for one step is read from the revised work procedure manual 3+. The procedure for performing this process is referred to as step N. From the explanation that has been read, the identification name associated with the part to be worked on (identification name (part tag)) and the identification name of the AR display data (identification name (object tag)) are recognized.

For example, in the case of a description that reads "Remove 'Screw X' with 'Driver Z'," in step S804, the work target part identification unit 64 recognizes "Screw X" as the identification name of the work target part. In step S807, the work target part identification unit 64 recognizes "Driver Z" as the identification name (object tag) associated with the AR display data.

In step S805, the work target part search unit 65 searches for an identification name (part tag) from the corrected equipment shape data 2+ from the corrected equipment shape data input unit 61. In step S806, the part position coordinate orientation calculation unit 66 calculates the coordinates and orientation of "screw X", the part found in the above search.

In step S807, "Driver Z" is recognized as the identification name (object tag) as described above. In step S808, a display method for indicating task content Y is searched for, and display object Z is obtained. That is, the identification name (object tag) is searched for in the AR display data 4 from the AR display data input unit 63. In step N, the virtual object found by the search, "Driver Z," is used as a virtual object to be displayed in AR, and data for virtual object 7 to be displayed in AR is created by the AR display virtual object creation unit 67.

Even if the same "Driver Z" is used in different steps, the position where "Driver Z" is displayed will be different when working in each step, so a virtual object is prepared to be displayed in AR as display information for each step.

In step S809, the object coordinate position and orientation setting unit 68 sets the coordinate position and orientation where object Z was acquired in step N of the work scenario. That is, the orientation of "screw X" calculated in the previous step is applied as the orientation of "driver Z" to rotate the axis of the AR display virtual object. Then, the coordinate position of the AR display virtual object 7 is moved so that the position of the head of "driver Z" hits the position of "screw X". Since processing is performed in this manner, the head position is set as a reference when the data for "driver Z" is prepared.

The AR display virtual object 7 is stored as data for step N of work scenario 5 (step S810).

In step S811, check whether there is a next step in the revised work procedure manual 3+. If there are still steps, return to step S803. If the revised work procedure manual 3+ has reached the final step and there are no more steps to proceed to, proceed to step S812.

Then, the work scenario output unit 69 outputs and saves the data set of work scenario 5, and ends the work scenario generation process in the work scenario creation unit 23. The generated work scenario is saved and used when necessary.

In addition, in step S808 above, when selecting and placing a virtual object to be displayed in AR based on the work content, if it is a screw, the part in the structural design and the position to be displayed in AR will be almost the same place.

On the other hand, the part that the worker operates to open and close is a part such as a handle, but in the structural design of the opening and closing mechanism, a part such as a hinge that acts as the axis of rotation may be more characteristic than the handle. For this reason, when extracting feature points in step S501 of the preprocessing flowchart in Figure 5, it is a good idea to have the modified equipment shape data 3+ contain information that associates parts, such as the structure that acts as the axis of rotation and the handle that operates it. One piece of data includes data on the mechanical parts that achieve opening and closing, data on the parts where the worker performs opening and closing operations, shape data for the open state, shape data for the closed state, and data on whether the normal state is open or closed.

The work procedure manual also describes the parts where opening and closing operations are performed, and the operation of "opening" or "closing." In the system of this embodiment, if the work is to "open," an arrow indicating the direction to the open state is placed as a virtual object displayed in AR on the work part that corresponds to the handle in the closed state. Also, if the work is to "close," an arrow indicating the direction to the closed state is placed as a virtual object displayed in AR on the work part that corresponds to the handle in the open state.

(Coordinate calculation)
Here, the coordinate calculation in the preprocessing will be described with reference to FIG.

The device shape data 2 is expressed as a set of three-dimensional coordinates (x, y, z) relating to the surface position of the device. The device shape is expressed by setting points on the device surface in a mesh pattern and giving coordinates to those points. Even if the surface is curved, two points can be approximated by a straight line if the number of points given coordinates is increased sufficiently and the intervals between each point are made small enough.

Suppose there is a screw hole at a certain point on the surface of a device. Suppose the coordinates of the four points surrounding the screw hole are (xm n, ym n, zm n), (xm n+1, ym n+1, zm n+1), (xm+1 n, ym+1 n, zm+1 n), and (xm+1 n+1, ym+1 n+1, zm+1 n+1). Since the points can be approximated by straight lines, a plane can be approximated from these four points.
Given three points, a plane that passes through those three points is determined. Since no plane passes through any four points, we find the plane ax+by+cz+d=0 where the sum of the distances between the plane and each point is the smallest. Here, the vector (a,b,c) is the normal vector that intersects the plane perpendicularly.

Also, prepare a plan view in which these points are projected onto the plane where z=0. The diagrams referenced in traditional media such as work instructions use such plan views projected into two dimensions. If the coordinates of the screw hole projected onto the plane are (xs, ys), then substitute this into the equation for the plane to calculate zs and obtain the position coordinates (xs, ys, zs) of the screw hole on the surface of the equipment.

The screw pointing to this screw hole has vector (a, b, c) and end coordinates (xs, ys, zs). If the length of the screw and screwdriver to be displayed is L, then the coordinates (xs, ys, zs) on vector (a, b, c) that satisfy (xs-xt) ² + (ys-yt) ² + (zs-st) ² = ^L2 become the start coordinates of the screwdriver. The AR display positions of the virtual objects of the screw and screwdriver are set as a line segment from the start coordinates (xt, yt, zt) to the end coordinates (xs, ys, zs).

The above coordinate values are based on the coordinate system of the 3D design data. When displaying as AR, it is necessary to associate the coordinate system of the design data with the coordinate system of the real space and calculate the coordinate values of the real space from the coordinate values of the design data. To achieve this, a common method is to set a coordinate point that is common to the design data and real space, and an anchor that serves as a reference point in the coordinate axis direction.

A characteristic position on the appearance of the target device is selected in advance, and its coordinates (xa, ya, za) in the design data are obtained. An anchor that can be recognized as a landmark by the AR display device is placed at the selected position. The AR display device is equipped with a camera and a sensor, which allows it to recognize the anchor position. The recognized anchor position is then set as the coordinate origin of the real space.
The information to be displayed in AR at the screw hole coordinates (xs, ys, zs) in the design data becomes a virtual object displayed at the coordinates (xs-xa, ys-ya, zs-za) in real space.

(Effects of the embodiment)
According to the above embodiment, a work scenario for AR display can be automatically created by combining a text-described work procedure manual with three-dimensional structural design data of the equipment to be worked on as shape data. Although there is processing such as referencing the target part using the work procedure manual and the equipment shape data, this processing is mechanical work and does not require work proficiency, and has the advantage that the processing can be completed in a relatively fixed short time.

In addition, according to this embodiment, the structural design data of the equipment to be worked on is used as equipment shape data, and the coordinates and orientation of the virtual object displayed to the worker in AR during the work procedure can be calculated and set. This has the advantage that there is no error caused by fine alignment, as occurs in the conventional case of manual placement.

In addition, this embodiment allows the operator to correctly identify the area to be operated, which has the effect of reducing operational errors such as misidentifying the target area, which may occur when performing manual work.

Furthermore, when using a wearable device for stereoscopic vision, there are physical differences between people, such as the distance between their eyes, which can cause variation in the positional relationship between real objects and virtual objects displayed in AR. For this reason, device users typically perform calibration in advance to correct how they see. In conventional scenario creation methods, if the creator who places the virtual objects does not calibrate properly, this can become a cause of error in the AR display. According to this embodiment of the present application, object placement is performed by calculation, so the above-mentioned causes of error can be eliminated.

In the above embodiment, for ease of understanding, the functional configuration diagrams of Figures 2, 3, and 6 have been used for explanation. However, the present invention is not limited to such divided configurations, and is generally applicable to cases where the above-mentioned functions are achieved as a whole by using shared circuits such as a common control device, memory, and arithmetic device in a time-division manner.

Other Embodiments
In addition, in the present invention, it is also possible to create a work scenario without relying on conventional equipment shape data and work procedure manuals.

Although several embodiments have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, combinations, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims.

Description of Reference Signs 11: Display device, 12: Target equipment, 13: Virtual driver, 21: Work scenario creation system, 22: Preprocessing unit, 23: Work scenario creation unit, 31: Equipment shape data input unit, 32: Work procedure manual input unit, 33: AR display data input unit, 34: Feature point coordinate extraction unit, 35: Work area selection unit, 36: Identification name assignment unit, 37: Work content tag description unit, 38: Corrected equipment shape data creation unit, 39: Corrected work procedure manual creation unit, 61: Corrected equipment shape data input unit, 62: Corrected work procedure manual input unit, 63: AR display data input unit, 64: Work target area identification unit, 65: Work target area search unit, 66: Area position coordinate orientation calculation unit, 67: AR display virtual object creation unit, 68: Object coordinate position orientation setting unit, 69: Work scenario output unit

In order to solve the above-mentioned problems, a work scenario creation system for AR display according to an embodiment of the present invention includes:
Based on a work procedure manual that shows the work to be performed on the actual target equipment as a chronological sequence of work steps and three-dimensional structural data of the target equipment, the spatial information obtained from the three-dimensional structural data is used to convert into a work scenario, which is a work procedure manual to which AR display data of a virtual display tool for the target equipment is added .

Claims (7)

From a work procedure manual that shows the actual target equipment to be worked on and the work to be performed in chronological order according to the work scenario displayed in AR,
A work scenario creation system for AR display that creates a work scenario by associating three-dimensional structural data of the target device and AR display data of a virtual object. The AR display work scenario creation system of claim 1 creates the work scenario by associating an identification name assigned to a part tag of an operating part of the target device with an identification name assigned to an object tag of the virtual object. a pre-processing unit that creates a revised work procedure manual by using three-dimensional structure data as equipment shape data of a real target device that is the subject of the work and a work procedure manual that shows a time series of work to be performed according to a work scenario displayed in AR; and
a part position and orientation calculation unit that calculates a part and orientation on a surface of the target device; and a work scenario creation unit that determines a display coordinate and orientation of the virtual object in accordance with a position coordinate and a location of the virtual object to create an AR display virtual display object and create an AR display work scenario;
A work scenario creation system for AR display having the above structure. The pre-treatment unit includes:
a device shape data input unit for inputting three-dimensional device shape data of the target device;
a work procedure input unit for inputting a work procedure showing, in time sequence, the work to be performed according to the work scenario displayed in AR;
an AR display data input unit for inputting AR display data;
a modified device shape data creating unit that creates modified device shape data;
A modified work procedure manual creation unit that creates a modified work procedure manual from the work procedure manual;
4. The system for creating a work scenario with AR display according to claim 3, further comprising: The work scenario creation unit includes:
a modified device shape data input unit to which modified device shape data is input;
a modified work procedure manual input unit for inputting a modified work procedure manual;
an AR display data input unit to which AR display data is input;
a part position coordinate orientation calculation unit that calculates the position coordinates and orientation of a work part;
an AR display virtual object creation unit that creates a virtual object for AR display;
4. The system for creating a work scenario with AR display according to claim 3, comprising: From a work procedure manual that shows the actual target equipment to be worked on and the work to be performed in chronological order according to the work scenario displayed in AR,
A work scenario creation method for AR display, which creates a work scenario by associating the three-dimensional structural data of the target device with the AR display data of the virtual display tool. The method for creating a work scenario for AR display according to claim 6, wherein an identification name associated with a part tag of an operation part of the target device is associated with an identification name assigned to an object tag of the virtual object to create a work scenario.

Images