JP7167654B2 - Information processing device, head mounted display, computer program and image display system - Google Patents

Description

The present invention relates to an information processing device, a head-mounted display, a computer program, and an image display system.

In recent years, techniques for assisting workers using mixed reality techniques have been known (see Patent Literature 1, for example). In the prior art described in Patent Document 1, a three-dimensional model of a virtual world (hereinafter referred to as virtual space) generated by a computer is superimposed on a metal workpiece in the real world (hereinafter referred to as "real space"). By displaying it on a head-mounted display, the worker is assisted in the work using the three-dimensional model.

JP 2017-044496

However, in the above-described prior art, if the alignment between the metal workpiece in the real space and the three-dimensional model in the virtual space is insufficient, it may be difficult for the operator to work.

The present invention has been made in consideration of such circumstances, and its object is to improve the accuracy of alignment between an object in the real space and a three-dimensional model in the virtual space. It is to provide a head mounted display, a computer program and an image display system.

One aspect of the present invention is an alignment unit that aligns a contour of an object in a real space detected by a 3D sensor provided in a head-mounted display and a contour of a 3D model in a virtual space, and based on the alignment, a superimposed image data generating unit for generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space; and a transmitting unit for transmitting the first superimposed image data to the head-mounted display. aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object to determine the outline of the object in the real space and the outline of the virtual space; a registration correcting unit that corrects the registration with the contour of the three-dimensional model, and the superimposed image data generating unit corrects the registration on the object in the physical space based on the correction result of the registration. The information processing apparatus generates second superimposed image data in which dimensional models are displayed in an overlapping manner, and the transmission unit transmits the second superimposed image data to the head mounted display.

According to one aspect of the present invention, a three-dimensional sensor, an alignment unit that aligns a contour of an object in a real space detected by the three-dimensional sensor, and a contour of a three-dimensional model in a virtual space, and based on the alignment, a superimposed image data generating unit for generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space; a reference position on the object detected by the three-dimensional sensor; Alignment correction for aligning a reference position on a dimensional model and correcting the alignment between the contour of the object in real space and the contour of the 3D model in virtual space using the outer dimension data of the object. wherein the superimposed image data generating unit generates second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result; A head-mounted display that displays the first superimposed image data and the second superimposed image data.

According to one aspect of the present invention, a computer is provided with an alignment step of aligning an outline of an object in a real space detected by a 3D sensor provided in a head-mounted display with an outline of a 3D model in a virtual space; a first superimposed image data generating step of generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on; and transmitting the first superimposed image data to the head-mounted display. a first transmission step of transmitting; aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model; an alignment correction step of correcting the alignment between the contour of the object and the contour of the three-dimensional model in the virtual space; and superimposing the three-dimensional model on the object in the real space based on the alignment correction result and a second transmission step of transmitting the second superimposed image data to the head-mounted display. .

According to one aspect of the present invention, a computer is provided with an alignment step of aligning an outline of an object in a real space detected by a 3D sensor included in a head-mounted display with an outline of a 3D model in a virtual space; a first superimposed image data generating step of generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on; and transmitting the first superimposed image data to the head-mounted display. a first display step of displaying; aligning a reference position on the object detected by the three-dimensional sensor with a reference position on the three-dimensional model; an alignment correction step of correcting the alignment between the contour of the object and the contour of the three-dimensional model in the virtual space; and superimposing the three-dimensional model on the object in the real space based on the alignment correction result and a second display step of displaying the second superimposed image data on the head-mounted display. .

One aspect of the present invention includes a head-mounted display provided with a three-dimensional sensor, and an information processing device, wherein the information processing device detects the contour of an object in real space detected by the three-dimensional sensor and the three-dimensional image in virtual space. A registration unit that aligns the outline of the model, and a superimposed image data generation unit that generates first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the registration. a transmission unit that transmits the first superimposed image data to the head mounted display; and a reference position on the object detected by the three-dimensional sensor and a reference position on the three-dimensional model are aligned, an alignment correction unit that corrects the alignment between the contour of the object in the real space and the contour of the three-dimensional model in the virtual space using the external dimension data of the superimposed image data generation unit, generating second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the result of the alignment correction, and transmitting the second superimposed image data to the head It is an image display system that transmits to the mount display.

According to the present invention, it is possible to improve the accuracy of alignment between an object in the real space and a three-dimensional model in the virtual space.

1 is a block diagram showing a configuration example of an image display system according to a first embodiment; FIG. It is a figure which shows the structural example of an external dimension data storage part. It is a figure which shows the structural example of a device arrangement|positioning data storage part. It is a top view which shows the example of apparatus arrangement|positioning of a work room. 4 is a flow chart showing an example of the procedure of an image display method according to the first embodiment; 2 is a block diagram showing another configuration example of the image display system according to the first embodiment; FIG. It is a block diagram which shows the structural example of the head mounted display which concerns on 2nd Embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing a configuration example of an image display system according to the first embodiment. In FIG. 1 , the image display system 1 includes a server device (information processing device) 10 , a terminal device 40 and a head mounted display 50 . The server device 10 and the terminal device 40 are provided with communication units 11 and 41, respectively, and communicate with each other.

The server device 10 includes a server communication unit 11, an alignment unit 12, a superimposed image data generation unit 13, an alignment correction unit 14, an external dimension data storage unit 15, and an equipment layout data storage unit 16. . The terminal device 40 includes a terminal communication section 41 , a terminal control section 42 and a display interface 43 . The head mounted display 50 has a three-dimensional sensor 51 .

The head mounted display 50 is a display device worn on the head of the operator. The terminal device 40 relays data between the server device 10 and the head mounted display 50, and the like. As the terminal device 40, for example, a mobile communication terminal device such as a smart phone may be used.

In the terminal device 40 , the terminal communication section 41 communicates with the server communication section 11 of the server device 10 . The terminal control unit 42 controls the terminal device 40 . The display interface 43 communicates with the head mounted display 50 . The display interface 43 and the head mounted display 50 may use, for example, a short-range wireless communication system as a communication system.

The head mounted display 50 displays an image so that it can be seen by the worker wearing the head mounted display 50 on his head. The three-dimensional sensor 51 detects objects in the physical space. An infrared camera, for example, may be used as the three-dimensional sensor 51 .

In the server device 10 , the server communication section 11 communicates with the terminal communication section 41 of the terminal device 40 . The alignment unit 12 aligns the contour of the object in the real space detected by the three-dimensional sensor 51 with the contour of the three-dimensional model in the virtual space. The superimposed image data generation unit 13 generates first superimposed image data in which the three-dimensional model in the virtual space is displayed superimposed on the object in the real space based on the alignment performed by the alignment unit 12 . The first superimposed image data is transmitted to the head mounted display 50 by the server communication section 11 (transmitting section) and displayed on the head mounted display 50 .

The alignment correction unit 14 aligns the reference position on the object detected by the three-dimensional sensor 51 with the reference position on the three-dimensional model, and uses the outer dimension data of the object to obtain the contour of the object in the real space. and the contour of the three-dimensional model in virtual space. The superimposed image data generation unit 13 generates second superimposed image data in which the three-dimensional model in the virtual space is displayed superimposed on the object in the real space based on the alignment correction result of the alignment correction unit 14 . The second superimposed image data is transmitted to the head mounted display 50 by the server communication section 11 (transmitting section) and displayed on the head mounted display 50 .

The outer dimension data storage unit 15 stores the outer dimension data of objects. FIG. 2 is a diagram showing a configuration example of the external dimension data storage unit 15. As shown in FIG. In the example of FIG. 2, the object is equipment provided in the work room where the workers work. As shown in FIG. 2, the outer dimension data storage unit 15 stores outer dimension data indicating outer dimensions of the device in association with a device type identifier (device type ID) indicating the type of the device. The external dimension data of a device is data indicating the horizontal (width), vertical (height), and depth dimensions of the device.

The equipment layout data storage unit 16 stores equipment layout data. FIG. 3 is a diagram showing a configuration example of the device layout data storage unit 16. As shown in FIG. As shown in FIG. 3, the device placement data storage unit 16 stores device type IDs of devices installed at the placement location in association with the placement location. In the example of FIG. 3, the device type ID corresponds to FIG.

The example of equipment layout data in the equipment layout data storage unit 16 shown in FIG. 3 corresponds to the equipment layout of the work room shown in FIG. FIG. 4 is a plan view showing an example of arrangement of equipment in a working room. In the working room shown in FIG. 4, the placement locations Pa1 to Pa6 are arranged in order in a line. Devices having respective device type IDs indicated by the device placement data in FIG. In the arrangement of arrangement locations Pa1 to Pa6, the device (Eid1) installed at one end arrangement location Pa1 is provided with a reference position Qa1 on the outer end of the arrangement. On the other hand, the device (Eid2) installed at the placement location Pa6 is provided with a reference position Qa2 on the outer end of the array.

In the working room shown in FIG. 4, the placement locations Pb1 to Pb6 are arranged in order in a line. Devices having respective device type IDs indicated by the device placement data in FIG. In the arrangement of arrangement locations Pb1 to Pb6, the device (Eid1) installed at one end arrangement location Pb1 is provided with a reference position Qb1 on the outer end of the arrangement. On the other hand, the device (Eid2) installed at the placement location Pb6 is provided with a reference position Qb2 on the outer end of the arrangement.

Next, with reference to FIG. 5, the operation of the image display system 1 according to this embodiment will be described. FIG. 5 is a flow chart showing an example of the procedure of the image display method according to this embodiment. 2, 3 and 4 will be used for explanation.

(Step S1) In the work room shown in FIG. 4, the worker wears the head-mounted display 50 on the head. Also, the operator carries the terminal device 40 or leaves the terminal device 40 in a place where the display interface 43 of the terminal device 40 and the head mounted display 50 can communicate with each other.

The worker uses the head-mounted display 50 to scan the real space of the work place. By scanning the real space of the work place, the three-dimensional sensor 51 of the head mounted display 50 detects each device installed at each arrangement place Pa1-Pa6, Pb1-Pb6, and the three-dimensional data of the detection result (hereinafter referred to as the real space space detection three-dimensional data) is output. The physical space detection three-dimensional data includes information representing the three-dimensional outline of each device in the physical space detected by the three-dimensional sensor 51 .

The physical space detection three-dimensional data is transmitted from the head-mounted display 50 to the display interface 43 of the terminal device 40 . The terminal control unit 42 of the terminal device 40 transmits the real space detection three-dimensional data received by the display interface 43 from the head mounted display 50 to the server device 10 via the terminal communication unit 41 . The server device 10 receives the real space detection three-dimensional data from the head mounted display 50 through the server communication unit 11 .

(Step S2) The alignment unit 12 of the server device 10 uses the real space detection three-dimensional data received from the head-mounted display 50 to determine the positions Pa1 to Pa6 and Pb1 to Pb6 in the real space of the work place. Recognize the 3D outline of the device. The alignment unit 12 aligns the recognized three-dimensional contour of each device with the contour of the three-dimensional model of the virtual space corresponding to each device. A three-dimensional model of the virtual space is set in the server device 10 in advance. For example, a three-dimensional polyhedron may be used as the three-dimensional model of the virtual space.

(Step S3) The superimposed image data generation unit 13 of the server device 10 superimposes the three-dimensional model of the virtual space corresponding to each device on each device in the real space based on the alignment by the alignment unit 12 in step S2. to generate the first superimposed image data to be displayed.

The server device 10 transmits the first superimposed image data to the terminal device 40 through the server communication section 11 . The terminal device 40 receives the first superimposed image data from the server device 10 through the terminal communication section 41 . The terminal control unit 42 of the terminal device 40 transmits the first superimposed image data received from the server device 10 to the head mounted display 50 via the display interface 43 .

The head mounted display 50 displays the first superimposed image data received from the terminal device 40 . As a result, the worker can see the three-dimensional model of the virtual space corresponding to each device in the real space superimposed on each device in the real space by the head-mounted display 50 at the work place in the work room shown in FIG. visually. However, in this state, there is a possibility that the contour of each device in the real space and the contour of the three-dimensional model in the virtual space corresponding to each device are not aligned sufficiently. If the alignment between the contour of the device in the real space and the contour of the 3D model in the virtual space corresponding to the device is insufficient, for the operator, the contour of the device in the real space and the virtual space corresponding to the device Since the outline of the 3D model looks out of alignment, it may be difficult to work. For this reason, in the present embodiment, alignment correction between the contour of the device in the physical space and the contour of the three-dimensional model corresponding to the device is performed by the following procedure.

(Step S4) The worker performs an operation of designating the reference positions Qa1 and Qa2 in the physical space at the work place in the work room shown in FIG. As an operation for designating the reference positions Qa1 and Qa2 in the physical space, for example, markers are affixed to the reference positions Qa1 and Qa2 in advance, and the operator scans the markers of the reference positions Qa1 and Qa2 with the head-mounted display 50 . Alternatively, the head-mounted display 50 scans the state in which the operator touches each of the reference positions Qa1 and Qa2 with his or her fingertips. The reference positions Qb1 and Qb2 in the physical space are also designated in the same manner as the reference positions Qa1 and Qa2.

By the operation of designating the reference positions Qa1 and Qa2 in the physical space, the three-dimensional sensor 51 of the head-mounted display 50 detects the respective reference positions Qa1 and Qa2, and generates the three-dimensional data of the detection result (hereinafter referred to as the three-dimensional data of the detected reference position). ) is output. The reference position detection three-dimensional data includes information indicating each of the reference positions Qa1 and Qa2 in the physical space detected by the three-dimensional sensor 51 . Similarly to the reference positions Qa1 and Qa2, the reference position detection three-dimensional data is output for the reference positions Qb1 and Qb2 in the physical space.

The reference position detection three-dimensional data is transmitted from the head mounted display 50 to the server device 10 via the terminal device 40 in the same manner as the physical space detection three-dimensional data described above. The server device 10 receives the reference position detection three-dimensional data from the head mounted display 50 through the server communication unit 11 .

(Step S5) The alignment correction unit 14 of the server device 10 recognizes the reference positions Qa1 and Qa2 in the physical space of the work place based on the reference position detection three-dimensional data received from the head mounted display 50 . The alignment correction unit 14 aligns the recognized reference positions Qa1 and Qa2 in the real space with the reference positions on the three-dimensional model in the virtual space, and determines the outer shape of each device installed at each placement location Pa1-Pa6. Using the dimension data, the alignment of the outline of each device installed at each placement location Pa1-Pa6 in the real space and the outline of each three-dimensional model in the virtual space is corrected. Similarly, for each device installed at each placement location Pb1-Pb6, the contour of each device in the real space and each three-dimensional model in the virtual space are based on the reference position detection three-dimensional data of the reference positions Qb1 and Qb2. Correct alignment with contours. The alignment correction method will be described below.

The reference position Qa1 in the physical space is provided in the device (Eid1) installed at the placement location Pa1. The reference position Qa2 in the physical space is provided in the device (Eid2) installed at the placement location Pa6. The alignment correction unit 14 aligns the recognized reference position Qa1 in the physical space with the reference position on the three-dimensional model in the virtual space corresponding to the device (Eid1) installed at the placement location Pa1. At the same time, the recognized reference position Qa2 in the physical space is aligned with the reference position on the three-dimensional model in the virtual space corresponding to the device (Eid2) installed at the placement location Pa6. Next, the alignment correction unit 14 corrects the placement locations Pa1, Pa2, Pa3, Pa4, Pa5, and Pa6 based on the data stored in the external dimension data storage unit 15 shown in FIG. 2 and the device placement data storage unit 16 shown in FIG. Acquire the arrangement order and external dimensions of each device installed in the Next, the alignment correction unit 14 calculates the distances from the reference positions Qa1 and Qa2 to each device existing between the reference position Qa1 and the reference position Qa2 (referred to as a first distance) according to the arrangement order of the devices. Calculate using the external dimensions of each device. Next, the alignment correction unit 14 sets the distance between the reference position Qa1 and the reference position Qa2 (referred to as a second distance) to 1, and calculates the position of each device as a ratio of the first distance to the second distance. Next, the alignment correction unit 14 corrects the contour of each three-dimensional model in the virtual space corresponding to each device based on the ratio of the first distances and the outer dimensions of each device. As a result, the contours of the three-dimensional models in the virtual space corresponding to the devices installed at the placement locations Pa1, Pa2, Pa3, Pa4, Pa5, and Pa6 accurately match the contours of the corresponding devices in the real space. Aligned.

The reference position Qb1 in the physical space is provided in the device (Eid1) installed at the placement location Pb1. The reference position Qb2 in the physical space is provided in the device (Eid2) installed at the placement location Pb6. The alignment correction unit 14 aligns the recognized reference position Qb1 in the physical space with the reference position on the three-dimensional model in the virtual space corresponding to the device (Eid1) installed at the placement location Pb1. At the same time, the recognized reference position Qb2 in the physical space is aligned with the reference position on the three-dimensional model in the virtual space corresponding to the device (Eid2) installed at the placement location Pb6. Next, the alignment correction unit 14 corrects the placement locations Pb1, Pb2, Pb3, Pb4, Pb5, and Pb6 based on the data stored in the external dimension data storage unit 15 shown in FIG. 2 and the device placement data storage unit 16 shown in FIG. Acquire the arrangement order and external dimensions of each device installed in the Next, the alignment correction unit 14 calculates the distances (referred to as third distances) from each of the reference positions Qb1 and Qb2 to each device existing between the reference position Qb1 and the reference position Qb2 according to the arrangement order of each device. Calculate using the external dimensions of each device. Next, the alignment correction unit 14 sets the distance between the reference position Qb1 and the reference position Qb2 (referred to as a fourth distance) to 1, and calculates the position of each device as a ratio of the third distance to the fourth distance. Next, the alignment correction unit 14 corrects the contour of each three-dimensional model in the virtual space corresponding to each device based on the ratio of the third distances and the outer dimensions of each device. As a result, the contours of the three-dimensional models in the virtual space corresponding to the devices installed at the placement locations Pb1, Pb2, Pb3, Pb4, Pb5, and Pb6 accurately match the contours of the corresponding devices in the real space. Aligned.

(Step S6) The superimposed image data generating unit 13 of the server device 10 creates virtual space 3 images corresponding to each device on each device in the real space based on the alignment correction result by the alignment correction unit 14 in step S5. Generate second superimposed image data in which the dimensional model is superimposed and displayed.

The second superimposed image data is transmitted from the server device 10 to the head mounted display 50 via the terminal device 40 in the same manner as the first superimposed image data described above.

The head mounted display 50 displays the second superimposed image data received from the terminal device 40 . As a result, the worker can see a three-dimensional model in the virtual space corresponding to each device in the real space superimposed on each device in the real space on the head-mounted display 50 at the work place in the work room shown in FIG. visually. In this state, the contour of each device in the real space and the contour of the three-dimensional model corresponding to each device are aligned with high accuracy. As a result, high-quality work support using the three-dimensional model of the virtual space can be provided to the worker. For example, when a model of work operation is shown to the worker using a three-dimensional model of virtual space, it can be shown to the worker as if he or she were operating a real work target device in the real space. In addition, for example, when a work procedure manual is shown to a worker using a three-dimensional model of a virtual space, the work procedure manual of the three-dimensional model of the virtual space can be placed in an appropriate position with respect to the position of the work target equipment in the real space. can be displayed well.

[Another configuration example of the image display system]
FIG. 6 is a block diagram showing another configuration example of the image display system according to this embodiment. In FIG. 6, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and descriptions thereof are omitted. In the image display system 1a shown in FIG. 6, the head mounted display 50a may include the display communication unit 52 and the terminal device 40 may not be provided. The display communication unit 52 communicates with the server communication unit 11 of the server device 10 . The head mounted display 50 communicates with the server device 10 through the display communication section 52 . The image display system 1a is the same as the image display system 1 shown in FIG.

As described above, according to this embodiment, it is possible to improve the accuracy of alignment between an object in the real space and a three-dimensional model in the virtual space. As a result, by superimposing the three-dimensional model of the virtual space on the object in the real space and displaying it on the head-mounted display, it is possible to provide high-quality work support to the worker using the three-dimensional model of the virtual space. can be done.

[Second embodiment]
FIG. 7 is a block diagram showing a configuration example of the head mounted display 60 according to the second embodiment. In FIG. 7, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. In the second embodiment, the alignment unit 12, the superimposed image data generation unit 13, the alignment correction unit 14, the outer dimension data storage unit 15, and the device layout data storage unit 16 of the server device 10 of the first embodiment are combined into a head-mounted display. 60, the head-mounted display alone is implemented. Other than this point, it is the same as the first embodiment.

Note that the head mounted display 60 may further include a display communication unit that communicates with the outside. As a result, when the work room to be worked on changes, according to the work room to be worked on, the relevant outer dimension data, equipment arrangement data, etc. are acquired from an external device by communication, and the outer dimension data storage unit 15 and the equipment are stored. Data stored in the arrangement data storage unit 16 or the like can be updated as appropriate.

Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are included within the scope of the present invention.

Alternatively, a computer program for realizing the functions of the devices described above may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by the computer system. Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices.
In addition, "computer-readable recording medium" includes writable nonvolatile memories such as flexible discs, magneto-optical discs, ROMs and flash memories, portable media such as DVDs (Digital Versatile Discs), and computer system built-in media. A storage device such as a hard disk that

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic Random Access Memory)), which holds the program for a certain period of time, is also included.
Further, the above program may be transmitted from a computer system storing this program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in a transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1... Image display system 10... Server apparatus 11... Server communication part 12... Alignment part 13... Superimposed image data generation part 14... Alignment correction part 15... External dimension data storage part 16... Equipment arrangement Data storage unit 40 Terminal device 41 Terminal communication unit 42 Terminal control unit 43 Display interface 50, 50a, 60 Head mounted display 51 Three-dimensional sensor 52 Display communication unit

Claims (5)

an alignment unit that aligns the contour of the object in the real space detected by the 3D sensor provided in the head-mounted display with the contour of the 3D model in the virtual space;
a superimposed image data generation unit that generates first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment;
a transmission unit that transmits the first superimposed image data to the head mounted display;
Aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object, the outline of the object in the real space and the outline of the object in the virtual space. an alignment correction unit that corrects the alignment with the contour of the three-dimensional model;
The superimposed image data generation unit generates second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result,
The transmission unit transmits the second superimposed image data to the head mounted display.
Information processing equipment. a three-dimensional sensor;
an alignment unit that aligns the contour of the object in the real space detected by the three-dimensional sensor with the contour of the three-dimensional model in the virtual space;
a superimposed image data generation unit that generates first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment;
Aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object, the outline of the object in the real space and the outline of the object in the virtual space. an alignment correction unit that corrects the alignment with the contour of the three-dimensional model;
The superimposed image data generation unit generates second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result,
A head-mounted display that displays the first superimposed image data and the second superimposed image data. to the computer,
an alignment step of aligning the contour of the object in the real space detected by the 3D sensor provided in the head-mounted display with the contour of the 3D model in the virtual space;
a first superimposed image data generating step of generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment;
a first transmission step of transmitting the first superimposed image data to the head mounted display;
Aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object, the outline of the object in the real space and the outline of the object in the virtual space. an alignment correction step of correcting said alignment with contours of the three-dimensional model;
a second superimposed image data generating step of generating second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result;
a second transmission step of transmitting the second superimposed image data to the head mounted display;
A computer program for executing to the computer,
an alignment step of aligning the contour of the object in the real space detected by the 3D sensor provided in the head-mounted display with the contour of the 3D model in the virtual space;
a first superimposed image data generating step of generating first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment;
a first display step of displaying the first superimposed image data on the head mounted display;
Aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object, the outline of the object in the real space and the outline of the object in the virtual space. an alignment correction step of correcting said alignment with contours of the three-dimensional model;
a second superimposed image data generating step of generating second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result;
a second display step of displaying the second superimposed image data on the head mounted display;
A computer program for executing A head-mounted display with a three-dimensional sensor and an information processing device,
The information processing device is
an alignment unit that aligns the contour of the object in the real space detected by the three-dimensional sensor with the contour of the three-dimensional model in the virtual space;
a superimposed image data generation unit that generates first superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment;
a transmission unit that transmits the first superimposed image data to the head mounted display;
Aligning the reference position on the object detected by the three-dimensional sensor with the reference position on the three-dimensional model, and using the external dimension data of the object, the outline of the object in the real space and the outline of the object in the virtual space. an alignment correction unit that corrects the alignment with the contour of the three-dimensional model;
The superimposed image data generation unit generates second superimposed image data in which the three-dimensional model is superimposed and displayed on the object in the physical space based on the alignment correction result,
The transmission unit transmits the second superimposed image data to the head mounted display.
Image display system.

Images