JP4934228B2 - Information processing apparatus, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow visitors of a facility or a factory to grasp the operating condition of equipment etc. with a feeling of presence. <P>SOLUTION: An information processor comprises: captured image input means which inputs a captured image of equipment imaged by an imaging device; measurement information input means which inputs measurement information measured by a sensor provided in the equipment; generation means which generates a virtual image showing an outside or internal situation of the equipment based on the measurement information input by the measurement information input means; and display control means which superposes the virtual image generated by the generation means and the captured image input by the captured image input means so as to be displayed on a display device. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  In the past, visitors to facilities and factories often saw actual facilities and equipment, and prepared panels and videos. On the other hand, for example, Patent Document 1 proposes a mechanism for superimposing and displaying element information present on the spot in accordance with the movement of a visitor on a real scene image taken with a mobile terminal or the like.

JP 2004-102835 A

Some facilities and factories can be dangerous, and visitors may not be able to access the equipment. Also, even if you get close, you may not see the inside of the facility.
For example, even if the technique of Patent Document 1 is applied to a facility or factory tour, the name of the facility can be seen as a virtual image, and there is a problem that the viewer cannot feel it.
In addition, in the factory, the operation status of the equipment is sometimes predicted by simulation, but the simulation result is merely displayed as a two-dimensional sectional view on the screen of the computer terminal. Therefore, there is a problem that it is not easy for the operator to grasp the simulation result.

  The present invention has been made in view of such problems, and it is an object of the present invention to convey the operating status of equipment and the results of simulations to visitors and workers of facilities and factories with a sense of reality.

Therefore, an information processing apparatus according to the present invention includes a captured image input unit that inputs a captured image of a facility imaged by the imaging device, and a measurement information input that inputs measurement information measured by the sensor from a sensor provided in the facility. Based on the measurement information input by the measurement information input means, a creation means for creating a virtual image representing an external or internal situation of the facility, a virtual image created by the creation means, and the imaging Display control means for superimposing and displaying the captured image input by the image input means, and control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment; And the creation means is based on the control information input by the control information input means and the measurement information input by the measurement information input means. Creates a virtual image representing the internal situation, the facility is a facility in a steel plant, the sensor is a plurality of temperature sensors inside or outside the facility, or a pressure sensor, or a pressure sensor, The control information is injection of oxygen or gas into the facility, or input of material into the facility.
By adopting such a configuration, it is possible to convey the operational status of the facilities in the steelworks with a sense of reality to visitors of facilities and factories.
The information processing apparatus corresponds to, for example, an AR server described later.

The information processing apparatus according to the present invention includes a measurement information input unit that inputs measurement information measured by the sensor from a sensor provided in the facility, and the facility information based on the measurement information input by the measurement information input unit. Creating means for creating a virtual image representing an external or internal situation, and a display control means for superimposing and displaying the virtual image created by the creating means on the equipment that is transmitted through a display device; Control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment, and the creating means, the control information inputted by the control information input means, and the measurement Based on the measurement information input by the information input means, create a virtual image representing the external or internal situation of the facility, the facility is a facility in the steel works, the sensor, Internal or external of the plurality of temperature sensors Bei or pressure sensor, or a pressure sensor, wherein the control information includes blowing of oxygen or gas in the to equipment or a material charging of the to equipment.
By adopting such a configuration, it is possible to convey the operational status of the facilities in the steelworks with a sense of reality to visitors of facilities and factories.
The information processing apparatus corresponds to, for example, an AR server described later.

  ADVANTAGE OF THE INVENTION According to this invention, the operating condition of equipment etc. can be conveyed with a sense of reality to the visitor of a facility or a factory.

FIG. 1 is a diagram illustrating an example of a system configuration of the facility guidance system according to the first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of the AR server. FIG. 3 is a diagram illustrating an example of a software configuration of the AR server according to the first embodiment. FIG. 4 is a diagram (part 1) illustrating an example of a superimposed display of a virtual image and a captured image according to the first embodiment. FIG. 5 is a diagram (part 2) illustrating an example of the superimposed display of the virtual image and the captured image according to the first embodiment. FIG. 6 is a third diagram illustrating an example of a superimposed display of the virtual image and the captured image according to the first embodiment. FIG. 7 is a flowchart illustrating an example of display control processing in the AR server according to the first embodiment. FIG. 8 is a diagram illustrating an example of a system configuration of the facility guidance system according to the second embodiment. FIG. 9 is a diagram illustrating an example of a software configuration of the AR server according to the second embodiment. FIG. 10 is a diagram (part 1) illustrating an example of a superimposed display of a virtual image and a captured image according to the second embodiment. FIG. 11 is a flowchart illustrating an example of display control processing in the AR server according to the second embodiment. FIG. 12 is a flowchart illustrating an example of display control processing in the AR server according to the third embodiment. FIG. 13A is a diagram (part 1) illustrating an example of a display of equipment for each display layer. FIG. 13B is a diagram (part 2) illustrating an example of a display of equipment for each display layer. FIG. 13C is a diagram (part 3) illustrating an example of a display of equipment for each display layer. FIG. 13D is a diagram (part 4) illustrating an example of a display of equipment for each display layer. FIG. 14 is a diagram illustrating an example of a system configuration of the facility guidance system according to the fifth embodiment. FIG. 15 is a diagram illustrating an example of a software configuration of the AR server according to the fifth embodiment. FIG. 16 is a flowchart illustrating an example of display control processing in the AR server according to the fifth embodiment. FIG. 17 is a diagram illustrating an example of a system configuration of the work support system according to the sixth embodiment. FIG. 18 is a diagram illustrating an example of a software configuration of the work support system according to the sixth embodiment. FIG. 19 is a flowchart illustrating an example of processing in the work support system according to the sixth embodiment. FIG. 20 is a diagram illustrating an example of a system configuration of the work support system according to the seventh embodiment. FIG. 21 is a diagram illustrating an example of the equipment marker correspondence table. FIG. 22 is a diagram illustrating an example of the facility usage status table. FIG. 23 is a diagram illustrating an example of a software configuration of the camera-equipped HMD. FIG. 24 is a flowchart illustrating an example of processing in the work support system according to the seventh embodiment. FIG. 25 is a diagram illustrating an example of a system configuration of the work support system according to the seventh embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a system configuration of the facility guidance system according to the first embodiment. As shown in FIG. 1, in the facility guidance system, an AR (Augusted Reality) server 1, a camera 2, sensors 3 _{1 to} 3 _5, and a display device 4 are connected via a network.
The AR server 1 inputs from the camera 2 a captured image of equipment in the factory (in the example of FIG. 1, a blast furnace) captured by the camera 2. In the following embodiments including this embodiment, an example of equipment in a factory will be described as an example of equipment in a factory. This does not limit the following embodiments including this embodiment.
Moreover, AR server 1, the sensor 3 ₁ to 3 ₅ at the measured metrology data (e.g., temperature information inside the blast furnace, etc.) and inputs from the sensors 3 ₁ to 3 _5. In FIG. 1, the number of sensors is shown as five, but the present embodiment is not limited to this, and may be less than five or more than five. In the present embodiment, the sensor 3 is provided outside and inside the equipment. For example, depending on the equipment, the sensor 3 may be provided only inside the equipment, or only outside the equipment. May be provided. For example, in the case of a blast furnace, the sensor 3 is provided outside and inside the blast furnace. Moreover, in the case of the converter mentioned later, the sensor 3 shall be provided in the inside of a converter. Moreover, in the case of the continuous casting machine mentioned later, the sensor 3 shall be provided in the exterior of the continuous casting machine. The sensor is not limited to a temperature sensor that measures temperature information. For example, a pressure sensor that measures pressure information, a pressure sensor that measures pressure information, a gas sensor that measures the type and concentration of gas, and the like. There may be. Sensor 3 ₁ to 3 ₅ are all the same type of sensor (e.g., temperature sensors, etc.), or may be a different type of sensor (e.g., temperature sensors, gas sensors, pressure sensors, etc.).
In the following, for the sake of simplicity, these sensors are simply referred to as sensors unless otherwise specified.
The AR server 1 creates a virtual image representing an external or internal situation of the facility based on the input measurement information, and displays the created virtual image and the captured image input from the camera 2 on the display device 4 in a superimposed manner. Control to do.
Visitors of the factory can grasp the operational status of the equipment in the dangerous and unaccessible area, the operational status inside the equipment, etc. with a sense of reality by viewing the display device 4.

FIG. 2 is a diagram illustrating an example of a hardware configuration of the AR server 1.
As shown in FIG. 2, the AR server 1 includes a CPU 11, a storage device 12, and a communication device 13 as a hardware configuration. The CPU 11 executes processing for controlling the AR server 1 based on a program stored in the storage device 12. The storage device 12 stores programs, data used when the CPU 11 executes processing, and the like. The communication device 13 controls communication between the AR server 1 and other devices (for example, the sensor 3, the camera 2, the display device 4, etc.).
When the CPU 11 executes a process based on a program, a software configuration described later and a process related to a flowchart described later are realized.

FIG. 3 is a diagram illustrating an example of a software configuration of the AR server 1 according to the first embodiment.
As illustrated in FIG. 3, the AR server 1 includes a captured image input unit 21, a measurement information input unit 22, a creation unit 23, and a display control unit 24 as a software configuration.
The captured image input unit 21 receives and inputs captured images of equipment in the factory captured by the camera 2 from the camera 2.
Measurement information input unit 22, the sensor 3 ₁ to 3 ₅ receives from the sensor 3 ₁ to 3 ₅ the measured information measured by the inputs.
The creation unit 23 creates a virtual image representing a situation outside or inside the facility based on the measurement information input by the measurement information input unit 22.
For example, the creation unit 23 receives the measurement information from the measurement information input unit 22 and also receives from the measurement information input unit 22 equipment information indicating which equipment the measurement information is measured from. Based on the received facility information, the creating unit 23 determines whether the facility indicated by the facility information is a facility set to generate a virtual image inside the facility. When the facility indicated by the facility information is a facility that is set to generate a virtual image inside the facility, the creation unit 23 is stored in association with the facility information from the storage device 12 or the like based on the facility information. The in-facility estimation model (in-facility estimation mathematical model formula) is selected. More specifically, in the storage device 12 or the like, for example, a blast furnace and an estimated model in the blast furnace are stored in association with each other. Further, for example, a converter and an in-converter estimation model are associated with the storage device 12 and the like. When the facility indicated by the facility information indicates a blast furnace, the creation unit 23 selects an in-blast furnace estimation model associated with the blast furnace from the storage device 12 or the like. Then, the creation unit 23 substitutes the measurement information input by the measurement information input unit 22 into the selected blast furnace estimation model, estimates the situation in the blast furnace, and generates a virtual image representing the situation in the blast furnace.
Each estimation model is stored in the storage device 12 as a logical model to the last, and is recalculated according to measurement information measured by the sensors 3 _{1 to} 3 ₅ , for example, and has a virtual shape close to the actual state. An image is generated. Therefore, a mathematical model expression is preferable, but it may be generated in advance as computer graphics based on the mathematical model expression, and the display may be changed by changing the parameters. More specifically, the height direction can be adjusted according to the input amount of raw materials in the blast furnace, or the surface position of the molten steel in the furnace can be changed based on the measured value of the sensor in the converter example described later. And so on. Or applying a temperature information measured by the sensor 3 ₁ to 3 ₅ relative to the blast furnace estimation model may generate the actual temperature distribution as a virtual image.
The display control unit 24 superimposes the virtual image representing the external or internal situation of the facility created by the creation unit 23 and the captured image of the facility input by the captured image input unit 21 on the display device 4. The display is controlled (see FIGS. 4 to 6).

FIG. 4 is a diagram (part 1) illustrating an example of a superimposed display of a virtual image and a captured image according to the first embodiment. FIG. 4 shows an example in which a captured image of a blast furnace and a virtual image inside the blast furnace created based on temperature information inside the blast furnace are displayed in a superimposed manner.
FIG. 5 is a diagram (part 2) illustrating an example of the superimposed display of the virtual image and the captured image according to the first embodiment. FIG. 5 shows an example in which a captured image of the converter and a virtual image inside the converter created based on temperature information inside the converter are displayed in a superimposed manner.
FIG. 6 is a third diagram illustrating an example of a superimposed display of the virtual image and the captured image according to the first embodiment. In FIG. 6, an image of a molten steel passing between the continuous caster and the continuous caster and a solidified state of the molten steel passing through the continuous caster created based on temperature information outside the continuous caster. An example is displayed in which a virtual image to be superimposed is displayed.

FIG. 7 is a flowchart illustrating an example of display control processing in the AR server 1 according to the first embodiment.
In step S <b> 10, the captured image input unit 21 receives and inputs, from the camera 2, a captured image of equipment in the factory captured by the camera 2.
In step S11, the measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ from the sensor 3 ₁ to 3 _5, and inputs.
In step S12, the creation unit 23 creates a virtual image representing the external or internal situation of the facility based on the measurement information input in step S11.
In step S13, the display control unit 24 superimposes the virtual image representing the external or internal situation of the facility created in step S12 and the captured image of the facility input in step S10 on the display device 4. Control to display.
In step S14, for example, the display control unit 24 determines whether or not to end the process shown in FIG. 7 based on whether or not display end information or the like is received from the display device 4 or the like. If the display control unit 24 determines to end the process, the display control unit 24 ends the process illustrated in FIG. 7. If the display control unit 24 determines not to end the process, the process returns to step S <b> 10.
Note that the captured image input unit 21 may always receive and input a captured image from the camera 2 as long as the camera 2 is capturing images. Similarly, the measurement information input unit 22 as long as the measurement by the sensor 3 ₁ to 3 ₅ are made, always receive measurement information from the sensor 3 ₁ to 3 _5, may be input. The same applies to the following embodiments. With this configuration, the AR server 1 creates a virtual image based on measurement information input in real time, and superimposes the virtual image on the captured image input in real time and displays the virtual image on the display device 4. Can be controlled.

  Hereinafter, according to the present embodiment, it is possible to convey the operational status of equipment and the like with a sense of reality to visitors of facilities and factories. In particular, since it is possible to show the status of the equipment that is currently in operation, instead of the conventional video image screening, it is possible to provide an unprecedented experience.

<Embodiment 2>
FIG. 8 is a diagram illustrating an example of a system configuration of the facility guidance system according to the second embodiment. As shown in FIG. 8, the system configuration of the facility guidance system of the second embodiment includes an operation server 5 newly compared to the system configuration of the facility guidance system of the first embodiment.
The operation server 5 transmits control information to equipment in the factory. The facility executes processing based on the control information. For example, when the equipment is a blast furnace, the operation server 5 gives the blast furnace the instruction to inject sinter ore to instruct the introduction of sinter or the like, or the coke to instruct the introduction of coke as an example of the material. Instruction information, reducing material blowing instruction information for instructing blowing of the reducing material, output instruction information for instructing output from the outlet, and the like are transmitted.
The control signal is not limited to these. When the equipment is a converter, oxygen injection instruction information for instructing oxygen injection from the top of the converter, oxygen, fuel gas, carbon dioxide gas from the bottom of the converter There are respective blowing instruction information for instructing blowing of inert gas or the like. Furthermore, there is instruction information for instructing to tilt the furnace body in the work of tilting the furnace body and charging iron scraps, or tilting the furnace body again after blowing and transferring the molten steel to the ladle. In addition, when the equipment is a continuous casting machine, there is pressure instruction information for increasing pressure or reducing pressure.
The AR server 1 inputs control information transmitted from the operation server 5 to the equipment. Then, the AR server 1 creates a virtual image representing an external or internal situation of the facility based on the input control information and the input measurement information, and the created virtual image and the imaging input from the camera 2 Control is performed so that the image is superimposed on the image and displayed on the display device 4.
Visitors of the factory can grasp the operational status of the equipment in the dangerous and unaccessible area, the operational status inside the equipment, etc. with a sense of reality by viewing the display device 4.

FIG. 9 is a diagram illustrating an example of a software configuration of the AR server 1 according to the second embodiment.
As shown in FIG. 9, the software configuration of the AR server 1 according to the second embodiment newly includes a control information input unit 25 as compared with the software configuration of the AR server 1 according to the first embodiment.
The control information input unit 25 receives and inputs control information transmitted from the operation server 5 to the equipment from the operation server 5.
The creation unit 23 creates a virtual image representing an external or internal situation of the facility based on the measurement information input by the measurement information input unit 22 and the control information input by the control information input unit 25. For example, as illustrated in the first embodiment, the creation unit 23 substitutes measurement information into an in-facility estimation model corresponding to equipment, estimates a situation in the equipment, and creates a virtual image corresponding to the estimation result.
At the same time, for example, when the control information is coke input instruction information, the creation unit 23 adds an object representing a state in which coke is input into the facility to the virtual image based on the control information, and finally A virtual image representing an internal situation of a facility.
FIG. 10 is a diagram (part 1) illustrating an example of a superimposed display of a virtual image and a captured image according to the second embodiment. In FIG. 10, the captured image of the blast furnace and the virtual image in which the coke is thrown into the blast furnace and the blast furnace created based on the temperature information inside the blast furnace and the instruction information of the coke thrown into the blast furnace are superimposed. An example displayed is shown.
According to the present embodiment, since the creation unit 23 creates a virtual image in consideration of not only measurement information but also control information, it is possible to create a virtual image with a more realistic feeling. Particularly in the converter, when the furnace body is tilted, the virtual image superimposed and displayed also changes with the movement, so that the sense of reality further increases.

FIG. 11 is a flowchart illustrating an example of display control processing in the AR server 1 according to the second embodiment.
In step S <b> 20, the captured image input unit 21 receives and inputs, from the camera 2, a captured image of equipment in the factory captured by the camera 2.
In step S21, the measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ from the sensor 3 ₁ to 3 _5, and inputs.
In step S <b> 22, the control information input unit 25 receives and inputs control information from the operation server 5. The control information input unit 25 inputs the control information every time control information is transmitted from the operation server 5 to the facility.
In step S23, the creation unit 23 creates a virtual image representing an external or internal situation of the facility based on the measurement information input in step S21 and the control information input in step S22.
In step S24, the display control unit 24 superimposes the virtual image representing the external or internal situation of the facility created in step S23 and the captured image of the facility input in step S20 on the display device 4. Control to display.
In step S25, for example, the display control unit 24 determines whether or not to end the processing illustrated in FIG. 11 based on whether or not the display end information is received from the display device 4 or the like. If the display control unit 24 determines to end the process, the display control unit 24 ends the process illustrated in FIG. 11. If the display control unit 24 determines not to end the process, the process returns to step S20.

  Hereinafter, according to the present embodiment, it is possible to convey the operational status of equipment and the like with a sense of reality to visitors of facilities and factories.

<Embodiment 3>
Some factories operate equipment 24 hours a day, 7 days a week, or do not operate equipment for maintenance or cleaning reasons. In the present embodiment, even when the equipment is not in operation, a method will be described in which the operating status of the equipment or the like is conveyed to a visitor of the facility or factory with a sense of reality. The system configuration and software configuration of the present embodiment are the same as those of the first embodiment unless otherwise specified.
The creation unit 23 according to the present embodiment determines whether or not the facility is operating based on the measurement information input from the measurement information input unit 22, and if the determination is that the facility is operating, When a virtual image is created based on this and it is determined that the facility is not operating, a virtual image generated in advance based on the theoretical value or the average value of the measurement information so far is acquired from the storage device 12 or the like.
For example, the creation unit 23 receives the measurement information from the measurement information input unit 22 and also receives from the measurement information input unit 22 equipment information indicating which equipment the measurement information is measured from. The creation unit 23 acquires a pre-generated virtual image associated with the facility information from the storage device 12 or the like based on the received facility information.

FIG. 12 is a flowchart illustrating an example of display control processing in the AR server 1 according to the third embodiment.
In step S <b> 30, the captured image input unit 21 receives and inputs a captured image of equipment in the factory captured by the camera 2 from the camera 2.
In step S31, the measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ from the sensor 3 ₁ to 3 _5, and inputs.
In step S <b> 32, the creation unit 23 determines whether the facility is operating based on the measurement information input from the measurement information input unit 22. For example, when the measurement information is not within a predetermined range, the creation unit 23 determines that the facility is not operating. If the creating unit 23 determines that the facility is operating, the process proceeds to step S33. If the creating unit 23 determines that the facility is not operating, the creating unit 23 proceeds to step 34.
In step S33, the creation unit 23 creates a virtual image representing the external or internal situation of the facility based on the measurement information input in step S31.
On the other hand, in step S34, the creation unit 23 obtains, from the storage device 12 or the like, a virtual image representing the external or internal situation of the equipment created in advance based on theoretical values or the like according to the equipment.
In step S35, the display control unit 24 creates a virtual image representing the external or internal situation of the facility created in step S33 or acquired in step S34, and the captured image of the facility input in step S30. Are superimposed and displayed on the display device 4.
In step S36, for example, the display control unit 24 determines whether or not to end the processing shown in FIG. 12 based on whether or not display end information or the like is received from the display device 4 or the like. If the display control unit 24 determines to end the process, the display control unit 24 ends the process illustrated in FIG. 12. If the display control unit 24 determines not to end the process, the process returns to step S30.

As described above, according to the present embodiment, when the equipment is in operation, the virtual image created based on the measurement information is displayed superimposed on the captured image of the equipment, so that the equipment or facility visitor can provide the equipment. It is possible to convey the operational status such as with a sense of reality. In addition, according to the present embodiment, when the equipment is not in operation, the virtual image created based on the theoretical value or the average value of the measurement information so far is displayed superimposed on the captured image of the equipment, It is possible to convey the operational status of facilities to facilities and factories with a sense of reality.
Although the system configuration and the software configuration of the embodiment are the same as the configuration of the first embodiment, when the configuration is the same as the configuration of the second embodiment, the creation unit 23 is based on measurement information and / or control information. You may make it determine whether an installation is operating. For example, the creation unit 23 determines that the equipment is not in operation when the measurement information is not within a predetermined range and / or when the control information is not transmitted from the operation server 5 to the equipment for a predetermined time. You may do it.

<Embodiment 4>
The display control unit 24 may transmit the display layer set for each facility to the display device 4 according to the screen operation of the visitor in the display device 4. 13A to 13D are diagrams illustrating an example of display of equipment for each display layer. FIG. 13A represents the appearance of the blast furnace. FIG. 13B represents the refractory brick inside the blast furnace. FIG. 13C shows the state of iron ore and coke in the blast furnace. FIG. 13D represents a cross section of the blast furnace.
When the display control unit 24 requests the display device 4 to display the display layer as shown in FIGS. 13A and 13B, the display control unit 24 acquires a virtual image of the display layer created in advance from the storage device 12 or the like, and acquires the acquired display. Control is performed so that the virtual image of the layer and the captured image input by the captured image input unit 21 are displayed superimposed on the display device 4.

  According to this embodiment, in response to a request from a visitor or the like, a virtual image in which the operating status of equipment is displayed in various display layers is displayed superimposed on the captured image of the equipment, so that the facility or factory can be displayed. It is possible to convey the operational status of facilities etc. to the visitors in a more realistic and detailed manner.

<Embodiment 5>
FIG. 14 is a diagram illustrating an example of a system configuration of the facility guidance system according to the fifth embodiment. As shown in FIG. 14, the system configuration of the facility guidance system of the fifth embodiment does not include the camera 2 as compared to the system configuration of the facility guidance system of the first embodiment. The display device 4 of the present embodiment is, for example, a transmissive liquid crystal film stretched on a window or the like.
AR server 1, the sensor 3 ₁ to 3 ₅ at the measured metrology data (e.g., temperature information inside the blast furnace, etc.) and inputs from the sensors 3 ₁ to 3 _5.
The AR server 1 creates a virtual image representing an external or internal situation of the facility based on the input measurement information, and the created virtual image is superimposed on the facility viewed from the window or the like where the display device 4 is installed. Control to be displayed.

FIG. 15 is a diagram illustrating an example of a software configuration of the AR server 1 according to the fifth embodiment.
As illustrated in FIG. 15, the AR server 1 includes a measurement information input unit 22, a creation unit 23, and a display control unit 24 as a software configuration.
The measurement information input unit 22 receives and inputs measurement information measured by the sensors 31 to 35 from the sensors 31 to 35.
The creation unit 23 creates a virtual image representing a situation outside or inside the facility based on the measurement information input by the measurement information input unit 22.
The display control unit 24 performs control so that the virtual image representing the external or internal state of the facility created by the creation unit 23 is displayed superimposed on the facility viewed from the window where the display device 4 is installed. Do.

FIG. 16 is a flowchart illustrating an example of display control processing in the AR server 1 according to the fifth embodiment.
In step S40, the measurement information input unit 22, the measurement information measured by the sensor 3 ₁ to 3 ₅ receives from the sensor 31 to 35 is input.
In step S41, the creation unit 23 creates a virtual image representing the external or internal situation of the facility based on the measurement information input in step S40.
In step S42, the display control unit 24 displays the virtual image representing the external or internal state of the facility created in step S41, superimposed on the facility viewed from the window or the like where the display device 4 is installed. Do the control.
In step S43, for example, the display control unit 24 determines whether or not to end the processing illustrated in FIG. 16 based on whether or not the display end information is received from the display device 4 or the like. If the display control unit 24 determines to end the process, the display control unit 24 ends the process illustrated in FIG. 16. If the display control unit 24 determines not to end the process, the process returns to step S40.

  As described above, according to this embodiment as well, it is possible to convey the operational status of facilities and the like with a sense of reality to visitors of facilities and factories.

<Embodiment 6>
Control of plant facilities in steelworks is still a prime example of computer control, and the situation inside blast furnaces and converters is predicted from the color of pig iron that can be seen through the blast furnace's observation windows and the flame that blows out of the converter's entrance. However, it is often performed based on the judgment of the operator according to the situation at that time. However, this determination is not easy because it requires skill, and if the operator's determination is not appropriate, it may lead to a decrease in the quality of products produced at the factory equipment.
Therefore, in this embodiment, when the factory equipment is controlled based on the judgment of the operator, the situation of the factory equipment when the control is performed (for example, the situation inside the blast furnace or the converter) is simulated. A work support system that presents the result of the simulation as a three-dimensional image to the operator will be described.
The facility guidance systems according to the first to fifth embodiments create a three-dimensional virtual image representing the current state of the facility based on measurement information and control information, and present it to the visitors. However, in the work support system according to the sixth embodiment, when the control is performed based on the information (control schedule information) indicating the content to be controlled in addition to the measurement information, The change is predicted by simulation, and the prediction result is presented to the operator. The operator looks at the prediction result and finally determines what kind of control is actually performed.

The work support system according to the present embodiment will be described in more detail as follows. FIG. 17 is a diagram illustrating an example of a system configuration of the work support system according to the sixth embodiment. As shown in FIG. 17, in the work support system, a server 101, a camera-equipped HMD 102 (hereinafter abbreviated as HMD 102), and sensors 103 _{1 to} 103 ₅ are connected via a wireless network.
FIG. 18 is a diagram illustrating an example of a software configuration of the work support system according to the sixth embodiment.
Compared with the software configuration of the AR server 1 of the first embodiment, the software configuration of the work support system of the present embodiment newly includes a control schedule information input unit 26.
In the present embodiment, software functions are distributed between the server 101 and the HMD 102. The server 101 includes a measurement information input unit 22, a control schedule information input unit 26, and a simulation unit 27 as a functional configuration of software. The HMD 102 includes a camera and a drawing function, and includes a captured image input unit 21 and a drawing / display control unit 28 as a software functional configuration. In this embodiment, the software functions are distributed between the server 101 and the HMD 102. However, the server 101 may have all the functions as in the first embodiment, or the like. The HMD 102 may have all the functions and is not particularly limited.

First, the function of the server 101 will be described.
Measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ as input data from the sensors 3 ₁ to 3 _5. At this time, the sensors 3 _{1 to} 3 ₅ transmit measurement information to the server 101 via the wireless network.
The control schedule information input unit 26 receives the control schedule information input by the operator as input data. As the control schedule information, as with the control information, in addition to sinter ore charging instruction information, coke charging instruction information, reducing agent blowing instruction information and tapping instruction information, oxygen, air and various gas blowing instructions and pressure There is instruction information. The operator can input the control schedule information via an input device connected to the server, or can input the control schedule information remotely via a portable terminal or the like.
The simulation unit 27 performs control of the content represented by the control schedule information based on the measurement information received by the measurement information input unit 22 and the control schedule information received by the control schedule information input unit 26 A simulation of the external or internal situation of the facility is performed. For example, the simulation unit 27 uses the measurement information (the current temperature of the blast furnace, etc.) and the control schedule information (the blowing instruction information indicating the blowing amount of oxygen and air for each tuyere) as an in-facility estimation model corresponding to the facility. Substitute, perform numerical analysis on the situation in the facility, and output numerical data representing the result.
As a technique using the in-facility estimation model, for example, Japanese Patent Laid-Open No. 8-295910 discloses a technique for simulating the operation of a blast furnace by introducing a mathematical model that models the state in the furnace. . By using such a technique, data such as temperature and pressure at each point of the three-dimensional coordinates in the furnace can be obtained.

Next, functions of the HMD 102 will be described.
The captured image input unit 21 receives, as input data, a captured image of equipment in a factory captured by a camera provided in the HMD 102.
When receiving the numerical data representing the simulation result by the simulation unit 27, the drawing / display control unit 28 draws a three-dimensional virtual image based on the numerical data. Further, the drawing / display control unit 28 superimposes a three-dimensional virtual image representing a simulation result on the outside or inside state of the drawn facility and the captured image of the facility input by the captured image input unit 21. And control to display on the display device of the HMD 102. At this time, the server 101 transmits numerical data representing the simulation result to the HMD 102 via the wireless network.

FIG. 19 is a flowchart illustrating an example of processing in the work support system according to the sixth embodiment.
In step S <b> 50, the captured image input unit 21 receives a captured image of equipment in a factory captured by a camera provided in the HMD 102 as input data.
In step S51, the measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ as input data from the sensors 3 ₁ to 3 _5.
In step S52, the control schedule information input unit 26 receives the control schedule information input by the operator.
In step S53, the simulation unit 27 executes a simulation for estimating an external or internal situation of the facility based on the measurement information input in step S51 and the control schedule information input in step S52, and the simulation result The numerical data representing is output. Then, the drawing / display control unit 28 creates a three-dimensional virtual image based on numerical data representing the simulation result.
In step S54, the drawing / display control unit 28 creates a three-dimensional virtual image representing the simulation result of the external or internal situation of the equipment created in step S53, and the captured image of the equipment input in step S50. Are superimposed and displayed on a display (display device) provided in the HMD 102.
In step S55, for example, the drawing / display control unit 28 determines whether or not to end the processing illustrated in FIG. 19 based on whether or not the display end information is received from the HMD 102 or the like. If the drawing / display control unit 28 determines to end the processing, the drawing / display control unit 28 ends the processing illustrated in FIG. 19. If the drawing / display control unit 28 determines not to end the processing, the processing step returns to S50.

For the process of superimposing and displaying the three-dimensional virtual image created by the drawing / display control unit 28 and the captured image, the blowing instruction information indicating the amount of oxygen or air blown for each tuyere is used as control schedule information. The following is a more specific description with reference to FIG.
Large blast furnaces are provided with tuyere for feeding air into the furnace at 30 to 40 locations below the main body. If the amount fed into the tuyere is different, the temperature distribution around each tuyere is respectively Different.
Therefore, an ID for identification (hereinafter abbreviated as tuyere ID) is assigned to each tuyere, and when the simulation unit 27 executes a simulation, the tuyere ID is designated as control schedule information and oxygen or The blowing instruction information representing the amount of blowing air is input. Thereby, data representing the blowing amount at each point of the three-dimensional coordinates corresponding to each tuyere is input. Moreover, the value calculated based on the measurement information from a sensor is input as an initial value of the temperature data at each point of the three-dimensional coordinates in the blast furnace.
Then, the simulation unit 27 uses the input control schedule information and measurement information as initial conditions, and performs a numerical analysis of a flow equation of motion and a heat conduction method equation based on a mathematical model representing a state in the blast furnace, thereby making the constant Various numerical data (temperature data, speed data, pressure data, etc.) at each point of the three-dimensional coordinates in the blast furnace after the elapse of time are calculated.
When the drawing / display control unit 28 receives the numerical data at each point of the three-dimensional coordinates in the blast furnace representing the simulation result from the simulation unit 27, the simulation result (temperature distribution, velocity distribution, pressure distribution) is received from the received numerical data. Etc.) is generated.
Further, when the created / display control unit 28 superimposes and displays the created three-dimensional virtual image and the captured image, for example, a part that is visible to the user based on the blast furnace image captured by the camera provided in the HMD 102 Identify the orientation and angle. More specifically, since the image of the blast furnace captured by the camera provided in the HMD 102 is almost the same as the image of the blast furnace captured in the user's field of view, images obtained by capturing the blast furnace from various angles and orientations in advance. Is stored in the memory in the HMD 102, and the image captured by the camera of the HMD 102 and the image stored in the memory are matched to identify the orientation and angle of the portion visible to the user. Then, the drawing / display control unit 28 controls the orientation of the three-dimensional virtual image representing the simulation result presented to the operator based on the identified azimuth and angle.

Note that the configuration may be such that the operator can adjust the orientation of the three-dimensional virtual image by himself / herself through the input device, and is not particularly limited.
According to this configuration, the operator can confirm in advance from the simulation results what the temperature distribution and pressure distribution in the blast furnace and converter will be based on the control he / she will perform. .
Thus, for example, if the simulation result in the blast furnace shows a good temperature distribution or the like, the operator performs the control as scheduled, and if the simulation result shows an unfavorable temperature distribution or the like, the control The content can be reviewed to select an appropriate operation. Therefore, even an inexperienced operator can appropriately operate factory equipment such as a blast furnace that requires skill. Moreover, since the internal temperature distribution and the like are presented as a three-dimensional virtual image superimposed on the actual blast furnace or converter, the operator can more intuitively understand the situation inside the facility, and the simulation Whether or not the result is good can be grasped instantaneously and accurately.
In this embodiment, the case where the non-transparent HMD 102 with a camera is used has been described as an example. However, a camera is included by using the transmissive HMD by applying the configuration of the fifth embodiment. It may be a configuration that is not. That is, the captured image input unit 21 is not provided in the HMD, and the drawing / display control unit 28 performs control so that the created three-dimensional virtual image is superimposed and displayed on the equipment that is actually visible from the HMD. Will do. In this case, if the HMD is equipped with a GPS or an acceleration sensor, the HMD can recognize from which azimuth and angle the user is capturing the equipment. Further, the relationship between the three-dimensional coordinates used in the simulation and the actual equipment orientation is defined in advance. As a result, even in an HMD that does not have a camera, the orientation of the three-dimensional virtual image representing the simulation result presented to the operator is controlled based on the identified direction and angle, so that the equipment can be seen from the transmission type HMD. It can be displayed superimposed.

<Embodiment 7>
At the steelworks, the hot metal coming out of the blast furnace is put into a kneading car (torpedo car) or the like and transported to a steelmaking factory and charged into the steelmaking furnace in a molten state. Therefore, as with the chaotic vehicle, as in the first embodiment, a virtual image representing the internal situation may be generated as a part of the factory equipment, and superimposed on the chaotic vehicle and presented to the worker. That is, measurement information such as the temperature and pressure of the inside and the surface of a chaotic vehicle is acquired by a sensor, and the temperature distribution generated based on the in-facility estimation model associated with the chaotic vehicle is expressed using this measurement information. The virtual image may be displayed so as to be superimposed on an actual chaotic vehicle.
Furthermore, in this embodiment, the structure which displays the order number corresponding to the hot metal currently conveyed by the chaotic vehicle may be sufficient. The work support system according to the present embodiment will be described in more detail as follows. FIG. 20 is a diagram illustrating an example of a system configuration of the work support system according to the seventh embodiment. As shown in FIG. 20, in the work support system, a server 101, a camera-equipped HMD 102 (hereinafter abbreviated as HMD 102), and sensors 104 _{1 to} 104 ₅ are connected via a wireless network.
In the seventh embodiment, various processes executed in the AR server 1 in the first embodiment are performed in the HMD 102. The function of the HMD 102 will be described in detail as follows.

The HMD 102 acquires a captured image of equipment in the factory (chaotic vehicle in the example of FIG. 20) captured by a camera integrally provided in the HMD 102. Further, the HMD 102 acquires measurement information (for example, temperature information inside the chaotic vehicle) measured by the sensors 104 _{1 to} 104 ₅ via wireless or the like.
Furthermore, in this embodiment, the marker for identifying each equipment is attached | subjected to the equipment in a factory, and HMD102 can identify each equipment, when the camera with which the self-machine was recognized recognizes each marker. is there. More specifically, the HMD 102 holds the equipment marker correspondence table shown in FIG. 21 in which equipment IDs for identifying each equipment and the respective markers are associated with each other in a memory provided in the own device, By recognizing the marker by, each facility is identified by reading out the facility ID associated with each marker from the facility marker correspondence table. FIG. 21 is a diagram illustrating an example of the equipment marker correspondence table.
In addition, the HMD 102 creates a virtual image representing an external or internal situation of the facility based on the acquired measurement information, and superimposes the created virtual image and a captured image captured by the camera on a display provided in the own device Control to display on (display device).
Furthermore, in this embodiment, the HMD 102 generates a virtual image indicating the order number to which the product in the facility is assigned based on the facility ID, and creates the virtual image of the created order number and the captured image of the facility captured by the camera. It is characterized in that it is superimposed and displayed on a display device provided in the own device. This point will be described in more detail as follows.
In the present embodiment, the server 101 stores a facility usage status table shown in FIG. 22 in which a facility ID for identifying each facility is associated with an order number. This equipment usage status table is updated by the operator when data is registered in the manufacturing management system in the factory. For example, when an operation of manufacturing pig iron in a blast furnace is started, the above-mentioned equipment usage status table is updated by an operator entering an order number assigned to pig iron manufactured in the blast furnace into the production management system. Is done. That is, in the present embodiment, the server 101 stores the data of the manufacturing management system, unlike the AR server 1 shown in FIG. FIG. 22 is a diagram illustrating an example of the facility usage status table.
As with the AR server 1 shown in FIG. 2, the hardware configuration of the HMD 102 includes a CPU, a storage device (memory), and a communication device. The CPU is based on programs and data included in the storage device. By executing the processing for controlling the HMD 102, the processing related to the software configuration and flowcharts to be described later is realized.

FIG. 23 is a diagram illustrating an example of a software configuration of the camera-equipped HMD. As illustrated in FIG. 23, the HMD 102 includes a captured image input unit 21, a measurement information input unit 22, a specific processing unit 29, a creation unit 23, and a display control unit 24 as a software configuration.
The captured image input unit 21 receives, as input data, a captured image of equipment in a factory captured by a camera provided in the HMD 102.
Measurement information input unit 22 receives the measurement information measured by the sensor 3 ₁ to 3 ₅ as input data from the sensors 3 ₁ to 3 _5.
The identification processing unit 29 identifies the order number to which the product in each facility is allocated. More specifically, the specific processing unit 29 reads out the equipment ID corresponding to the marker attached to the equipment in the factory captured by the camera from the equipment marker correspondence table described above.
In the example shown in FIG. 20, when the specific processing unit 29 captures the marker A attached to the chaotic vehicle A and the marker B attached to the chaotic vehicle B from the equipment marker, the specific processing unit 29 uses the equipment marker correspondence table. The facility ID corresponding to each marker is read out, and the facility to which the marker A is attached is the chaotic vehicle A, and the facility to which the marker B is attached is identified as the chaotic vehicle B.
Further, the specification processing unit 29 reads out the order number corresponding to each equipment ID from the above-described equipment usage status table, and specifies the order number to which the product in each equipment is allocated.
In the example illustrated in FIG. 20, when identifying the chaotic vehicle A and the chaotic vehicle B, the specific processing unit 29 reads out the order ID corresponding to each equipment ID from the equipment usage status table. It is specified that pig irons of “ABC123” and “EDF456” are transported, and pig irons of order IDs “G123” and “X456” are carried in the chaotic vehicle B.

The creation unit 23 creates a virtual image representing an external or internal situation of the facility based on the measurement information input by the measurement information input unit 22 input by the measurement information input unit 22. The creation unit 23 creates a virtual image representing the order number specified by the specification processing unit 29.
The display control unit 24 superimposes the virtual image representing the external or internal state of the facility created by the creation unit 23 and the captured image of the facility input by the captured image input unit 21 to display the display device of the HMD 102 Control to display on. In addition, the display control unit 24 superimposes the virtual image representing the order number created by the creation unit 23 and the captured image of the facility input by the captured image input unit 21 and displays them on the display device of the HMD 102. Control as follows.
FIG. 20 shows a state in which the virtual image representing the order number created by the creating unit 23 is superimposed on the chaotic vehicle image captured by the display control unit 24 with the camera.
Note that FIG. 20 does not show an example of superimposing a virtual image representing the situation outside or inside the facility, but it is possible to display by switching the display layer using the configuration of the fourth embodiment. It is.
In the example illustrated in FIG. 20, the configuration in which the marker is used to identify the factory equipment has been described. However, instead of the marker, a wireless IC tag such as an RFID may be used to identify the factory equipment. That is, the HMD 102 has a wireless IC tag reader function, a wireless IC tag storing each equipment ID is attached to each equipment, and the equipment ID may be directly read from the equipment. Not done.

FIG. 24 is a flowchart illustrating an example of processing in the work support system according to the seventh embodiment.
In step S <b> 60, the captured image input unit 21 receives and inputs captured images of facilities in the factory captured by the camera provided in the HMD 102.
In step S61, the measurement information input unit 22 is a sensor 3 ₁ to 3 ₅ receives from the sensor 3 ₁ to 3 ₅ the measured information measured by the inputs.
In step S62, the identification processing unit 29 identifies the facility ID of each facility based on the marker captured by the camera provided in the HMD 102, and further, the product stored in each facility based on the identified facility ID. Specify the order number.
In step S63, the creation unit 23 creates a three-dimensional virtual image representing the external or internal situation of the facility based on the measurement information input in step S61. Further, the creation unit 23 creates a virtual image representing the order number based on the order number specified in step S62.
In step S64, the display control unit 24 superimposes and displays the three-dimensional virtual image representing the external or internal situation of the facility created in step S63 and the captured image of the facility input in step S60. Control to display on the device 4. In addition, the display control unit 24 performs control so that the virtual image representing the order number created in step S63 and the captured image of the facility input in step S60 are superimposed and displayed on the display device 4.
In step S65, for example, the display control unit 24 determines whether or not to end the processing illustrated in FIG. 24 based on whether or not display end information or the like is received from the display device 4 or the like. If the display control unit 24 determines to end the process, the display control unit 24 ends the process illustrated in FIG. 24. If the display control unit 24 determines not to end the process, the process returns to S60.
Note that the present embodiment can be applied not only to factory equipment but also to products waiting to be shipped. In this case, since it is not necessary to superimpose and display a virtual image representing the inside or outside of the facility, the process of skipping S61 and not generating a virtual image representing the inside or outside of the facility in S63 is performed. There may be, and it does not specifically limit.

FIG. 25 is a diagram illustrating an example of a system configuration of the work support system according to the seventh embodiment. In the example shown in FIG. 25, the order number is displayed on the coil waiting for shipment. In this example, markers 105 _{1 to} 105 ₃ are attached to the coils waiting to be shipped, and the order number of each coil can be acquired by capturing this marker with the HMD 102. Since it is the same as the example demonstrated using FIG. 20, description is abbreviate | omitted.
In the example shown in FIG. 25, not only the order number but also the loading order instruction sheet is displayed by AR. In this case, the operator registers in advance information regarding the order of loading (information defining the order of loading using the order number) in the server 101. Then, when the specification processing unit 29 specifies the order number of the product, information regarding the loading order registered in the server 101 is also read, and the creation unit 23 creates a virtual image representing the loading order instruction.
As a result, workers can instantly grasp the assigned order number and loading order for products, semi-finished products, and materials stored in the facility simply by looking at the facilities and products in the factory. Therefore, operations such as preparation for shipment and transportation to the next process can be performed smoothly, and mistakes in operation can be prevented.
In this embodiment, the case where the non-transparent HMD 102 with a camera is used has been described as an example. However, a camera is included by using the transmissive HMD by applying the configuration of the fifth embodiment. It may be a configuration that is not. In that case, the captured image input unit 21 is not provided in the HMD, and the creation unit 23 performs control so that the created three-dimensional virtual image is displayed superimposed on the equipment that is actually visible from the HMD. It will be.
The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

  For example, in the said embodiment, it demonstrated by the example which shows the mode in facilities, such as a blast furnace of a steel mill and a converter which it is difficult to look inside. However, each of the above-described embodiments can be applied to facilities that handle liquids and powders in a transparent casing as food manufacturing, chemical / pharmaceutical manufacturing, and the like. When each of the above-described embodiments is applied to such a facility, the liquid or powder being charged is directly shown to a visitor, and the attributes of the liquid or powder (material name, temperature, viscosity, acidity, alcohol content) 1) and the like can be superimposed and displayed in a display form such as a tag. In this way, more specific information can be obtained from the tag information while viewing the real-time operation status, and a tour service that is easier to understand for the visitor can be provided.

In the above-described embodiment, a so-called non-transmissive display or a transmissive display has been described as an example. However, the present invention can be applied to a non-transmissive projector or a transmissive projector. Can do. For example, instead of the display device 4, a display device provided with a glasses-type display unit mounted on the visitor's head or the like may be used. For example, when the AR server 1 superimposes and displays the captured image and the virtual image, the AR server 1 performs coordinate alignment between the captured image and the virtual image. As a method of coordinate alignment, for example, there is a method in which a fixed marker is arranged in the facility, and the coordinate alignment between the captured image and the virtual image is performed based on the fixed marker in the image captured by the camera 2. . In addition, when using the display apparatus provided with the spectacles type display part with which a visitor's head etc. are equipped, the AR server 1 is based on the positional information, direction information, etc. from a display apparatus. Estimate whether or not the image is viewed, and adjust the coordinates to match the estimated position.
In addition, you may implement combining embodiment mentioned above arbitrarily.
The AR server 1 described above is an example of a computer.

1 AR server 11 CPU
12 storage device 13 communication device

Claims (10)

A captured image input means for inputting a captured image of the facility imaged in the imaging device;
Measurement information input means for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input by the measurement information input means, a creation means for creating a virtual image representing an external or internal situation of the facility;
Display control means for superimposing and displaying the virtual image created by the creating means and the captured image input by the captured image input means on a display device;
Control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment;
Have
The creation means creates a virtual image representing an external or internal situation of the facility based on the control information input by the control information input means and the measurement information input by the measurement information input means. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is an information processing apparatus that is oxygen or gas blown into the facility or a material is charged into the facility. A measurement information input means for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input by the measurement information input means, a creation means for creating a virtual image representing an external or internal situation of the facility;
Display control means for displaying the virtual image created by the creation means in a superimposed manner on the equipment that is transmitted and displayed from a display device;
Control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment;
Have
The creation means creates a virtual image representing an external or internal situation of the facility based on the control information input by the control information input means and the measurement information input by the measurement information input means. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is an information processing apparatus that is oxygen or gas blown into the facility or a material is charged into the facility. Control schedule information input means for inputting control schedule information representing control details scheduled for the equipment ;
Based on the control schedule information input by the control schedule information input means and the measurement information input by the measurement information input means, the equipment when the control represented by the control schedule information is performed A simulation means for performing a simulation for estimating an external or internal situation;
Further comprising
The information processing apparatus according to claim 1 , wherein the creating unit creates a three-dimensional virtual image representing the simulation result from numerical data representing the simulation result . Further comprising specifying means for specifying an order number corresponding to the product in the facility ;
The information processing apparatus according to claim 1 , wherein the creating unit creates a virtual image representing the order number specified by the specifying unit. The creation unit creates a virtual image representing an external or internal situation of the facility based on the measurement information input by the measurement information input unit when the facility is operating.
When the equipment is operating, the display control means superimposes the virtual image created by the creation means and the captured image input by the captured image input means and displays them on the display device, The information processing apparatus according to claim 1, wherein when the equipment is not in operation, a virtual image created in advance and a captured image input by the captured image input unit are superimposed and displayed on the display device. The creation unit creates a virtual image representing an external or internal situation of the facility based on the measurement information input by the measurement information input unit when the facility is operating.
When the equipment is in operation, the display control means displays the virtual image created by the creation means in a superimposed manner on the equipment transmitted through a display device, and the equipment is in operation. The information processing apparatus according to claim 2, wherein a virtual image created in advance is displayed so as to be superimposed on the facility that is transmitted through the display device when the virtual image is not displayed . An information processing method executed by an information processing apparatus,
A captured image input step for inputting a captured image of the facility imaged in the imaging device;
A measurement information input step for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input in the measurement information input step, a creation step of creating a virtual image representing an external or internal situation of the facility;
A display control step for superimposing and displaying the virtual image created in the creation step and the captured image input in the captured image input step on a display device;
A control information input step for inputting control information transmitted from the control device for controlling the equipment to the equipment;
Including
In the creation step, a virtual image representing an external or internal situation of the facility is created based on the control information input in the control information input step and the measurement information input in the measurement information input step. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is an information processing method in which oxygen or gas is blown into the facility or material is charged into the facility. An information processing method executed by an information processing apparatus,
A measurement information input step for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input in the measurement information input step, a creation step of creating a virtual image representing an external or internal situation of the facility;
A display control step of superimposing and displaying the virtual image created in the creation step on the equipment that is transmitted through a display device; and
A control information input step for inputting control information transmitted from the control device for controlling the equipment to the equipment;
Including
In the creation step, a virtual image representing an external or internal situation of the facility is created based on the control information input in the control information input step and the measurement information input in the measurement information input step. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is an information processing method in which oxygen or gas is blown into the facility or material is charged into the facility. Computer
A captured image input means for inputting a captured image of the facility imaged in the imaging device;
Measurement information input means for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input by the measurement information input means, a creation means for creating a virtual image representing an external or internal situation of the facility;
Display control means for superimposing and displaying the virtual image created by the creating means and the captured image input by the captured image input means on a display device;
Control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment;
To function,
The creation means creates a virtual image representing an external or internal situation of the facility based on the control information input by the control information input means and the measurement information input by the measurement information input means. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is a program in which oxygen or gas is blown into the facility or material is charged into the facility. Computer
A measurement information input means for inputting measurement information measured by the sensor from a sensor provided in the facility;
Based on the measurement information input by the measurement information input means, a creation means for creating a virtual image representing an external or internal situation of the facility;
Display control means for displaying the virtual image created by the creation means in a superimposed manner on the equipment that is transmitted and displayed from a display device;
Control information input means for inputting control information transmitted from the control device for controlling the equipment to the equipment;
To function,
The creation means creates a virtual image representing an external or internal situation of the facility based on the control information input by the control information input means and the measurement information input by the measurement information input means. And
The facility is a facility in the steelworks,
The sensor is a plurality of temperature sensors inside or outside the facility, or a barometric pressure sensor, or a pressure sensor,
The control information is a program in which oxygen or gas is blown into the facility or material is charged into the facility.

Images