JP3274290B2 - Video display device and video display system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image display device and an image display system for visualizing information using a portable computer or the like and presenting the information to a user.

[0002]

2. Description of the Related Art In a large-scale plant represented by a nuclear power plant, a road, a railway, an aircraft, and a water and sewage system, regular maintenance work is performed in order to provide stable services. In particular, in each of the above-mentioned systems, it is required to perform enormous inspection items safely and reliably. Therefore, workers who perform these tasks need special skills and experience, but they need to work in a dangerous and contaminated environment. In recent years, it has become difficult to grow. As a means for solving these problems, a computer system has been developed which supports efficient and easy maintenance work.

FIG. 13 is a block diagram showing a conventional computer system (patrol inspection system) for supporting maintenance work disclosed in Japanese Patent Application Laid-Open No. 03-143204. In the figure, reference numeral 101 denotes a portable computer; Is the image display section,
103 is a key input unit. The purpose of this system is to improve the work efficiency of workers and to prevent omission of inspection.

Next, the operation will be described. In this patrol inspection system, the portable computer 1 as shown in FIG.
01 is carried by the worker. In the patrol inspection system, the device name of the device to be patroled and the inspection items are displayed on the screen of the image display unit 102 of the portable computer 101 as shown in the screen example of FIG. 14, and the display of the inspected device is inverted and displayed. I do.

FIG. 15 is a block diagram showing a configuration of a site terminal patrol / inspection support device disclosed in Japanese Patent Laid-Open No. 5-89392. In the figure, reference numeral 201 denotes a substation device as a patrol target device; 202 is a patrol inspection computer, 2
03 is a portable terminal. This patrol and inspection support device,
The purpose is to improve the work efficiency of patrol members during patrols and inspections and reduce the load on patrol members.

Next, the operation will be described. The inspection inspection computer 202 is installed in the control room, and the knowledge base 22
2, monitoring data 221, treatment guidance 223, additional inspection / inspection item data 224, and diagnostic means 226 for judging the equipment status from these information. Inspection items and treatment guidance are displayed on the portable terminal 203 at the site where communication with the computer is possible via transmission means. The patrol member patrols the substation equipment 201 according to the displayed patrol / inspection items.
Perform an inspection and enter the result via the portable terminal 203,
The portable terminal 203 compares the result with the patrol inspection computer 202.
Send to The inspection inspection computer 202 diagnoses the transmitted inspection / inspection, and if additional inspection / inspection is required, adds an additional inspection / inspection item to the diagnosis result and adds the inspection / inspection item to the portable terminal 20.
Send to 3.

FIG. 16 is a block diagram showing the configuration of a visual information display device disclosed in Japanese Patent Application Laid-Open No. Hei 5-191839. In the drawing, 330 is a liquid crystal display device, 340 is a reflection mirror, and 350 is a half mirror. . This device is
It is not directly related to the maintenance work of equipment such as plants, but is displayed by superimposing stereoscopic three-dimensional video information such as CG (computer graphics), television video, video video, etc. on the landscape that the user can see with the naked eye. The purpose is to let them.

In FIG. 16, in order to present three-dimensional image information, an image to be projected on an eye is projected on a liquid crystal display device 330, and the image is projected on a half mirror 350 via an optical system such as a reflection mirror 340. . The user can view the outside scenery and the image of the liquid crystal display device 330 on the half mirror 350 in a superimposed manner. By preparing two sets of this mechanism and projecting them to the left and right eyes, the scenery can be three-dimensionally superimposed on CG, television video, video video, and the like.

[0009]

Since the conventional video display system is configured as described above, guidance on the patrol route, device operation, check items, and drawings of the target device are displayed on the portable computer at the site. Although it is thought that it is possible to carry the guidance, check items, drawings of the target device, etc. on the patrol route and device operation drawn in advance on paper as a portable computer, only to reduce the workload of searching Therefore, there is a problem that the burden on the worker cannot be substantially reduced. That is,
When working with these devices, the worker must move his / her gaze between the device to be inspected / repaired and the screen on which the information is displayed. Also, no matter how much useful information is presented on the screen of the portable computer, if the worker does not look at the portable computer, this system does not support the system. In fact, considering that some kind of inspection work is performed using such a support system, it is a burden for the worker to check the guidance information displayed on the screen in detail and then check and repair the target equipment. And thus neglected to look at the screen.

[0010] Further, when a drawing related to the target device is presented on the screen of the portable computer, the worker considers which part of the drawing corresponds to which part of the inspection / repair target device and considers the head. It is necessary to overlap the inspection / repair equipment with the drawing in the method, and there is also a problem that an operator error may occur in this part.

As described above, JP-A-3-14320
In the case of using a support system for maintenance and patrol work using a portable computer as disclosed in Japanese Unexamined Patent Application Publication No. 4 (1994) and Japanese Unexamined Patent Application Publication No. 5-89392, it is necessary to force a worker to move his / her gaze, check / repair target equipment, The operator must superimpose the information on the screen. One method of solving this problem is to use an image display device mounted on the head as in the above-mentioned Japanese Patent Application Laid-Open No. HEI 5-191839, to provide operation guidance for inspection / repair target equipment, check items,
It is conceivable that a drawing or the like for the target device is superimposed and displayed on the inspection / repair target device or its periphery. By using this method, the worker can obtain information such as guidance and drawings created by the computer without turning his / her eyes away from the inspection / repair target device.

However, when such a head-mounted image display device shown in FIG. 16 is used, the optical system devices such as the liquid crystal display device 330, the reflection mirror 340, and the half mirror 350 are moved to the operator's head. The operator must support these weights with his / her neck during work, and feel uncomfortable due to the presence of a mirror or the like in the field of view. In addition, these devices are mounted on a helmet or the like, and the worker wears it on the head.At this time, it is impossible to completely fix the helmet on the head, so if the worker moves, The relative position between the eyeball and the optical system is not fixed, and the CG projected on the half mirror 350,
Video such as video shakes. Studies have shown that such shaking causes workers to experience the same symptoms as motor sickness. From these facts, it is not realistic to perform maintenance work such as inspection and repair over a long period of time using the head-mounted visual information presenting device because it places a great burden on workers. Inspection and repair work is usually performed by a plurality of workers, but with a support system using this head-mounted image display device, information can be presented only to the wearing worker. There was a problem.

[0013] The invention of claim 1 has been made to solve the above-mentioned problems, and in order to reduce the burden on the workers at the time of work inspection during inspection and repair work of a plant, a plurality of workers are provided. It is an object of the present invention to obtain an image display device which enables the user to simultaneously view information on inspection and does not need to move his or her line of sight from the inspection / repair target device.

A second object of the present invention is to provide a video display device which can reduce the burden on the worker at the time of work inspection in the inspection and repair work of the plant and can be carried by the worker. .

According to a third aspect of the present invention, there is provided an image display apparatus which enables an operator to make an accurate decision when a maintenance method and an operation method need to be changed depending on the internal state of an object. With the goal.

A fourth object of the present invention is to provide a video display device capable of reliably executing maintenance work and operation work required at a work site.

A fifth object of the present invention is to provide an image display device capable of preventing a worker and a person around the worker from danger in advance.

A sixth object of the present invention is to provide a video display device which enables a worker to take an action required at a work site accurately and promptly.

A further object of the present invention is to provide a video display device which can easily execute drawing of a figure according to the shape of an object.

An object of the present invention is to provide a video display system in which a worker at a work site and an operator at the center can easily communicate with each other.

It is another object of the present invention to provide a video display system capable of easily drawing a figure according to the shape of an object.

[0022]

According to a first aspect of the present invention, there is provided an image display apparatus which projects images using image projection means based on information obtained from a position and orientation meter side image and images obtained from a photographing means. Projection information search means for searching the projection information database and the object database for the image information relating to the image to be imaged and the arrangement and shape of the object, based on the relative positional relationship between the image projection means and the object and the shape of the object Video generating means for generating a video to be projected from the retrieved video information.

An image display apparatus according to a second aspect of the present invention includes a portable computer which can be carried by a user, the portable computer functioning as a support means, and a position and orientation measuring means, a projection information database. Means, object database means, projection information search means, and image generation means.

According to a third aspect of the present invention, there is provided an image display device, comprising: an internal state of an unmeasurable object estimated by using an internal state estimation database including a dynamic characteristic model; The apparatus further comprises means for visualizing at least one of the internal state of the target that cannot be performed and adding the visualized video to the video projected on the target and its surroundings.

According to a fourth aspect of the present invention, there is provided a video display apparatus comprising: a maintenance and operation method determining database for storing a maintenance method and an operation method of a device including an object; and a maintenance method or an operation method suitable for a photographed object. Means for determining using the maintenance and operation method determination database and adding an image relating to the maintenance method or the operation method to the image projected on the object and its surroundings.

According to a fifth aspect of the present invention, there is provided an image display apparatus which obtains a sensor such as a radiation, a temperature, a gas concentration and the like and a spatial distribution of radiation, temperature, a gas concentration and the like obtained from the sensor which cannot be perceived by a human. Means for adding an image relating to a spatial distribution to an image projected on the object and its surroundings.

According to a sixth aspect of the present invention, there is provided an image display apparatus which determines image information for supporting a user's actions such as guidance of a dangerous area and a moving route to a user by using a dangerous area and a moving route determining database. And means for adding an image relating to the determined support information to the image projected on the periphery thereof.

According to a seventh aspect of the present invention, the image projection means of the image display apparatus controls the optical axis of the visible laser light to thereby control the relative position between the image projection means and the object and the shape of the object. Means for projecting images, moving images, or characters in real time.

An image display system according to an eighth aspect of the present invention is an image display system, comprising: image information relating to an image to be projected by the image projection means based on information obtained from the position and orientation meter side means and an image obtained from the photographing means. And projection information retrieval means for retrieving the arrangement and shape of the object from the projection information database and the object database, and the image information retrieved based on the relative positional relationship between the image projection means and the object and the shape of the object. A portable computer having an image generating means for generating an image to be projected from a portable computer, and a central computer communicable with the portable computer via a communication means, wherein the image is taken by an imaging means installed in the portable computer. Remote display means for displaying an image of the object, and remote input means for processing the input information to project the input information to the object and its surroundings using the image projection means. Those with a central side computer which.

In the image display system according to the ninth aspect of the present invention, the image projection means installed in the portable computer controls the optical axis of the visible laser light to determine the relative position between the image projection means and the object. It includes means for projecting a video, a moving image, or a character in real time according to the shape of the object.

[0031]

According to a first aspect of the present invention, there is provided an image display apparatus, wherein the projection information retrieval means displays information obtained from the position and orientation meter side means, that is, the position and orientation of the support means, and an image of an object obtained from the photographing means. Based on the image projection means, the image information relating to the image to be projected around the object, including the object, and the arrangement and shape of the object are searched from the projection information database and the object database. An image to be projected is generated from the searched image information based on the relative positional relationship of the object and the shape of the object. The image is projected onto the object and its surroundings via the image projection means. Thus, in maintenance and inspection of a plant or the like, for example, diagnosis of equipment including an object, operation guidance stored as a database, video information related to a drawing of the equipment, and the like, can be directly performed by the video projection means on the object to be inspected / repaired and its surroundings. In addition to this, it is not necessary for a user such as a worker to move his or her line of sight from the inspection / repair target device, and a plurality of workers can simultaneously view information related to the inspection.

According to a second aspect of the present invention, there is provided a video display device comprising:
In the inspection and repair work of the plant, etc., the burden on the worker at the time of work inspection can be reduced, and the worker can easily carry the work.

The image display device according to the third aspect of the present invention
At least one of the internal state of the unmeasurable target and the internal state of the target that is measurable but cannot be directly seen by the user, estimated using the internal state estimation database including the dynamic characteristic model, etc. Visualized,
The visualized image is added to the image projected on the object and its surroundings. As a result, video information relating to the internal state of the equipment can also be directly projected to the inspection / repair equipment and its surroundings by the video projection means provided in the portable computer, etc. Need to be moved and the maintenance method and operation method need to be changed depending on the internal state of the target.
Workers can make accurate decisions.

According to a fourth aspect of the present invention, there is provided an image display device comprising:
A maintenance method or an operation method suitable for the photographed object is determined using the maintenance and operation method determination database, and an image relating to the maintenance method or the operation method is added to the image projected on the object and its surroundings. As a result, video information related to maintenance or operation guidance can also be directly projected to the inspection / repair equipment and its surroundings by the video projection means provided in the portable computer, etc. It is not necessary to move the work, and the maintenance work and the operation work required at the work site can be reliably executed.

According to a fifth aspect of the present invention, there is provided a video display device comprising:
The spatial distribution of information that cannot be perceived by humans, such as radiation, temperature, and gas concentration, obtained from the sensor is obtained, and an image relating to the spatial distribution is added to the image projected on and around the object.
As a result, image information on the spatial distribution of gas, heat, radiation, etc., captured by the sensor can be projected directly to inspection and repair equipment and its surroundings by image projection means provided in portable computers and the like. Inspection /
It is not necessary to move the line of sight from the repair target device, and the user and the people around it can be avoided from danger in advance.

The video display device according to the invention of claim 6 is:
The danger area for the user, video information that assists the user's action such as guidance of the travel route is determined using the danger area and the travel route determination database, and the video related to the support information determined as the video projected on the target object and its surroundings is determined. Add.
As a result, the video information relating to the guidance of the moving route or the patrol route and the danger area can also be directly projected onto the inspection / repair equipment and its surroundings by the video projection means provided in the portable computer, etc. The user does not need to move his line of sight from the device to be inspected / repaired.
The user can take the action required at the work site accurately and promptly.

According to a seventh aspect of the present invention, the image projecting means of the image display apparatus controls the optical axis of the visible laser light, so that the image projecting means can be controlled in accordance with the relative position between the image projecting means and the object and the shape of the object. Project video, video, or text in real time. This makes it easy to draw a figure according to the shape of the object.

In the video display system according to the present invention, the central computer capable of communicating with the portable computer via the communication means captures an object photographed by the photographing means installed in the portable computer using the remote display means. Displays an image of an object.
The remote input means processes the input information to project the input information input by the operator onto the target object and its surroundings using the video projection means. Thus, the worker at the site carrying the portable computer and the operator of the central computer can easily communicate with each other.

According to a ninth aspect of the present invention, the image display means of the image display system controls the optical axis of the visible laser light, so that the relative position of the image projection means and the object and the shape of the object can be adjusted. Project video, video, or text in real time. This makes it easy to draw a figure according to the shape of the object.

[0040]

【Example】

Embodiment 1 FIG. An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an image display apparatus according to an embodiment of the present invention. In the figure, reference numeral 1 denotes a camera (photographing means), 2 denotes a position using a gyro, a magnetic sensor, an ultrasonic sensor, light, and the like. A position and orientation measurement device constituted by a measurement device, an image projection device (image projection means) 3 for projecting information created by the computer onto an object, and a portable computer 4. The camera 1, the position and orientation measurement device 2, and the image projection device 3 are attached to a portable computer 4. That is, the portable computer 4 functions as a support unit that supports the camera 1 and the video projection device 3 (also supports the position and orientation measurement device 2).

Reference numeral 5 denotes a highly accurate position of the portable computer 4 using information from the camera 1 and the position and orientation measurement device 2,
Based on the position and orientation calculation result of the portable computer 4 obtained from the position and orientation measurement unit 5, the position and orientation measurement unit 6 calculates and outputs the orientation, and 7 is based on the position and orientation calculation result 6 of the portable computer 4.
An object placement and shape database (object database) 10 to be described later is searched, which object is to be displayed with data, and what kind of information to present is searched from the projection information database 8 to be projected. A projection information search unit (projection information search means) for outputting information, 8 is a projection information database storing information to be presented to a user who carries the portable computer 4, and 9 is a position and orientation calculation result 6 of the portable computer 4. A video information generation unit that searches for an object arrangement to be projected, a shape database 10, calculates a relative positional relationship with the projection object, and converts and outputs video information output from the projection information search unit 7. (Image generation means). Reference numeral 10 denotes an object arrangement and object database having an object arrangement and shape. The portable computer 4 includes all the above processes and databases. The position and orientation measurement device 2 and the position and orientation measurement unit 5 correspond to a position and orientation measurement unit.

Next, the operation will be described. FIG. 2 is an image diagram for explaining the operation of the video display device according to this embodiment. The image display device is used for maintenance and monitoring of power plants, substation equipment, chemical plants, buildings, railways, roads, water and sewage systems, and the like. A user such as a worker or a patrolman who performs maintenance and inspection of a plant or the like carries a video display device and patrols the inside of the plant. In the example shown in FIG.
4 shows projected information in the operation of the video display device in opening and closing work on the valve 11. The user points the video display device at a target valve 11. At this time, the image display device uses one or both of the image information from the camera 1 and the information from the position and orientation measurement device 2 oriented in the direction of the valve 11, and the position of the portable computer 4,
Measure posture. Projection information search unit 7 of portable computer 4
Searches the object arrangement and shape database 10 based on the position and orientation calculation result 6 of the portable computer 4, and determines which object the data is to be displayed on. Further, the projection information database 8 is searched for what projection information is to be presented to the determined object, and the projection information is output to the video information generation unit 9. That is, the portable computer 4 determines what kind of object is present in the direction in which the user turns the portable computer 4, searches for information on the object, and projects the information by superimposing the information on the object. For this purpose, the relative positional relationship with the object and the shape of the object are calculated. Then, the video information generation unit 9 of the portable computer 4 converts the video information by the calculation so that the video information can be superimposed on the target and projected, and projects the video information on the target.

In the example shown in FIG. 2, the portable computer 4 recognizes the valve 11 from the image information of the valve 11 and the like, projects projection information about the valve 11 "turn the valve to the right", an arrow indicating the direction of turning the valve, An arrow indicating the direction in which the fluid flows is projected on the wall behind the valve, the valve, and the like.

As described above, in a plant or the like, a user such as a worker moves while carrying the portable computer 4. When the user directs the portable computer 4 to illuminate a flashlight or the like attached to the portable computer 4 to a part of interest or an object that the user wants to know in detail while traveling, the portable computer 4 can position itself, It calculates which object is interesting from the information of the camera, and displays the information corresponding to the object on the object. By simply pointing the portable computer at an object of interest, the user can project information about the object onto the object and obtain necessary information without moving his / her eyes. Of users can share the information at the same time. In addition, various information can be obtained in a natural manner while looking at the target object, and additional maintenance work can be reduced. Further, as will be described later in another embodiment, the portable computer 4 at the site and the central computer on the central side provided in the control room or the like are configured to be able to communicate via a communication device such as a wireless device. You may. In this case, a person who operates a central computer in the control room can give an easy-to-understand instruction and advice to a user at the site by controlling the projection function. In addition, by adding information about the traveling route and the dangerous area to the projection information, the user can intuitively and without overlooking the traveling route and the dangerous area.

Further, unlike the conventional video display device, the user does not need to wear the video display device on his / her head.
Physical load on the user can be reduced.

Embodiment 2 FIG. FIG. 3 is an explanatory diagram showing a configuration of a video display device according to another embodiment of the present invention. The video display device according to this embodiment has substantially the same configuration as that of FIG. 1, except that the video projection device 3, the camera 1, and the portable computer 4
Is not carried by the user and is attached to a ceiling, a wall, or the like.

The video display apparatus according to the first embodiment judges the user's interest from the position and the posture of the portable computer. However, the video display apparatus according to the present embodiment uses the finger of the user instead of using this. Using a process of acquiring an action indicating a target from a camera and specifying a target of interest of the user, a video processed by a computer is projected on the target of interest by superimposing the image on the target of interest of the user. Also, instead of pointing at the user,
Give the user something like a stick attached to the position measuring device, and use the processing to identify the object of interest of the user from the position and orientation of the stick, such as identifying the target object, using the process of identifying the user's interest. The image processed by the computer is projected on the object.

FIG. 4 is a system configuration diagram of a video display apparatus according to this embodiment. In the figure, the same or corresponding components as those of the first embodiment shown in FIG. 21 is a calculation result of the position and orientation of the camera 1, 22 is a target specifying unit of the user's interest, 23 is a pan head control unit, 24 is a cloud for supporting the camera 1, the image projection device 3, etc. and directing them toward the object. The table 25 is a computer corresponding to the portable computer 4 in FIG. In this embodiment, unlike the first embodiment, the mounting positions of the camera 1, the position and orientation measuring device 2, and the video projecting device 3 are not a portable computer but a pan head controlled by a computer mounted on a wall or the like. 24. That is, the head 24
It corresponds to the support means.

Next, the operation will be described. The position and orientation measurement unit 5 calculates the position and orientation of the camera 1 based on information from the camera 1 and the position and orientation measurement device 2, and outputs the result as a position and orientation calculation result 21. User interest target specifying unit 22
Performs a process of specifying an object of interest of the user from the action pointed by the user as described above, and searches the projection information database 8 for information to be projected. The video information generation unit 9
The relative position information between the camera and the object is calculated based on the information on the object of interest of the user, and the information to be projected can be displayed on the object based on the relative position information and the shape of the object. To convert information to be projected. The camera platform control unit 23 controls the camera platform 24 so that the camera always faces the direction of the object of interest to the user.

According to this embodiment, the same effects as those of the first embodiment can be obtained. In addition, the user can simply point at the object of interest without carrying the computer, and can obtain the object. In addition to projecting information about the object, necessary information can be obtained without moving the line of sight, and a plurality of users can share the information at the same time.

Embodiment 3 FIG. FIG. 5 is a block diagram showing a configuration of a video projection device in a video display device according to another embodiment of the present invention. In the drawing, the same or corresponding components as those of the first embodiment shown in FIG. Is attached, 31 is a visible laser light generator that generates visible laser light, 32 is a mirror, 33 is an axis that rotates the mirror about a horizontal axis, 34 is an axis that rotates the mirror about a vertical axis, Reference numeral 35 denotes a laser light generation control device that controls on / off of the visible laser light generation device 31, and 36 denotes a mirror surface control that combines a drive device such as a motor installed on each of the shafts 33 and 34 and a device that controls the drive device. Device.

Next, the operation will be described. The video information generated by the video information generation unit 9 of the video display device is transmitted to the mirror 32 by the operation of the axes 33 and 34 controlled by the mirror surface control device 36.
, And the light beam from the visible laser light generator 31 is reflected to an arbitrary place, and the image information is projected. The laser light from the visible laser light generator 31 is turned on and off by the laser light generation controller 35. With these controls, the video projection device 3 can project an arbitrary character or graphic at an arbitrary location around an object such as a plant.

Embodiment 4 FIG. FIG. 6 is a block diagram showing a configuration of an internal state estimation / display function of a video display device according to another embodiment of the present invention. FIG. 6 shows a function of adding information to the projection information database 8 in FIG. 1. The video display device according to this embodiment functions by combining FIG. 1 and FIG. 6, the same or corresponding components as those of the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and reference numeral 41 denotes a site or a site environment measured by a measurement sensor attached to the portable computer 4. An on-site environment measuring unit, 42 is an internal state estimation database including a dynamic characteristic model for estimating the internal state of an unmeasurable target, and 43 is an unmeasurable using the on-site environment measurement data and the internal state estimation database An internal state estimating unit for estimating an internal state of a target; a user input / output unit for performing a dialogue with a user to determine which part of the internal state data is to be projected;
Is the internal state estimation database 42 and the internal state estimation unit 43
And a central computer including: The portable computer 4 and the central computer 45 are connected by communication means.
This communication means is not shown in FIG.

Next, the operation will be described. Hereinafter, the function of the image display device according to the present embodiment will be described by taking as an example a case where the temperature distribution state of the coolant inside the reactor is displayed.

From the site environment measuring section 41, the central computer 4
In 5, measurable data such as reactor inlet temperature, coolant flow rate, reactor output (neutron flux) and the like are input, and the reactor thermal characteristic equation and the reactor nuclear characteristic equation stored in the internal state estimation database 42. The internal state estimating unit 43 calculates the temperature distribution of the coolant inside the reactor using the reactor dynamic characteristic model configured as described above. Next, the user input / output unit 44 knows the internal position of the reactor (for example, the distance from the entrance) where the internal state is to be displayed, obtains the corresponding coolant temperature from the calculation result, and obtains the result via the communication means. The information is transmitted to the portable computer 4 and added to the projection information database 8 in FIG. In the reactor equipped with the reactor internal coolant temperature sensor, the internal state estimation function is skipped, and only the internal state display screen is edited.

As a result, the internal state of the object that cannot be directly seen by the user can be projected onto the object and its surroundings, and the image can be presented intuitively to the user.
Decision making can be accurately performed when a change in the maintenance method or operation method is required according to the internal state of the target. For example, if the temperature of the nuclear fuel is too high, it is highly likely that the fuel has been damaged, and therefore, after the operation is stopped, the inspection work on the reactor fuel is performed immediately.

Embodiment 5 FIG. FIG. 7 is a block diagram showing a configuration of a display function of a device maintenance method and an operation method in a video display device according to another embodiment of the present invention. FIG. 7 shows a function of adding information to the projection information database 8 in FIG. 1. This embodiment functions by combining FIG. 1 and FIG. In FIG. 7, the same or corresponding components as those of the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and reference numeral 41 denotes a site or a site environment measured by a measurement sensor attached to the portable computer 4. The on-site environment measuring unit, 51 is a maintenance method determination database constructed based on maintenance procedures and know-how such as inspection and repair, a past repair history, and the like, and 52 is constructed based on operation procedures and know-how of individual devices on the site. An equipment operation method determination database 53 uses a site environment data and a maintenance method determination database 51 to determine a maintenance procedure and method suitable for the current situation, and a maintenance method determination unit 54 uses a site environment data and an operation method determination database 54. The operation method determining unit 55 for searching and determining an operation method suitable for the current situation from the operation method determination database 52, and the determined maintenance method Characters for display to the user a maintenance mode display database consisting of drawings and the like, characters for displaying the determined operating method to the user 56, instructions display database consisting of drawings and the like,
Reference numeral 57 denotes a maintenance method display database 55 at the time of maintenance, and a database search and transfer unit which searches the operation method display database 56 at the time of operation based on the maintenance method or the operation method and writes information into the projection information database 8, and 58 denotes a maintenance method determination unit. It is a central computer including an operation method determining unit, a database search and transfer unit, and the like.

Next, the operation will be described. Hereinafter, the function of the image display device according to this embodiment will be described by taking as an example a case where a maintenance method in a nuclear power plant is determined and displayed.

[0059] Measurement data is input from the on-site environment measurement unit 41 to grasp the current plant state.
Using the maintenance method determination database 51 composed of HEN (Ifusen) rules, the maintenance method determination unit 53 determines the following information on the maintenance method.

(1) Priority of equipment to be maintained (2) Prerequisites for performing maintenance (for example, system isolation) feasibility (3) Maintenance method for each equipment (for example, disassembly method, inspection method, Parts replacement method, order of each work)

Next, based on these pieces of information, the database search and transfer section 57 searches the maintenance method display database 55 and transfers the information to the projection information database 8.
Thereafter, according to the configuration of FIG. 1, the video information generation unit 9 of the portable computer 4 projects the image on the target. At the time of operating the device, information on the operation is projected onto the operating device and its surroundings in the same manner.

In this way, a maintenance method and an operation method suitable for the current situation are determined, the results are summarized on a display screen which is easy for the user to understand, projected on the object and its surroundings, and the video information is presented to the user. Therefore, maintenance work and operation work required on site can be reliably executed.

Embodiment 6 FIG. FIG. 8 is a block diagram showing a configuration of a site environment display function in a video display device according to another embodiment of the present invention. FIG. 8 shows a function of adding information to the projection information database 8 in FIG. 1. This embodiment functions by combining FIG. 1 and FIG.
In FIG. 8, the same or corresponding components as those of the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and 41 is a site or a site environment is measured by a measurement sensor attached to the portable computer 4. The on-site environment measuring unit 61 uses an input / output device such as a pointing device or a display of the portable computer 4 to display a type of a sensor and a display method (isobars, intervals of isoconcentration lines, etc.) used for display.
Is a user information input / output unit that interactively obtains information from the user, and a site 62 that converts the site environment data obtained from the sensor and the spatial distribution of the information obtained from the sensor according to the user's request into an image. Environmental Information Converter, 6
Reference numeral 3 denotes a database transfer unit that writes the site environment information converted so that it can be expressed in an image into the projection information database 8.

Next, the operation will be described. Hereinafter, the function of the image display device according to this embodiment will be described by taking as an example a case where the on-site environment is always visualized during maintenance work in a nuclear power generation facility.

The site environment measuring section 41 measures the site environment by using a measuring sensor attached to the site or the portable computer 4. The measured site environment data is converted into an image by the site environment information conversion unit 62 in accordance with the user's request (what to display and in what display format) obtained by the user information input / output unit 61. The data is converted into a form that can be expressed and sent to the database transfer unit 63. The information is written into the projection information database 8 by the database transfer unit 63, and thereafter, according to the configuration of FIG.
The video information generation unit 9 of the portable computer 4 projects such information on a wall or floor around the object.

In this way, the radiation, temperature, and gas concentration that cannot be perceived by the human being are measured by the on-site environment measuring unit 41 and displayed in a visual form, so that the user and the human being around the user can be effectively displayed. You can avoid danger.

Embodiment 7 FIG. FIG. 9 is a diagram showing a method for determining a dangerous area / moving route in a video display apparatus according to another embodiment of the present invention.
FIG. 3 is a block diagram illustrating a configuration of a display function. FIG. 9 shows FIG.
1 shows a function of adding information to the projection information database 8, and this embodiment functions by combining FIG. 1 and FIG. In FIG. 9, the first embodiment shown in FIG.
The same reference numerals are given to the same or corresponding parts as the components of the above, 41 is a site environment measuring unit for measuring the site environment by a measuring sensor attached to the site or the portable computer 4, and 71 is a user's action area. , A dangerous area determination database composed of dangerous area determination rules based on position information of an opening having a low overhead and on-site environmental data;
2 is a moving route determination database composed of moving route determination rules, 73 is a dangerous area determining unit that determines a dangerous area suitable for the current situation using the site environment data and the dangerous area determining database 71, and 74 is a site environment data. A moving route determining unit that determines a moving route suitable for the current situation using the moving route determining database 72; 75, a dangerous area display database including characters, drawings, and the like for displaying the determined dangerous area; and 76, a determined moving path. Is a moving route display database made up of characters, drawings, etc. for displaying the information. A database search and transfer unit 77 searches both the danger area display database 75 and the moving route display database 76 and writes information into the projection information database 8. ,
Reference numeral 78 denotes a central computer including a dangerous area determination unit 73, a movement route determination unit 74, and a database search and transfer unit 77.

Next, the operation will be described. Hereinafter, the function of the image display device according to this embodiment will be described by taking, as an example, a case where a moving route is determined and displayed in a nuclear power generation facility.

The moving route determining section 74 determines a moving route according to the following steps.

(1) The measurement data is input from the on-site environment measurement unit 41 to grasp the current state of the nuclear power generation facility.

(2) Using the movement prohibition rules stored in the movement route determination database 72, the movement prohibition area is obtained without omission.

(3) A movable area is obtained by excluding the current prohibited area from the entire movable area (an area that can be moved if the conditions are satisfied).

(4) The moving route is determined so that the moving distance is the shortest in the movable area.

Next, the database search and transfer section 77 searches the movement route display database 76 for an appropriate display according to the dangerous area display movement route, and transfers the display information to the projection information database 8. According to
The video information generator 9 of the portable computer 4 projects information on the target. Also in the dangerous area display, the dangerous area determining unit 73 obtains the dangerous area and projects and displays the dangerous area in the same procedure as described above, so that the user can be informed.

As described above, the video display apparatus according to the present embodiment determines the danger area / moving route suitable for the current situation, and collects the determined danger area / moving route on the display screen which is easy for the user to understand, and displays the target area and its surroundings. Since the image is projected and the video information is presented to the user, the user can take an action required on site accurately and promptly.

Embodiment 8 FIG. FIG. 10 is a block diagram showing the overall configuration of a video display system according to another embodiment of the present invention. In the figure, the same or corresponding parts as those of the first embodiment shown in FIG. Reference numeral 81 denotes a central computer, 82a and 82b denote communication devices for exchanging data between the portable computer 4 and the central computer 81, and 83 denotes an image of the site taken by the camera 1 installed in the portable computer 4. A remote display device (remote display means) 84 for displaying on the central computer 81, a remote device 84 for projecting information input by a pointing device such as a pen or a mouse to the site using the video projection device 3 installed in the portable computer 4. An input device (remote input means). Reference numeral 85 denotes a display data conversion unit that converts data transmitted by the communication device into a format that can be displayed on the central computer, and 86 denotes a video display unit that displays the converted video data. The remote display device 83 includes a display data conversion unit 85 and a video display unit 86.
Reference numeral 87 denotes an input device, such as a pen or a mouse, which is input by an operator on an image on the image display unit 86. The remote input device 84 includes an input device 87 and an input data conversion unit 88. In this way, the image of the site is displayed on the central computer 81 at a remote location, and the input from the central computer 81 is sent to the portable computer and projected on the site, so that the worker on the site and the central Communication of the operator's intention becomes easy.

Next, the operation will be described. FIG. 11 is an explanatory diagram showing a data communication method between the portable computer 4 and the central computer 81. First, a worker at the site installs the camera 1 and the video projection device 3 on the portable computer 4, and turns the camera 1 and the video shooting device toward a target object.
The portable computer 4 displays an image captured by the camera 1 on the portable computer 4 and simultaneously transmits digital data of the image to the central communication device 82b via a communication device 82a such as a wireless device at the site. The central computer 81 receives the radio wave via the communication device 82b, converts the digital data into image data, and displays the image data on the image display device. The central computer 81 converts image data such as handwritten information input on the central side into digital data, and transmits the digital data to the communication device 82a at the site via the central communication device 82b. The portable computer 4 can display the information input from the central computer 81 on the site by projecting the transmitted data to the site using the video projecting device 3 installed in the portable computer 4. it can.

Next, the remote display device 83 and the remote input device 8
Operation 4 will be described with reference to the flowchart shown in FIG. First, the image is transmitted to the communication device 82b on the central side via the communication device 82a on the site photographed by the camera installed on the central computer 81 (step ST80). The display data conversion unit 85 obtains data from the communication device 82b and converts the data into image data that can be displayed on the central computer 81 (step ST81). The converted image data is displayed on the video display unit 86 of the remote display device 83 (step S
T82). Next, handwriting, graphics, and the like are input from the input device 87 of the central computer 81 on the video sent from the site.
When pointer information or the like is input (step ST83), the input image data is converted into digital data by the input data conversion unit 88 (step ST8).
4). The converted data is transmitted to the communication device 82a connected to the portable computer 4 via the communication device 82b (step ST85), and the input data is projected to the site by the video projection device 3 installed in the portable computer 4. (Step ST
86).

As described above, according to the video display system of this embodiment, the video of the site is displayed on the central computer 81.
And the input from the central computer 81 is projected to the site, so that the worker at the site and the operator at the central side can easily communicate with each other.

[0080]

As described above, according to the first aspect of the present invention, the image should be projected using the image projection means on the basis of the information obtained from the position and orientation meter side means and the image obtained from the photographing means. Projection information search means for searching the projection information database and the object database for video information and the arrangement and shape of the object with respect to the image, based on the relative positional relationship between the image projection means and the object and the shape of the object, And video generating means for generating a video to be projected from the retrieved video information, so that the necessary video information generated by the video generating means is superimposed on the object within the field of view of the user who operates the video display device. Since the information is displayed, the user can make correspondence between the presented information and the target object without moving his / her gaze, and can perform operations such as maintenance and inspection accurately and easily. There is an effect that it is possible to see the video information in a human. Furthermore, there is an effect that the operator can intuitively understand the correspondence between the information such as the drawings and the actual state of the equipment in the case of maintenance inspection of the plant or the like. Further, since the worker does not need to wear the video display device on the head, there is an effect that the physical load on the worker can be reduced.

According to the second aspect of the present invention, there is provided a portable computer which can be carried by a user, and the portable computer functions as a support means, and furthermore, a position and orientation measuring means, a projection information database means, an object The configuration including the database unit, the projection information search unit, and the video generation unit has an effect that the user can easily carry it.

According to the third aspect of the present invention, the internal state of an unmeasurable object estimated using an internal state estimation database including a dynamic characteristic model and the like can be measured but the object cannot be directly viewed by the user. Visualizing at least one of the internal state of the object and the image projected onto the object and its surroundings, the apparatus further comprises means for adding the visualized image. Thus, when a change in the maintenance method and the operation method is required, there is an effect that a user such as a worker can accurately make a decision.

According to the fourth aspect of the present invention, the maintenance and operation method determination database storing the maintenance method and operation method of the equipment including the object, and the maintenance and operation method suitable for the photographed object are maintained and operated. Means for determining using the method determination database and adding an image relating to a maintenance method or an operation method to the image projected on the object and its surroundings. Thus, there is an effect that the maintenance work and the operation work to be performed can be surely executed.

According to the fifth aspect of the present invention, radiation, temperature,
A sensor for gas concentration and the like, and a means for obtaining a spatial distribution of information that cannot be perceived by humans, such as radiation, temperature, gas concentration, etc. obtained from the sensor, and adding an image relating to the spatial distribution to an image projected on an object and its surroundings. Is provided so that the user and the people around it can be avoided from danger in advance.

According to the sixth aspect of the present invention, the video information that supports the user's actions such as the guidance of the dangerous area and the moving route for the user is determined using the dangerous area and the moving route determination database, and the object and its surroundings are determined. Since the apparatus further comprises means for adding an image related to the determined support information to the image to be projected on the work site, there is an effect that the user can take an action required at the work site accurately and promptly.

According to the seventh aspect of the present invention, by controlling the optical axis of the visible laser light, an image, a moving image, and a moving image can be displayed in real time according to the relative position of the image projecting means and the object and the shape of the object. Alternatively, since the apparatus is configured to include means for projecting a character, there is an effect that a figure corresponding to the shape of the object can be easily drawn.

According to the invention of claim 8, based on the information obtained from the position and orientation meter side means and the image obtained from the photographing means, the video information on the video to be projected by the video projection means and the object Projection information retrieval means for retrieving the arrangement and shape of the object from the projection information database and the object database, and projecting from the retrieved image information based on the relative positional relationship between the image projection means and the object and the shape of the object. A portable computer comprising an image generation unit for generating an image,
A central computer communicable with the portable computer via a communication means, a remote display means for displaying an image of the object photographed by the photographing means provided in the portable computer, and a video projection of the input information inputted. A central computer having remote input means for processing input information to project the object and its surroundings by using the means, so that a worker at the site carrying the portable computer and the central computer There is an effect that the operator of the computer can easily communicate.

According to the ninth aspect of the present invention, by controlling the optical axis of the visible laser light, an image, a moving image, Alternatively, since the apparatus is configured to include means for projecting a character, there is an effect that a figure corresponding to the shape of the object can be easily drawn.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating a video display device according to an embodiment of the present invention.

FIG. 2 is an image diagram for explaining an operation of the video display device shown in FIG.

FIG. 3 is an explanatory diagram showing a configuration of a video display device according to another embodiment of the present invention.

FIG. 4 is a configuration diagram showing the video display device shown in FIG.

FIG. 5 is a configuration diagram showing an image projection device of an image display device according to another embodiment of the present invention.

FIG. 6 is a configuration diagram showing a part of a video display device according to another embodiment of the present invention.

FIG. 7 is a configuration diagram showing a part of a video display device according to another embodiment of the present invention.

FIG. 8 is a configuration diagram showing a part of a video display device according to another embodiment of the present invention.

FIG. 9 is a configuration diagram showing a part of a video display device according to another embodiment of the present invention.

FIG. 10 is a configuration diagram showing a video display system according to another embodiment of the present invention.

11 is an explanatory diagram showing a method of using the video display system shown in FIG.

12 is a flowchart showing a flow of processing of the video display system shown in FIG.

FIG. 13 is a configuration diagram showing a portable computer in a conventional maintenance patrol work support system.

FIG. 14 is an explanatory diagram showing an example of a screen of a display unit of the portable computer shown in FIG.

FIG. 15 is a configuration diagram showing another example of a conventional maintenance patrol work support system.

FIG. 16 is a configuration diagram showing a conventional video display device.

[Explanation of symbols]

1 camera (photographing means), 2 position and orientation measuring device (position and orientation measuring device), 3 image projecting device (image projecting device), 4
Portable computer, 5 position and orientation measurement unit (position and orientation measurement unit), 7 projection information search unit (projection information search unit), 8
Projection information database, 9 video information generation unit (video generation means), 10 object arrangement, shape database (object database), 42 internal state estimation database, 5
1 Maintenance method determination database, 52 Operation method determination database, 71 Danger area determination database, 72
Moving route determination database, 81 Central computer, 83
Remote display device (remote display means), 84 Remote input device (remote input means).

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-7-44108 (JP, A) JP-A-7-56624 (JP, A) JP-A-3-63208 (JP, U) (58) Survey Field (Int.Cl. ⁷ , DB name) G01B 11/00-11/30 G05B 23/02

Claims (9)

(57) [Claims] 1. An image capturing means for capturing an image of an object, an image projecting means for projecting an image on and around the object imaged by the image capturing means, and moving the image capturing means and the image projecting means. Using at least one of a supporting means for supporting as possible, an image obtained from the photographing means, and a position measuring device such as a magnetic sensor, a gyro, an ultrasonic sensor, etc., supported by the supporting means. Position and orientation measurement means for measuring the position and orientation of the support means, a projection information database for storing video information relating to images projected by the video projection means, and storing arrangement, orientation and shape information of a plurality of objects. An image to be projected using the image projecting means based on an object database, information obtained from the position and orientation meter side means, and an image obtained from the photographing means. Projection information searching means for searching the projection information database and the object database for video information relating to an image and the arrangement and shape of the object, and the relative positional relationship between the image projecting means and the object and the shape of the object. And a video generating means for generating a video to be projected from the retrieved video information based on the video information. 2. The image display device includes a portable computer that can be carried by a user, the portable computer functioning as the support unit, and the position and orientation measurement unit, the projection information database unit, The image display device according to claim 1, further comprising the object database unit, the projection information search unit, and the image generation unit. 3. The image display device according to claim 1, wherein the internal state of the unmeasurable target estimated by using an internal state estimation database including a dynamic characteristic model and the target of a target that can be measured but cannot be directly viewed by a user. 3. The image processing apparatus according to claim 1, further comprising: a unit that visualizes at least one of the internal state and adds the visualized image to an image projected on an object and its surroundings.
3. The video display device according to 1. 4. The image display device according to claim 1, further comprising: a maintenance and operation method determining database storing a maintenance method and an operation method of the device including the object; and a maintenance and operation method suitable for the photographed object. 3. A means for determining using the operation method determination database and adding an image relating to the maintenance method or the operation method to an image projected on the object and its surroundings, further comprising: The image display device according to the above. 5. The image display device according to claim 1, further comprising a sensor for detecting radiation, temperature, gas concentration, and the like;
Means for obtaining a spatial distribution of information that cannot be perceived by humans, such as temperature and gas concentration, and adding an image relating to the spatial distribution to an image projected on an object and its surroundings. The video display device according to claim 2. 6. The image display device determines, using a risk area and a movement route determination database, image information that supports the user's actions such as a danger area for the user and guidance of a movement path, and places the information on an object and its surroundings. 3. The image display device according to claim 1, further comprising: a unit for adding an image relating to the determined support information to an image to be projected. 7. The image projection unit controls an optical axis of a visible laser beam, so that an image, a moving image, or a video is displayed in real time according to a relative position of the image projection unit and an object and a shape of the object. 7. The video display device according to claim 1, further comprising a unit for projecting a character. 8. A portable computer comprising: a photographing means for photographing an object; and an image projecting means for projecting an image around the object photographed by the photographing means and around the object. Of measuring the position and orientation of the portable computer using at least one of the image obtained from the above and a position measuring device such as a magnetic sensor, a gyro, an ultrasonic sensor, etc. supported by the portable computer. Attitude measurement means, a projection information database that stores video information related to images projected by the video projection means, an object database that stores arrangement, attitude and shape information of a plurality of objects, and the position and orientation meter side means Based on the obtained information and the image obtained from the photographing means, the image information on the image to be projected by the image projecting means and the arrangement and shape of the object Projection information searching means for searching the projection information database and the object database for images, and an image projected from the searched image information based on the relative positional relationship between the image projecting means and the object and the shape of the object. A portable computer comprising: a video generating means for generating an image; and a central computer communicable with the portable computer via a communication means, the object being photographed by the photographing means provided in the portable computer. A central computer having remote display means for displaying an image of an object, and remote input means for processing the input information so that the input information is projected onto the object and its surroundings using the image projection means; Video display system. 9. The image projection means controls an optical axis of a visible laser beam, so that an image, a moving image, or a video is displayed in real time in accordance with the relative position of the image projection means and an object and the shape of the object. The video display system according to claim 8, further comprising means for projecting characters.