JPH0886615A - Image display apparatus and image display system - Google Patents

Abstract

PURPOSE: To allow a plurality of workers to view data related to the inspection at the same time by generating an image to be projected from retrieved image data based on the relative positional relationship between an image-projecting means and an object and the shape of the object. CONSTITUTION: Either or both of image data from a camera 1 and data from a position/posture-measuring device 2 are used to measure the position and posture of a portable computer 4. A projection data-retrieving part 7 of the computer 4 retrieves an arrangement/shape database 10 based on a position/ posture calculation result 6 of the computer 4, thereby judging for which of objects data are to be displayed. Further, the retrieving part 7 retrieves from a projection data database 8 what kind of projection data is to be given to the detected object, and outputs the projection data to an image data-generating part 9. The generating part 9 of the computer 4 converts the image data by calculations so as to project the image data in an overlapping state on the object, and projects the data to the object via an image-projecting device 3.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video display device and a video display system for visualizing information and presenting it to a user by using a portable computer or the like.

[0002]

2. Description of the Related Art In large-scale plants represented by nuclear power plants, roads, railways, aircrafts, and water and sewer systems, regular maintenance work is performed in order to provide stable service. In particular, in each of the above systems, it is required to carry out enormous inspection items safely and surely. Therefore, the workers who perform these tasks require special skills and experience, but it is necessary to work in a dangerous and contaminated environment. In recent years, it has become difficult to grow them. As a means for solving these problems, a computer system has been developed that supports efficient and easy maintenance work.

FIG. 13 is a block diagram showing a computer system (patrol inspection system) for supporting the conventional maintenance work disclosed in Japanese Patent Laid-Open No. 03-143204.
In the figure, 101 is a portable computer, 102 is an image display unit, and 103 is a key input unit. This system aims to improve the work efficiency of workers and prevent inspection omissions.

Next, the operation will be described. In this patrol inspection system, the portable computer 1 as shown in FIG.
01 is carried by the worker. In the inspection inspection system, the device name and inspection item of the inspection target device are displayed on the screen of the image display unit 102 of the portable computer 101 as shown in the screen example of FIG. 14, and the display of the inspection completed device is reversed and displayed. To do.

FIG. 15 is a block diagram showing the construction of a patrol / inspection support device for a field terminal disclosed in Japanese Patent Laid-Open No. 5-89392, in which 201 is a substation device which is a patrol target device. 202 is a computer for patrol inspection, 2
Reference numeral 03 is a portable terminal. This patrol and inspection support device
The purpose is to improve the work efficiency of patrols and inspections and reduce the load on patrols.

Next, the operation will be described. The inspection inspection computer 202 is installed in the control room, and the knowledge base 22
2. Monitoring data 221, treatment guidance 223, additional inspection / inspection item data 224, and diagnostic means 226 for judging the device status from these information are provided. Display the inspection item and the treatment guidance on the portable terminal 203 at the site that can communicate with the computer via the transmission means. The patrolman patrols the substation device 201 according to the displayed patrol / inspection items.
Check and enter the result via the portable terminal 203,
The portable terminal 203 sends the result to the inspection inspection computer 202.
Send to. The inspection inspection computer 202 diagnoses the transmitted inspection / inspection, and if additional inspection / inspection is necessary, the additional inspection / inspection item is added to the diagnosis result to carry the portable terminal 20.
Send to 3.

FIG. 16 is a block diagram showing the structure of the visual information display device disclosed in Japanese Patent Laid-Open No. 5-1991839, in which 330 is a liquid crystal display device, 340 is a reflection mirror, and 350 is a half mirror. . This device
It is not directly related to the maintenance work of equipment such as plants, but displays the stereoscopic three-dimensional image information such as CG (computer graphics), TV image, and video image on the landscape that the user can see with the naked eye. The purpose is to let.

In FIG. 16, in order to present three-dimensional image information, an image to be projected on the eye is displayed on the liquid crystal display device 330, and the image is projected on the half mirror 350 through the optical system such as the reflection mirror 340. . The user can see the outside scenery and the image of the liquid crystal display device 330 on the half mirror 350 in an overlapping manner. By preparing two sets of this mechanism and projecting them to the left and right eyes, the landscape and the CG, TV image, video image, etc. can be superimposed three-dimensionally.

[0009]

Since the conventional video display system is configured as described above, the guidance for the patrol route, device operation, check items, and drawings of the target device are displayed on the portable computer in the field. However, it is easy to carry around the inspection route, guidance for device operation, check items, drawings of the target device, etc. drawn on paper as a portable computer and reduce the work of searching. Therefore, there is a problem in that the burden on the worker cannot be substantially reduced. That is,
When working with these devices, the worker has to look back and forth between the inspection / repair target device and the screen on which the information is displayed. In addition, no matter how much useful information is presented on the screen of the portable computer, if the worker fails to look at the surface of the portable computer, it does not mean that this system is supporting. In fact, considering that some kind of inspection work is performed using such a support system, it is a burden on the worker to check the guidance information displayed on the screen in detail and then inspect and repair the target device. This will increase the number of users and even make them neglect to look at the screen.

Further, when a drawing concerning the target device is presented on the screen of the portable computer, the worker considers which part of the drawing corresponds to which part of the inspection / repair target device, and then the head. There is also a problem in that it is necessary to superimpose the inspection / repair device and the drawing on the inside, and an operator's error may occur in this part.

As described above, Japanese Patent Application Laid-Open No. 3-14320
When using a maintenance / patrol work support system using a portable computer, such as Japanese Patent Laid-Open No. 4 and Japanese Patent Laid-Open No. 5-89392, the operator is forced to move his / her line of sight, and the inspection / repair target device, Workers must superimpose the information presented on the screen. As one method for solving this problem, an image display device mounted on the head as in the above-mentioned Japanese Patent Laid-Open No. 5-1991839 is used, and operation guidance for the inspection / repair target device, check items,
It is conceivable that the drawings for the target device may be displayed in an overlapping manner on the target device for inspection / repair and its surroundings. Using this method, an operator can obtain information such as guidance and drawings created by a computer without diverting the line of sight from the inspection / repair target device.

However, when such a head-mounted image display device shown in FIG. 16 is used, the optical system devices such as the liquid crystal display device 330, the reflection mirror 340, and the half mirror 350 are attached to the operator's head. The operator must support these weights with his / her neck while working, and feels uncomfortable due to the presence of a mirror or the like in the field of view. In addition, when these devices are mounted on a helmet and the like, the worker wears them on the head, but at this time, it is impossible to fix the helmet to the head completely. , The relative position between the eyeball and the optical system is not constant, and the CG projected on the half mirror 350,
Images such as videos will shake. There are also research reports that such shaking causes workers to experience the same symptoms as car sickness. For these reasons, it is not realistic to perform maintenance work such as inspection and repair for a long time by using the head-mounted visual information presenting device because it imposes a great burden on the operator. In addition, the inspection / repair work is usually performed by a plurality of workers, but with a support system using this head-mounted image display device, information can be presented only to the wearing worker. There was a problem.

The invention of claim 1 has been made to solve the above-mentioned problems, and a plurality of workers are provided in order to reduce the burden on the workers at the time of work inspection in plant inspection / repair work. It is an object of the present invention to obtain an image display device that allows the user to see the information regarding the inspection at the same time and does not have to move the line of sight from the inspection / repair target device.

It is an object of the present invention to provide a video display device which can reduce the burden on a worker at the time of work inspection in a plant inspection / repair work and can be carried by the worker. .

According to a third aspect of the present invention, there is provided a video display device which enables a worker to make a precise decision when a maintenance method or an operation method needs to be changed depending on the internal state of a target. With the goal.

It is an object of the present invention to provide a video display device capable of surely executing maintenance work and operation work required at a work site.

It is an object of the present invention to provide an image display device capable of avoiding danger to workers and humans around them in advance.

It is an object of the present invention to provide a video display device which enables a worker to accurately and swiftly perform an action required at a work site.

It is an object of the present invention to provide a video display device capable of easily drawing a graphic corresponding to the shape of an object.

It is an object of the present invention to provide a video display system in which a worker on the work site and an operator on the center side can easily communicate with each other.

It is an object of the present invention to provide a video display system capable of easily drawing a graphic corresponding to the shape of an object.

[0022]

According to a first aspect of the present invention, there is provided a video display device for projecting by using a video projecting means based on information obtained from the position / orientation side means and an image obtained from the photographing means. Projection information search means for searching the projection information database and the object database for the video information about the video to be displayed and the arrangement and shape of the object, and based on the relative positional relationship between the image projection means and the object and the shape of the object. And a video generation means for generating a video to be projected from the searched video information.

The video display device according to the invention of claim 2 is provided with a portable computer which can be carried by a user, and the portable computer functions as a supporting means, and also has a position and orientation measuring means and a projection information database. It includes means, object database means, projection information search means, and image generation means.

In the image display device according to the third aspect of the present invention, the internal state of the unmeasurable object estimated by using the internal state estimation database including the dynamic characteristic model and the direct measurement by the user although it is measurable. It is further provided with a means for visualizing at least one of the internal states of the target that cannot be processed and adding the visualized video to the video projected on the target and its surroundings.

A video display device according to a fourth aspect of the present invention includes a maintenance and operation method determination database that stores a maintenance method and an operation method of a device including an object, and a maintenance method or an operation method suitable for a photographed object. It further comprises a means for making a decision using a maintenance and operation method decision database and adding an image relating to the maintenance method or the operation method to the image projected on the object and its surroundings.

According to a fifth aspect of the present invention, there is provided a video display device, which calculates a sensor for radiation, temperature, gas concentration, etc., and a spatial distribution of information such as radiation, temperature, gas concentration, etc., which cannot be perceived by humans, It further comprises means for adding an image relating to the spatial distribution to the image projected onto the object and its surroundings.

The image display device according to the invention of claim 6 determines the image information for supporting the user's actions such as the danger area for the user and the guidance of the movement route by using the danger area and movement route determination database, and the target object. And a means for adding an image relating to the determined support information to the image projected on the periphery thereof.

The image projection means of the image display device according to the seventh aspect of the present invention controls the optical axis of the visible laser light so as to respond to the relative position of the image projection means and the object and the shape of the object. It also includes means for projecting an image, a moving image, or a character in real time.

In the video display system according to the invention of claim 8, the video information on the video to be projected by the video projection means is based on the information obtained from the position and orientation meter side means and the video obtained from the photographing means. And projection information retrieval means for retrieving the position and shape of the object from the projection information database and the object database, and image information searched based on the relative positional relationship between the image projection means and the object and the shape of the object. A central computer that is capable of communicating with a portable computer through a communication unit, the portable computer having an image generation unit that generates an image to be projected from, and is photographed by a photographing unit installed in the portable computer. A remote display means for displaying an image of the target object and a remote input means for processing the input information so as to project the input information input to the target object and its surroundings by using the image projection means. Those with a central side computer which.

The image projection means installed in the portable computer in the image display system according to the ninth aspect of the present invention controls the optical axis of the visible laser light to determine the relative position of the image projection means and the object. It includes means for projecting an image, a moving image, or characters in real time according to the shape of the object.

[0031]

In the image display device according to the present invention, the projection information retrieval means displays the information obtained from the position and orientation meter side means, that is, the position and orientation of the supporting means, and the image of the object obtained from the photographing means. On the basis of the image projection means and the object database, the image information about the image to be projected around the object including the object and the arrangement and shape of the object are searched from the projection information database and the object database. An image to be projected is generated from the searched image information based on the relative positional relationship between the object and the shape of the object. The image is projected onto the object and its surroundings via the image projection means. Thereby, in the maintenance and inspection of the plant, for example, the image of the equipment including the target object, the operation guidance stored as a database, the image information about the device drawing, etc., can be directly inspected and repaired by the image projection means and the surrounding area. By projecting the image onto the device, a user such as a worker does not need to move his / her line of sight from the inspection / repair target device, and a plurality of workers can see the information regarding the inspection at the same time.

The video display device according to the invention of claim 2 is
In addition to being able to reduce the burden on the worker at the time of work inspection during plant inspection / repair work, the worker can easily carry it.

The video display device according to the invention of claim 3 is
At least one of the internal state of the unmeasurable object estimated using the internal state estimation database consisting of dynamic characteristic models and the internal state of the object that is measurable but cannot be directly seen by the user. Is visualized,
The visualized image is added to the image projected on the object and its surroundings. As a result, the image information related to the internal state of the device can be projected directly onto the inspection / repair device and its surroundings by the image projection means provided in the portable computer, etc. When there is no need to move and the change of maintenance method and operation method is required depending on the internal state of the target,
Workers can make accurate decisions.

The video display device according to the invention of claim 4 is
A maintenance method or an operation method suitable for the photographed object is determined using the maintenance and operation method determination database, and an image related to the maintenance method or the operation method is added to the image projected on the object and its surroundings. As a result, the image information related to maintenance or operation guidance can be projected directly onto the inspection / repair device or its surroundings by the image projection means provided in the portable computer, etc. Moreover, it is possible to reliably perform the maintenance work and the operation work required at the work site, without the need to move.

The video display device according to the invention of claim 5 is
The spatial distribution of information that cannot be perceived by humans, such as radiation, temperature, and gas concentration, obtained from the sensor is obtained, and the image related to the spatial distribution is added to the image projected on the object and its surroundings.
As a result, the image information related to the spatial distribution of gas, heat, radiation, etc. captured by the sensor can be directly projected onto the inspection / repair device and its surroundings by the image projection means equipped in the portable computer etc. Users such as staff members
It is not necessary to move the line of sight from the device to be repaired, and it is possible to prevent the user and people around the user from danger in advance.

The video display device according to the invention of claim 6 is
Video information that supports the user's actions such as danger area and movement route guidance to the user is determined using the danger area and movement route determination database, and the image related to the support information determined as the image projected on the object and its surroundings is displayed. Add.
As a result, the video information relating to the guidance of the travel route or the patrol route and the dangerous area can be directly projected onto the inspection / repair device or its surroundings by the image projection means provided in the portable computer or the like. The user does not have to move his or her line of sight from the equipment to be inspected or repaired.
The user can accurately and swiftly take the action required at the work site.

According to the seventh aspect of the invention, the image projecting means of the image display device controls the optical axis of the visible laser light so that the relative position of the image projecting means and the object and the shape of the object are controlled. Project video, video, or text in real time. As a result, it is possible to easily execute drawing of a figure according to the shape of the object.

In the image display system according to the invention of claim 8, the central computer, which can communicate with the portable computer through the communication means, is an object photographed by the photographing means installed in the portable computer using the remote display means. Display the image of the object.
Further, the remote input means processes the input information input by the operator to project the input information onto the object and its surroundings by using the image projection means. As a result, a worker on the site carrying the portable computer and an operator of the central computer can easily communicate with each other.

The image display means of the image display system according to the ninth aspect of the present invention controls the optical axis of the visible laser light so that the relative position of the image projection means and the object and the shape of the object are controlled. Project video, video, or text in real time. As a result, it is possible to easily execute drawing of a figure according to the shape of the object.

[0040]

【Example】

Example 1. An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image display apparatus according to an embodiment of the present invention. In the figure, 1 is a camera (imaging means), 2 is a position using a gyro, a magnetic sensor, an ultrasonic sensor, light, etc. A position / orientation measuring device including a measuring device, 3 is a video projection device (video projection means) for projecting information created by a computer onto an object, and 4 is a portable computer. The camera 1, the position / orientation measuring device 2, and the image projection device 3 are attached to the portable computer 4. That is, the portable computer 4 functions as a support unit that supports the camera 1 and the image projection device 3 (also supports the position and orientation measurement device 2).

Reference numeral 5 is a highly accurate position of the portable computer 4 using information from the camera 1 and the position and orientation measuring device 2,
A position / orientation measuring unit for calculating and outputting the attitude, 6 is a position / orientation calculation result of the portable computer 4 obtained from the position / orientation measuring unit 5, and 7 is based on the position / orientation calculation result 6 of the portable computer 4,
The object layout and shape database (object database) 10 to be described later are searched to determine which object the data is to be displayed in, and what kind of information to present is searched from the projection information database 8 and projected. A projection information retrieval unit (projection information retrieval means) that outputs information, 8 is a projection information database that stores information to be presented to a user who carries the portable computer 4, and 9 is a position and orientation calculation result 6 of the portable computer 4. , An object arrangement to be projected, searching from the shape database 10, calculating a relative positional relationship with the object to be projected, etc., and converting and outputting the image information output from the projection information searching unit 7 (Image generation means). Reference numeral 10 denotes an object arrangement, an object arrangement having a shape, and a shape database 10. The portable computer 4 includes all the above processes and databases. The position / orientation measuring device 2 and the position / orientation measuring unit 5 correspond to the position / orientation measuring means.

Next, the operation will be described. FIG. 2 is an image diagram for explaining the operation of the video display device according to this embodiment. The image display device is used for maintenance and monitoring of power generation plants, substation equipment, chemical plants, buildings, railways, roads, water and sewer systems, and the like. A user, such as a worker or a patrolman who performs maintenance and inspection of the plant or the like, carries a video display device and travels around the plant. In the example shown in FIG.
The projected information in the operation of the image display device in the opening and closing work on the valve 11 is shown. The user turns the image display device toward the target valve 11. At this time, the video display device uses one or both of the video information from the camera 1 directed toward the valve 11 and the information from the position and orientation measurement device 2, or the position of the portable computer 4,
Measure your posture. Projection information search unit 7 of portable computer 4
On the basis of the position / orientation calculation result 6 of the portable computer 4, the object placement / shape database 10 is searched to determine which object the data is to be displayed on. Further, the projection information database 8 is searched for what projection information is to be presented to the determined object, and the projection information is output to the video information generation unit 9. That is, the portable computer 4 determines what kind of object is in the direction in which the user points the portable computer 4, retrieves information about the object, and projects the information by superimposing it on the object. Therefore, the relative positional relationship with the object and the shape of the object are calculated. Then, the video information generation unit 9 of the portable computer 4 converts the video information by the calculation so that the video information can be projected on the target object in an overlapping manner, and projects the video information on the target object.

In the example shown in FIG. 2, the portable computer 4 recognizes the valve 11 from the image information of the valve 11 and the like, and projection information about the valve 11 "turn the valve to the right", an arrow indicating the direction of turning the valve, An arrow indicating the direction of fluid flow is projected on the wall behind the valve, the valve, or the like.

Thus, in a plant or the like, a user such as a worker carries the portable computer 4 and moves. When the user points the portable calculator 4 at a portion of the user who is interested in moving or wants to know more in detail while shining a flashlight or the like attached to the portable calculator 4, the portable calculator 4 determines its own position and posture. The target object is calculated based on the camera information, and the corresponding information is displayed on the target object. The user simply points the portable computer at the object of interest and the information about the object is projected onto the object, and the necessary information can be obtained without moving the line of sight. Users will be able to share that information at the same time. Further, various information can be obtained in a natural form while looking at the object, and maintenance work can be reduced. Further, as will be described later in another embodiment, the portable computer 4 in the field and the central computer on the central side provided in the control room or the like are configured to communicate with each other via a communication device such as a wireless device. May be. In this case, a person who operates the central computer in the control room controls the projection function to give the user on the spot easy-to-understand instructions and advice. In addition, by adding information about the patrol route and the dangerous area to the projection information, the user can obtain the patrol route and the dangerous area intuitively and without overlooking.

Further, unlike the conventional video display device, the user does not need to wear the video display device on the head,
The physical load on the user can be reduced.

Example 2. FIG. 3 is an explanatory diagram showing the configuration of a video display device according to another embodiment of the present invention. The video display device according to this embodiment has almost the same configuration as that of FIG. 1, except that the video projection device 3, the camera 1, and the portable computer 4 are provided.
Is not carried by the user and is attached to the ceiling, wall, etc.

Although the video display device according to the first embodiment determines the target of the user's interest from the position and orientation of the portable computer, the video display device according to this embodiment uses the finger of the user instead of using it. The action of pointing the object is obtained from the camera, and the process of identifying the target of the user's interest is used to project the image processed by the computer on the target to be superimposed on the target. Also, instead of the user pointing,
An object that the user is interested in is used by giving the user an object such as a stick with a position measuring device, and using processing for specifying the target of the user such as specifying the object from the position and orientation of the stick. The image processed by the computer is superimposed and projected onto the object.

FIG. 4 is a system configuration diagram of a video display device according to this embodiment. In the figure, the same or corresponding parts as those of the first embodiment shown in FIG. Reference numeral 21 is a position / orientation calculation result of the camera 1, 22 is a target specifying unit of the user's interest, 23 is a platform control unit, 24 is a cloud for supporting the camera 1 and the image projection device 3 and directing them toward the target object. A table 25 is a computer corresponding to the portable computer 4 in FIG. In this embodiment, unlike the first embodiment, the mounting positions of the camera 1, the position / orientation measuring device 2 and the image projecting device 3 are not a portable computer but a pan head controlled by a computer mounted on a wall or the like. It is attached to 24. That is, the platform 24
Corresponds to the support means.

Next, the operation will be described. The position / orientation measuring unit 5 calculates the position / orientation of the camera 1 based on the information from the camera 1 and the position / orientation measuring device 2, and outputs it as a position / orientation calculation result 21. User's interest target specifying unit 22
Performs a process of specifying the target of the user's interest based on the action pointed by the user as described above, and retrieves information to be projected from the projection information database 8. The video information generation unit 9
Information about the target object of interest to the user is calculated as relative position information between the camera and the target object, and based on this relative position information and the shape of the target object, the information to be projected can be superimposed and displayed on the target object. To convert the information to be projected. The camera platform control unit 23 controls the camera platform 24 so that the camera always faces the object of interest to the user.

The same effects as those of the first embodiment can be obtained by this embodiment, and the user can simply point to the object of his / her interest without carrying a computer, and The information about the object can be projected onto the target object and the necessary information can be obtained without moving the line of sight, and a plurality of users can share the information at the same time.

Example 3. FIG. 5 is a block diagram showing the configuration of a video projection device in a video display device according to another embodiment of the present invention. In the figure, the same or corresponding parts as those of the components of the first embodiment shown in FIG. 31 is a visible laser light generator for generating visible laser light, 32 is a mirror, 33 is an axis for rotating the mirror about a horizontal axis, 34 is an axis for rotating the mirror about a vertical axis, Reference numeral 35 is a laser light generation control device for controlling on / off of the visible laser light generation device 31, and 36 is a mirror surface control in which a drive device such as a motor installed on each of the shafts 33 and 34 and a device for controlling the drive device are combined. It is a device.

Next, the operation will be described. The video information generated by the video information generation unit 9 of the video display device is reflected by the mirror 32 by the operation of the axes 33 and 34 controlled by the mirror surface control device 36.
Of the visible laser light generator 31 is converted into the movement of the light beam, and the light beam from the visible laser light generation device 31 is reflected at an arbitrary position to project the image information. Further, the laser light generation control device 35 controls on / off of the laser light from the visible laser light generation device 31. By these controls, the image projection device 3 can project an arbitrary character or figure in an arbitrary place around an object such as a plant.

Example 4. FIG. 6 is a block diagram showing the configuration of the internal state estimation / display function of the video display device according to another embodiment of the present invention. FIG. 6 shows the function of adding information to the projection information database 8 in FIG. 1, and the video display device according to this embodiment functions by combining FIG. 1 and FIG. In FIG. 6, the same or corresponding parts as those of the constituent elements of the first embodiment shown in FIG. 1 are designated by the same reference numerals, and reference numeral 41 measures the on-site environment by a measurement sensor attached to the on-site or portable computer 4. A site environment measuring unit, 42 is an internal state estimation database including a dynamic characteristic model for estimating the internal state of an unmeasurable object, and 43 is an unmeasurable state using the site environment measurement data and the internal state estimation database. An internal state estimation unit that estimates the internal state of the target, 44 is a user input / output unit that interacts with the user as to which portion of the internal state data is projected, 45
Is an internal state estimation database 42 and an internal state estimation unit 43.
It is a central computer that includes and. The portable computer 4 and the central computer 45 are connected by communication means,
This communication means is not shown in FIG.

Next, the operation will be described. The function of the image display apparatus according to this embodiment will be described below by taking the case of displaying the temperature distribution state of the coolant inside the nuclear reactor as an example.

From the site environment measuring unit 41 to the central computer 4
In FIG. 5, measurable data such as reactor inlet temperature, coolant flow rate, reactor output (neutron flux), etc. are input, and the reactor thermal characteristic equation and reactor nuclear characteristic equation stored in the internal state estimation database 42 are input. The internal state estimating unit 43 calculates the temperature distribution of the coolant inside the nuclear reactor using the nuclear reactor dynamics model. Next, in the user input / output unit 44, the internal position of the reactor (for example, the distance from the inlet) where the internal state should be displayed is known, the coolant temperature corresponding to the internal temperature is obtained from the calculation result, and the result is obtained via the communication means. It is transmitted to the portable computer 4 and added to the projection information database 8 in FIG. In a reactor equipped with a reactor internal coolant temperature sensor, the internal state estimation function is skipped and only the internal state display screen is edited.

As a result, the internal state of the object that the user cannot see directly can be projected onto the object and its surroundings, and the image can be intuitively presented to the user.
It is possible to accurately make a decision when the maintenance method or the operation method needs to be changed according to the internal state of the target. For example, if the temperature of the nuclear fuel is too high, it is highly likely that the fuel has been damaged, so that the inspection work of the nuclear fuel is focused immediately after the shutdown.

Example 5. FIG. 7 is a block diagram showing the configuration of the display function of the device maintenance method and the operation method in the video display device according to another embodiment of the present invention. FIG. 7 shows the function of adding information to the projection information database 8 in FIG. 1, and this embodiment functions by combining FIG. 1 and FIG. In FIG. 7, the same or corresponding parts as those of the constituent elements of the first embodiment shown in FIG. 1 are designated by the same reference numerals, and reference numeral 41 measures the on-site environment by a measurement sensor attached to the on-site or portable computer 4. Site environment measuring unit, 51 is a maintenance method and know-how such as inspection and repair, maintenance method decision database constructed based on past repair history, etc. 52 is constructed based on operation procedure and know-how of individual equipment on site. A device operation method determination database, 53 is a maintenance procedure determining unit that determines a maintenance procedure and a method that match the current situation using the site environment data and maintenance method determination database 51, and 54 is a site environment data and operation method determination database 54. And an operation method determination unit for retrieving and determining an operation method suitable for the current situation from the operation method determination database 52, and 55 is the determined maintenance method. Characters for display to the user a maintenance mode display database consisting of drawings and the like, characters for displaying the determined operating method to the user 56, instructions display database consisting of drawings and the like,
Reference numeral 57 denotes a maintenance method display database 55 at the time of maintenance, and operation method display database 56 at the time of operation based on the maintenance method or the operation method, and a database search / transfer section for writing information to the projection information database 8, 58 a maintenance method determination section, It is a central computer including an operation method determination unit, a database search transfer unit, and the like.

Next, the operation will be described. The function of the image display device according to this embodiment will be described below by taking the case of determining and displaying the maintenance method in the nuclear power plant as an example.

Measurement data is input from the site environment measuring unit 41 to grasp the current plant state, and mainly the IF-T
The maintenance method determination unit 53 determines the following information regarding the maintenance method by using the maintenance method determination database 51 including the HEN rule.

(1) Priority order of equipment to be maintained (2) Whether or not preconditions (for example, system isolation) for carrying out maintenance are satisfied (3) Maintenance method of each equipment (for example, disassembly method, inspection method, (Part replacement method, order of each work)

Next, the database retrieval / transfer section 57 retrieves the maintenance method display database 55 on the basis of these pieces of information, and transfers those pieces of information to the projection information database 8.
Thereafter, according to the configuration of FIG. 1, the video information generation unit 9 of the portable computer 4 projects the image on the object. Even when the device is operated, the information about the operation is projected on the operation device and its surroundings by the same method.

By this, the maintenance method / operation method suitable for the present situation is determined, and it is collected on a display screen that is easy for the user to understand and projected on the object and its surroundings, and the image information is presented to the user. Therefore, maintenance work and operation work required at the site can be surely executed.

Example 6. FIG. 8 is a block diagram showing the configuration of a site environment display function in a video display device according to another embodiment of the present invention. FIG. 8 shows a function of adding information to the projection information database 8 in FIG. 1, and this embodiment works by combining FIG. 1 and FIG.
In FIG. 8, the same or corresponding parts as those of the constituent elements of the first embodiment shown in FIG. 1 are designated by the same reference numerals, and reference numeral 41 indicates the on-site environment by a measurement sensor attached to the on-site or portable computer 4. The on-site environment measuring unit 61 is an input / output device such as a pointing device or a display of the portable computer 4, the type of sensor used for displaying, a display method (interval between isobars, isoconcentration lines, etc.).
A user information input / output unit for interactively acquiring the information with the user, and 62 is a site for converting so that the site environment data obtained from the sensor and the spatial distribution of the information obtained from the sensor at the request of the user can be expressed in an image. Environmental information conversion unit, 6
Reference numeral 3 is a database transfer unit for writing in the projection information database 8 the site environment information converted so that it can be expressed as an image.

Next, the operation will be described. The function of the image display apparatus according to this embodiment will be described below by taking as an example the case where the site environment is constantly visualized in the maintenance work at the nuclear power generation facility.

The on-site environment measuring unit 41 measures the on-site environment by a measurement sensor attached to the site or the portable computer 4. The measured on-site environment data is converted into an image by the on-site environment information conversion unit 62 according to the user's request (what to display and in what display format) obtained by the user information input / output unit 61. It is converted into a form that can be expressed and sent to the database transfer unit 63. The information is written in the projection information database 8 by the database transfer unit 63, and thereafter, according to the configuration of FIG.
The video information generation unit 9 of the portable computer 4 projects the information on the wall or floor around the object.

In this way, the radiation, temperature, and gas concentration that cannot be perceived by humans are measured by the site environment measuring unit 41 and displayed in a visible form, so that the user and the humans around them can be effectively displayed. You can avoid it from danger.

Example 7. FIG. 9 shows determination of a dangerous area / moving route in an image display device according to another embodiment of the present invention.
It is a block diagram which shows the structure of a display function. FIG. 9 shows FIG.
2 shows the function of adding information to the projection information database 8 in this example, and this embodiment functions by combining FIG. 1 and FIG. 9, the first embodiment shown in FIG.
The same reference numerals are given to the same or corresponding parts as those of the above, 41 is a site environment measurement unit that measures the site environment by a measurement sensor attached to the site or the portable computer 4, 71 is a user's action area Database for dangerous area determination, which consists of location information of openings with low overhead and dangerous area determination rules based on site environment data, 7
Reference numeral 2 is a travel route determination database configured by travel route determination rules, 73 is a dangerous area determination unit that determines a dangerous area that matches the current situation using the on-site environmental data and the dangerous area determination database 71, and 74 is the on-site environmental data. A travel route determination unit that determines a travel route that suits the current situation using the travel route determination database 72, 75 is a danger area display database including characters and drawings for displaying the determined danger area, and 76 is the determined travel path. A moving path display database consisting of characters and drawings for displaying, and 77 is a database search / transfer section that searches both the danger area display database 75 and the moving path display database 76 and writes the information to the projection information database 8. ,
Reference numeral 78 is a central computer that includes the dangerous area determination unit 73, the movement route determination unit 74, and the database search transfer unit 77.

Next, the operation will be described. The function of the image display device according to this embodiment will be described below by taking the case of determining and displaying the moving route in the nuclear power generation facility as an example.

The movement route determination unit 74 determines the movement route according to the following steps.

(1) Input measurement data from the site environment measuring unit 41 to grasp the current state of the nuclear power generation facility.

(2) Using the movement prohibition rules stored in the movement route determination database 72, the movement prohibition area is obtained without omission.

(3) The movable area is obtained by excluding the current prohibited area from the entire moving area (movable area if the conditions are satisfied).

(4) The movement route is determined so that the movement distance becomes the shortest in the movable area.

Next, the database search / transfer unit 77 searches the travel route display database 76 for an appropriate display in accordance with the dangerous area display travel route, transfers the display information to the projection information database 8, and thereafter the configuration shown in FIG. According to
The video information generation unit 9 of the portable computer 4 projects the information on the object. Also in the dangerous area display, the dangerous area determination unit 73 can find the dangerous area and project and display the dangerous area by the same procedure as described above, and notify the user of that fact.

In this way, the video display apparatus according to this embodiment determines a dangerous area / movement route that suits the current situation, collects the dangerous area / movement path on a display screen that is easy for the user to understand, and displays it on the object and its surroundings. Since the image is projected and the image information is presented to the user, the user can accurately and swiftly perform the action required on site.

Example 8. FIG. 10 is a block diagram showing the overall configuration of a video display system according to another embodiment of the present invention. In the figure, the same or corresponding parts as those of the first embodiment shown in FIG. 81 is a central computer, 82a and 82b are communication devices for exchanging data between the portable computer 4 and the central computer 81, and 83 is a scene image taken by the camera 1 installed in the portable computer 4. A remote display device (remote display means) for displaying on the central computer 81, 84 is a remote for projecting information input by a pointing device such as a pen or a mouse on the site by using the image projection device 3 installed in the portable computer 4. It is an input device (remote input means). Reference numeral 85 is a display data conversion unit that converts the data sent by the communication device into a format that can be displayed on the central computer, and 86 is a video display unit that displays the converted video data. The remote display device 83 includes a display data conversion unit 85 and a video display unit 86.
Reference numeral 87 is an input device such as a pen or a mouse that the operator inputs on the image on the image display unit 86. The remote input device 84 includes an input device 87 and an input data conversion unit 88. In this way, the image of the site is displayed on the central computer 81 at the remote location, and the input from the central computer 81 is sent to the portable computer and projected on the site, so that the workers on the site and the central computer Communication of operator's will becomes easy.

Next, the operation will be described. FIG. 11 is an explanatory diagram showing a data communication method between the portable computer 4 and the central computer 81. First, a worker on site installs the camera 1 and the image projection device 3 on the portable computer 4, and directs the camera 1 and the image capturing device toward the target object.
The portable computer 4 displays the image captured by the camera 1 on the portable computer 4 and simultaneously transmits the digital data of the image to the central communication device 82b via the communication device 82a such as a wireless device in the field. The central computer 81 which has transmitted the radio wave via the communication device 82b converts the digital data into image data and displays the image data on the image display device. Further, the central computer 81 converts image data such as handwritten information input on the central side into digital data, and transmits the digital data to the on-site communication device 82a via the central communication device 82b. The portable computer 4 can display the information input from the central computer 81 on the site by projecting the transmitted data on the site by using the image projection device 3 installed in the portable computer 4. it can.

Next, the remote display device 83 and the remote input device 8
The operation of No. 4 will be described according to the flowchart shown in FIG. First, it is transmitted to the communication device 82b on the center side via the communication device 82a on the scene photographed by the camera installed in the computer 81 on the center side (step ST80). The display data conversion unit 85 acquires data from the communication device 82b and converts it into image data that can be displayed on the central computer 81 (step ST81). The converted image data is displayed on the video display unit 86 of the remote display device 83 (step S
T82). Next, on the video sent from the scene, handwriting, graphics, and the like from the input device 87 of the central computer 81.
When pointer information or the like is input (step ST83), the input image data is converted into digital data by the input data conversion unit 88 (step ST8).
4). The converted data is transmitted to the communication device 82a connected to the portable computer 4 via the communication device 82b (step ST85), and the input data is projected on the site by the image projection device 3 installed in the portable computer 4. (Step ST
86).

As described above, according to the video display system of this embodiment, the video of the scene is displayed on the central computer 81.
In addition, since the input from the central computer 81 is projected on the site, the workers on the site and the operator on the central side can easily communicate with each other.

[0080]

As described above, according to the first aspect of the invention, the image projection means should be used to project based on the information obtained from the position / orientation side means and the image obtained from the photographing means. Based on the projection information search means for searching the projection information database and the object database for the video information on the video and the layout and shape of the object, and the relative positional relationship between the video projection means and the object and the shape of the object, Since it is configured to include a video generation unit that generates a video to be projected from the searched video information, the necessary video information generated by the video generation unit is superimposed on the target object within the visual field of the user who handles the video display device. Since the information is displayed, the user can take the correspondence between the presented information and the object without moving the line of sight, can perform the work such as maintenance and inspection accurately and easily, and can also perform multiple operations. There is an effect that it is possible to see the video information in a human. Further, there is an effect that the operator can intuitively understand the correspondence between the information such as the drawing and the actual state of the device in the case of maintenance and inspection of the plant or the like. Further, since it is not necessary for the worker to wear the image display device on the head, there is an effect that the physical load on the worker can be reduced.

According to the second aspect of the present invention, the portable computer is equipped with a portable computer that can be carried by the user, and the portable computer functions as a supporting means, and the position / orientation measuring means, the projection information database means, and the object. Since the database means, the projection information retrieval means, and the image generation means are included, there is an effect that the user can easily carry it.

According to the third aspect of the present invention, the internal state of the unmeasurable object estimated using the internal state estimation database including the dynamic characteristic model and the object which is measurable but cannot be directly seen by the user. Since it is configured to further include a means for visualizing at least one of the internal states of the above, and adding the visualized image to the image projected on the object and its surroundings, depending on the internal state of the object. When the maintenance method and the operation method are required to be changed, a user such as a worker can make an accurate decision.

According to the invention of claim 4, a maintenance and operation method determination database for storing a maintenance method and an operation method of a device including an object and a maintenance method or an operation method suitable for the photographed object are maintained and operated. Requested at the work site such as plant maintenance / inspection because it was configured using the method decision database and was further equipped with a means for adding the image regarding the maintenance method or operation method to the image projected on the object and its surroundings. This has the effect of reliably performing the required maintenance work and operation work.

According to the invention of claim 5, radiation, temperature,
A sensor such as a gas concentration and a means for obtaining a spatial distribution of information that cannot be perceived by a human such as radiation, temperature and gas concentration obtained from the sensor, and adding an image relating to the spatial distribution to an image projected on an object and its surroundings. Since it is configured to further include, there is an effect that the user and humans around the user can be avoided from danger in advance.

According to the invention of claim 6, the image information for supporting the user's actions such as the dangerous area for the user and the guidance of the moving route is determined by using the dangerous region and moving route determination database, and the object and its surroundings are determined. Since it is configured to further include means for adding the image regarding the determined support information to the image to be projected on the screen, there is an effect that the user can appropriately and swiftly perform the action required at the work site.

According to the invention of claim 7, by controlling the optical axis of the visible laser light, an image, a moving image, a moving image, a moving image can be displayed in real time according to the relative position of the image projecting means and the object and the shape of the object. Alternatively, since it is configured to include a means for projecting a character, there is an effect that a figure corresponding to the shape of the object can be easily drawn.

According to the invention of claim 8, based on the information obtained from the position / orientation side means and the image obtained from the photographing means, the image information relating to the image to be projected by the image projecting means and the object. Projection information retrieval means for retrieving the arrangement and shape of the object from the projection information database and the object database, and projecting from the retrieved video information based on the relative positional relationship between the image projection means and the object and the shape of the object. A portable computer having an image generation means for generating an image;
A central computer capable of communicating with a portable computer via a communication means, a remote display means for displaying an image of an object photographed by a photographing means installed in the portable computer, and an image projection of input input information. Since it is configured to include a central computer having remote input means for processing the input information to project it on the object and its surroundings by using the means, the field worker carrying the portable computer and the central computer. There is an effect that the operator of the computer can easily communicate.

According to the ninth aspect of the present invention, by controlling the optical axis of the visible laser light, the image, the moving image, the moving image, the moving image, the real-time image can be displayed in real time according to the relative position of the image projecting means and the object and the shape of the object. Alternatively, since it is configured to include a means for projecting a character, there is an effect that a figure corresponding to the shape of the object can be easily drawn.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a video display device according to an embodiment of the present invention.

FIG. 2 is an image diagram for explaining an operation of the video display device shown in FIG.

FIG. 3 is an explanatory diagram showing a configuration of an image display device according to another embodiment of the present invention.

4 is a configuration diagram showing the video display device shown in FIG.

FIG. 5 is a configuration diagram showing a video projection device of a video display device according to another embodiment of the present invention.

FIG. 6 is a configuration diagram showing a part of an image display device according to another embodiment of the present invention.

FIG. 7 is a block diagram showing a part of an image display device according to another embodiment of the present invention.

FIG. 8 is a configuration diagram showing a part of an image display device according to another embodiment of the present invention.

FIG. 9 is a configuration diagram showing a part of an image display device according to another embodiment of the present invention.

FIG. 10 is a block diagram showing an image display system according to another embodiment of the present invention.

11 is an explanatory diagram showing a method of using the video display system shown in FIG.

12 is a flowchart showing a processing flow of the video display system shown in FIG.

FIG. 13 is a configuration diagram showing a portable computer in a conventional maintenance patrol work support system.

14 is an explanatory diagram showing a screen example of a display unit of the portable computer shown in FIG.

FIG. 15 is a configuration diagram showing another example of a conventional support system for maintenance inspection work.

FIG. 16 is a configuration diagram showing a conventional video display device.

[Explanation of symbols]

1 camera (imaging means), 2 position and orientation measuring device (position and orientation measuring means), 3 video projection device (video projection means), 4
Portable computer, 5 Position / orientation measuring unit (position / orientation measuring means), 7 Projection information retrieval unit (projection information retrieval means), 8
Projection information database, 9 Image information generation unit (image generation means), 10 Object arrangement, Shape database (Object database), 42 Internal state estimation database, 5
1 maintenance method determination database, 52 operation method determination database, 71 dangerous area determination database, 72
Travel route determination database, 81 Central computer, 83
Remote display device (remote display means), 84 Remote input device (remote input means).

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 1/00 G08C 15/00 D H04N 7/18 D

Claims (9)

[Claims] 1. A photographing means for photographing an object, a picture projecting means for projecting a picture on the object photographed by the photographing means and its surroundings, and moving the photographing means and the picture projecting means. At least one of a supporting unit that supports as much as possible, an image obtained from the image capturing unit, and a position measuring device supported by the supporting unit, such as a magnetic sensor, a gyro, or an ultrasonic sensor, is used. A position / orientation measuring means for measuring the position and orientation of the supporting means, a projection information database for storing image information on an image projected by the image projecting means, and arrangement, orientation and shape information of a plurality of objects. An image to be projected using the image projecting means based on the object database, the information obtained from the position and orientation meter side means, and the image obtained from the photographing means. Projection information search means for searching the projection information database and the object database for the image information about the image and the arrangement and shape of the object, and the relative positional relationship between the image projection means and the object and the shape of the object. An image display device comprising: an image generation unit that generates an image to be projected based on the retrieved image information. 2. The image display device includes a portable computer that can be carried by a user, and the portable computer functions as the supporting means, and the position / orientation measuring means, the projection information database means, The image display device according to claim 1, further comprising the object database unit, the projection information search unit, and an image generation unit. 3. The image display device includes an internal state of a non-measurable object estimated using an internal state estimation database including a dynamic characteristic model, and an internal state of an object that is measurable but cannot be directly seen by a user. 3. The apparatus according to claim 1, further comprising means for visualizing at least one of the internal state and adding the visualized image to the image projected on the object and its surroundings.
The video display device described in. 4. The image display device includes a maintenance and operation method determination database that stores a maintenance method and an operation method of a device including an object, and a maintenance method or an operation method suitable for the photographed object. 3. The method according to claim 1, further comprising means for adding a video image related to the maintenance method or the operation method to a video image that is determined using an operation method determination database and is projected on an object and its surroundings. The image display device described. 5. The image display device comprises a sensor for radiation, temperature, gas concentration, etc., and radiation obtained from the sensor,
2. A means for obtaining a spatial distribution of information such as temperature and gas concentration that cannot be perceived by humans and adding an image relating to the spatial distribution to an image projected on an object and its surroundings. The video display device according to claim 2. 6. The image display device determines image information for supporting a user's behavior such as a dangerous area for a user and guidance of a moving route by using a dangerous area and a moving route determination database, and determines the image information around the object and its surroundings. The image display device according to claim 1 or 2, further comprising means for adding an image regarding the determined support information to an image to be projected. 7. The image projecting means controls the optical axis of the visible laser light so that an image, a moving image, or a real-time image is displayed in real time according to the relative position of the image projecting means and the object and the shape of the object. 7. The image display device according to claim 1, further comprising means for projecting characters. 8. A portable computer provided with a photographing means for photographing an object, and an image projecting means for projecting an image on the object photographed by the photographing means and its surroundings, the photographing means. Position for measuring the position and orientation of the portable computer by using at least one of the image obtained from the above and a position measuring device such as a magnetic sensor, a gyro, or an ultrasonic sensor supported by the portable computer. Attitude measuring means, a projection information database that stores image information about an image projected by the image projecting means, an object database that stores arrangement, orientation, and shape information of a plurality of objects, and from the position and orientation meter side means. Based on the obtained information and the image obtained from the photographing means, the image information regarding the image to be projected by the image projecting means and the arrangement and shape of the object An image projected from the retrieved image information based on the relative positional relationship between the image projecting means and the object and the shape of the object, A central computer that is capable of communicating with the portable computer through a communication unit, the object being photographed by the photographing unit installed in the portable computer. A central computer having remote display means for displaying an image of an object and remote input means for processing the input information to project the input information input to the object and its surroundings by using the image projection means. Video display system. 9. The image projecting means controls the optical axis of the visible laser light so that an image, a moving image, or a real-time image is displayed in real time according to the relative position of the image projecting means and the object and the shape of the object. The image display system according to claim 8, further comprising means for projecting characters.