JPH09128310A - Information sharing device/method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To share information made to correspond to an object and a place between a computer and a computer in a remote place. SOLUTION: A memo 3011 is adhered to the unit 4010 of a job site, which is projected on the display 1010 of a computer 1000. When a portable computer 3000 is brought near to the unit 4010 on the job site, the memo 3011 is displayed on a display 3010. If the memo 3011 is inputted when the portable computer 3000 is near the unit 4010, the memo 3011 is displayed near the unit 4010 projected on the display 1010. Thus, information can be adhered in accordance with the object and the place from the remote place or at that place, and information can be referred to from the remote place or at that place, and therefore, information can be shared.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing system for associating information in a computer with objects and places in the real world, and more particularly to an information sharing method and an information sharing apparatus between a work site and a remote computer.

[0002]

2. Description of the Related Art Conventional man-machine interface technology has attempted to create a world (virtual world) as similar to the real world as possible by simulating the real world. However, the two were only similar, and they were completely independent worlds. Therefore, when operating in a virtual world built in a computer, it is impossible to know what the real world is, and conversely, when working in the real world, information in the virtual world cannot be obtained. When we want to exchange information between two worlds, humans have to go back and forth between the two worlds.

For example, it is assumed that a device under repair is hung on a piece of equipment placed at the plant site (such an action is called "tagging"). However, simply hanging up the bill does not indicate from the console in the operating room that the device is being repaired. In order for the console to know that the device being repaired is hanging, it is necessary to input something to the computer and tell it that the device is being repaired. In other words, it is necessary to place a tag not only on the device in the real world but also on the device in the virtual world. Thus, in the conventional man-machine technology, humans had to make an effort to match information in two similar worlds.

[0004]

As described above, in the prior art, it was necessary to go to the site to put information such as a memo on a remote object or place. Further, it is difficult to remotely refer to information such as a memo attached to a remote object or place, and there is a problem that it is necessary to go to the site to refer to the information.

[0005]

In order to solve the above-mentioned problems, the present invention provides a display position correspondence means for associating a position on a display means with designated information, and a display position corresponding by the display position correspondence means. Physical correspondence means for associating the physical position or physical object with the information based on the above, and a predetermined distance range from the physical position or physical object correlated by the physical correspondence means. The information output means outputs the above information when Thereby, information associated with a physical position or a physical object on the display means can be referred to at the physical position or physical object using the information output means. The display means includes, for example, a map, a system diagram, a device configuration diagram, a video image taken by a camera, a stored image,
A drawing or the like is displayed, and the physical correspondence means associates information with a physical position or object based on the display content on the display means. As the information output means, for example, a portable computer equipped with a display or a speaker, or a computer equipped with a display or a speaker installed near the physical position or a physical object is used. The portable computer has a position sensor for knowing the current position and a reading device (for example, a wireless tag reading device or a bar code reading device) for reading the identification information of the device. Furthermore, the physical objects mentioned here refer to all beings including life forms such as humans and animals.

According to the present invention, when the information is designated, the correspondence means for associating with the physical position or the physical object within the predetermined distance range from the current position, and the physical means associated with the correspondence means. The information output unit outputs the information associated with the position or the object by the corresponding unit in association with the display object representing the physical position or the physical object. This makes it possible to refer to information associated with a physical position or a physical object, for example, remotely using the information output means. The corresponding means may be, for example, a portable computer with a display or a speaker, or a computer with a display or a speaker installed near the physical position or a physical object. The portable computer has a position sensor for knowing the current position and a reading device (for example, a wireless tag reading device or a bar code reading device) for reading the identification information of the device. Further, the information output means displays, for example, a map, a system diagram, a device configuration diagram, a video image taken by a camera, a memory image, a memory drawing, etc., and outputs the information associated by the associating means in association therewith. To do. For example, the information associated with the position
Display at the corresponding position on the map.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A field work support system according to an embodiment of the present invention will be described with reference to the drawings.

This system supports information sharing between computers in various places in the plant. In particular, information between a computer at the site (a computer carried by a field worker or a computer installed at the site or embedded in a device at the site) and a computer installed away from the site Support sharing based on where the computer is located in the field. When information is input to the computer at the site, the information is associated with the device at or near the input computer. When a remote computer such as an operation room displays a video of the scene, the input information associated with the video is displayed at the place shown in the video. Further, when the computer displays a map or system diagram of the site, information is displayed on the map or system diagram based on where on the site the information was input. Conversely, if information is associated with a device displayed in the video of the site displayed on the display in the operation room, a computer near the device (a computer carried by a site worker or installed at the site or installed in the site) The associated information is displayed on a computer embedded in the device).

A usage pattern of the present site work support system is shown in FIG. Computer 1000 display 10
At 10 is displayed an image of the site photographed by the camera 1100 installed at the remote site. The device 4010 on the spot is shown in the image. On the display 2010 of another computer 2000, a systematic diagram of the equipment installed at the site is displayed. Field worker 400
0 carries the portable computer 3000 and inspects the device 4010 installed on the site. When the worker 4000 inputs the memory 3011 “Inspection” on the display 3010 of the portable computer 3000, the memo 3011 is also displayed on the image of the device 4010 displayed on the display 1010 of the computer 1000 that is remote.
Is displayed. Also, the memo 3011 is stored in the computer 20.
It is also displayed in the vicinity of the symbol 2012 on the system diagram displayed on the display 2010 of 00. Here, the symbol 2012 is a symbol representing the device 4010 on the spot.

The memo 3011 is displayed on the computer 1000 when the image of the device 4010 is displayed on the display 1010. The memory 3011 is not displayed when the image of the device 4010 is not displayed. When the device 4010 is out of the shooting range of the camera 1100 due to an operation such as pan of the camera 1100, or the image of another camera shooting another place is displayed on the display 101.
When it is displayed at 0, the memo 3011 is not displayed. Similarly, also in the computer 2000, the device 4
When the symbol corresponding to 010 is not displayed, the memo 3011 is not displayed.

Further, when another worker comes to the device 4010 with a portable computer, a memo 3011 is displayed on the portable computer.

In the above, the portable computer 30 on site
The example has been described in which the information input from 00 is displayed on the computers 1000 and 2000 located away from the site.
Next, an example in which information input from a remote computer 1000 or 2000 is displayed on a mobile computer on site will be described with reference to FIG.

Display 10 of computer 1000
A memo 3011 “Inspection” is created on the PC 10 and attached to the image of the device 4010 displayed on the display 1010. The specific method for attaching will be described later. When the worker 4000 working on the site comes to the device 4010 side with the portable computer 3000, a memo 3011 is also displayed on the display 3101 of the portable computer 3000. At that time, the portable computer 3
The memo 3011 is displayed in 000 only when the portable computer is near the device 4010, and when the portable computer moves and approaches another device, the device 4010 is displayed.
The memo 3011 associated with is no longer displayed. The memo 3011 is also displayed near the symbol 2012 on the system diagram displayed on the display 2010 of the computer 2000.

The same applies when the memo 3011 is created on the computer 2000. That is, when a memo 3011 is created on the display 2010 and attached to the symbol 2012 in the system diagram, the device 40 on the spot is displayed.
The memo 3011 is also displayed on the display of the computer in the vicinity of 10. The memo 3011 is also displayed on the image of the device 4010 displayed on the display 1010 of the computer 1000.

In the above example, the computer 1000 displayed the image taken by the camera, but the image of the scene may be taken in advance. That is, images of various parts of the scene are taken and stored in a storage device accessible by the computer 1000. Computer 1
000 reads the stored image from the storage device and displays it on the display 1010. If information is attached to the device shown in the displayed still image, the information is displayed on the still image. Further, the information pasted on the still image is displayed on a portable computer or another computer at the site.

In the above example, the example of inputting the memo "Inspection" has been described, but it goes without saying that the information to be input is not limited to text and may be any information that can be handled by a computer. For example, graphic data, audio information, video information,
Numerical data, image information, etc. may be used. These pieces of information are created by using an application program for inputting and editing each piece of information. For example, text information such as a memo is created using a text editor program or a word processor program, and video information is created using a video editing program for capturing a video and editing it.

In addition to newly inputting, information already existing in the computer may be copied or moved.
For example, information stored in advance in the computer such as control data, sensor values, guidance information may be used.
It goes without saying that an icon may be displayed when displaying such information, and detailed contents of the information may be presented when the icon is designated.

In the above example, the case where the portable computer is used has been described, but the same information is applied to the computer installed in the site or the computer installed in the field device (these are referred to as an embedded computer). It goes without saying that sharing methods can be applied. For example, it may be considered that the computer 3000 is installed in the device 4010.

The computer 100 will now be described with reference to FIG.
A method of pasting information on the device displayed in the image at 0 will be described. To paste the information to the device, simply place the information on the image on the device you want to paste. That is, the icon display 1031 of the information to be pasted is displayed on the device 10 on the display 1010 using the pointer 1030.
Move to 10 images. The movement is usually performed by using an operation called drag. Drag means pointer 103
This is an operation in which the button of the pointing device that is linked to 0 is pressed and held. When the pointer 1030 enters the area 1020 of the device 4010, the area 1020 is displayed graphically and it is confirmed that the icon 1031 has been moved to the area of the device 4010. When the button of the pointing device is released at this point, the icon 1031 is attached to the device 4010. Upon releasing the button, the graphic display in area 1020 disappears. The area 1020 is an area defined on the image in advance according to the position of the device 4010. In this embodiment, the shape of the region 1020 is rectangular, but it goes without saying that other shapes such as a polygon and a circle may be used. in this way,
An area on the image is defined in advance for a device to which information may be attached. These areas are defined for each camera, and according to the operation of the camera (pan, zoom, etc.), the position and size of the area can be changed appropriately according to the operation amount so that the area does not deviate from the displayed device. Hold on.
The method of pasting information on the device shown in the video is not limited to the above method.

FIG. 2 shows the configuration of the portable computer 3000. The external appearance image is shown in FIG. This computer 3000 has a CPU like a normal computer
(Central processing unit) 11, main memory 12, disk 1
3, a display 3010 and a bus 24. CPU1
The text and graphic information generated by the display 1 is displayed on the display 3 via the display interface 15.
010. The transparent tablet 16 is overlaid and integrated on the display 3010. The pointing position of the pen 3 is detected by the tablet 16 and reported to the CPU 11 via the pen interface 17. Similarly, the tablet 16 also detects that the button 4 associated with the pen 3 has been pressed or released, and reports it to the CPU 11 via the pen interface 17. The user uses the pen 3 to input characters and graphic information into the computer 3000.

The computer 3000 is equipped with a camera 5 for recording the surrounding conditions as a still image or a moving image. The camera 5 is attached to the computer body by a flexible arm that can be bent freely, and by appropriately bending this arm, it is possible to freely take pictures of the surroundings. The video processing device 18 converts the video signal from the camera 5 from analog to digital according to an instruction from the CPU 11, further compresses it, and records it in the memory 12 as image data of an appropriate format. The video processing device 18 expands the compressed image data stored in the memory 12 according to an instruction from the CPU 11 and displays it on the display 3010 via the display interface 15.

The computer 3000 is equipped with a microphone 19 for recording voice and a speaker 20 for outputting voice. The voice processing device 21 is a CPU
According to the instruction from 11, the voice signal from the microphone 19 is converted from analog to digital, and further compressed and recorded in the memory 12 as voice data of an appropriate format. The audio processing device 21 expands the compressed audio data stored in the memory 12 according to an instruction from the CPU 11, converts the compressed audio data into an analog audio signal, and outputs the analog audio signal to the speaker 20.

The RFID tag reader 1 is a computer 3
The information stored in the wireless tags within a certain distance range from 000 is read. The wireless tag is also called a data carrier, holds information, and transmits the held information to the reading device 1 in response to an inquiry from the wireless tag reading device 1. Information communication between the wireless tag and the reading device 1 is performed via the wireless tag antenna 22. A wireless tag holding identification information for each device is attached to each device in the field. The portable computer 3000 reads the identification information of a wireless tag attached to a nearby device via the wireless tag reading device 1. This allows the portable computer 3000 to identify a device near it. The wireless tag reading device 1 can only read information of wireless tags within a certain distance range. When there are a plurality of wireless tags within the readable distance range, all the information can be read.

In the above embodiment, the portable computer 30
Although the RFID tag and its reader were used to identify the device near 00, other methods can of course be used. For example, a bar code system or a position sensor may be used.

Reference numeral 14 is a wireless LAN device for wirelessly exchanging information with other computers. Reference numeral 23 is a communication antenna which is a part of the wireless LAN device 14. The computer 3000 receives another computer, for example, the computer 1000, via the wireless LAN device 14.
It exchanges information with 2000 and other portable computers.

A method of referring to information pasted on a device and a method of pasting information to a device on the portable computer 3000 will be described with reference to FIG. FIG. 4 shows a portable computer 30.
The display form of the display 3010 of 00 is shown.
In the portable computer 3000, the RFID tag reader 1
To read the information of the RFID tag in the vicinity and recognize the device in the vicinity. The recognized device is the display 30
10 is displayed as a device list 3020. When multiple devices are in the vicinity, they are displayed in a list in the order in which the multiple devices were recognized. By default, the information list 3021 displays a list of titles of information attached to the first recognized device (valve 1 in FIG. 5). To view a list of information pasted on other recognized devices (valve 2 and motor), select the desired item in the device list with the pen 3
You can specify using. FIG. 5 shows that three pieces of information, a memo, an inspection record, and a caution for use are attached to the valve 1. By default, the information (memo in the case of FIG. 5) at the top of the information list is displayed on the display 30.
It is displayed in a format such as 3023 on 10. When it is desired to refer to other information (inspection record or caution), a desired item in the information list may be indicated with the pen 3.
To attach the information to the device, the device to which the information is to be attached is designated by using the pen 3 in the device list, and then the new creation 3022 of the information is designated. Then, the information creation sheet opens in a format like 3023. If it is text information,
As it is, the characters may be input with the pen 3 in the “content” field. Other information may be created by using a program for creating the text information, and may be copied in the “content” column or read from a file. If the information created in this way is saved, the device list 3020
The information is pasted to the currently selected device in the
The title is additionally displayed on the information list 3021.
In the “date and time” column, the portable computer 3000 automatically inserts the current date and time by referring to a built-in timer (not shown in FIG. 2). If a wireless tag is also attached to the worker 4000 so that the portable computer 3000 can also identify the user ID of the worker 4000, the computer 3000 can be automatically inserted in the “creator” field.
Does not need to be entered.

A method of implementing the present site work support system will be described with reference to FIGS. 6 and 7. The overall configuration of this system is shown in FIG. Computers 1000, 2000, and 2100 installed away from the site are wired LAN 500
Connected using 0. On-site mobile computer 3
000, 3100 and the embedded computer 3200 installed at the site are connected using a wireless LAN. Wireless LA
The computer connected by N exchanges information with the computer connected to the wired LAN 5000 via the hub 5010. An image from a camera placed on the spot is sent to the computers 1000 and 2000 via a matrix switcher 5020.

The computer 1000 holds information about what is shown in the image and what is shown in the image using a data structure as shown in FIG. The area objects 1102-1104 correspond to the individual devices shown in the video. The area information of the area object is information about where and how large the corresponding device appears in the image.
For example, it is represented by the position and size of the rectangular area. The device ID is the identification number of the on-site device corresponding to the area. In the area list 1101, all pointers to area objects in the field of view of one camera are registered.

The information attached to each device is stored in the computer 2100 using a data structure as shown in FIG. This information is accessed from all other computers. Needless to say, this information may be stored in any of the computers connected to the network shown in FIG. The correspondence information list 2110 includes a memo 2111 attached to one device and inspection information 211.
2, pointers to all information such as the parts list 2113 are stored. The correspondence information list 2110 is held in correspondence with the device ID.

The information pasting to the device on the image described with reference to FIG. 5 is realized by the steps as shown in FIG. While the pointer 1030 is dragging information, it is constantly checked whether or not the pointer 1030 has entered the area. When the pointer 1030 enters any area, the area is displayed graphically. When the button of the pointing device is released, the area object corresponding to the area containing the pointer 1030 is determined (step 6001). The device ID held by the determined area object is read (step 6002), the computer 2100 is accessed via the wired LAN 5000, and the dragged information is registered in the correspondence information list corresponding to the device ID (step 600).
3).

The processing steps when the portable computer 3000 described in FIG. 4 refers to the information attached to the device will be described with reference to FIG. First, the RFID tag reading device 1 is used to read the information of the RFID tag attached to the nearby device, and the device ID of the nearby device is identified (step 6011). The device name corresponding to the identified device ID is displayed on the display 30 as a device list 3020.
10 is displayed (step 6012). Equipment list 30
Correspondence information list 2110 corresponding to the top 20 devices
Is read from the computer 2100 via the wireless LAN and the wired LAN 5000, and the title of the correspondence information is displayed as the information list 3021 (step 601).
3). The information corresponding to the highest item in the information list 3021 is read from the computer 2100 by tracing the pointer registered in the correspondence information list 2110 and displayed on the display 3010 (step 6014).

In the above embodiment, the wireless tag is attached to each device, but it may be attached to other devices. For example, it may be attached to a person or an animal. This makes it possible to attach information to people and animals and share the information from nearby computers or remote computers.

By using the on-site work support system of the present embodiment, information can be easily and immediately shared between the on-site worker and the operator in the remote operation room. Conventionally, the notes and tags attached to the equipment on site could not be read by the operator in the operation room. In this system, instead of pasting memos and tags on the equipment in the field, by creating memos and tags on the portable computer and making them accessible from a remote computer, the operator in the operation room can also perform other tasks in the field. Members can also refer easily. On the contrary, control information that can only be referred to in the operation room can be easily referred to on site. As a result, it is possible to tag the equipment on the site without going to the site remotely, and to remotely refer to the tag placed by the worker on the site.

A distributed work support system according to another embodiment of the present invention will be described with reference to FIG. This system is for workers who have a portable computer and work on the street.
The center supports information sharing between operators who support those workers. 7000 is a computer installed in the center, and a map of the street where the worker works is displayed on the display 7001 thereof. When the operator of the center wants to send the instruction information 7002 to the worker at the site A, the pointer 7003 is used to send the instruction information 702.
02 is moved to the position corresponding to the site A on the map. Then, the instruction information 7002 is displayed on the portable computer 7100 of the worker at the site A. On the other hand, when the worker at the site B wants to inform the operator of the center of the situation, when the situation information 7004 is created on the portable computer 7200, the site B on the map displayed on the display 7001 of the computer 7000 of the center is displayed. The status information 7004 is displayed at the display position corresponding to. As described above, it is possible to easily share information between workers who work in a distributed manner in various places on the street and operators in the center.

[0035]

The effects of the present invention are as follows.

1) Information can be associated with local objects and places from a distance. The pasted information can be referred to using a local computer. Then, it is possible to refer to the information that is locally associated with the thing or place that is in the field from a distant place. In addition, the local computer can display only the information about the place or objects in the vicinity, so that only the necessary information can be easily referred. Furthermore, since information associated with remote objects and places can be referred to by watching the images of those objects and places, information can be easily searched and retrieved. Then, while watching images of remote objects and places,
Since information can be associated with these things and places, you can work as if you were at the site.

[Brief description of the drawings]

FIG. 1 is a diagram showing a usage pattern of a field work support system according to the present invention.

FIG. 2 is a diagram showing a configuration of a portable computer for supporting on-site work.

FIG. 3 is a diagram showing an external appearance of a mobile computer for supporting on-site work.

FIG. 4 is a diagram showing a display form of a portable computer for supporting on-site work.

FIG. 5 is a diagram showing a method of pasting information on a device shown in a video.

FIG. 6 is a diagram showing an overall configuration of a field work support system.

FIG. 7 is a diagram showing a data structure for managing a display area of a device on a video.

FIG. 8 is a diagram showing a data structure for managing information attached to a device.

FIG. 9 is a diagram showing processing steps for realizing information pasting on a video.

FIG. 10 is a diagram showing processing steps for referring to information pasted on a device by a portable computer.

FIG. 11 is a diagram showing a usage pattern of a wide area distributed work system according to the present invention.

[Explanation of symbols]

1 ... Wireless tag reader, 14 ... Wireless LAN device, 1
000, 2000, 2100, 6000 ... Computer, 1020 ... Area on image of device 4010, 1030
… A pointer that moves in conjunction with the pointing device, 3
000, 3100, 6100, 6200 ... Portable computer, 3200 ... Embedded computer, 4010 ... One of on-site equipment, 5000 ... Wired LAN, 5010 ... Wireless L
AN hub, 6001 ... A display displaying a map.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Harumi Uchigasaki 7-1-1 Omika-cho, Hitachi-shi, Ibaraki Hitachi Ltd. Hitachi Research Laboratory, Hitachi Ltd. (72) Inventor Yukihiro Kawamata 7-chome, Omika-cho, Hitachi-shi, Ibaraki No. 1 in Hitachi Research Laboratory, Hitachi, Ltd. (72) Inventor Hideaki Suzuki No. 1-1, Omika-cho, Hitachi, Hitachi, Ibaraki (72) In Hitachi Research Laboratory, Hitachi, Ltd. (72) Atshiko Nishikawa Hitachi, Ibaraki Prefecture 5-2-1 Omika-cho, Ltd. Omika Factory, Hitachi Ltd.

Claims (10)

[Claims] 1. A display position correspondence means for associating a position on a display means with designated information, and a physical position or a physical object and the information based on the display position matched by the display position correspondence means. And a physical correspondence means for associating with, and an information output means for outputting the information when in a predetermined distance range from a physical position or a physical object correlated by the physical correspondence means. An information sharing device characterized by. 2. The physical position or physical object according to claim 1, wherein an image taken by a camera is displayed on the display means, and the physical correspondence means is shown at the corresponding position. An information sharing device, wherein the information is associated with the information. 3. The physical position or physical object indicated by the display object displayed at the position on the display means associated with the physical correspondence means according to claim 1, and the information. An information sharing device characterized by being associated with each other. 4. The information sharing apparatus according to claim 1 or 3, wherein a map is displayed on the display means. 5. Corresponding means for associating a physical position or a physical object within a predetermined distance range from the current position with designated information, and a physical position correlated by the corresponding means. Alternatively, the information sharing device comprises an information output unit that outputs information associated with the position or the object by the association unit in association with a display object representing a physical object. 6. The information output means according to claim 5, wherein the information output means stores the information associated with the physical location or physical object appearing in the image captured by the camera by the association means. An information sharing device characterized by outputting. 7. The information sharing apparatus according to claim 5, wherein said information output means outputs information associated with a physical position by said corresponding means in correspondence with a position on a map. . 8. Corresponding means for associating a physical position or a physical object within a predetermined distance range from the current position with specified information, and a physical position correlated by the corresponding means or An information sharing device, comprising: an information output unit that outputs information associated with the position or the object when the object is within a predetermined distance range from the physical object. 9. A display position corresponding step for associating a position on the display means with designated information, and a physical position or a physical object based on the display position associated by the display position corresponding step and the information. And a physical correspondence step of associating with each other, and an information output step of outputting the information when in a predetermined distance range from a physical position or a physical object correlated by the physical correspondence step. Information sharing method characterized by. 10. Corresponding step of correlating with a physical position or a physical object within a predetermined distance range from the current position when the information is designated, and a physical position correlated by the corresponding step. Alternatively, the information sharing method comprises an information output step of outputting the information associated with the position or the object in the association step in association with a display object representing a physical object.