JP2003058239A - Interactive state monitoring, displaying and driving device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interactive monitoring, displaying and operation controlling device for selectively and intuitively recognizing a state of a monitoring object by utilizing the relation of an actual space and a virtual space and permitting even an operation to the object through a communication system. SOLUTION: At the time of accessing VRML text data 14 and Java module 14a of a server 10 from a terminal 50, 3D image of a plant is displayed at the terminal. Data of a sensor 30 are fetched to the description of the VRML text 14 through a sensor actuator control system 20, a sensor I/O control module 11, a sensor information acquisition module 13 and the Java module 14a and display at the terminal becomes contents detected by the sensor. When an operator performs the operation on the terminal, the Java module 14a detects it, actuator operation data are generated and impressed through an actuator transmission module 15 and an actuator control module 17 to the actuator control system 20 and an object operated by the operator is driven to a desired state.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention can monitor a desired target in a physical space from an arbitrary terminal device, which is suitable for monitoring a desired target to be monitored or controlling a desired target to be controlled by remote control. 3D virtual space is displayed in real time in real time, and 3
An event is generated in the three-dimensional virtual space and the event is reflected in the state of the corresponding object in the real space,
The present invention relates to an interactive / state monitoring / display and driving device for a real space.

[0002]

2. Description of the Related Art When a physical space is monitored in a security system or the like to monitor the situation, a television camera is generally installed to photograph the space to be monitored, and the moving image data taken by the observer stands by. It is transmitted to a place and monitored in real time, or recorded on a video tape and checked when necessary. In the case of such a system, you can install multiple TV cameras so that you can shoot the entire monitored space,
Often, the camera is configured to scan the field of view to ensure a constant field of view.

Further, as a simpler monitoring system which does not use a television camera, for example, doors are opened / closed, lights are turned on / off, temperature, humidity and the like are detected by a sensor and displayed in a control room or the like. Display on the panel,
Depending on the condition, there is a system that calls attention with a warning lamp.

[0004]

By the way, in a surveillance system such as a security system using a television camera as described above, it is not always necessary to obtain all the information of an image captured by the television camera, or all the information. It is often not desirable to be monitored. For example, it is not necessary to monitor the condition of the trash can, the placement of chairs, and the clutter of documents, and for people who work in the monitored space, they can be a little relaxed. It is uncomfortable and mentally unpleasant to monitor all of them.

Further, in the above-mentioned conventional monitoring system, when a wide space such as the whole building is to be monitored, a large number of cameras and a large number of monitoring monitors are usually installed, and a supervisor is required. Accordingly, it is necessary to grasp the overall state of the building while switching the image to be displayed or moving the surveillance range by panning the camera. However, such a work is troublesome to operate, and there is a possibility that blind spots in the monitoring area or missing events of the monitoring target may occur, so a system that allows for easier and intuitive grasp of the state is desired. It is rare.

Further, in the conventional monitoring system using such a moving image, since the moving image data is transmitted in real time, the amount of information to be transmitted is huge, and the information is transmitted only to a specific monitoring place such as a management room. However, it is not suitable for monitoring from any place via a network. However, in a system that monitors a specific event, it may be desirable for the monitor to be able to monitor it at any place such as at home or on the move, and it is desired to handle such a case.

In addition, in a system for notifying a specific physical quantity or the like detected by a sensor without using moving image data, the above-mentioned problem of transmission is solved, but information is needed to monitor a complicated target. Too little, which is not preferable. In such a case, the state can be properly grasped only by a supervisor who is familiar with the real space and the environment where the monitoring target is placed.

Therefore, an object of the present invention is to display an arbitrary monitored object in the physical space on an arbitrary terminal device so that the state can be selectively, intuitively and appropriately grasped, and the grasped contents can be displayed. It is an object of the present invention to provide an interactive / state monitoring / displaying and driving device that organically and efficiently performs desired operation or control in the real space based on the above.

[0009]

In recent years, a text language for constructing a 3D virtual space, simulating a real space, and displaying it on a computer is known. One of the most popular such text languages is VRML (Virtual R).
eality Modeling Language). This VRML is
Although it can be said that it is a three-dimensional version of HTML (Hyper Text Markup Language) which is widely used as a two-dimensional description means for Internet websites, since the content of VRML is text, there is almost no load during transmission. In the 3D world described in VRML, it is possible to use the VRML browser to view the virtual space and walk in the virtual space. You can go anywhere and see anywhere in the created virtual space. The VRML3D world created on the server side can also be accessed and viewed from the client side.

The interface device of the present invention uses a 3D description language represented by such VRML to describe a desired real space in a virtual space in a computer. It displays only the necessary information in the real space, enables transmission with the amount of information that does not impose a load on the transmission via the network, and enables operation and control in the real space based on interactive operations. .

According to the first aspect of the present invention, a sensor for detecting the state of the monitored object, an actuator for driving the controlled object for changing the state of the monitored object, the monitored object and the control. A means for displaying a monitored space including an object by a three-dimensional image, the three-dimensional display means for distinguishably displaying the state of the monitored object detected by the sensor, and the displayed monitoring. An interactive, state monitoring, display and drive device is provided which has an actuator drive means for driving the actuator to drive the displayed controlled object to change the state of the object.

[0012] Preferably, by accessing a corresponding browser, the browser can access the monitored space 3
It has data describing the monitored space in text, which enables a three-dimensional display, and data updating means for updating the described data based on the state of the monitored object detected by the sensor.

According to a second aspect of the present invention, a communication system, a server connected to the communication system, a terminal device connected to the communication system and capable of communicating with the server, and connected to the server. A monitoring target that includes a sensor that detects the state of the monitoring target and an actuator that drives the control target that changes the state of the monitoring target, and the server includes the monitoring target and the monitoring target that includes the control target. Space 3
A three-dimensional display means for displaying the state of the monitored object detected by the sensor in a distinguishable manner; and a means for displaying the state of the monitored object displayed by the three-dimensional image, An actuator drive unit that drives the actuator to drive the displayed controlled object, and the terminal device displays the three-dimensional image by accessing the three-dimensional display unit of the server. Display means and a command to drive the actuator to drive the displayed control object in order to change the state of the displayed monitor object, and the actuator via the communication system. There is provided an interactive / state monitoring / display and driving device having a command receiving means for notifying the driving means.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Regarding one embodiment of the present invention,
This will be described with reference to FIGS. 1 and 2. As the present embodiment, a factory management system for performing factory security management, facility management, etc. will be exemplified. FIG. 1 is a block diagram showing the overall configuration of a factory management system 1 according to an embodiment of the present invention. The factory management system 1 includes a server 10, a sensor / actuator control system 20, a sensor / actuator 30, a computer network 40, and a plurality of user terminals 50.

First, the configuration of each part of the factory management system 1 will be described. The server 10 is a server that provides an interface environment between a physical space and a 3D virtual space on a computer, and reflects a desired state of the physical space on the 3D virtual space, and a physical space based on an operation in the virtual space. It controls. The server 10 will be described in detail with reference to FIG. FIG. 2 is a block diagram showing the configuration of the server 10. The server 10 includes a sensor input / output (I / O) control module 11, a sensor data text file 12, and a sensor information acquisition module 1.
3, VRML text data 14, Java module 14a, actuator transmission module 15, actuator data text file 16 and actuator control module 17.

Sensor I / O control module 11
Periodically and periodically drives an I / O interface (not shown) in the server 10 to acquire data from various sensors installed in a factory for monitoring a management target via the sensor / actuator control system 20. To do. The sensor I / O control module 11 stores the acquired data in the sensor data text file 12.

The sensor data text file 12 is a file in which the detection data of various sensors installed in the factory generated by the sensor I / O control module 11 is recorded, and is accessed by the sensor information acquisition module 13.

The sensor information acquisition module 13 is Java
When the sensor data text file 12 is accessed by the a module 14a, the described sensor data is read and output to the Java module 14a and reflected in the VRML text data 14.

The VRML text data 14 is stored in the server 1
It is data describing the 3D virtual space created in 0, and the 3D virtual space is displayed on the user terminal 50 when accessed by the user terminal 50. In the present embodiment, the VRML text data 14 enables the desired monitoring target object of the management target factory,
Objects to be controlled and operated are displayed as a 3D image.

The Java module 14a, for example, periodically receives the sensor information acquisition module 13 at predetermined time intervals.
Also, the actuator transmission module 15 is activated, and the data text describing the sensor data is accessed via the sensor information acquisition module 13 to acquire the sensor data. Then, based on the acquired sensor data, VR
The data indicating the state of the object in the ML text data 14 is updated. That is, the 3D virtual space data is updated based on the acquired real space data,
The state of the real space is reflected in the virtual space.

Further, for example, one of the plurality of user terminals 50
When a desired object is operated by one,
When any object is controlled according to a control program prepared in advance, the actuator transmission module 15 is activated, the actuator operating text is accessed, and the actuator operating data is written.

The actuator transmission module 15 has a J
It is activated by the ava module 14a and writes actuator actuation data as an actuator data text file 16.

Actuator data text file 1
Reference numeral 6 is a file in which the actuator actuation data written by the actuator transmission module is recorded and read by the actuator control module 17.

Actuator control module 1
7 regularly or periodically accesses the actuator data text file 16 to read actuator operation data, drives an I / O interface in the server 10 (not shown), and via the sensor / actuator control system 20, Controls various actuators installed in factories under management. More than,
This is the configuration of the server 10.

The sensor / actuator control system 20 takes in data from various sensors installed in the factory under the control of the server 10 and outputs the data to the server 10. Further, the sensor / actuator control system 20 generates a signal for driving the actuator as needed based on the control signal from the server 10 to actually drive various actuators.

The sensor and the actuator 30 are provided for an object to be managed in a factory, in particular, an object to be monitored and operated / controlled, and a sensor for detecting the state of the object, and drive the object to a desired state. It is an actuator. Specifically, the sensor and the actuator 30 are, as shown in the figure, a sensor that detects the open / closed state of the door and an actuator that opens and closes the door, a sensor that detects the lighting state of the illumination, an actuator that turns on / off the illumination, and a window. Sensors that detect the open / closed state and actuators that open and close windows, sensors that detect the status of employees coming to work, sensors that detect the status of various safety systems in the factory, and the alarm systems that are turned on / off Actuators, sensors that detect the operating status of devices and equipment in factories and actuators that drive those devices and equipment, sensors that detect the operating status of elevators, and actuators for emergency stop of elevators, workers and robots, etc. And robots that detect the placement status of robots Actuators that drive, sensors that detect damage to important structures and the like and actuators that drive such structures, and sensors that monitor the environment inside and outside the factory and actuators that control its state. .

The computer network 40 is a communication network that connects the server 10 and the user terminal 50, and in the present embodiment, is a LAN provided between the persons involved in the factory to be managed.

The plurality of user terminals 50 are computer terminal devices connected to the server 10 via the computer network 40 and having a 3D browser function for viewing a 3D virtual space. In the case of a computer terminal device having such a 3D browser function, an arbitrary device accesses the 3D virtual space configured in the 3D virtual space description language configured on the server 10 via the computer network 40, and its state Can be monitored and desired operation can be performed.

Next, the operation of the factory management system 1 having such a configuration will be described. First, the objects that need to be managed, for example in factories, such as doors, windows
Sensors for detecting the states of various machines / equipment, worker's placement status, robot's placement status, environment-related information, information about buildings and main structures are provided in advance. If necessary, an actuator for actually performing the processing of objects that need to be operated and controlled by remote control during management and objects that need to be processed in an emergency is provided. These sensors and actuators 30 are connected to the server 10 via a sensor / actuator control system 20. That is, the server 1 of the data detected by the sensor and actuator 30 via the sensor / actuator control system 20.
It is set such that the data is loaded into 0, and the actuator is actually driven based on the control or operation data from the server 10.

Further, the 3D image information showing the factory to be managed, that is, the image information of the objects corresponding to the positional relationship in the actual factory, at least the objects to be managed and their states can be intuitively grasped. Three-dimensional virtual space data in which such an object is described is installed in the server 10 as the VRML text 14.

In such a state, for example, when the factory manager accesses the server 10 from the user terminal 50 installed at home or the like via the computer network 40, the 3D browser of the user terminal 50 causes the VRML to operate.
By accessing the text 14 and the Java module 14a, a 3D image of the factory is displayed on the user terminal 50.
This 3D image is an image that conforms to the actual structure, configuration, layout, etc. of the factory so that the situation of the factory can be grasped as described above, and more detailed display contents include
It is an image centering on an object to be monitored or controlled.

On the other hand, in the factory management system 1 in operation, data from various sensors 30 installed in the factory are used as the sensor / actuator control system 2.
It is fetched by the sensor I / O control module 11 of the server 10 via 0 and stored as a sensor data text file 12. The sensor data text file 12 is appropriately accessed by the sensor information acquisition module 13 activated by the Java module 14a, and the sensor information is captured in the virtual space description of the VRML text 14. As a result, the 3D virtual space of the factory displayed on the user terminal 50 is displayed according to the content detected by the sensor 30.

In such a state, for example, when the supervisor operates the object by clicking the object of the factory controlled object displayed in the virtual space on the 3D browser of the user terminal 50, Ja
The va module 14a detects the event, generates the operation data of the corresponding actuator, and activates the actuator transmission module 15. The activated actuator transmission module 15 transfers the operation data of the actuator to the actuator data text file 1
Store as 6. The actuator data text file 16 is accessed by the actuator control module 17 and applied to the sensor / actuator control system 20.

In the sensor / actuator control system 20, the input control data is converted into a signal suitable for actually driving the actuator, if necessary, and applied to the corresponding actuator. As a result, the object operated by the administrator is driven to a desired state. Also, the state of the driven object is
Immediately detected by the sensor, the VRML text of the server 10 is updated according to the signal flow described above. As a result, the state of that object in the 3D virtual space of the user terminal 50 is also updated to the state of the driven result.

As a result, the situation of the object to be monitored in the physical space of the factory is represented in the 3D virtual space of the server 10 substantially in real time according to the physical situation, and as a result, displayed on the user terminal 50. Even in the 3D image, the situation is displayed in real time. Further, when the observer accesses the 3D virtual space via the user terminal 50 and causes an event, for example, by clicking a button or a door, the actuator in the real space is actually driven and controlled. Also, the situation of the factory (real space) caused by this operation is immediately reflected in the virtual space via the sensor.

As described above, in the factory management system 1 of the present embodiment, the user can monitor and control the physical space via the interactive 3D virtual space interface constructed on the server 10. More specifically, the state of the factory is selectively displayed by a 3D (3D) image and only necessary objects unlike a television monitor. Therefore, the supervisor can intuitively, reliably, and accurately grasp the situation of the factory. As a result, for example, when an emergency occurs, it is possible to immediately deal with those phenomena.

In the present invention, the data of only the necessary objects is transmitted based on the VRML text data, not the actual moving image data. Therefore, even if it is applied to a system via a network, the data is not transmitted. There is no problem based on transmission volume. As a result, any client computer that is connected to the network and has a 3D browser can perform monitoring, control, and operation work from any computer, and the convenience is significantly improved.

The present invention is not limited to the above-described embodiment, but various suitable modifications can be made. For example, in the above-described embodiment, the computer network that connects the server 10 and the user terminal 50 is a LAN, but any network such as WAN or the Internet may be used.

The sensor data text file does not necessarily have to be written only from the sensor I / O control module of the server 10. For example,
Data may be collected by another server on the network, a client computer dedicated to data collection, or the like, and written by this. With such a configuration, more sensor data can be collected from a plurality of machines, and more appropriate monitoring and management can be performed.

Also, the sensors and actuators may be distributed and arranged via a network.

Further, the present invention may be an object of monitoring or control of any event, and the object of application is not limited at all. For example, facility security, route guidance,
It can be applied to home security, management of buildings, hospitals, offices, factories, maintenance of aircraft, etc.

[0042]

According to the present invention, an arbitrary monitored object in the physical space is displayed on an arbitrary terminal device so that the state of the monitored object can be selectively, intuitively and appropriately grasped. It is possible to provide an interface device and a system thereof that can directly and appropriately perform desired operation or control in the physical space based on the grasped contents.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a factory management system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a server of the factory management system shown in FIG.

[Explanation of symbols]

1 ... Factory management system 10 ... server 11 ... Sensor I / O control module 12 ... Sensor data text file 13 ... Sensor information acquisition module 14 ... VRML text data 14a ... Java module 15 ... Actuator transmission module 16 ... Actuator data text file 17 ... Actuator control module 20 ... Sensor / actuator control system 30 ... Sensor and actuator 40 ... Computer network 50 ... User terminal

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/18 H04N 7/18 D 13/02 13/02 13/04 13/04 (72) Inventor's tree Hideo Jiichi 2512-9 Isshiki, Hayama-cho, Miura-gun, Kanagawa 102 (72) Mycastle Yutaka Ohira 1-236 Tikicho, Hachioji-shi, Tokyo Soka University F-term (reference) 5B050 AA00 BA08 BA09 BA11 CA06 CA08 EA26 FA02 5C054 AA02 DA01 DA09 EA01 FD02 GB12 HA18 5C061 AA06 AA21 AB01 AB14 AB21 5E501 AA30 AC05 AC32 BA03 BA05 CA02 DA05 FA46 5H223 DD07 EE30 FF03 FF05

Claims (3)

[Claims] 1. A sensor for detecting a state of a monitored object, an actuator for driving a controlled object for changing the state of the monitored object, a monitored object space including the monitored object and the controlled object. A three-dimensional display means for displaying the state of the monitored object detected by the sensor in a distinguishable manner, and changing the state of the displayed monitored object, An interactive, state monitoring / displaying and driving device, comprising actuator driving means for driving the actuator to drive the displayed controlled object. 2. The three-dimensional display means enables the three-dimensional display of the monitoring target space on the browser by accessing the corresponding browser, and the data describing the monitoring target space in text. The interactive / state monitoring / displaying and driving apparatus according to claim 1, further comprising a data updating unit that updates the described data based on a state of the monitored object detected by a sensor. 3. A communication system, a server connected to the communication system, a terminal device connected to the communication system and capable of communicating with the server, and connected to the server to detect the state of a monitored object. A sensor and an actuator that drives a control target that changes the state of the monitoring target, and the server displays the monitoring target and a monitoring target space including the control target by a three-dimensional image. And a three-dimensional display unit for distinguishably displaying the state of the monitored object detected by the sensor, and driving the actuator to change the state of the displayed monitored object. An actuator driving unit that drives the displayed controlled object, wherein the terminal device accesses the three-dimensional display unit of the server. A display unit for displaying the three-dimensional image, and a command for driving the actuator to drive the displayed control target in order to change the state of the monitored target displayed, An interactive / state monitoring / displaying and driving device having a command receiving means for notifying the actuator driving means via a communication system.