CN113632122A - Maintenance method, maintenance server, and program - Google Patents

Maintenance method, maintenance server, and program Download PDF

Info

Publication number
CN113632122A
CN113632122A CN202080023845.6A CN202080023845A CN113632122A CN 113632122 A CN113632122 A CN 113632122A CN 202080023845 A CN202080023845 A CN 202080023845A CN 113632122 A CN113632122 A CN 113632122A
Authority
CN
China
Prior art keywords
maintenance
composite image
information
server
terminal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202080023845.6A
Other languages
Chinese (zh)
Inventor
谷本好史
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Murata Machinery Ltd
Original Assignee
Murata Machinery Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Murata Machinery Ltd filed Critical Murata Machinery Ltd
Publication of CN113632122A publication Critical patent/CN113632122A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/21—Monitoring or handling of messages
    • H04L51/224—Monitoring or handling of messages providing notification on incoming messages, e.g. pushed notifications of received messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Human Resources & Organizations (AREA)
  • Operations Research (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Quality & Reliability (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • Tourism & Hospitality (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Transfer Between Computers (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Access from an external terminal to a device is made easy and quick. A method for maintaining equipment in an automated warehouse system (1) using maintenance terminals (21a, 21b) provided at a maintenance site (2) includes: a step (S11) for creating a live image (FM) that represents the automated warehouse system (1); a step (S21-S23) of acquiring access information (AC); a step (S41-S44) of associating access information (AC) for accessing the corresponding device with an object (O) in the live image (FM) to generate a composite image (SI); a step (S37) for outputting the composite image (SI) to the maintenance terminals (21a, 21 b); a step (S3) for receiving a selection of an object (O); and a step (S51-S53) of establishing a communication session between the corresponding device and the maintenance terminal (21a, 21b) using the access information (AC) associated with the selected object (O).

Description

Maintenance method, maintenance server, and program
Technical Field
The present invention relates to a maintenance method for maintaining a device in a job site, a maintenance server for the maintenance job, and a program for causing a computer to execute the maintenance method.
Background
Conventionally, a technique for accessing a server or a terminal in a LAN and a terminal outside the LAN via a VPN (Virtual Private Network) is known (for example, see patent document 1). In the system of patent document 1, when a server or a terminal in a specific LAN is accessed from an external terminal via a VPN, information for connection to a gateway of the VPN is transmitted from the server that manages the information to the external terminal. The information for connecting to the gateway includes an address of the gateway and the like.
Documents of the prior art
Patent document
Patent document 1: japanese patent No. 5254909
Disclosure of Invention
Problems to be solved by the invention
It is considered that the above-described technique of connecting an external terminal to a terminal in a specific LAN or the like by VPN connection is applied to a work site provided with a plurality of devices such as an automated warehouse. This enables access to the devices in the job site from the external terminal via the network.
However, the conventional VPN connection technology is not assumed to be used when a plurality of devices exist at a work site. For example, in the conventional VPN connection technology, access information (address) to a gateway connected to the VPN is provided, but information related to the arrangement of terminals and servers in the LAN is not provided.
In a work site having a plurality of devices, there are cases where a target device is specified according to a location of the device in the work site, but if information on the device placement is not provided as in the conventional VPN connection, it is sometimes difficult to quickly access the target device from an external terminal.
The invention aims to make the access from an external terminal to a device easy and quick in a system for accessing the device in a job site from a terminal outside the job site.
Means for solving the problems
Hereinafter, a plurality of embodiments will be described as means for solving the problem. These means may be combined arbitrarily as required.
A maintenance method according to an aspect of the present invention is a method performed by a maintenance server for maintaining a device in a job site using a maintenance terminal provided at a maintenance site outside the job site. The maintenance method includes the following steps.
And creating a live view image indicating the work site.
And obtaining access information for accessing the device.
A step of generating a composite image by associating an object representing a device in the live image with access information for accessing the device.
And outputting the composite image to the maintenance terminal.
Accepting selection of an object in the composite image by using the maintenance terminal.
A step of establishing a communication session between the device corresponding to the selected object and the maintenance terminal using the access information associated with the selected object.
In the above-described maintenance method for maintaining a device in a work site using an external maintenance terminal, a composite image including an object indicating the arrangement and form of the device in the work site is generated. The generated composite image is output to an external maintenance terminal.
Further, when access information for accessing the corresponding device is associated with an object of the composite image and a certain object within the composite image is selected by the maintenance terminal, a communication session is established between the device corresponding to the object and the maintenance terminal using the access information associated with the selected object.
Thus, the user of the maintenance terminal can easily specify the device to be accessed with reference to the composite image, and can quickly access the specified device.
There may be a plurality of work sites. In this case, the live view image, the access information, and the composite image are generated for each of the plurality of job sites.
Thus, the user of the maintenance terminal can easily specify the devices existing in the plurality of work sites, and can quickly access the specified devices.
The maintenance method described above may further include the following steps.
A step of receiving abnormality information indicating an abnormality occurring in the device from the device.
And a step of generating a composite image by associating the abnormality information with an object corresponding to the device in which the abnormality has occurred.
Thus, the user who maintains the terminal can easily and quickly access information related to the abnormality occurring in the device.
The maintenance method described above may further include the following steps.
A step of generating a new composite image using new access information corresponding to the change of the device when the device in the job site has changed after the composite image is generated.
And outputting a new composite image to the maintenance terminal.
Thus, when there is a change in the equipment at the work site, the latest composite image reflecting the change can be provided to the maintenance terminal.
The maintenance method described above may further include the step of notifying the maintenance terminal that the composite image has been updated when a new composite image is generated.
This makes it possible to notify the user of the maintenance terminal of the update of the composite image and the presence of a change in the equipment at the work site.
The maintenance method described above may further include the step of displaying the dialog box together with the composite image. The dialog box provides a dialog function between a maintenance worker using the maintenance terminal and an operator using an operator terminal installed at a work site. In this case, a notification that the synthetic image has been updated is displayed in the dialog box.
This makes it possible to notify the worker at the work site of the update of the composite image and the presence of a change in the work site equipment.
A maintenance server according to another aspect of the present invention is a server for maintaining a device in a job site using a maintenance terminal provided at a maintenance site outside the job site. The maintenance server is provided with a control unit.
The control unit creates a live image representing a work site, acquires access information for accessing the device, associates an object representing the device in the live image with the access information for accessing the device to generate a composite image, outputs the composite image to the maintenance terminal, receives selection of an object in the composite image by using the maintenance terminal, and establishes a communication session between the device corresponding to the selected object and the maintenance terminal by using the access information associated with the selected object.
In the maintenance server, a composite image including an object indicating the arrangement and the form of the equipment in the work site is generated. The generated composite image is output to an external maintenance terminal.
Further, when access information for accessing the corresponding device is associated with an object of the composite image and a specific object within the composite image is selected by the maintenance terminal, a communication session is established between the device corresponding to the object and the maintenance terminal using the access information associated with the selected object.
Thus, the user of the maintenance terminal can easily specify the device to be accessed with reference to the composite image, and can quickly access the specified device.
There may be a plurality of work sites. In this case, the control unit generates a live view image, access information, and a composite image for each of the plurality of job sites.
Thus, the user of the maintenance terminal can easily specify the devices existing in the plurality of work sites, and can quickly access the specified devices.
The control unit may receive abnormality information indicating an abnormality occurring in the device from the device, and generate a composite image by associating the abnormality information with an object corresponding to the device in which the abnormality has occurred.
Thus, the user who maintains the terminal can easily and quickly access information related to the abnormality occurring in the device.
When there is a change in the device in the job site after the composite image is generated, the control unit may generate a new composite image using new access information corresponding to the change in the device and output the new composite image to the maintenance terminal.
Thus, when there is a change in the equipment at the work site, the latest composite image reflecting the change can be provided to the maintenance terminal.
The control unit may notify the maintenance terminal that the composite image has been updated when a new composite image is generated.
This makes it possible for the user of the maintenance terminal to notify the user of the update of the composite image and also to notify that there is a change in the work field device.
The control unit may display the dialog box together with the composite image. The dialog box provides a dialog function between a maintenance worker using the maintenance terminal and an operator using an operator terminal installed at a work site. In this case, the control unit displays a notification that the composite image has been updated in the dialog box.
This makes it possible to notify the worker at the work site of the update of the composite image and the presence of a change in the work site equipment.
Another aspect of the present invention relates to a program for causing a computer to execute the maintenance method described above.
Effects of the invention
By generating the composite image and outputting the composite image to an external maintenance terminal, it is possible to easily specify a device to be accessed by referring to the composite image in the external maintenance terminal, and to quickly access the specified device.
Drawings
Fig. 1 is a diagram showing an example of the overall configuration of a communication system.
Fig. 2 is a diagram showing an example of the configuration of the automated warehouse system.
Fig. 3 is a diagram showing an example of image data.
Fig. 4 is a diagram showing an example of measurement data of the sensor.
Fig. 5 is a diagram showing an example of the configuration of the maintenance site.
Fig. 6 is a diagram (1) showing an example of the maintenance information.
Fig. 7 is a diagram (2) showing an example of the maintenance information.
Fig. 8 is a diagram showing an example of a hardware configuration of the maintenance server.
Fig. 9 is a diagram showing a hardware configuration of the relay server.
Fig. 10A is a diagram showing an example of a gateway list.
Fig. 10B is a diagram showing a configuration of the composite image creation information.
Fig. 11A is a diagram showing an example of the data configuration of the first device list.
Fig. 11B is a diagram showing an example of the data configuration of the second device list.
Fig. 12 is a flowchart showing an outline of the operation of the maintenance server.
Fig. 13 is a flowchart showing the initial setting operation.
Fig. 14 is a flowchart showing an operation of acquiring access information.
Fig. 15 is a diagram showing transmission and reception of information during acquisition of access information.
Fig. 16 is a flowchart showing the operation of the maintenance server in the operation until the output of the composite image.
Fig. 17 is a diagram showing transmission and reception of information during an operation until output of a composite image.
Fig. 18 is a diagram showing an example of a login screen.
Fig. 19 is a flowchart showing an operation of generating a composite image.
Fig. 20 is a flowchart showing a communication session establishment operation of the maintenance server.
Fig. 21 is a diagram showing transmission and reception of information in the communication session establishment operation.
Fig. 22 is a flowchart showing the operation of the maintenance server in the case where there is a change in the device.
Fig. 23 is a diagram showing transmission and reception of information in a case where there is a change in the device.
Fig. 24 is a diagram showing an example of notification that the composite image has been updated.
Fig. 25 is a diagram showing an example of an initial screen.
Fig. 26A is a diagram showing an example of a composite image.
Fig. 26B is an enlarged view of a part of the composite image.
Fig. 27 is a diagram showing an example of a display screen on which a composite image and a dialog box are simultaneously displayed.
Fig. 28 is a diagram showing an example of a display screen on which access information is displayed as a list of characters.
Detailed Description
1. First embodiment
(1) Integral construction of communication system
The maintenance server according to the present invention will be explained below. The maintenance server is a server for maintaining a device installed at a work site using a maintenance terminal installed at a maintenance site located at a place separate from the work site. Therefore, the maintenance server is installed in a communication system including devices and terminals in the work site and terminals existing at the maintenance site.
A communication system 100 according to a first embodiment, which is an example of the above-described communication system, will be described below with reference to fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of a communication system.
The communication system 100 according to the first embodiment includes the automatic warehouse system 1, the maintenance site 2, the maintenance server 3, and the relay server 4, which are examples of the work site.
The automated warehouse system 1 is a system for loading and unloading goods, which is disposed in a logistics center or the like, for example. The automated warehouse system 1 includes an automated warehouse 11 (an example of a device) for loading and unloading goods (fig. 2). The communication system 100 of the present embodiment includes a plurality of automated warehouse systems 1.
The maintenance site 2 is, for example, a site where maintenance personnel who perform maintenance of the automated warehouse 11 in the automated warehouse system 1 reside, and includes terminals ( maintenance terminals 21a and 21b) for performing maintenance of the automated warehouse system 1. The maintenance terminals 21a, 21b of the maintenance site 2 can access the respective devices of the plurality of automated warehouse systems 1.
The maintenance server 3 is connected to (the gateway device of) the automatic warehouse system 1 and the maintenance site 2 via a network NW (e.g., a WAN). The maintenance server 3 stores a database that stores information (referred to as operation information) on (each equipment of) the automated warehouse 11 stored in the automated warehouse system 1 and information (referred to as maintenance information) used for maintaining the equipment managed at the maintenance site 2.
The maintenance server 3 supplies the stored operation information and/or maintenance information to the operator terminals used by the operators using the automated warehouse 11 in the automated warehouse system 1 and the maintenance terminals 21a and 21b used by the maintenance personnel at the maintenance site 2 as needed. Thus, in the communication system 100, the operation information and the maintenance information can be shared between the operator terminal and the maintenance terminals 21a and 21b via the maintenance server 3.
Further, when the maintenance terminals 21a and 21b are used to maintain the devices in the automated warehouse system 1, the maintenance server 3 mediates a communication session between the maintenance terminals 21a and 21b and the devices. How the mediation is performed will be described in detail later.
The relay server 4 is connected to (the gateway device of) the automated warehouse system 1 and the maintenance site 2 via the network NW. The relay server 4 is a server that relays communication between the automated warehouse system 1 and the maintenance site 2.
The relay server 4 is connected to the maintenance server 3 via the network NW, and supplies various information stored in the relay server 4 to the maintenance server 3 in response to a request from the maintenance server 3.
In the communication system 100 according to the present embodiment, transmission and reception of commands and the like between the devices (gateway device, camera and sensor of the automatic warehouse system 1) and the maintenance server 3, which will be described later, are performed using, for example, the WebSocket protocol.
In the communication system 100 shown in fig. 1, only 1 maintenance site 2 exists, but the present invention is not limited to this, and a plurality of maintenance sites 2 may exist in the communication system 100. For example, not only the maintenance of the plurality of automated warehouse systems 1 at the 1 maintenance site 2 but also the maintenance of the 1 automated warehouse systems 1 from the plurality of maintenance sites 2 may be performed.
(2) Automated warehouse system
(2-1) automated warehouse
A specific configuration of the automated warehouse system 1 included in the communication system 100 will be described below with reference to fig. 2. Fig. 2 is a diagram showing an example of the configuration of the automated warehouse system. The plurality of automated warehouse systems 1 have the same basic configuration, with only differences in the number of facilities and the like. Therefore, the basic configuration of the representative automated warehouse system 1 will be described below.
The automated warehouse system 1 has an automated warehouse 11. The automated warehouse 11 includes racks, stacker cranes, conveyors, and the like, and performs loading and unloading, transferring, and storing of goods.
In the present embodiment, cameras CA1 to CA4 capable of capturing still images and moving images having a predetermined time length, for example, are provided in the stacker crane and the conveyor of the automated warehouse 11, and the status of the stacker crane and the conveyor can be acquired as visual operation information FI. Cameras CA1 to CA4 are, for example, fisheye cameras. The cameras CA1 to CA4 can acquire image data (moving images) shown in fig. 3 as operation information FI, for example. Fig. 3 is a diagram showing an example of image data.
Further, a sensor SE (for example, a temperature/humidity sensor) for measuring ambient temperature and humidity is provided near the automated warehouse 11, and the ambient temperature and humidity can be acquired as the operation information FI. The sensor SE can acquire, for example, data showing the measured temperature and humidity in a line graph or a bar graph as shown in fig. 4 as measurement data (operation information FI). Fig. 4 is a diagram showing an example of measurement data of the sensor.
Further, the stacker crane is provided with a sensor SE for measuring a predetermined physical quantity, and when the stacker crane collapses, the level of vibration and/or sound generated by the collapse of the load can be measured as the operation information FI. Examples of such a sensor SE include an acceleration sensor and an acoustic sensor.
In the automatic warehouse system 1 shown in fig. 2, 4 cameras CA1 to CA4 and 1 sensor SE are provided, but the number of cameras and sensors provided in the automatic warehouse 11 is not limited to the above, and may be any number according to the scale of the automatic warehouse system 1.
(2-2) network constitution of automated warehouse System
The network configuration for mutual communication of the devices in the automated warehouse system 1 is further explained using fig. 2. The network of the automated warehouse system 1 is constructed with a first switching hub SH 1. As shown in fig. 2, the first switching hub SH1 connects the cameras CA1, CA2 with the first external storage device 13 a. The first external Storage device 13a is, for example, a NAS (Network Attached Storage) device, and stores image data acquired by cameras CA1 and CA 2.
Further, a control device (not shown) such as a stacker crane and a conveyor of the automated warehouse 11 is connected to the first switching hub SH 1.
The network of the automated warehouse system 1 is constructed with a second switching hub SH 2. The second switching hub SH2 connects the cameras CA3, CA4, the sensor SE, and the second external storage device 13 b. The second external storage device 13b is, for example, a NAS device, and stores image data acquired by the cameras CA3 and CA4 and measurement data acquired by the sensor SE.
The second switching hub SH2 is also connected to the first switching hub SH1, and image data acquired by the cameras CA1 and CA2 can be stored in the second external storage device 13b, and image data acquired by the cameras CA3 and CA4 and data acquired by the sensor SE can be stored in the first external storage device 13 a.
The network of the automated warehouse system 1 constitutes a server having a function of managing the automated warehouse system 1. Specifically, the system includes a delivery management server 15a and an information management server 15 b.
The transportation management server 15a is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. The conveyance management server 15a can communicate with stacker cranes, conveyors, and the like of the automated warehouse 11, and control the stacker cranes and the conveyors to control entry and exit of the goods in the automated warehouse 11.
In the present embodiment, the delivery management server 15a is connected to the first switching hub SH 1.
The information management server 15b is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. The information management server 15b manages various data (for example, data acquired by the cameras CA1 to CA4 and the sensor SE) stored in the first external storage device 13a and the second external storage device 13 b. The information management server 15b acquires information managed by the delivery management server 15a as necessary.
In the present embodiment, the information management server 15b is connected to the second switching hub SH 2.
The network of the automated warehouse system 1 includes an operator terminal 17. The operator terminal 17 is, for example, a personal computer, a tablet terminal, or the like. The operator terminal 17 is a terminal used by an operator who operates the automated warehouse 11, and can communicate with the maintenance server 3 via the network NW. Thus, the operator can use the operator terminal 17 to view the operation information FI and the maintenance information MI stored in the maintenance server 3. When the operator terminal 17 is a tablet terminal, the operator can carry the operator terminal with the installation place of the device to be maintained (automatic warehouse, camera, sensor) to perform the maintenance operation. When a terminal other than the tablet terminal (for example, a personal computer) is the operator terminal 17, the operator terminal 17 may access the cameras CA1 to CA4, the sensor SE, and the like.
In the automated warehouse system 1 shown in fig. 2, only 1 operator terminal 17 is provided, but the present invention is not limited to this, and a plurality of operator terminals 17 may be provided.
The network of the automated warehouse system 1 is configured with a first gateway device 19. The first gateway device 19 is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), and a network interface. The first gateway device 19 may be hardware that realizes a part or all of its functions by SoC or the like.
The first gateway device 19 is connected to the second switching hub SH2 and the external network NW, and can relay communication between each device in the network configuration of the automated warehouse system 1 and the outside. Therefore, the first gateway device 19 stores a first device list EL1 (described later) listing information on devices connected to the first gateway device 19 in the storage device.
Upon receiving an instruction from the relay server 4 to establish a communication session between each device (cameras CA1 to CA4, sensor SE, control devices of the stacker crane and the conveyor, etc.) of the automatic warehouse system 1 and the maintenance terminals 21a and 21b, the first gateway device 19 performs control to establish a VPN session with the second gateway device 23 (described later) of the maintenance site 2.
The first gateway device 19 has a function of uploading image data of the cameras CA1 to CA4, measurement data of the sensors SE, abnormality information indicating an abnormality occurring in each device, and the like, which are stored in the first external storage device 13a and the second external storage device 13b, to the maintenance server 3 at a predetermined timing, as the operation information FI. The first gateway device 19 can upload information related to maintenance of each device, operation information of each device, a photograph of the device taken by an operator or the like, information related to parts order of each device, and the like as operation information FI to the maintenance server 3.
The predetermined time may be appropriately determined, for example, at regular intervals (for example, at every 1 hour, every 1 day, or the like), at a time when new operation information FI is stored, at a time when existing operation information FI is updated, at a time when the capacity of stored operation information FI is a certain size, or the like.
(3) Maintenance site
A specific configuration of the maintenance site 2 included in the communication system 100 will be described below with reference to fig. 5. Fig. 5 is a diagram showing an example of the configuration of the maintenance site.
The maintenance site 2 has maintenance terminals 21a and 21 b. The maintenance terminals 21a, 21b are connected to a third switching hub SH 3. The maintenance terminals 21a and 21b are, for example, personal computers, and are used when maintenance personnel residing at the maintenance site 2 perform maintenance of the equipment of the automated warehouse system 1 and the like.
The maintenance site 2 has a second gateway device 23. The second gateway device 23 is a computer system having various interfaces such as a CPU, a storage device (RAM, ROM, SSD, hard disk, etc.), a network interface, and the like. The second gateway device 23 may be hardware that realizes a part or all of its functions by an SoC or the like.
The second gateway device 23 is connected to the third switching hub SH3 and an external network, and relays communication between the maintenance terminals 21a and 21b and the outside. Therefore, the second gateway device 23 stores a second device list EL2 (described later) listing information on the maintenance terminals 21a and 21b connected to the second gateway device 23 in the storage device.
The second gateway device 23 executes control of establishing a VPN session with the first gateway device 19 after issuing a VPN session establishment request from the first gateway device 19 of the automated warehouse system 1.
The second gateway device 23 has a function of uploading information stored in a backbone system (not shown) provided in the maintenance terminals 21a and 21b and/or the maintenance site 2 to the maintenance server 3 as maintenance information MI at a predetermined timing. The predetermined time can be appropriately determined, for example, at regular intervals (for example, at intervals of 1 hour, at intervals of 1 day, or the like), at a time when new maintenance information MI is stored, at a time when existing maintenance information MI is updated, at a time when the capacity of stored maintenance information MI is a certain size, or the like.
Examples of the maintenance information MI include a parts list which is a list of parts of the equipment shown in fig. 6, a layout diagram which shows the arrangement of the equipment (automated warehouse 11) in the automated warehouse system 1 shown in fig. 7, and the like. Fig. 6 and 7 are diagrams showing an example of the maintenance information. The second gateway device 23 can upload, as the maintenance information MI, an inspection history of each equipment of the automated warehouse system 1, an inspection report of each equipment, and the like to the maintenance server 3.
The maintenance server 3 can determine which equipment of the automatic warehouse system 1 is related to the maintenance information MI received from the second gateway apparatus 23 (maintenance site 2) based on the file name, for example.
(4) Maintenance server
A specific hardware configuration of the maintenance server 3 included in the communication system 100 will be described below with reference to fig. 8. Fig. 8 is a diagram showing an example of a hardware configuration of the maintenance server.
As shown in fig. 8, the maintenance server 3 is a computer system having a processor 31, a RAM32, a ROM33, a network interface 34, and a storage device 35.
The processor 31 is, for example, an SoC integrating a CPU or a CPU and various interfaces into 1 chip. The processor 31 executes programs stored in the storage device 35 and the like, and performs various information processes performed by the maintenance server 3.
The RAM32 stores temporarily necessary information such as programs stored in the storage device 35. The ROM33 stores programs, settings, and the like for controlling the maintenance server 3.
As described above, the processor 31, the RAM32, and the ROM33 are hardware components that perform control for realizing the functions of the maintenance server 3. Therefore, hereinafter, these 3 hardware components are referred to as "control unit" of the maintenance server 3. That is, the control unit executes the program stored in the storage device 35 to realize the function (operation) of the maintenance server 3 described later.
The network interface 34 performs communication with an external terminal and/or device or the like. The network interface 34 is, for example, an ethernet (registered trademark) card that communicates with an external terminal and/or device by wired communication, a wireless LAN interface that communicates with an external terminal and/or device by wireless communication, or the like.
When the maintenance server 3 transmits various information and data to the outside, the control unit outputs the information and data to the outside via the network interface 34. When the maintenance server 3 receives various information and data from the outside, the information and data are input from the outside to the control unit via the network interface 34.
The storage device 35 stores a program executed by the maintenance server 3, various kinds of information stored in the maintenance server 3, and various kinds of setting information. The various information stored in the maintenance server 3 according to the present embodiment is operation information FI related to the equipment (automatic warehouse 11) of the automatic warehouse system 1 and maintenance information MI related to maintenance of the equipment managed at the maintenance site 2.
On the other hand, the setting information stored in the storage device 35 of the maintenance server 3 according to the present embodiment is user information UI, a gateway list GL, and composite image creation information GI.
The user information UI is information for managing users who can log in the maintenance server 3. The gateway list GL is a list for identifying the first gateway device 19 of each of the automatic warehouse systems 1 and the second gateway device 23 of each of the maintenance sites 2. The composite image creation information GI is information used when creating a composite image SI (fig. 26A) for each of the automated warehouse systems 1 and each of the maintenance sites 2 and generating a composite image SI for each of the automated warehouse systems 1 and when determining from which of the maintenance terminals 21a and 21b of the maintenance site 2 an access is made.
With the above configuration, the maintenance server 3 can receive and store the operation information FI related to the equipment of the automatic warehouse system 1 and the maintenance information MI related to the maintenance of the equipment from the automatic warehouse system 1 and the maintenance site 2.
As will be described later, the maintenance server 3 having this configuration can generate an image (referred to as a composite image SI) in which an object O (described later) of the live image FM is associated with access information AC to a device corresponding to the object O. In the composite image SI, the operation information FI and the maintenance information MI relating to the corresponding equipment can be associated with each object O.
The maintenance server 3 may have a display 36 in addition to the above configuration. The display 36 is a display for displaying various information on the maintenance server 3, for example, a liquid crystal display, an organic EL display, or the like. Thus, the user can manage the maintenance server 3 while referring to the information displayed on the display 36, for example.
The maintenance server 3 may also have an input interface 37 that accepts input from a user. The input interface 37 is an input device such as a keyboard, a mouse, or a touch panel. Thereby, the user can directly manage the maintenance server 3 using the input interface 37.
Further, even if the input interface 37 and/or the display 36 described above are not provided, the maintenance server 3 can be managed while referring to information relating to the maintenance server 3, for example, using a terminal that can access the maintenance server 3 via the network interface 34.
(5) Relay server
A specific hardware configuration of the relay server 4 included in the communication system 100 will be described below with reference to fig. 9. Fig. 9 is a diagram showing a hardware configuration of the relay server.
As shown in fig. 9, the relay server 4 is a computer system having a processor 41, a RAM42, a ROM43, a network interface 44, and a storage device 45.
The processor 41 is, for example, an SoC integrating a CPU or a CPU and various interfaces into 1 chip. The processor 41 executes a program stored in the storage device 45 and the like, and performs various information processing performed in the relay server 4.
The RAM42 stores temporarily necessary information such as programs stored in the storage device 45. The ROM43 stores programs, settings, and the like for controlling the relay server 4.
The network interface 44 performs communication with an external terminal and/or device or the like. The network interface 44 is, for example, an ethernet (registered trademark) card that communicates with an external terminal and/or device by wired communication, a wireless LAN interface that communicates with an external terminal and/or device by wireless communication, or the like.
The storage device 45 is a storage device such as a hard disk or SSD that stores the program executed by the relay server 4 and the registration information LI. Further, the storage device 45 stores a setting file or the like in which settings necessary for operating the relay server 4 are described.
In other embodiments, the relay server 4 may also have a display 46. The display 46 is a display for displaying various information on the relay server 4, for example, a liquid crystal display, an organic EL display, or the like. Thus, the user can manage the relay server 4 while referring to the information displayed on the display 46, for example.
In other embodiments, the relay server 4 may have an input interface 47 that accepts input from a user. The input interface 47 is an input device such as a keyboard, a mouse, or a touch panel. Thereby, the user can directly manage the relay server 4 using the input interface 47.
Further, even if the input interface 47 and/or the display 46 are not provided, the relay server 4 can be managed while referring to information on the relay server 4, for example, using a terminal that can access the relay server 4 via the network interface 44.
(6) Information managed in a communication system
(6-1) information managed by the maintenance server
Hereinafter, information stored in the maintenance server 3, the relay server 4, the first gateway device 19, and the second gateway device 23 will be described. First, information managed by the maintenance server 3 will be described. As described above, the maintenance server 3 stores the user information UI, the composite image creation information GI, and the gateway list GL in the storage device 35.
As shown in fig. 10A, the gateway list GL is a list in which the identification information of each automated warehouse system 1 is associated with the identification information of the first gateway device 19 included in the automated warehouse system 1, for example. The gateway list GL stores the identification information of each maintenance site 2 in association with the identification information of the second gateway device 23 of the maintenance site 2. Fig. 10A is a diagram showing an example of a gateway list.
The identification information of each automatic warehouse system 1 and each maintenance site 2 is, for example, the identification numbers of the automatic warehouse system 1 and the maintenance site 2. The identification number of the automatic warehouse system 1 in the identification information is also included in the composite image creation information GI described later. In the example shown in fig. 10A, the identification number of the automatic warehouse system 1 is 100 to 199, and the identification number of the maintenance site 2 is 200 to 299. The identification numbers of the automated warehouse system 1 and the maintenance site 2 are not limited to the example shown in fig. 10A, and can be set arbitrarily.
On the other hand, the identification information of the first gateway device 19 and the second gateway device 23 is, for example, a registration ID for the first gateway device 19 and the second gateway device 23 to register the relay server 4. Further, the network addresses (IP addresses) of the first gateway device 19 and the second gateway device 23 may be set as the identification information of the gateway devices.
The user information UI stores a user ID and a password for logging in the maintenance server 3. Further, in the user information UI, a user type is associated with each user ID. The user type indicates whether the user ID is a type of a maintenance worker or a type other than the maintenance worker such as an operator. Further, information of each user (for example, the user's affiliation, contact information, authority level, and the like) may be stored in the user information UI.
As shown IN fig. 10B, the composite image creation information GI includes identification information IN, live image FM, related information AI, and first device list EL 1. Fig. 10B is a diagram showing a configuration of the composite image creation information. The identification information IN is information for identifying the automated warehouse system 1, and is the same as the identification information of the gateway list GL.
The field image FM is an image representing the automated warehouse system 1. In particular, the live image FM is a live image or digital model of the automated warehouse system 1. The field image FM is shown by imaging each device of the automated warehouse system 1. The device shown as an image is referred to as an object O.
The related information AI is information in which the operation information FI and the maintenance information MI are related to the equipment of the automatic warehouse system 1. Further, the association information AI establishes an association with the equipment of the automated warehouse system 1 and access information AC (described later) for accessing the equipment from the maintenance terminals 21a, 21 b.
The first device list EL1 is a list of information on devices connected to the first gateway apparatus 19. The specific configuration will be described in detail later.
(6-2) information managed by the Relay Server
Next, information managed by the relay server 4 will be described. The login information LI managed by the relay server 4 stores a user ID for logging in the relay server 4. The login information LI includes user IDs for the first gateway device 19 and the second gateway device 23. That is, the first gateway device 19 and the second gateway device 23 register the relay server 4.
In addition, in the login information LI, the user ID for the gateway apparatus is associated with the network address (for example, IP address) of the gateway apparatus having the user ID. In this way, the relay server 4 can know the network address of the corresponding gateway device from the user ID of the gateway device, and can access the gateway device.
(6-3) information managed by gateway device
Finally, information managed by the gateway apparatus will be described. Each gateway device manages a list of information on devices connected in its own LAN. The first gateway apparatus 19 manages the first device list EL1 shown in fig. 11A. On the other hand, the second gateway apparatus 23 manages the second device list EL2 shown in fig. 11B. Fig. 11A is a diagram showing an example of the data configuration of the first device list. Fig. 11B is a diagram showing an example of the data configuration of the second device list.
The equipment list shown in fig. 11A and 11B is an example, and the kinds and the number of the equipment to be listed are different depending on the scale of the automatic warehouse system 1 and the maintenance site 2.
(6-3-1) first device List
As shown in fig. 11A, the first device list EL1 managed by the first gateway apparatus 19 has a device name recording portion R1, an address recording portion R2, a sub-ID recording portion R3, a subtype recording portion R4, and a state recording portion R5.
The device name recording section R1 records the device name. In the first device list EL1, as shown in fig. 11A, names of cameras CA1 to CA4 (crane #1 camera to conveyor #2 camera), a name of a sensor SE (crane #1 sensor), a name of the first external storage device 13a (external storage device #1), a name of the second external storage device 13b (external storage device #2), a name of the information management server 15b (information management server), and a name of the transportation management server 15a (transportation management server) are recorded.
The address recording portion R2 records a network address assigned to a device capable of connecting with the gateway apparatus. In the first device list EL1, IP addresses "192.168.0.1" to "192.168.0.4" are recorded in the rows having the device names of the cameras CA1 to CA4, respectively. Further, an IP address of "192.168.0.11" is recorded in the row having the device name of the sensor SE.
Further, IP addresses "192.168.0.21" and "192.168.0.22" are recorded in the row having the device names of the external storage device #1 to the external storage device #2, respectively, and IP addresses "192.168.0.31" and "192.168.0.32" are recorded in the row having the device names of the information management server and the transportation management server, respectively.
In addition, IP addresses "192.168.0.111" and "192.168.0.112" are recorded in each row having a control device name (control panel) of the stacker crane, and IP addresses "192.168.0.41" and "192.168.0.42" are recorded in each row having a control device name (PLC) of the conveyor.
As described later, the IP address recorded in the address recording portion R2 is used when devices connected to the gateway apparatus communicate with each other via the gateway apparatus when a communication session is established between the gateway apparatuses.
The sub-ID recording portion R3 records a login ID (sub-ID) for the device to log in to the relay server 4. In the first device list EL1, the child IDs of "cam 001" to "cam 004" are recorded in the rows having the device names of the cameras CA1 to CA4, respectively, and the child ID of "sen 001" is recorded in the row having the device name of the sensor SE.
The child IDs of "ctrl 001" and "ctrl 002" are recorded in the row having the device name of the "control panel", and the child IDs of "PLC 001" and "PLC 002" are recorded in the row having the device name of the "PLC".
The subtype recording section R4 records a user type (subtype) indicating a category of the device connected to the gateway apparatus. In the first device list EL1, a subtype of "camera" is recorded in a row having device names of cameras CA1 to CA4, and a subtype of "sensor" is recorded in a row having a device name of sensor SE.
In addition, the subtype of "dial" is recorded in the row of the device name having "dial", and the subtype of "PLC" is recorded in the row of the device name having "PLC".
The status recording section R5 records whether or not each device recorded in the device list is in a state capable of communicating with the gateway apparatus. In the example of the first device list EL1, for example, "NG" is recorded in the line of the device names of the camera CA3 and the camera CA4, and "OK" is recorded in the other lines. That is, the cameras CA3, CA4 cannot communicate with the first gateway device 19.
In this way, the status recording portion R5 records "OK" for devices that can communicate with the gateway apparatus and "NG" for devices that cannot communicate among the devices (devices) connected to the gateway apparatus.
As shown in fig. 11A, information containing at least 1 line worth of information of the first device list EL1, that is, information of the IP address, device name, sub ID, sub type, and status for 1 device is access information AC, which is associated with the object O of the live image FM.
(6-3-2) second device list
Next, a second device list EL2 managed by the second gateway apparatus 23 will be described with reference to fig. 11B. The second device list EL2 has a device name recording portion R1, an address recording portion R2, a sub-ID recording portion R3, a subtype recording portion R4, and a status recording portion R5, as in the first device list EL 1.
In the second device list EL2, the name of the maintenance terminal 21a (maintenance terminal #1) and the name of the maintenance terminal 21b (maintenance terminal #2) are recorded in the device name recording part R1. In the address recording portion R2, IP addresses "172.28.0.2" and "172.28.0.3" are recorded in the rows having the device names of the maintenance terminals 21a and 21b, respectively.
On the other hand, no information is recorded in the sub-ID recording portion R3, the sub-type recording portion R4, and the status recording portion R5 of the second device list EL 2. This indicates that the maintenance terminals 21a and 21b cannot log in the relay server 4.
(7) Maintaining summary actions of servers
The operation of the maintenance server 3 of the communication system 100 according to the present embodiment will be described below. First, a schematic operation of the maintenance server 3 will be described with reference to fig. 12. Fig. 12 is a flowchart showing an outline of the operation of the maintenance server.
When accessing each device of the automated warehouse system 1 using the remote maintenance terminals 21a and 21b, the maintenance server 3 can easily identify a desired device and can quickly access the desired device. To realize this function, the maintenance server 3 executes processing according to the flowchart of fig. 12. In addition, the actions shown in fig. 12 are actions for 1 automated warehouse system 1. The maintenance server 3 individually executes the operations shown in fig. 12 for each of the plurality of automated warehouse systems 1 included in the communication system 100.
Specifically, first, in step S1, the maintenance server 3 executes an initial setting operation for generating the composite image SI. After the initial setting operation is performed, the maintenance server 3 generates the composite image SI in step S2 using the information created and acquired in the initial setting operation. The maintenance server 3 outputs the generated composite image SI to the corresponding maintenance terminals 21a, 21 b. Upon receiving the composite image SI, the maintenance terminals 21a and 21b cause a display (not shown) to display the composite image SI.
After outputting the composite image SI, the maintenance server 3 accepts selection of the object O in the composite image SI using the maintenance terminals 21a, 21b in step S3.
When the selection of the object O is accepted in step S3 (yes in step S3), the maintenance server 3 issues an instruction for establishing a communication session between the device corresponding to the selected object O and the maintenance terminals 21a, 21b using the access information AC associated with the selected object O in step S4.
(8) Maintaining specific actions of a server
(8-1) initial setting operation
The initial setting operation in step S1 will be specifically described below with reference to fig. 13. Fig. 13 is a flowchart showing the initial setting operation.
In the initial setting operation, first, in step S11, a live image FM is created and stored in the storage device 35 of the maintenance server 3.
When the live image FM is a live image of the automatic warehouse system 1, live images are captured at a plurality of locations of the automatic warehouse system 1. Next, the live images FM are generated by connecting the plurality of live images in consideration of the positions of the automatic warehouse system 1 at which the live images are captured. Further, the live image FM may be shot as a panoramic image.
On the other hand, when the live image FM is a digital model of the automatic warehouse system 1, a three-dimensional image of the automatic warehouse system 1 drawn by using CAD or the like is used as the live image FM.
Next, in step S12, the maintenance server 3 acquires access information AC for accessing each device of the automated warehouse system 1 from the maintenance terminals 21a and 21 b. In the present embodiment, the access information AC is acquired from the relay server 4. The operation of acquiring the access information AC will be described in detail later.
After the access information AC is acquired, in step S13, the maintenance server 3 associates the acquired access information AC with the object O in the live image FM. The association can be performed as follows, for example.
First, in the live image FM, an image indicating the equipment of the automatic warehouse system 1 is determined as the object O. Thereafter, the coordinate values of the specified objects O are acquired. The coordinate value of the object O may be, for example, an arrangement position (coordinate value) of a pixel representing the object O in the live image FM.
Next, the coordinate value of the object O and the access information AC of the device corresponding to the object O are stored in association with each other in the association information AI.
By executing the above-described steps S11 to S13, the maintenance server 3 obtains information necessary for generating the composite image SI. That is, the maintenance server 3 associates the live image FM, the related information AI, the first device list EL1 including the access information AC, and the identification information IN of the automated warehouse system 1 to obtain the composite image creation information GI.
In step S13, the maintenance server 3 receives the operation information FI and the maintenance information MI from the automatic warehouse system 1 and the maintenance site 2. Then, the maintenance server 3 stores the operation information FI and the maintenance information MI of the corresponding equipment in association with each object O of the live image FM in the association information AI. At this time, the maintenance server 3 associates the links to the operation information FI and the maintenance information MI with the objects O.
As described above, the operation information FI includes abnormality information indicating an abnormality occurring in the equipment of the automatic warehouse system 1. Therefore, by associating the operation information FI with each object O of the live image FM, the abnormality information can be associated with the object O corresponding to the device in which the abnormality has occurred, and the composite image SI can be generated.
Further, as the initial setting operation, the maintenance server 3 may execute setting operations such as user registration for logging in the maintenance server 3 and access restriction to information for each user.
(8-2) Access information acquisition action
Next, the operation of step S12 of the above-described initial setting operation, that is, the operation of acquiring the access information AC will be described in more detail with reference to fig. 14 and 15. Fig. 14 is a flowchart showing an operation of acquiring access information. Fig. 15 is a diagram showing transmission and reception of information during acquisition of access information.
In order to acquire the access information AC, the maintenance server 3 transmits a device list request signal to the relay server 4 in step S21.
Specifically, the maintenance server 3 first determines the identification number of the automated warehouse system 1 in which the desired access information AC exists. Then, in the gateway list GL, the registration ID of the first gateway device 19 associated with the identified identification number is specified.
After determining the login ID, the maintenance server 3 transmits the determined login ID together with the device list request signal to the relay server 4 (step S201).
The relay server 4 that has received the request signal determines which of the registration IDs transmitted together with the request signal is the registration ID of the first gateway device 19 of the automatic warehouse system 1. After that, the relay server 4 transmits a device list request signal to the identified first gateway device 19 (step S202).
Upon receiving the device list request signal from the relay server 4 (step S202), the first gateway apparatus 19 transmits the first device list EL1 held by itself to the relay server 4 (step S203).
After that, if the desired first device list EL1 is acquired from the first gateway device 19 (step S203), the relay server 4 transmits the first device list EL1 to the maintenance server 3 (step S204).
Upon receiving the first device list EL1 from the relay server 4 (yes in step S22), the maintenance server 3 stores the acquired first device list EL1 in the storage device 35 as the synthetic image creation information GI. After that, the maintenance server 3 extracts the access information AC of each device from the stored first device list EL 1.
As described above, the maintenance server 3 can receive the desired first device list EL1 from the corresponding first gateway device 19 via the relay server 4, and extract and acquire the access information AC from the received first device list EL 1.
(8-3) generating/outputting operation of composite image
Next, a specific operation of generating and outputting the composite image SI in step S2 will be described with reference to fig. 16 and 17. Fig. 16 is a flowchart showing the operation of the maintenance server in the operation until the output of the composite image. Fig. 17 is a diagram showing transmission and reception of information during an operation until output of a composite image. Hereinafter, a case where the maintenance person of the maintenance site 2 accesses the maintenance server 3 using the maintenance terminal 21a will be described as an example.
The generation and output of the composite image SI is started by the maintenance terminal 21a issuing a connection request to the maintenance server 3 from the maintenance server 3 whose initial setting operation is completed and which is operating (step S301). Specifically, for example, the address of the maintenance server 3 is specified in the web browser of the maintenance terminal 21a, and the web browser outputs a connection request to the maintenance server 3.
When receiving the connection request (yes in step S31), the maintenance server 3 outputs the login screen I1 to the maintenance terminal 21a for which there is the connection request in step S32 (step S302). For example, the login screen I1 shown in fig. 18 is output. Fig. 18 is a diagram showing an example of a login screen.
The maintenance person inputs his or her user ID into the user ID input field T1 and inputs his or her password into the password input field T2 on the login screen I1 displayed on the maintenance terminal 21 a. Thereafter, by pressing the login button B1, the input user ID and password are output to the maintenance server 3 as authentication information (step S303).
When receiving the authentication information from the maintenance terminal 21a (yes in step S33), the maintenance server 3 performs an authentication action in step S34. Specifically, the maintenance server 3 compares the received authentication information, i.e., the user ID and the password, with the user ID and the password stored in the user information UI.
As a result of the comparison, if the authentication fails due to the fact that the received authentication information is not present in the user information UI or the received authentication information is not matched with the user ID and the password stored in the user information UI (no in step S34), the maintenance server 3 ends the composite image output process.
When the authentication fails, the maintenance server 3 notifies the maintenance terminal 21a that wants to log in of the failure of the authentication. The maintenance server 3 may end the composite image output process after the authentication has failed a predetermined number of times.
On the other hand, when the authentication is successful by the received authentication information (yes in step S34), the maintenance server 3 outputs an initial screen I2 (fig. 25) to the maintenance terminal 21a for which the authentication is successful in step S25 (step S304). The configuration of the initial screen I2 will be described later.
After the maintenance terminal 21a displays the initial screen I2, the maintenance person selects a room desired to enter using the initial screen I2. The operator terminal 17 or the maintenance terminal 21a transmits a request for participation in a session to the selected room to the maintenance server 3 as a request for entering the session (step S305).
Upon receiving the session admission request from the maintenance terminal 21a, the maintenance server 3 generates a synthetic image SI in step S36. The operation of generating the composite image SI executed in step S36 will be described in further detail below with reference to fig. 19. Fig. 19 is a flowchart showing an operation of generating a composite image.
First, in step S41, the maintenance server 3 selects the operation information FI and the maintenance information MI that can be viewed by the target maintenance person and the devices that the maintenance person can access, for the devices corresponding to the objects O of the live image FM.
Specifically, the maintenance server 3 determines the operation information FI and the maintenance information MI that can be viewed by the maintenance person identified by the user ID. Further, the equipment that the maintenance person can access is determined.
Next, the maintenance server 3 refers to the related information AI and extracts the operation information FI and the maintenance information MI associated with each object O, and the operation information FI and the maintenance information MI that can be viewed by the maintenance person who is the object among the access information AC. Further, access information AC of the device that the maintenance person can access is extracted.
Further, the maintenance server 3 associates each object O with the operation information FI, the maintenance information MI, and the access information AC extracted for the corresponding equipment, and generates individual association information for each of the targeted maintenance staff.
After the unique related information is generated, the maintenance server 3 determines in step S42 whether or not the operation information FI, the maintenance information MI, or the access information AC is related to each object O of the live image FM in the unique related information.
If none of the operation information FI, the maintenance information MI, and the access information AC is associated with the object O in the unique associated information (no in step S42), the process of generating the synthetic image SI proceeds to step S44.
On the other hand, if any of the operation information FI, the maintenance information MI, and the access information AC is associated with the object O in the unique associated information (yes in step S42), the maintenance server 3 arranges a predetermined icon at the coordinate position of the live image FM where the object O is located in step S43. The predetermined icon indicates that the operation information FI and/or the maintenance information MI are associated with the object O.
Further, if the device corresponding to the object O is accessible using the maintenance terminal 21a, the access information AC of the corresponding device is displayed in the icon of the object O. As described later, the type (shape) of the icon to be arranged differs depending on the type of the operation information FI and/or the maintenance information MI to be associated with each other.
After the placement of the icons, the maintenance server 3 clicks the icon to generate a link that can refer to the operation information FI and the maintenance information MI selected in step S41, based on the unique related information.
At the same time, by clicking on the icon on which the access information AC is displayed, a command is generated to output an instruction (referred to as a session establishment instruction) to establish a communication session using the access information AC selected in step S41.
Here, the link to the operation information FI and the maintenance information MI is information indicating a storage location of the operation information FI and/or the maintenance information MI in the maintenance server 3. The link is, for example, a URL for accessing the operation information FI and/or the maintenance information MI.
In the automatic warehouse system 1 having a large number of facilities, there are usually a plurality of operation information FI and maintenance information MI associated with 1 object O (facility). Therefore, in the present embodiment, links to the operation information FI and the maintenance information MI associated with each object O are tabulated and managed as a list. In the present embodiment, links to the list are associated with the objects O.
On the other hand, since the access information AC is different for each device, the command to output the communication session establishment instruction is directly associated with each object O.
Through the above-described processing, in the live image FM, an icon can be arranged in the object O associated with any of the operation information FI, the maintenance information MI, and the access information AC, while an icon can be not arranged in the object O not associated with any of these pieces of information.
After the icons are arranged as necessary for the objects O, in step S44, it is confirmed whether or not the above-described steps S41 to S43 have been performed for all the objects O of the live image FM. That is, it is checked whether or not the operation information FI, the maintenance information MI, and the access information AC to be associated with each other are determined for all the objects O.
If the association of the information is not determined for all the objects O (no in step S44), the above-described S41 to S43 are further executed for the other objects O.
On the other hand, when the association of information is determined for all the objects O (yes in step S44), the generation of the composite image SI (fig. 26A) is terminated.
A specific configuration will be described later, but if any of the operation information FI, the maintenance information MI, and the access information AC is associated with the object O of the live image FM in the composite image SI, an icon is placed on the object O.
The shape of the icon differs depending on the type of information associated with the object O, and the access information AC is displayed together with the icon on the object O associated with the access information AC.
By such a composite image SI, it is possible to indicate to the maintenance person which kind of operation information FI, maintenance information, and access information AC is associated with which object O in a visually organized state.
Returning to fig. 16 and 17, after the composite image SI is generated as described above, the maintenance server 3 transmits the display screen I3 on which the dialog box a4 (fig. 27) is provided, together with the composite image SI, to the maintenance terminal 21a used by the maintenance person who generates the user ID of the composite image SI in step S37 (step S306).
For example, a dialog a4 is provided outside the region where the composite image SI is displayed.
When the operator of the automated warehouse system 1 accesses the maintenance server 3 using the operator terminal 17, the above steps S31 to S37 and steps S41 to S44 are also executed. That is, the composite image SI is generated, and the display screen I3 is output to the operator terminal 17. However, the access information AC is not associated with the object O of the composite image SI in this case. That is, the operator using the operator terminal 17 can read only the operation information FI and the maintenance information MI with reference to the composite image SI.
Further, the maintenance server 3 associates the identification information IN of the automated warehouse system 1 represented by the generated composite image SI with the composite image SI. When the object O of the composite image SI is selected (clicked), the identification information IN is included IN a selection result (described later) of the object O and transmitted to the maintenance server 3.
(8-4) summary of actions until output of composite image
The operations up to the output of the composite image SI by the maintenance server 3 described with reference to fig. 13 to 19 can be summarized as follows.
In the present embodiment, the maintenance server 3 creates a field image FM indicating the automatic warehouse system 1 (step S11), acquires access information AC for accessing a device (step S12, steps S21 to S23), generates a composite image SI by associating the access information AC for accessing the device with an object O indicating the device in the field image FM (step S13, steps S41 to S44), and outputs the composite image SI to the maintenance terminals 21a and 21b (step S37).
The access information AC for accessing the corresponding device is associated with the object O of the composite image SI generated as described above. Thus, the user (maintenance person) of the maintenance terminal 21a or 21b can easily specify the device to be accessed with reference to the composite image SI.
In the present embodiment, the maintenance server 3 acquires the operation information FI from the automatic warehouse system 1, and generates the composite image SI by associating the operation information FI with the corresponding object O. As described above, since the operation information FI includes the abnormality information indicating the abnormality occurring in the equipment of the automatic warehouse system 1, the maintenance person can easily and quickly access the information relating to the abnormality occurring in the equipment.
In the present embodiment, the maintenance server 3 outputs the display screen I3 on which the dialog box a4 is displayed together with the composite image SI. Further, the maintenance server 3 sets a dialog a4 together with the composite image SI generated using the user ID of the operator.
Thus, using the dialog box a4, the dialog function can be used between the maintenance person using the maintenance terminals 21a, 21b and the operator using the operator terminal 17.
(9) Communication session establishment actions
(9-1) communication Session establishment Act
As described above, when the composite image SI is generated and output, the display screen I3 that simultaneously displays the composite image SI and the dialog a4 is displayed on the maintenance terminals 21a, 21 b. By displaying the display screen I3, the provision of the composite image SI that can easily specify the accessible device is started. At the same time, the supply of the operation information FI and the maintenance information MI stored in the maintenance server 3 is started.
That is, the maintenance worker can easily and quickly access the device corresponding to the desired object O by clicking the icon disposed in the object O with reference to the composite image SI. At the same time, the operation information and the maintenance information MI associated with the object O can be accessed.
The following describes operations until a communication session is established between the maintenance terminals 21a and 21b and the device corresponding to the object O identified in the composite image SI, with reference to fig. 20 and 21. Fig. 20 is a flowchart showing a communication session establishment operation of the maintenance server. Fig. 21 is a diagram showing transmission and reception of information in the communication session establishment operation.
The establishment of a communication session between the maintenance terminal 21a of the login maintenance server 3 and the control panel (equipment name: crane #1 control panel) of the stacker crane of the automated warehouse 11 will be described as an example.
If the icon with access information AC is clicked on the composite image SI (fig. 26A) displayed on the display screen I3 of the maintenance terminal 21a (yes in step S3 of fig. 12), the information of the clicked icon is transmitted from the maintenance terminal 21a to the maintenance server 3 as the result of selection of the object O (step S501).
For example, when the icon with the access information AC of "ctrl 001192.168.0.111" is clicked, the maintenance terminal 21a transmits its own IP address (IN the LAN), the identification number of the maintenance site 2 where the maintenance terminal 21a is located, the access information AC, and the identification information IN associated with the composite image SI to the maintenance server 3 as the selection result of the object O.
In addition, when an icon not indicating that the access information AC is associated with the object O is clicked, the maintenance server 3 executes an operation associated with the icon.
Upon receiving the selection result of the object O from the maintenance terminal 21a, the maintenance server 3 determines which first gateway apparatus 19 the device that the maintenance server 3 wants to access is connected to in step S51.
Specifically, the maintenance server 3 determines which of the identification information of the first gateway devices 19 the identification information IN of the received selection result is associated with IN the gateway list GL. For example, when receiving the identification number of "101" as the identification information IN, the maintenance server 3 refers to the gateway list GL of fig. 10A to specify the first gateway device 19 having the registration ID "vpn 001".
Further, in step S51, the maintenance server 3 identifies the second gateway device 23 connected to the maintenance terminal 21a that transmitted the selection result of the object O. For example, when receiving the identification number of "201", the maintenance server 3 refers to the gateway list GL of fig. 10A to specify the second gateway device 23 having the registration ID "vpn 002".
Next, the maintenance server 3 determines access information AC of the device which the maintenance server 3 wants to access in step S52. Specifically, the maintenance server 3 determines the access information AC transmitted as a result of the selection of the object O as the access information AC of the device that is desired to access.
Further, in step S52, the maintenance server 3 specifies the IP address of the maintenance terminal 21a from the address of the maintenance terminal 21a received as the selection result of the object O.
Thereafter, in step S53, the maintenance server 3 transmits the registration ID (vpn001) of the first gateway device 19 specified in steps S51 to S52, the specified registration ID (vpn002) of the second gateway device 23, the specified access information AC, and the IP address of the maintenance terminal 21a to the relay server 4 as a connection request to the first gateway device 19 (step S502).
By executing the above-described steps S51 to S53, the maintenance server 3 can transmit an instruction for establishing a communication session between the maintenance terminal 21a and the equipment of the automated warehouse system 1 to the relay server 4 in the communication system 100.
After transmitting the instruction for establishing the communication session between the maintenance terminal 21a and the equipment of the automated warehouse system 1, the establishing action of the communication session is mainly performed by the relay server 4, the first gateway apparatus 19, and the second gateway apparatus 23. The set-up operation will be described below with reference to fig. 21.
The relay server 4 that has received the connection request specifies the network address of the first gateway device 19 from the user ID "vpn 001" of the connection request destination. Further, the network address of the second gateway device 23 to which the maintenance terminal 21a is connected is determined based on the user ID "vpn 002".
Then, the determined network address of the second gateway device 23, the access information AC, and the IP address of the maintenance terminal 21a are transmitted to the determined first gateway device 19 as a connection request (step S503).
The first gateway device 19 that has received the connection request determines whether or not to accept the connection request. When it is determined that the connection request is to be accepted, the first gateway device 19 transmits a notification indicating that the connection request is accepted to the second gateway device 23 having the network address included in the received connection request (step S504).
The second gateway apparatus 23 that has received the notification transmits the second device list EL2 of its own to the first gateway apparatus 19 that is the transmission source of the notification (step S505).
The first gateway apparatus 19 which has received the second device list EL2 transmits the first device list EL1 owned by itself to the second gateway apparatus 23 of the transmission source of the received device list (step S506).
Through the above steps S505 and S506, after exchanging the first device list EL1 and the second device list EL2 that the first gateway device 19 and the second gateway device 23 have with each other, a VPN session is established between them.
As described above, by establishing the VPN session between the first gateway apparatus 19 and the second gateway apparatus 23, the maintenance terminal 21a can access the device connected to the first gateway apparatus 19 via the second gateway apparatus 23 and the first gateway apparatus 19.
Further, when a VPN session is established between the first gateway apparatus 19 and the second gateway apparatus 23, the respective devices of the maintenance terminal 21a and the automated warehouse system 1 can communicate with each other using the IP addresses recorded in the address recording part R2 of the replaced first device list EL1 and second device list EL 2.
For example, after the VPN session is established, the device (device name: control panel of crane #1 control panel) of the automatic warehouse system 1 notifies the IP address "172.28.0.2" of the maintenance terminal 21a that the VPN connection to the device is possible together with the IP address "192.168.0.111" of the device.
The maintenance terminal 21a that has received the notification starts, for example, a maintenance application of the device. Thereby, the user (maintenance person) of the maintenance terminal 21a can perform maintenance of the device (control panel) using the application.
Through the VPN session, the devices of the maintenance terminal 21a and the automated warehouse system 1 can transmit and receive data to and from each other with security ensured by using IP addresses (for example, IP addresses such as "192.168. times. times.," 172.28. times. ") used in the LAN via the first gateway device 19 and the second gateway device 23.
(9-2) summary related to communication session establishment action
The operation of establishing a communication session between the maintenance terminals 21a and 21b and the equipment of the automated warehouse system 1 described with reference to fig. 20 and 21 can be summarized as follows.
As described above, when the maintenance terminals 21a and 21b select the specific object O (the object O to which the access information AC is attached) in the composite image SI (step S3 and step S501), the maintenance server 3 establishes a communication session between the device corresponding to the selected object O and the maintenance terminals 21a and 21b using the access information AC associated with the selected object O (steps S51 to S53 and step S502).
Thus, the user (maintenance person) of the maintenance terminal 21a or 21b can easily specify the device to be accessed with reference to the composite image SI, and can quickly access the specified device.
(10) Functions capable of being performed during display of composite image SI
(10-1) overview of functions that can be performed in displaying the composite image SI
After the display screen I3 shown in fig. 27 described above is displayed, the maintenance staff can perform a conversation in the maintenance terminals 21a and 21b using the conversation field a 4. Further, various functions using the composite image SI can be executed in addition to the dialog.
For example, the new composite image SI can be generated by associating the dialog content in the dialog box a4 with the composite image SI. When the dialog content in the dialog box a4 is clicked, the position (object O) of the composite image SI associated with the dialog content can be "jumped" (accessed).
Further, new operation information FI and maintenance information MI may be generated, or existing operation information FI and maintenance information MI may be edited and associated with the composite image SI.
Not only the operation information FI and the maintenance information MI are updated, but also the maintenance server 3 can reflect these changes in the composite image SI when the connection state of the equipment in the automatic warehouse system 1 changes, when there is a change in the equipment in the automatic warehouse system 1, or when there is an increase or decrease in the number of the equipment.
(10-2) update operation of composite image accompanying device change
Hereinafter, an operation in the communication system 100 in the case where there is a change in the device will be described with reference to fig. 22 and 23. Fig. 22 is a flowchart showing the operation of the maintenance server in the case where there is a change in the device. Fig. 23 is a diagram showing transmission and reception of information in a case where there is a change in the device.
For example, when a new gateway device is added to the automated warehouse system 1 and/or a device is removed and/or added to the existing automated warehouse system 1, the first gateway device 19 of the automated warehouse system 1 in which the device is changed generates (updates) the first device list EL1 corresponding to the new device change. The first gateway device 19 transmits the new first device list EL1 to the relay server 4 (step S601).
The relay server 4 that received the new first device list EL1 transmits it to the maintenance server 3 (step S602). At this time, the relay server 4 transmits the identification information IN (for example, the login ID) of the first gateway apparatus 19 that transmitted the new first device list EL1 together with the new first device list EL 1.
The maintenance server 3 that received the new first device list EL1 updates the composite image SI to correspond to the new first device list EL 1. The updating of the composite image SI is performed as follows.
First, in step S61, the maintenance server 3 performs new association of each access information AC of the new first device list EL1 with the object O in the composite image SI.
Specifically, the maintenance server 3 first determines which composite image creation information GI should be changed, based on the identification information IN transmitted together with the new first device list EL 1. Next, the first device list EL1 of the corresponding composite image creation information GI is replaced with a new first device list EL 1. Further, the maintenance server 3 associates the object O with the new access information AC included in the new first device list EL1, and generates new association information AI.
In addition, the association of the object O with the new access information AC in the above step S61 can be realized by executing the above-described steps S11 to S13 (fig. 13) and steps S21 to S23 (fig. 14) based on the new first device list EL 1. Therefore, a detailed description is omitted here.
After performing the new association between the object O and the access information AC, the maintenance server 3 generates a new composite image SI based on the new composite image creation information GI in step S62, and outputs the new composite image SI to the maintenance terminals 21a and 21b (step S603).
In addition, the generation and output of the new synthesized image SI of the above-described step S52 can be realized by performing steps S31 to S37 (fig. 16) and steps S41 to S44 (fig. 19) based on the new association. Therefore, a detailed description is omitted here.
In this way, when there is a change in the equipment of the automated warehouse system 1 after the generation of the composite image SI, the new composite image SI is generated and output using the new access information corresponding to the change in the equipment, and when there is a change in the equipment in the automated warehouse system 1, the latest composite image SI reflecting the change can be supplied to the maintenance terminals 21a and 21 b.
After generating the new composite image SI as described above, the maintenance server 3 notifies the maintenance terminals 21a and 21b that the composite image SI is updated in step S63.
For example, as shown in fig. 24, the notification is an instruction to display a pop-up window PU having a display of "updated composite image" on the display screen I3 in a predetermined area (lower area in the example of fig. 24) of the dialog a 4. Fig. 24 is a diagram showing an example of notification that the composite image has been updated. The notification may be a command for causing the maintenance terminals 21a and 21b to generate a predetermined sound.
Alternatively, a message of "updated composite image" may be issued to the dialog box a 4. Further, the message may include a link to the updated data, and the updated data in the composite image SI may be accessed from the link in the message.
In this way, when a new composite image SI is generated, the maintenance server 3 notifies the maintenance terminals 21a and 21b of the fact, and the maintenance person using the maintenance terminals 21a and 21b can easily recognize that a new composite image SI (the latest composite image SI) is generated, that is, that the composite image SI is updated. Further, the maintenance person can recognize that there is a change of the equipment in the automated warehouse system 1 by recognizing that there is an update of the composite image SI.
When the maintenance person using the maintenance terminals 21a and 21b has a conversation with the operator using the operator terminal 17, the above-described pop-up window is also displayed in the conversation field a4 of the operator terminal 17. Thus, the operator of the automated warehouse system 1 can also notify that there is a change in the equipment in the automated warehouse system 1 together with the update of the composite image SI.
(11) Structure of picture display
(11-1) initial Picture
The configuration of the display screen displayed on the display of the maintenance terminals 21a and 21b will be briefly described below. The display screen shown below is an example, and the configuration and/or appearance of the display screen can be appropriately changed.
First, a configuration of an initial screen I2 displayed on the operator terminal 17 and the maintenance terminals 21a and 21b after the maintenance server 3 is registered will be described with reference to fig. 25. Fig. 25 is a diagram showing an example of an initial screen. The initial screen I2 has a use history display section a 1. The use history display field a1 displays a use history list L1 for displaying a list of use histories, an update button B2 for updating the use histories, and a memo field T3 for recording memos.
The initial screen I2 has a room selection bar a 2. A selection button B3 for selecting which room of the dialog to enter is displayed in the room selection column a 2. When any one of the selection buttons B3 is pressed, it is required to enter the room of the dialog of the selected button.
Further, the button of the room which the user cannot enter or exit is not displayed or is invalid (cannot be selected).
The initial screen I2 has a user information change field A3. In the user information change field a3, an information edit field T4 for editing information of the user (user ID, password, name of the user who has established an association with the user ID), an image change button B4 for changing the image (avatar) of the user, a category change field M1 for changing the category of the account, and an update button B5 for updating the information of the user if pressed are displayed.
(11-2) composite image
Next, the structure of the composite image SI will be described with reference to fig. 26A and 26B, and fig. 26A is a diagram showing an example of the composite image. Fig. 26B is an enlarged view of a part of the composite image. The composite image SI displays a plurality of objects O representing the equipment of the automated warehouse system 1. Specifically, the various automated warehouses 11 included in the automated warehouse system 1 are displayed as objects O.
In the composite image SI, an icon is arranged in each object O. In the example of the composite image SI shown in fig. 26A, eight kinds of icons (ICO1 to ICO8) are arranged. The type of icon displayed in the composite image SI can be arbitrarily determined according to the type of information related to the device, and the like.
The first icon ICO1 has a shape with a circle surrounding the "i" character. The first icon ICO1 indicates that, in the equipment in which the icon is placed, various information (such as a manual) related to the equipment is associated as the operation information FI and/or the maintenance information MI. Further, a character "×" INFO "(" × "is a name of a device, the same will be applied hereinafter) is given below the first icon ICO1, and specifically indicates an object in which the first icon ICO1 is disposed.
The second icon ICO2 has a shape that mimics a miniature camera. The second icon ICO2 indicates that image data acquired by the camera CA provided in the device in which the icon is arranged is associated as operation information FI. Note that a character "camera" is given below the second icon ICO2, and specifically indicates an object on which the second icon ICO2 is disposed.
The third icon ICO3 has a shape that mimics a thermometer and hygrometer. The third icon ICO3 indicates that, in the device indicated by the icon, the data (temperature and humidity) acquired by the sensor SE installed in the device is associated as the operation information FI. Note that a character of ". dot.. wet temperature" is given below the third icon ICO3, and specifically indicates the target in which the third icon ICO3 is disposed.
The fourth icon ICO4 has the shape of a document. The fourth icon ICO4 indicates that, in the device labeled with this icon, the part list of the device is associated as the maintenance information MI. Further, characters of "× parts list" are marked below the fourth icon ICO4, and specifically, which object the fourth icon ICO4 is disposed in is shown.
The fifth icon ICO5 has a shape that mimics a large camera. A fifth icon ICO5 indicates that image data acquired by a large camera with the icon attached thereto is associated as operation information FI. Further, a character "camera" is attached below the fifth icon ICO5, and this indicates that the fifth icon ICO5 is disposed as an object of the large camera.
The sixth icon ICO6 has a shape that is marked with an arrow next to the inverted triangular shape of the figure. The sixth icon ICO6 is provided at a predetermined position of the live image FM (composite image SI), and indicates that the view of the composite image SI is moved in the direction of the icon when the icon is clicked.
When the sixth icon ICO6 is pressed or operated, the center of the field of view of the composite image SI becomes the position of the selected sixth icon ICO6, and the direction of the field of view becomes the direction of the front face of the sixth icon ICO 6. This enables realization of virtual reality like moving the automated warehouse system 1 in the composite image SI.
The seventh icon ICO7 is provided at a predetermined position (above the composite image SI in the example of fig. 26A) of the live image FM (composite image SI) and has a shape of a switch. In addition, a character of "View Mode" is marked below the lower side. The seventh icon ICO7 has a function of switching between display and non-display of the first icon ICO1 to the sixth icon ICO6 by clicking.
The eighth icon ICO8 is provided at a predetermined position (above the composite image SI in the example of fig. 26A) of the live image FM (composite image SI) and has a shape of a switch. Further, a character of "VPN" is marked thereon. The eighth icon ICO8 has a function of switching between display and non-display of the access information AC in the composite image SI by clicking.
As shown in fig. 26B (an enlarged view of a portion surrounded by a broken line in fig. 26A), several icons included in the composite image SI display the access information AC. In the example shown in fig. 26B, "ctrl 001192.168.0.111" and the child ID and IP address of the device (control panel of stacker crane) are displayed as the access information AC between the description of "PL 1_ INFO" and the first icon ICO 1. Further, "cam001192.168.0.1" and the child ID and IP address of the device (camera CA) are displayed as the access information AC between the description of "PL 1_ crane camera" and the second icon ICO 2.
(11-3) display of composite image and dialog box
The configuration of the display screen I3 on which the composite image SI and the dialog a4 are simultaneously displayed will be described below with reference to fig. 27. Fig. 27 is a diagram showing an example of a display screen on which a composite image and a dialog box are simultaneously displayed.
As shown in fig. 27, on the display screen I3, the room selection field a2 and the dialog field a4 described above are displayed outside the area where the composite image SI is displayed. In the example shown in fig. 27, the room selection field a2 is displayed on the left side of the composite image SI, and the dialog field a4 is displayed on the right side.
In the dialog box a4, the dialog contents that are performed at a predetermined time in the dialog are arranged in time series as the speech bubble CH. In the dialog box a4, a file selection box FS is disposed. The file selection field FS selects a file stored in the operator terminal 17, the maintenance terminal 21a, or the like, and adds the selected file to the contents of the conversation (speech bubble CH).
The file attached to the dialog content can be associated with a predetermined position or object O of the composite image SI as the operation information FI or the maintenance information MI by dragging the file to the composite image SI.
The file associated with the contents of the session can be associated with the operation information FI when the above-described operation is performed by the operator terminal 17, and can be associated with the maintenance information MI when the above-described operation is performed by the maintenance terminals 21a and 21b, for example.
Further, as shown in fig. 28, by switching the display state on the display screen I3, the access information AC associated with the object O of the composite image SI can also be displayed in a list by characters. Fig. 28 is a diagram showing an example of a display screen on which access information is displayed as a list LST of characters.
2. Other embodiments
While one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. In particular, the plurality of embodiments and modifications described in the present specification can be arbitrarily combined as needed.
The order of the processes in the flowcharts and the contents of the processes in the steps described above can be changed without departing from the scope of the present invention.
(A) The composite image SI may not have an icon indicating that the access information AC is associated with the object O. In this case, for example, when a cursor (e.g., a mouse cursor) is arranged on the composite image SI, if the access information AC is associated with the object O pointed by the cursor, the access information AC of the object O may be displayed. Further, the access information AC itself may be displayed without being associated with the object O, and for example, a message asking whether or not to access the device corresponding to the object O may be displayed. That is, the access information AC may be associated with the object O in the composite image SI, and the specific display mode is not limited.
(B) In addition to the user ID and the password, user authentication using human body information (such as a fingerprint, a blood vessel (vein) pattern, an iris pattern, a face pattern, and the like) of the worker and the maintenance person can be used.
Further, for example, user authentication using identification information (for example, a MAC address, an IP address, and the like) of the worker terminal 17 and the maintenance terminals 21a and 21b can be used. In this case, the viewing authority can be set in units of the operator terminal 17, the maintenance terminals 21a and 21b, the LAN group, the automated warehouse system 1, the maintenance site 2, and the like.
(C) The communication system 100 described above can be used in a system in which information is shared at isolated locations, in addition to the use of information sharing between the automated warehouse system 1 and the maintenance site 2 that maintains the facility of the automated warehouse system 1.
(D) The dialog a4 may be displayed in a state of overlapping a part of the composite image SI. In this case, the dialog a4 and/or the composite image SI may be moved on the screen.
Industrial applicability
The present invention can be widely applied to a communication system for remotely maintaining equipment in an operation site.
Description of the symbols:
100: a communication system; 1: an automated warehouse system; 11: an automated warehouse; 13 a: a first external storage device; 13 b: a second external storage device; 15 a: a delivery management server; 15 b: an information management server; 17: an operator terminal; 19: a first gateway device; 2: maintaining a base; 21a, 21 b; maintaining the terminal; 23: a second gateway device; 3: maintaining the server; 31: a processor; 32: a RAM; 33: a ROM; 34: a network interface; 35: a storage device; 36: a display; 37: an input interface; 4: a relay server; 41: a processor; 42: a RAM; 43: a ROM; 44: a network interface; 45: a storage device; 46: a display; 47: an input interface; a1: a history display section; a2: a room selection bar; a3: a user information change bar; a4: a dialog bar; AC: accessing the information; AI: correlation information; b1: a login button; b2: an update button; b3: a selection button; b4: an image change button; b5: an update button; CA: a camera; CA 1-CA 4: a camera; CH: a speech bubble; EL 1: a first device list; EL 2: a second device list; FI: operation information; FM: a live image; FS: a file selection field; GI: synthesizing image production information; GL: a gateway list; i1: a login picture; i2: an initial picture; i3: displaying a picture; ICO 1: a first icon; ICO 2: a second icon; ICO 3: a third icon; ICO 4: a fourth icon; ICO 5: a fifth icon; ICO 6: a sixth icon; ICO 7: a seventh icon; ICO 8: an eighth icon; IN: identifying information; l1: using the history list; and LI: login information; m1: a category change field; MI: maintaining the information; NW: a network; o: an object; PU (polyurethane): popping up a window; r1: a device name recording section; r2: an address recording section; r3: a sub-ID recording section; r4: a subtype recording section; r5: a status recording section; and SE: a sensor; SH 1: a first switching hub; SH 2: a second switching hub; SH 3: a third switching hub; and (3) SI: synthesizing an image; t1: a user ID input field; t2: a password input field; t3: a memo bar; t4: an information editing column; UI (user interface): and (4) user information.

Claims (13)

1. A maintenance method performed by a maintenance server for maintaining a device in a job site using a maintenance terminal of a maintenance site installed outside the job site, comprising:
creating a site image representing the work site;
acquiring access information for accessing the device;
generating a composite image by associating an object indicating the device in the live image with the access information for accessing the device;
outputting the synthesized image to the maintenance terminal;
receiving selection of the object in the composite image by using the maintenance terminal; and
and establishing a communication session between the device corresponding to the selected object and the maintenance terminal using the access information associated with the selected object.
2. The maintenance method according to claim 1,
there are a plurality of the above-mentioned work sites,
the site image, the access information, and the composite image are generated for each of a plurality of job sites.
3. The maintenance method according to claim 1 or 2,
the maintenance method further comprises:
Receiving abnormality information indicating an abnormality occurring in the device from the device; and
and associating the abnormality information with the object corresponding to the device in which the abnormality has occurred to generate the composite image.
4. The maintenance method according to any one of claims 1 to 3,
the maintenance method further comprises:
a step of generating a new composite image using new access information corresponding to a change in the device, when the device in the job site has been changed after the composite image is generated; and
and outputting the new composite image to the maintenance terminal.
5. The maintenance method according to claim 4,
the maintenance method may further include the step of notifying the maintenance terminal that the composite image has been updated when the new composite image is generated.
6. The maintenance method according to claim 5,
the maintenance method further includes displaying a dialog box providing a dialog function between a maintenance worker using the maintenance terminal and an operator using an operator terminal installed at the work site together with the composite image,
And displaying a notification that the composite image has been updated in the dialog box.
7. A maintenance server for maintaining a device in a work site by using a maintenance terminal of a maintenance site installed outside the work site,
the maintenance server is provided with a control part,
the control part is used for controlling the operation of the motor,
creating a site image representing the work site;
obtaining access information for accessing the device;
generating a composite image by associating an object representing the device in the live image with the access information for accessing the device;
outputting the synthesized image to the maintenance terminal;
receiving selection of the object in the composite image by using the maintenance terminal; and
and establishing a communication session between the device corresponding to the selected object and the maintenance terminal using the access information associated with the selected object.
8. The maintenance server of claim 7,
there are a plurality of the above-mentioned work sites,
the control unit generates the site image, the access information, and the composite image for each of a plurality of work sites.
9. The maintenance server according to claim 7 or 8,
The control unit receives abnormality information indicating an abnormality occurring in the device from the device, and generates the composite image by associating the abnormality information with the object corresponding to the device in which the abnormality has occurred.
10. The maintenance server according to any one of claims 7 to 9,
when there is a change in the device in the job site after the composite image is generated, the control unit generates a new composite image using new access information corresponding to the change in the device, and outputs the new composite image to the maintenance terminal.
11. The maintenance server of claim 10,
when the new composite image is generated, the control unit notifies the maintenance terminal that the composite image has been updated.
12. The maintenance server of claim 11,
the control unit displays a dialog box providing a dialog function between a maintenance worker using the maintenance terminal and an operator using an operator terminal installed at the work site, together with the composite image, and displays a notification that the composite image has been updated in the dialog box.
13. A process in which, in the presence of a catalyst,
causing a computer to perform the maintenance method of any one of claims 1 to 6.
CN202080023845.6A 2019-04-05 2020-03-04 Maintenance method, maintenance server, and program Pending CN113632122A (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2019-072891 2019-04-05
JP2019072891A JP7456094B2 (en) 2019-04-05 2019-04-05 Maintenance method and maintenance server
PCT/JP2020/009045 WO2020202993A1 (en) 2019-04-05 2020-03-04 Maintenance method, maintenance server, and program

Publications (1)

Publication Number Publication Date
CN113632122A true CN113632122A (en) 2021-11-09

Family

ID=72668629

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080023845.6A Pending CN113632122A (en) 2019-04-05 2020-03-04 Maintenance method, maintenance server, and program

Country Status (5)

Country Link
US (1) US11829958B2 (en)
EP (1) EP3951686A4 (en)
JP (1) JP7456094B2 (en)
CN (1) CN113632122A (en)
WO (1) WO2020202993A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7010417B1 (en) * 2021-01-14 2022-01-26 住友電気工業株式会社 Information provision system, server device, information provision method, and information provision program

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3406760B2 (en) 1996-01-26 2003-05-12 株式会社東芝 Monitoring and control equipment
JP2002032880A (en) 2000-07-14 2002-01-31 Hitachi Service & Engineering (East) Ltd Remote maintenance system for equipment
ITBO20000608A1 (en) 2000-10-18 2002-04-18 Gd Spa METHOD AND AUTOMATIC MACHINE FOR THE PROCESSING OF A PRODUCT
US20070233844A1 (en) 2006-03-29 2007-10-04 Murata Kikai Kabushiki Kaisha Relay device and communication system
JP4222397B2 (en) 2006-09-12 2009-02-12 村田機械株式会社 Relay server
US9244941B2 (en) * 2009-03-18 2016-01-26 Shutterfly, Inc. Proactive creation of image-based products
JP5254909B2 (en) 2009-09-02 2013-08-07 日本電信電話株式会社 Setting information distribution system, setting information distribution method, distribution server, and distribution server program
JP2011055455A (en) 2009-09-04 2011-03-17 Murata Machinery Ltd Management relay server and relay communication system
US9581994B2 (en) * 2011-04-05 2017-02-28 Fisher-Rosemount Systems, Inc. Methods and apparatus to manage process control resources
JP2013183447A (en) 2012-03-05 2013-09-12 Murata Mach Ltd Relay communication system, relay server and management server
JP2016192000A (en) 2015-03-31 2016-11-10 横河電機株式会社 Operation support device and operation support method
JP2016201007A (en) 2015-04-13 2016-12-01 Ｉｈｉ運搬機械株式会社 Transportation facility remote maintenance system
US11487353B2 (en) * 2016-11-14 2022-11-01 Logitech Europe S.A. Systems and methods for configuring a hub-centric virtual/augmented reality environment
US20180275940A1 (en) * 2017-03-27 2018-09-27 Wipro Limited Personalized display system and method for dynamically displaying user information
JP6845072B2 (en) 2017-04-21 2021-03-17 ファナック株式会社 Factory equipment maintenance support equipment and maintenance support system
US10996742B2 (en) * 2017-10-17 2021-05-04 Logitech Europe S.A. Input device for AR/VR applications
US11314399B2 (en) * 2017-10-21 2022-04-26 Eyecam, Inc. Adaptive graphic user interfacing system
US10809884B2 (en) * 2017-11-06 2020-10-20 The Sherwin-Williams Company Paint color selection and display system and method
US11049075B2 (en) * 2017-12-13 2021-06-29 Ariel D. Weindling Victim reporting and notification system and alert mechanism for organizations
US20220083978A1 (en) * 2017-12-13 2022-03-17 Ariel D. Weindling Misconduct metrics reporting generation and rendering engine apparatuses, methods, systems and media
US10846935B2 (en) * 2018-04-12 2020-11-24 PRO Unlimited Global Solutions, Inc. Augmented reality campus assistant
US11042769B2 (en) * 2018-04-12 2021-06-22 PRO Unlimited Global Solutions, Inc. Augmented reality badge system
US11080935B2 (en) * 2019-04-03 2021-08-03 Wayfair Llc Systems and methods for scene-independent augmented reality interfaces
US11457730B1 (en) * 2020-10-26 2022-10-04 Wells Fargo Bank, N.A. Tactile input device for a touch screen

Also Published As

Publication number Publication date
WO2020202993A1 (en) 2020-10-08
EP3951686A1 (en) 2022-02-09
US11829958B2 (en) 2023-11-28
EP3951686A4 (en) 2022-12-07
JP2020170452A (en) 2020-10-15
US20220164777A1 (en) 2022-05-26
JP7456094B2 (en) 2024-03-27

Similar Documents

Publication Publication Date Title
JP6364241B2 (en) Inspection support device, inspection support method, inspection support program
EP3148160B1 (en) Information processing apparatus, information processing method, and program
JP2020087245A (en) Error display system, error display method, and information processing device
CN113631489B (en) Maintenance method and maintenance server
JP2008234185A (en) Nurse assignment apparatus and computer program
CN109977081A (en) A kind of content sharing method and terminal device
CN113632122A (en) Maintenance method, maintenance server, and program
JP2023112117A (en) Test result reporting method and test result reporting system
JP5977586B2 (en) Information processing system, information processing apparatus, information processing method, and program
CN110945539A (en) Order management system, order management device, and program
US11463298B2 (en) Mediating apparatus, device management system, communication control method, and non-transitory recording medium
KR100946708B1 (en) System of Guiding Visitor and Method there of
JP2022091538A (en) Information processing device, request management system, request management method, and program
WO2021054073A1 (en) Maintenance system, server, maintenance method, and program
JP6522416B2 (en) Business communication system and computer program
JP7018677B1 (en) Work status management system
WO2010143415A1 (en) Communication control system, communication control device, communication control method and communication control program
JP3737503B2 (en) Monitoring system, monitoring method and program thereof
JP7340573B2 (en) Information processing method, information processing system
JP2002354552A (en) Server device, server program and record medium thereof
JP7324481B2 (en) Business card exchange system, application program, business card exchange method
JP7310628B2 (en) Program and communication system
JP2021051593A (en) Image processing system, image processing device and method
JP2024025318A (en) Information processing apparatus, information processing system, task management method, and program
JP2019091123A (en) Information processing system, terminal device, information processor, and information processing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination