JP2022112342A - Equipment construction confirmation system, and equipment construction confirmation method - Google Patents

Abstract

To provide an equipment construction confirmation system which reduces burden of an operator engaging in confirmation work of equipment construction and to secure accuracy in equipment confirmation.SOLUTION: An equipment construction confirmation system 1 has an equipment detecting unit 130 for detecting equipment included in an image data 111 obtained by capturing a site; a terminal detecting unit 135 for acquiring equipment attributes as information for use in identification of the detected equipment from the image data to detect terminals of the equipment included in the image data; an electric wire detecting unit 140 for acquiring terminal attributes as information for use in identification of the detected terminal from the image data to detect an electric wire connected to the terminal included in the image data; and a wire connection appropriateness confirming unit 145 for acquiring electric wire attributes as information for use in identification of the detected electric wire from the image data to confirm appropriateness of the wire connection between the terminal and the electric wire by comparison of the terminal attributes of the terminal with the electric wire attributes of the electric wire connected with the terminal.SELECTED DRAWING: Figure 1

Description

The present invention relates to an equipment construction confirmation system and an equipment construction confirmation method.

Japanese Patent Laid-Open No. 2002-200001 describes a construction site image determination device configured for the purpose of facilitating comprehension of the ability of a field worker who performs construction at a construction site. The judging device learns a learning device using a plurality of learning construction site images and learning data including good/bad judgments for each learning construction site image. Then, based on the output data of the learning device, the construction site image to be judged is judged to be good or bad.

Patent Literature 2 describes an uninterruptible replacement device for a high-voltage instrument transformer configured for the purpose of uninterruptible replacement of a high-voltage instrument transformer. In the above uninterruptible switching device, between the power supply side and the load, the PT is connected to the V connection on both sides of the high voltage switch of the circuit in which the high voltage switch and the instrument transformer are connected, and the power supply side of the high voltage switch PT is connected to , the secondary side of each PT is connected to the control unit that has the functions of voltage detection / phase detection / display and high voltage switch operation function, and the power meter is connected to the instrument transformer .

JP 2019-75078 A JP-A-2-60434

When constructing equipment for power distribution, confirmation (checks, pass/fail judgments, inspections, etc.) is performed on site to ensure that appropriate equipment has been installed and that wiring between equipment has been performed correctly. For example, during construction work related to high-voltage power receiving equipment, it is necessary to ensure that each terminal of multiple devices (high-voltage transformer, measuring instrument, test switch (check terminal), etc.) installed at the site has a wire connection to each terminal. The operator visually checks whether the connection is good or bad. However, since the number of terminals to be visually checked at such sites is more than tens, reducing the burden on workers and ensuring confirmation accuracy have become issues.

Japanese Patent Laid-Open No. 2002-200002 describes a judgment device that judges whether a construction site image to be judged is good or bad based on the output data of a learning device. However, this judging device is designed for the purpose of making it possible to easily grasp the abilities of field workers who carry out construction work at the construction site. It does not contribute to ensuring Further, the uninterruptible replacement device described in Patent Document 2 is merely configured for the purpose of uninterrupted replacement of a high-voltage instrument transformer.

The present invention has been made in view of such a background, and is an equipment construction confirmation system that can reduce the burden on workers who perform confirmation work in equipment construction and ensure the accuracy of equipment confirmation. And the purpose is to provide an equipment construction confirmation method.

One of the present inventions for achieving the above object is an information processing system (equipment construction confirmation system) for confirming the equipment installed at the site during equipment construction. a device detection unit that detects the device that is connected to the image data and specifies device attributes that are information for identifying the detected device; A terminal detection unit that specifies terminal attributes, which is information for identifying terminals, and information for detecting electric wires connected to the terminals included in the image data and identifying the detected electric wires. A wire detection unit that identifies wire attributes, and a connection quality check that checks whether the connection between the terminals and the wires is good or bad by comparing the terminal attributes of the terminals with the wire attributes connected to the terminals. and

In this way, the equipment construction confirmation system of the present invention automatically acquires the terminal attributes of the equipment terminals and the wire attributes of the wires connected to the terminals included in the image data obtained by photographing the site, By comparing, it is possible to check whether the connection between the terminal and the electric wire is good or not, so that the burden on the operator who performs the confirmation work in the equipment construction can be reduced, and the confirmation accuracy of the equipment can be ensured.

Another aspect of the present invention is the equipment construction confirmation system described above, wherein the equipment detection unit detects the equipment included in the image data and obtains the equipment attributes using machine learning that performs image recognition. A device detection model that is a model is used, and the terminal detection unit detects terminals included in the image data and acquires the terminal attributes using a terminal detection model that is a machine learning model that performs image recognition. The electric wire detection unit uses an electric wire detection model, which is a machine learning model for image recognition, to detect electric wires included in the image data and acquire the electric wire attributes.

In this way, the equipment construction confirmation system of the present invention detects terminals and electric wires by means of the equipment detection model, which is a machine learning model that performs image recognition, so equipment confirmation work can be performed efficiently and reliably.

Another aspect of the present invention is the equipment construction confirmation system described above, wherein the terminal attribute is information attached to the terminal or the vicinity of the terminal, and the wire attribute is information attached to the coating of the wire. This is the attached information.

In this way, the equipment construction confirmation system of the present invention converts information originally attached to equipment on site (information attached to terminals and information attached to wire coatings) as terminal attributes and wire attributes. Therefore, it is possible to easily and flexibly apply to various sites without the need for preparation such as attaching separate information to the equipment installed at the site.

Another aspect of the present invention is the equipment construction confirmation system described above, wherein a storage unit for storing unloaded equipment information, which is information about the equipment unloaded to the site, and the equipment detection unit specify Checking whether or not the device installed at the site is the same device as the device delivered to the site by comparing the attribute of the device and the information on the delivered device. and a device identification unit.

In this way, the equipment construction confirmation system of the present invention automatically confirms whether or not the equipment installed at the site is the same as the equipment unloaded to the site. It is possible to prevent the wrong equipment from being installed at the site, thereby enhancing the certainty and safety of the construction work.

Another aspect of the present invention is the equipment construction confirmation system described above, which detects a clock device that provides the current date and time, and an information display device included in image data photographed on site, and detects the information display device. The displayed date and time, which is the date and time displayed on the device, is detected, and by comparing the detected displayed date and time with the current date and time provided from the timing device, it is confirmed whether or not the displayed date and time are correct. A date and time display confirmation unit for outputting information based on the result is further provided.

In this way, the system for confirming equipment work according to the present invention automatically determines whether or not the date and time displayed on the information display installed at the site is correct. It is possible to reduce the burden on the operator to check whether the information is correct or not.

Another aspect of the present invention is the equipment construction confirmation system described above, wherein the date and time display confirmation unit detects the information display included in the image data photographed at the site and confirms the display date and time. Detection is performed using a date and time display detection model, which is a machine learning model that performs image recognition.

In this way, the equipment construction confirmation system of the present invention uses the equipment detection model, which is a machine learning model for image recognition, to detect the information display device and display date and time contained in the image data taken at the site. Therefore, it is possible to efficiently and reliably detect the information display device and the display date and time.

In addition, the problems disclosed by the present application and their solutions will be clarified by the description of the mode for carrying out the invention and the drawings.

ADVANTAGE OF THE INVENTION According to this invention, while aiming at the burden reduction of the worker who performs confirmation work in equipment installation, the precision of equipment confirmation can be ensured.

It is a figure which shows the main functions with which an apparatus installation confirmation system is provided. This is an example of equipment installed on-site in high-voltage power receiving equipment construction. This is an example of equipment installed on-site in high-voltage power receiving equipment construction. It is an example of out-of-warehouse apparatus information. It is an example of a confirmation result. It is an example of an information processing device that realizes an equipment installation confirmation system. 4 is a flowchart for explaining learning processing; 9 is a flowchart for explaining confirmation processing; 4 is a flowchart for explaining device identification processing; 7 is a flowchart for explaining connection confirmation processing; 9 is a flowchart for explaining date and time confirmation processing; It is an example of a confirmation result display screen.

DETAILED DESCRIPTION OF THE INVENTION Embodiments for carrying out the invention will be described below. In the following description, the same or similar configurations may be denoted by common reference numerals and descriptions thereof may be omitted.

FIG. 1 shows main functions provided in an information processing system (hereinafter referred to as "equipment installation confirmation system 1") described as one embodiment. The equipment construction confirmation system 1 performs on-site confirmation of the installation status of equipment (check, pass/fail judgment, inspection etc.). In this embodiment, the construction to which the equipment construction confirmation system 1 is applied is construction related to high-voltage power receiving equipment (installation work and replacement work for high-voltage meters (high-voltage transformers (meter transformers) and measuring instruments (watt-hour meters)). (hereinafter referred to as "high-voltage power receiving facility construction"), however, the type of construction to which the equipment construction confirmation system 1 is applied is not necessarily limited.

The equipment construction confirmation system 1, for example, determines the quality of connection of wires (core wires of cables) to terminals provided in equipment installed at the site based on an image of the site (whether the connection is performed correctly). (Is there any)? As a result, it is possible to reduce the burden on the operator involved in judging whether the connection is good or bad, improve the work efficiency, and improve the confirmation accuracy.

In addition, the equipment construction confirmation system 1, for example, based on the image of the site, the same equipment (type and serial number) as the equipment that was unloaded for the site (hereinafter referred to as "unloaded equipment") (matching equipment) is installed on site. As a result, it is possible to prevent equipment different from the unloading equipment from being installed at the site by mistake, thereby increasing the certainty and safety of the construction work.

In addition, the equipment construction confirmation system 1, for example, based on the photographed image of the site, the operation status of the equipment installed at the site (in this example, whether the date and time held by the watt-hour meter is correct) This makes it possible to reduce the burden on the operator who checks the operation status of the equipment and to improve the work efficiency.

FIG. 2A shows an example of equipment (high-voltage transformer 2, test switch 3, weighing scale 4, and information display 5) installed on-site in high-voltage power receiving facility construction.

Among the devices illustrated, the high-voltage transformer 2 is installed, for example, at a consumer side (for example, a product manufacturing factory) that has concluded a high-voltage power receiving contract with a power provider such as an electric power company. The high-voltage transformer 2 (VCT) is a device that supplies the power supplied from the power supply side to the load side, and is a voltage transformer (VT: Voltage Transformer) that converts the high voltage supplied from the power supply side to a low voltage. It functions as a current transformer (CT) that converts a large current supplied from the power supply side into a small current.

As shown in the figure, the high-voltage transformer 2 includes a group of primary terminals 21a (UK, VK, WK) to which three power lines 71a for supplying power from the power supply side are connected, and a group of terminals 21a (UK, VK, WK) for supplying power to the load side. and a primary terminal group 21b (UL, VL, WL) to which each of the three distribution lines 71b is connected. The coating colors of the three power lines 71a are different. Also, the colors of the coatings of the three distribution lines 71b are different. In addition, since the coating of each electric wire constituting the power line 71a and the distribution line 71b is colored differently, for example, by comparing the color of the coating of the electric wire with the color of the terminal, , it is possible to visually confirm whether or not the connection between the terminal and the electric wire is performed correctly. Further, when information for identifying each electric wire is attached to the coating of each electric wire, the quality of the connection can be confirmed by collating the information with the information attached to each terminal. As will be described later, the equipment construction confirmation system 1 performs image recognition and collation of such information to efficiently and reliably determine whether the connection is good or bad.

In addition, the high-voltage transformer 2 has seven electric wires of a cable 72 (seven-core cable) for providing the measuring instrument 3 with the low voltage after conversion by the voltage transformer and the small current after conversion by the current transformer. It has a secondary terminal group 22 (1S, P1, P3, 3S, 3L, P2, 1L) to be connected. Each of the seven wires has a different coating color.

Further, the high-voltage transformer 2 is provided with a display plate (not shown) on which information for determining whether or not the high-voltage transformer 2 that has been taken out of the warehouse and the high-voltage transformer 2 that has been installed at the site is the same. ing. In this example, the display board includes a display of the type and manufacturing number of the high-voltage transformer 2 in a predetermined format (characters, bar code, QR code (registered trademark), etc.).

The test switch 3 (also referred to as a "check terminal") is a device used for the purpose of safely and reliably performing testing and replacement of the measuring instrument 4 on site. It is provided so as to be interposed between the scale 4 and the scale 4 . In some cases, the test switch 3 is attached to the high-voltage transformer 2 . In order to avoid a non-weighing state and a power failure state during replacement work of the measuring device 4, a substitute measuring device may be temporarily attached to the test switch 3.

As shown in the figure, the exemplified test switch 3 has terminal groups 31a (1S, P1, P3, 3S, 3L, P2, 1L) to which seven electric wires of a cable 72 (seven-core cable) are respectively connected. and a terminal group 31b (1S, P1, P3, 3S, 3L, P2, 1L) to which seven electric wires of a cable 73 (seven-core cable) connected to the weighing scale 4 are connected. Further, the test switch 3 illustrated as an example is provided with a detachable contact plug 32 for controlling the opening and closing of each terminal between the terminal group 31a and the terminal group 31b. The spigot 32 is used, for example, when installing a replacement weighing scale.

The meter 4 is a three-phase three-wire watt-hour meter capable of measuring active power and reactive power. The measuring instrument 4 is obligated to be replaced every predetermined period by the Measurement Law. The measuring instrument 4 illustrated has a terminal group 41a (1S, P1, P3, 3S, 3L, P2, 1L) to which seven electric wires of a cable (seven-core cable) are respectively connected. Moreover, the weighing scale 4 illustrated has a terminal group 41b (C1, C2, DT, SG, D) to which the information display 5 and the cable 74 are connected.

The weighing scale 4 also has a function of outputting date and time information such as the current date and time used for measuring weighing values. In addition, the weighing instrument 4 illustrated has a communication function, and communicates with other devices such as the notification destination of the weighing value. The weighing scale 4 is provided with a display plate (not shown) for determining (identifying) whether the weighing scale 4 taken out of the warehouse and the weighing scale 4 installed at the site are the same. In this example, the display plate includes a display in a predetermined format (characters, bar code, QR code (registered trademark), etc.) for the type and manufacturing number of the weighing instrument 4 .

A display device for information such as power consumption (hereinafter referred to as "information display device 5") is communicably connected to the meter 4 via a cable 74, and the measured value measured by the meter 4 (active power amount, reactive power, average power factor, etc.), the operating state of the measuring instrument 4, the current date and time held by the measuring instrument 4, and other information 55 is displayed. The illustrated information display 5 has a user interface for setting the date and time information of the scale 4 by the operator.

FIG. 2B shows another example of equipment (high-voltage transformer 2, test switch 3, weighing scale 4, and information display 5) installed on-site in high-voltage power receiving equipment construction. The configurations of the high-voltage transformer 2 and the test switch 3 illustrated are the same as in FIG. 2A (the high-voltage transformer 2 is omitted in the figure). Regarding the weighing scale 4, the basic configuration is the same as that of FIG. 2A, but the connection method with the information display 5 is different. The information display device is connected via a cable 74 (4-core cable) in which the electric wire is connected to the part (P3, P2) and part (DT, SG) of the terminal group 41b (C1, C2, DT, SG, D). 5 and is communicably connected. The information display 5 includes a terminal group 51 (P3, P2, DT1, SG1, DT2, SG2) to which each electric wire of the cable 74 is connected.

Returning to FIG. 1, as shown in FIG. 1, the equipment construction confirmation system 1 includes a storage unit 110, a model learning unit 120, an image data acquisition unit 125, an equipment detection unit 130, a terminal detection unit 135, an electric wire detection unit 140, a connection It has the functions of a pass/fail checking section 145 , a delivery device identification section 150 , a date/time display checking section 155 , and a confirmation result generating section 160 .

Among the functions described above, the storage unit 110 stores image data 111 , learning models 112 , learning data 113 , delivered device information 114 , and confirmation results 115 .

The image data 111 is data acquired by the image data acquisition unit 125, and is data obtained by encoding an image shot on site in a predetermined format. The image data 111 may be one frame (one frame) clipped from a moving image. The image data 111 may be captured by a person using a smartphone, a digital still camera, or the like, or may be captured automatically.

As shown in FIG. 1, the learning model 112 includes machine learning models of a device detection model 1121, a terminal detection model 1122, a wire detection model 1123, a connection quality confirmation model 1124, and a date display detection model 1125.

Among these, the device detection model 1121 is a machine learning model for detecting that a predetermined device appears in the image data 111 . When a predetermined device appears in the image data 111, the device detection model 1121 further acquires information identifying the device (type, serial number, etc., hereinafter referred to as “device attributes”). The device detection model 1121 acquires the device attribute using, for example, the form of the device shown in the image data 111 as a feature amount. Also, the device detection model 1121 acquires device attributes by recognizing (character recognition, bar code recognition, QR code (registered trademark) recognition, etc.) display on a display plate attached to the device, for example.

The terminal detection model 1122 is a machine learning model for detecting that a terminal of a given device appears in the image data 111 . When the terminal detection model 1122 detects that the terminal of the device is shown in the image data 111, the terminal detection model 1122 further includes information for identifying the terminal of the device (characters, numbers, colors attached to the terminal or near the terminal, Other terminal characteristics (hereinafter referred to as "terminal attributes") are acquired. The terminal detection model 1122 acquires the terminal attribute using, for example, the form of the terminal appearing in the image data 111 as a feature amount.

The electric wire detection model 1123 is a machine learning model for detecting an electric wire connected to a terminal of a predetermined device when the image data 111 shows the terminal. When the electric wire detection model 1123 detects an electric wire connected to a terminal, the electric wire detection model 1123 further provides information for identifying the electric wire (the color of the electric wire covering, the characters written on the electric wire covering, the tag attached to the electric wire, Other various characteristics of the electric wire (hereinafter referred to as "attributes of the electric wire") are specified. The electric wire detection model 1123 acquires the electric wire attribute using, for example, the shape of the electric wire appearing in the image data 111 as a feature amount.

The connection quality confirmation model 1124 confirms the quality of connection between the terminal detected by the terminal detection model 1122 and the electric wire connected to the terminal detected by the electric wire detection model 1123 (whether or not the connection is made in a correct relationship). or) is confirmed by comparing the terminal attribute of the terminal and the wire attribute of the wire. For example, the connection quality confirmation model 1124 confirms the quality of the connection between the terminal and the wire based on whether the color or character attached to the terminal matches the color or character attached to the wire.

The date display detection model 1125 is a machine learning model for detecting that the image data 111 shows the date and time displayed by the information display device 5 . The date and time display detection model 1125 further identifies (recognizes) the displayed date and time when the date and time is displayed. The date and time display detection model 1125 detects the date and time displayed in the image data 111 and identifies the date and time, for example, using a character recognition technique.

Although the types of the machine learning models described above are not necessarily limited, they are realized by, for example, deep learning (DNN (Deep Neural Network), CNN (Convolutional Neural Network), etc.). Further, each machine learning model may be a machine learning model (anomaly detection model) obtained by learning a steady state by an autoencoder (VAE (Variational Autoencoder) or the like), for example. Further, detection of an object based on the image data 111 by each machine learning model includes, for example, "sliding window method", "HOG (Histograms of Oriented Gradients) feature amount", "region proposal method", "Faster R-CNN", "YOLO (You Only Look Once)", "SSD (Single Shot Detector)", "End-to-END learning", etc.). In addition, the feature amount used by each machine learning model may be set by a human system, and for example, a known feature amount extraction method (SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), FAST (Features from Accelerated Segment Test), BRIEF (Binary Robust Independent Elementary Features), ORB (Oriented FAST and Rotated BRIEF), etc.) may be used.

As shown in FIG. 1, the learning data 113 includes learning data 1131 used for learning the device detection model 1121, learning data 1132 used for learning the terminal detection model 1122, learning data 1133 used for learning the electric wire detection model 1123, and connection quality. Learning data 1134 used for learning the confirmation model 1124 and learning data 1135 used for learning the date/time display detection model 1125 are included.

The learning data 1131 used for learning the device detection model 1121 is, for example, data in which image data showing a device and the type of the device (correct answer information, label) are associated with each other. The learning data 1131 may include, for example, data in which image data in which the device is not captured and information indicating that the device is not captured are associated with each other. In order to improve the accuracy of the device detection model 1121, learning of the device detection model 1121 is performed by photographing the device in various modes (various modes with different shooting conditions such as lighting, shooting direction, and shooting position). This is done using a sufficient amount of training data 1131 based on image data.

The learning data 1132 used for learning the terminal detection model 1122 is, for example, data in which image data showing a terminal is associated with information (correct answer information, label) indicating the terminal attribute of the terminal. The learning data 1132 may include, for example, data in which image data in which the terminals are not shown are associated with data indicating that the terminals are not shown. In order to improve the accuracy of the terminal detection model 1122, the terminal detection model 1122 is learned using images of terminals photographed in various modes (various modes with different shooting conditions such as lighting, shooting direction, shooting position, etc.). This is done using a sufficient amount of training data 1132 based on the data.

The learning data 1133 used for learning the electric wire detection model 1123 is, for example, data in which image data showing an electric wire and information (correct answer information, label) indicating electric wire attributes of the electric wire are associated with each other. The learning data 1133 may include, for example, data in which image data in which electric wires are not shown and data indicating that electric wires are not shown are associated with each other. In order to improve the accuracy of the electric wire detection model 1123, the learning of the electric wire detection model 1123 is performed using images of electric wires photographed in various modes (various modes with different photographing conditions such as light adjustment, photographing direction, and photographing position). This is done using a sufficient amount of learning data 1133 based on the data.

The learning data 1134 used for learning the connection quality confirmation model 1124 is, for example, data in which a combination of terminal attributes and wire attributes is associated with information (correct answer information, label) indicating whether the combination is a correct combination. is. In order to improve the accuracy of the connection quality confirmation model 1124, learning of the connection quality confirmation model 1124 is performed using a sufficient amount of learning data 1134 for various combinations of terminal attributes and wire attributes.

The learning data 1135 used for learning the date/time display detection model 1125 is, for example, data obtained by associating image data showing the date/time display displayed by the information display device 5 with information indicating the date/time (correct information, labels). be. In order to improve the accuracy of the date/time display detection model 1125, the learning of the date/time display detection model 1125 requires a sufficient amount of learning data for various combinations of image data showing the date/time display and information indicating the date/time. 1134.

The delivered device information 114 shown in FIG. 1 includes information (type and serial number) specifying devices delivered to the site. The equipment installation confirmation system 1 acquires the out-of-warehouse equipment information 114, for example, via a user interface or by communication.

FIG. 3 shows an example of the out-of-warehouse device information 114. As shown in FIG. The delivered device information 114 is composed of one or more entries (records) having items such as delivery date and time 1141, site ID 1142, format 1143, manufacturing number 1144, and the like. One entry of the out-of-warehouse equipment information 114 corresponds to one out-of-warehouse equipment. Among the above items, the date and time when the device is taken out of the warehouse is set in the delivery date and time 1141 . The site ID 1142 is set with a site ID that is an identifier of the site where the device is installed. The format 1143 is set with the format of the device. The serial number 1144 is set with the serial number of the device.

Returning to FIG. 1 , the confirmation result 115 includes information indicating the results confirmed by the connection quality confirmation unit 145 , the delivered equipment identification unit 146 , and the date/time display confirmation unit 155 . The content of the confirmation result 115 is provided to a user such as an operator by the confirmation result generator 160 .

An example of the confirmation result 115 is shown in FIG. The confirmation result 115 illustrated as an example is composed of a plurality of entries (records) having items such as a device ID 1151, a format 1152, a manufacturing number 1153, a connection status 1154, a delivery identification 1155, a date and time display 1156, and the like. One of the entries in the confirmation result 115 corresponds to one piece of equipment installed on site. Among the above items, the device ID 1151 is set with the device ID of the device provided at the site. In the format 1152, information indicating the format of the device (model number, model number, etc.) is set. The serial number 1153 is set with the serial number of the device. The connection state 1154 is set with the determination result of the connection quality by the connection quality confirmation unit 145 for the terminal of the device. A result of identification by the delivered device identification unit 150 is set in the delivered device identification 1155 . In the date/time display 1156, the result of confirming the date/time display by the date/time display confirmation unit 155 is set.

The model learning unit 120 shown in FIG. Learning of the electric wire detection model 1123, learning of the connection quality confirmation model 1124 based on the learning data 1134, and learning of the date display detection model 1125 based on the learning data 1135 are performed. Note that the learning data 113 may be divided into data used for learning and data used for verification, and the learning model 112 trained with the data used for learning may be verified using the data used for verification.

The image data acquisition unit 125 acquires image data captured on site. The image data acquired by the image data acquisition unit 125 is stored in the storage unit 110 as the image data 111 .

The device detection unit 130 uses the device detection model 1121 to detect a device included in the image data 111, and when a device is detected, acquires the device attribute of the device.

The terminal detection unit 135 uses the terminal detection model 1122 to detect a terminal appearing in the image data 111 and, if a terminal is detected, acquires the terminal attribute of the terminal.

The wire detection unit 140 uses the wire detection model 1123 to detect the wire appearing in the image data 111 and, if the wire is detected, acquires the wire attribute of the wire.

The connection quality confirmation unit 145 determines the quality of the connection between the terminal and the electric wire using the connection quality confirmation model 1124 .

The out-of-warehouse equipment identification unit 150 collates the type (type, serial number) of the equipment specified by the equipment detection unit 130 with the out-of-warehouse equipment information 114 to determine whether the equipment installed at the site has been unloaded for the site. Confirm (identify) whether or not it is the same as the equipment that was purchased.

The date/time display confirmation unit 155 uses the date/time display detection model 1125 to detect the date/time display appearing in the image data 111, and if the date/time display is detected, specifies the date/time (identification, character recognition). The date/time display confirmation unit 155 also compares the specified date/time with the date/time held by the equipment construction confirmation system 1 to determine whether or not the specified date/time is correct.

As a result of confirming the quality of the connection between the terminal and the electric wire confirmed by the connection quality confirmation unit 145, the confirmation result generation unit 160 determines that the equipment installed at the site by the unloaded equipment identification unit 150 is the equipment unloaded for the site. and the confirmation result of whether the date and time display by the date and time display confirmation unit 155 is correct are stored as the confirmation result 115 . The confirmation result generation unit 160 outputs the confirmation result 115 at any time (screen display, voice output, etc.).

FIG. 5 shows an example of hardware of an information processing device (computer) used for realizing the equipment installation confirmation system 1 shown in FIG. The illustrated information processing apparatus 10 includes a processor 11 , a main memory device 12 , an auxiliary memory device 13 , an input device 14 , an output device 15 , a communication device 16 , a timer device 17 and a photographing device 18 . The equipment installation confirmation system 1 may be composed of a plurality of information processing devices 10 that are communicatively connected. In addition, the information processing apparatus 10 is a virtual information processing resource provided in whole or in part using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. may be implemented using Also, all or part of the functions provided by the information processing apparatus 10 may be realized by services provided by the cloud system via an API (Application Programming Interface) or the like, for example. Further, all or part of the functions provided by the information processing device 10 are realized using, for example, SaaS (Software as a Service), PaaS (Platform as a Service), IaaS (Infrastructure as a Service), etc. can be anything. A part of the information processing device 10 may be, for example, a portable information terminal such as a smart phone, a tablet, or a notebook personal computer that is brought into the construction site.

In the figure, the processor 11 includes, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), AI (Artificial Intelligence) chip or the like.

The main storage device 12 is a device that stores programs and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory (NVRAM (Non Volatile RAM)), or the like.

The auxiliary storage device 13 is, for example, an SSD (Solid State Drive), a hard disk drive, an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage system, an IC card, an SD card, or an optical recording device. They are a read/write device for a recording medium such as a medium, a storage area of a cloud server, and the like. Programs and data can be read into the auxiliary storage device 13 via a recording medium reading device or the communication device 16 . Programs and data stored (stored) in the auxiliary storage device 13 are read into the main storage device 12 at any time.

The input device 14 is an interface that receives input from the outside, and includes, for example, a keyboard, mouse, touch panel, card reader, pen-input tablet, voice input device, and the like.

The output device 15 is an interface for outputting various information such as processing progress and processing results. The output device 15 is, for example, a display device (liquid crystal monitor, LCD (Liquid Crystal Display), graphic card, etc.) that visualizes the above various information, a device (audio output device (speaker, etc.)) that converts the above various information into sound. , a device (printing device, etc.) that converts the above various information into characters. For example, the information processing device 10 may be configured to input and output information with another device via the communication device 16 .

The input device 14 and the output device 15 constitute a user interface for receiving and presenting information from the user.

The communication device 16 is a device that realizes communication with other devices. The communication device 16 is a wired or wireless communication interface that realizes communication with other devices via a communication network such as the Internet. modules and the like.

The clock device 17 is a device that generates highly accurate current date and time, and is configured using an RTC (Real Time Clock) or the like. The clock device 17 may calibrate the date and time with the date and time acquired by the communication device 16 by NTP (Network Time Protocol).

The image capturing device 18 captures still images or moving images, and generates image data 111 based on the captured video. The photographing device 18 is, for example, a camera attached to a smart phone used by an on-site worker or a digital still camera. In addition to performing image recognition on the image data 111 acquired by the imaging device 18, for example, on-site information obtained by object detection sensors such as LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) and millimeter wave radar A machine learning model learned using learning data with parameters may be used to improve detection accuracy of objects (devices, terminals, electric wires).

For example, an operating system, a file system, a DBMS (DataBase Management System) (relational database, NoSQL, etc.), a KVS (Key-Value Store), etc. may be installed in the information processing apparatus 10 .

The above-described functions provided by the equipment construction confirmation system 1 are obtained by the processor 11 of the information processing device 10 reading and executing a program stored in the main storage device 12, or by hardware of the information processing device 10 ( FPGA, ASIC, AI chip, etc.). The information processing apparatus 10 stores the various types of information (data) described above, for example, as files managed by a database table or a file system.

Next, main processing performed by the equipment installation confirmation system 1 will be described.

FIG. 6 shows processing (hereinafter referred to as , referred to as “learning processing S600”). The model learning unit 120 individually uses learning data 1131 to 1135 corresponding to each of the device detection model 1121, the terminal detection model 1122, the electric wire detection model 1123, the connection quality confirmation model 1124, and the date display detection model 1125. (S611). Note that learning of the learning model 112 usually requires a large amount of information processing resources. For example, the learning of the learning model 112 is performed using an API (Application Programming Interface) for machine learning provided by the cloud system. may

FIG. 7 is a flowchart for explaining the processing (hereinafter referred to as "confirmation processing S700") performed by the equipment installation confirmation system 1 when confirming the installation status of equipment at the site. An on-site worker or the like (hereinafter referred to as "user"), for example, operates a smartphone, which is a component of the equipment construction confirmation system 1, and uses the equipment construction confirmation system 1 to check the installation status of the equipment. conduct. The confirmation process S700 will be described below with reference to FIG.

First, the equipment construction confirmation system 1 allows a user to specify a work type (either of "identification of the unpacked equipment and the equipment installed at the site", "confirmation of connection quality", or "confirmation of display date and time"). Accept (S701). When the user designates "equipment identification" (S702: equipment identification), the equipment construction confirmation system 1 executes the equipment identification processing S710, and then ends the confirmation processing S700. When the user designates "confirmation of connection quality" (S702: connection quality), the equipment construction confirmation system 1 executes the connection confirmation process S720, and then ends the confirmation process S700. When the user designates "confirm display date and time" (S702: display date and time), the equipment construction confirmation system 1 executes the date and time confirmation process S730, and then ends the confirmation process S700.

FIG. 8 is a flowchart for explaining the device identification processing S710 of FIG. The device identification processing S710 will be described below with reference to FIG. Note that the device identification processing S710 is performed, for example, by application software of a smartphone owned by the user.

First, the image data acquisition unit 125 of the equipment construction confirmation system 1 acquires the image data of the site and stores it as the image data 111 (S811). The image data is, for example, image data captured by a user operating a smartphone on site. The image data may be captured as a still image, or may be one frame (one frame) cut out from moving image data.

Subsequently, the device detection unit 130 of the device construction confirmation system 1 detects devices included in the image data 111 (S812). If the device cannot be detected (S813: NO), the process returns to S811. On the other hand, when a device is detected (S813: YES), the device detection unit 130 acquires device attributes of the detected device (S814). Note that the processing of S811 to S814 is performed in real time while the user takes an image of equipment installed at the site with an imaging device of a smartphone, for example.

Subsequently, the delivered device identification unit 150 of the device construction confirmation system 1 compares the device attribute acquired by the device detection unit 130 with the delivered device information 114 (S815). As a result of the comparison, if the device attribute is the same as that of the device delivered to the site (S816: YES), the delivered device identification unit 150 outputs information indicating that the device has been identified (screen display, voice output, etc.) (S817). On the other hand, if the equipment attributes are different from those of the equipment delivered to the site (S816: NO), the delivered equipment identification unit 150 outputs information indicating a warning to the effect that the equipment is different from the equipment delivered to the site. Output (screen display, audio output, etc.) is performed (S818).

In S<b>819 , the delivered device identification unit 150 sets the result of checking the delivered device to the delivered device identification 1155 of the confirmation result 115 . For example, the out-of-warehouse identification unit 150 identifies information indicating "OK" if the out-of-warehouse equipment is installed at the site, and a warning if a different equipment from the out-of-warehouse equipment is installed at the worksite. Set to 1155.

FIG. 9 is a flowchart for explaining the connection confirmation process S720 of FIG. The connection confirmation process S720 will be described below with reference to FIG.

First, the image data acquisition unit 125 of the equipment construction confirmation system 1 acquires the image data of the site and stores it as the image data 111 (S911). The image data is, for example, image data captured by a user operating a smartphone on site. The image data may be captured as a still image, or may be one frame (one frame) cut out from moving image data.

Subsequently, the device detection unit 130 of the device installation confirmation system 1 detects devices included in the image data 111 (S912). If the device cannot be detected (S913: NO), the process returns to S911. On the other hand, when a device is detected (S913: YES), the device detection unit 130 acquires device attributes of the detected device (S914).

Subsequently, the terminal detection unit 135 of the equipment construction confirmation system 1 detects terminals included in the image data (S915). If the terminal cannot be detected (S916: NO), the process returns to S911. On the other hand, when a terminal is detected (S916: YES), the terminal detection unit 135 acquires the terminal attribute of the detected terminal (S917).

Subsequently, the electric wire detection unit 140 of the equipment construction confirmation system 1 detects electric wires included in the image data (S918). If the electric wire cannot be detected (S919: NO), the process returns to S911. On the other hand, when an electric wire is detected (S919: YES), the electric wire detection unit 140 acquires electric wire attributes of the detected electric wire (S920).

Note that the processing of S911 to S920 is performed in real time while the user takes an image of equipment installed at the site with an imaging device of a smartphone, for example.

Subsequently, the connection quality confirmation unit 145 of the equipment construction confirmation system 1 compares the terminal attribute and the wire attribute to confirm the quality of the connection of the wire to each terminal (S921). If the connection is not properly performed (S922: NO), the connection quality checking unit 145 outputs information (screen display, audio output, etc.) identifying the terminal and wire for which the connection is not properly performed, and warning information. (S923), the process proceeds to S925. On the other hand, if the connection is properly performed (S922: YES), the connection quality confirmation unit 145 outputs (screen display, voice output, etc.) that the connection is good (S924).

In S<b>925 , the connection quality confirmation unit 145 sets the result of confirming the connection to the connection status 1154 of the confirmation result 115 . For example, the connection quality checking unit 145 sets information indicating “OK” if the connection is correct, and information indicating an incorrect terminal if the connection is not correct, in the connection status 1154 .

Subsequently, the connection quality confirmation unit 145 generates the confirmation result 115 (S925).

FIG. 10 is a flowchart for explaining the date and time confirmation process S730 of FIG. The date and time confirmation processing S730 will be described below with reference to FIG.

First, the image data acquisition unit 125 of the equipment construction confirmation system 1 acquires the image data of the site and stores it as the image data 111 (S1011). The image data is, for example, image data captured by a user operating a smartphone on site. The image data may be captured as a still image, or may be one frame (one frame) cut out from moving image data.

Subsequently, the device detection unit 130 of the device construction confirmation system 1 detects the information display device 5 included in the image data 111 (S1012). If the information display 5 cannot be detected (S1013: NO), the process returns to S1011. On the other hand, when the information display device 5 is detected (S1013: YES), the date/time display confirmation unit 155 of the equipment construction confirmation system 1 detects the date/time display included in the image data (S1014). If the date and time display cannot be detected (S1015: NO), the process returns to S1011. On the other hand, if the date and time display is detected (S1015: YES), the process proceeds to S1016.

Note that the processing of S1011 to S1015 is performed in real time while the user takes an image of equipment installed at the site with an imaging device of a smartphone, for example.

In S1016, the date/time display confirmation unit 155 compares the detected date/time with the date/time acquired from the clock device 17 (hereinafter referred to as "system date/time"). If the detected date and time are not correct (S1017: NO), the date and time display confirmation unit 155 outputs information (date and time calibration instruction) prompting the user to calibrate the date and time (screen display, voice output, etc.) (S1018), and proceeds to S1020. move on. On the other hand, if the detected date and time are correct (S1017: YES), the date and time display confirmation unit 155 outputs information indicating that the date and time are correct (screen display, voice output, etc.).

In S<b>1020 , the date/time display confirmation unit 155 sets the date/time confirmation result to the date/time display 1156 of the confirmation result 115 . For example, the date/time display confirmation unit 155 sets the date/time display 1156 with information indicating "OK" if the connection is correct, and indicating that the connection is incorrect if the connection is not correct.

By the way, the screen showing the result of each of the above processes (device identification process S710, connection confirmation process S720, date and time confirmation process S730) can be displayed, for example, on the output device 15 (display device) of the information processing device 10 such as a smartphone operated by the user. ) may be displayed.

FIG. 11 shows an example of the above screen (hereinafter referred to as "confirmation result display screen 1100"). On the confirmation result display screen 1100 shown as an example, the contents of the confirmation result 115, that is, information about the equipment to be confirmed (equipment ID 1151, type 1152, serial number 1153) and the results of the above processes (connection status 1154 , delivery identification 1155, date and time display 1156) are displayed. The user can efficiently perform confirmation work by referring to the confirmation result display screen 1100 .

As described above, the equipment construction confirmation system 1 automatically acquires the terminal attributes of the equipment terminals and the wire attributes of the wires connected to the terminals included in the image data 111 obtained by photographing the site, By comparing these, it is possible to automatically check whether the connection between the terminal and the electric wire is good or bad, thereby reducing the burden on the operator in the installation work of the device and ensuring the accuracy of checking the status of the device. In addition, the equipment construction confirmation system 1 detects terminals and electric wires by means of equipment detection models, which are machine learning models that perform image recognition. can do.

In addition, since the equipment construction confirmation system 1 uses the information attached to the terminal and the information attached to the coating of the electric wire, which are the information originally attached to the on-site equipment, as the terminal attribute and the electric wire attribute, It is possible to flexibly and easily apply to various sites without the need to add separate information to the equipment installed at the site.

In addition, since the equipment construction confirmation system 1 automatically confirms whether or not the equipment installed at the site is the same as the equipment delivered to the site, the equipment different from the delivered equipment can be checked. can be prevented from being installed on site by mistake, and the certainty and safety of construction work can be enhanced.

In addition, since the equipment construction confirmation system 1 automatically determines whether or not the date and time displayed on the information display installed at the site is correct, it is It is possible to reduce the burden on the operator for confirming whether or not.

Although the embodiments of the present invention have been described in detail above, the above description is for facilitating understanding of the present invention, and does not limit the present invention. It goes without saying that the present invention can be modified and improved without departing from its spirit, and that equivalents thereof are included in the present invention. For example, the above embodiments have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations. Moreover, it is possible to add, delete, or replace a part of the configuration of the above embodiment with another configuration.

1 equipment construction confirmation system, 10 information processing device, 110 storage unit, 111 image data, 112 learning model, 1121 equipment detection model, 1122 terminal detection model, 1123 electric wire detection model, 1124 connection quality confirmation model, 1125 date and time display detection model, 113 learning data 1131 learning data used for learning the device detection model 1132 learning data used for learning the terminal detection model 1133 learning data used for learning the electric wire detection model 1134 learning data used for learning the connection quality confirmation model 1135 Learning data used for learning the date and time display detection model 114 Delivery device information 115 Confirmation result 120 Model learning unit 125 Image data acquisition unit 130 Device detection unit 135 Terminal detection unit 140 Electric wire detection unit 145 Connection quality Confirmation unit 150 Delivery equipment identification unit 155 Date and time display confirmation unit 160 Confirmation result generation unit S600 Learning process S700 Confirmation process S710 Equipment identification process S720 Connection confirmation process S730 Date confirmation process

Claims (8)

An information processing system that supports confirmation work performed on-site during equipment construction,
a device detection unit that detects devices included in image data obtained by photographing a scene and acquires device attributes, which are information identifying the detected devices, from the image data;
a terminal detection unit that detects terminals of the device included in the image data and acquires terminal attributes, which are information identifying the detected terminals, from the image data;
a wire detection unit that detects wires connected to the terminals included in the image data and acquires wire attributes, which are information identifying the detected wires, from the image data;
a connection quality confirmation unit that confirms quality of connection between the terminal and the wire by comparing the terminal attribute of the terminal with the wire attribute of the wire connected to the terminal;
Equipment construction confirmation system. The equipment construction confirmation system according to claim 1,
The device detection unit detects devices included in the image data and acquires the device attributes using a device detection model, which is a machine learning model for performing image recognition;
The terminal detection unit detects terminals included in the image data and acquires the terminal attributes using a terminal detection model, which is a machine learning model for performing image recognition,
The electric wire detection unit detects electric wires included in the image data and acquires the electric wire attributes using an electric wire detection model, which is a machine learning model that performs image recognition.
Equipment construction confirmation system. The equipment construction confirmation system according to claim 1 or 2,
The terminal attribute is information attached to the terminal or the vicinity of the terminal,
The wire attribute is information attached to the coating of the wire,
Equipment construction confirmation system. The equipment construction confirmation system according to claim 1 or 2,
a storage unit that stores unloaded equipment information, which is information about the equipment unloaded to the site;
By comparing the device attributes specified by the device detection unit with the delivered device information, it is possible to determine whether or not the device installed at the site is the same device as the device delivered to the site. a delivery device identification unit that confirms whether
A device construction confirmation system further comprising: The equipment construction confirmation system according to claim 1 or 2,
a clock that provides the current date and time;
An information display device included in image data taken on site is detected, a display date and time, which is the date and time displayed on the information display device, is detected, and the detected display date and time is provided from the timing device. a date and time display confirmation unit for confirming whether or not the displayed date and time is correct by comparing with the current date and time, and outputting information based on the confirmation result;
A device construction confirmation system further comprising: The equipment construction confirmation system according to claim 5,
The date and time display confirmation unit detects the information display device and the display date and time included in the image data taken at the site using a date and time display detection model that is a machine learning model.
Equipment construction confirmation system. A method for supporting confirmation work performed on-site during equipment construction, comprising:
The information processing device
a step of detecting a device included in image data obtained by photographing a scene, and obtaining device attributes, which are information identifying the detected device, from the image data;
a step of detecting a terminal of the device included in the image data and obtaining a terminal attribute, which is information identifying the detected terminal, from the image data;
a step of detecting a wire connected to the terminal included in the image data and obtaining a wire attribute, which is information for identifying the detected wire, from the image data;
confirming whether the connection between the terminal and the electric wire is good or bad by comparing the terminal attribute of the terminal with the electric wire attribute of the electric wire connected to the terminal;
A method for confirming equipment construction, comprising: The equipment construction confirmation method according to claim 7,
The information processing device is
detecting devices included in the image data and acquiring the device attributes using a device detection model, which is a machine learning model for performing image recognition;
detecting terminals included in the image data and acquiring the terminal attributes using a terminal detection model, which is a machine learning model for performing image recognition;
A wire detection model, which is a machine learning model that performs image recognition, is used to detect wires included in the image data and acquire the wire attributes.
Equipment construction confirmation method.

Images