JP7117272B2 - Maintenance support system and method - Google Patents

Description

The present invention relates generally to assisting in maintenance (eg, maintenance) of equipment to be serviced, such as equipment in an elevator.

In general, in the maintenance business of elevators and escalators, quality is maintained by having maintenance personnel regularly inspect each device of the elevators and escalators so that passengers can use the elevators safely and replace parts as necessary. ing.

Here, there are a wide variety of parts that make up each device of the elevator, such as brake linings, guides, rollers, and electromagnetic contactors. Furthermore, even with the same brake lining, these parts differ in shape and performance depending on the delivery year, model, load capacity, etc. of the elevator. While there are such a wide variety of parts, the maintenance personnel must identify the replacement standards and replacement methods for each part and the model when arranging the parts, which is complicated and not easy.

Conventionally, the technology described in Patent Document 1 is known as a system for providing information on the maintenance of target equipment to maintenance personnel in response to such problems.

In Patent Document 1, by attaching wirelessly communicable tags containing individual identification information of maintenance target devices in advance to a plurality of maintenance target devices provided in a plant, maintenance personnel can register with the tags. A system is described in which maintenance information such as part models and manuals can be grasped even if there are multiple devices by carrying a terminal that can read and write maintenance information.

Japanese Patent Application Laid-Open No. 2016-1400

There are hundreds of thousands of elevators and escalators subject to maintenance, for example, and these elevators are installed indoors or outdoors, and are used for a wide variety of purposes, such as apartments, stations, and distribution facilities. The variety is huge. Therefore, it is difficult to attach tags to all maintenance target parts.

SUMMARY OF THE INVENTION The present invention has been made to solve such problems, and its object is to support maintenance without installing a tag for each part.

A maintenance support system receives imaged data indicating an imaged image of one or more parts among a plurality of parts constituting a device to be maintained, which is imaged by a camera, and detects one of the parts in the imaged image indicated by the imaged data. Among the above components, component information including information indicating the degree of deterioration of the target component, which is a component focused on based on the captured image, is displayed.

Maintenance can be supported without installing a tag for each part.

1 is a block diagram of an overall system according to one embodiment of the present invention; FIG. FIG. 2 is an explanatory diagram of functions of a component determination system; It is a figure which shows the flow of a component determination process. It is a figure which shows the 1st necessity determination. It is a figure which shows the 2nd necessity determination. It is a figure which shows the flow of the necessity determination Y. FIG. It is a figure which shows an example of a component determination result screen. It is a figure which shows another example of a component determination result screen.

In the following description, an "interface device" may be one or more interface devices. The one or more interface devices may be at least one of the following:
- One or more I/O (Input/Output) interface devices. An I/O interface device is an interface device to at least one of an I/O device and a remote display computer. The I/O interface device to the display computer may be a communications interface device. The at least one I/O device may be any of a user interface device, eg, an input device such as a keyboard and pointing device, and an output device such as a display device.
- One or more communication interface devices. The one or more communication interface devices may be one or more of the same type of communication interface device (e.g., one or more NICs (Network Interface Cards)) or two or more different types of communication interface devices (e.g., NIC and It may be an HBA (Host Bus Adapter).

Also, in the following description, "memory" may be one or more memory devices, typically a main memory device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

Also, in the following description, a "persistent storage device" is one or more persistent storage devices. A permanent storage device is typically a non-volatile storage device (for example, an auxiliary storage device), and specifically, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

Also, in the following description, "storage" may be at least memory of memory and persistent storage.

Also, in the following description, a "processor" is one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may be another type of processor device such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. At least one processor device may be a processor core. At least one processor device may be a broadly defined processor device such as a hardware circuit (for example, FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)) that performs part or all of processing.

Also, in the following description, "DB" is an abbreviation for database. In the following explanation, the expression "xxxDB" may be used to describe information that can be obtained as an output for an input, but the information may be data of any structure (for example, structured data (or unstructured data), or a learning model such as a neural network that generates an output for an input. Therefore, "xxxDB" can be called "xxx information". One DB may be divided into two or more DBs, or all or part of two or more DBs may be one DB.

In addition, in the following description, the function may be described using the expression “kkk unit”, but the function may be realized by executing one or more computer programs by a processor, or may be realized by executing one or more computer programs. It may be realized by the above hardware devices (for example, FPGA or ASIC), or may be realized by a combination thereof. The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

In the following embodiments, the equipment is the equipment that the elevator has.

For example, information about parts of an elevator is provided to an information processing terminal (hereinafter referred to as a "maintenance staff terminal") used by a maintenance worker who inspects the elevator on site. The maintenance terminal is an information processing terminal equipped with a camera and a monitor (an example of a display device), such as a tablet PC (tablet personal computer) or a smart phone. The maintenance support system according to the present embodiment includes a function realized by executing one or more programs on a maintenance terminal (hereinafter referred to as a maintenance support unit) and a component determination system that communicates with the maintenance terminal. at least one.

Input image data of the parts captured by the camera of the maintenance terminal, and the part determination system compares the image data with the three-dimensional model of each part (information indicating the appearance of the part) stored in the part appearance DB. Then, the component shown in the imaged image indicated by the imaged data is determined. This determined part may be hereinafter referred to as a "part of interest". The part determination system outputs the part information of the target part to the maintenance terminal. The output parts information is displayed on the monitor by the maintenance terminal.

The part information is specified from the part appearance DB. The part appearance DB contains, for each part, for example, drawing information, appearance information (for example, photographs and 3D models of parts in a normal state and a state that has reached a replacement standard), part types, dimensions, specifications, etc. attribute information.

Furthermore, when the component shape specified from the component appearance DB using the imaging data reaches the replacement standard, the component determination system determines, for example, the degraded part or the size of the part and the need to replace the part. and parts information is output to the terminal for maintenance personnel.

Furthermore, the parts determination system manages a maintenance DB. The maintenance DB includes, for example, maintenance data for each part such as aging information and accumulated operating time information for each part, and next replacement date information for each part calculated based on the replacement period for each part preset by the designer. Further, the maintenance DB stores data for measuring the degree of deterioration of parts at any time by a remote monitoring device that manages each elevator. The parts determination system determines whether or not the latest value of the measurement data regarding the target part has reached a replacement determination threshold previously determined for the target part.

In addition, the part determination system refers to the maintenance DB for the target part, and determines that replacement is necessary when the aging and accumulated operation information have reached the next replacement date. Alternatively, the parts determination system determines that replacement is necessary when measurement data of the part measured by the remote monitoring device reaches a replacement determination threshold value. The parts determination system outputs the parts information indicating the result of such replacement determination to the maintenance terminal.

Furthermore, in this embodiment, when the part of interest needs to be replaced, at least one of automatically creating an estimate for the part of interest and placing an order for the part of interest is performed.

According to the present embodiment, if a maintenance worker takes an image of an arbitrary part, he/she can know the part information of the imaged part (for example, the model, specifications, and necessity of replacement of the part). Such maintenance support is possible without installing a tag for each part.

Details of an embodiment of the present invention will be described below with reference to the drawings. In the following description, the "component ID" may include at least one of a component identifier (for example, an identification number) and a name.

FIG. 1 is a block diagram of the overall system according to one embodiment of the present invention.

In this embodiment, there are roughly three systems.

The first system is the maintenance terminal 1 . The maintenance staff terminal 1 is carried by, for example, a maintenance staff 4 who maintains the equipment 3 of the elevator in the customer building 2 . The maintenance terminal 1 is an information processing terminal equipped with a camera and a monitor. The monitor is a touch panel that serves as both an input device and a display device, but the maintenance terminal 1 may have an input device such as a keyboard or pointing device in addition to the monitor as the display device. The maintenance terminal 1 includes an interface device for wireless communication, a storage device, and a processor connected to them. Parts of the device 3 are imaged by the camera. The maintenance staff terminal 1 transmits imaging data representing a captured image and information associated with the imaging data representing the number of elevators to the parts determination system 5 . Furthermore, the maintenance staff terminal 1 creates estimates for parts and places orders for parts.

The second system is the parts determination system 5 . The parts determination system 5 may be a server configured with one or more computers, or may be a system implemented on a plurality of computational resource pools such as a cloud platform. The parts determination system 5 manages a parts appearance DB 7 storing information on the ID and appearance (for example, shape) of each part, and a maintenance DB 8 storing information on necessity of replacement of parts for each elevator. The part determination system 5 also communicates with the maintenance terminal 1 by radio, inputs imaging data and elevator number information, and outputs part model information and replacement necessity information to the maintenance terminal 1 . The component determination system 5 includes a reception unit 61 that receives imaging data, a component determination unit 62 that identifies a target component and its degree of deterioration by referring to the component appearance DB 7 based on the image data, and a component of interest and its degree of deterioration. and a display unit 63 for displaying parts information including information indicating on the maintenance staff terminal 1 (transmitting parts information for display to the maintenance staff terminal 1).

The third system is procurement system 9 . The procurement system 9 may be a server configured with one or more computers, or may be a system implemented on a plurality of computing resource pools such as a cloud platform. The procurement system 9 manages an inventory DB71. The inventory DB 71 stores information indicating the amount of money required for creating an estimate and placing an order, the number of items in stock, and the delivery date. Specifically, for example, the inventory DB 71 has a record storing information indicating the part name, model number, number (identification number), gold, quantity in stock, and delivery date for each part. When the maintenance staff terminal 1 prepares an estimate for parts or arranges an order for parts, the procurement system 9 transmits necessary information to the maintenance staff terminal 1 .

In this embodiment, for example, the following processes are performed. In addition, in this embodiment, one or more computer programs executed on the maintenance terminal 1 include a predetermined AR (Augmented Reality) application. As a result, a maintenance support unit, which is a function including an AR camera function, is realized. The maintenance support unit can superimpose an object such as text or a texture image on the image obtained by the camera and display it on the monitor.

In S<b>101 , the maintenance staff terminal 1 takes an image of an arbitrary part of the equipment 3 with a camera in response to the operation by the maintenance staff 4 . Also, the maintenance staff terminal 1 inputs the number of the elevator having the device 3 . The machine number may be manually input by maintenance personnel 4 or may be received from control panel 72 that controls device 3 of the elevator. The maintenance terminal 1 transmits imaged data associated with the machine number to the parts determination system 5 .

In S102, the reception unit 61 receives the imaging data.

In S103, the component determination unit 62 performs the following processing. The component determination unit 62 stores the received imaging data in a predetermined storage area (for example, the component appearance DB 7). Based on the component appearance DB 7, the component determination unit 62 identifies the component information (for example, information indicating the component ID and model type) of the target component, which is the component shown in the captured image indicated by the received captured data. . The part determination unit 62 determines whether or not the target part needs to be replaced. The maintenance DB 8 is referred to as necessary in determining whether replacement is necessary.

In S104, the display unit 63 displays the parts information including information indicating the determination result as to whether or not the part of interest needs to be replaced on the maintenance staff terminal 1 (transmits the parts information to the maintenance staff terminal 1). Here, the display is, for example, an image obtained by a camera (for example, an imaged image indicated by the transmitted imaging data or a current image obtained by the camera) by the maintenance support unit of the maintenance terminal 1. A component information object (for example, text or an icon indicating the presence of component information) is displayed in association with the target component currently displayed. Which range of the captured image is the target component range may be specified as follows, for example. That is, the maintenance terminal 1 identifies the scale when the maintenance terminal 1 captures an image, and identifies the relevant range, which is the range of the target component in the captured image, based on the scale and the analysis result of the captured image. Range information, which is information indicating the applicable range, is associated with the part information. The maintenance support unit specifies the applicable range based on the range information, and associates the object of the parts information with the specified applicable range. The computer program executed by the maintenance terminal 1 may be a general Web browser. In that case, the function corresponding to the maintenance support unit is realized in the parts determination system 5, and as a result, the parts determination system 5 and association of the object of the part information with the applicable range.

The part information of the part of interest is displayed, and the part of interest can be quoted or ordered. For example, in S<b>105 , the maintenance support unit of the maintenance staff terminal 1 receives a quote request or an order request from the maintenance staff 4 . Such acceptance may be possible regardless of the degree of deterioration of the target component, or may be possible only when the target component requires replacement. For example, information indicating the address (for example, URL) of the procurement system 9, which is a system related to quotation or ordering, is included in the parts information in S104, or is stored in the maintenance terminal 1 in advance, When making an estimate or placing an order, communication with the procurement system 9 is possible using the address.

When the estimate request is accepted, in S106, the maintenance support unit of the maintenance staff terminal 1 transmits an inquiry associated with the part ID in the part information to the procurement system 9, and receives the inquiry from the procurement system 9 in response to the inquiry. Estimate information is generated based on the obtained procurement support information. The procurement support information is, for example, information indicating at least the amount of the amount, the number of stocks, and the delivery date of the part of interest, and is information specified from the inventory DB 71 by the procurement system 9 using the part ID as a key. The estimate information includes a part ID and information obtained from the procurement support information (for example, information indicating at least the amount of the amount, the number of items in stock, and the delivery date). The maintenance support unit displays the estimate information, for example, an estimate screen showing contents based on the procurement support information (for example, a screen displaying the part ID, the price, and the delivery date) on the monitor. The maintenance support unit may receive approval (in other words, order request) for the estimate information (in other words, procurement support information). Also, the maintenance support unit may pass the estimate information and the customer's e-mail address to the mailer (e-mail software) of the maintenance terminal 1 . In this case, an e-mail containing the estimate information may be sent to the customer's e-mail address by a mailer.

When the order request is accepted, the maintenance support unit of the maintenance staff terminal 1 transmits the order request associated with the part ID of the target part to the procurement system 9 in S107. Before sending the order request, an order screen showing the order details based on the procurement support information (for example, a screen displaying the part ID, amount and delivery date) is displayed on the monitor by the maintenance support department. A purchase order request may be sent if approval is obtained. In response to the order request, the procurement system 9 reduces the inventory quantity by 1 for the part of interest corresponding to the part ID associated with the order request, and processes the order.

Inquiries for quotations and order requests may be made through the parts determination system 5 . That is, processing may be performed between the maintenance terminal 1 and the procurement system 9 via the parts determination system 5 .

FIG. 2 is a diagram for explaining the functions of the component determination system 5. As shown in FIG.

The component appearance DB 7 includes information on the ID and shape of each component. Specifically, for example, the component appearance DB 7 includes field imaging data and drawing data for each component. Take one component as an example (“target component” in the description of FIG. 2).

The field imaging data includes, for example, a plurality of 3D models respectively corresponding to a plurality of degrees of deterioration of the target part. "Multiple deterioration degrees" differ in stages, such as no deterioration (new), one or more deterioration degrees corresponding to no need to replace, and one or more deterioration degrees corresponding to need to replace. Including degree of deterioration. The 3D model may be imaging data or 3D scan data of the target part. The data format of the 3D model may be any format.

The drawing data indicates the appearance including at least the size and shape of the size (for example, the size of each part), shape, and color of the target part for each of the plurality of degrees of deterioration.

Next, the maintenance DB 8 includes the following two types of replacement necessity information for each part by machine number. Take one component as an example (“target component” in the description of FIG. 2).

The first type of replacement necessity information includes information indicating a replacement cycle set in advance by a designer or the like using the past failure record of the target part. If the operating time (elapsed time after replacement) of the target part has reached the replacement cycle of the target part, it is determined that the target part needs to be replaced.

The second type of replacement necessity information is the amount of operation obtained from data measured by a remote monitoring device (for example, data measured by a sensor provided in the device 3) regarding the deterioration of the target part. It includes a replacement determination threshold value set by, for example. If the most recent operating amount (for example, the value of measured data) reaches the threshold, it is determined that the target part needs to be replaced. The details of the replacement necessity determination will be described later with reference to FIGS. 4A and 4B.

The parts determination unit 62 uses the parts appearance DB 7 and the maintenance DB 8 to perform, for example, the following.

The component determining unit 62 receives imaging data captured by the maintenance staff terminal 1 as an input, and identifies components appearing in the captured image indicated by the imaging data. Specifically, for example, the component determination unit 62 learns a support model, which is a model that inputs imaging data and outputs a component ID, according to AI technology such as deep learning, so that the input imaging data indicates A component can be identified from the component appearance DB 7 based on the characteristics of the component shown in the captured image. For example, in the component appearance DB 7, as described above, the 3D model and the appearance of each component are defined according to the degree of deterioration. A model and appearance are identified, and as a result, other information (eg, part name and model) of the part corresponding to the 3D model and appearance is identified.

In addition, the component determination unit 62 determines whether or not the external appearance (for example, shape or color) of the specified component needs to be replaced as a process to be performed next after specifying the component by the above-described method using the imaging data captured by the maintenance staff terminal 1 as input. judge. Appearance data such as shapes that have reached replacement criteria due to wear, for example, may be accumulated in the part appearance DB 7 (for example, appearances that have reached replacement criteria may be learned in advance based on accumulated imaging data). Utilizing AI technology such as deep learning, the part determination unit 62 evaluates the similarity of the characteristics of the imaging data with the characteristics of normal parts and parts that have reached the replacement criteria due to wear, and the similarity with normal parts is low. Alternatively, if the degree of similarity with the part that has reached the replacement criteria is high, it may be determined that replacement is necessary. Further, if the degree of deterioration corresponding to the specified 3D model or appearance corresponds to the degree of deterioration corresponding to the need for replacement, it may be determined that the replacement is required. Determining whether or not a part needs to be replaced from the part appearance by referring to the part appearance DB 7 is the "necessity determination X" in this embodiment.

Further, the part determination unit 62 determines whether or not the specified part needs to be replaced based on the replacement necessity information in the maintenance DB 8 . In the present embodiment, "necessity determination Y" refers to determining whether or not a part needs to be replaced based on the actual operating time and amount of operation of the part with reference to the maintenance DB 8 .

Both the parts appearance DB 7 and the maintenance DB 8 may include a list of parts (for example, part IDs) belonging to each machine number. The parts determination unit 62 may refer to the range corresponding to the machine number from the maintenance staff terminal 1 for both the parts appearance DB 7 and the maintenance DB 8 . As a result, it can be expected that identification of part information and determination of necessity of replacement can be performed at high speed.

FIG. 3 is a diagram showing the flow of component determination processing.

In S<b>1 , the component determination unit 62 acquires the imaging data and machine number information received by the reception unit 61 from the maintenance terminal 1 .

In S2, the component determination unit 62 identifies the component by referring to the component appearance DB 7 based on the imaging data. The part identified here is the target part.

In S3, the part determination unit 62 performs a necessity determination X that determines whether or not the target part needs to be replaced based on the degree of deterioration corresponding to the appearance of the target part.

In S4, the parts determination unit 62 searches the maintenance DB 8 using the ID of the part of interest and machine number information as keys to perform necessity determination Y, which is to determine whether or not the part of interest needs to be replaced.

In S5, the part determination unit 62 checks the result of the necessity determination X. FIG. If the result of the necessity judgment X is that replacement is required, part information including information indicating that replacement is required is transmitted to the maintenance terminal 1 .

If the result of the necessity determination X is that the replacement is unnecessary, the part determination unit 62 checks the result of the necessity determination Y in S6. If the result of the necessity determination Y is that replacement is required, the component information including information indicating that replacement is required is transmitted to the maintenance terminal 1 . On the other hand, if the result of the necessity determination Y is that replacement is not required, the component information including information indicating that replacement is not required is transmitted to the maintenance terminal 1 .

Instead of being performed regardless of the result of the necessity determination X, the necessity determination Y may be performed when the result of the necessity determination X indicates that replacement is unnecessary.

FIG. 4A is a diagram showing a first necessity determination. FIG. 4B is a diagram illustrating the second necessity determination.

As described above, the maintenance DB 8 is a DB that stores the following first type replacement necessity information and second type replacement necessity information regarding each part of each elevator machine. The necessity determination Y includes the first necessity determination shown in FIG. 4A and the second necessity determination shown in FIG. 4B.

The first necessity determination is a necessity determination using the first type of replacement necessity information. The replacement cycle can be set by utilizing past failure records. For example, the replacement cycle is set in advance by the designer based on the operating hours and the like for each component of the same type. Specifically, for example, the first necessity determination is performed according to the procedure shown in FIG. 4A.

There is first-class replacement necessity information including past failure record data. The first type of replacement necessity information includes, for example, information indicating the failure occurrence time and the number of failures for each component of the same type. Here, the "failure occurrence time" can be calculated from the time from the installation date or replacement date registered for each component to the failure occurrence date. Note that the failure occurrence time may be replaced with a deterioration parameter corresponding to each component, such as the number of times of operation or power-on time.

In SW1, the part determination unit 62 calculates the shape parameter m of the target part by performing Weibull analysis based on the first replacement necessity information (for example, failure occurrence time and failure number data).

In SW2, the part determination unit 62 determines whether m≧1 for the shape parameter m of the target part obtained by the Weibull analysis.

In S-W3, the component determination unit 62 determines whether or not the component of interest with m≧1 has a high degree of importance (A rank).

Parts of interest with m≧1 are wear-out failure type parts. If the target component is of A rank, in SW4, the component determining unit 62 calculates a failure occurrence cycle with A1 (A1>0)% reliability. “A1% reliability failure occurrence period” means a period in which a failure is allowed to occur with a probability of (100−A1)%. If the target component is not of rank A, then in S-W5, the component determination unit 62 calculates a failure occurrence cycle with a reliability of A2% (A2>0 and A1>A2). The component determination unit 62 sets the failure occurrence cycle calculated in SW4 or SW5 as the replacement cycle.

Note that the following reliability calculation formula used in general Weibull analysis may be adopted for calculating the failure occurrence period described above.
Failure rate F(t) = 1-exp(-(t/η) ^m

For example, the failure occurrence period with 95% reliability is exp(ln(η)+(ln(ln(1/(1-0.05))))/m).

In SW6, the component determination unit 62 determines whether or not the component of interest with m<1 is of high importance (A rank).

A component of interest with m<1 is a random failure type component. If the target part is of A rank, in S-W7, the part determination unit 62 calculates a failure occurrence period that is B1 times the actual usage of the previous year (for example, B1>1). If the part of interest is a part other than the A-rank part, in S-W8, the part determination unit 62 calculates a replacement occurrence cycle that is B2 times the actual usage of the previous year (for example, B2≧1 and B1>B2). . The component determination unit 62 sets the failure occurrence cycle calculated in SW7 or SW8 as the replacement cycle.

In S-W9, the component determination unit 62 determines whether or not the operating time of the target component has reached the set replacement cycle. When it is determined that the replacement cycle has arrived, the part determination unit 62 determines that replacement is necessary. When it is not determined that the replacement cycle has arrived, the part determination unit 62 determines that replacement is unnecessary. A result of the first necessity determination is registered in the maintenance DB 8 for the target part. Whether or not the replacement cycle has been reached can be determined by calculating (replacement cycle−current cumulative operating time).

On the other hand, the second necessity determination is a necessity determination using the second type of replacement necessity information. The second type of replacement necessity determination information includes, for example, measurement data on deterioration of each part by a remote monitoring device.

As shown in FIG. 4B, measurement data related to the deterioration of each part (for example, the number of brake operations in the case of brake linings, etc.) by a sensor or the like installed in the device 3 is collected by a remote monitoring device (not shown) connected to the device 3. After being connected, the measurement data is periodically accumulated in the second type of replacement necessity information (for example, a type of DB) in chronological order together with the measurement date. Measured data resulting from the deterioration of each component includes, for example, running time, temperature, humidity, number of pulses, energization time, number of starts, torque, current, and the like.

The second necessity determination is obtained by the following procedure using the measurement data as described above.

In S-R1, the part determination unit 62 acquires the latest value of the measurement data (remote monitoring data) corresponding to the target part from the second type replacement necessity information.

In S-R2, the part determination unit 62 determines whether or not the latest value of the measurement data acquired in S-R1 exceeds a threshold value set in advance by the designer. If the determination result of S-R2 is true, the determination result is replacement required. If the determination result of S-R2 is false, the determination result does not need to be exchanged. Information indicating the determination result is registered in the maintenance DB 8 for the target part.

FIG. 5 is a diagram showing the flow of necessity determination Y. As shown in FIG.

The necessity determination Y may be at least one of the first necessity determination and the second necessity determination. When the determination result of one of the first necessity determination and the second necessity determination is false, the other necessity determination may be performed. One example is as shown in FIG.

That is, in S-D1, the component determination unit 62 performs the first necessity determination. If the determination result of S-D1 is true, the determination result indicates that replacement is required.

If the determination result in S-D1 is false, in S-D2, the component determining unit 62 performs a second necessity determination. If the determination result of S-D2 is true, the determination result indicates that replacement is required. If the determination result of S-D2 is also false, the determination result indicates that exchange is unnecessary.

FIG. 6 is a diagram showing an example of a component determination result screen.

This screen 601 is, for example, a GUI (Graphical User Interface), and is displayed when the maintenance staff terminal 1 is a tablet PC. This screen 601 has an appearance area 611 and a detail area 612 .

An appearance area 611 displays the appearance and name of the device corresponding to the machine number, and the captured image.

A detail area 612 displays at least a part of the identified parts information, a UI (User Interface) 631 for receiving a quote request, and a UI 632 for receiving an order request.

FIG. 7 is a diagram showing another example of the component determination result screen.

This screen 701 is displayed, for example, when the maintenance terminal 1 is a smart phone. On this screen 701, an object 702 of part information is displayed according to the AR function. When the object 702 is designated by the maintenance staff 4 (for example, when the object 702 is touched), the display unit 63 displays a parts determination result screen 703 showing details of parts information. The part determination result screen 703 displays an image 721 indicating the identified 3D model, an object 722 indicating the deterioration position, and at least part of the part information 731 .

The component determination unit 62 identifies the deterioration position of the component based on at least one of the identified degree of deterioration of the target component and the analysis result of the captured image, , the position corresponding to the deteriorated position is determined as the highlighted display position. The highlighting position is highlighted by the display unit 63 . An example of highlighting is the display of object 722 . Object 722 may be a texture image or other type of object covering the area corresponding to the degraded location. As a mode of highlighting, any mode such as display of different colors or shades of light can be adopted. This allows maintenance personnel to notice deterioration that is difficult to notice with the naked eye, for example, at important diagnostic locations such as hot spots.

Note that the part specified by the part determination unit 62 and displayed by the display unit 63 is not necessarily the part shown in the captured image. In other words, the determined parts are not always correct.

Therefore, the part determination unit 62 may specify N parts of interest (candidates) based on the captured image, and the part of interest having the highest degree of similarity among them may be set as a display target. When maintenance personnel 4 inputs that the component of interest is incorrect, the component of interest having the next highest degree of similarity may be displayed. When maintenance personnel 4 inputs that the component of interest to be displayed is correct, the accumulated imaging data may be associated with the data of the correct component. Based on the accumulation of such ties, the component determination unit 62 may learn the support model described above. As a result, it is expected that the accuracy of component determination will be improved.

The above description can be summarized, for example, as follows.

The maintenance support system includes a reception section 61 and a display section 63 . The receiving unit 61 receives imaging data indicating a captured image of one or more components captured by a camera among a plurality of components constituting the equipment 3 to be maintained. The display unit 63 displays information indicating the degree of deterioration of a target component, which is a component focused on based on the captured image among one or more components shown in the captured image indicated by the imaging data. Display information. This makes it possible to support maintenance without installing a tag for each part.

The maintenance support system is implemented by at least one of a maintenance terminal 1 (an example of an information processing terminal) and a component determination system 5 (an example of a computer system capable of communicating with an information processing terminal). For example, the maintenance staff terminal 1 may have at least one of the reception unit 61 and the display unit 63 instead of the parts determination system 5 .

The component information of the target component is displayed in association with the image of the target component among the captured images displayed on the monitor of the maintenance terminal 1 . This makes it easy for maintenance personnel to understand which component information is displayed in the captured image.

The part information includes a corresponding model, which is a three-dimensional model according to the degree of deterioration of the part of interest, among the three-dimensional models of the part of interest. As a result, even if the degree of deterioration of the target component is difficult to see with the naked eye, the maintenance personnel can grasp the degree of deterioration from the corresponding model. Note that the position of the model determined based on at least one of the degree of deterioration of the component of interest and the image of the component of interest in the captured image may be highlighted. This allows maintenance personnel to notice deterioration that is difficult to notice with the naked eye, for example, at important diagnostic locations such as hot spots.

A maintenance support system includes a parts determination unit 62 . The component determination unit 62 identifies the component information of the target component based on the captured image indicated by the captured data and the component appearance DB 7 (an example of component appearance information) containing information on the ID and shape of each component. By analyzing the captured image, it is possible to identify the appearance of the component. Based on the appearance, the component appearance DB 7, which includes information on the shape of each component, is referred to, so it is possible to accurately identify the component information. Be expected.

The part appearance DB 7 stores, for each part, a plurality of three-dimensional models corresponding to a plurality of degrees of deterioration of the part (this may include a three-dimensional model of a part with a zero degree of deterioration (i.e., normal)) and dimensions of the part. It contains information indicating The component determination unit 62 identifies the shape and scale of the captured component by analyzing the imaging data. The part determination unit 62 identifies a three-dimensional model and dimensions that match the identified shape and scale from the part appearance DB 7 . In this way, even if the target component is degraded, it is expected that the appearance of the target component (appearance obtained from the captured image) will match the 3D model according to the degree of deterioration. Identification is expected.

The component determination unit 62 determines whether or not the component of interest needs to be replaced based on the specified degree of deterioration. The component information includes information indicating whether or not the target component needs to be replaced. This allows maintenance personnel to know whether replacement is necessary according to the degree of deterioration.

When the identified degree of deterioration of the target component is smaller than the degree of deterioration indicating that replacement is necessary, the component determination unit 62 obtains information including at least one of information indicating the past failure status and status indicating the operating status of each component. Determination of necessity of replacement of the target part from the degree of deterioration of the target component (eg, replacement cycle or operation amount (eg, number of operations)) specified based on the maintenance DB 8 (an example of maintenance information) . Thus, when it is determined from the part appearance DB 7 (in other words, from the appearance of the target part) that the target part does not need to be replaced, the maintenance DB 8 is used to determine whether the target part needs to be replaced. Therefore, it is expected that the accuracy of determining whether or not the part of interest needs to be replaced is improved.

Determining whether the component of interest needs to be replaced includes at least one of a first determination of necessity and a second determination of necessity according to the replacement cycle. As a result, it is expected that the accuracy of determining whether or not the part of interest needs to be replaced is improved.

The component determination unit 62 performs the following in the first necessity determination.
・Determine whether the deterioration of the part of interest is wear type or accidental type.
When the wear type is determined, a replacement period is calculated according to the reliability determined according to the importance of the target part.
- If an accidental type is determined, a replacement cycle is calculated according to the importance of the target part and the replacement history of the target part.
- Determine whether or not the number of days elapsed from the latest replacement date of the target component has reached the calculated replacement cycle.

The component determination unit 62 performs the following in the second necessity determination.
- Identify the operation amount of the target part from the maintenance DB 8 .
- Determine whether or not the identified operating amount is equal to or greater than the operating amount that indicates that the target component needs to be replaced.

When the determination result of one of the first necessity determination and the second necessity determination is false (when it is determined that replacement is not necessary), the component determination unit 62 performs the other necessity determination. As a result, it is expected that the accuracy of determining whether or not the part of interest needs to be replaced can be further improved.

The component determination unit 62 accumulates the received imaging data in the component appearance DB 7, for example. The location (storage area) where the imaging data is accumulated may be any location. The component determination unit 62 determines component IDs of one or more target components for the received image data from the component appearance DB 7 using a support model, which is a model that inputs image data and outputs component IDs. The part determination unit 62 receives, for example, a maintenance worker's selection of a part appearing in the captured image among the one or more target parts. The component determination unit 62 performs learning for updating the support model based on the relationship between the imaging data and the component ID of the selected component. As a result, it is expected that the accuracy of the support model will be improved, and thus the accuracy of the characteristics of the part information will be improved.

Procurement support information (for parts of interest) that is information indicating at least one of the amount, the number of stocks, and the delivery date obtained from the inventory DB 71 (an example of inventory information) containing information indicating the price, the number of inventory, and the delivery date of each part The parts of interest are ordered in response to approval of the display of the procurement support information). This makes it easy for maintenance personnel to make arrangements for replacement of the part of interest.

Equipment to be maintained is equipment that the elevator has. The parts appearance DB 7 includes information about the equipment and parts that the elevator of each elevator has. Information indicating the number of the elevator is associated with the imaging data. The component determination unit 62 identifies the component information of the target component from the information corresponding to the machine number indicated by the information associated with the imaging data in the component appearance DB 7 . As a result, the reference range of the part appearance DB 7 is narrowed down to the information corresponding to the machine number. Therefore, high-speed identification of parts information is expected.

Although one embodiment of the present invention has been described above, this is an example for explaining the present invention, and is not intended to limit the scope of the present invention only to this embodiment. The present invention can also be implemented in various other forms.

1: maintenance personnel terminal, 5: parts determination system, 7: parts appearance DB, 8: maintenance DB

Claims (12)

a receiving unit that receives imaged data representing a captured image of one or more components captured by a camera among a plurality of components that constitute the equipment to be maintained;
Display of component information including information indicating the degree of deterioration of the target component, which is a component focused on based on the captured image among the one or more components captured in the captured image indicated by the imaging data. display and
with
The part information includes a corresponding model, which is a three-dimensional model according to the degree of deterioration, among the three-dimensional models of the target part,
Maintenance support system. A position of the corresponding model determined based on at least one of the degree of deterioration of the target component and the image of the target component in the captured image is highlighted.
The maintenance support system according to claim 1 . Further comprising a component determination unit that identifies component information of the target component based on the imaged image indicated by the imaged data and component appearance information including information on the ID and appearance of each component,
The maintenance support system according to claim 1. The part determination unit determines whether or not the target part needs to be replaced based on the specified degree of deterioration,
The part information includes information indicating the necessity of replacement determined for the part of interest,
The maintenance support system according to claim 3. a receiving unit that receives imaged data representing a captured image of one or more components captured by a camera among a plurality of components that constitute the equipment to be maintained;
Display of component information including information indicating the degree of deterioration of the target component, which is a component focused on based on the captured image among the one or more components captured in the captured image indicated by the imaging data. a display unit for
a component determination unit that identifies component information of the target component based on the imaged image indicated by the imaged data and component appearance information including information on the ID and appearance of each component;
with
The part appearance information includes, for each part, information indicating a plurality of three-dimensional models corresponding to a plurality of degrees of deterioration of the part and the dimensions of the part,
The parts determination unit
By analyzing the imaging data, identifying the shape and scale of the part shown,
identifying a three-dimensional model and dimensions that conform to the identified shape and scale from the part appearance information;
The part corresponding to the specified three-dimensional model and dimensions is the target part,
The identified degree of deterioration of the part of interest is the degree of deterioration corresponding to the identified three-dimensional model,
Maintenance support system. a receiving unit that receives imaged data representing a captured image of one or more components captured by a camera among a plurality of components that constitute the equipment to be maintained;
Display of component information including information indicating the degree of deterioration of the target component, which is a component focused on based on the captured image among the one or more components captured in the captured image indicated by the imaging data. a display unit for
a component determination unit that identifies component information of the target component based on the imaged image indicated by the imaged data and component appearance information including information on the ID and appearance of each component;
with
The part information includes information indicating whether or not the part of interest needs to be replaced, and
When the identified degree of deterioration of the target component is smaller than the degree of deterioration indicating the need for replacement, the component determination unit determines at least one of information indicating past failure status and status indicating operating status for each component. Determining whether or not the part of interest needs to be replaced from the degree of deterioration of the part of interest, which is specified based on maintenance information that is information included;
Maintenance support system. Determining whether the component of interest needs to be replaced includes at least one of a first determination of necessity in accordance with a replacement cycle and a second determination of necessity in accordance with an operation amount,
The component determination unit, in the first necessity determination,
Determining whether the deterioration related to the target component is a wear type or an accidental type,
When the wear type is determined, calculating the replacement cycle according to the reliability determined according to the importance of the target part,
If the accidental type is determined, calculating a replacement cycle according to the importance of the target part and the replacement history of the target part,
determining whether or not the number of days elapsed from the latest replacement date of the target component has reached the calculated replacement cycle;
Determining whether or not the target component needs to be replaced includes a second determination of necessity or non-necessity according to the amount of operation,
The parts determination unit, in the second necessity determination,
identifying the operating amount of the target part from the maintenance information;
Determining whether the specified operation amount is equal to or greater than the operation amount that means that the attention part needs to be replaced;
The maintenance support system according to claim 6 . The part determination unit performs the necessity determination of the other when the determination result of one of the first necessity determination and the second necessity determination is false.
The maintenance support system according to claim 7 . The parts determination unit
accumulating the accepted imaging data;
determining the component IDs of one or more components of interest for the received imaging data from the component appearance information using a support model, which is a model that receives imaging data as an input and outputs a component ID;
Receiving selection of a component appearing in the captured image among the one or more components of interest;
Learning to update the support model based on the relationship between the imaging data and the part ID of the selected part,
The maintenance support system according to claim 5 or 6 . In response to approval to display the procurement support information, which is information indicating at least the amount of the amount, the number of stocks, and the delivery date obtained from the inventory information including the information indicating the amount, the number of inventory, and the delivery date of each part, the attention parts are ordered,
The maintenance support system according to claim 5 or 6 . The device to be maintained is a device possessed by an elevator,
The part appearance information includes information about the equipment and parts that the elevator of the elevator has for each elevator number,
Information indicating the number of the elevator is associated with the imaging data,
The part determination unit identifies part information of the target part from information corresponding to a machine number indicated by information associated with the imaging data, among the part appearance information.
The maintenance support system according to claim 5 or 6 . The component information is displayed in association with the image of the target component in the captured image,
The maintenance support system according to any one of claims 1 to 11.

Images