JP2017097856A - Information processing device, information processing method, information processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support maintenance work on the basis of the state of a device shown in previously created work procedure information and the state of an actual device becoming the object of maintenance.SOLUTION: An information processing device for supporting work in the maintenance of the device includes a storage part which stores component list information for each work procedure of the maintenance and component definition information for specifying a component possessed by the device, a specification part which specifies a work object component handled in the work procedure and the number of first components which is the number of the work object components on the basis of the component list information, a photographing part which generates the image data of an image obtained by photographing the device, an analysis part which specifies the work object component in the image data and the number of second components which is the number of the work object components on the basis of the component definition information, and a display control part which displays maintenance information on the basis of the result of comparing the number of the first components with the number of the second components.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, an information processing system, and a program.

  2. Description of the Related Art Conventionally, a method using an electronic manual or the like obtained by digitizing a manual indicating operation methods of various devices is known.

  For example, a method of providing support information that is highly convenient for using an electronic manual is known. Specifically, based on the image feature amount of the part used in the device, a similar image search process is performed to identify the part identifier, and support information such as help information is extracted from the identified part identifier. Is known (see, for example, Patent Document 1).

  However, in the maintenance of the apparatus, the state of the apparatus indicated in the work procedure information created in advance may be different from the state of the actual apparatus that is a maintenance target. In such a case, in the conventional technology, in the maintenance of the device, the difference between the state of the device indicated in the work procedure manual and the state of the device that is the object of the maintenance is to what extent in the maintenance work. In many cases, it cannot be judged whether it has an influence.

  An object of one aspect of the present invention is to support maintenance work based on the state of an apparatus indicated in work procedure information created in advance and the state of an actual apparatus that is a maintenance target.

In order to solve the above-described problem, an information processing apparatus that supports work in apparatus maintenance, which is an aspect of the present invention, includes parts list information for each maintenance work procedure, and parts included in the apparatus. Based on the parts list information, a part to be handled that is handled in the work procedure, and a first part number that is the number of parts to be worked are identified based on the part list information. A specific unit that performs the processing, a photographing unit that generates image data of an image obtained by photographing the device, a second part that is the number of the work target part and the work target part in the image data based on the part definition information. A display control unit that displays maintenance information based on a result of comparing the number of parts, an analysis unit that identifies the number of parts, and the number of first parts and the number of second parts;
It is characterized by providing.

  Maintenance work can be supported on the basis of the state of the device indicated in the work procedure information created in advance and the state of the actual device to be maintained.

It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows an example of a function structure of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the work content database of 1st embodiment. It is a figure which shows an example of the components definition database of 1st embodiment. It is a figure which shows an example of the components list database of 1st embodiment. It is a figure which shows an example of the notification content database of 1st embodiment. It is a flowchart which shows an example of the whole process by the information processing apparatus of 1st embodiment. It is a 1st figure explaining the maintenance using the information processing apparatus of 1st embodiment. It is a 2nd figure explaining the maintenance using the information processing apparatus of 1st embodiment. It is a figure which shows the example of a display of warning information. It is a figure which shows the example of a display of an "OK" icon. It is a figure which shows an example of the components list database of 2nd embodiment. It is a flowchart which shows an example of the whole process by the information processing apparatus of 2nd embodiment. It is a figure which shows the example of a display of the maintenance information in 2nd embodiment.

  Embodiments of the present invention will be described below.

<First Embodiment>
A user may perform maintenance on the apparatus. In this maintenance work, a plurality of procedures are often performed. Hereinafter, an example in which a plurality of procedures are performed by the user in the maintenance work will be described. It is assumed that a part is removed from the apparatus in the maintenance procedure. In this case, it is assumed that there is a part in the user's hand during the work, that is, in a state where a part of the maintenance work is completed. On the other hand, when the maintenance work is completed, that is, when all of the plurality of procedures are completed, the parts that are at the user's hand are attached to the apparatus according to the procedure, and the parts are not left at the user's hand.

<Example of hardware configuration of information processing apparatus>
FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing apparatus according to an embodiment of the present invention. Hereinafter, an example in which the information processing apparatus is a so-called head mounted display (hereinafter referred to as “HMD”) 10 will be described.

  As illustrated, the HMD 10 includes a display device 10H1, a touchpad 10H2, a network interface 10H3, a photographing device 10H4, and a CPU (Central Processing Unit) 10H5. Further, the HMD 10 includes a ROM (Read-Only Memory) 10H6, a RAM (Random Access Memory) 10H7, and a flash memory 10H8.

  The display device 10H1 is, for example, a display. That is, the display device 10H1 is an example of an output device that outputs an image or the like.

  The touch pad 10H2 is an example of an input device for a user to input an operation. That is, the touch pad 10H2 inputs a command or the like by a user's gesture operation.

  The network interface 10H3 is an example of an interface for inputting / outputting data or the like to / from an external device such as a server via a network by wire or wireless. For example, the network interface 10H3 is an antenna and a processing IC (Integrated Circuit).

  The imaging device 10H4 is, for example, a camera. That is, the imaging device 10H4, when the user is performing maintenance, for example, when the user is performing maintenance, captures all or part of the device that is the maintenance target, and generates an image.

  The CPU 10H5 is an example of an arithmetic device that performs various processes and processing of various data, and a control device that controls each hardware and the like.

  The ROM 10H6, the RAM 10H7, and the flash memory 10H8 are examples of storage devices. Specifically, the ROM 10H6 or the flash memory 10H8 stores input data, programs, and the like. The RAM 10H7 is a storage area or the like where the program to be executed is expanded.

  Note that the hardware configuration is not limited to the illustrated configuration. For example, the information processing apparatus may further include an arithmetic device or a control device. Further, the information processing apparatus may be composed of a plurality of apparatuses.

  Hereinafter, the HMD 10 will be described as an example in which it is used by being worn on the head of a user who performs maintenance of the apparatus, that is, an operator. Further, in the following example, the HMD 10 is a so-called transmission (See-Through) type, and a user wearing the HMD 10 will be described as an example in which a line of sight is transmitted and the state outside the HMD 10 can be seen. .

  FIG. 2 is a functional block diagram illustrating an example of a functional configuration of the information processing apparatus according to the embodiment of the present invention. Specifically, the HMD 10 includes a display control unit 10F1, a touch pad control unit 10F2, a display control unit 10F3, a content management unit 10F4, a storage control unit 10F5, and a network control unit 10F6. The HMD 10 also includes a component number identification unit 10F7, a content creation unit 10F8, a photographing unit 10F9, an analysis unit 10F10, a determination unit 10F11, and a database storage unit 10F12.

  First, the database storage unit 10F12 will be described. The database storage unit 10F12 is realized by the ROM 10H6, the RAM 10H7, the flash memory 10H8, and the like.

  The database storage unit 10F12 stores a work content database 210, a component definition database 220, a component list database 230, and a notification content database 240. Each database is stored in the database storage unit 10F12 in advance by, for example, an administrator of the HMD 10.

  The work content database 210 stores work content information indicating work content in maintenance for each apparatus. The component definition database 220 stores component definition information for identifying components included in a device from image data obtained by capturing each device.

  The parts list database 230 stores parts list information indicating a list of parts to be handled in each procedure of work contents. The parts list database 230 stores the number of missing parts allowed for each part.

  The notification content database 240 stores notification content information notified as a determination result by the determination unit 10F11. Details of each database will be described later.

  The display control unit 10F1 displays maintenance information based on the analysis result by the analysis unit 10F10 and the notification content database 240.

  For example, when the user performs a touch operation with a finger, the touch pad control unit 10F2 receives the touch operation. Next, the touch pad control unit 10F2 inputs an operation by a touch gesture such as a tap and a swipe based on the received touch operation. The touch pad control unit 10F2 is realized by, for example, the touch pad 10H2 (FIG. 1).

  The display control unit 10F3 controls an image displayed on the display. The display control unit 10F3 is realized by, for example, the display device 10H1 (FIG. 1).

  The content management unit 10F4 manages the database storage unit 10F12. That is, the content management unit 10F4 updates each database stored in the database storage unit 10F12 and inputs necessary data from an external device. The content management unit 10F4 is realized by, for example, the CPU 10H5 (FIG. 1).

  The storage control unit 10F5 stores a database input from an external device via a network or the like in the database storage unit 10F12. The storage control unit 10F5 is realized by, for example, the CPU 10H5 and the flash memory 10H8 (FIG. 1).

  The network control unit 10F6 controls network communication such as a wireless LAN (Local Area Network). The network control unit 10F6 is realized by, for example, the network interface 10H3.

  Based on the component list database 230, the component number specifying unit 10F7 specifies the number of components handled in each procedure indicated by the work content information (hereinafter referred to as “first component number”). In other words, the part number specifying unit 10F7 specifies the parts associated with each procedure according to the maintenance work content information and the number of parts (first part number). The component number specifying unit 10F7 is realized by, for example, the CPU 10H5.

  The content creation unit 10F8 creates maintenance information to be displayed based on the determination result by the determination unit 10F11. More specifically, the content creation unit 10F8 includes maintenance information including an image output by the analysis unit 10F10 as a result of analyzing the image data captured by the imaging unit 10F9 and notification content information based on the determination result. Create and pass to display control unit 10F1. The content creation unit 10F8 is realized by, for example, the CPU 10H5.

  The imaging unit 10F9 generates image data of an image including parts handled in maintenance for the apparatus. Note that the photographing unit 10F9 is realized by, for example, the photographing apparatus 10H4 (FIG. 1).

  The analysis unit 10F10 refers to the component definition database 220, analyzes the image data generated by the imaging unit 10F9, and calculates the number of components handled in maintenance and the number of components (hereinafter, the second number of components) in the image data. Identify. The analysis unit 10F10 may generate schematic image data indicating a handled part from the image data, and output the generated image data to the content creation unit 10F8. The analysis unit 10F10 is realized by, for example, the CPU 10H5.

  The determination unit 10F11 compares the first component number with the second component number. Further, when there is a difference between the first component number and the second component number as a result of the comparison, the determination unit 10F11 refers to the component list database 230 and determines whether or not the difference is less than the allowable number. judge. In other words, the determination unit 10F11 determines whether or not the difference between the number of first components and the number of second components is a difference that has no effect or a small effect when the maintenance work is continued.

  In addition, the difference which has no influence or the difference which has little influence is a grade which can be disregarded, for example, the difference by a lot etc. or the difference by the work mistake which can be disregarded. On the other hand, the difference that has an effect is a difference that affects subsequent procedures. For example, influential differences are caused by serious work mistakes. The determination unit 10F11 is realized by, for example, the CPU 10H5.

  In the HMD 10, maintenance information corresponding to the determination result is created by the content creation unit 10F8 and displayed by the display control unit 10F1.

  In the HMD 10, when there is a difference between the number of first parts and the number of second parts, maintenance information according to each state such as “warning state” and “error state” is displayed according to the determination result. You may let them. Further, in the HMD 10, when there is no difference between the first component number and the second component number, the work content information may be displayed as maintenance information.

  The “warning state” is a state when the difference between the number of first parts and the number of second parts is determined to be a difference having no effect or a difference having a small effect. In this case, the HMD 10 can continue to support maintenance work according to the procedure.

  The “error state” is a state when the difference between the number of first parts and the number of second parts is determined to be a difference that affects subsequent procedures and the like. In this case, the HMD 10 prompts the maintenance work to be interrupted.

<Each database>
The databases stored in the database storage unit 10F12 will be described below with reference to FIGS.

  FIG. 3 is a diagram illustrating an example of a work content database according to the first embodiment. The work content database 210 stores work procedures performed on the work content in association with each work content.

  The work content database 210 includes a procedure No. as an information item. , Still image file, message, and other items are associated with the item “procedure No.”. In the following description, information including the value of the item “procedure No.” and the values of other items in the work content database 210 is referred to as work content information.

  The value of the item “Procedure No.” indicates the number of the procedure in the work content. The value of the item “still image file” indicates the corresponding procedure No. It is the image data which shows the work content performed in. The value of the item “message” indicates a message (text data) displayed together with the image data that is the value of the item “still image file”.

  In the example of FIG. 1, image data “Step 1.jpg” and text data “Open the control panel,...” Are displayed as work content information.

  Note that the work content information of the present embodiment may be used when the maintenance information is created by the content creation unit 10F8. In the example of FIG. In the work content information 210-1 of No. 1, the message is “... Remove the $ 1 screw on the front board”. In this case, “$ 1” can be a variable.

  In the present embodiment, a first part number of a parts list database 230 (to be described later) may be substituted into a variable included in the message to be a part of the work content information, or the second part number may be substituted to store one piece of maintenance information. It is good as a part.

  Further, the work content in the maintenance is not limited to the work shown in FIG. The maintenance work content may be one procedure. On the other hand, the maintenance work may be performed in succession by a plurality of works or procedures.

  FIG. 4 is a diagram illustrating an example of a component definition database according to the first embodiment. The component definition database 220 is provided for each type of device to be maintained.

  The component definition database 220 includes a component ID, feature amount data, and a component name as information items. The item “component ID” is associated with other items. In the following description, information including the value of the item “component ID” and the values of other items in the component definition database 220 is referred to as component definition information.

  The value of the item “component ID” is an identifier for identifying the component. The value of the item “feature data” indicates a key point group. The key point is calculated from the extreme value of the amount of change indicating the gradient of the image. The key points are configured by position, angle, size, 128-dimensional histogram vector, and the like.

  The value of the item “part name” indicates the name of the part identified by the corresponding part ID.

  The component definition database 220 of this embodiment is referred to when image data is analyzed by the analysis unit 10F10.

  FIG. 5 is a diagram illustrating an example of a parts list database according to the first embodiment. The parts list database 230 of this embodiment is provided for each work content.

  The parts list database 230 includes a procedure No. as an information item. , Part ID, number of first parts, warning threshold (first threshold), and error threshold (second threshold). In the present embodiment, the second threshold value is larger than the first threshold value.

  In the parts list database 230, the item “procedure No.” is associated with other items. In the following description, information in which the value of the item “procedure No.” is associated with the values of other items in the parts list database 230 is referred to as parts list information.

  In this embodiment, for example, when the difference between the number of first parts and the number of second parts is larger than the first threshold value, the difference between the two has a significant influence on the continuation of work, and is allowed. Judge that it is not.

  In this embodiment, when the difference between the number of first parts and the number of second parts is larger than the second threshold, the difference between the two does not affect the continuation of the work or does not make the continuation difficult. It is determined that it is acceptable.

  Therefore, for example, when the second threshold value is “0”, such as the component ID “pid_2”, it can be understood that no significant difference is permitted. On the other hand, when the second threshold is “3” as in the component ID “pid — 3”, it can be seen that the component allows a difference of up to three.

  FIG. 6 is a diagram illustrating an example of a notification content database according to the first embodiment. The notification content database 240 of this embodiment is provided for each work content.

  The notification content database 240 includes a procedure No. as an information item. Have warnings, errors. In the notification content database 240, the item “procedure No.” is associated with other items. In the following description, information including the value of the item “procedure No.” and the values of other items is referred to as notification content information.

  The value of the item “warning” indicates contents to be notified when it is determined that the difference between the number of first parts and the number of second parts is acceptable. Specifically, the value of the item “warning” is information for notifying that the number of parts associated with the work content information is different from the number of parts of the actual device, for example.

  The value of the item “error” indicates the content notified when it is determined that the difference between the number of first components and the number of second components is not allowed. Specifically, the value of the item “error” indicates that, for example, the difference between the number of parts associated with the work content information and the number of parts of the actual device exceeds the allowable range, and the maintenance work is continued. It is information for notifying that it is difficult.

  The notification content database 240 of this embodiment is referred to by the content creation unit 10F8 and the display control unit 10F1.

  Next, the operation of the HDM 10 of this embodiment will be described with reference to FIG.

  FIG. 7 is a flowchart illustrating an example of overall processing performed by the information processing apparatus according to the first embodiment.

  The display control unit 10F1 receives input of the model of the device to be maintained and the work content, refers to the work content database 210, and causes the display device 10H1 to display work content information indicating a procedure included in the work content. (Step S701).

  Specifically, the display control unit 10F1 includes the procedure No. among the procedures included in the work content information. The work content information with the smallest value is selected and displayed.

  Subsequently, the HMD 10 captures the target component by the imaging unit 10F9, and acquires image data including the target component (step S702). In the example of FIG. 7, when the input of the model and the work content is received, the work content information is displayed first. However, the present invention is not limited to this. In the present embodiment, the image captured by the imaging unit 10F9 may be displayed on the display device 10H1 until the maintenance information is displayed after receiving the input of the model and the work content.

  In addition, the HMD 10 of the present embodiment may cause the display control unit 10F1 to display guide information or the like that prompts the operator to shoot the maintenance target device. Specifically, for example, the display control unit 10F1 guides the operator to a position where the device range in the image captured by the imaging unit 10F9 is equivalent to the range included in the image 71 of the procedure image IMG1. Information may be displayed on the display device 10H1.

  Subsequently, the HMD 10 refers to the component list database 230 by the component number specifying unit 10F7 and determines whether or not there are a plurality of component IDs corresponding to the procedure in which the work content information is displayed (step S703). ).

  If a plurality of component IDs do not exist in step S703, the component number identification unit 10F7 proceeds to step S705 described later. In this case, since the part ID of the part for which the first part number and the second part number are obtained is one, the first part number and the second part number are obtained by the following processing for the part having the part ID. .

  In step S703, when there are a plurality of component IDs, the component number identification unit 10F7 selects the component with the first component ID (step S704). The type of component selected here is a component for which the number of first components and the number of second components are to be obtained.

  Subsequently, the component number identification unit 10F7 identifies the first component number of the target component (step S705). Specifically, for example, the number-of-parts specifying unit 10F7 refers to the parts list database 230 based on the input model and work content, and executes an execution procedure No. in the input model and work content. The parts list information including is extracted. Then, the component number identification unit 10F7 identifies the first component number included in the component list information.

  Subsequently, the HMD 10 analyzes the image data acquired in step S702 by the analysis unit 10F10, and specifies the second number of parts of the target part (step S706).

  Hereinafter, analysis of image data by the analysis unit 10F10 will be described. The analysis unit 10F10 of the present embodiment refers to the component definition database 220 and analyzes whether or not the target component is included in the image data based on the feature amount data included in the component information of the target component.

  Image analysis by the analysis unit 10F10 is realized by so-called pattern matching, for example. The analysis unit 10F10 analyzes whether or not a subject that is the same as or similar to the target component is captured in the image data. In the image analysis, the type and size of parts may be classified. The data used for image analysis may be an image.

  For example, SURF (Speed-Up Robust Features) is used as an algorithm for analyzing an image. When SURF is used, the influence of illumination, scale, angle, etc., on which a subject is photographed can be reduced. In addition, since the data indicating the feature amount often has a smaller data capacity than the image data, if the data indicating the feature amount is used, the data capacity for performing image analysis can be reduced.

  In SURF, first, a sample image of a part to be a subject is input in advance. Next, based on the sample image, a key point group is calculated from the extreme value of the amount of change such as the gradient of the image. Subsequently, the analysis unit 10F10 collates the image data based on the feature amount data. If it is determined by collation that there is a subject that is the same as or similar to the feature data, the analysis unit 10F10 assumes that the target component is included in the image indicated by the image data.

  In addition, for example, a QR code (registered trademark), an AR marker, and the like are pasted on the part in advance, and the analysis unit 10F10 performs analysis to recognize the part based on the read QR code (registered trademark) and the AR marker. May be.

  Next, in the HMD 10, the determination unit 10F11 calculates the difference between the first component number and the second component number. Further, the determination unit 10F11 refers to the parts list database 230, and in the input work content, the currently executed procedure No. The parts list information including is extracted. Then, the determination unit 10F11 compares the difference between the first component number and the second component number with a second threshold value included in the extracted component list information, and determines whether the difference is equal to or smaller than the second threshold value ( Step S707).

  If the difference is not equal to or smaller than the second threshold value in step S707, that is, if the difference is greater than the second threshold value, the HMD 10 selects an error notification by referring to the notification content database 240 by the determination unit 10F11 (step S708). Then, the process proceeds to step S711 described later. At this time, the determination unit 10F11 may hold the component ID of the selected component and the content of the selected notification in association with each other.

  In step S707, when the difference is equal to or smaller than the second threshold, the determination unit 10F11 determines whether the difference is equal to or smaller than the first threshold (step S709).

  If the difference is not equal to or smaller than the first threshold value in step S709, that is, if the difference is larger than the first threshold value, the determination unit 10F11 selects a warning notification with reference to the notification content database 240 (step S710), which will be described later. Proceed to step S711. At this time, the determination unit 10F11 may hold the component ID of the selected component and the content of the selected notification in association with each other.

  Subsequently, the HMD 10 determines whether or not processing has been performed for all the component IDs associated with the procedure in which the work content information is displayed (step S711). If it is determined in step S711 that processing has not been performed for all component IDs, the HMD 10 selects the next component type (step S712) and returns to step S705.

  In step S711, when processing has been performed for all component IDs, the HMD 10 creates maintenance information to be passed to the display control unit 10F1 by the content creation unit 10F8 (step S713).

  Hereinafter, the processing of the content creation unit 10F8 will be described. First, a case where error notification is selected will be described.

  The content creation unit 10F8 refers to the notification content database 240 and refers to the procedure No. of the designated work content. The notification contents information including is extracted. Then, the content creation unit 10F8 creates maintenance information including the error notification content included in the notification content information and the image output as the analysis result of step S706. The content creation unit 10F8 passes this maintenance information to the display control unit 10F1.

  Next, a case where warning notification is selected will be described.

  The content creation unit 10F8 refers to the notification content database 240. Next, the content creation unit 10F8 sets the procedure No. of the designated work content. The notification contents information including is extracted. Then, the content creation unit 10F8 associates the notification content of the warning included in the notification content information with the image output as the analysis result in step S706.

  Also, the content creation unit 10F8 refers to the work content database 210, and specifies the procedure number of the designated work content. Get message of work contents information including.

  Then, the content creation unit 10F8 creates maintenance information including an image output as an analysis result, a warning notification content, and a message included in the work content information, and passes the maintenance information to the display control unit 10F1.

  When the message included in the work content information includes a symbol “$” or the like indicating a variable, a message in which the second number of parts is substituted for this variable may be included in the maintenance information.

  Next, the processing of the content creation unit 10F8 when the difference between the first component number and the second component number is equal to or less than the first threshold value will be described.

  In this case, the content creation unit 10F8 displays the image output as the analysis result of step S706 and the procedure number No. of the designated work content. Maintenance information including a message of work content information including “” may be created and passed to the display control unit 10F1. In addition, the content creation unit 10F8 has a procedure No. of the designated work content. The work content information including may be transferred to the display control unit 10F1 as maintenance information as it is.

  In addition, when there are a plurality of components handled in the procedure, the determination unit 10F11 of the present embodiment holds the content of the selected notification for each component ID, and displays the content of the notification for each component. Maintenance information may be created.

  Subsequently, the HMD 10 causes the display control unit 10F1 to display the maintenance information passed from the content creation unit 10F8 on the display device 10H1 (step S714).

  Subsequently, the HMD 10 determines whether or not the next procedure has been selected (step S715).

  When the next procedure is selected in step S715, the HMD 10 refers to the work content information of the next procedure (step S716) and returns to step S703. At this time, the HMD 10 may display the work content information of the next procedure in accordance with the operation of the worker.

  If the next procedure is not selected in step S715, the HMD 10 ends the process. The case where the next procedure is not selected is, for example, the case where all procedures are completed.

  Hereinafter, the processing of the HMD 10 of the present embodiment will be specifically described with reference to FIGS.

  FIG. 8 is a first diagram illustrating maintenance using the information processing apparatus according to the first embodiment. Hereinafter, as shown in the figure, an example in which the worker 1 who is a user of the HMD 10 wears the HMD 10 and performs maintenance will be described.

  In this example, the worker 1 maintains an MFP (Multifunction Peripheral) 2 that is an example of the apparatus. That is, the worker 1 attaches / detaches parts such as screws to / from the MFP 2 in each procedure during maintenance of the MFP 2. In other words, in the following example, the parts handled in the maintenance are screws, and the number of parts is the number of screws. In the following example, it is assumed that the MFP model is “XX” and the work content performed by the worker 1 is “screw replacement”.

  In the present embodiment, the worker 1 wears the HMD 10 and inputs the model “XX” of the MFP 2 to be maintained and the work content “screw replacement”. Then, the display control unit 10F1 refers to the work content database 210, extracts work content information corresponding to the work content “screw replacement” of the input model “XX”, Are displayed on the display device 10H1 in order from the work content information of the younger procedure.

  In FIG. 1 shows an example in which the work content information 210-1 is displayed (see FIG. 3).

  At this time, the display control unit 10F1 displays the work content information 210-1 as the procedure image IMG1 to the worker 1. The procedure image IMG1 includes an image 81 indicated by a still image file name included in the work content information 210-1, and a message 82.

  In the message included in the work content information 210-1, “... Remove $ 1 screw” is stated. Therefore, the display control unit 10F1 reads the work content and procedure No. from the parts list database 230. The part list information 230-1 corresponding to the part number 230-1 is extracted, and the first part number “6” in the part list information 230-1 is substituted into the variable “$ 1” in the message to obtain work content information (see FIG. 5) ).

  When the procedure image IMG1 is displayed, the worker 1 can grasp the contents of the procedure in the maintenance. Specifically, the procedure image IMG1 shows the contents of the “unscrew” procedure, the position of the target screw, the number of screws removed by the procedure, and the like. Therefore, when the procedure image IMG1 is displayed by the display control unit 10F1, the worker 1 can perform this procedure for the MFP 2 and can easily perform maintenance.

  Note that the maintenance target device is not limited to the MFP 2. The apparatus to be maintained may be office equipment other than the MFP.

  When the HMD 10 is a so-called immersive type, the MFP 2 may be a so-called simulation model or the like instead of an actual machine.

  FIG. 9 is a second diagram for explaining maintenance using the information processing apparatus of the first embodiment. FIG. 9A is a diagram illustrating an example in which work content information is displayed. FIG. 9B is a diagram illustrating an example of an image of an analysis result by the analysis unit 10F10. FIG. 9C is a first diagram illustrating a display example of maintenance information. FIG. 9D is a second diagram illustrating a display example of maintenance information.

  FIG. 9 illustrates an example in which the work content “screw replacement” is performed in the maintenance of the MFP 2 of the model “XX” as in the example of FIG. 8.

  First, the display control unit 10F1 displays the procedure image IMG1 as shown in FIG.

  At this time, the component number specifying unit 10F7 specifies the components included in the procedure image IMG1 and the number of the components (first component number).

  The procedure image IMG1 includes a procedure No. 1, the number-of-parts specifying unit 10 </ b> F <b> 7 reads the procedure number “Screw Replacement” from the parts list information 230-1. The part handled at 1 is specified as the part with the part ID “pid1”, and the first part number is specified as “6”.

  Next, the analysis unit 10F10 specifies the second number of parts.

  In the HDM 10, the photographing unit 10 </ b> F <b> 9 photographs the MFP 2 that is the object of maintenance by the operation of the worker 1 and generates image data of the MFP 2.

  When the image of the MFP 2 is captured by the imaging unit 10F9 and the analysis result image 91 is generated by the analysis unit 10F10, the content creation unit 10F8 generates an image IMG2 including the image 91 as illustrated in FIG. Generate. The image IMG2 is an image showing a part or the whole of the device ahead of the operator's line of sight.

  In the example shown in FIGS. 9A and 9B, the number of screws SC is different. Specifically, the number of first components shown in FIG. 9A is “6”. On the other hand, the second number of parts shown in FIG. 9B is “4”. Therefore, the difference between the number of first components and the number of second components is “2”.

  In the present embodiment, when the determination unit 10F11 compares the first component number with the second component number and obtains the difference between the first component number and the second component number, the difference and the component list information 230- 1 is compared with the first and second thresholds included in the first.

  The first threshold included in the parts list information 230-1 is “1”, and the second threshold is “3”.

  The difference “2” between FIG. 9A and FIG. 9B is larger than the first threshold and smaller than the second threshold. Therefore, the content creation unit 10F8 refers to the notification content database 240 and determines the procedure number “Screw replacement” of the work content “No. The warning notification content 83 of the first notification content information 240-1 is associated with the image 81. In addition, the procedure No. The message of the work content information 210-1 of 1 includes the variable “$ 1”. Therefore, the content creation unit 10F8 creates a message 82A in which the second component number “4” is substituted for the variable “$ 1” of the message of the work content information 210-1, and associates it with the image 81.

  Then, the content creation unit 10F8 creates an image IMG3 as maintenance information including the image 81, the notification content 83, and the message 82A, passes it to the display control unit 10F1, and displays it on the display device 10H1.

  FIG. 9C shows an example in which an image IMG3 is displayed. The image IMG3 includes an image 81, a message 82A, and notification contents 83. The notification content 83 shown in FIG. 9C is an icon image indicating a warning.

  According to the present embodiment, by displaying the notification content 83, it is possible to present to the worker that the state is different from the image IMG2 indicating the work content information 210-1.

  In the present embodiment, for example, when the notification content 83 is selected by an operator's operation, the display control unit 10F1 may display warning information. The warning information may be information notifying that a part has a defect. Details of the warning information will be described later.

  FIG. 9D illustrates an example in which an image IMG4 is displayed as maintenance information when the difference between the first component number and the second component number is greater than the second threshold.

  In this case, the content creation unit 10F8 refers to the notification content database 240, and performs the procedure No. The notification content 84 of the error in the notification content information 240-1 is associated with the image 81. In the image IMG4, information 85 indicating the missing part is also displayed. In the example of FIG. 9D, the component ID “pid_1” of the screw SC is displayed as the component ID of the missing component. Further, in the example of FIG. 9D, an error icon 86 is displayed.

  In the example of FIG. 9, the case where one part is handled in the procedure has been described. However, when a plurality of parts are handled, an error or warning notification may be displayed as maintenance information for each part.

  Next, warning information will be described with reference to FIG. FIG. 10 is a diagram illustrating a display example of warning information.

  An image 101 illustrated in FIG. 10 is an example of a screen displayed on the display device 10H1 when the notification content 83 in FIG. 9C is selected.

  The image 101 may include, as warning information, a message indicating the difference between the first component number and the second component number, and the component IDs of components having different numbers. The warning information may further display information such as “the number of parts may vary depending on the lot of the apparatus”.

  When information such as “the number of parts may differ depending on the lot of the apparatus” is further displayed, the storage control unit 10F5 displays data indicating whether the number of parts changes when the lot of the apparatus is different. It may be memorized. Based on this data, the determination unit 10F11 can determine whether or not to display information such as “the number of parts may differ depending on the lot of the apparatus”.

  In the present embodiment, the display control unit 10F1 may display an “OK” icon or the like when the difference between the first component number and the second component number is equal to or less than the first threshold value.

  FIG. 11 is a diagram illustrating a display example of the “OK” icon. As illustrated, the display control unit 10F1 further displays an “OK” icon 111 on the procedure image IMG1 illustrated in FIG. This case includes the case where there is no difference between the first component number and the second component number, that is, the first component number is equal to the second component number.

  The “OK” icon 111 may also be displayed when the difference between the first component number and the second component number is equal to or smaller than the first threshold value.

  Therefore, for example, the procedure No. In the case of 1, even if there is a difference between the first component number and the second component number, if the difference is one, the image shown in FIG. 11 is displayed.

  Further, when the difference between the first component number and the second component number is larger than the first threshold value, an image IMG3 shown in FIG. 9C is displayed.

  When a value equal to or greater than “1” is input to the first threshold as in the component ID “pid_1” in FIG. 5, for example, the number of components varies depending on the lot even if the same model is used. It is. In addition, when a value greater than or equal to “1” is input to the first threshold value, even if there is a component that has been forgotten to be installed in the previous maintenance, and there is no problem even if there is no component that has been omitted, etc. Good. In these cases, even if there is a difference, the determination unit 10F11 may determine that the normal state and display the “OK” icon or the like illustrated in FIG.

  When information such as “the number of parts may differ depending on the lot of the apparatus” is further displayed, the storage control unit 10F5 displays data indicating whether the number of parts changes when the lot of the apparatus is different. It may be memorized. Based on this data, the determination unit 10F11 can determine whether or not to display information such as “the number of parts may differ depending on the lot of the apparatus”.

  As described above, by comparing the number of first components and the number of second components, the determination unit 10F11 can determine the state of the device indicated by a predetermined procedure in the maintenance of the device and the state of the device to be maintained. It can be detected that there is a difference. Next, the content creation unit 10F8 creates maintenance information based on the detected difference. The display control unit 10F1 displays the maintenance information created by the content creation unit 10F8.

  For example, when a difference is detected, the content creation unit 10F8 creates maintenance information corresponding to the difference in the number of parts. If it does in this way, display control part 10F1 can display maintenance information according to the difference in the number of parts.

  Further, there is a method of preparing contents corresponding to each difference, assuming that there is a difference between the state of the apparatus indicated by the predetermined procedure and the state of the apparatus to be maintained. In this method, content is prepared for each difference, and thus the data capacity often increases. On the other hand, in this embodiment, since maintenance information is created according to the difference, the data capacity stored by the storage control unit 10F5 can be reduced.

  For example, even if a manual, that is, information on maintenance is prepared for each model, the number of parts may differ depending on the lot. In this case, if the same model is used, the maintenance work is the same, but the number of parts differs depending on the lot, so it is often necessary to facilitate manuals for each lot. Therefore, since there are many types of manuals, the data capacity may increase. Even in such a case, in the present embodiment, it is not necessary to prepare content for each lot.

  Further, according to the present embodiment, the maintenance information corresponding to the difference between the state of the apparatus assumed in advance in the manual or the like and the state of the apparatus on which the operator is working is displayed. Therefore, according to the present embodiment, it is possible to display maintenance information corresponding to the current state of the apparatus. Therefore, according to the present embodiment, maintenance work can be supported based on the state of the apparatus indicated in the work content information created in advance and the state of the actual apparatus that is the object of maintenance.

  Furthermore, according to the present embodiment, when there is a difference between the state of the device indicated in the work content information created in advance and the state of the actual device to be maintained, the influence caused by this difference The degree can be presented to the worker.

  Therefore, according to the present embodiment, even if there is a difference between the state of the apparatus indicated in the work content information created in advance and the state of the actual apparatus that is the object of maintenance, The degree of influence can be easily grasped. In other words, according to the present embodiment, the operator can grasp the difference between the state of the device indicated in the manual or the like and the state of the device in front of the worker who is going to perform maintenance. it can. Furthermore, according to the present embodiment, it is possible to notify the worker whether the influence of the difference is negligible or not.

  In the present embodiment, the example in which the component is removed from the apparatus in the procedure in the maintenance work has been described. However, the present invention is not limited to this. This embodiment can also be applied in a procedure for attaching a part once removed to an apparatus.

  In this case, the difference between the number of parts associated with the work content and the procedure and the number of parts identified from the analysis result of the image data of the device taken during or after the attachment of the parts And the same processing as described above may be performed.

  In this way, the worker can confirm whether or not a significant difference has occurred after the parts are attached.

Second Embodiment
In the second embodiment, for example, an HMD having the same hardware configuration and functional configuration as in the first embodiment is used. The second embodiment is different from the first embodiment in the parts list database compared to the first embodiment.

  FIG. 12 is a diagram illustrating an example of a parts list database according to the second embodiment. The parts list database 230A of this embodiment is provided for each work content. Also. In the component list database 230A, the first component number for each lot is associated with the component ID associated with each procedure.

  The parts list database 230A includes a procedure No. as an information item. , Lot, component ID, first component number, warning threshold (first threshold), and error threshold (second threshold). The item “lot” indicates the number of parts.

  In the example of FIG. 12, for example, the procedure No. The value of the item “lot” is the same for the component with the component ID “pid_1” handled by 1 and the component with the component ID “pid_2”. Therefore, even if these parts are in different lots, the procedure No. “ The first part number in 1 indicates the same thing.

  In the example of FIG. 12, for example, the procedure No. It can be seen that the part number “pid — 3” handled at 2 has a different number of first parts for the lot “A” and the lot “B”.

  Depending on a lot to be manufactured, the apparatus may change the number of parts, the type of parts, or both. For example, the number of parts may increase or decrease due to improvements in the apparatus, changes in specifications, and destinations.

  In the present embodiment, by referring to this parts list database 230A, for each lot of parts handled in the work content procedure, the state of the device indicated in the work content information created in advance and the actual maintenance target It is possible to detect a difference in the state of the device.

  Next, the operation of the HDM 10 of this embodiment will be described with reference to FIG.

  FIG. 13 is a flowchart illustrating an example of overall processing performed by the information processing apparatus according to the second embodiment.

  The processing from step S1301 to step S1304 in FIG. 13 is the same as the processing from step S701 to step S704 in FIG.

  In step S1304, when the first part is selected, the HMD 10 uses the determination unit 10F11 to refer to the part list database 230A and determine whether the first part number is different for each lot corresponding to the part ID. (Step S1305).

  Hereinafter, the process of step S1305 will be specifically described. In the parts list database 230A, for example, the procedure No. In the case of 1, the part ID of the handled part is “pid_1”, and the corresponding lot is “common” (see FIG. 12). Therefore, in this case, the determination unit 10F11 determines that the first part number for each lot corresponding to the part ID is common.

  Further, in the parts list database 230A, for example, the procedure No. 2, the part ID of the handled part is “pid — 3”, the corresponding lots are “A” and “B”, and the first part numbers are “6” and “3”, respectively. (See FIG. 12). Therefore, in this case, the determination unit 10F11 determines that the number of first components is different for each lot corresponding to the component ID.

  In step S1305, if the number of first parts for each lot is not different, that is, if the number of first parts for each lot is the same, the HMD 10 proceeds to step S1307 to be described later.

  In step S1305, when the number of first parts differs for each lot, the HMD 10 specifies the first lot by the part number specifying unit 10F7 (step S1306), and proceeds to step S1307.

  The processing from step S1307 to step S1312 is the same as the processing from step S705 to step S710 in FIG.

  Subsequently, the HMD 10 determines whether or not the processing in steps S1307 to S1312 has been executed for all the first component numbers associated with the selected component ID in the component list database 230A (step S1313).

  In step S1313, when all the first component numbers have not been processed, the HMD 10 selects the next lot (step S1314) and the component number specifying unit 10F7 returns to step S1307.

  If all the first component numbers have been processed in step S1313, the HMD 10 proceeds to step S1314.

  The processing from step S1315 to step S1319 is the same as the processing from step S711 to step S715 of FIG.

  In this embodiment, in step S1310 and 1312, the content of the selected notification may be stored in association with the component ID and the lot.

  In this embodiment, in step S1316, the content of the selected notification may be displayed for each part ID and lot.

  FIG. 14 is a diagram illustrating a display example of maintenance information in the second embodiment.

  An image (screen) 141 shown in FIG. 14 is an example of an image displayed as maintenance information. In the image 141, an icon 142 for notifying a warning, information 143 for identifying a lot in which the difference between the number of first parts and the number of second parts is greater than a first threshold, and information 144 indicating a manufacturing number of the identified lot , Is included.

  In this case, for example, except for lot B, the difference between the number of first parts and the number of second parts is equal to or less than the first threshold, and in the case of lot B, the difference between the number of first parts and the number of second parts is greater than the first threshold. You can see that

  In this embodiment, in this way, by notifying an error or warning for each lot, if the lot of the parts of the apparatus that will be maintained from now is other than lot B, the work is performed according to the work content information. Can be easily grasped.

  As described above, when the data shown in the above (Table 4) is used, the display control unit 10F1 (FIG. 2) displays information on the lot when the number of parts of the apparatus differs depending on the lot. Can do. That is, even if the number of parts of the apparatus varies depending on the lot, the display control unit 10F1 can display maintenance information that matches the state of the apparatus that is the object of maintenance.

<Modification>
In the embodiment according to the present invention, the target component is not limited to a screw. For example, the target component may be a mechanical component such as a spring or a sheet metal. The target part is preferably a part having a large number of parts used in the apparatus, such as a screw.

  The embodiment according to the present invention is not limited to the HMD. For example, it may be realized by an information processing system that combines an imaging device such as a camera and an information processing device such as a PC (Personal Computer). That is, in the information processing system, the imaging device and the information processing device are connected by wire or wirelessly, and an image generated by the imaging device is transmitted to the information processing device. Next, in the information processing system, the information processing apparatus may perform various processes, display a processing result or an image based on the processing result, or input a user operation.

  In the embodiment according to the present invention, a plurality of information processing apparatuses may perform processing or storage in a distributed, parallel, or redundant manner. For example, the embodiment according to the present invention may be realized by an information processing system having an information processing apparatus such as a server and an information processing apparatus such as an HMD connected to the server or the like via a network. Specifically, the server or the like may store various processes and data, while the HMD or the like may display a processing result and an image to the user and input a user operation.

  The embodiment of the present invention may be realized by a program for causing a computer such as an information processing apparatus or an information processing system having one or more information processing apparatuses to execute an information processing method.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims. Or it can be changed.

10 HMD
2 MFP
10F1 Display control unit 10F7 Number-of-parts specification unit 10F8 Content creation unit 10F10 Analysis unit 10F11 Determination unit 10F12 Database storage unit 210 Work content database 220 Component definition database 230, 230A Parts list database 240 Notification content database

Japanese Patent No. 4150651

Claims (10)

An information processing apparatus that supports work in apparatus maintenance,
A storage unit that stores component list information for each maintenance work procedure, and component definition information for identifying components included in the device;
Based on the parts list information, a part to be handled that is handled in the work procedure, and a specifying unit that specifies the number of first parts that is the number of parts to be worked,
A photographing unit for generating image data of an image obtained by photographing the device;
Based on the part definition information, an analysis unit that identifies the part to be worked on in the image data and a second part number that is the number of parts to be worked on;
A display control unit for displaying maintenance information based on a result of comparing the first part number and the second part number;
An information processing apparatus comprising: The maintenance information is
An image output as an analysis result of the image data;
The information processing apparatus according to claim 1, further comprising: a message according to a comparison result between the first component number and the second component number. The display control unit
The information processing apparatus according to claim 2, wherein when there is a difference between the first component number and the second component number, maintenance information corresponding to the difference between the first component number and the second component number is displayed. . A determination unit that determines whether a difference between the first component number and the second component number is greater than a first threshold;
The display control unit
The information processing apparatus according to claim 3, wherein maintenance information including warning information is displayed when the difference is greater than the first threshold.   The information processing apparatus according to claim 4, wherein the first number of parts and the first threshold value are stored for each lot of the apparatus. The determination unit determines whether or not a difference between the first component number and the second component number is greater than a second threshold;
The display control unit
6. The information processing apparatus according to claim 4, wherein maintenance information including error information is displayed when the difference is greater than the second threshold value.   The information processing apparatus according to claim 6, wherein the first number of parts and the second threshold value are stored for each lot of the apparatus. An information processing system having one or more information processing devices for supporting work in device maintenance,
A storage unit that stores component list information for each maintenance work procedure, and component definition information for identifying components included in the device;
Based on the parts list information, a part to be handled that is handled in the work procedure, and a specifying unit that specifies the number of first parts that is the number of parts to be worked,
A photographing unit for generating image data of an image obtained by photographing the device;
Based on the part definition information, an analysis unit that identifies the part to be worked on in the image data and a second part number that is the number of parts to be worked on;
A display control unit for displaying maintenance information based on a result of comparing the first part number and the second part number;
An information processing system comprising: An information processing method by an information processing device that supports work in maintenance of the device, wherein the information processing device includes:
Based on the parts list information for each maintenance work procedure stored in the storage unit, the work target parts to be handled in the work procedure and the first part number that is the number of the work target parts are identified,
Generating image data of an image taken of the device;
Based on the part definition information for specifying the part of the device, which is stored in the storage unit, the work target part in the image data, the second part number that is the number of the work target part, Identify
An information processing method for displaying maintenance information based on a result of comparing the first number of parts and the second number of parts. A program executed by an information processing device that supports work in maintenance of the device,
Based on the parts list information for each work procedure of the maintenance stored in the storage unit, a process for specifying a work target part to be handled in the work procedure and a first part number that is the number of the work target parts; ,
Processing for generating image data of an image obtained by photographing the device;
Based on the part definition information for specifying the part of the device, which is stored in the storage unit, the work target part in the image data, the second part number that is the number of the work target part, Processing to identify
A program that causes the information processing apparatus to execute a process of displaying maintenance information based on a result of comparing the first number of parts and the second number of parts.

Images