JP7496226B2 - Inspection support system, inspection support device, and inspection support method - Google Patents

Description

The present invention relates to an inspection support system, an inspection support device, and an inspection support method that support inspection work performed while moving between multiple locations.

Many places are now digitalizing and managing maintenance work procedures. In particular, it is common for the work performed by experienced inspectors to be recorded on video, and for unskilled inspectors to use the recorded video as a manual when performing inspection work.

As part of such efforts, there are techniques described in Patent Documents 1 and 2.
Patent Document 1 discloses an on-site work implementation status management device and management method, which "has a work data acquisition unit 101 that acquires work process information from sensors installed on the worker and the equipment used and from the equipment itself, a work location information acquisition unit 102 that acquires location information on the location where the work is performed, a work time information acquisition unit 103 that acquires time information on the time when the work was performed, an imaging unit 104 that acquires video information related to the work, and a work information recording unit 105 that collectively records and manages the information acquired by the work data acquisition unit, work location information acquisition unit, work time information acquisition unit and imaging unit on a task by task basis" (see abstract).

Patent Document 2 also discloses a production preparation activity program and its recording medium that "displays video data of actual workers working on an existing production line. Displays an element work screen 200 for inputting element work information relating to each of a plurality of element works of one work process contained in the work video data. Outputs element work data based on the input element work information. Executes mass production prototypes in a virtual space based on the element work data, and outputs virtual video data of the mass production prototype. Makes the virtual video data and work video data commonly available to a plurality of functional modules that each execute independent production preparation activity processes" (see abstract).

JP 2008-84068 A JP 2011-150569 A

As mentioned above, the work performed by an experienced inspector is recorded as a video, and an unskilled inspector performs the inspection work using the recorded video as a manual. In this case, if the experienced inspector records a series of videos from the start to the end of the inspection work, the work of the experienced inspector can be reduced. For example, when inspecting multiple locations in one inspection work, by recording a series of videos from the start to the end of the inspection work, the experienced inspector does not have to turn the recording on and off each time, and the burden on the experienced inspector can be reduced.

However, when using this method, parts that are not necessary for the inspection work are recorded in the video. Therefore, it is necessary to extract the parts necessary for the inspection work from the recorded video.

The technology disclosed in Patent Document 1 does not describe the process of extracting necessary parts from recorded video. Therefore, with the technology disclosed in Patent Document 1, it is necessary to turn recording on and off each time a necessary part of the inspection work is reached, which increases the burden on experienced inspectors.

In the technology disclosed in Patent Document 2, the recorded video is edited using a GUI (Graphical User Interface), which is essentially a manual input process. This method requires a lot of editing work and is not efficient.

The present invention was made in light of this background, and its objective is to realize efficient manual creation and inspection work.

In order to solve the above-mentioned problems, the present invention provides an inspection support system that supports inspection work performed while moving between multiple locations, and is characterized in that it comprises a first terminal unit held by a first person and having a first imaging unit that captures video and a first position acquisition unit that acquires position information; an extraction unit that generates an extracted video by extracting from a first video captured by the first imaging unit a range of video sandwiched between start information, which is specified information in the first video or specified information input from outside at the time the first video is being captured, and end information, which is specified information in the first video or specified information input from outside at the time the first video is being captured; and an output unit that, when the extracted video is captured, outputs first position information, which is the position information acquired by the first position acquisition unit, in association with the extracted video.
Other solutions will be described in the embodiments as appropriate.

The present invention makes it possible to realize efficient manual creation and inspection work.

1 is a diagram showing a configuration of an inspection support system according to an embodiment of the present invention; FIG. 2 is a functional block diagram of a head-mounted device. FIG. 2 is a functional block diagram of a moving image recording server. FIG. 2 is a functional block diagram of a work processing server. 11 is a diagram showing an example of work procedure video information stored in a work procedure video DB. FIG. FIG. 2 is a diagram showing an example of a checklist stored in a checklist DB. 10 is a flowchart showing a procedure of a work video creation process in the first embodiment. 10 is a flowchart showing a procedure for creating an abnormality operation video in the first embodiment. 5 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector in accordance with a created manual video in the first embodiment. FIG. 4 is a diagram showing an inspection operation in the first embodiment. 13 is a flowchart showing a procedure of a work video creation process in the second embodiment. 13 is a flowchart showing a procedure for an abnormality operation video creation process in the second embodiment. 13 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector according to a created manual video in the second embodiment. FIG. 13 is a diagram showing an example of work procedure video information in the third embodiment. 13 is a flowchart showing a procedure of a work video creation process in the third embodiment. 13 is a flowchart showing a procedure for an abnormality operation video creation process in the third embodiment. 13 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector according to a created manual video in the third embodiment. FIG. 23 is a diagram showing an example of work procedure video information in the fourth embodiment. 13 is a flowchart showing a procedure of a work video creation process in the fourth embodiment. 13 is a flowchart showing a procedure for creating an abnormality operation video in the fourth embodiment. 13 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector following a created manual video in the fourth embodiment. FIG. 13 is a functional block diagram of a work processing server according to the fifth embodiment. FIG. 11 is a diagram illustrating an example of a display item switching process. FIG. 13 is a diagram showing a configuration of an inspection support system according to a sixth embodiment. FIG. 23 is a functional block diagram of a moving image recording server in the sixth embodiment. 23 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector according to a created manual video in the sixth embodiment. 13 is a flowchart showing a procedure for a first unskilled inspector video storage process.

Next, a form for implementing the present invention (referred to as an "embodiment") will be described in detail with reference to the drawings as appropriate.

[First embodiment]
<System Configuration>
FIG. 1 is a diagram showing a configuration of an inspection support system 1 according to this embodiment.
1, the inspection support system 1 includes a head-mounted device 100, a video recording server 200, a work processing server 300, a work procedure video DB 410, and a checklist DB 430. Details of each device and the processes performed by each device will be described later. The video recording server 200 and the work processing server 300 are collectively referred to as the inspection support server.

The head-mounted device 100 is a glasses-like device that an inspector wears on his/her head. As described later in FIG. 2, the head-mounted device 100 is equipped with a camera 131, a microphone 134, a speaker 133, etc., and is capable of capturing video, acquiring audio, and outputting audio. As shown in FIG. 1, the glasses on the head-mounted device 100 serve as a display 132. By wearing such a head-mounted device 100, the information displayed on the display 132 can be viewed while using both hands. The display 132 is configured so that the actual scenery and the displayed information can be viewed simultaneously.

Here, an experienced inspector P1 wears a head-mounted device 100a (first terminal unit), and an unskilled inspector P2 wears a head-mounted device 100b (second terminal unit).

The video recording server 200 extracts necessary parts from the video captured by the head-mounted device 100a worn by the skilled inspector P1, and stores the extracted video (extracted video) in the work procedure video DB 410. In this way, a manual video is created for use by the unskilled inspector P2 when performing inspection work. Note that the manual video includes a work video, which is a video of the inspection work performed by the skilled inspector P1 when no abnormality is detected in the inspection equipment, and an abnormality work video, which is a video of the work performed when an abnormality is detected in the inspection equipment.

When the unskilled inspector P2 performs an inspection task, the work processing server 300 acquires a video corresponding to the inspection task from the work procedure video DB 410 based on the position information (second position information) of the head-mounted device 100b worn by the unskilled inspector P2. The work processing server 300 then outputs the acquired video to the display 132 of the head-mounted device 100b. In this way, the unskilled inspector P2 can proceed with the inspection task while checking the inspection procedure performed by the skilled inspector P1 through the video. Furthermore, when the work processing server 300 detects information that the inspection has been completed, it performs a process of checking the corresponding inspection item in the checklist 431 (see FIG. 6) stored in the checklist DB 430.

The work procedure video DB 410 stores videos of the inspection work performed by the skilled inspector P1 as manual videos.
The checklist DB 430 stores a checklist 431 (see FIG. 6) for inspection work.
The work procedure video DB 410 and the checklist DB 430 may be integrated. In addition, the work procedure video DB 410 and the checklist DB 430 may be configured not to be included in the inspection support system 1 by being installed as a cloud or the like.

(Head-mounted device 100)
FIG. 2 is a functional block diagram of the head mounted device 100.
The head mounted device 100 includes a memory 110, a CPU (Central Processing Unit) 121, and a transmission/reception device 122. The head mounted device 100 also includes a camera 131, a display 132, a speaker 133, a microphone 134, a sensor 135, and a position acquisition device 136.
Here, the sensor 135 is a temperature sensor, an odor sensor, etc. The temperature sensor may be a thermograph or the like.
The position acquisition device 136 is a device that identifies the position of the head mounted device 100. Methods for identifying the position using the position acquisition device 136 include a global positioning system (GPS), a barcode, Wi-Fi information, and simultaneous localization and mapping (SLAM).

The memory 110 is a read only memory (ROM). When the CPU 121 executes a program stored in the memory 110, a transmission/reception unit 111 and an output processing unit 112 are realized.
The transmitting/receiving unit 111 performs processes of transmitting moving images and position information to the moving image recording server 200 and transmitting/receiving various information to/from the operation processing server 300 .
The output processing unit 112 performs a process of outputting information to the display 132 .

(Movie recording server 200)
FIG. 3 is a functional block diagram of the moving image recording server 200.
The video recording server 200 (constituting the inspection support server) includes a memory 210, a CPU 221, and a storage device 222 such as a hard disk (HD). The video recording server 200 also includes an input device 231 such as a keyboard or a mouse, a display 232, and a transmission/reception device 233.

The program stored in the storage device 222 is loaded into the memory 210. Then, the CPU 221 executes the loaded program to realize the registration processing unit 211, and the extraction unit 212 and storage processing unit 213 that constitute the registration processing unit 211.

The extraction unit 212 extracts the parts necessary as a manual from the video captured by the head-mounted device 100a worn by the skilled inspector P1 and sent to the extraction unit 212. The extraction method will be described later.
The storage processing unit 213 stores the video extracted by the extraction unit 212 in the work procedure video DB 410 .

(Work processing server 300)
FIG. 4 is a functional block diagram of the work processing server 300.
The work processing server 300 (constituting the inspection support server) includes a memory 310, a CPU 321, and a storage device 322 such as a hard disk (HD). The work processing server 300 also includes an input device 331 such as a keyboard or a mouse, a display 332, and a transmission/reception device 333.

The program stored in the storage device 322 is loaded into the memory 310. Then, the CPU 321 executes the loaded program to realize the inspection processing unit 311, and the position determination unit 312, analysis processing unit 313, acquisition processing unit 314, inspection determination unit 315, and check processing unit 316 that constitute the inspection processing unit 311.

The position determination unit 312 determines whether or not the head mounted device 100b has reached the inspection position based on the position information transmitted from the head mounted device 100b.
The analysis processing unit 313 performs an analysis to determine whether or not an abnormality has occurred in the inspection equipment based on information such as video transmitted from the head-mounted device 100b.
The acquisition processing unit 314 acquires a video corresponding to the currently performed task from the task procedure video DB 410 .

The inspection determination unit 315 determines whether the inspection work has been completed, etc., based on whether predetermined information has been input.
The check processing unit 316 performs processing such as inputting a check into a check list 431 (see FIG. 6) in a check list DB 430 .

(Work procedure video DB410)
FIG. 5 is a diagram showing an example of work procedure video information 411 stored in the work procedure video DB 410. As shown in FIG.
The work procedure video information 411 has columns for ID, location information, work ID, video file, start time, end time, normal/abnormal, and device information.
The ID is a number that is automatically assigned by the moving image recording server 200 .
The location information is information relating to the location where the corresponding inspection work is to be performed.
The work ID is an ID related to the inspection work. The work ID is automatically assigned by the video recording server 200. Note that while the ID is a number assigned regardless of the work, it is preferable that the work ID be assigned so that it can be grouped by inspection work or inspection equipment. For example, as shown in the second embodiment, when multiple inspection works are performed using the same inspection equipment, it is preferable that the respective work IDs be assigned as "1-1", "1-2", "1-3", etc.

The video file field stores the work video (manual video) of the corresponding inspection work. Note that the video file field may store the video file of the work video itself, or may store the path where the video file is stored.
The start time is the time when the work video starts, and the end time is the time when the work video ends. The start time and the end time may be the playback times of the video before extraction, or may be the actual times.

The normal/abnormal column stores information on whether the inspection work was completed normally or whether an abnormality was detected.
The equipment information field stores information about the equipment to be inspected for the corresponding inspection work. The equipment information is stored by the video recording server 200 based on, for example, location information and installation information of the equipment to be inspected.

(Checklist DB430)
FIG. 6 is a diagram showing an example of the checklist 431 stored in the checklist DB 430. As shown in FIG.
The checklist 431 has columns for data indicating classification of location information, work ID, normal/abnormal, completion check, abnormality measures required, and check method.
The location information and job ID fields store information about the location where the inspection job is being performed and the job ID.
The normal/abnormal column contains information indicating whether no abnormality was detected in the inspection equipment (normal) or whether an abnormality was detected (abnormal) as a result of the inspection work.
In the completion check field, a check is entered when the inspection work is completed.
In the "measures required for abnormality" column, a check mark is entered if the abnormality-requiring measures described below are required.

At least one of the items from reading the value, operation/non-operation, and presence/absence of abnormality is entered in the data column representing the classification of the check method. The value in the reading of the value is the output value of the inspection device for determining that the inspection result is abnormal, the output value of the inspection device during inspection, etc. are input. The output value of the inspection device for determining that the inspection result is abnormal is a value input in advance via the input device 231 of the video recording server 200, etc. The output value of the inspection device during inspection is input from an input device (not shown) of the head-mounted device 100, or input based on the voice input to the microphone 134. For example, when the unskilled inspector P2 utters the voice "output value is 100", the value "100" is input in the data column representing the classification of the corresponding check method. Alternatively, the output value of the inspection device during inspection may be input as the equivalent value of the meter in the image captured by the camera 131. In this case, the check processing unit 316 extracts the value of the meter in the image by a method such as recognizing the value of the meter in the image by machine learning.

Operating/non-operating is information on whether the inspection work was performed while the inspection equipment was operating or not operating. The determination of whether the inspection work was performed while the inspection equipment was operating or not operating is made, for example, by the check processing unit 316 referring to the operation schedule and inspection time of the inspection equipment.

The presence or absence of an abnormality is information on whether or not an abnormality was detected as a result of the inspection work. The presence or absence of an abnormality can also be substituted with the normal/abnormal column.

<Flowchart>
(Work video creation process)
Fig. 7 is a flowchart showing the procedure of the work video creation process in the first embodiment. In the flowchart of this embodiment, the movements of a person are shown by dashed squares. Fig. 7 shows a flowchart in the case where no abnormality is detected in the inspection equipment.
First, the skilled inspector P1 puts on the head-mounted device 100a and starts the inspection work (S101).
The camera 131 of the head mounted device 100a captures a video (S102). Furthermore, the position acquisition device 136 acquires position information of the head mounted device 100a (S103).
Thereafter, the transmitting/receiving unit 111 of the head mounted device 100a transmits the acquired video and the position information in association with each other to the video recording server 200. Note that the video and the position information may be continuously transmitted to the video recording server 200 as they are captured, or may be temporarily stored in a RAM (not shown) of the head mounted device 100a and then transmitted to the video recording server 200 together.

The extraction unit 212 of the video recording server 200 searches for start information and end information from the acquired video (S111). The start information and end information are predetermined images in the video. For example, if there is an image in the video of the skilled inspector P1 holding up his index finger, the extraction unit 212 sets that image as start information. Also, if there is an image in the video of the skilled inspector P1 holding up his index finger and middle finger, the extraction unit 212 sets that image as end information. The process of searching for start information and end information in the video is performed using machine learning, etc.

Next, the extraction unit 212 extracts the moving image in the range between the start information and the end information from the acquired moving image (S112).
Then, the storage processing unit 213 stores the extracted video and the position information corresponding to the video in the work procedure video DB 410 (S113). At this time, "normal" is stored in the normal/abnormal column of the work procedure video information 411 shown in Fig. 5. The video extracted and stored in the work procedure video DB 410 becomes the work video.

Here, the start information is an image of the expert inspector P1 holding up his index finger, and the end information is an image of the expert inspector P1 holding up his index finger and middle finger, but this is not limited to this. Other actions of the expert inspector P1 may be used. Alternatively, for example, if there is an image of a piece of paper with the word "start" written on it, this image may be used as the start information. Similarly, if there is an image of a piece of paper with the word "end" written on it, this image may be used as the end information. Alternatively, the start information may be when the expert inspector P1 nods his head once, and the end information may be when the expert inspector nods his head twice. The head-nodding action can be easily detected by the image moving up and down in the video. Also, when an image of the expert inspector P1 pointing at the inspection device with his index finger is detected, the image may be used as the start information. Also, the start information may be an image of the expert inspector P1 making a circle with his thumb and index finger (a so-called OK sign). Also, the extraction unit 212 may obtain the start information by detecting zooming by the camera 131, and obtain the end information by detecting unzooming by the camera 131.

(Video creation process for abnormal operation)
Fig. 8 is a flowchart showing the procedure of the abnormality occurrence work video creation process in the first embodiment. In Fig. 8, a case where an experienced inspector P1 detects an abnormality in an inspection device will be described.
The processes in steps S201 to S213 differ from the processes shown in FIG.
(A1) In step S201, the start information and end information are changed from those in step S101 in Fig. 7. For example, in step S201, shaking the hand from side to side with the hand open is set as start information when an abnormality is detected. Also, shaking the hand from side to side with the hand open except for the thumb is set as end information when an abnormality is detected.
(A2) The storage destination of the video in step S213 is the work procedure video DB 410, but “abnormal” is stored in the normal/abnormal column of the work procedure video information 411 shown in FIG.
Here, the video extracted and stored in the work procedure video DB 410 becomes the abnormality time work video.
As mentioned above, the work video and the abnormality operation video will be collectively referred to as the manual video as appropriate.

(Inspection work by unskilled inspector P2)
FIG. 9 is a flowchart showing the procedure of an inspection work process performed by the unskilled inspector P2 in accordance with the created manual video in the first embodiment.
First, the unskilled inspector P2 puts on the head-mounted device 100b and starts the inspection work (S301).
Then, the work processing server 300 acquires map information, for example, from the work procedure video DB 410, and transmits it to the head-mounted device 100b. The output processing unit 112 of the head-mounted device 100b outputs an overall map to the display 132 based on the acquired map information (S302). At this time, the output processing unit 112 indicates the direction of the next inspection location with an arrow or the like. The direction of the next inspection location is indicated based on the position information, etc., of the work procedure video information 411 shown in FIG. 5.
The camera 131 of the head-mounted device 100b starts capturing images (S303). The camera 131 continues capturing video during the inspection work. The captured video is transmitted to the work processing server 300 or the video recording server 200.

In step S302, the output processing unit 112 may display on the overall map based on the current location information acquired from the position acquisition device 136. Alternatively, the output processing unit 112 may output the next inspection location to be visited on the overall map.

While the inspection work is being performed, the position acquisition device 136 of the head-mounted device 100b worn by the unskilled inspector P2 starts acquiring position information of its own position (S304). The position acquisition device 136 constantly acquires position information of its own position during the inspection work. In addition, the transmission/reception device 122 continues to transmit the acquired position information to the work processing server 300.
Then, the position determination unit 312 of the work processing server 300 determines whether or not the unskilled inspector P2 (the head-mounted device 100b itself) has reached a position where the inspection work is performed (referred to as an inspection position) (S305). The inspection position is a position within a predetermined distance (nearby) from the position indicated by the position information stored in the work procedure video information 411 shown in FIG. 5.
When the inspection position has not been reached (S305→No), the inspection processing unit 311 returns the process to step S305.

If the inspection position has been reached (S305 → Yes), the head-mounted device 100b transmits the video captured by the camera 131 to the work processing server 300.
Then, the analysis processing unit 313 of the work processing server 300 performs an analysis process to analyze the transmitted video (S306). In the analysis process, the transmitted video is compared with the video stored in the work procedure video DB 410 to determine whether or not an abnormality has occurred. This determination is performed using, for example, machine learning.
Then, the analysis processing unit 313 determines whether or not an abnormality has been detected as a result of the analysis process using the video transmitted from the head mounted device 100b (S307).

If no abnormality is detected (S307→No), the acquisition processing unit 314 acquires a work video associated with the current position information from the work procedure video DB 410 (S311). The work video acquired at this time is a work video of a record in which "normal" is stored in the normal/abnormal column of the work procedure video information 411 shown in Fig. 5. The acquired work video is transmitted to the head-mounted device 100b by the transmission/reception unit 111.
Then, the output processing unit 112 of the head mounted device 100b outputs the transmitted work video to the display 132 (S312).
The unskilled inspector P2 performs the inspection work in accordance with the work video displayed on the display 132 (S313). The work video can be played back in slow motion, for example.

Next, the inspection determination unit 315 determines whether the inspection work has been completed (S314). For example, the unskilled inspector P2 holds his fingers with the thumb extended over the camera 131 of the head-mounted device 100. The inspection determination unit 315 recognizes the image of the fingers with the thumb extended as a completion signal, and determines that the completion signal has been acquired. In other words, the inspection determination unit 315 determines that the inspection work has been completed.

When it is determined that the inspection work is completed (S314→Yes), the check processing unit 316 performs a check process to input a check indicating that the inspection is complete for the currently performed inspection in the checklist 431 shown in FIG. 6 (S315). In step S315, the check processing unit 316 inputs the location information of the displayed work video into the location information in the checklist 431 in FIG. 6. The check processing unit 316 also inputs the work ID of the displayed work video into the work ID column of the checklist 431. The check processing unit 316 then inputs "normal" into the normal/abnormal column of the checklist 431 and inputs a check into the completion check column. The check processing unit 316 also inputs information into the data column representing the classification of the check method in the checklist 431 by the above-mentioned method. The output processing unit 112 may output the checklist 431 into which information has been input to the display 132.

Next, if the inspection work has not been completed (S314→No), the inspection processing unit 311 returns the process to step S314.

If an abnormality is detected in step S307 (S307→Yes), the output processing unit 112 of the head-mounted device 100b performs an alert process (S321). In the alert process, a display is displayed indicating that an abnormality has been detected in the inspection equipment and that abnormality work will now be performed. Alternatively, a voice output may be generated from the speaker 133 indicating that an abnormality has been detected and that abnormality work will now be performed.

The acquisition processing unit 314 acquires a video (abnormal work video) from the work procedure video DB 410 based on the position information (S322). The abnormal work video acquired at this time is a video of a record in which "abnormal" is stored in the normal/abnormal column of the work procedure video information 411 shown in FIG. 5.
When there are multiple corresponding abnormality operation videos, the output processing unit 112 of the head-mounted device 100b outputs thumbnails of the corresponding abnormality operation videos to the display 132. Then, the unskilled inspector P2 may select a thumbnail corresponding to the abnormality operation video to be performed.

The abnormality operation video acquired by the acquisition processing unit 314 is transmitted to the head mounted device 100b.
The output processing unit 112 of the head mounted device 100b outputs the acquired abnormality operation video to the display 132 (S323).
Then, the unskilled inspector P2 performs the abnormality work based on the abnormality work video displayed on the display 132 (S324).

Then, the inspection determination unit 315 determines whether the abnormality work is completed by determining whether a completion signal has been acquired (S325). Here, if a completion signal that is an image of fingers with the thumb extended is detected, the result is "Yes." Also, if a completion signal that is an image of fingers with the thumb and index finger extended is detected, the result is "No."

If the completion signal has not been acquired (S325→No), the inspection processing unit 311 performs an abnormality countermeasure required process of inputting information "abnormality countermeasure required" into the corresponding item in the check list 431 (S326).
In step S326, the check processing unit 316 inputs the position information of the displayed work video into the position information in the checklist 431 of FIG. 6. The check processing unit 316 also inputs the work ID of the displayed work video into the work ID column of the checklist 431. The check processing unit 316 then inputs "abnormal" into the normal/abnormal column of the checklist 431, does not input a check into the completion check column, and inputs a check into the abnormality countermeasure required column. Since no check is input into the completion check column, it is indicated that the inspection work is not completed, and since a check is input into the abnormality countermeasure required column, it is indicated that confirmation and countermeasure work by a person other than the unskilled inspector P2 is required. In other words, information on abnormality countermeasure required is input into the checklist 431. Furthermore, the check processing unit 316 inputs information into the column of data representing the classification of the check method in the checklist 431 by the above-mentioned method.
The output processing unit 112 may output the checklist 431 after the information input has been completed to the display 132 .

In addition, in step S326, for example, a video captured by the camera 131 of the head-mounted device 100b is transmitted to the video recording server 200. The video is sent from the task processing server 300 to the video recording server 200, but may also be sent directly from the head-mounted device 100b to the video recording server 200.

Then, the video recording server 200 stores the video in the work procedure video DB 410. At this time, it is preferable to distinguish the work ID from the video by the experienced inspector P1 to indicate that it is a video stored in step S326. In this way, the abnormal state can be confirmed later. Furthermore, by examining the video of the abnormality, facility management companies and the like can determine the need for updating equipment, and can use this information when creating budgets.

When a completion signal is acquired (S325→Yes), the check processing unit 316 performs a check process to input a check indicating that the inspection is complete for the currently performed inspection in the checklist 431 shown in FIG. 6, and inputs abnormality occurrence information into the checklist 431 (S327). In step S327, the check processing unit 316 inputs the location information of the displayed work video into the location information in the checklist 431 in FIG. 6. The check processing unit 316 also inputs the work ID of the displayed work video into the work ID column of the checklist 431. The check processing unit 316 then inputs "abnormal" into the normal/abnormal column of the checklist 431, and inputs a check into the completion check column. The output processing unit 112 may output the checklist 431 into which information input has been completed to the display 132.

At this time, the video recording server 200 may store the video captured by the camera 131 of the head-mounted device 100b in the work procedure video DB 410. At this time, the work ID may be distinguished from videos taken by the skilled inspector P1 to indicate that the video was stored in step S326.

<Inspection work>
FIG. 10 is a diagram showing an inspection operation in the first embodiment.
In the example of FIG. 10, an inspection of equipment installed in a building B is performed.
10, solid line L1 indicates the work flow of experienced inspector P1. The experienced inspector P1 performs inspection work at points P21 to P23 while moving in the direction of the arrow shown on solid line L1. Note that reference symbol P11 indicates the start point of the inspection work, and reference symbol P12 indicates the end point of the inspection work.

Then, dashed line L2 indicates the work flow of unskilled inspector P2. Unskilled inspector P2 follows the map information displayed in step S302 of FIG. 9 and the displayed arrow toward the first inspection location, reference symbol P21 (arrow of dashed line L2). When unskilled inspector P2 reaches the vicinity of reference symbol P21, a work video is displayed on display 132 of head-mounted device 100b worn by unskilled inspector P2. Unskilled inspector P2 performs the inspection work following the displayed work video.

When the inspection work at P21 is completed, a check is entered in the completion check column of the corresponding record in the checklist 431 of the checklist DB 430.
Then, the direction to the next inspection location, reference symbol P22, is displayed by an arrow on the display 132 of the head-mounted device 100b.
The skilled inspector P1 follows the arrow displayed in step S302 in Fig. 9 to the next inspection location P22 (arrow L2). After that, he performs the same inspection work as at P21. The skilled inspector P1 also performs the same inspection work at P23.

When the map is displayed in step S302 of FIG. 9, an overall map such as that shown in FIG. 10 is displayed.

Effects of the First Embodiment
According to the first embodiment, the manual video required by the unskilled inspector P2 can be automatically generated simply by recording the inspection work performed by the skilled inspector P1 for one day or several days. This reduces the burden of creating the manual video. In addition, the skilled inspector P1 does not need to turn recording on and off every time, which eliminates factors that hinder the work of the skilled inspector P1. This allows for efficient inspection work.

In addition, the work processing server 300 also inputs the checklist 431, which reduces the workload of the unskilled inspector P2. Therefore, the unskilled inspector P2 can concentrate on the inspection work, thereby improving the efficiency and accuracy of the inspection work.

[Second embodiment]
In the second embodiment, a case will be described in which a plurality of inspection operations are performed at one location (one inspection device).
Since each configuration of the inspection support system 1 in the second embodiment is similar to that in the first embodiment, the description thereof will be omitted here.

(Work video creation process)
FIG. 11 is a flowchart showing the procedure of the work video creation process in the second embodiment. In FIG. 11, the same process as in FIG. 7 is assigned the same step number and the description is omitted. When performing multiple inspection tasks at one location (one inspection device), the skilled inspector P1 captures multiple start information and end information during the inspection of one location (one inspection device). In other words, the skilled inspector P1 captures multiple start signals and end signals during the inspection of one location (one inspection device). Whether the inspection starts at the same inspection location or different inspection locations is determined based on the position information.

After step S113, the extraction unit 212 determines whether there is other start information (S121).
If there is no other start information (S121→Yes), the registration processing unit 211 ends the process.

If there is other start information (S121→No), the extraction unit 212 determines, based on the position information, whether or not the start information is start information at a position nearby the moving images extracted so far (S122).
If the information is about the start of a nearby position relative to the moving image thus far (S122→Yes), the extraction unit 212 returns the process to step S111.
If the start information is not at a nearby position for the video so far (S122→No), the process moves to the next inspection location (S123).

(Video creation process for abnormal operation)
FIG. 12 is a flowchart showing the procedure of the abnormality operation video creation process in the second embodiment.
The processing in steps S201 to S223 differs from that in FIG. 11 in (A1) and (A2) described in FIG. 8, and therefore description thereof will be omitted here.

(Inspection work)
Fig. 13 is a flowchart showing the procedure of the inspection work process performed by the unskilled inspector P2 according to the created manual video in the second embodiment. In Fig. 13, the same steps as those in Fig. 9 are given the same step numbers and the description thereof will be omitted.
After step S311, the acquisition processing unit 314 determines whether or not there are multiple work videos associated with nearby position information for the current position in the work procedure video DB 410 (S331). Nearby position information is position information indicating a position within a predetermined distance from the current position. The work video in step S331 is a video of a record in which "normal" is stored in the normal/abnormal column of the work procedure video information 411 in FIG. 5.
If there are not multiple work videos (S331→No), the output processing unit 112 of the head mounted device 100b performs the process of step S312.

If there are multiple work videos (S331→Yes), the output processing unit 112 of the head-mounted device 100b outputs thumbnails of the multiple work videos to the display 132. Then, the unskilled inspector P2 selects one of the displayed thumbnails (S332). After that, the output processing unit 112 performs the process of step S312 for the work video of the selected thumbnail.

After step S322, the acquisition processing unit 314 determines whether or not there are multiple abnormality work videos associated with the neighboring position information of the current position in the work procedure video DB 410 (S333). Here, the abnormality work video in step S333 is a video of a record in which "abnormality" is stored in the normal/abnormal column of the work procedure video information 411 in FIG. 5.
If there are not multiple abnormality operation videos (S333→No), the output processing unit 112 of the head mounted device 100b performs the process of step S323.

If there are multiple abnormality-occurrence work videos (S333→Yes), the output processing unit 112 of the head-mounted device 100b outputs thumbnails of the multiple abnormality-occurrence work videos to the display 132. Then, the unskilled inspector P2 selects one of the displayed thumbnails (S334). After that, the output processing unit 112 performs the process of step S323 for the abnormality-occurrence work video of the selected thumbnail.

According to the second embodiment, even if there are multiple inspection tasks at one location (one inspection device), manual videos can be created for each task.

[Third embodiment]
The third embodiment records audio (acquires audio information) along with video.
The configurations of the inspection support system 1 in the third embodiment are similar to those in the first embodiment except that work procedure video information 411b is stored in the work procedure video DB 410, and therefore will not be described here.

(Work procedure video DB410)
FIG. 14 is a diagram showing an example of the work procedure video information 411 b stored in the work procedure video DB 410. As shown in FIG.
The work procedure video information 411b shown in FIG. 14 has the same structure as the work procedure video information 411 shown in FIG. 5, except that a column for audio information is added.
The voice information field stores voice information for the corresponding inspection work. Note that the voice information field may store the voice information itself, or may store a path where the voice information is stored.

<Flowchart>
(Work video creation process)
Fig. 15 is a flowchart showing the procedure of the work video creation process in the third embodiment. In Fig. 15, the same processes as those in Fig. 7 are given the same step numbers and the description thereof will be omitted.
When an inspection operation is started (S101), the camera 131 of the head-mounted device 100a captures video and the microphone 134 records audio (S102b).
Then, after step S103, the transmitter/receiver 111 of the head-mounted device 100a transmits the acquired video, the audio information recorded at the time the video was being captured, and the location information to the video recording server 200 in association with each other.

Next, after step S111, the extraction unit 212 extracts a range of moving images between the start information and the end information from the acquired moving images, and also extracts audio information associated with the extracted moving images (S112b).
Then, the storage processing unit 213 stores the extracted video, the audio information corresponding to the video, and the position information in the work procedure video DB 410 (S113b). At this time, “normal” is stored in the normal/abnormal column in the work procedure video information 411b in FIG.

In the third embodiment, a specific sound may be used as the start information and the end information. For example, the extraction unit 212 acquires the start information and the end information by using the sound of "start" as the start information and the sound of "end" as the end information.

(Video creation process for abnormal operation)
FIG. 16 is a flowchart showing the procedure of the abnormality operation video creation process in the third embodiment.
The processing in steps S201 to S213b differs from that in FIG. 15 in the points (A1) and (A2) described in FIG. 8, and therefore will not be described here.

(Inspection work)
Fig. 17 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector P2 according to a created manual video in the third embodiment. In Fig. 17, the same steps as those in Fig. 9 are given the same step numbers and the description thereof will be omitted.
After the result of the determination in step S305 is "Yes," the analysis processing unit 313 performs an analysis process based on the video captured by the camera 131 of the head-mounted device 100b and the audio information acquired by the microphone 134 (S306b). The content of the analysis process is to detect an abnormality by machine learning using the video and audio information, similar to step S306 in Fig. 9.

If the result of step S307 is "No," the acquisition processing unit 314 acquires the work video and audio information associated with the location information in the vicinity of the current location from the work procedure video DB 410 (S311b). Here, the acquired work video and audio information are the video and audio information of a record in which "normal" is stored in the normal/abnormal column in the work procedure video information 411b in FIG. 14.
Then, the output processing unit 112 of the head-mounted device 100b outputs the acquired work video to the display 132 worn by the unskilled inspector P2, and outputs the acquired audio from the speaker 133 (S312b).

After step S321, the output processing unit 112 of the head-mounted device 100b acquires the abnormality-state work video and the audio information associated with the abnormality-state work video from the work procedure video DB 410 based on the position information (S322b). Here, the abnormality-state work video and audio information acquired are the work video and audio information of a record in which "abnormality" is stored in the normal/abnormal column in the work procedure video information 411b in FIG. 14.
Then, the output processing unit 112 outputs the acquired abnormality operation video to the display 132, and outputs the acquired audio from the speaker 133 (S323b).

According to the third embodiment, the verbal explanation by the experienced inspector P1 and the sounds of the inspection equipment, such as motor sounds, can be used as a manual, thereby improving the accuracy and efficiency of the inspection by the unskilled inspector P2.

[Fourth embodiment]
In the fourth embodiment, information from the sensor 135 is registered together with the video as a video manual. Each configuration of the inspection support system 1 in the fourth embodiment is similar to that in the first embodiment, and therefore description thereof will be omitted here.

(Work procedure video DB410)
FIG. 18 is a diagram showing an example of the work procedure video information 411c stored in the work procedure video DB 410.
The work procedure video information 411c shown in Fig. 18 has the same configuration as the work procedure video information 411 shown in Fig. 5, except that a column for sensor information is added. The sensor information is information input from the sensor 135 (see Fig. 2) of the head-mounted device 100.
The sensor information field stores sensor information for the corresponding inspection work. Note that the sensor information field may store the sensor information itself, or may store a path where the sensor information is stored.

(Work video creation process)
Fig. 19 is a flowchart showing the procedure of the work video creation process in the fourth embodiment. In Fig. 19, the same processes as those in Fig. 7 are given the same step numbers and the description thereof will be omitted.
When an inspection operation is started (S101), the camera 131 of the head-mounted device 100a captures video, and the sensor 135 acquires sensor information (S102c).
Then, after step S103, the transmitter/receiver 111 of the head-mounted device 100a transmits the acquired video, the sensor information collected at the time the video was captured, and the position information to the video recording server 200 in association with each other.

Next, after step S111, the extraction unit 212 extracts a range of moving images between the start information and the end information from the acquired moving images, and extracts sensor information associated with the extracted moving images (S112c).
Then, the storage processing unit 213 stores the extracted video, the sensor information corresponding to the video, and the position information in the work procedure video DB 410 (S113c). At this time, “normal” is stored in the normal/abnormal column in the work procedure video information 411c in FIG.

(Video creation process for abnormal operation)
FIG. 20 is a flowchart showing the procedure of the abnormality operation video creation process in the fourth embodiment.
The processing in steps S201 to S213c differs from that in FIG. 15 in (A1) and (A2) described in FIG. 8, and therefore description thereof will be omitted here.

(Inspection work)
Fig. 21 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector P2 according to a created manual video in the fourth embodiment. In Fig. 21, the same steps as those in Fig. 9 are denoted by the same step numbers and will not be described.
After the result of the determination in step S305 is "Yes," the analysis processing unit 313 performs an analysis process based on the video captured by the camera 131 of the head-mounted device 100b and the sensor information acquired by the sensor 135 (S306c). The content of the analysis process is to detect an abnormality by machine learning using the video and the sensor information, similar to step S306 in Fig. 9.

If the result of step S307 is "No," the storage processing unit 213 acquires the work video and sensor information associated with the position information in the vicinity of the current position from the work procedure video DB 410 (S311c). Here, the acquired work video and audio information are the video and sensor information of a record in which "normal" is stored in the normal/abnormal column in the work procedure video information 411c in FIG. 18.
Then, the output processing unit 112 of the head-mounted device 100b outputs the acquired work video to the display 132 of the head-mounted device 100b worn by the unskilled inspector P2, and also outputs the acquired sensor information (S312c). The output of the sensor information may be, for example, as follows.
(B1) If the sensor 135 is a thermograph or an infrared camera, a thermographic image or an infrared image is output to the display 132. In this case, it is desirable to display in parallel thermographic images when creating a work video or when creating a work video during an abnormality.
(B2) In the case of an odor sensor or the like, a voltage value or the like acquired from the sensor 135 is displayed on the display 132. In this case as well, it is desirable to display the voltage value when creating the work video or when creating the work video under abnormal conditions in parallel with the voltage value (sensor information) acquired in step S311c.
In addition, it is preferable that the display of the sensor information can be switched ON/OFF by the unskilled inspector P2.

After step S321, the output processing unit 112 acquires the abnormality work video and the sensor information associated with the abnormality work video from the work procedure video DB 410 based on the position information of the vicinity of the current position (S322c). Here, the abnormality work video and audio information acquired are the video and sensor information of the record in which "abnormality" is stored in the normal/abnormal column in the work procedure video information 411c in FIG. 18.
Then, the output processing unit 112 outputs the acquired abnormality operation video to the display 132, and also outputs the acquired sensor information (S323c). The method of outputting the sensor information in step S323c is the same as the output method in step S312c.

According to the fourth embodiment, the unskilled inspector P2 can refer to the temperature, odor, etc. of the inspection equipment, thereby improving the accuracy and efficiency of the inspection by the unskilled inspector P2.

[Fifth embodiment]
In the fifth embodiment, the display is reduced according to the level of skill of the unskilled inspector P2.
(Work processing server 300d)
FIG. 22 is a functional block diagram of the work processing server 300d in the fifth embodiment.
The work processing server 300d differs from the work processing server 300 shown in Fig. 4 in that a switching processing unit 317 is added to an inspection processing unit 311d. Other configurations are the same as those in Fig. 4, so descriptions thereof will be omitted here.
The switching processing unit 317 performs processing to switch the inspection items to be output depending on the skill level of the unskilled inspector P2.

<Flowchart>
(Display item switching process)
FIG. 23 is a diagram illustrating an example of the display item switching process.
The switching processing unit 317 performs a display item switching process for switching (selecting) the inspection items to be output in accordance with the skill level of the unskilled inspector P2 (S401).
The display item switching process is performed, for example, by the following process.
(B1) The switching processing unit 317 performs a display item switching process in accordance with an input by the unskilled inspector P2 himself/herself. For example, the unskilled inspector P2 judges his/her own level of skill, and selects and inputs an inspection item that he/she judges to be unnecessary as an item not to be displayed.
(B2) The switching processing unit 317 performs a display item switching process in accordance with an input by the administrator. For example, the administrator judges the level of expertise of the unskilled inspector P2, and selects and inputs inspection items that are judged unnecessary as items not to be displayed.
(B3) The switching processing unit 317 performs display item switching processing by machine learning. For example, the switching processing unit 317 determines whether to display or hide an inspection item by machine learning based on the inspection speed of the unskilled inspector P2 or the time required for the inspection.

In the fifth embodiment, the work video creation process, the abnormality work video creation process, and the inspection work process are the same as those shown in Figures 7, 8, and 9, and therefore will not be illustrated or described here.

Sixth Embodiment
In the sixth embodiment, a video of an inspection by a non-skilled inspector P2 is also registered in the work procedure video DB 410.
In addition, the work video creation process and the abnormality operation video creation process in the sixth embodiment are similar to those in FIG. 7 and FIG. 8, and therefore illustration and description thereof will be omitted here.

<System Configuration>
FIG. 24 is a diagram showing the configuration of an inspection support system 1e according to the sixth embodiment.
The inspection support system 1e shown in FIG. 24 differs from the inspection support system 1e shown in FIG. 1 in the following points.
(C1) A video is being transmitted from the head mounted device 100b to the video recording server 200e.
(C2) The video recording server 200e stores the video transmitted from the head mounted device 100b in the work procedure video DB 410.

(Movie recording server 200e)
FIG. 25 is a functional block diagram of a moving image recording server 200e in the sixth embodiment.
The moving image recording server 200e differs from the moving image recording server 200 shown in Fig. 3 in that a selection processing unit 214 is added to a registration processing unit 211e. Other configurations are the same as those in Fig. 3, so descriptions thereof will be omitted here.
When storing videos captured by the head-mounted device 100b worn by the unskilled inspector P2 in the work procedure video DB 410, the sorting processing unit 214 distinguishes between videos to be stored and videos to be discarded, and stores the videos to be stored in the work procedure video DB 410.

(Inspection work)
Fig. 26 is a flowchart showing the procedure of an inspection work process performed by a non-skilled inspector P2 according to a created manual video in the sixth embodiment. In Fig. 26, the same steps as those in Fig. 9 are given the same step numbers and the description thereof will be omitted.
Then, after step S314, the transmitting/receiving unit 111 of the head mounted device 100 transmits the video captured by the camera 131 of the head mounted device 100b and the position information to the video recording server 200e. Note that the video may be once transmitted to the work processing server 300, and then transmitted from the work processing server 300 to the video recording server 200e.
Thereafter, the video recording server 200e performs a non-skilled inspector video storage process in which the transmitted video and the position information are associated with each other and stored in the work procedure video DB 410 (S341). The process of step S341 will be described later.
The process performed in step S341 is the process described later with reference to FIG.

After step S327, the transmitting/receiving unit 111 of the head mounted device 100 transmits the captured video and the position information to the video recording server 200e. Note that the video may be once transmitted to the work processing server 300, and then transmitted from the work processing server 300 to the video recording server 200e.
Thereafter, the video recording server 200e performs a non-skilled inspector video storage process in which the transmitted video and position information are associated with each other and stored in the work procedure video DB 410 (S341).

(Unskilled inspector video storage processing)
FIG. 27 is a flow chart showing the procedure of the unskilled inspector video storage process carried out in step S341 in FIG.
In steps S501 and S502, the same processes as those in steps S111 and S112 in FIG. 7 are performed on the video of the inspection work of the unskilled inspector P2.
Then, the selection processing unit 214 of the video recording server 200e performs a selection process on the video extracted in step S502 (S503). In the selection process, the selection processing unit 214 judges whether or not the video to be registered is already stored in the work procedure video DB 410. The video to be registered is not already stored in the work procedure video DB 410 when the inspection equipment has been updated. In this case, if an update flag or the like is attached to the equipment information of the work procedure video, the selection processing unit 214 judges the video of the inspection work by the unskilled inspector P2 as a video to be stored in the work procedure video DB 410. Note that the video stored here may be a video by a somewhat skilled unskilled inspector P2. In step S503, the video judged not to be stored in the work procedure video DB 410 is discarded.

Then, the storage processing unit 213 stores the video determined to be stored in step S503 in the work procedure video DB 410 (S504). At this time, it is preferable that the work ID of the work procedure video information 411 in FIG. 5 is distinguishable from the video by the skilled inspector P1. Also, if "No" is determined in step S307 in FIG. 17, "Normal" is stored in the normal/abnormal column of the work procedure video information 411 in FIG. 5. Then, if "Yes" is determined in step S307 in FIG. 17, "Abnormal" is stored in the normal/abnormal column of the work procedure video information 411 in FIG. 5.

Note that the selection process in step S503 may be omitted so that all videos by the unskilled inspector P2 are stored in the work procedure video DB 410. In this way, data for determining whether an equipment is abnormal or normal by machine learning can be collected.
In addition, by omitting the selection process in step S503 and storing all videos by the unskilled inspector P2 in the work procedure video DB 410, it is possible to provide materials for checking the work content of the unskilled inspector P2 later. In addition, since falsification is impossible, it is possible to guarantee the transparency and reliability of the inspection work. Furthermore, by checking the work videos and the videos of the inspection work by the unskilled inspector P2, the parent maintenance company can compare the state of the inspection work of the maintenance subsidiary, making it easy to select a good subsidiary.

In addition, by performing the selection process in step S503, the volume of videos stored in the work procedure video DB 410 can be reduced.

According to the inspection support system 1 in each embodiment, the following on-site applications are conceivable.
(Z1) Maintenance work on medical equipment in medical institutions and nursing facilities.
(Z2) Maintenance work on building equipment by a security company.
(Z3) Equipment maintenance work in local governments, schools, gymnasiums, etc.
(Z4) Equipment maintenance work at water purification plants.
(Z5) Verification work at each facility (medical institution).
(Z6) In addition, by handing over the generated manual video to a subcontractor for maintenance, even if a different company is contracted to perform the maintenance, the subcontractor can easily perform the maintenance. In addition, the parent company of the maintenance can earn income by renting out the generated manual video or the program that embodies the inspection support system 1.

In the head-mounted device 100, the speaker 133 and the microphone 134 may be omitted except in the third embodiment. Similarly, in the head-mounted device 100, the sensor 135 may be omitted except in the fourth embodiment. Although the sensor 135 is configured to be provided in the head-mounted device 100, it may be installed, for example, on the floor, wall, ceiling, etc. of the building B or on the inspection equipment. In this case, the information output from the sensor 135 may be sent to the work processing server 300, and the information may be output to the display 132 of the head-mounted device 100b. At this time, the storage processing unit 213 of the video recording server 200 associates the information (sensor information) output from the sensor 135 installed on the floor, wall, ceiling, etc. of the building B or on the inspection equipment with the extracted video and stores it in the work procedure video DB 410. With this configuration, in addition to the temperature sensor and the odor sensor, a pressure sensor, a water temperature sensor, a vibration sensor, etc. can be installed and the information can be used.

In addition, although each embodiment describes an example of equipment inspection, the inspection support system 1 in each embodiment may also be applied to inspections of things other than equipment, such as water leaks and overflows in water purification plants.

In each embodiment, the analysis processing unit 313 determines whether or not there is an abnormality in the inspection equipment based on the video, sound, etc. acquired by the head-mounted device 100b. However, this is not limited to the above, and when the unskilled inspector P2 detects an abnormality, the inspection determination unit 315 may determine that an abnormality has occurred in the inspection equipment by giving a specific signal.
For example, if an abnormality occurs in the inspection equipment, the unskilled inspector P2 may, during the inspection work, hold his or her thumb and index finger outstretched over the camera 131 of the head-mounted device 100b. In this case, the inspection determination unit 315 recognizes the image of the fingers with the thumb and index finger outstretched as an abnormality signal.

In addition, when determining whether an abnormality in the inspection equipment has occurred based on a signal from unskilled inspector P2, if multiple videos of work performed during an abnormality exist, thumbnails of the multiple videos of work performed during an abnormality may be output to display 132 of head-mounted device 100b. Then, unskilled inspector P2 selects from the thumbnails a thumbnail of a video of work performed during an abnormality that corresponds to the currently occurring abnormality. Then, the video of work performed during an abnormality that corresponds to the selected thumbnail may be output to display 132.

Furthermore, a smartphone, a tablet terminal, or the like may be used instead of the head-mounted device 100. Furthermore, for example, an experienced inspector P1 may wear the head-mounted device 100, and an unskilled inspector P2 may carry a smartphone to perform inspection work.
The moving image recording server 200 and the task processing server 300 may be separate servers or may be an integrated server.

In addition, in each embodiment, inspection equipment and work videos are assigned one-to-one, but it is also possible to assign multiple inspection equipment to one work video. For example, this is the case when multiple inspection equipment of the same type are provided, or when similar inspection equipment exists. In other words, this is the case when "inspection equipment B" of the same type as "inspection equipment A" exists, and also when "inspection equipment C" exists that is similar to "inspection equipment A" and has almost the same inspection work procedure. "Inspection equipment A", "inspection equipment B", and "inspection equipment C" may be provided in the same building or equipment, or in different buildings, equipment, or bases. Here, a base refers to equipment that exists in different regions, for example, one base is in prefecture A and another base is in prefecture B.

In this case, the position information of multiple inspection devices is stored in the position information column of the work procedure video information 411 shown in FIG. 5. For example, in the above example, the position information of "inspection device A," "inspection device B," and "inspection device C" is stored in the position information column of the work procedure video information 411. Then, when a work video of "inspection device A" is generated based on the inspection work by the skilled inspector P1, the skilled inspector P1 inputs the position information of "inspection device B" and "inspection device C," which can perform similar inspection work, via the input device 231 of the video recording server 200, etc.

In this case, the location (inspection location) where the unskilled inspector P2 performed the inspection work is entered into the location information column of the checklist 431 shown in FIG. 6. That is, when the unskilled inspector P2 inspects "inspection device A", the location information (inspection location) of "inspection device A" is entered into the location information column of the checklist 431. Similarly, when the unskilled inspector P2 inspects "inspection device B", the location information (inspection location) of "inspection device B" is entered into the location information column of the checklist 431, and when the unskilled inspector P2 inspects "inspection device C", the location information (inspection location) of "inspection device C" is entered into the location information column of the checklist 431. In this way, efficient management of extracted videos is possible.

The checklist 431 can also be stored in a work procedure video DB in association with a work video. The checklist 431 is a checklist for an inspection task in the associated work video.
In this case, the checklist 431 may be input in advance as text data or image data via the input device 231 of the video recording server 200, or may be generated based on the audio in the work video. Then, in step S113 of Fig. 7, the storage processing unit 213 of the video recording server 200 associates the extracted work video with the checklist 431 for this work video and stores them in the work procedure video DB 410. In this way, the checklist 431 and the manual video (work video) of the inspection work can be managed in association with each other. This enables efficient management of the checklist 431.

The present invention is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

In addition, the above-mentioned configurations, functions, each unit 111-112, 211-213, 311-317, work procedure video DB 410, checklist DB 430, etc. may be realized in hardware by designing some or all of them as an integrated circuit, for example. In addition, as shown in Figures 2 to 4, 22, and 25, the above-mentioned configurations, functions, etc. may be realized in software by a processor such as the CPU 121, 221, or 321 interpreting and executing a program that realizes each function. Information such as the program, table, and file that realizes each function can be stored in a recording device such as the memory 110, 210, or 310, or a solid state drive (SSD), or a recording medium such as an integrated circuit (IC) card, a secure digital (SD) card, or a digital versatile disc (DVD).
In addition, in each embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other.

1, 1e Inspection support system 100, 100a, 100b Head-mounted device 100a Head-mounted device (first terminal unit)
100b Head-mounted device (second terminal unit)
131 Camera (first imaging unit, second imaging unit)
132 Display (display unit, sensor information output unit)
133 Speaker (audio output unit)
134 Microphone (voice acquisition unit)
135 Sensor (sensor part)
136 Position acquisition device (first position acquisition unit, second position acquisition unit)
200, 200e Video recording server (inspection support device)
212 Extraction unit 213 Storage processing unit (output unit)
214 Selection processing unit (display selection unit)
300, 300d Work processing server 314 Acquisition processing unit (video acquisition unit)
410 Work procedure video DB (video storage unit, stores extracted videos, extracted audio information, and extracted sensor information)
431 Checklist P1 Expert Inspector (First Person)
P2 Unskilled inspector (second person)
S112 Extraction of video (extraction step)
S113: Storage of video and position information (output step)
S311 Acquire work video (video acquisition step)
S312 Output of work video (display step)

Claims (16)

An inspection support system that supports inspection work performed while moving to multiple locations,
a first terminal unit having a first imaging unit for capturing a video and a first position acquisition unit for acquiring position information, the first terminal unit being carried by a first person;
an extraction unit that generates an extracted moving image by extracting, from a first moving image captured by the first imaging unit, a range of the moving image sandwiched between start information, which is predetermined information in the first moving image or predetermined information input from outside at the time when the first moving image is being captured, and end information, which is predetermined information in the first moving image or predetermined information input from outside at the time when the first moving image is being captured;
an output unit that outputs first position information, which is the position information acquired by the first position acquisition unit when the extracted moving image is captured, in association with the extracted moving image;
An inspection support system comprising: The inspection support system according to claim 1 , wherein the start information and the end information are predetermined images in the first moving image. A second terminal unit has a display unit and a second location acquisition unit that acquires location information, and is owned by a second person different from the first person;
a moving image acquisition unit that acquires the extracted moving image based on second location information that is the location information acquired by the second location acquisition unit;
The display unit is
The inspection support system according to claim 1 , further comprising: displaying the extracted video acquired by the video acquisition unit. The second terminal unit has a second imaging unit,
The inspection support system according to claim 3, characterized in that while the extracted video is displayed on the display unit, a second video, which is a video captured by the second imaging unit, is stored in a video memory unit that stores the videos. the first terminal unit has a voice acquisition unit that acquires voice information;
the second terminal unit has a voice output unit that outputs the voice information,
The extraction unit extracts, from the audio information, the audio information associated with the extracted moving image as extracted audio information;
The output unit outputs the extracted moving image and the extracted audio information in association with each other;
The inspection support system according to claim 3 , wherein the audio output unit outputs the extracted audio information associated with the extracted moving image displayed on the display unit. The inspection support system according to claim 5 , characterized in that the start information and the end information are predetermined audio information among the audio information acquired by the audio acquisition unit at the time when the first video is being captured. A plurality of pieces of position information are associated with the extracted video,
The inspection support system according to claim 3 , characterized in that the video acquisition unit acquires the extracted video by comparing the second position information with a plurality of pieces of position information associated with the extracted video. The inspection support system according to claim 3 , further comprising: a display selection unit that selects the extracted moving image to be displayed on the display unit depending on a proficiency level of the second person. The second terminal unit has a second imaging unit,
The inspection support system according to claim 3, further comprising a check processing unit that, when predetermined information in a second video, which is a video captured by the second imaging unit, or predetermined information input from outside and indicating that the inspection work by the second person has been completed, is input while the extracted video is being displayed on the display unit, enters a check in an item corresponding to the second location information in a checklist stored in a checklist memory unit. the first terminal unit has a sensor unit having at least one of a temperature sensor and an odor sensor,
The extraction unit extracts, from the sensor information acquired from the sensor unit, the sensor information associated with the extracted moving image as extracted sensor information;
The inspection support system according to claim 1 , wherein the output unit outputs the extracted video and the extracted sensor information in association with each other. A building in which the equipment to be inspected is installed or a sensor unit installed in the equipment,
the extraction unit acquires sensor information that is information output from the sensor unit at a time when the first moving image is being captured;
The inspection support system according to claim 1 , wherein the output unit outputs the sensor information and the extracted video in association with each other. The inspection support system according to claim 1 , wherein the output unit outputs the extracted video and a checklist of the inspection work corresponding to the extracted video in association with each other. The first terminal is a head-mounted device.
2. The inspection support system according to claim 1 . an extraction unit that generates an extracted video by extracting, from a first video captured by a first imaging unit of a first terminal unit carried by a first person, a video within a range between start information, which is predetermined information in the first video or predetermined information input from outside at the time when the first video is being captured, and end information, which is predetermined information in the first video or predetermined information input from outside at the time when the first video is being captured;
an output unit that outputs first position information, which is position information acquired by a first position acquisition unit of the first terminal unit when the extracted video is captured, in association with the extracted video;
An inspection support device comprising: An inspection support system that supports inspection work while moving between multiple locations,
A first terminal unit having a first imaging unit that captures a moving image and a first position acquisition unit that acquires position information, the first terminal unit being carried by a first person;
an extraction step of generating an extracted moving image by extracting, from a first moving image captured by the first imaging unit, a range of the moving image sandwiched between start information, which is predetermined information in the first moving image or predetermined information input from outside at the time when the first moving image is being captured, and end information, which is predetermined information in the first moving image or predetermined information input from outside at the time when the first moving image is being captured;
an output step of outputting first position information, which is the position information acquired by the first position acquisition unit when the extracted moving image is captured, in association with the extracted moving image;
An inspection support method comprising the steps of: The inspection support system includes:
a second terminal unit having a display unit and a second location acquisition unit that acquires location information, the second terminal unit being owned by a second person different from the first person;
a moving image acquisition step of acquiring the extracted moving image based on second position information, which is the position information acquired by the second position acquisition unit;
a display step of displaying the extracted moving image on the display unit;
The inspection support method according to claim 15, further comprising the steps of:

Images