JP2021086401A - Management apparatus and program - Google Patents

Abstract

To provide a management apparatus and a program configured to effectively utilize information registered in advance on a work by associating text information on the work with work log information.SOLUTION: In a management system 100, a management apparatus 10 includes: a registration unit 14 which stores, for each of a plurality of works, text information described for the works and log information to be acquired when the work is executed, in association with each other; and a search unit 15 which searches for the registration unit with a search query input by a user, and outputs a search result for a work that matches at least one of the text information and the log information associated with the work with the search query.SELECTED DRAWING: Figure 1

Description

The present invention relates to a management device and a program capable of effectively utilizing the information registered regarding the work by linking the text information about the work and the work log information.

There are various technologies related to searching documents related to business.

For example, Patent Document 1 relates to a similar document extraction device capable of accurately extracting a document group in which a plurality of description levels similar to a key document are mixed, and assumes that the search target document is a document written about some work. .. The business management support device of Patent Document 2 finds similar cases from past business using activity log information and meteorological observation information, estimates the types of risks that may occur in the future, and evaluates the contents. Then, the manual information is searched based on the estimated type of risk and the evaluation result of the content, and presented to the terminal of the worker who carries out the work.

In addition, there are various technologies related to support for on-site work such as data collection utilizing IT (information technology) as described in Patent Documents 3 to 11 below. Such technology is expected to be used in the framework of IT service management (ITSM, Non-Patent Document 1) related to field work and the like.

In the safety management system of Patent Document 3, the safety management data for each worker including the state information and the accident information acquired from the worker terminal is compared with the preset state judgment standard data, and the worker is in a dangerous state. Notify when it is determined that there is. The target data are "heat index and temperature / humidity", "acceleration and position", "pulse and fatigue", "noise and fatigue", and "movement amount and rest time".

In the plant management system of Patent Document 4, in a situation where a worker carries a terminal at a work site such as a plant, a sensor is specified from the position of the terminal to collect data about a measurement target, and images, sounds, etc. collected at the same time. It recognizes the operating status of the measurement target in cooperation with the operator and presents the information to support the operator to the operator. In order to recognize the operating status of the worker, data is consciously collected from the worker.

In the work support system for interactive recognition of Patent Document 5, when a worker asks a question by voice, the situation at the work site is collected from video, voice, and a sensor attached to the target device to recognize the state of the target device. , Tell the worker the degree of completion of the task. The information processing device of Patent Document 6 analyzes the captured image and collects and analyzes the environmental information, voice, and biological information at the time of shooting so that the situation of the subject can be recognized and the realism of the site can be shared. Display this recognition status. In the work check sheet read-through / registration method of Patent Document 7, the progress of work is managed by registering the read-through of the work check sheet using voice recognition technology.

In the status management system of Patent Document 8, in order to support the management of the machining status (image, date and time, position) of the machining target such as the joint member, the status information including the image information is mapped on the map information. In the work support system of Patent Document 9, a work diary that records the work environment and work history is automatically created by the worker wearing a wearable terminal in maintenance / inspection work, and at that time, the skill information of the worker is also included. Since it is remembered, it also provides help information that matches the skill of the worker.

In the maintenance support terminal / system of Patent Document 10, in maintenance support, by storing information on supporter candidates in advance, a supporter is selected and joint work is carried out when the worker cannot work alone. To do. In the construction information acquisition device, management system, and information terminal of Patent Document 11, in order to collect information that can be used for future construction from past captured images and search for similar projects, the images taken during construction are used. Recognize the imprinted text and the image of the object and use it as meta information.

Japanese Unexamined Patent Publication No. 2018-73353 Patent No. 6349284 JP-A-2018-180573 International Publication No. 2017/169276 JP-A-2018-160232 Japanese Unexamined Patent Publication No. 2018-170714 Patent No. 5684419 Patent No. 6066797 Japanese Patent Application Laid-Open No. 2005-085190 Japanese Unexamined Patent Publication No. 2001-160097 Japanese Unexamined Patent Publication No. 2019-066920

"What is ITSMS?" [Online], 2017, [Searched on October 23, 1st year of Reiwa], Internet <URL: https://www.itsmf-japan.org/aboutus/itsms.html>

However, the prior art has not always been able to give a sufficient result to a user who wants to obtain knowledge about the work or the like by searching a document related to the work or the like that has already been registered.

In the technique of Patent Document 1, since the description level between documents is different, even when a document having a high degree of similarity is extracted, the extracted document may contain information up to the level of detail desired by the user. It is not always similar to the level of detail. For example, even if a user who starts work after extraction wants to know information on work risks and preventive measures, the target information is not always written in the extracted document. Therefore, the extraction result based on the high degree of similarity between documents in Patent Document 1 is not always advantageous for the user who wants to acquire the know-how and risk of the work when trying to carry out similar work in the future. It was a thing.

In the technique of Patent Document 2, by finding similar cases in the past, it is possible to efficiently find preventive measures for risks and to find prompt countermeasures when risks occur. Furthermore, because it uses the raw environmental information of the site called meteorological observation information, the document is highly accurate and easy for the user to understand. However, there is a problem that the response activity log information is a technology related to work that is affected by meteorological observation information, and cannot be applied to indoor operation work that is not affected by meteorological observation information.

Further, although the techniques of Patent Documents 3 to 11 relate to assistance of work, etc., they do not deal with documents related to work, etc., and therefore cannot give knowledge to a user who searches for documents related to work, etc. Met.

In view of the above problems of the prior art, the present invention provides a management device and a program capable of effectively utilizing the information registered for the work by linking the text information about the work and the work log information. The purpose is to do.

In order to achieve the above object, the present invention is a management device, and for each of a plurality of operations, the text information describing the work and the log information acquired when the work is performed are associated with each other. Search the registration unit using the registration unit stored in the above and the search query input by the user, and the search query hits at least one of the text information or the log information associated with the work. It is characterized by including a search unit that outputs a search result for. Further, the program is characterized in that the computer functions as the management device.

According to the present invention, not only the text describing the work but also the log information of the work that can be collected by remote support or the like can be searched, so that the search related to the work can be efficiently performed and the work is registered. It is possible to effectively utilize the information provided.

It is a functional block diagram of the management system which concerns on one Embodiment. It is a flowchart of the operation of the management system which concerns on one Embodiment. It is a flowchart of the procedure of registering the work data about the individual work which concerns on one Embodiment in the registration part. It is a functional block diagram of a terminal including the functional block which shows the element structure of the log acquisition part. It is a figure which shows the example of the screen display which presents the detailed work data of the work to the user by the presentation part. It is a figure which shows the example of the interface which provides the log information to a user as a graph as an example corresponding to the example of FIG. It is a figure which shows the hardware configuration in a general computer apparatus 70.

FIG. 1 is a functional block diagram of a management system according to an embodiment. The management system 100 includes a management device 10 and a terminal 20. The management device 10 can be configured as a server, and can be implemented as a document management system (a document management system having extended functions according to the present embodiment) in a company or the like. By configuring the terminal 20 as a wearable terminal such as an HMD (head-mounted display, optical see-through type or video see-through type, the same shall apply hereinafter), the work user Uw who performs the work can carry it to the work site. It is possible. There may be one or more terminals 20, but FIG. 1 shows any one of such one or more terminals 20. The management device 10 and the terminal 20 are configured to be able to communicate with each other via the Internet, LAN, or the like.

The management device 10 includes a text reception unit 11, a log reception unit 12, a recognition unit 13, a registration unit 14, a search unit 15, and a presentation unit 16. The terminal 20 includes a log acquisition unit 21 and an auxiliary presentation unit 22.

FIG. 2 is a flowchart of the operation of the management system 100 according to the embodiment. After the database is built in the management system 100 in advance (step S1), the user searches using the database of the management system 100 (step S2). ) Is possible. (Since the search in step S2 is executed for each individual user according to the wishes of the individual users, the search in step S2 may not be performed depending on the user.) As will be described later, the user. From the standpoint of, it is possible to search the management system 100 for each work and other cases as necessary, and to register texts and logs related to the work based on the obtained knowledge. By adding the registered information, the database of the management system 100 will be updated each time.

In step S1, the registration unit 14 as a database is constructed, and then the process proceeds to step S2. In step S2, by using the constructed registration unit 14, the search user Us searches the management device 10 to obtain a search result, and the flow ends. That is, the management system 100 builds the database of the registration unit 14 in advance in step S1, and then enables the user Us to search in step S2. The search in step S2 can be performed by any user not only once but as many times as necessary by using the registered registration unit 14 that has already been constructed.

In addition, in FIG. 2, in order for the management system 100 to return the search result to the search from the user, it is necessary for the management system 100 to have a database sufficient for the search to be constructed in advance. , The step that the user actually searches using this is distinguished as step S2. Step S1 is not necessary from the standpoint of an individual user rather than the standpoint of the management system 100, and the user searches the management system 100 (a database of a predetermined size has already been constructed) for each work as needed. (Step S2) can be performed, and texts and actual work related to the work can be performed based on the obtained knowledge. By adding the registered information, the database of the management system 100 will be updated each time. Step S1 is configured as the accumulation of the information registered and updated for each work over many works. With respect to this large number of tasks, the user in charge of each task may change each time.

That is, the database construction in step S1 of FIG. 2 can be constructed by accepting text input related to the work and log collection in the actual work, respectively, for a large number of work in business or the like. (As mentioned above, if a database of a predetermined size sufficient for the search has already been constructed, the user can perform the search in step S2 in order to obtain reference information on this work.) For the sake of explanation here, for the sake of explanation. Each ID of a large number of works (N in total) is identified by the integers i = 1,2,3, ..., N and written as work W (i). The individual work W (i) is, for example, maintenance work in communication equipment. The registration unit 14 constructed as a database contains the text information T (i) received by the text reception unit 11 and the log information L (i) received by the log reception unit 12 for each work W (i), and this log information. The database information DB associated with the recognition result R (i) obtained by applying the recognition process by the recognition unit 13 to L (i) is saved. That is, this database information DB can be written as the following equation (1) in set notation.
DB = {(T (i), L (i), R (i)) | i = 1,2,3,…, N}… (1)

For the sake of explanation, of the entire database information DB of equation (1), the individual information related to work W (i) is provided as work data D (i) = (T (i), L (i), R (i)). Notated as. (Therefore, it can be expressed as DB = {D (i) | i = 1,2,3, ..., N}.) FIG. 3 shows work data D for individual work W (i) according to one embodiment. It is a flowchart of the procedure of registering (i) in the registration unit 14. In step S1 of FIG. 2, the flowchart of FIG. 3 is executed for each of a large number of work W (i) (i = 1,2,3, ..., N), and individual work data D (i) is stored in the registration unit 14. ) Is accumulated.

As shown, the flow of FIG. 3 consists of seven sequential steps S11-S17. Of these, steps S11, S12, S13, and S16 are steps related to text input and work execution by the user, but as explained above, if a database sufficient for search has already been constructed, the user can perform text input and work. By performing a search on the management system 100 at that time, reference information can be obtained as a search result. This search step is shown as sub-steps S110, S120, S130 and S160 contained within steps S11, S12, S13 and S16 and can be performed as options.

Note that the sub-steps S110, S120, S130 and S160 in FIG. 3 are examples of situations in which the user feels the need to search the management system 100 and actually searches for the management system 100 during text creation or work ( This is an example of (including preparation before work). The user is free to manage according to his / her own needs even in any situation other than the text creation and work as shown in FIG. 3 (that is, regardless of whether the work text or the like is created or work is performed). It is possible to search for system 100. That is, the search timing in step S2 of FIG. 2 may be any timing desired by the individual user in this way.

In step S11, the user Ut1 inputs the work plan text T1 (i) related to the work W (i) to the text reception unit 11, and the text reception unit 11 accepts the input of the work plan text T1 (i). Then proceed to step S12. In this step S11, as step S110, the user Ut1 (corresponding to the search user Us in this case) may perform a search. In step S12, the user Ut2 reads and confirms the work text T1 (i) received for the work W (i), and then inputs the work approval text T2 (i) for the work W (i) to the text reception unit 11. Then, after the text reception unit 11 accepts the input of the work approval text T2 (i), the process proceeds to step S13. In this step S12, as step S120, the user Ut2 (corresponding to the search user Us in this case) may perform a search.

The user Ut1 who inputs the work plan text T1 (i) in step S11 is, for example, a user who actually performs this work W (i), and in order to obtain approval to carry out the work W (i) in business, for example, Create and enter the work plan text T1 (i) by yourself before actually performing the work W (i). The work plan text T1 (i) is configured to describe the plan of the work W (i), for example, including a description of the contents of the work W (i). If the user Ut1 is a user who is not accustomed to inputting the work plan text T1 (i), the user Ut1 searches in step S110 and performs a work plan text for a work similar to the work W (i) (work). By narrowing down the approval text and work report text) in the search unit 15 and presenting the narrowed down results using the presentation unit 16, work while referring to data on work similar to work W (i). Since the plan text T1 (i) can be input, the efficiency of the work plan input work is improved.

Further, the user Ut2 who inputs the work approval text T2 (i) in step S12 is, for example, a user who has the approval authority to perform this work W (i), and inputs the work plan text T1 (i) of the work W (i). After reading, create and enter a work approval text T2 (i) that gives approval for this plan. The work approval text T2 (i) may be configured to include a description such as advice for the work W (i) as an expression to approve the performance of the work W (i). , It may be composed of a fixed phrase indicating approval. If user Ut2 is an approver who is not accustomed to inputting work approval text T2 (i), user Ut2 may perform work approval text for work similar to work W (i) in step S120 (work plan text, work plan text, etc.). By narrowing down the work report text (which may be the work report text) in the search unit 15 and presenting the narrowed down results using the presentation unit 16, the work approval text T2 (work approval text T2 (which may be the work report text) is referred to while referring to the data about the work similar to the work W (i). Since i) can be entered, the efficiency of approval work will be improved.

The input of the work plan text T1 (i) and the work approval text T2 (i) in steps S11 and S12 can be viewed or input via the interface of the existing document management system (for example, via a web page) in the text reception unit 11. By using an enable browser, etc.) to assign i, which is an ID related to work W (i), in advance, and then sending an e-mail to the corresponding users Ut1 and Ut2 via this ID, etc. You can prompt for input. Although not shown in FIG. 3, if it is determined that the contents of the work plan text T1 (i) are insufficient and the user Ut2 does not approve in step S12, the user Ut1 is notified to that effect. , User Ut1 may again enter the work plan text T1 (i) properly described in step S11.

In step S13, the user Uw carrying the terminal 20 performs work W (i) at the site in accordance with the work plan text T1 (i) in step S11 (and the work approval text T2 (i) in step S12). Then, after the log acquisition unit 21 acquires the log information L (i) related to the work W (i) during the execution, the process proceeds to step S14. In step S13 (including during preparation before work execution), user Uw (corresponding to search user Us in this case) may perform a search as step S130. In step S14, the log information L (i) acquired by the log acquisition unit 21 of the terminal 20 in step S13 is transmitted to the management device 10, and the log reception unit 12 receives the log information L (i), and then the step. Proceed to S15.

The log information L (i) acquired by the log acquisition unit 21 in step S13 may be composed of one or more arbitrary types of time-series information that records the work W (i) on the time series at the site. Yes, the details will be described later. The user Uw who carries the terminal 20 in step S13 and thereby enables the log acquisition unit 21 to acquire the log information L (i) inputs the work plan text T1 (i) of the work W (i) in step S11. It may be the same as the user Ut1 or another user. If you are another user, refer to the work plan text T1 (i) in step S11 (and the work approval text T2 (i) in step S12) to understand the work W (i), and then work W ( You can start on i). If the user Uw is a user who is not accustomed to the execution of the work W (i), the user Uw narrows down the registration data for the work similar to the work W (i) in the search unit 15 in step S130. However, by presenting the result using the presentation unit 16, it is possible to prepare for carrying out the work W (i) while referring to the data on the matter similar to the work W (i). It is possible to facilitate the work safely and to improve the efficiency of the work.

In step S15, the recognition unit 13 applies the recognition process to the log information L (i) received by the log reception unit 12 in step S14, so that the work W recorded in the log information L (i) ( After obtaining the recognition result R (i) for i), the process proceeds to step S16.

Here, it is assumed that the log information L (i) is recorded for the work W (i) over each time t (t _S ≤ t ≤ t _{E ) from the start time t S} to the end time t _E. Of the log information L (i), the information at time t is referred to _{as L (i) [t].} (That is, it can be expressed as L (i) = {L (i) _[t] | t _S ≤ t ≤ t _E)}. ) The recognition result R (i) obtained by the recognition unit 13 is configured as the recognition result of the work content at each time t given as the _{identifier TR (i) [t].} (That is, R (i) = {TR (i) _[t] | t _S ≤ t ≤ t _E )}. )

_{In the above, the recognition result identifier TR (i) [t] at} each time t is used for explanation in relation to the log information L (i) on the time series, but the work W on the time series by the user Uw ( Since i) is generally composed of events for each interval in the time series, the recognition result for each _{time interval TD = {t | t a} ≤ t <t _{b} of such an individual event is actually} The identifiers TR (i) _[TD] will be obtained. (The recognition result identifiers TR (i) _[t] at each time t in the time interval TD have the same contents.)

For example, when the work W (i) is the maintenance work of the server, the recognition result R (i) can be composed of the identifier (ID) corresponding to the recognition result for each time interval of the individual event as follows. Become.
Recognition result identifier TR (i) _[TD1] = ID corresponding to the recognition result of "walking" in the first time interval TD1 = {t | t ₀ ≤ t ≤ t _1}
Second time interval TD2 = {t | t ₁ ≤ t ≤ t ₂ } recognition result identifier TR (i) _[TD2] = ID corresponding to the recognition result of "stop"
Third time interval TD3 = {t | t ₂ ≤ t ≤ t ₃ } recognition result identifier TR (i) _[TD3] = ID corresponding to the recognition result of "cable insertion / removal"
4th time interval TD4 = {t | t ₃ ≤ t ≤ t ₄ } recognition result identifier TR (i) _[TD4] = ID corresponding to the recognition result of "walking"

In the above, the example of the recognition result of the individual event to which the identifier is given describes the work operation of the worker such as "walking", "stopping", and "cable insertion / removal". As a result, the recognition result of the individual event may be configured as a description that some trouble has occurred. That is, the recognition result can be an arbitrary content that describes the individual events (events related to the worker, events related to the server, etc. that is the work target, events related to the work environment, etc.) that make up the work W (i). Yes, as will be described later, it is defined in advance and an identifier is given.

In step S15, the recognition process for the recognition unit 13 to obtain the recognition result R (i) as the time series of the recognition result identifier for the time series log information L (i) is a model learned in advance by machine learning. It is possible by using. The details will be described later together with the details of the log information L (i) acquired by the log acquisition unit 21 in step S13.

In step S16, the user Ut3 inputs the work report text T3 (i) relating to the work W (i) to the text reception unit 11, and the text reception unit 11 accepts the input of the work report text T3 (i). Then proceed to step S17. In this step S16, as step S160, the user Ut3 (corresponding to the search user Us in this case) may perform a search. The user Ut3 who inputs in step S16 may be a user who can grasp and report the implementation status of work W (i) in the field, and actually performed work W (i) in step S13. It may be the same user as the user Uw, it may be the same as the user Ut1 who input the work plan text T1 (i) in step S11, or it may be another user. The work report text T3 (i) is created and entered by user Ut3 as a report of the results of performing the planned and approved work W (i). For example, if there is an unexpected trouble in work W (i), enter it so that it is possible to give knowledge to prevent similar trouble when similar work is performed in the future, including a description about this. Is desirable. If user Ut3 is an approver who is not accustomed to inputting work report text T3 (i), user Ut3 may use work report text, etc. for work similar to work W (i) in step S160 (work plan text, etc.). By narrowing down the work approval text (which may be the work approval text) in the search unit 15 and presenting the narrowed down results using the presentation unit 16, the work report text T3 (work report text T3 (which may be a work approval text) is referred to while referring to data on work similar to work W (i). Since i) can be input, the efficiency of work report input will be improved.

The input of the work report text T3 (i) in step S16 is also existing in the text reception unit 11 in the same manner as the input of the work plan text T1 (i) and the work approval text T2 (i) in steps S11 and S12. After confirming i, which is the assigned ID for work W (i), using the interface of the document management system (web page browser, etc.), electronically to the corresponding user Ut3 via this ID. Input can be prompted by sending an e-mail or the like.

In step S17, the work data D (i) for the work W (i) obtained in the above steps S11 to S16 is registered (recorded) in the registration unit 14, and the flow ends. As already explained, the work data D (i) has text information T (i) and log information L (i) such as "D (i) = (T (i), L (i), R (i))". ) And the recognition result R (i). The text information T (i) registered in the registration unit 14 is the work plan text T1 (i), the work approval text T2 (i) and the work report text T3 (i) received by the text reception unit 11 in steps S11, S12 and S16. ), And (that is, T (i) = (T1 (i), T2 (i), T3 (i))), the log information L (i) registered in the registration unit 14 is a step. If the log information L (i) received by the log receiving unit 12 in S14 is used, and the recognition result R (i) registered in the registration unit 14 is the recognition result R (i) recognized by the recognition unit 13 in step S15. Good.

As described above, individual work data D (i) is registered in the registration unit 14 by executing the flow of FIG. 3 for each of a large number of work W (i) (i = 1,2,3, ...). , The database information DB = {D (i) | i = 1,2,3, ..., N} of the registration unit 14 is constructed by accumulating these, which is the step S1 of FIG. 2 already described. The search in step S2 of FIG. 2 can be performed with reference to the database information DB constructed in this way. As already explained, in the situation where the database information DB is constructed as sufficient for the search, the execution timing of step S2 may be any timing when the user feels the need for the search, and the user performs the desired search. Can be done. Examples of the timing at which the search in step S2 is executed include steps S110, S120, S130, and S160 in FIG. 3, which are examples of timings at which the search is useful. Regardless of the timing at which step S2 is executed, the search can be specifically performed as follows.

In one embodiment, in step S2, the user Us inputs the word Qt (word, phrase, sentence, etc.) and the state Qr as the search query into the search unit 15 as the search key (search query), and the registration unit 14 in the search unit 15. Searches the database information DB stored in, and outputs the work data D (i) that hits this search query Qt, Qr to the presentation unit 16 as a search result list {D (i)}. Here, regarding the state Qr, a menu or the like in which the identifier and the corresponding recognition result (for example, the recognition result such as "walking" or "stopping" described above) are described so that the state Qr can be specified as the identifier of the recognition result in the recognition unit 13. Is prepared in advance, and the user Us may specify the state Qr by an identifier by referring to the menu or the like. Here, since the log information L (i) is raw data including video and the like, the recognition result R (i) is searched by specifying the state Qr instead of directly searching the log information L (i). ..

The presentation unit 16 displays the search result list {D (i)} on the display to display the search result to the user Us. The user Us who browses this search result list {D (i)} is selected in the presentation unit 16 by selecting and inputting a desired case (work W (i)) from the list to the presentation unit 16. As the details of the case, the details of the work data D (i) can be further presented to the user Us.

Here, when the search unit 15 searches the work data D (i) = (T (i), L (i), R (i)) with the search queries Qt and Qr, the text related to the work is performed by the phrase Qt. The information T (i) may be searched, and the recognition result R (i) corresponding to the work log information L (i) may be searched by the state Qr. That is, the search result list {D (i)} can be expressed as shown in (a) below.
{D (i)} = {D (i) | “Qt ∈ T (i)” and “Qr ∈ R (i)”}… (a)

The above search example is a database composed of (1) text information T (i) and (2) log information L (i) as work log-related information and its recognition result R (i). This is the case when the user Us specifies the AND search for the registration unit 14 of the above, and the search of only one of them shown in (b) or (c) below or the OR search shown in (d) below is performed. The user Us may specify to obtain the search result {D (i)}. In this way, in step S2, the search conditions are specified for the registration unit 14 as a database composed of two sets of text information and log-related information (conditions such as AND and OR for the two sets of database). User Us can search including.
{D (i)} = {D (i) | "Qt ∈ T (i)"}… (b)
{D (i)} = {D (i) | "Qr ∈ R (i)"}… (c)
{D (i)} = {D (i) | “Qt ∈ T (i)” or “Qr ∈ R (i)”}… (d)

In the case of (b) above, in order to improve the convenience of the user when presenting the search results, the search results are divided into a plurality of categories based on the status data (distinguished by the status identifier). You may make a presentation. For example, if there are many cases that include a work state that indicates "cable removal" work during the narrowed down work, the user is divided into those that include the cable removal work and those that do not, including the sorting criteria. If presented, the user can easily guess what kind of matter is specifically included in each matter group. Regarding categorization, it is based on a preset rule base (for example, as in the above example, the most states included in the search results are categorized according to whether or not the most states are included in each work). Just do it.

On the other hand, when the details of the work data D (i) selected from the search result list {D (i)} in the presentation unit 16 are further presented to the user Us, the log information L (i) is also included. The entire work data D (i) = (T (i), L (i), R (i)) can be provided as a target for viewing or viewing by the user Us.

In particular, the search according to the embodiment shown in FIGS. 2 and 3 described above can produce the following effects. That is, of the work data D (i) = (T (i), L (i), R (i)), the text information T (i) works in the same framework as the existing document management system W (i). Since it is composed of text information that describes plans, approvals, reports, etc. regarding the above, if the conventional method search that targets only this text information T (i) is adopted, the user Us desires. Search results may not be available. For example, when user Us wants to search for work cases of cable replacement, even if the word "cable" is used for searching, "server maintenance work" (work W (work W)) is normally performed for "cable". Assuming that the text information T (k) described in terms of a different level (in this case, the higher level) from the "cable" such as "maintenance" is registered with respect to (k)), the word "cable" Is not included, so the result of this work W (k) cannot be referenced.

On the other hand, in the present embodiment, the log information L (k) as the information related to the work log and the recognition result that enables the search are performed instead of the single database configuration of only the text information T (k) as in the conventional case. Efficient search is possible due to the dual database configuration including R (k). That is, even if the text information T (k) related to the work W (k) does not include the word "cable", the recognition result R (k) obtained by automatically recognizing the log information L (k) by the recognition unit 13 Since it is possible to include work-related identifiers using cables in the identifiers, the results of work W (k) are used not only as search results by the phrase query Qt for "cable", but also using cables. It can also be referred to as a search result by the state query Qr corresponding to the work-related identifier, and unlike the conventional method search, it is possible to effectively utilize the business record without burying it. Then, when the search related to "cable" by the query Qt and / or QR is hit, and when the log information L (k) contains a video, this video is presented by the presentation unit 16 and the user Us is informed. It is also possible to present work images related to "cables", which makes it possible to make more effective use of business records. (Here, the business record is used as an explanation example, but the same effective utilization is possible for any record related to the work.)

Similarly, even if the text has the same content, there may be notational fluctuations in which the expression changes depending on the person. Therefore, even if the phrase query Qt alone cannot be used for efficient search, it is specified as the state query Qr in this embodiment. By defining various predetermined states (states in which notational fluctuations cannot exist) as possible states in advance, by using the state query QR as well, the notational fluctuations in the case of text only are not affected. It becomes possible to search efficiently.

When the search unit 15 searches for the text information T (i) with the word Qt as a search query, the text information T (i) contains the word Qt (“Qt ∈ T (i)”). ) Is a hit. Further, when the search unit 15 searches the log information L (i) by the state Qr as a search query for the recognition result R (i), the state Qr of the recognition result R (i) is specified by the identifier. If even one of the specified state Qr is included in the time series of the recognition result R (i) (when "Qr ∈ R (i)"), it may be considered as a hit.

It should be noted that one or more words and states may be specified together with the search condition for each of the words Qt and the state Qr as the search query. The search condition may be specified by an arbitrary logical expression input by the user U2, such as an arbitrary logical operation such as AND, OR, or NOT, or a combination thereof. For example, it is possible to search for "text information T (i) does not include the phrase Qt" (NOT condition) and "recognition result R (i) contains two states Qr1 and Qr2" (AND condition). ..

Hereinafter, details of each part of the management system 100 that realizes the operation according to the embodiment shown in FIGS. 2 and 3 will be described.

<Log acquisition unit 21>
FIG. 4 is a functional block diagram of the terminal 20 including a functional block showing an element configuration of the log acquisition unit 21. The log acquisition unit 21 includes a photographing unit 31, a recording unit 32, a position measurement unit 33, a displacement measurement unit 34, an environmental information acquisition unit 35, and a biological information acquisition unit 36 as a configuration for acquiring various log information. It is provided with an auxiliary recognition unit 37 that performs real-time recognition processing on log information at the site where user Uw is working.

The auxiliary recognition unit 37 and the auxiliary presentation unit 22 provided in the terminal 20 are configured to assist the user Uw when performing work in the field, and the user Uw omits them in step S13 of FIG. It is also possible to perform work W (i) and acquire the log information L (i) in the log acquisition unit 21, but by using the auxiliary recognition unit 37 and the auxiliary presentation unit 22, the user Uw can use the work W (i). It becomes possible to carry out work W (i) while receiving assistance for). In particular, new work after a database information DB for a large number (N) of work W (i) (i = 1,2,3, ..., N) sufficient for searching has been constructed and registered in the registration unit 14. When the user Uw executes W (N + 1), it is possible to receive assistance utilizing a similar case of the work W (N + 1) recorded in the database information DB of the registration unit 14. (It is also possible to conduct a survey by prior search before the work in step S130 described above.) The assistance provided by the auxiliary recognition unit 37 and the auxiliary presentation unit 22 will be described later, and the log acquisition unit 21 will be described below. Each part (excluding the auxiliary recognition part 37) will be described.

The shooting unit 31 is configured to include a camera as hardware, and shoots an image when the user Uw is performing work W (i). When the terminal 20 is configured to include, for example, an HMD, the photographing unit 31 is realized by mounting a camera facing the line-of-sight direction of the user Uw so as to capture the field of view of the user Uw wearing the HMD. can do.

The recording unit 32 is configured to include a microphone as hardware, and includes the voice when the user Uw is performing the work W (i) (including the voice of the user Uw himself and other voices generated at the work site). ) Is recorded. The position measurement unit 33 is configured to include a positioning sensor as hardware, and measures the position information of the terminal 20 carried by the user Uw (that is, the position information of the user Uw who is working). Any existing method may be used for measuring the position information by the positioning sensor, and GPS (Global Positioning System), indoor GPS, a combination thereof, or the like may be used.

The displacement measuring unit 34 includes a displacement sensor (accelerometer, angular velocity sensor, orientation sensor, etc.) as hardware, and measures the displacement information of the terminal 20 and / or the user Uw carrying the terminal 20. When the displacement measuring unit 34 measures the displacement information of the user Uw, the displacement information is measured as the movement of the user Uw for each body part (for example, head, right hand, left hand, etc.) of the user Uw by an arbitrary existing motion sensor. You may do so.

The environmental information acquisition unit 35 is configured to include various environmental information sensors that acquire environmental information such as illuminance, atmospheric pressure, temperature, humidity, noise, and odor as hardware, and all or part of these various environmental information. To get. This environmental information relates to the environment of the work site where the user Uw carrying the terminal 20 exists.

The biological information acquisition unit 36 is configured to include various biological information (vital data) sensors as hardware, and the body temperature, heart rate, pulse rate, respiratory rate, and the body temperature, heart rate, pulse rate, and respiratory rate of the user Uw who is carrying the terminal 20 for work. Acquire all or part of various biological information such as sweating amount and brain waves. Any existing biometric information sensor may be used, and the biometric information of the user Uw can be acquired by the user Uw wearing it as a wearable sensor.

In step S13 of FIG. 3, the log acquisition unit 21 has a configuration in which all or only an arbitrary part of each of the above units 31 to 36 is provided, so that the video and audio when the user Uw is performing the work W (i) in the field. , Position information, displacement information, environmental information, and any part of biometric information are acquired as log information L (i) on the time series along the progress of work W (i), and the log of the management device 10 Send to reception department 12. This transmission may be performed in real time along with the progress of work W (i), or during work W (i), log information L (memory, etc.) in a storage mechanism (memory, etc.) (not shown) in the terminal 20 Each time i) is acquired, it may be saved sequentially and transmitted after work W (i) is completed.

<Learning for recognition processing in recognition unit 13>
As described above, the recognition unit 13 recognizes the time-series log information L (i) along with the time progress of the work W (i) received by the log reception unit 12 using a pre-learned model. By applying, the recognition result R (i) of the time series along the time progress of the work W (i) is obtained.

In one embodiment, this model learning individually infers a model of the recognition result R (i) related to a predetermined attribute from a predetermined combination of log information L (i) composed of various types of information. Teacher data may be given to each for learning.

For example, from the video and audio of the log information L (i) and all or part of the sensor data related to the body movement and living body of the user Uw, the user's operation content and work status (event) of the recognition result R (i). A first model for inferring such things (for example, "walking", "stopping", "during cable exchange") at each time t may be learned and constructed with the corresponding teacher data. .. Further, for example, the working environment (for example, "normal" or "abnormal") in the recognition result R (i) is inferred at each time t from the temperature and / or humidity which is the environmental information in the log information L (i). The second model may be learned and constructed given the corresponding teacher data. When the recognition unit 13 recognizes, the first model obtains the recognition result of the user's operation content and work status at each time t, and the second model obtains the recognition result of the work environment at each time t. The obtained one can be obtained as the recognition result R (i). Separate recognition result identifiers are defined in advance for the first model and the second model, and when the user Us searches with the search unit 15, one or more identifiers in the first model and a first model are used. One or more identifiers in the two models, and / or one of them, can be specified as the state Qr of the search query.

In another embodiment, all of the log information L (i) composed of various kinds of information is input, and a single model for obtaining the recognition result R (i) at each time t is given with the corresponding teacher data. You may try to learn and build after leaving it. For example, when the log information L (i) is composed of five types of video, audio, sensor data related to the body movement and living body of the user Uw, temperature and humidity, the whole recognition result R (i) is taken as input. May be inferred. For example, the user's operation content and work status are "walking", "stopping", and "cable replacement", and the working environment is "normal" or "abnormal". As a combination of these, 3 When there can be x2 = 6 types of recognition results, it may be inferred which of these 6 types is applicable.

In any of the embodiments, model learning can be performed after giving teacher data by any existing method using machine learning such as a neural network. At this time, examples of labels given as recognition results of individual events during work in the teacher data are, for example, "walking", "stopping", and "cable" regarding the user's operation content and work status as illustrated above. It is "under replacement" or the like, and is, for example, "normal" or "abnormal" about the working environment. The identifiers that make up the recognition result R (i) on the time series are made up of the identifiers that correspond to the recognition results of this individual event at each time t.

The text of the plan, approval, and report regarding the work received by the text reception unit 11 is not always described, and even for small individual events during the work, a text description is given in advance as a learning label in the teacher data to deal with it. By defining the identifier to be used, in the present embodiment, it is possible to ensure that the state Qr (specified by the identifier) by the search unit 15 is hit in the search as the search query. The text description of the individual event at the time of this learning may be used for the description in the menu or the like for explaining the state QR corresponding to the identifier, which is provided to the user Us searched by the search unit 15.

When inferring with a neural network or the like, a likelihood vector in which the likelihood corresponding to the individual event given by the individual learning label is listed as the value of each element is obtained as the inference result, but the recognition unit 13 recognizes it. When doing so, the label content corresponding to the maximum likelihood may be used as the recognition result. For example, a three-dimensional likelihood vector when recognizing time t, which is a model for inferring which of the three types of user's operation content and work status is "walking", "stopping", and "cable replacement". When (0.2,0.1,0.7) is obtained, the recognition result at this time t may be "during cable replacement", which gives the maximum likelihood of 0.7. If none of the likelihoods exceeds a predetermined threshold, the recognition result text at this time t may not be given as no recognition result.

Hereinafter, supplementary explanations and various additional or modified embodiments of the present invention will be described.

<1 ... Presentation of search results by the presentation unit 16 when searching in step S2>
As described above, the search result by the search unit 15 is output to the presentation unit 16, and the user Us is searched for in the following manners (1) to (4) by interactive processing with the user Us. The results are shown.
(1) In the presentation unit 16, the search result list {D (i)} as a summary of the search results is displayed to the user Us. In the display of this search result list {D (i)}, only the matter name, date and time, place, etc. of the work W (i) may be displayed as a list, and further, the text information T (i) and the recognition result R ( It may include a part of i) (eg, the writing part of the text information T (i)) or a summary.
(2) The user Us inputs the designation of the work W (i) whose details are to be confirmed from the displayed search result list {D (i)} to the presentation unit 16. (For the sake of explanation, it is assumed that work W (i) is specified.)
(3) The presentation unit 16 provides the user Us with detailed work data D (i) = (T (i), L (i), R (i)) regarding the work W (i) specified for the user Us. Present. At this time, the hit portion of the search query Qt and Qr may be presented with an editing effect such as highlighting.
(4) User Us confirms the individual contents of the presented detailed work data D (i) = (T (i), L (i), R (i)) together with the hit points of the search queries Qt and Qr. ..

FIG. 5 is a diagram showing an example of a screen display DP (i) in which detailed work data D (i) of the work W (i) is presented to the user Us by the presentation unit 16 of the above (3). Here, as an example of the text information T (i) presented, an example of the work plan text T1 (i) is shown, and the order of the work status and the required time thereof are described in a table format, and the work is simple. The schedule is abbreviated. (Actually, the corresponding work approval text T2 (i) and work report text T3 (i) are also displayed to the user Us, but they are omitted in FIG. 5.) The example of FIG. 5 is a log. This is an example of the case where the information L (i) is composed of the video shot by the shooting unit 31, and the recognition result as an identifier on the time axis for this video on the seek bar SB along the time axis of this video. Is shown as sections R1, R2, R3, and texts indicating the contents of specific examples corresponding to the identifiers in the recognition result R (i) are given in balloon form as explanatory displays e1, e2, e3, respectively. This text may be the same as the text explaining the state corresponding to the identifier in the menu or the like for designating the identifier when the user Us searches in the search unit 15 as the state search query QR.

In the example of FIG. 5, the user Us uses the phrase search query Qt to indicate the work status as "installation of a new server" and the same content as "installation of a new server", which means the search query Qr according to the state specified by the identifier. As the screen display DP (i) for showing the work data D (i) as the search result when the search condition is "OR", the presentation unit 16 is the hit location of this search query Qt, Qr. The "Installation" section (a part of the text information T1 (i) and a part of the recognition result R (i), the explanation display e2 for the section R2) is highlighted. In the example of FIG. 5, the search query Qt = "installation of a new server" by chance is hit in the text information T1 (i), and the search query Qr = "installation of a new server" means the same state. Is an example of a hit in the recognition result R (i), but it is possible that only one of them is hit by "OR search". For example, in the example of FIG. 5, it is assumed that the search query Qt = "cable exchange" and the search query Qr = "cable exchange" (actually specified by an identifier) of the same content are searched by the search condition "OR". Then, the work plan text T1 (i) does not hit, and only the explanation display e3 corresponding to the section R3 of the recognition result R (i) is displayed as a hit. In this way, even if the description such as "cable replacement" does not exist in the text information T (i) as already explained, the work W (i) to which "cable replacement" corresponds is performed via the recognition result R (i). It is possible to make it a hit.

In the present embodiment, the text information T (i) in the work data D (i) = (T (i), L (i), R (i)) of the search result via the presentation unit 16 in (4) above. Not only that, the log information L (i) corresponding to the recognition result R (i) is also provided to the user Us so that it can be referred to. In the example of FIG. 5, when the log information L (i) is configured to include a video, a seek bar SB is provided as an interface for providing the log information L (i) so that it can be referred to, and a section R1 which is a recognition result on the time axis , R2, R3 are shown with the text as the corresponding recognition result R (i) linked as explanatory displays e1, e2, e3.

In order to confirm the log information L (i) as a video, the user Us specifies, for example, the section R2 on the seek bar SB or the explanatory text e2 thereof (specified by clicking with the mouse cursor, etc.), so that the presentation unit 16 can display the section R2. The video section R2 in which the corresponding "installation of a new server" is captured is played back as a moving image display mv2, and the user Us can view and check this moving image display mv2. Further, when the voice recorded by the recording unit 32 is also included in the log information L (i), the recorded voice is displayed when the video shot by the shooting unit 31 is displayed as the moving image display mv2 by specifying the section R2. May also be played.

In the example of FIG. 5, when the log information L (i) includes video and audio, the log information L (i) associated with the recognition result R (i) of the work status recognized from the log information L (i) is used. This is an example in which the presentation unit 16 provides the text information T (i) to the user Us so that it can be referred to, and the log information L (i) provides information such as various sensor information other than video and audio (shown in FIG. 4). Even when the information is configured to include the information acquired by each unit 33 to 36 in the log acquisition unit 21, the information is common in time series, so the interface is similar to that of the seek bar SB shown in FIG. The log information L (i) and the recognition result R (i) may be provided to the user Us together with the text information T (i) so that they can be referred to.

FIG. 6 shows an example of an interface that provides the log information L (i) as a graph GR so that the user Us can refer to it, as an example corresponding to the example of FIG. For example, when the environmental information acquisition unit 35 acquires one-dimensional time-series data such as temperature, the log information L is obtained by such a graph GR (in this example, the horizontal axis represents time and the vertical axis represents the acquired data value). (i) can be provided to the user Us for reference. Therefore, the vertical axis of the graph GR may be given a scale of acquired data values, and the horizontal axis may be given a scale of time, and the corresponding numerical values may be given and presented. Further, when the user Us moves the mouse cursor to a data point on the line of the graph GR and specifies the data point, the graph value at the position of the point (time on the horizontal axis and acquired data value on the vertical axis). May be presented in the form of a pop-up display or the like. For multidimensional time-series data, a plurality of such graph GRs may be provided, or the change behavior of each dimension of the time-series data may be listed in the graph GR.

<2 ... Utilization of information on management system 100 and implementation of work assistance using this information>
In FIG. 2, it is assumed that the database information DB of the registration unit 14 is constructed in step S1 and then the search is performed in step S2, and as a flow for preparing each work data D (i) to be registered in this step S1, FIG. Explained. When trying to perform a new work W (N + 1) after the database information DB = {D (i) | i = 1,2,3,…, N} has been constructed over a large number of N cases. Uses these (step S2 in FIG. 2 and the entire flow in FIG. 3) in cooperation with each other, and assists the database information DB in performing this work W (N + 1) in the actual work. It is possible to use it as content for.

That is, when performing a new work W (N + 1), the flow of FIG. 3 is carried out for this work W (N + 1), and text information T (N + 1) is prepared as a business document. By actually performing work W (N + 1) to acquire log information L (N + 1) and obtaining recognition result R (N + 1), the database information DB can be converted to this work data D (N + 1). It may be updated by adding, and the updated database information DB = {D (i) | i = 1,2,3,…, N, N + 1} may be obtained.

In addition, when performing a new work W (N + 1), work plan, approval and report texts T1 (N + 1), T2 (N + 1), T3 (N + 1) in steps S11, S12 and S16. When the user inputs to the text reception unit 11, the user searches for a work case similar to the work W (N + 1) by performing the search in step S2 as steps S110, S120, S160. By doing so, it is possible to use it as a reference for writing appropriate sentences, and by grasping the precautions (such as troubles) in similar work, for example, the actual work W (N +) It is possible to work consciously to avoid similar troubles when facing 1). Similarly, in order to carry out the work of step S13, the user can search the work case in step S130 and obtain knowledge.

In other words, by using the management system 100 as described above, the knowledge of past work can be easily referred to, and new people who are unfamiliar with the relevant work or approval work can engage in actual work or approve it. Even if this is the case, efficient succession will be possible.

When actually executing a new work W (N + 1), what is selected by the user Us from the search results related to the searched similar work is transmitted from the presentation unit 16 to the auxiliary presentation unit 22 of the terminal 20 as work auxiliary content. However, it may be displayed on the HMD, for example, as work auxiliary content when the user Uw performs the work in the field. In addition, instead of the user Uw, the on-site user Uw remotely inputs the search queries Qt and Qr to the management device 10 via the Internet, LAN, etc., and selects and inputs the desired case to assist the work. It may be used as content.

Further, when the work W (N + 1) is performed at the site, the log information L (i) acquired by each unit 31 to 36 shown in FIG. 4 is read and recognized by the auxiliary recognition unit 37 in real time at each time t. By doing so, the work status information at time t is obtained as a recognition result and output to the auxiliary presentation unit 22, and when the auxiliary presentation unit 22 performs the auxiliary information according to the work status information at the site, the user Uw performs the work. As the work auxiliary content of, for example, it may be displayed on the HMD.

The fact that the auxiliary recognition unit 37 obtains the work status information from the log information in real time is learned in advance using the teacher data in the same way that the recognition unit 13 obtains the recognition result R (i) from the log information L (i). This can be done by using a learning model. The entire model learned by the recognition unit 13 may be used as it is in the auxiliary recognition unit 37, or only a part of the model may be used. For example, the recognition unit 13 learns including "trouble" such as an accident in the individual event to be recognized, and the auxiliary recognition unit 37 outputs the likelihood of this "trouble" in chronological order, and the likelihood value. When the threshold value is exceeded, the presentation unit 16 may make a presentation (video display or audio output) that calls attention to the trouble, such as "need attention".

<3-1> As the search conditions specified by the state query QR for the search unit 15, the order in which a plurality of state queries Qr occur (the order in which they appear in time series) is specified, and the duration of each of the one or more state Qrs. It is also possible to specify the search conditions including the specification of, and thereby perform the search in the search unit 15. For example, it may be specified that the three states Qr1, Qr2, and Qr3 appear in this order, and at this time, the search may be performed including the condition of allowing / not allowing another state to appear in between. (For example, if it is allowed, the work including the state time series of "Qr1 → Qr4 → Qr2 → Qr3" will be the search result, but if it is not allowed, it will not be the search result.) Regarding the duration of the state, for example, the state Qr (cable) It is possible to specify that the status related to related work) is "continue for 10 minutes or more".

<3-2> The examples of FIGS. 5 and 6 are examples of confirming a specific section of time-series log information by directly specifying the searched user Us. The presentation unit 16 presents a plurality of sections composed of log information on the time series (the type of log information (for example, video) is specified by the user) for a certain operation included in the search result. At that time, a plurality of sections are presented in a predetermined order according to a predetermined standard in the presentation unit 16 without being directly specified by the user (for example, in a browser display that scrolls from top to bottom, the one with the earliest order is higher. It may be presented by arranging and arranging them). The order of presenting the sections may be, for example, the order of priority of the sections. As for the priority of the interval, the value estimated separately from the state identifier by the recognition unit 13 may be used in the same manner as the “trouble” likelihood estimated by the auxiliary recognition unit 37 as a time series.

<3-3> The presentation unit 16 presents the log information to the user Us so that it can be referred to. However, since it is generally assumed that the log information is enormous, the objects that can be presented are selected in advance. As described above, it may be used as a candidate for the target to be specified by the user or arranged in the order determined by the presentation unit 16. The criteria for pre-selecting the objects that can be presented are the threshold judgment for the priority of the section used when deciding the order of presenting the above sections and the judgment for the duration of the section (for example, the section judged to be short is from the presentation target. Judgment to exclude) may be used. If the size of the log information is determined to be small by the threshold value determination, this selection may be omitted.

<3-4> In the registration unit 14, the text information T (i) and the log information L () are set as data D (i) = (T (i), L (i), R (i)) for each work W (i). i) and the recognition result R (i) are stored, but only the text information T (i) (a part of) may be stored. Specifically, if there is a work W (i) in which step S11 in FIG. 3 is at least completed, but steps S13 and subsequent steps are not completed, the data D (i) is stored in such a state. Sometimes. (Even in such work W (i), the state query Qr cannot be used and only the phrase query Qt can be used when searching, but it is possible to search only for the text information T (i). is there.)

<4> The text reception unit 11 receives the text information T1 (i), T2 (i), T3 (i) of the plan, approval, and report of the work W (i) in steps S11, S12, and S16, and texts are received by these. Information T (i) = (T1 (i), T2 (i), T3 (i)) is assumed to be composed, but this is an example of preparing text information for business, and text information T (i) ) Can be accepted as input from the user, and the mode, format, and individual content may be any other. For example, in the example of the work plan text T1 (i) of FIG. 5, the work schedule is included, but the work schedule is described in a separate procedure manual or the like, and the worker may refer to this when working. If it is an agreement, the detailed schedule may be omitted.

<5> With reference to FIG. 2, it is assumed that the user Us performs the search in step S2 at an arbitrary timing after the database of a scale sufficient for the search is constructed in the registration unit 14, but depending on the content of the search query. The corresponding data may not be registered in the registration unit 14. In this case, the search unit 15 may return to the user Us that there is no data that hits the search query, for example, by a predetermined message such as "No applicable matter".

<6> In the flow of FIG. 3, the order from step S13 for work execution and log acquisition to step S17 for registration is assumed to be "S13-> S14-> S15-> S16-> S17", but step S13 is completed. Later, since step "S14-> S15" (log input and recognition) and step S16 (work report text input) can be performed independently, the order of these can be changed to "S13-> S16-> S14-> S15-> S17". It may be carried out in the order of, or it may be carried out in parallel as "S13-> (" S14-> S15 "and S16)-> S17".

<7> In the process of obtaining the recognition result R (i) from the log information L (i) in the recognition unit 13, instead of or in addition to the method using the learning model already described, the above-mentioned Patent Documents 3 to 7 All or part of the method may be used. When the auxiliary recognition unit 37 obtains the work status information as the work risk degree, it may be obtained as the likelihood of "trouble" by the method using the learning model already described, or the above-mentioned Patent Documents 3, 4 and 4 may be obtained. All or part of the methods 9 and 10 may be used. As the method of presenting the search result in the presentation unit 16, all or a part of the methods of the above-mentioned Patent Documents 4, 8 and 9 may be used.

<8> As a method for further narrowing down the search results by the search queries Qt and Qr (performing an AND search) in accepting a search from a user in the search unit 15, an image (each in the video) as in the above-mentioned Patent Document 11 The frame image at time t) and its exif information (image shooting date / time, shooting position, shooting direction, camera information, shooting condition information, etc.) may be used.

<9> FIG. 7 is a diagram showing a hardware configuration in a general computer device 70, and each of the management device 10 and the terminal 20 in the management system 100 is one or more computer devices having such a configuration. It is feasible as 70. The computer device 70 is a CPU (central processing unit) 71 that executes a predetermined instruction, and a dedicated processor 72 (GPU (graphic calculation device)) that executes a part or all of the execution instructions of the CPU 71 on behalf of the CPU 71 or in cooperation with the CPU 71. And deep learning dedicated processor, etc.), RAM73 as the main storage device that provides a work area for the CPU71 and the dedicated processor 72, ROM74 as the auxiliary storage device, communication interface 75, display 76, camera 77, mouse, keyboard, touch panel, etc. It includes an input interface 78 that accepts user input, a microphone 79, sensors 80 composed of one or more types of sensors, a processor 81, and a bus BS for exchanging data between them. The shooting unit 31 is a camera 77, the recording unit 32 is a microphone 79, and each unit 33 to 36 that acquires log information other than video and audio in the log acquisition unit 21 is a corresponding dedicated sensor included in the sensors 80. It can be realized. Both the presentation unit 16 and the auxiliary presentation unit 22 can be realized by the display 76 and the speaker 81.

Each part of the management device 10 and the terminal 20 can be realized by a CPU 71 and / or a dedicated processor 72 that reads and executes a predetermined program corresponding to the function of each part from the ROM 74. Here, when the shooting-related processing is performed, the camera 77 further operates in conjunction with each other, and when the video viewing processing is performed, the display 76 and the speaker 81 further operate in conjunction with each other to transmit and receive data. When the communication-related processing related to is performed, the communication interface 75 further operates in conjunction with it. Further, when the log acquisition unit 21 acquires various log information, the camera 77 operates in conjunction with the video, for example, as described above.

<10> The log acquisition unit 21 is realized by a sensor or the like provided in the terminal 20 configured as a wearable terminal, but all or part of the log information is separately from the terminal 20 installed at the work site. It may be acquired by sensors provided in the device or equipment, and the log acquisition unit 21 may receive the log information acquired by the external sensors by communication. For example, the temperature and humidity may be acquired from a thermometer or a hygrometer as a device separate from the terminal 20 installed at the site. As the image acquired by the photographing unit 31, an image taken by a surveillance camera or the like installed at the site as a device separate from the terminal 20 may be used.

100 ... management system, 10 ... management device, 20 ... terminal
11 ... Text reception department, 12 ... Log reception department, 13 ... Recognition department, 14 ... Registration department, 15 ... Search department, 16 ... Presentation department
21 ... Log acquisition department, 22 ... Auxiliary presentation department

Claims (9)

For each of the plurality of tasks, a registration unit that stores the text information described about the task and the log information acquired when the task is performed in association with each other.
A search unit that searches the registration unit using a search query input by the user and outputs a search result for a work in which the search query hits at least one of the text information or log information associated with the work. A management device characterized by being provided with. The search unit searches the user for a search input in which both or one of the first search query for the text information associated with the work and the second search query for the log information associated with the work is specified. The management device according to claim 1, which is configured to be acceptable as an input and outputs a search result for a work that hits the search input. In the registration unit, the text information and the log are obtained by applying the recognition process to the log information on the time series acquired when the work is performed, and the recognition result regarding the individual event on the time series. I remember it by linking it to the work together with the information.
The management device according to claim 1 or 2, wherein the search unit examines whether or not the corresponding recognition result hits the search query when searching the log information. In the search unit, when the search is performed and the search result for the work in which the search query is hit is output, the text information and the log information associated with the work are output so that the user can refer to them. The management device according to any one of claims 1 to 3. The log information is acquired as log information on a time series when the work is performed and stored in the registration unit.
A presentation unit that presents the search results output by the search unit to the user is further provided.
The management device according to any one of claims 1 to 4, wherein the presenting unit presents the log information and information about the work status recognized from the log information to the user together with the text information. .. The log information is acquired as log information on a time series when the work is performed and stored in the registration unit.
A presentation unit that presents the search results output by the search unit to the user is further provided.
In the work included in the search result, the presenting unit extracts in advance the priority and / or the information of the section in which the work status specified by the user occurs in the log information on the time series. The management device according to any one of claims 1 to 5, wherein the duration of the section satisfies a predetermined condition. In the registration unit, the text information and the log are obtained by applying the recognition process to the log information on the time series acquired when the work is performed, and the recognition result regarding the individual event on the time series. I remember it by linking it to the work together with the information.
In the search unit, as a search query specification for the log information, it is specified that one or more specified types of events among the recognized individual events appear in a specified order, and / or the recognized individual event. The management device according to any one of claims 1 to 6, wherein the designation of the duration of one or more designated events among the individual events is received and searched. The management device according to any one of claims 1 to 7, wherein the log information includes video, audio, or one or more types of sensor information collected during work. A program characterized in that the computer functions as the management device according to any one of claims 1 to 8.

Images