JP2018116571A - Management system, management method, and management program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a management system, management method, and management program for efficiently recognizing a location position of a management object and performing management.SOLUTION: A management server 20 of a management system 10 includes: a location information storage section 27 for storing a management object identifier for identifying a management object and positional information of the management object in association with each other; and a control section 21 for acquiring an image from a mobile unit 11 attached to a mobile body. The control section 21 acquires an image photographed by the mobile unit 11 and photographic position information, specifies a location position using the photographic position information, for the management object included in the image, and records the location position in the location information storage section 27.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a management system, a management method, and a management program for managing the location of a management target.

The equipment used at the construction site varies depending on the work situation. For this reason, it is necessary to accurately manage the location of each device.
Therefore, conventionally, a technique for managing the status of materials and equipment has been studied (for example, see Non-Patent Document 1). In the technique described in Non-Patent Document 1, an IC tag attached to an IC tag or equipment carried by a construction worker is read by an IC tag reader installed in an elevator. Thereby, the moved equipment and materials, the moving time and the moving destination are recorded.

  Furthermore, a technique for managing the operating status of equipment used at a construction site has been studied (for example, see Patent Document 1). In the technique described in Patent Document 1, the state of a device is determined using measurement information acquired from an acceleration sensor attached to the device.

JP2013-54682A

Obayashi Corporation, “Developing a“ lifting management system ”that shows the loading status of materials and equipment”, [online], [October 14, 2016 search], Internet <http://www.obayashi.co .jp / press / news20160328_1>

  When managing equipment using the technology described in Non-Patent Document 1, it is necessary to attach an IC tag to each of the equipment to be managed. Therefore, it is impossible to manage materials and equipment that are not attached with IC tags.

  The present invention has been made in view of the above problems, and an object thereof is to provide a management system, a management method, and a management program for efficiently grasping and managing the location of a management target.

  A management system that solves the above problems includes a management information storage unit that stores management target identification information that identifies a management target and position information of the management target in association with each other, and an image from a photographing device attached to a moving body. A control unit that acquires an image captured by the image capturing device and image capturing position information, and acquires the image capturing position information for a management target included in the image. The location is specified, and the location is recorded in the management information storage unit in association with the management target identification information of the management target. Accordingly, the location of the management target can be efficiently grasped and managed using the moving body imaging device.

  In the management system, the control unit identifies a usage state of a management target based on the image, and records the usage state in the management information storage unit in association with the management target identification information and the location. It is preferable. Thereby, it is also possible to grasp the usage state of the management target.

  -In the management system, further comprising a moving body information storage unit for recording the location of the moving body, the control unit specifies the shooting position where the image information was captured as the location of the moving body, It is preferable to record the location of the moving body in the moving body information storage unit. Thereby, not only the location of the management target but also the location of the moving body can be grasped and managed together.

  In the management system, when the location of the moving body is included in the imaging prohibited area, the imaging apparatus preferably stops imaging. As a result, when the moving body enters the photographing prohibited area, photographing can be prohibited.

  According to the present invention, the location of a management target can be efficiently grasped and managed.

1 is an overall configuration diagram of a management system in an embodiment. The perspective view which shows the attachment position of the mobile body unit in embodiment. It is a block diagram of the information memorize | stored in each memory | storage part of the management system in embodiment, Comprising: (a) is a to-be-photographed-information storage part, (b) is a moving image information storage part, (c) is a location information storage part. The flowchart explaining the process sequence of the management process in embodiment. 6 is a flowchart for explaining a processing procedure of subject recognition processing in the embodiment. Explanatory drawing explaining the image which identifies the management object in embodiment. The flowchart explaining the process sequence of the location confirmation process in embodiment.

  Hereinafter, an embodiment in which a management system, a management method, and a management program for managing the location of a management target are embodied will be described with reference to FIGS. In this embodiment, movable equipment (for example, a welding machine, a pump, etc.) used at a construction site is managed as a management target.

  As shown in FIG. 1, the management system 10 of this embodiment includes a management server 20 connected to a plurality of mobile units 11.

  As shown in FIG. 2, a wearable terminal that functions as a photographing device is used for the mobile unit 11. This mobile unit 11 is attached to the side of the helmet H1 of the worker who works while moving at the construction site.

As shown in FIG. 1, the mobile unit 11 includes an imaging unit 12, a transmission unit 13, and a control unit 14.
The photographing unit 12 is constituted by a CCD camera or the like. The photographing unit 12 photographs the front of the worker and generates a moving image (image).

The transmission part 13 transmits the moving image image | photographed with the imaging | photography part 12 to the management server 20 for every predetermined capacity | capacitance.
The control unit 14 controls the start and stop of shooting of the shooting unit 12 and controls the start and stop of transmission of the transmission unit 13. The control unit 14 holds imaging device identification information that identifies the mobile unit 11.

  The management server 20 includes an input unit 16 and a display unit 17. The input unit 16 is a keyboard or a pointing device. The display unit 17 is a display and displays the location of the management target.

  The management server 20 includes a control unit 21, a general building information storage unit 22, a feature point information storage unit 23, a prohibited area information storage unit 24, a subject information storage unit 25, a moving image information storage unit 26, and a location information storage unit 27. Yes.

  The control unit 21 includes a CPU, a RAM, a ROM, and the like, and performs processes described later (each process such as a management stage, a position specifying stage, and a subject specifying stage). By executing the management program for that purpose, the control unit 21 functions as a management unit 211, a position specifying unit 212, and a subject specifying unit 213.

The management unit 211 executes processing for managing the location of the management target. In the present embodiment, a moving image taken by the mobile unit 11 is acquired, and management targets are managed by this moving image.
The position specifying unit 212 estimates the shooting position and shooting direction based on the moving image, and executes a process of specifying the shooting field of view. In the present embodiment, the position specifying unit 212 specifies a shooting position using known v-SLAM (visual Simultaneous Localization And Mapping).

  The subject specifying unit 213 executes processing for specifying the management target of the subject (object) within the specified field of view and the state of the subject. The subject specifying unit 213 of the present embodiment uses a deep learning convolutional neural network. Specifically, the subject specifying unit 213 inputs the image of the divided region of the shooting frame to the learned classification model generated by learning, and specifies the management object and state or material. For this learning, the subject specifying unit 213 uses the large amount of teacher data stored in the subject information storage unit 25 based on a publicly known technique to store the learned classification model learned by deep learning. 25.

  The general building information storage unit 22 stores general building information related to the building under construction. In this embodiment, BIM (Building Information Modeling) data is used as building general information. This general building information includes an object, attribute information, and location information constituting the building. An object is constituted by a three-dimensional model to which an object ID is assigned. The attribute information includes information regarding specifications such as object ID, object name, object type, area, material, product information, and cost. The location information includes information regarding the position (coordinates) where this object is placed.

The object ID is an identifier for specifying the object.
The object name is the name of this object.
The type, area, material, product information, and cost are the type (structure type and joinery type), area, material, product information, and cost of this object.

  In the location information, data relating to an identifier for specifying a place (a work area) where this object is placed is recorded. A work zone is a group that summarizes multiple dwelling units (rooms) in a building in terms of space (adjacent dwelling units) and time (working time) when proceeding with the construction work of the building in stages. For example, a work zone such as each floor of a building or a part of a floor of a floor is determined.

  The feature point information storage unit 23 stores information on feature points at the construction site for specifying the location (imaging position) of the mobile unit 11. This feature point information is information related to a portion that is a feature of an image when taken from the inside of a building under construction. The feature point information includes a feature point identifier, a feature amount of an image that specifies the feature point, and a position (xyz coordinate) of the feature point.

  The prohibited area information storage unit 24 stores prohibited area information for determining a prohibited area for photographing. This prohibited area information is registered in advance before execution of a management process described later. The prohibited area information includes data related to the prohibited area identifier, area name, and position (coordinates) regarding the prohibited area. The photographing prohibited area is, for example, a toilet used by an operator.

The subject information storage unit 25 stores information used for specifying the subject and determining the state of the subject. These pieces of information are registered in advance before execution of management processing described later.
Specifically, as illustrated in FIG. 3A, the subject information storage unit 25 stores a learned classification model 251, teacher data 252, and individual recognition data 255.

  In the learned classification model 251, a learned classification model generated by deep learning for specifying a management target and a state using the teacher data 252 is recorded. The learned classification model 251 is associated with the management target identifier and the state identifier.

  In the management object identifier data area, data relating to an identifier (here, a name) for specifying the type of an object is recorded. Here, the kind of an object is a building element (for example, building material or accommodation) or material used for construction, or a device (for example, a welding machine or a pump) used at a construction site.

  An identifier for specifying the state of the subject specified by the management target identifier is recorded in the state identifier data area. For example, when the subject is a welder, an identifier indicating use or stop is recorded.

The teacher data 252 is teacher data used to calculate a deep learning learned classification model. The teacher data 252 includes data relating to a management target identifier used as an output layer, a state identifier, and a learning image used as an input layer.
In the management object identifier data area and the state identifier data area, data relating to an identifier for identifying the type of the management object (thing) and an identifier for identifying the state of the management object are recorded.

  The individual recognition data 255 is data used for recognizing an individual in order to identify the subject as the same kind of object (building element or equipment). The individual recognition data 255 includes data related to a management target identifier, an individual identifier, and individual identification information.

In the management object identifier data area, data relating to an identifier for specifying the type of an object to be managed (for example, a welder or a pump) is recorded.
In the individual identifier data area, data relating to an identifier for individually specifying each management target is recorded.

  In the individual identification information data area, information for distinguishing this management target from other individuals in the same object is recorded in the image. This individual specifying information is, for example, information on the color of the colored marker assigned to the management target, the position in the management target, and the like.

  As shown in FIG. 3B, the moving image information storage unit 26 stores moving image management information 260 related to moving images taken by the mobile unit 11. This moving image management information 260 is composed of data relating to a moving image identifier, a moving image, shooting device identification information, shooting date and time, shooting position, and shooting direction.

Data relating to an identifier for identifying each moving image is recorded in the moving image identifier data area.
In the moving image data area, moving images shot by the mobile unit 11 are recorded.
In the photographing device identification information data area, the photographing date / time data area, the photographing position data area, and the photographing direction data area, the device (moving body unit 11) that photographed the moving image, the photographing date / time, the photographing position, and the photographing direction, respectively. Data about is recorded.

  As shown in FIG. 3C, the location information storage unit 27 stores information regarding the location of each management target. The location information 270 is registered when the management target is carried into the construction site. The location information 270 includes data on an individual identifier, a management target identifier, a location, a state, and a date and time.

In the individual identifier data area, data relating to an identifier for specifying each management target is recorded.
In the management target identifier data area, data relating to an identifier for specifying the type of management target is recorded.

In the location data area, data relating to the position where the management target is detected is recorded.
In the status data area, data relating to the status of the management target is recorded.
In the date / time data area, data related to the date and time when the management target is detected at the location is recorded.

(Management process)
Next, a management process for managing a management target using the management system 10 configured as described above will be described with reference to FIG.

  When starting work at the construction site, the worker activates the mobile unit 11 attached to the helmet H1. In this case, the control unit 14 of the mobile unit 11 starts shooting a moving image in the shooting unit 12 and records it in the memory. In this case, the moving image identifier and the shooting date and time are recorded in the memory in association with the moving image. And the control part 14 transmits the moving image recorded on memory with the imaging device identification information to the management server 20 for every predetermined capacity via the transmission part 13. FIG.

  The control part 21 of the management server 20 performs the acquisition process of the moving image from the mobile body unit 11 (step S1-1). Specifically, the management unit 211 of the control unit 21 receives a moving image having a predetermined capacity from the mobile unit 11 together with the imaging device identification information. And the management part 211 produces | generates the moving image management information 260 containing the received moving image, a moving image identifier, imaging | photography date, and imaging device identification information, and records it on the moving image information storage part 26. FIG.

  Next, the control part 21 of the management server 20 performs a position estimation process (step S1-2). Specifically, the position specifying unit 212 of the control unit 21 specifies, as candidate feature points, feature points that may appear in the current frame among the feature points used in specifying the latest position. Here, the position specifying unit 212 predicts the moving direction and moving speed of the worker using a plurality of pieces of frame information, and specifies candidate feature points based on the moving direction and moving speed. Then, the position specifying unit 212 searches for a feature point that matches the feature amount of the candidate feature point in the current frame (image). When a matching feature point is specified, the shooting position and shooting angle are specified using the position (xyz coordinates) of the specified feature point, and recorded in the moving image management information 260. The position specifying unit 212 specifies a shooting range (shooting field of view) from these shooting positions, shooting angles, and images.

  Next, the control unit 21 of the management server 20 executes a determination process as to whether or not it is a photographing prohibited area (step S1-3). Specifically, the management unit 211 of the control unit 21 compares the estimated shooting position with the position (coordinates) of the prohibited area information in the prohibited area information storage unit 24, and whether the prohibited area includes a shooting position. Determine whether or not.

  When it is determined that the shooting position is within the shooting prohibited area (“YES” in step S1-3), the control unit 21 of the management server 20 deletes the moving image acquired from the mobile unit 11. Then, the control unit 21 waits for reception of the next moving image.

  On the other hand, when it is determined that the shooting position is outside the shooting prohibited area (“NO” in step S1-3), the control unit 21 of the management server 20 executes subject recognition processing (step S1-4). Here, the subject specifying unit 213 of the control unit 21 specifies the subject (object) included in the frame, the location of the subject, information for identifying the subject (object ID or individual identifier), and the like. Details of the subject recognition processing will be described later.

  Next, the control part 21 of the management server 20 performs the information addition process to a building comprehensive information storage part (step S1-5). Specifically, when the subject to which the object ID is assigned is included in the frame, the management unit 211 of the control unit 21 uses the image, the shooting date and time as the attribute information of the object of the general building information corresponding to the object ID. Record the location and state of the subject.

  Next, it is determined whether or not there is a management target whose position information is to be updated (step S1-6). Specifically, the management unit 211 of the control unit 21 determines that there is a management target to be updated when the subject to which the management target identifier is assigned is included in the frame.

  Here, when it is determined that there is a management target for updating the position information (in the case of “YES” in step S1-6), the control unit 21 of the management server 20 executes a process for updating the position information of the management target ( Step S1-7). Specifically, the management unit 211 of the control unit 21 searches the location information storage unit 27 for the latest location information 270 including the management target identifier and the individual identifier specified in the frame. When the location of the corresponding location information 270 is different from the specified location, the management unit 211 generates new location information 270 and records it in the location information storage unit 27. In this case, the management unit 211 includes, in the location information 270, the individual identifier, the management target identifier, the location, the state, and the date and time (imaging date and time).

  When only the management target identifier is specified, the management unit 211 searches the location information 270 including the same management target identifier and the same location. When the corresponding location information 270 is extracted, the date and time of the location information 270 is updated to the shooting date and time of the newly specified image. If the corresponding location information 270 is not extracted, new location information 270 is generated and recorded in the location information storage unit 27.

  On the other hand, when it is determined that there is no management target whose position information is to be updated (“NO” in step S1-6), the control unit 21 of the management server 20 skips the process of step S1-7.

  And the control part 21 of the management server 20 performs the determination process of whether it left the field (step S1-8). When the worker leaves the site, the exit is input to the mobile unit 11. In this case, the control unit 14 of the mobile unit 11 determines to leave the site, and sends the exit information to the management server 20. When it is determined that the exit information is not received and the exit from the site is not made (in the case of “NO” in step S1-8), the control unit 21 of the management server 20 continues the processing after step S1-1. To do.

  On the other hand, if the exit information is received and it is determined that the exit has occurred (in the case of “YES” in step S1-8), the management unit 211 of the control unit 21 ends the management process.

(Subject recognition processing)
Next, the above-described subject recognition process (step S1-4) will be described with reference to FIG.
First, the control unit 21 of the management server 20 executes region division processing on the captured image (step S2-1). Specifically, the subject specifying unit 213 of the control unit 21 divides a plurality of regions included in each frame (image) of the captured moving image using a known method. For example, the region division is performed using the edge included in the photographed image, the hue information or the saturation information of the pixel.

And the control part 21 of the management server 20 specifies each division area as a process target sequentially, and repeats the next process.
The control unit 21 of the management server 20 executes management target and state prediction processing (step S2-2). Specifically, the subject specifying unit 213 of the control unit 21 uses the image of each divided region as an input value, and uses the learned classification model 251 stored in the subject information storage unit 25 to manage objects and states or materials. Is calculated. Here, although the management target and state cannot be specified by deep learning, such as a pillar, a wall, and a floor, if the material can be specified, the material is specified.

  Assume that the image 500 shown in FIG. 6 is acquired. In this case, the control unit 21 generates a plurality of divided regions based on the contour lines in the image. And the kind and state of management object are specified by deep learning. For example, as shown in FIG. 6, the stopped welding machine M to be managed is specified at the center position of the image. In deep learning, learning is performed using an image (a welding machine M in use) when a spark is emitted, and the use state is determined.

Next, the control unit 21 of the management server 20 sequentially identifies each subject (each management target) as a processing target, and repeats the following processing.
Here, first, the control unit 21 of the management server 20 executes a location position and individual information specifying process (step S2-3). Specifically, the subject specifying unit 213 of the control unit 21 specifies the location of the subject of the image using the shooting position and shooting direction of the image, and records it in the memory of the control unit 21. Then, the subject specifying unit 213 of the control unit 21 performs a determination process as to whether or not an object corresponding to the subject management target identifier is recorded in the architectural general information storage unit 22 at the location of the subject. Here, when the corresponding object is recorded, the object ID of this object is specified and recorded in the memory of the control unit 21.

  On the other hand, when the corresponding object is not recorded in the building general information storage unit 22, the subject specifying unit 213 of the control unit 21 searches the image of the subject for an image portion corresponding to the individual specifying information. When the individual specifying information is included in the image part, the subject specifying unit 213 specifies the individual identifier of the individual recognition data 255 in which the individual specifying information is recorded, and records it in the memory of the control unit 21.

  For example, when the welding machine M is specified in the image 500 shown in FIG. 6, it is determined whether the individual identification information of the welding machine is included in the image of the welding machine M in the image 500. Here, when the marker M1 of the individual specifying information is detected in the image of the welding machine M, the subject specifying unit 213 specifies the individual identifier corresponding to the individual specifying information.

(Location confirmation process)
Next, processing for confirming the location of the management target will be described with reference to FIG. For example, it is assumed that the worker confirms the location of a management target such as a device used for work to be started.

  First, the control unit 21 of the management server 20 executes an acquisition process of target information to be searched (step S3-1). Specifically, the management unit 211 of the control unit 21 displays a search screen on the display of the display unit 17. This search screen includes an input field to be searched and a search execution button. Here, an individual identifier or a management target identifier can be used as a search target. The operator uses the input unit 16 to input information related to the management target whose location is to be confirmed in the input field of the search screen, and selects a search execution button.

  The control unit 21 of the management server 20 executes search execution processing (step S3-2). Specifically, the management unit 211 of the control unit 21 extracts location information 270 having an individual identifier or management target identifier that matches the acquired search target information in the location information storage unit 27.

  And the control part 21 of the management server 20 performs the display process of a location (step S3-3). Specifically, the management unit 211 of the control unit 21 causes the display of the display unit 17 to display a three-dimensional model of the building that matches the current progress of the construction work. Then, the management unit 211 displays the location of the management target corresponding to the search target information on the three-dimensional model. In this case, when information related to the state is recorded in the extracted location information 270, the management unit 211 displays the information including the state related information. Thereby, the worker can grasp the location of the management target.

According to this embodiment, the following effects can be obtained.
(1) In this embodiment, the control unit 21 of the management server 20 specifies the subject included in the captured image and the location of the subject in the subject recognition process (step S1-4), and updates the position information of the management target. Is executed (step S1-7). Thereby, it is possible to efficiently grasp and manage the location of the management target using the mobile unit 11.

  (2) In the present embodiment, the control unit 21 of the management server 20 identifies the use state of the management target in the subject recognition process (step S1-4), and executes the update process of the management target position information (step S1-4). S1-7). Thereby, it is possible to efficiently grasp the usage state of the management target.

  (3) In the present embodiment, the control unit 21 of the management server 20 executes the location and individual information specifying process (step S2-3) after the management target and state prediction process (step S2-2). . Thereby, not only a subject (object) can be specified from an image, but also an individual can be managed while being distinguished from other same-type objects.

  (4) In this embodiment, the mobile unit 11 uses a wearable terminal that functions as a photographing device. This mobile unit 11 is attached to the side of the helmet H1 of the worker who works while moving at the construction site. Thereby, the location of the management target located around the operator can be grasped only by the worker moving for work.

  (5) In this embodiment, when it determines with a photography prohibited area (in the case of "YES" in step S1-3), the control part 21 of the management server 20 deletes the moving image. Thereby, it is possible to automatically delete the image in the photographing prohibited area so as not to save it.

  (6) In this embodiment, the control part 21 of the management server 20 performs the acquisition process of the target information to search, a search execution process, and the display process of a location (steps S3-1 to S3-3). Thereby, the current location of the searched management target can be displayed.

Moreover, you may change the said embodiment as follows.
In the above embodiment, the control unit 21 of the management server 20 specifies the image capturing position and angle using v-SLAM. The specification of the image shooting position and angle is not limited to processing using v-SLAM. For example, PDR (Pedestrian Dead Reckoning) may be used. In this case, the mobile unit 11 is provided with an acceleration sensor, a gyro sensor, or the like. Acceleration information (acceleration direction, magnitude, etc.) relating to acceleration applied to the acceleration sensor and the mobile unit 11 is detected in three-axis directions. The gyro sensor detects angular velocity information (angular velocity direction, magnitude, and the like) related to the angular velocity applied to the mobile unit 11 in three axis directions. And the transmission part 13 transmits the detected value of the acceleration sensor and gyro sensor at the time of imaging | photography to the management server 20 with an image.

  In the above embodiment, the control unit 21 of the management server 20 executes the position estimation process (step S1-2) and the photographing prohibited area determination process (step S1-3). These processes may be executed in the mobile unit 11. In this case, when the control unit 14 of the mobile unit 11 determines that the shooting is prohibited, shooting may be stopped. Specifically, when the control unit 14 determines that it is a shooting prohibited area, shooting is stopped for a predetermined period.

In addition, an alarm may be output that informs that it is a photographing prohibited area. This alarm causes the operator to stop shooting.
Then, when it is determined in the position estimation process that the shooting prohibited area has been left, the control unit 14 of the mobile unit 11 resumes image shooting and image transmission to the management server 20.
In addition, the moving body unit 11 may be provided with a cover member (for example, a lens cover) that opens and closes an opening for taking light into the photographing apparatus in the photographing apparatus. When the control unit 14 determines that it has arrived at the photographing prohibited area, the control unit 14 moves the cover to shield the opening.

  In the above embodiment, the imaging unit 12 of the moving body unit 11 images the front of the worker wearing the helmet H1 to which the moving body unit 11 is attached. The shooting direction is not limited to the front, but may be the rear or the left-right direction with respect to the traveling direction. Moreover, you may make it image | photograph a half circumference (180 degree | times) and a full circumference (360 degree | times).

  In the above-described embodiment, movable devices such as welding machines and pumps used at the construction site have been described as management targets. The management target is not limited to such a device. For example, the present invention can be applied to an aerial work vehicle, a scaffold, and the like. Furthermore, the management target is not limited to an object, and may be a person.

  In the above embodiment, the control unit 21 of the management server 20 identifies the location of the management target and manages this management target. In addition to this, the location (current location) of the worker wearing the helmet H1 to which the mobile unit 11 is attached may be managed. In this case, an operator identification table for specifying an operator is stored in the control unit 21. Furthermore, the management server 20 includes an operator information storage unit that stores the location of the operator. In this worker identification table, imaging device identification information for identifying the mobile unit 11 and worker identification information for identifying the worker are associated with each other. This worker uses the helmet H1 to which the moving body unit 11 of the imaging device identification information associated in the worker identification table is attached. When the management server 20 specifies the shooting position of the image, the management server 20 specifies the worker identifier corresponding to the shooting device identification information acquired together with the image using the worker identification table. Then, the specified photographing position is stored in the worker information storage unit in association with the worker identifier. As a result, not only the management object but also the location of the operator who has taken the image can be managed together.

  In the above embodiment, the mobile unit 11 has been described as a separate body from the helmet H1. However, the present invention is not limited to this, and a smart helmet in which all devices of the mobile unit 11 are integrated with the helmet H1 may be used.

  In the management target and state prediction process (step S2-2) of the above embodiment, the control unit 21 of the management server 20 specifies the use state using an image. The method of specifying the usage state is not limited to using an image. For example, it is possible to determine the use state of the management target using information accompanying the image, such as sound included in the moving image. In this case, the control unit 21 determines a sound generated when the management target is used by using deep learning. In this case, the management target state is learned using the frequency characteristics of the generated sound.

  In the above embodiment, the control unit 21 of the management server 20 specifies the location of the management target based on the image acquired from the mobile unit 11. In addition, a warning about a management target whose location has not been confirmed for a long time may be output to the display unit 17. Specifically, the control unit 21 holds an unconfirmed period. Then, the control unit 21 periodically calculates the final confirmation reference date and time before the unconfirmed period from the current date and time. Further, the control unit 21 uses the location information storage unit 27 to identify the date and time when the location was last confirmed (location confirmation date and time) for each individual identifier. And the control part 21 searches the individual identifier in which the location confirmation date after the last confirmation reference date is not recorded. When an individual identifier with no location confirmation date and time recorded after the final confirmation reference date and time is extracted, the control unit 21 displays the individual identifier of the location information 270 on the display unit. As a result, it is possible to urge the management to be reviewed for management targets that have not been used for a long period of time and cannot be used.

In the subject recognition processing of the above-described embodiment, the subject specifying unit 213 of the control unit 21 performs region division processing (step S2-1) on the captured image, and management target and state prediction processing (for each divided region) Step S2-2) was performed. The management target and state prediction process is not limited to each area division, but may be performed by area extraction by object detection.
Moreover, the management target and state prediction processing was executed using deep learning. The management target and state prediction processing may be performed using a known SIFT (Scale-Invariant Feature Transform) method or the like as long as it can be specified using an image.

  In the above embodiment, the control unit 21 specifies the shooting position from the image. The shooting position information for specifying the shooting position of the image is not limited to the image information, and for example, information such as a global positioning system (GPS) may be used.

  In the above embodiment, the control unit 21 of the management server 20 acquires the captured moving image from the mobile unit 11. The image used for the control unit 21 to manage the management target is not limited to a moving image, and a still image taken at a predetermined time interval can be used.

  M ... welder, H1 ... helmet, M1 ... marker, 10 ... management system, 11 ... moving body unit, 12 ... photographing part, 13 ... transmission part, 14, 21 ... control part, 16 ... input part, 17 ... display part , 20 ... management server, 22 ... building general information storage unit, 23 ... feature point information storage unit, 24 ... prohibited area information storage unit, 25 ... subject information storage unit, 26 ... moving picture information storage unit, 27 ... location information storage unit , 211 ... management unit, 212 ... position specifying unit, 213 ... subject specifying unit, 251 ... learned classification model, 252 ... teacher data, 255 ... individual recognition data, 260 ... video management information, 270 ... location information, 500 ... image .

Claims (6)

A management information storage unit that stores management target identification information that identifies a management target and location information of the management target in association with each other;
A management system including a control unit that acquires an image from a photographing device attached to a moving body,
The controller is
Obtaining an image photographed by the photographing device and photographing position information,
For the management target included in the image, specify the location using the shooting position information,
A management system, wherein the location is recorded in the management information storage unit in association with the management target identification information of the management target. The control unit identifies a usage state of a management target based on the image,
The management system according to claim 1, wherein the usage state is recorded in the management information storage unit in association with the management target identification information and the location. A moving body information storage unit for recording the location of the moving body;
The said control part specifies the imaging position which image | photographed the said image information as a location position of the said mobile body, and records the location position of the said mobile body in the said mobile body information storage part. 2. The management system according to 2.   4. The management system according to claim 3, wherein when the location of the moving body is included in a photographing prohibited area, the photographing apparatus stops photographing. A management information storage unit that stores management target identification information that identifies a management target and location information of the management target in association with each other;
A management method for managing the management target using a management system including a control unit that acquires an image from an imaging device attached to a moving body,
The controller is
Obtaining an image photographed by the photographing device and photographing position information,
For the management target included in the image, specify the location using the shooting position information,
A management method, wherein the location is recorded in the management information storage unit in association with the management target identification information of the management target. A management information storage unit that stores management target identification information that identifies a management target and location information of the management target in association with each other;
A management program that manages the management target using a management system including a control unit that acquires an image from an imaging device attached to a moving body,
The control unit
Obtaining an image photographed by the photographing device and photographing position information,
For the management target included in the image, specify the location using the shooting position information,
A management program that functions as means for recording the location in the management information storage unit in association with the management target identification information of the management target.