JP2023005746A - Work management system, robot device, work management method, and computer program - Google Patents

Abstract

To determine the presence/absence of an abnormality in the work with an image captured by a camera of a robot device that can autonomously travel while moving.SOLUTION: A work management system comprises: a work management information storage part which stores work management information indicating a schedule of the work executed in a work site by a worker; an object recognition part which recognizes whether or not a person is included in an image captured by a robot device; a work abnormality determination part which determines the presence/absence of an abnormality in the work in a place and at the time of capturing of the image on the basis of the work management information when the person is included in the image; and a determination result storage part which stores a result of the determination.SELECTED DRAWING: Figure 1

Description

The present invention relates to a work management system, a robot device, a work management method, and a computer program.

Conventionally, Patent Document 1, for example, describes a technique for assisting a worker who performs construction work of a building. In the technology described in Patent Document 1, in order to provide work support information to workers, a worker terminal owned by the worker or a robot capable of autonomous travel captures the state of the work of each worker photographed by a camera. The image information shown and the audio information collected by the microphone are transmitted to the central management device. Based on the construction-related information corresponding to the image information and the sound information, the central management device specifies the work to be performed by each worker.

JP 2017-220109 A

2. Description of the Related Art In recent years, it has become important to monitor whether workers are doing wrong work at a work site in an information processing facility such as a data center. For example, if work such as settings for a server device or a communication device is performed incorrectly in a data center, a failure that affects many users may occur. For this reason, it is important to monitor whether the worker is performing work that is different from the scheduled work, and to prevent failures from occurring. Here, when installing surveillance cameras in a facility, a large number of surveillance cameras are required to eliminate blind spots. For this reason, it is conceivable that the presence or absence of an abnormality in the work is determined based on an image captured by a camera while an autonomously traveling robot is moving.

However, with the technique described in Patent Literature 1, it is necessary to specify which worker the image information to be referenced corresponds to, in order to specify the work to be performed by each worker. For this reason, the worker terminal that transmits the image information to the central management device also transmits the worker ID to the central management device. On the other hand, when an autonomously traveling robot transmits image information to a central management device, Patent Document 1 does not describe how to acquire a worker ID corresponding to the image information. It is difficult to specify which worker the image information corresponds to.

SUMMARY OF THE INVENTION The present invention has been made in consideration of such circumstances, and its object is to determine the presence or absence of an abnormality in a work by an image taken by a camera while an autonomously traveling robot device is moving. It is in.

(1) One aspect of the present invention includes a robot device that autonomously travels and captures images of its surroundings, and a server device that communicates with the robot device. an object recognition unit for recognizing whether or not a person is included in the image taken by the robot device; and a person is included in the image. a work abnormality determination unit that determines whether or not there is an abnormality in the work at the location and time when the image was taken, based on the work management information, and a determination result storage unit that stores the result of the determination. , is a work management system.
(2) In one aspect of the present invention, when the work abnormality determination unit determines that there is an abnormality in the work, the elapsed time from the most recent scheduled work end time for the time when the image was captured at the location where the image was captured The work management system according to (1) above, wherein the reason for an abnormality in work is determined based on time.
(3) In one aspect of the present invention, the work abnormality determination unit determines the work abnormality level to be higher than a reference level when it is determined that there is a work abnormality a plurality of times within a certain period at the same place. , the work management system according to (1) or (2) above.
(4) According to one aspect of the present invention, the above (1) to (3), wherein the work abnormality determination unit does not determine whether or not there is an abnormality in the work when the location where the image is captured is a predetermined location. Any work management system.
(5) In one aspect of the present invention, the object recognition unit calculates the distance between the worker and the robot device based on the image captured by the robot device or the point cloud information of the object measured by the robot device. The work abnormality determination unit identifies the position of the worker based on the obtained distance, and identifies the work scheduled at the identified position of the worker, according to the above (1) to (4) Any work management system.
(6) In one aspect of the present invention, the object recognition unit recognizes the number of people from a plurality of images captured by the robot device at different times, and the work abnormality determination unit recognizes the number of people recognized and the work schedule. The work management system according to any one of (1) to (5) above, wherein the presence or absence of an abnormality in the work is determined based on the number of workers.
(7) An aspect of the present invention is (6) above, wherein the server device further includes a robot control unit that instructs the robot device to re-capture when the recognized number of people is less than the number of people scheduled to work. work management system.
(8) An aspect of the present invention is the work management system according to (7) above, wherein the robot control unit causes the robot device to move from a location where the previous image was taken and to retake the image.

(9) One aspect of the present invention is a robot device that autonomously travels and captures images of its surroundings, and includes a work management information storage unit that stores work management information indicating a schedule of work to be performed by a worker at a work site. an object recognition unit for recognizing whether or not a person is included in the image captured by the robot apparatus; and when the image includes a person, the image is captured based on the work management information. and a determination result storage unit for storing the result of the determination.

(10) One aspect of the present invention includes a robot device that autonomously travels and captures images of its surroundings, and a server device that communicates with the robot device, wherein the server device performs work performed by a worker at a work site. an object recognition step in which the server device recognizes whether or not a person is included in the image taken by the robot device; a work abnormality determination step of determining whether or not there is an abnormality in the work at the place and time when the image was taken, based on the work management information, when a person is included in the image; and and a judgment result storage step of storing the judgment result.
(11) According to one aspect of the present invention, a robot device that autonomously travels and photographs the surroundings includes a work management information storage unit that stores work management information indicating a schedule of work to be performed by a worker at a work site, an object recognition step in which the robot device recognizes whether or not a person is included in an image taken by the robot device; Work management including: a work abnormality determination step of determining whether or not there is an abnormality in the work at the location and time when the image was captured based on the information; and a determination result storage step of storing the determination result in the robot device. The method.

(12) One aspect of the present invention is work management information in which a server device that communicates with a robot device that autonomously travels and photographs the surroundings stores work management information that indicates a schedule of work to be performed by a worker at a work site. an object recognition step of recognizing whether or not a person is included in an image taken by the robot device, in the computer of the server device, the object recognition step comprising a storage unit; A computer program for executing a work abnormality determination step of determining whether or not there is an abnormality in the work at the location and time when the image was captured, based on work management information, and a determination result storage step of storing the result of the determination. is.
(13) In one aspect of the present invention, a robot device that autonomously travels and photographs the surroundings includes a work management information storage unit that stores work management information indicating a schedule of work to be performed by a worker at a work site, an object recognition step of recognizing whether or not a person is included in the image taken by the robot device, and an object recognition step, in the computer of the robot device, of recognizing whether or not a person is included in the image taken by the robot device; a computer program for executing a work abnormality determination step of determining whether or not there is a work abnormality at the location and time when the image is captured; and a determination result storage step of storing the determination result.

According to the present invention, it is possible to determine whether or not there is an abnormality in the work based on the image captured by the camera while the autonomously traveling robot device is moving.

1 is a block diagram showing a configuration example of a work management system according to one embodiment; FIG. 4 is a flow chart showing an example of the overall procedure of a work management method according to one embodiment; It is a figure which shows a concrete example in the work management method which concerns on one Embodiment. It is a figure which shows the structural example of the work management information which concerns on one Embodiment. 4 is a diagram showing a configuration example of robot control information according to one embodiment; FIG. It is a figure which shows the structural example of the determination result which concerns on one embodiment. FIG. 7 is an explanatory diagram of determination results shown in FIG. 6 ; 1 is a block diagram showing a configuration example of a robot device according to an embodiment; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
FIG. 1 is a block diagram showing a configuration example of a work management system according to the first embodiment. In FIG. 1 , the work management system 1 includes a server device 10 and a robot device 20 . The work management system 1 manages work performed by workers at work sites within the facility. A facility to be managed may be, for example, an information processing facility. Examples of information processing facilities include data centers and server rooms.

The server device 10 includes a communication unit 11, a control unit 12 (robot control unit 121, object recognition unit 122, work abnormality determination unit 123), and a storage unit 13 (robot control information 131, work management information 132, determination result 133). and The robot device 20 includes a communication unit 21 , a control unit 22 (an image acquisition unit 221 , a self-position estimation unit 222 and an autonomous travel control unit 223 ), a camera unit 23 , a LiDAR unit 24 and a travel unit 25 .

The robot device 20 autonomously travels and photographs the surroundings. The movement range in which the robot device 20 moves is a preset range in which the work site in the facility to be managed can be photographed.

The server device 10 communicates with the robot device 20 via the communication network NW. The communication network NW may consist of wired lines or wireless lines, or may consist of wired lines and wireless lines. For example, the communication unit 21 of the robot device 20 may be connected to the wireless line of the communication network NW, while the communication unit 11 of the server device 10 may be connected to the wired line of the communication network NW.

The server device 10 receives the image captured by the robot device 20 via the communication network NW. The server device 10 determines whether or not there is an abnormality in the work based on the image captured by the robot device 20 .

The robot device 20 and the server device 10 shown in FIG. 1 will be described in detail below.

[Robot device]
The robot device 20 has mainly hardware functions such as the camera unit 23, the LiDAR unit 24, and the traveling unit 25, and the main functions such as the control unit 22 (the image acquisition unit 221, the self-position estimation unit 222, and the autonomous traveling control unit 223). It has a function by software.

The communication unit 21 communicates with the communication unit 11 of the server device 10 via the communication network NW.

The camera unit 23 images the surroundings of the robot device 20 . The LiDAR unit 24 measures the distance (point group information) and the like to an object existing around the robot apparatus 20 by LiDAR (Light Detection and Ranging) technology. The traveling unit 25 includes a traveling mechanism that causes the robot device 20 to travel, a drive device that drives the traveling mechanism, and the like, and causes the robot device 20 to travel according to commands from the autonomous travel control unit 223 .

The control unit 22 controls the robot device 20 . Each function of the control unit 22 is realized by the control unit 22 having computer hardware such as a CPU (Central Processing Unit) and memory, and the CPU executing a computer program stored in the memory. . The controller 22 may be configured using a general-purpose computer device, or may be configured as a dedicated hardware device.

The image acquisition unit 221 acquires an image captured by the camera unit 23 (hereinafter referred to as a camera image). The image acquisition unit 221 acquires camera images based on image acquisition requests received from the server device 10 . The image acquisition unit 221 transmits camera images to the server device 10 .

The self-position estimation unit 222 matches the point group information measured by the LiDAR unit 24 with map information (not shown) provided in the robot device 20 by SLAM (Simultaneous Localization and Mapping) technology, and determines the current state of the robot device 20. Estimate location. The map information is map information including the range in which the robot device 20 is scheduled to travel.

When the user or application designates the destination of the robot device 20, the autonomous travel control unit 223 generates a route from the current position to the destination using path planning technology. The autonomous travel control unit 223 issues a command to the travel unit 25 to make the robot device 20 travel along the route.

[Server device]
The server device 10 has mainly software functions such as the control unit 12 (robot control unit 121, object recognition unit 122, work abnormality determination unit 123).

The communication unit 11 communicates with the communication unit 21 of the robot device 20 via the communication network NW.

The storage unit 13 stores various information. The storage unit 13 stores robot control information 131 , work management information 132 and determination results 133 .

The control unit 12 controls the server device 10 . The robot control unit 121 transmits a robot control command for controlling the robot device 20 to the robot device 20 . The robot control unit 121 transmits robot control commands such as movement commands and image acquisition requests to the robot device 20 based on the robot control information 131 .

The object recognition unit 122 recognizes whether or not a person is included in the camera image taken by the robot device 20 . For example, SSD (Single Shot MultiBox Detector) and Yolo (You only live once) can be applied as the object recognition technology. Note that the object recognition unit 122 may recognize the number of people (persons) included in the camera image.

Based on the work management information 132, the work abnormality determination unit 123 determines whether or not there is an abnormality in the work at the location and time when the camera image was captured when a person is included in the camera image. The work management information 132 is information indicating a schedule of work to be performed by the worker at the work site. The storage unit 13 stores a determination result 133 indicating the determination result of the work abnormality determination unit 123 .

Each function of the server device 10 is realized by the server device 10 having computer hardware such as a CPU and memory, and the CPU executing a computer program stored in the memory. The server device 10 may be configured using a general-purpose computer device, or may be configured as a dedicated hardware device. For example, the server device 10 may be configured using a server computer connected to a communication network such as the Internet. Also, each function of the server device 10 may be realized by cloud computing. The server device 10 may be implemented by a single computer, or may be implemented by distributing the functions of the server device 10 to a plurality of computers.

Next, with reference to FIG. 2, the overall procedure of the work management method according to this embodiment will be described. FIG. 2 is a flow chart showing an example of the overall procedure of the work management method according to this embodiment.

(Step S<b>1 ) The server device 10 acquires a camera image captured by the robot device 20 .

(Step S2) The server device 10 recognizes whether or not the camera image obtained from the robot device 20 includes a person.

(Step S3) If a person is included in the camera image, proceed to step S4; otherwise, terminate the processing of FIG.

(Step S<b>4 ) If a person is included in the camera image, the server device 10 determines whether there is an abnormality in the work at the location and time when the camera image was captured, based on the work management information 132 .

(Step S5) The server apparatus 10 records the determination result 133 of the presence/absence of an abnormality in the work.

Next, details of the work management method according to the present embodiment will be described with reference to FIGS. 3 to 7. FIG. 3 to 7 are diagrams showing a specific example of the work management method according to this embodiment.

Here, the facility of the data center 300 shown in FIG. 3 will be described as an example of the work site. As shown in FIG. 3, eight racks are installed in the data center 300 facility. Devices such as servers and routers are stored in each rack. A worker works on a device to be worked in a passage on the side of the rack in which the device to be worked is stored.

FIG. 4 is a diagram showing a configuration example of work management information according to this embodiment. An example of the work management information 132 in FIG. 4 is an example of a schedule of work to be performed by a worker at a work site within the facility of the data center 300 in FIG. In FIG. 4, work No. 1 to work No. A schedule of four tasks up to 4 is shown.

(Work No. 1) From 9:00 to 10:00, five workers of work company A perform the work of work content AAA in the third work passage.
(Work No. 2) From 9:30 to 10:15, three workers of work company B perform the work BBB in the 7th work passage.
(Work No. 3) From 12:00 to 13:00, one worker of work company C performs the work of work content CCC in the No. 3 work passage.
(Work No. 4) From 14:00 to 15:00, two workers of the work company D perform the work DDD in the work passage No. 3.

FIG. 5 is a diagram showing a configuration example of robot control information according to this embodiment. An example of the robot control information 131 in FIG. 5 corresponds to a robot control instruction for controlling the robot device 20 within the facility of the data center 300 in FIG. FIG. 5 shows instruction No. 1 to instruction No. Six robot control instructions up to 6 are shown.

(Instruction No. 1) At 9:00, an instruction is given to move to the destination P1 "X coordinate = 5.0, Y coordinate = 10.0", and when the destination P1 is reached, an image acquisition request is made. . When the robot device 20 captures images of the surroundings at the destination P1, a camera image of the No. 1 work passage and a camera image of the No. 6 work passage are obtained.

(Instruction No. 2) Instruction No. 1 (corresponding to the opening time "NULL" in FIG. 5), command to move to destination P2 "X coordinate = 10.0, Y coordinate = 10.0", and arrive at the destination P2 Then, an image acquisition request is made. When the robot device 20 captures images of the surroundings at the destination P2, a camera image of the No. 2 work passage and a camera image of the No. 7 work passage are obtained.

(Instruction No. 3) Instruction No. 2 (corresponding to the opening time "NULL" in FIG. 5), command to move to destination P3 "X coordinate = 15.0, Y coordinate = 10.0", and arrive at the destination P3 Then, an image acquisition request is made. When the robot device 20 captures images of the surroundings at the destination P3, a camera image of the No. 3 work aisle and a camera image of the No. 8 work aisle are acquired.

(Instruction No. 4) Instruction No. 3 (corresponding to the opening time "NULL" in FIG. 5), command to move to the destination P4 "X coordinate = 20.0, Y coordinate = 10.0", and arrive at the destination P4 Then, an image acquisition request is made. When the robot device 20 captures images of the surroundings at the destination P4, a camera image of the No. 4 work aisle and a camera image of the No. 9 work aisle are acquired.

(Instruction No. 5) Instruction No. 4 (corresponding to the opening time "NULL" in FIG. 5), command to move to destination P5 "X coordinate = 25.0, Y coordinate = 10.0", and arrive at the destination P5 Then, an image acquisition request is made. When the robot device 20 captures images of the surroundings at the destination P4, a camera image of work aisle No. 5 and a camera image of work aisle No. 10 are acquired.

(Instruction No. 6) At 9:30, an instruction is given to move to the destination P1 "X coordinate = 5.0, Y coordinate = 10.0", and when the destination P1 is reached, an image acquisition request is made. . When the robot device 20 captures images of the surroundings at the destination P1, a camera image of the No. 1 work passage and a camera image of the No. 6 work passage are obtained.

Based on the robot control information 131 shown in FIG. 1 to instruction No. Six robot control instructions up to 6 are sequentially transmitted to the robot device 20 . The robot device 20 can read those command numbers. 1 to instruction No. According to the six robot control commands up to 6, the robot sequentially moves to the destination, photographs the surroundings, and transmits the photographed camera images to the server device 10.例文帳に追加

The camera unit 23 is provided with cameras at both ends of the robot device 20. For example, by capturing images at the destination P1 with both cameras, a camera image of the No. 1 work passage (camera image of the No. 1 work passage) is obtained. ) and a camera image of work aisle No. 6 (camera image of work aisle 6). Alternatively, the camera unit 23 is provided with a camera in front of the robot device 20. For example, at the destination P1, the robot device 20 faces toward the No. 1 work passage and takes an image, and then faces toward the No. 6 work passage to take an image. , and a camera image of the No. 1 work aisle and a camera image of the No. 6 work aisle may be acquired. The image acquiring unit 221 assigns an identifier to the camera image acquired from the camera unit 23 at the destination P1, for example, to identify whether the camera image is the camera image of the first work passage or the camera image of the sixth work passage. After that, the camera image is transmitted to the server device 10 .

The server device 10, for example, commands No. When the camera image of the No. 1 work passage and the camera image of the No. 6 work passage are acquired by 1, the object recognition unit 122 recognizes whether or not a person is included in the camera images. Next, when a person is included in the camera image, the work abnormality determination unit 123 determines whether or not there is an abnormality in the work at the location and time when the image was captured, based on the work management information 132 of FIG. .

For example, the work abnormality determination unit 123 determines the command No. 1, based on the work management information 132 of FIG. Determine whether there is an abnormality in the work until 9:30. In this case, in the work management information 132 of FIG. 4, there is no work scheduled for the first work passage from 9:00 to 9:30. Therefore, there is a possibility that the worker shown in the camera image of the No. 1 work passage is doing an unscheduled wrong work. Therefore, the work abnormality determination unit 123 determines that the command No. 1, based on the work management information 132 of FIG. It is determined that there is an abnormality in the work until 9:30.

On the other hand, instruction No. 3, the work management information 132 in FIG. I have work to do. Therefore, the work abnormality determination unit 123 determines that the command No. 3, if a person is included in the camera image of work passage No. 3 acquired in step 3, based on the work management information 132 of FIG. It is judged that there is no abnormality in the work until 9:30.

The object recognition unit 122 recognizes the number of people included in the camera image, and the work abnormality determination unit 123 determines whether the number of workers in the work management information 132 and the number of people included in the camera image match or disagree. , the presence or absence of an abnormality in the work may be determined.

For example, instruction no. 3, the work management information 132 in FIG. is. Therefore, the work abnormality determination unit 123 determines that the command No. 3, if the number of people included in the camera image of the No. 3 work passage obtained in step 3 is 5, based on the work management information 132 of FIG. It is determined that there is no abnormality in the work at the time "between 9:00 and 9:30". On the other hand, the work abnormality determination unit 123 determines that the command No. 3, if the number of people included in the camera image of the No. 3 work passage acquired in step 3 is not 5, based on the work management information 132 of FIG. It is determined that there is an abnormality in the work at the time "between 9:00 and 9:30".

FIG. 6 is a diagram showing a configuration example of determination results according to the present embodiment. An example of the determination result 133 in FIG. 1, work no. 3 and work no. It corresponds to 4. FIG. 6 shows the result No. 1 to result No. Three determination results up to 3 are shown. Also, FIG. 7 is an explanatory diagram of the determination result 133 of FIG. FIG. 7 shows an explanation of the determination of whether or not there is an abnormality in the work in the No. 3 work passage.

(Result No. 1) A person is included in the camera image of the No. 3 work passage acquired at 9:30, but the work No. 3 in the No. 3 work passage in the work management information 132 of FIG. 1 (working time "9:00 to 10:00"), the determination result is no abnormality. Also, the abnormality level is "0" corresponding to no abnormality.

(Result No. 2) A person is included in the camera image of work aisle 3 acquired at 10:30, but there is no work in work aisle 3 that matches the work management information 132 in FIG. Therefore, the judgment result is abnormal. At this time, if the work abnormality determination unit 123 determines that there is a work abnormality, the work abnormality determination unit 123 determines the work based on the elapsed time from the most recent scheduled work end time with respect to the shooting time at the shooting location of the camera image. determine the reason for the abnormality. Result no. In the case of 2, the most recent work No. in the 3rd work passage. The elapsed time "30 minutes" from the scheduled work end time "10:00" of 1 is within the predetermined working time excess threshold "60 minutes". Therefore, the work abnormality determination unit 123 determines that the reason for the abnormality of the work is "work time exceeded". Moreover, the abnormality level is "1" corresponding to "work time exceeded" with abnormality.

(Result No. 3) A person is included in the camera image of work aisle 3 acquired at 11:30, but there is no work in work aisle 3 that matches the work management information 132 in FIG. Therefore, the judgment result is abnormal. At this time, if the work abnormality determination unit 123 determines that there is a work abnormality, the work abnormality determination unit 123 determines the work based on the elapsed time from the most recent scheduled work end time with respect to the shooting time at the shooting location of the camera image. determine the reason for the abnormality. Result no. In the case of 3, the elapsed time "90 minutes" from the past scheduled work end time "10:00" in the No. 3 work passage exceeds the predetermined working time excess threshold "60 minutes". Therefore, the work abnormality determination unit 123 determines that the reason for the abnormality of the work is "unplanned work". In addition, the abnormality level is "2" corresponding to "unplanned work" with abnormality.

Note that the working time excess threshold value may be preset in the server device 10 as a constant value, or may be preset in the server device 10 according to the content of the most recent work in the past.

In addition, the work abnormality determination unit 123 may determine the abnormality level to be higher than the reference level when it is determined that there is an abnormality in the work in the same work passage a plurality of times within a certain period of time. As a result, the administrator can be notified that the abnormal level of work is higher than usual.

Next, some modifications of this embodiment will be described.

(Modification 1)
The work abnormality determination unit 123 may not determine the presence or absence of work abnormality when the camera image is captured at a predetermined location. For example, in FIG. 3, if there is an entrance/exit of the facility in the No. 6 work aisle, there is a person in the No. 6 work aisle regardless of the work in the No. 6 work aisle, and there is no person in the No. 6 work aisle camera image. may be included. In this case, the work abnormality determination unit 123 does not determine whether or not there is an abnormality in the work with respect to the camera image of the No. 6 work passage. As a result, it is possible to obtain the effect of preventing erroneous determination of the presence or absence of an abnormality in the work.

(Modification 2)
The object recognition unit 122 recognizes the number of people from a plurality of camera images captured by the robot device 20 at different times, and the work abnormality determination unit 123 compares the recognized number of people with the planned number of workers in the work management information 132 ( The presence or absence of an abnormality in the work may be determined based on whether it matches or disagrees with the number of people scheduled to work. At this time, the robot control unit 121 may instruct the robot device 20 to re-shoot when the number of people recognized by the object recognition unit 122 is less than the number of people scheduled to work. This is because there is a possibility that fewer workers than the actual number were recognized because the workers were photographed overlapping each other. Further, the robot control unit 121 may cause the robot device 20 to move from the location where the previous image was taken and re-image. For example, in FIG. 3, when the camera image of the No. 1 work aisle is captured at the destination P1 (one end of the No. 1 work aisle) the first time, the second time is the destination P1 of the No. 1 work aisle. causes the camera image of the No. 1 work aisle to be re-captured on the opposite side (the other end of the No. 1 work aisle). As a result, it is possible to obtain the effect of preventing the number of workers from being recognized less than the actual number because the workers are photographed overlapping each other.

(Modification 3)
The object recognition unit 122 acquires the distance between the worker and the robot device 20 based on the image captured by the robot device 20 or the point group information of the object measured by the robot device 20, and the work abnormality determination unit 123 The position of the worker may be specified based on the obtained distance, and the work scheduled at the specified position of the worker may be specified. In order to deal with situations where multiple tasks are performed simultaneously or consecutively in one aisle, the relevant tasks are identified based on more detailed information on the work position, and then the presence or absence of abnormalities in the work is identified. It is for judging.

Specifically, the server device 10 includes in the work management information 132 information indicating detailed work locations on work passages (for example, information on work locations divided into a mesh in a two-dimensional space). The object recognition unit 122 acquires the distance from the worker to the robot device 20 based on the point group information measured by the LiDAR unit 24, for example. Alternatively, for example, the camera unit 23 has a depth camera with a built-in depth sensor that acquires depth information, and the object recognition unit 122 calculates the distance from the worker to the robot device 20 based on the image captured by the depth camera. get. Based on the acquired distance, the work abnormality determination unit 123 identifies the worker's position (the work place divided into meshes), identifies the work scheduled at the identified worker's position, and performs the work. judge whether there is any abnormality. Further, in order to increase the accuracy of identifying the position of the worker, the robot device 20 acquires camera images or point cloud information on both sides of the passage, and the server device 10 acquires the camera images or point clouds acquired on both sides of the passage. The information may be used to locate the worker.

(Modification 4)
The robot device 20 may capture a camera image at an arbitrary timing, and the server device 10 may determine whether there is an abnormality in the work based on the camera image captured at an arbitrary timing. In this case, the robot device 20 adds photographing information indicating the location and time at which the camera image was photographed to the camera image. The server device 10 recognizes the location and time when the camera image was captured based on the image capturing information added to the camera image.

[Second embodiment]
FIG. 8 is a block diagram showing a configuration example of a robot device according to the second embodiment. In FIG. 8, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and descriptions thereof are omitted. In the second embodiment, the functions provided in the server device 10 in the above-described first embodiment are provided in the robot device 20a, so that the robot device alone constitutes a work management system.

Specifically, the robot device 20a of FIG. 8 is different from the robot device 20 of FIG. A section 26 (robot control information 131, work management information 132, determination result 133) is added.

According to each of the above-described embodiments, it is possible to obtain the effect that the presence or absence of an abnormality in the work can be determined based on the images captured by the camera while the robot device 20 capable of autonomous travel is moving.

As a result, for example, it is possible to improve the overall service quality of various services that use work management systems. , promote sustainable industrialization, and contribute to the expansion of innovation.

Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are included within the scope of the present invention.

Alternatively, a computer program for realizing the functions of the devices described above may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by the computer system. Note that the “computer system” referred to here may include hardware such as an OS and peripheral devices.
In addition, "computer-readable recording medium" includes writable nonvolatile memories such as flexible discs, magneto-optical discs, ROMs and flash memories, portable media such as DVDs (Digital Versatile Discs), and computer system built-in media. A storage device such as a hard disk that

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM (Dynamic Random Access Memory)), which holds the program for a certain period of time, is also included.
Further, the above program may be transmitted from a computer system storing this program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in a transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

Reference Signs List 1 work management system 10 server device 20, 20a robot device 11, 21 communication unit 12, 22, 22a control unit 13, 26 storage unit 121 robot control unit 122 object Recognition unit 123 Work abnormality determination unit 131 Robot control information 132 Work management information 133 Determination result 23 Camera unit 24 LiDAR unit 25 Travel unit 221 Image acquisition unit 222 self-position estimation unit, 223... autonomous driving control unit, NW... communication network

Claims (13)

A robot device that autonomously travels and shoots the surroundings,
a server device that communicates with the robot device,
The server device
a work management information storage unit that stores work management information indicating a schedule of work to be performed by a worker at a work site;
an object recognition unit that recognizes whether or not an image captured by the robot device includes a person;
a work abnormality determination unit that determines whether or not there is an abnormality in the work at the location and time when the image was captured based on the work management information when a person is included in the image;
A determination result storage unit that stores the result of the determination,
Work management system. When determining that there is an abnormality in the work, the work abnormality determination unit determines the reason for the work abnormality based on the elapsed time from the last scheduled work end time with respect to the shooting time at the location where the image was shot. determine the
The work management system according to claim 1. The work abnormality determination unit determines the work abnormality level to be higher than a reference level when it is determined that there is an abnormality in the work multiple times within a certain period at the same place.
The work management system according to claim 1 or 2. The work abnormality determination unit does not determine whether or not there is an abnormality in the work when the location where the image is captured is a predetermined location.
The work management system according to any one of claims 1 to 3. The object recognition unit acquires the distance between the worker and the robot device based on the image captured by the robot device or the point group information of the object measured by the robot device,
The work abnormality determination unit identifies the position of the worker based on the acquired distance, and identifies the work scheduled at the identified worker's position.
The work management system according to any one of claims 1 to 4. The object recognition unit recognizes the number of people from a plurality of images captured at different times by the robot device,
The work abnormality determination unit determines whether there is a work abnormality based on the recognized number of people and the number of people scheduled to work.
The work management system according to any one of claims 1 to 5. The server device further comprises a robot control unit that instructs the robot device to re-shoot when the recognized number of people is less than the number of people scheduled to work.
The work management system according to claim 6. The robot control unit causes the robot device to move from a previous shooting location and reshoot.
The work management system according to claim 7. A robot device that autonomously travels and photographs the surroundings,
a work management information storage unit that stores work management information indicating a schedule of work to be performed by a worker at a work site;
an object recognition unit that recognizes whether or not an image captured by the robot device includes a person;
a work abnormality determination unit that determines whether or not there is an abnormality in the work at the location and time when the image was captured based on the work management information when a person is included in the image;
a determination result storage unit that stores the result of the determination;
A robotic device comprising: A robot device that autonomously travels and photographs the surroundings, and a server device that communicates with the robot device,
The server device includes a work management information storage unit that stores work management information indicating the schedule of work to be performed by the worker at the work site,
an object recognition step in which the server device recognizes whether or not an image captured by the robot device includes a person;
a work abnormality determination step in which the server device determines whether or not there is an abnormality in the work at the place and time when the image was taken based on the work management information when a person is included in the image;
a judgment result storage step in which the server device stores the judgment result;
work management methods, including; A robot device that autonomously travels and shoots the surroundings has a work management information storage unit that stores work management information indicating the schedule of work to be performed by the worker at the work site,
an object recognition step in which the robot device recognizes whether or not an image captured by the robot device includes a person;
a work abnormality determination step in which the robot device determines whether or not there is an abnormality in the work at the location and time at which the image was captured based on the work management information when a person is included in the image;
a determination result storage step in which the robot device stores the result of the determination;
work management methods, including; A server device that autonomously travels and communicates with a robot device that photographs the surroundings has a work management information storage unit that stores work management information indicating the schedule of work to be performed by the worker at the work site,
In the computer of the server device,
an object recognition step of recognizing whether or not an image captured by the robot device includes a person;
a work abnormality determination step of determining whether or not there is an abnormality in the work at the place and time when the image was taken, based on the work management information, when a person is included in the image;
a judgment result storage step of storing the judgment result;
A computer program for executing A robot device that autonomously travels and shoots the surroundings has a work management information storage unit that stores work management information indicating the schedule of work to be performed by the worker at the work site,
In the computer of the robotic device,
an object recognition step of recognizing whether or not an image captured by the robot device includes a person;
a work abnormality determination step of determining whether or not there is an abnormality in the work at the place and time when the image was taken, based on the work management information, when a person is included in the image;
a judgment result storage step of storing the judgment result;
A computer program for executing

Images