JP6902181B2 - Monitoring system, monitoring method and monitoring program - Google Patents

Description

The present invention relates to a monitoring system, a monitoring method, and a monitoring program for monitoring a person in a specific space.

It is necessary to ensure security in specific spaces such as data centers and food factories. In this case, a person may be photographed in a specific space, and face recognition technology may be used to determine whether or not the photographed person is a worker. However, when wearing dustproof clothing, a mask, or the like, it is difficult to determine the person.

Therefore, a surveillance system for collating a person using a surveillance camera and authentication information by electric field communication is also being studied (see, for example, Patent Document 1). The surveillance system described in this document includes an authentication controller and an image recognition server. The authentication controller executes the authentication process based on the authentication information received from the authentication terminal by electric field communication. The image recognition server recognizes the person included in the image of the shooting area taken by the camera, and is authenticated by the recognized person and the authentication controller based on the position information of the authentication terminal and the coordinate information of the shooting area of the camera. Match with a person.

Japanese Unexamined Patent Publication No. 2012-8802

In the technology described in the literature, equipment such as a surveillance camera and an electric field communication device is required. In particular, it is not easy to accurately acquire the position information of the authentication terminal by electric field communication.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a monitoring system, a monitoring method, and a monitoring program capable of efficiently grasping the allocation of personnel.

A monitoring system that solves the above problems is a monitoring system that includes a control unit connected to a photographing device that photographs a specific space and monitors the specific space, and the control unit is the identification from the photographing device. A photographed image of a space is acquired, a human body image included in the photographed image is extracted, a coloring specific area in the human body image is specified, and personnel arrangement information is output according to the color arrangement of the coloring specific area. .. As a result, the staffing information is output according to the color scheme of the coloring specific area included in the specific space, so that the staffing can be efficiently grasped. For example, when the photographing device is originally arranged, it can be monitored while using the existing equipment.

-In the monitoring system, the coloring specific area is colored by a color arrangement according to a role, and the control unit is connected to a work information storage unit that stores the number of people for each role assumed in the specific space. The control unit calculates the number of people for each role according to the color arrangement specified in the captured image, and the calculated number of people for each role and the number of people for each role stored in the work information storage unit. It is preferable to output the comparison result with. This makes it possible to easily grasp the difference from the original staffing.

-In the monitoring system, the control unit is connected to a reference color information storage unit that stores a reference reference color, and the control unit specifies a reference color setting area in the captured image and the reference color. It is preferable to acquire the detection color of the set area and use the result of comparing the detection color with the reference color stored in the reference color information storage unit to perform color correction of the coloring specific area. As a result, even when the color scheme of the coloring specific region looks different depending on the situation, the color scheme can be accurately specified by the color correction.

-In the monitoring system, the control unit is connected to a shooting location correction information storage unit that stores color correction information in association with the shooting device, and the control unit identifies the shooting device that shot the shot image. It is preferable that the color correction information associated with the photographing device is used in the photographing place correction information storage unit to perform color correction of the coloring specific region. As a result, the color scheme can be accurately specified by using the color correction information corresponding to the photographing device in advance.

-In the monitoring system, it is preferable to use the head position in the human body image as the coloring specific region. As a result, the coloring specific region can be accurately specified by the characteristic portion (edge portion) in the human body image.

According to the present invention, the staffing can be efficiently monitored.

Explanatory drawing explaining the structure of the monitoring system in this embodiment. It is explanatory drawing explaining the structure of the information stored in each storage part of the monitoring system in this embodiment, (a) is a work information storage part, (b) is a color information storage part, (c) is history information. Memory. The front view explaining the clothing of the person in this embodiment. It is explanatory drawing explaining the processing procedure of management processing in this embodiment, (a) is a flow chart of monitoring processing, (b) is explanatory drawing explaining each processing of monitoring processing.

Hereinafter, an embodiment in which the monitoring system, the monitoring method, and the monitoring program are embodied will be described with reference to FIGS. 1 to 4.
As shown in FIG. 1, in the present embodiment, the personnel W0 in the work area (specific space) S0 of the food factory is monitored.

Here, as shown in FIG. 3, each person W0 wears a clean suit, a clean cap (hat), and a mask. The clean cap functions as a coloring specific area, and the parietal region and the lateral region are colored with a color corresponding to the role of the personnel W0. Specifically, when the person W0 plays the role of a leader, he / she wears a clean cap of a color (for example, blue) that identifies the leader. Further, when the person W0 plays the role of a worker, he / she wears a clean cap of a color (for example, red) that identifies the worker.

The monitoring system 10 shown in FIG. 1 includes a photographing device 15 and a monitoring server 20.
The photographing device 15 photographs the work area S0 and transmits the captured image to the monitoring server 20. The photographing device 15 of the present embodiment is arranged at a position where the personnel W0 can be monitored, such as a position in the diagonally upward direction or a position in the lateral direction with respect to the specific space.
Further, the photographing device 15 transmits the generated photographed image to the monitoring server 20. The photographing device 15 stores the photographing device identifier for identifying the transmission source.

The monitoring server 20 includes a control unit 21, a work information storage unit 22, a color information storage unit 23, and a history information storage unit 25. Further, the monitoring server 20 includes an input unit 27 and a display unit 28.
The input unit 27 includes a keyboard and a pointing device, and inputs information (data) to be stored in the work information storage unit 22 and the color information storage unit 23.
The display unit 28 includes a display and displays the result of the monitoring process.

The control unit 21 includes control means (CPU, RAM, ROM, etc.) and performs processes (each process such as a monitoring step, a human body identification step, a color identification step, etc.) described later. Therefore, by executing the monitoring program stored in the storage unit, the control unit 21 functions as the monitoring unit 211, the human body identification unit 212, and the color identification unit 213.

The monitoring unit 211 identifies the number of people by role in the work area using the captured image, and executes a process of monitoring the status of the worker. The monitoring unit 211 stores a photographing device management table for specifying the work area S0 corresponding to the photographing device identifier of the photographing device 15.

The human body identification unit 212 executes a process of identifying the human body in the image. In the present embodiment, the human body identification unit 212 identifies a rectangular human body image including a human body recognition area by pattern recognition. Then, the human body identification unit 212 stores a coloring area identification program that specifies a head area including a clean cap of the coloring specific area in the layout (coordinates) of the human body image. In the present embodiment, in the rectangle including the human body recognition area, the area specified at a predetermined ratio is specified as the head area.
The color specifying unit 213 executes a process of specifying the color of the head area.

As shown in FIG. 2A, the work information storage unit 22 stores work management information 220 related to a work process performed in a specific space. The work management information 220 is stored in advance before the monitoring process is performed. The work management information 220 includes a work identifier, a work area, a work time zone, and the number of people by role.

In the work identifier data area, data regarding an identifier for identifying this work process is stored.
In the work area data area, information (for example, an identifier) for specifying an area (place) where this work process is executed is recorded.

In the work time zone data area, data related to the time zone (start date and time and end date and time) when the work process is performed in this work area is stored.
In the number-of-role data area, data on the number of people for each role required for the work process assumed in this work area is stored. The number of people associated with the role identifier is stored in the number-of-role data area. The role identifier is an identifier for identifying each role.

As shown in FIG. 2B, the color information storage unit 23 stores color information 230 related to colors for identifying a role assigned to a person. This color information 230 is stored in advance before the monitoring process is performed. The color information 230 includes label color information associated with the role identifier.

In the role identifier data area, data related to the identifier for identifying the role is recorded.
In the label color information data area, data regarding the color scheme (label color) assigned to the personnel in the role of this role identifier is recorded. In the present embodiment, the color scheme name and the RGB value of this color scheme are stored as the data related to the label color information.

As shown in FIG. 2C, the history information storage unit 25 stores history information 250 regarding the result of monitoring using the image captured by the photographing device 15. This history information 250 is recorded when the monitoring process is executed. The history information 250 includes data regarding the shooting date / time, the work area, the image, the number of detected persons by role, and the presence / absence of a warning.

In the shooting date / time data area, data related to the date and time when the image used for monitoring was shot is recorded.
In the work area data area, information (for example, an identifier) for identifying the work area (location) where the image was taken is recorded.

The image data used for monitoring is recorded in the image data area.
In the role-specific detected number data area, data regarding the number of people detected by role detected from this image is recorded.
In the presence / absence of warning data area, as a result of monitoring using this image, flag data regarding whether or not a warning has been given is recorded.

<Monitoring process>
Next, with reference to FIG. 4, a monitoring process executed by using the above-mentioned monitoring system 10 will be described. This monitoring process is executed repeatedly on a regular basis.

First, as shown in FIG. 4A, the control unit 21 of the monitoring server 20 executes an image reading process (step S1-1). Specifically, the monitoring unit 211 of the control unit 21 periodically acquires an image of the work area S0 from the photographing device 15 together with the photographing device identifier of the photographing device 15. The monitoring unit 211 identifies the work area according to the image pickup device identifier by using the image pickup device management table, generates the history information 250 including the work area, the acquired date and time (shooting date and time), and the image, and stores the history information. Record in section 25.

Next, the control unit 21 of the monitoring server 20 executes the human body detection process (step S1-2). Specifically, the human body identification unit 212 of the control unit 21 specifies the contour shape included in the image acquired from the photographing device 15. Then, the human body identification unit 212 detects a rectangular image (human body image R0 shown in FIG. 3) including the human body recognition region by pattern recognition (model fitting technique or the like) of the human body shape.

For example, in the work area S0 in FIG. 4B, three human body images are detected.
Then, the control unit 21 of the monitoring server 20 repeats the following processing for each detected human body image.

First, the control unit 21 of the monitoring server 20 executes the head identification process (step S1-3). Specifically, the human body identification unit 212 of the control unit 21 specifies the head area H0 shown in FIG. 3 in the specified human body image R0 by using the coloring region identification program.

Next, the control unit 21 of the monitoring server 20 executes the color information extraction process (step S1-4). Specifically, the color specifying unit 213 of the control unit 21 analyzes the histogram of the image of the specified head area H0 and extracts 16 representative colors.

Then, the control unit 21 of the monitoring server 20 executes the color determination process (step S1-5). Specifically, the color specifying unit 213 of the control unit 21 uses the area ratio of each representative color in the head area H0 and the similarity between the representative colors to control the dominant color (RGB) in the head area. The statistically representative color) is specified in the value, and the label color closest to this dominant color is estimated. In this case, the color identification unit 213 removes noise colors such as a clean suit color scheme (for example, white or light blue) and general hair color (for example, black or brown) from the area ratio and similarity, and the remaining color is removed. The label color (red or blue) is specified by the color scheme.
The above processing is repeatedly executed for each human body image.

Next, the control unit 21 of the monitoring server 20 executes the result display process (step S1-6). Specifically, the monitoring unit 211 of the control unit 21 aggregates the label colors specified for each human body image by label color. Then, the monitoring unit 211 records the number of people for each label color in the history information 250 and displays it on the display unit 28.

Further, the monitoring unit 211 specifies the work area S0 corresponding to the photographing device 15 that captured the read image, and specifies the working time zone corresponding to the shooting date and time of the image. The monitoring unit 211 specifies the work management information 220 including the specified work area S0 and the work time zone in the work information storage unit 22. The monitoring unit 211 compares the number of people by role of the specified work management information 220 with the total number of people by label color.

Here, if the number of people by role and the total number of people by label color do not match, the monitoring unit 211 of the control unit 21 issues a warning on the display unit 28 in addition to the number of people by label color. Output and record the warning "Yes" flag in the history information 250. Further, when the number of people by role and the number of people by total label color match, the monitoring unit 211 records the warning "none" flag in the history information 250.

According to this embodiment, the following effects can be obtained.
(1) In the present embodiment, the control unit 21 of the monitoring server 20 extracts the human body image R0 included in the captured image acquired from the photographing device 15 and executes the human body detection process (step S1-2). The control unit 21 determines the label color of the coloring specific region in each detected human body image R0 (step S1-5). As a result, the staffing information is output according to the color scheme of the head area H0 included in the human body image R0 of the person in the working area S0, so that the staffing can be efficiently grasped. For example, when the photographing device 15 is originally arranged in the work area, it can be monitored while using the existing equipment.

(2) In the present embodiment, the control unit 21 of the monitoring server 20 sets the number of people stored in the work information storage unit 22 by role and the totaled label color in the result display process (step S1-6). If the number of people does not match, a warning is output. This makes it possible to easily grasp the difference from the original staffing.

(3) In the present embodiment, the head area H0 in the human body image R0 is used as the coloring specific region. Thereby, the coloring specific region can be accurately specified by the characteristic portion (end portion) in the human body image R0.

Moreover, the said embodiment may be changed as follows.
-In the above embodiment, the control unit 21 of the monitoring server 20 estimates the label color closest to the dominant color in the head area in the color determination process (step S1-5). Before this color determination process, the color of the image may be corrected. Specifically, in the work area S0, the reference color setting area may be arranged on a wall or the like, and the photographing device 15 may photograph a photographed image including the reference color setting area. In this case, the control unit 21 of the monitoring server 20 connects to the reference color information storage unit that stores the reference reference color. Then, the control unit 21 specifies the color (detection color) of the reference color setting region captured in the image acquired from the photographing device 15. The control unit 21 compares the detected color with the reference color stored in the reference color information storage unit, and sets the stored reference color information as a label color so as to be the color of the acquired reference color region. Generate a color correction chart that corrects the color scheme used.

Then, the control unit 21 identifies in the color correction chart the color closest to each representative color in the head area H0 among the colors of the generated color correction chart, and estimates the label color. As a result, even when the color scheme of the coloring specific region looks different depending on the situation, the color scheme can be accurately specified by the color correction.

Further, color correction may be performed for each imaging location or each imaging device. For example, when color correction is performed for each imaging device, the monitoring server 20 includes a imaging location correction information storage unit that stores color correction information in association with the imaging device identifier of each imaging device 15. The imaging location correction information storage unit stores color correction information corrected in advance according to the color characteristics of the imaging device 15 based on the reference color acquired from the imaging device 15. The control unit 21 is connected to the imaging location correction information storage unit. When the captured image is acquired from the photographing device, the control unit 21 estimates the label color by using the color correction information stored in the photographing location correction information storage unit. As a result, the color scheme can be accurately specified by using the color correction information corresponding to the photographing device 15 in advance. A color correction chart may be used together with this color correction information.

-In the above embodiment, one photographing device 15 is provided for one working area. The work area (specific space) photographed by the photographing device 15 is not limited to one, and a plurality of work areas may be photographed. In this case, the control unit 21 stores the photographing device management table that specifies the working area in association with the photographing device identifier of the photographing device 15 and the photographing direction. Then, the control unit 21 acquires the shooting direction at the time of shooting the image from the shooting device 15, and specifies the work area by using the data for specifying the work area in association with the shooting direction. As a result, when the photographing device 15 can photograph a plurality of work areas, it is possible to efficiently grasp the personnel arrangement.

In the above embodiment, the control unit 21 of the monitoring server 20 displays the total number of people by label color in the result display process (step S1-6) on the display unit 28 and stores it in the work information storage unit 22. A warning was output when the number of people by role and the number of people by total label color did not match. The result display is not limited to this, and may be any information on the placement of personnel specified from the image. For example, only when there is an area where a warning is output, it is displayed on the display unit 28 or an e-mail is notified to the administrator. You may do it.

In the above embodiment, the control unit 21 of the monitoring server 20 identifies the head area H0 in the rectangular human body image R0 including the human body recognition area in the head identification process (step S1-3), and the head area H0 is specified. The color determination process of H0 (step S1-5) was executed. The method for specifying the coloring specific region is not limited to the above-mentioned method as long as it can be specified using the coordinates in the human body image R0.

-In the above embodiment, the head area H0 at the head position was used as the coloring specific region. The coloring specific area is not limited to the head position, and a part of another human body (for example, shoulder, chest, back, etc.) may be used as the coloring specific area.

H0 ... Head area, R0 ... Human body image, S0 ... Work area, W0 ... Personnel, 10 ... Surveillance system, 15 ... Imaging device, 20 ... Surveillance server, 21 ... Control unit, 22 ... Work information storage unit, 23 ... Color Information storage unit, 25 ... History information storage unit, 27 ... Input unit, 28 ... Display unit, 211 ... Monitoring unit, 212 ... Human body identification unit, 213 ... Color identification unit, 220 ... Work management information, 230 ... Color information, 250 … History information.

Claims (5)

A color information storage unit that stores label color information related to role identifiers,
A work information storage unit that stores information about work areas, work hours, and the number of people by role,
It is a monitoring system that has a control unit connected to a photographing device that photographs a specific space and monitors the specific space.
The control unit
In the human body image specified by pattern recognition, the colored area identification information for specifying the head area including the cap colored with the color according to the role as the colored specific area is retained.
A photographed image of the specific space is acquired from the photographing device, and the photographed image is acquired.
The human body image included in the photographed image is extracted by distinguishing each person by pattern recognition.
In the human body image of each person, in the rectangle including the human body recognition area, the area specified at a predetermined ratio is specified as the head area.
Image analysis of the head area was performed to identify the color scheme, and the number of people for each label color was calculated.
Information on the number of people by role according to the work area of the specific space, the work time zone, and the work time zone corresponding to the shooting date and time of the captured image is acquired from the work information storage unit.
A monitoring system characterized in that a warning is output when the number of people by label color and the number of people by role do not match. The control unit is connected to a reference color information storage unit that stores a reference reference color as a reference.
The control unit
A reference color setting area is specified in the captured image, and the detection color of the reference color setting area is acquired.
Monitoring system according to claim 1, wherein the sensing color, using the result of comparison between the reference color stored in said reference color data storing unit, and performing color correction of the coloring specific area. The control unit is connected to a shooting location correction information storage unit that stores color correction information in association with the shooting device.
The control unit identifies a photographing device that has captured the captured image, and uses the color correction information associated with the photographing device in the photographing location correction information storage unit to perform color correction of the coloring specific region. The monitoring system according to claim 1 or 2, wherein the monitoring system is characterized. A color information storage unit that stores label color information related to role identifiers,
A work information storage unit that stores information about work areas, work hours, and the number of people by role,
It is a monitoring method for monitoring the specific space by using a monitoring system provided with a control unit connected to a photographing device for photographing the specific space.
The control unit
In the human body image specified by pattern recognition, the colored area identification information for specifying the head area including the cap colored with the color according to the role as the colored specific area is retained.
A photographed image of the specific space is acquired from the photographing device, and the photographed image is acquired.
The human body image included in the photographed image is extracted by distinguishing each person by pattern recognition.
In the human body image of each person, in the rectangle including the human body recognition area, the area specified at a predetermined ratio is specified as the head area.
Image analysis of the head area was performed to identify the color scheme, and the number of people for each label color was calculated.
Information on the number of people by role according to the work area of the specific space, the work time zone, and the work time zone corresponding to the shooting date and time of the captured image is acquired from the work information storage unit.
A monitoring method characterized in that a warning is output when the number of people by label color and the number of people by role do not match. A color information storage unit that stores label color information related to role identifiers,
A work information storage unit that stores information about work areas, work hours, and the number of people by role,
A monitoring program that monitors the specific space using a monitoring system equipped with a control unit connected to a photographing device that photographs the specific space.
The control unit holds colored area identification information for specifying a head area including a cap colored with a color according to a role as a colored specific area in a human body image specified by pattern recognition.
The control unit
A photographed image of the specific space is acquired from the photographing device, and the photographed image is acquired.
The human body image included in the photographed image is extracted by distinguishing each person by pattern recognition.
In the human body image of each person, in the rectangle including the human body recognition area, the area specified at a predetermined ratio is specified as the head area.
Image analysis of the head area was performed to identify the color scheme, and the number of people for each label color was calculated.
Information on the number of people by role according to the work area of the specific space, the work time zone, and the work time zone corresponding to the shooting date and time of the captured image is acquired from the work information storage unit.
A monitoring program characterized in that it functions as a means for outputting a warning when the number of people by label color and the number of people by role do not match.

Images