JP2022012400A - Image processing device, control method of the same, and program - Google Patents

Abstract

To provide a technology that suppresses occurrences of suspicious behaviors by presenting areas in a monitored area that should be patrolled.SOLUTION: A device sets an area to be patrolled on image data and identifies whether a predetermined person detected in the image data has circled the aforementioned area. Then, using a result of the identification, the system decides whether to give a notification to encourage a person to patrol the aforementioned area, and controls the notification to be given.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing device, a control method for the image processing device, and a program.

In recent years, shoplifting damage has become more serious in various facilities such as retail stores and large-scale commercial stores, and there is a need on the store side to reduce the damage caused by shoplifting. In response to this need, there is a technology to prevent shoplifting by analyzing the video taken by the surveillance camera in real time to detect suspicious behavior that is suspected of shoplifting and notifying the store clerk or security guard of the detection result. be.

As a technique for detecting suspicious behavior, for example, there is a method described in Patent Document 1. In Patent Document 1, feature values such as the position, orientation, and line-of-sight direction of a person are extracted from image data obtained from an image taken by a camera, and these feature values are analyzed to detect a person suspected of being shoplifted. When a person suspected of shoplifting is detected, the store clerk or security guard is notified, and the store clerk or security guard speaks to the target person to suppress shoplifting.

JP-A-2017-076171

However, the method described in Patent Document 1 cannot prevent the occurrence of suspicious behavior even if the suspicious behavior can be detected from the image data.

An object of the present invention is to provide a technique for suppressing the occurrence of suspicious behavior by presenting an area to be looked around in a monitored area.

The present invention has been detected by an acquisition means for acquiring image data, a setting means for setting an area to be patrolled on the image data, a detection means for detecting a predetermined person from the image data, and the detection means. A specific means for identifying whether the predetermined person has patrolled the area, a determination means for determining whether to give a notification for urging the patrol to the area by using the specific result by the specific means, and the above-mentioned. It is characterized by comprising a control means for controlling the notification on the condition that the notification is determined by the determination means.

According to the present invention, the occurrence of suspicious behavior can be suppressed by presenting an area to be looked around in the monitored area.

It is a figure which showed the example of the hardware composition of the image processing apparatus by embodiment. It is a block diagram which shows the function and composition of the image processing apparatus and the member connected to the image processing apparatus in 1st Embodiment. It is a figure which shows the example of the setting screen displayed on the display device. It is a figure which shows the example of the instruction screen which is displayed on display by an instruction device. It is a flowchart which shows the flow of the process which the image processing apparatus creates a patrol instruction diagram in 1st Embodiment. It is a block diagram which shows the function and composition of the image processing apparatus and the member connected to it in the modification. It is a figure which shows the example which expressed the priority of a patrol instruction by a color in the modification. It is a figure which shows the example which displays the image of the patrol area by using the instruction device in 2nd Embodiment. It is a figure which shows the example which displays the image of the patrol area by using the instruction device in 2nd Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are given the same reference numbers, and duplicate explanations are omitted.

FIG. 1 is a diagram showing an example of a hardware configuration of an image processing device 100. The image processing device 100 includes a CPU 101, a ROM 102, a RAM 103, a hard disk drive (HDD) 104, a communication unit 105, an input device 106, and a display device 107.

The communication unit 105 is connected to the network 108. The CPU 101 reads out the control program recorded in the ROM 102 and executes various processes. The RAM 103 is used as a temporary storage area such as a main memory or a work area. The HDD 104 is used for storing image data. Note that the HDD 104 is not always necessary when the CPU 101 acquires image data via the network 108. The communication unit 105 is a circuit that communicates via the network 108. When the CPU 101 acquires the image data via the HDD 104, the communication unit 105 is not always necessary. The display device 107 is a display device such as a display that displays image data, a result of suspicious person detection, a patrol instruction, and the like. The input device 106 is a keyboard, a mouse, or the like, and has a device for the user to manually set the selection of the image data displayed on the display device 107, the setting of the area designation on the image data, and the like.

As described above, the hardware configuration of the image processing apparatus 100 has the same hardware components as the hardware configuration mounted on the personal computer (PC). Therefore, various functions realized by the image processing apparatus 100 can be implemented as software that operates on a PC. The image processing device 100 can realize various functions of the image processing device 100 of FIG. 1 and the flowchart of FIG. 5 to be described later by executing a program by the CPU 101.

(First Embodiment)
FIG. 2 is a block diagram showing the functions and configurations of the image processing apparatus 100 of FIG. 1 and the members connected to the image processing apparatus 100 in the first embodiment. Each block shown here can be realized by an element such as a computer CPU or a mechanical device in terms of hardware, and can be realized by a computer program or the like in terms of software, but here, it is realized by their cooperation. It depicts a functional block that will be used. Therefore, it is understood by those skilled in the art who have touched this specification that these functional blocks can be realized in various forms by a combination of hardware and software.

The image processing device 100 includes an image acquisition unit 202, a human body detection unit 203, a setting unit 204, an employee determination unit 205, a patrol determination unit 206, and a patrol instruction unit 207.

Further, the image processing device 100 is connected to a video acquisition device 201 such as a surveillance camera and an instruction device 208 via a network 108 such as the Internet.

The image acquisition unit 202 inputs image data obtained from the image transmitted from the image acquisition device 201 into the image processing device 100. The video may be a video file such as an avi file or a move file, or may be a real-time video sent from a web camera. Further, the number may be one or a plurality.

The human body detection unit 203 detects the human body included in the image data. A known method can be used for human body detection, and for example, there is a method of performing pattern matching based on the shape of the human body, but the method is not limited to this.

The setting unit 204 specifies an area to be patrolled on the image data. The method of specifying may be the specification of the coordinates by accepting the numerical input from the user, or the specification of the coordinates using the UI may be accepted from the user. FIG. 3 shows an example of specifying the coordinates of the patrol target area using the UI.

FIG. 3 shows a setting screen displayed on the display device 107 of the image processing device 100, and the setting screen 300 shows an in-store view (301) that gives a bird's-eye view of the inside of the store and surveillance camera icons 302A to 302E showing the arrangement of surveillance cameras. including.

The setting unit 204 sets the patrol areas 303A to 303E on the in-store map (301) according to the user's instruction. At this time, the photographing areas taken by each of the surveillance camera icons 302A to 302E correspond to the patrol areas 303A to 303E.

When a plurality of patrol areas are set on the in-store map (301), a tag or the like that distinguishes each set patrol area may be displayed.

In this embodiment, one patrol area is set for one surveillance camera, but when monitoring a wide area with one surveillance camera, such as when using a surveillance camera equipped with a wide-angle lens, 1 A plurality of patrol areas may be set for the area captured by the surveillance cameras.

The employee determination unit 205 determines whether the human body detected by the human body detection unit 203 is an employee (predetermined person) of the store. A known method can be used to determine whether an employee is an employee. For example, a method of determining whether or not the employee is an employee may be used based on whether or not the feature data of the face of the employee registered in advance and the feature data of the detected face image of the human body match. Further, a method may be used in which a color code (identifier) is attached to the clothes of the employee and the color code is photographed by a surveillance camera to detect the employee. Employees also include observers entrusted with monitoring by stores.

The patrol determination unit 206 determines whether or not the employee has patrol in the patrol area set by the setting unit 204. In this embodiment, if an employee is detected in the patrol area, it is determined that the patrol area has been patroled, but the present invention is not limited to this. A method of determining that the employee has been patrolled by staying in the patrol area for a predetermined time, or a method of detecting the flow line of the employee and determining that the employee has been patrolled by crossing the patrol area may be used.

The determination result (specific result) by the patrol determination unit 206 is stored in the RAM 103 or the HDD 104 of the image processing apparatus 100 as the patrol determination information. The patrol determination information includes at least a date and time when the patrol is determined and an ID for distinguishing which patrol area has been patroled when a plurality of patrol areas are set. The patrol determination unit 206 is an example of the specific means in the present invention.

The patrol instruction unit 207 identifies whether or not an employee has been patrolled in the patrol area by using the determination result by the patrol determination unit 206. Then, when it is specified that there is no patrol of the employee, the instruction device 208 is instructed to issue a patrol instruction to the employee. As a method of specifying whether or not an employee has been patrolled, the patrolling determination unit 206 identifies that there was no patrolling in the patrolling area where the determination result of having been patrolled was not obtained for a certain period of time. On the other hand, it is specified that there was a patrol in the patrol area where the judgment result that the patrol has been completed was obtained for a certain period of time. As another embodiment, the setting unit 204 sets the residence time of the employee deemed to have stayed in the patrol area and the number of times the employee is detected per hour as a threshold value, and the result of the patrol determination unit 206 is the threshold value. If the above is not satisfied, it may be specified that the employee has not been patrolled. That is, it may be specified that the employee has been patrolled in the patrol area where the employee can be continuously detected for a predetermined time or when the employee is detected a predetermined number of times.

Further, as another embodiment, the patrol determination unit 206 may perform until the patrol of the employee is specified. The patrol instruction unit 207 is an example of the determination means and the control means in the present invention.

The instruction method by the instruction device 208 is preferably an instruction using a display display (in-store map (301) or identification number of a patrol area), but the patrol instruction can be an instruction method using a sound or other device in addition to the display display. I do not care. Figure 4 shows an example of patrol instructions to employees.

FIG. 4 shows an instruction screen displayed on display by the instruction device 208, and the instruction screen 400 includes a patrol instruction diagram (401) and a patrol instruction area 402 reflecting the instruction result of the patrol instruction unit 207.

The patrol instruction diagram (401) shows an area corresponding to the in-store diagram (301). The patrol instruction area 402 indicates an area identified by the patrol instruction unit 207 that no employee has been patrolled. FIG. 4 shows an example in which the patrol area of 403C is specified as the patrol instruction area 402.

In this embodiment, the patrol instruction area 402 is identified and displayed in a color different from that of the patrol designated areas 403A, 403B, 403D, and 403E, so that the patrol instruction area 402 is instructed to patrol, but the present invention is not limited to this. .. For example, when a patrol instruction is issued, a message prompting patrol to the patrol instruction area (such as "Please patrol 403C") may be displayed on the display.

Next, the process until the patrol instruction diagram (401) is created will be described with reference to FIG.

FIG. 5 is a flowchart showing a flow chart of a process of creating a patrol instruction diagram for the image processing device 100 to give a patrol instruction to an employee by the instruction device 208 in the first embodiment.

When an instruction to set a patrol area on the in-store map (301) is received from the user via the input device 106 in S501, the setting unit 204 determines an area for detecting the human body and determining whether the employee has patroled. To.

After that, in S502, the human body detection unit 203 detects the human body with respect to the image data acquired from the image acquisition unit 202.

In S503, the employee determination unit 205 determines whether or not the person detected by the human body detection unit 203 is an employee. If it is determined to be an employee, the process shifts to S504, and if it is determined not to be an employee, the process returns to S502.

In S504, the patrol determination unit 206 determines whether or not the person determined to be an employee in S503 is included in the patrol area set in S501. The determination result by the patrol determination unit 206 is stored in the RAM 103 or the HDD 104 as the patrol determination information. The patrol determination information includes at least a date and time when the patrol is determined and an ID for distinguishing which patrol area has been patroled when a plurality of patrol areas are set.

In S505, the patrol instruction unit 207 uses the patrol determination information obtained in S504 to determine whether there is a patrol area in which the employee has not patrol for a certain period of time. If there is a patrol area where the employee has not patrol for a certain period of time, the process is transferred to S506 and a patrol instruction diagram (401) is created. The created patrol instruction diagram (401) is displayed on the display device 107 by using the instruction device 208. On the other hand, if there is no patrol area where the employee has not patroled for a certain period of time, the process returns to S502.

By repeating the above processes of S502 to S506, the image processing device 100 gives the employee an efficient patrol instruction in the store by using the instruction device.

(Variation example of the first embodiment)
In the first embodiment, in S506 of FIG. 5, for a predetermined area (patrol area), it is determined whether the area (patrol instruction area) to be instructed to patrol is triggered by the detection of an employee.

However, in the first embodiment, shoplifting and places and times where there are many suspicious behaviors that are considered to be shoplifting are not taken into consideration.

Therefore, in the modified example, the patrol instruction unit 207 instructs the patrol instruction device 208 to patrol the patrol instruction area in which the suspicious behavior is detected, giving priority to the patrol instruction area in which the suspicious behavior is not detected. Give instructions.

FIG. 6 is a block diagram showing the functions and configurations of the image processing device 100 of FIG. 1 and the members connected to the image processing device 100 in the modified example. Here, only the functions and configurations different from the block diagram of FIG. 2 will be described.

In the modified example, the image processing device 100 has a suspicious behavior determination unit 610. The suspicious behavior determination unit 610 determines the suspicious behavior of the human body detected by the human body detection unit 203. As an example of the method of determining suspicious behavior, there is a method of determining the movement of the human hand (such as taking an object and putting it in a bag) or the direction of the line of sight (such as the line of sight is swaying). The suspicious behavior may be determined by any of the known methods, not limited to the above-mentioned method of determining suspicious behavior.

When executing the process of S505, the patrol instruction unit 607 also determines whether the region determined to be patrol by the patrol determination unit 606 includes the result of the suspicious behavior determination unit 610. Then, using the information as to whether or not the result of the suspicious behavior determination unit 610 is included, in S506, a priority is given and an instruction is given to the instruction device 608.

FIG. 7 is an example in which the priority of the patrol instruction in the modified example is represented by a color. In the modified example, the highest priority is patrol when the patrol is not performed and suspicious behavior is detected, the next is when the patrol is not performed and no suspicious behavior is detected, and the next is when the patrol is completed and suspicious behavior is detected. The patrol instruction is given by separating the colors of the instruction area 402, but the present invention is not limited to this. It suffices if the user can identify the patrol instruction area 402 having different priorities, and the thickness of the frame indicating the patrol instruction area 402 may be changed according to the priority, or the mark indicating the priority may be superimposed on the patrol instruction area 402. You may do it.

Further, the method of determining the priority is not limited to this, and the priority may be determined according to the date and time when the suspicious behavior was detected, the day of the week, and the type of the suspicious behavior, and the order of the priorities of each condition is also shown in FIG. It is not limited to the example shown.

As described above, according to the modified example, it is possible to efficiently issue a patrol instruction by giving priority to the area to be patrol in consideration of the detection result of the suspicious behavior.

(Second embodiment)
In the first embodiment, it was assumed that the instruction device 208 uses the patrol instruction diagram (401) to encourage the employee to patrol, but in the second embodiment, the instruction device 208 is a surveillance camera. It is assumed that a camera viewer that displays the image of is used.

8 and 9 are examples of monitoring the patrol area using a camera viewer.

FIG. 8 is an example of displaying the patrol instruction area image 805 using the instruction device 208 when the patrol instruction area 803 is included in the patrol instruction diagram (801). In FIG. 8, a pop-up screen 804 is displayed so that the user can easily determine which patrol instruction area is the image, but the present invention is not limited thereto. When a plurality of patrol instruction areas are included, the patrol instruction area image 805 may be displayed on the pop-up screen 804 by using the priority shown in the example of FIG. 7 and setting a time difference in descending order of priority.

FIG. 9 is an example in which the size of the image of the patrol instruction area displayed by the camera viewer 900 is determined and displayed using the priority shown in the example of FIG. 7. The video 901, the video 902, and the video 903 are displayed in the order of the video size in descending order of priority. At this time, the image size and layout shown in FIG. 9 are examples, and are not limited thereto.

As described above, according to the second embodiment, it is possible to effectively encourage the employee to monitor the patrol area by using the camera viewer.

As described above, according to the present invention, it is possible to efficiently instruct employees to patrol the store in order to suppress shoplifting.

(Other examples)
INDUSTRIAL APPLICABILITY The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads the program. It can also be realized by the processing to be executed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments and can be modified and modified in various ways without departing from the spirit and scope of the invention. Therefore, a claim is attached to publicize the scope of the invention.

100 Image processing device 101 CPU
102 ROM
103 RAM
104 Hard disk drive (HDD)
105 Communication unit 106 Input device 107 Display device

Claims (11)

The acquisition method for acquiring image data and
A setting means for setting an area to be patrolled on the image data, and
A detection means for detecting a predetermined person from the image data,
A specific means for identifying whether the predetermined person detected by the detection means has patrolled the area, and
A decision-making means for deciding whether to give a notice for urging a patrol to the area by using the specific result by the specific means.
An image processing apparatus comprising: a control means for controlling the notification on the condition that the notification is determined by the determination means. The image processing apparatus according to claim 1, wherein the determination means determines to perform the notification when the predetermined person is detected a predetermined number of times in the area. The image processing apparatus according to claim 1, wherein the determination means determines to give the notification when the predetermined person is continuously detected for a predetermined time in the region. The image processing apparatus according to any one of claims 1 to 3, wherein the acquisition means acquires the image data obtained by capturing an image of the inside of a store. The image processing device according to claim 4, wherein the predetermined person is an employee of the store. The image processing apparatus according to claim 5, wherein the detection means detects a person who matches the characteristic data indicating the characteristics of a pre-registered employee as the predetermined person. The image processing apparatus according to claim 5, wherein the detection means detects the predetermined person by detecting an identifier attached to the clothes of the employee. The setting means sets a plurality of the areas, and the setting means sets a plurality of the areas.
The image processing apparatus according to any one of claims 1 to 7, wherein the specific means and the determination means execute each process in each of the plurality of the regions. The detection means further detects a person who has performed a suspicious action, and the detection means further detects the person.
The control means controls to perform the notification so that the area in which the person who has performed the suspicious action is detected and the area in which the person who has performed the suspicious action is not detected among the areas for which the patrol is promoted by the notification can be distinguished. The image processing apparatus according to claim 8. The acquisition process to acquire the image and
A setting process for setting an area to be patrolled on the image data, and
A detection step of detecting a predetermined person from the image data, and
A specific step of specifying whether or not the predetermined person detected in the detection step has patrolled the region, and
Using the specific result in the specific step, the notification is performed on condition that a decision step for deciding whether to give a notification for urging the patrol to the region and a decision to give the notification in the determination step are performed. And the control process to control
A method for controlling an image processing apparatus, which comprises. A program for making a computer function as each means of the image processing apparatus according to any one of claims 1 to 9.

Images