JP2021068201A - Network system, suspicious person detection device, and suspicious person detection method - Google Patents

Abstract

To provide a network system, a suspicious person detection device, and a suspicious person detection method for efficiently detecting a suspicious person.SOLUTION: A network system 1 for detecting a suspicious person includes: a camera 300 for photographing the vicinity of a place where a commodity is placed; a server 100 having a control unit for detecting a person performing a predetermined operation based on an image provided by the camera 300, counting the number of times of detection for each person by identifying the person, and outputting a warning when the predetermined number of times are reached; and a store terminal 200 and an administrator terminal 400 for issuing a warning.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for detecting a suspicious person at a product display place or the like.

Conventionally, a technique for detecting a suspicious person at a product sales office or the like has been known. For example, Japanese Patent Application Laid-Open No. 1 (Patent Document 1) discloses a control device, a control method, and a control program. According to Patent Document 1, the control device is searched by a search unit that searches for advertisement information that is information about an advertisement output as a response message based on the characteristics of sound in a conversation between the dialogue agent system and the user, and a search unit. It is characterized by including an output control unit that controls to output an advertisement related to the advertisement information as a response message. Therefore, the control device can output a message suitable for the user.

Japanese Unexamined Patent Publication No. 2019-10474

An object of one aspect of the present invention is to provide a technique for efficiently detecting a suspicious person.

According to one aspect of the present invention, a camera for photographing the vicinity of the place where the product is placed and a person performing a predetermined operation are detected based on the image of the camera, and the person is identified. As a result, a suspicious person detection system including a control unit for counting the number of detected times for each person and outputting a warning when the predetermined number of times is reached is provided.

As described above, according to one aspect of the present invention, a technique for efficiently detecting a suspicious person is provided.

It is a figure which shows the whole structure of the network system 1 which concerns on 1st Embodiment. It is a block diagram which shows the structure of the server 100 which comprises the network system 1 which concerns on 1st Embodiment. It is an image diagram which shows the suspicious person data 122A which concerns on the 1st Embodiment. It is a block diagram which shows the structure of the store terminal 200 which concerns on 1st Embodiment. It is a block diagram which shows the structure of the camera 300 which concerns on 1st Embodiment. It is a block diagram which shows the structure of the settlement machine 500 which concerns on 1st Embodiment. It is a block diagram which shows the structure of the administrator terminal 400 which concerns on 1st Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 1st Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 2nd Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 3rd Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 4th Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 5th Embodiment. It is an image diagram which shows the suspicious person data 122A which concerns on 6th Embodiment. It is a sequence diagram which shows the information processing of the network system 1 which concerns on 6th Embodiment. It is a figure which shows the whole structure of the network system 1 which concerns on 7th Embodiment. It is a block diagram which shows the structure of the vending machine which concerns on 7th Embodiment. It is a figure which shows the whole structure of the network system 1 which concerns on 8th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of them will not be repeated.
<First Embodiment>
<Overall configuration of network system 1>

First, with reference to FIG. 1, the overall configuration of the network system 1 according to the present embodiment will be described. The network system 1 mainly controls a server 100 for realizing a suspicious person detection service, a camera 300 for photographing the surroundings of a product display shelf 700, and controls the camera 300 and handles product purchase data. It includes a store terminal 200 for performing a service, a settlement machine 500 for performing a settlement process for purchasing a product, and the like.
<Outline of operation of network system 1>

Next, an outline of the operation of the network system 1 according to the present embodiment will be described. With reference to FIG. 1, the camera 300 photographs the surroundings of the product display shelf 700 in the store, and transmits the captured image to the store terminal 200 by wired communication or wireless communication. The store terminal 200 transmits the captured image to the server 100 via the Internet, a carrier network, or the like. The server 100 detects a person who is suspiciously behaving based on the captured image, and notifies the store terminal 200, the administrator terminal 400, or the like of a warning when the person is repeatedly detected.

It is preferable that even a person who is determined to behave suspiciously does not notify a warning when the product is purchased.

As described above, the network system 1 according to the present embodiment automatically recognizes that a person who has repeated suspicious behavior has been detected, and reports the information to that effect to the administrator or the like. Hereinafter, a specific configuration of the network system 1 for realizing such a function will be described in detail.
<Configuration of server 100>

The configuration of the server 100 constituting the network system 1 will be described with reference to FIG. First, the configuration of the server 100 will be described. The server 100 includes a CPU (Central Processing Unit) 110, a memory 120, an operation unit 140, and a communication interface 160 as main components.

The CPU 110 controls each part of the camera 300 by executing a program stored in the memory 120 or an external storage medium.

The memory 120 is realized by various RAMs (Random Access Memory), various ROMs (Read Only Memory), and the like. In other words, the memory 120 includes a volatile memory 121 and a non-volatile memory 122. The memory 120 may be included in the server 100, may be detachable from various interfaces of the server 100, or may be a recording medium of another device accessible from the server 100. You may. The memory 120 stores a program executed by the CPU 110, data generated by executing the program by the CPU 110, input data, and other databases used for processing and services according to the present embodiment.

For example, the non-volatile memory 122 stores suspicious person data 122A as shown in FIG. The suspicious person data 122A includes a person ID, feature data such as a face for identifying the person, face image data itself, and suspicious for each visitor who has been determined to behave suspiciously. Stores the correspondence with the number of times it is determined that the person is acting and is identified as the person.

Returning to FIG. 2, the operation unit 140 receives an instruction from the service administrator or the like and inputs the instruction to the CPU 110.

The communication interface 160 is realized by a communication module such as a wireless LAN or a wired LAN. The communication interface 160 exchanges data with the camera 300, another server, or the like by wired communication or wireless communication.
<Structure of store terminal 200>

Next, the configuration of the store terminal 200 constituting the network system 1 according to the present embodiment will be described. With reference to FIG. 4, the store terminal 200 according to the present embodiment has a CPU 210, a memory 220, a display 230, an operation unit 240, a communication interface 260, a speaker 270, and a microphone 280 as main components. And include.

The CPU 210 controls each part of the store terminal 200 by executing a program stored in the memory 220.

The memory 220 is realized by various RAMs, various ROMs, and the like. The memory 220 stores application programs for various services, data generated by executing programs by the CPU 210, data received from the server 100 and the camera 300, data input via the operation unit 240, and the like.

The display 230 displays an image or text based on the data from the CPU 210. The operation unit 240 is composed of a pointing device, a switch, and the like, and inputs various commands from the user to the CPU 210. The store terminal 200 may have a touch panel 250 including a display 230 and an operation unit 240.

The communication interface 260 is realized by wireless communication compliant with WiFi (registered trademark), an antenna for wireless communication with a station on a carrier network, a connector for wired communication, and the like. The CPU 210 receives the captured moving image data from the camera 300 via the communication interface 260, uploads the captured moving image data to the server 100 via the router or the Internet, and notifies the server 100 of a warning about a suspicious person. Is received, payment information and purchase information from the settlement machine 500 are received, and the payment information and purchase information are transmitted to the server 100.

The speaker 270 outputs sound based on the signal from the CPU 210. The microphone 280 acquires voice and inputs a voice signal to the CPU 210.
<Camera 300 configuration>

The configuration of the camera 300 will be described with reference to FIG. The camera 300 includes a CPU 310, a memory 320, an image sensor 330, a communication interface 360, and the like as main components.

The CPU 310 controls each part of the camera 300 by executing a program stored in the memory 320 or an external storage medium.

The memory 320 is realized by various RAMs, various ROMs, and the like. The memory 320 stores a program executed by the CPU 310, data generated by executing the program by the CPU 310, image data acquired by the image sensor 330, and the like.

The image sensor 330 captures a moving image or a still image based on the signal from the CPU 310 and delivers the moving image or the still image to the CPU 310.

The communication interface 360 is realized by a communication module such as a wireless LAN or a wired LAN. The communication interface 360 exchanges data with other devices such as a server by wired communication or wireless communication. In the present embodiment, the CPU 310 uploads the image data obtained by taking a picture using the image sensor 330 to the server 100 via the communication interface 360.
<Configuration of payment machine 500>

The configuration of the checkout machine 500 will be described with reference to FIG. The settlement machine 500 includes a CPU 510, a memory 520, a settlement unit 530, a communication interface 560, and the like as main components.

The CPU 510 controls each part of the checkout machine 500 by executing a program stored in the memory 520 or an external storage medium.

The memory 520 is realized by various RAMs, various ROMs, and the like. The memory 520 stores a program executed by the CPU 510, data generated by executing the program by the CPU 510, and the like.

The settlement unit 530 has an optical sensor, a contact sensor, a short-range communication antenna, etc., and detects inserted coins and banknotes based on a signal from the CPU 510, reads a bar code of a product, and performs a QR code (QR code). The acquired information is input to the CPU 510 by reading a registered trademark), reading an IC card, or communicating with an RFID tag.

The communication interface 560 is realized by a communication module such as a wireless LAN or a wired LAN. The communication interface 560 exchanges data with other devices such as a store terminal 200 and a payment server by wired communication or wireless communication. In the present embodiment, the CPU 510 transmits payment information to the payment server or store terminal 200 via the communication interface 560 based on the information acquired via the payment unit 530, or the product management server or store terminal. Send purchase information to 200.
<Configuration of administrator terminal 400>

Next, the administrator terminal 400 constituting the network system 1 according to the present embodiment includes a personal computer and a server of a head office or a security company that controls the store, a head office manager, a store manager, a security guard, or the like. It is a device used by a person who needs information that a suspicious person has been detected, such as a smartphone or tablet held by the company. As shown in FIG. 7, the administrator terminal 400 according to the present embodiment may have the same configuration as the store terminal 200, and the description will not be repeated here.
<Information processing in network system 1>

Next, with reference to FIG. 8, the information processing of each device in the network system 1 according to the present embodiment will be described.

First, the CPU 310 of the camera 300 uses the image sensor 330 to take a picture of the vicinity of the product display shelf 700 (step S102). The CPU 310 transmits the captured moving image data to the store terminal 200 via the communication interface 260 (step S104).

The CPU 210 of the store terminal 200 receives the moving image data via the communication interface 260 (step S112). The CPU 210 transmits moving image data to the server 100 via the communication interface 260 (step S114).

The CPU 110 of the server 100 receives moving image data from the store terminal 200 via the communication interface 160 (step S122). At this time, the CPU 110 temporarily stores the moving image data in the volatile memory 121. Based on the moving image data, the CPU 110 determines whether or not there is a person who behaves suspiciously around the product display shelf 700 (step S124).

The method for determining whether or not there is a person who is suspicious is not particularly limited. For example, the CPU 110 is programmed with a plurality of judgment steps for identifying suspicious behavior such as hand movement, face movement, and movement path, and performs suspicious behavior when a predetermined condition is satisfied. You may judge that. Alternatively, the CPU 110 may prepare a large number of moving image data corresponding to the sluggish behavior as teacher data and detect the suspicious behavior by using AI (Artificial Intelligence).

If no suspicious behavior is detected (NO in step S124), the CPU 110 ends the process related to the moving image from the store terminal 200 this time.

When a suspicious behavior is detected (YES in step S124), the CPU 110 cuts out a face portion of the person from the moving image data and creates feature data for identifying the person (step S126). At this time, the CPU 110 temporarily stores the face image data and the feature data in the volatile memory 121.

In addition, in order to create feature data, it is preferable to use a plurality of images. That is, it is preferable that the CPU 110 creates feature data by cutting out a plurality of, for example, five face images at timings from the moving image data.

The CPU 110 refers to the suspicious person data 122A and searches whether a suspicious person matching the feature data has already been registered (step S128).

If a suspicious person matching the feature data is already registered, the CPU 110 increases the number of times the suspicious person is detected (step S130). If a suspicious person matching the feature data has not been registered yet, new suspicious person data is created based on the feature data this time.

The CPU 110 determines whether or not the target person has purchased the product (step S132). For example, if the target person is in front of the checkout machine 500 and the purchase information is acquired from the checkout machine 500, it is determined that the target person has purchased the product.

When the target person purchases the product (YES in step S132), the CPU 110 resets the number of detections to 0 (step S142). The CPU 110 ends the processing related to the moving image from the store terminal 200 this time.

When the target person leaves without purchasing the product (NO in step S132), it is determined whether or not the number of detections is a predetermined number of times, for example, 5 times or more. (Step S134).

When the number of detections is less than a predetermined number (NO in step S134), the CPU 110 ends the process related to the moving image from the store terminal 200 this time.

When the number of detections is equal to or greater than a predetermined number (YES in step S134), the CPU 110 stores the face image data in the specific area of the memory 120 as suspicious person information (step S136). For example, the CPU 110 temporarily stores the moving image data received in step S122 and the face image data cut out in step S126 in the volatile memory 121, and in step S136, the image data of the face portion and the moving image It is preferable to store the image data itself and the like in the non-volatile memory 122.

The CPU 110 notifies the store terminal 200 and the administrator terminal 400 of a warning that a suspicious person has been detected via the communication interface 160 (step S138). At this time, the CPU 110 may transmit a suspicious person's face image, moving image, or the like as suspicious person information to the store terminal 200 or the administrator terminal 400. As a result, in the store terminal 200 and the administrator terminal 400, the CPUs 210 and 410 display a warning message indicating that a suspicious person has been detected on the displays 230 and 430, and the speakers 270 and 470 indicate that a suspicious person has been detected. The warning is output as a voice (step S151).

As described above, regarding the network system 1 according to the present embodiment, by searching for a suspicious person with the feature point data of the face image, the privacy is improved as compared with the case where the face image data itself of all the visitors is saved. Can be protected. Then, when suspicious behavior is detected more than a predetermined number of times, the face image data can be warned as suspicious person information, so that security can be ensured.
<Second embodiment>

In the first embodiment, the image data acquired by the server 100 is temporarily stored, and when a predetermined condition is satisfied as a suspicious person, the image data is non-volatilely stored or registered as a suspicious person. It was something to do. However, as shown in FIG. 9, the CPU 110 may also have a configuration in which the moving image data received in step S122 and the face image data cut out in step S126 are stored in the non-volatile memory 122.

In this case, when no suspicious behavior is detected (NO in step S124), it is preferable that the CPU 110 erases the target moving image data and face image data from the non-volatile memory 122 (step). S244).

When the number of detections is less than a predetermined number (NO in step S134), the CPU 110 preferably erases the target moving image data and face image data from the non-volatile memory 122 (step S244). ).
<Third embodiment>

Alternatively, as shown in FIG. 10, the data transmitted from the store terminal 200 to the server 100 may be restricted. In the following, only the parts different from the first embodiment will be described, and the description will not be repeated for the same parts.

The CPU 210 of the store terminal 200 receives the moving image data via the communication interface 260 (step S112). The CPU 210 masks the face portion and then transmits the moving image data to the server 100 via the communication interface 260 (step S314).

The CPU 110 of the server 100 receives moving image data from the store terminal 200 via the communication interface 160 (step S122). Based on the moving image data, the CPU 110 determines whether or not there is a person who behaves suspiciously around the product display shelf 700 (step S124).

If no suspicious behavior is detected (NO in step S124), the CPU 110 ends the process related to the moving image from the store terminal 200 this time.

When a suspicious behavior is detected (YES in step S124), the CPU 110 requests the store terminal 200 for the characteristic data of the person via the communication interface 160 (step S326).

The CPU 210 of the store terminal 200 receives the request from the server 100 (step S316). The CPU 210 cuts out a face portion of a person from the moving image data and creates feature data for identifying the person (step S318). The CPU 210 transmits the feature data to the server 100 via the communication interface 260 (step S320).

The CPU 110 receives the feature data (step S327), refers to the suspicious person data 122A, and searches whether a suspicious person matching the feature data has already been registered (step S128).

If a suspicious person matching the feature data is already registered, the CPU 110 increases the number of times the suspicious person is detected (step S130). If a suspicious person matching the feature data has not been registered yet, new suspicious person data is created based on the feature data this time.

The CPU 110 determines whether or not the target person has purchased the product (step S132). For example, if the target person is in front of the checkout machine 500 and the purchase information is acquired from the checkout machine 500, it is determined that the target person has purchased the product.

When the target person purchases the product (YES in step S132), the CPU 110 resets the number of detections to 0 (step S142). The CPU 110 ends the processing related to the moving image from the store terminal 200 this time.

When the target person leaves without purchasing the product (NO in step S132), it is determined whether or not the number of detections is a predetermined number of times, for example, 5 times or more. (Step S134).

When the number of detections is less than a predetermined number (NO in step S134), the CPU 110 ends the process related to the moving image from the store terminal 200 this time.

When the number of detections is equal to or greater than a predetermined number (YES in step S134), the CPU 110 requests face image data from the store terminal 200 via the communication interface 160 (step S335).

The CPU 210 of the store terminal 200 receives the request from the server 100 (step S336). The CPU 210 cuts out a face portion from the moving image data and transmits the face image data to the server 100 via the communication interface 260 (step S337).

The CPU 110 stores the received face image data as suspicious person information in a specific area of the memory 120 (step S136).
<Fourth Embodiment>

Other devices may play a part or all of the roles of the devices constituting the network system 1 of the above embodiment. Alternatively, a part or all of the roles of one device may be shared by a plurality of devices, or a part or all of the roles of a plurality of devices may be shared by one device. For example, the role of the server 100 may be played by a plurality of servers on the cloud.

Hereinafter, as an example, a case where the store terminal 200 plays a part of the role of the server 100 will be described. In the present embodiment, it is preferable that the store terminal 200 and the server 100 hold the suspicious person data 122A. Information processing of each device in the network system 1 according to the present embodiment will be described with reference to FIG.

First, the CPU 310 of the camera 300 uses the image sensor 330 to take a picture of the surroundings of the product display shelf 700 (step S102). The CPU 310 transmits the captured moving image data to the store terminal 200 via the communication interface 260 (step S104).

The CPU 210 of the store terminal 200 receives the moving image data via the communication interface 260 (step S112). Based on the moving image data, the CPU 210 determines whether or not there is a person who behaves suspiciously around the product display shelf 700 (step S424).

If no suspicious behavior is detected (NO in step S424), the CPU 210 ends the process related to the moving image this time.

When a suspicious behavior is detected (YES in step S424), the CPU 210 cuts out a face portion of the person from the moving image data and creates feature data for identifying the person (step S426).

The CPU 210 refers to the suspicious person data 122A and searches whether a suspicious person matching the feature data has already been registered (step S428).

If a suspicious person matching the feature data is already registered, the CPU 210 increases the number of times the suspicious person is detected (step S430). If a suspicious person matching the feature data has not been registered yet, new suspicious person data is created based on the feature data this time.

The CPU 210 determines whether or not the target person has purchased the product (step S432). For example, if the target person is in front of the checkout machine 500 and the purchase information is acquired from the checkout machine 500, it is determined that the target person has purchased the product.

When the target person purchases the product (YES in step S432), the CPU 210 resets the number of detections to 0 (step S442). The CPU 210 ends the processing related to the moving image this time.

When the target person leaves without purchasing the product (NO in step S432), it is determined whether or not the number of detections is a predetermined number of times, for example, 5 times or more. (Step S434).

When the number of detections is less than a predetermined number (NO in step S434), the CPU 210 ends the process related to the moving image this time.

When the number of detections is equal to or greater than a predetermined number of times (YES in step S434), the CPU 210 cuts out a face portion from the moving image data and uses the face image data as suspicious person information as suspicious person information on the server 100. (Step S436).

Then, in the store terminal 200, the CPU 210 displays a warning message indicating that a suspicious person has been detected on the display 230, or causes the speaker 270 to output a warning message indicating that a suspicious person has been detected by voice (step S151). ..

The CPU 110 of the server 100 receives the face image data from the store terminal 200 via the communication interface 160 (step S435). The CPU 110 stores the face image data as suspicious person information in a specific area of the memory 120 (step S136).

The CPU 110 notifies the administrator terminal 400 of a warning that a suspicious person has been detected via the communication interface 160 (step S138). As a result, in the administrator terminal 400, the CPU 410 displays a warning message indicating that a suspicious person has been detected on the display 430, and causes the speaker 470 to output a warning message indicating that a suspicious person has been detected by voice.
<Fifth Embodiment>

In the above embodiment, the image is stored in the non-volatile memory 122 and the number of times is counted only when a suspicious behavior is detected. However, as shown in FIG. 12, the number of times may be counted each time a person is detected, and a warning may be output when the count reaches a predetermined number of times. Then, the number of detections may be reset when the purchase of the product is accepted. In the following, only the parts different from the first embodiment will be described, and the description will not be repeated for the same parts.

First, the CPU 310 of the camera 300 uses the image sensor 330 to take a picture of the surroundings of the product display shelf 700 (step S102). The CPU 310 transmits the captured moving image data or still image data to the store terminal 200 via the communication interface 260 (step S104).

The CPU 210 of the store terminal 200 receives moving image data or still image data via the communication interface 260 (step S112). The CPU 210 transmits moving image data or still image data to the server 100 via the communication interface 260 (step S114).

The CPU 110 of the server 100 receives moving image data or still image data from the store terminal 200 via the communication interface 160 (step S122). At this time, the CPU 110 temporarily stores the moving image data or the still image data in the volatile memory 121.

The CPU 110 cuts out a face portion of a person from the moving image data or the still image data, and creates feature data for identifying the person (step S126). At this time, the CPU 110 temporarily stores the face image data in the volatile memory 121.

The CPU 110 refers to the suspicious person data 122A and searches whether a suspicious person matching the feature data has already been registered (step S128).

If a suspicious person matching the feature data is already registered, the CPU 110 increases the number of times the suspicious person is detected (step S130). If a suspicious person matching the feature data has not been registered yet, new suspicious person data is created based on the feature data this time.

The CPU 110 determines whether or not the target person has purchased the product (step S132). For example, if the target person is in front of the checkout machine 500 and the purchase information is acquired from the checkout machine 500, it is determined that the target person has purchased the product.

When the target person purchases the product (YES in step S132), the CPU 110 resets the number of detections to 0 (step S142). The CPU 110 ends the processing related to the moving image from the store terminal 200 this time.

When the target person leaves without purchasing the product (NO in step S132), it is determined whether or not the number of detections is a predetermined number of times, for example, 5 times or more. (Step S134).

When the number of detections is less than a predetermined number (NO in step S134), the CPU 110 ends the process related to the image from the store terminal 200 this time.

When the number of detections is equal to or greater than a predetermined number (YES in step S134), the CPU 110 cuts out a face portion from the moving image data or the still image data and stores it in a specific area of the memory 120 as suspicious person information. (Step S136). For example, the CPU 110 temporarily stores the moving image data or still image data received in step S122 and the face image data cut out in step S126 in the volatile memory 121, and in step S136, the image of the face portion. The data, the moving image data itself, and the like are stored in the non-volatile memory 122 for a long period of time.

The CPU 110 notifies the store terminal 200 and the administrator terminal 400 of a warning that a suspicious person has been detected (step S138). As a result, in the store terminal 200 and the administrator terminal 400, the CPUs 210 and 410 display a warning message indicating that a suspicious person has been detected on the displays 230 and 430, and the speakers 270 and 470 indicate that a suspicious person has been detected. The warning is output as a voice (step S151).
<Sixth Embodiment>

In addition to the number of times the suspicious behavior is detected, the CPUs 110 and 210 preferably store the date and time of the suspicious behavior in the memories 120 and 220 when the suspicious behavior is determined. For example, as shown in FIG. 13, the CPUs 110 and 210 may accumulate the history of the detection date and time for each suspicious person as suspicious person information, or for each store or display area without associating with the suspicious person. The date and time when the suspicious behavior occurred may be accumulated.

Then, in step S138, the CPU 110 notifies the store terminal 200 and the administrator terminal 400 of a warning such as the fact that a suspicious person has been detected and the date and time when the suspicious behavior has occurred. As a result, in the store terminal 200 and the administrator terminal 400, the CPUs 210 and 410 display warning messages such as the fact that a suspicious person has been detected on the displays 230 and 430, the date and time when the suspicious behavior occurred, and the speaker. A warning such as the fact that a suspicious person has been detected in 270 or 470 and the date or time when the suspicious behavior has occurred is output by voice (step S151).

Alternatively, as shown in FIG. 14, when a suspicious behavior is detected (YES in step S124), the CPU 110 determines the date and time when the store terminal 200 or the administrator terminal 400 has a suspicious behavior. Notify a warning (step S125). As a result, in the store terminal 200 and the administrator terminal 400, the CPUs 210 and 410 display warning messages such as the date and time when the displays 230 and 430 have suspicious behavior, and the speakers 270 and 470 have suspicious behavior. A warning such as a date or time is output as a voice (step S127).
<7th embodiment>

In the above embodiment, the store where there is no clerk near the product display shelf 700 has been described, but the same configuration and control process may be executed for the unmanned vending machine. As for the unmanned vending machine, as shown in FIG. 15, a product display shelf 700, a camera 300, a checkout machine 500, a store terminal 200, and the like are included in one device or housing. It may be. That is, for example, as shown in FIG. 16, the store terminal 200A as a vending machine is equipped with an image sensor 330, a settlement unit 530, and the like.
<Eighth Embodiment>

Further, the suspicious person detection function of the network system 1 can be used even in a store having a clerk as shown in FIG.
<Summary>

As described above, in the above-described embodiment, a camera for photographing the vicinity of the place where the product is placed and a person performing a predetermined operation are detected based on the image of the camera, and the relevant person is detected. A suspicious person detection system including a control unit for counting the number of times detected for each person by identifying the person and outputting a warning when the predetermined number of times is reached is provided.

Preferably, the control unit is implemented by a communication terminal that is locally connected to the camera. The communication terminal provides the server with image data of a person who has reached a predetermined number of times.

Preferably, the control unit is realized by a communication device locally connected to the camera and a server connected to the communication device via a communication network. The server stores the image data of the person who has reached a predetermined number of times in the non-volatile memory.

Preferably, the control unit is realized by a communication device locally connected to the camera and a server connected to the communication device via a communication network. The server deletes the image data of the person who has not reached the predetermined number of times from the memory.

Preferably, when the identified person purchases the product, the control unit resets the number of times it has been counted.

Preferably, the control unit detects a person performing a predetermined operation based on the image of the camera, creates feature data, identifies the person, and counts the number of times detected for each person. I will do it.

Preferably, the control unit also outputs the timing at which a predetermined operation is performed.

In the above embodiment, an image is acquired from the camera via the communication interface and the communication device interface, a person performing a predetermined operation is detected from the image, and the person is identified. A suspicious person detection device including a control unit for counting the number of times and outputting a warning when a predetermined number of times is reached is provided.

In the above embodiment, a suspicious person detection method is provided. The suspicious person detection method includes a step in which the camera takes a picture of the place where the product is placed, a step in which the control unit detects a person performing a predetermined operation based on the image taken by the camera, and a control unit. However, it includes a step of counting the number of times detected for each person by identifying the person, and a step of outputting a warning when the control unit reaches a predetermined number of times.

In the above embodiment, a suspicious person detection method is provided. The suspicious person detection method includes a step in which the control unit acquires an image captured by the camera via a communication interface and a step in which the control unit detects a person performing a predetermined operation based on the image captured by the camera. The control unit includes a step of counting the number of times detected for each person by identifying the person, and a step of outputting a warning when the control unit reaches a predetermined number of times.

In the above embodiment, the number of times detected for each person is determined by identifying the person who was photographed based on the camera for photographing the vicinity of the place where the product is placed and the image of the camera. A suspicious person detection system is provided that counts, outputs a warning when a predetermined number of times is reached, and when a specified person purchases a product, a control unit for resetting the number of times accumulated up to that point is provided.

Preferably, the control unit counts the number of detections only for a person performing a predetermined operation.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown not by the above description but by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

1: Network system 100: Server 110: CPU
120: Memory 121: Volatile memory 122: Non-volatile memory 122A: Suspicious person data 140: Operation unit 160: Communication interface 200: Store terminal 200A: Vending machine 210: CPU
220: Memory 230: Display 240: Operation unit 250: Touch panel 260: Communication interface 270: Speaker 280: Microphone 300: Camera 310: CPU
320: Memory 330: Image sensor 360: Communication interface 400: Administrator terminal 420: CPU
430: Display 470: Speaker 500: Settlement machine 510: CPU
520: Memory 530: Settlement unit 560: Communication interface 700: Display shelf

Claims (10)

A camera for taking pictures near the place where the product is placed,
To detect a person performing a predetermined operation based on the image of the camera, count the number of detections for each person by identifying the person, and output a warning when the predetermined number of times is reached. Suspicious person detection system equipped with a control unit. The control unit is realized by a communication terminal locally connected to the camera.
The network system according to claim 1, wherein the communication terminal provides image data of a person who has reached a predetermined number of times to a server. The control unit is realized by a communication device locally connected to the camera and a server connected to the communication device via a communication network.
The network system according to claim 1, wherein the server stores image data of a person who has reached a predetermined number of times in a non-volatile memory. The control unit is realized by a communication device locally connected to the camera and a server connected to the communication device via a communication network.
The network system according to claim 1, wherein the server deletes image data of a person who has not yet reached a predetermined number of times from the memory. The network system according to any one of claims 1 to 4, wherein when the specified person purchases a product, the control unit resets the number of times counted up to that point. The control unit detects a person performing a predetermined operation based on the image of the camera, creates feature data, identifies the person, and counts the number of times detected for each person. The network system according to any one of claims 1 to 5. The network system according to any one of claims 1 to 6, wherein the control unit also outputs a timing at which the predetermined operation is performed. Communication interface and
An image is acquired from the camera via the communication device interface, a person performing a predetermined operation is detected from the image, and the number of detected times is counted by identifying the person, and the predetermined number of times is reached. Then, a suspicious person detection device including a control unit for outputting a warning. It is a suspicious person detection method
The step that the camera takes a picture of the place where the product is placed,
A step in which the control unit detects a person performing a predetermined operation based on an image captured by the camera.
A step of counting the number of times the control unit detects the person by identifying the person, and
A suspicious person detection method including a step in which the control unit outputs a warning when a predetermined number of times is reached. It is a suspicious person detection method
The step that the control unit acquires the captured image of the camera via the communication interface,
A step in which the control unit detects a person performing a predetermined operation based on an image captured by the camera.
A step of counting the number of times the control unit detects the person by identifying the person, and
A suspicious person detection method including a step in which the control unit outputs a warning when a predetermined number of times is reached.

Images