JP2021043996A - Notification device and notification system - Google Patents

Abstract

To easily implement notification.SOLUTION: A notification device according to an example of one embodiment comprises: a detection unit which detects and identifies a user in a predetermined facility on the basis of an image that is captured in the predetermined facility; and a notification unit which notifies the user of predetermined information by changing an illumination mode of an illumination device that illuminates the predetermined facility, on the basis of a result of detection by the detection unit.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to notification devices and notification systems.

Conventionally, there is known a technique for authenticating a user using an image captured by a camera. As an example of such technology, the face of the worker is authenticated using the image captured by the camera, the working condition of the worker is managed using the authentication result, and the working condition of the worker satisfies the workable requirement. If not, there is known a technique for notifying an operator of a warning by displaying a warning on a terminal device used by the operator.

Japanese Patent Application No. 2016-0246889

However, in the prior art, it may not be possible to easily realize the notification.

For example, if a suspicious person's face is authenticated using an image in the facility and a suspicious person is detected, a warning is displayed on the terminal device used by the clerk to notify the presence of the suspicious person. A mode of application is conceivable. However, with such a technique, since the store clerk keeps checking the terminal device, it cannot be said that the warning can be easily notified.

An object to be solved by the present invention is to provide a notification device and a notification system that easily realize notification.

The notification device according to an example of the embodiment is a detection unit that detects a user of a predetermined facility from an image taken by the imaging unit, and a lighting device that illuminates a position where an object associated with the user is arranged. It is characterized in that it is provided with a notification unit that notifies the user of the position of the object by changing the illumination mode.

According to the notification device according to an example of the embodiment, notification can be easily realized.

FIG. 1 is a diagram showing an example of an information management system according to an embodiment. FIG. 2 is a diagram showing an example of a lighting device according to an embodiment. FIG. 3 is a diagram showing an example of the functional configuration of the information management device according to the embodiment. FIG. 4 is a diagram showing an example of information registered in the external information database according to the embodiment. FIG. 5 is a diagram showing an example of information registered in the internal information database according to the embodiment. FIG. 6 is a diagram showing an example of information registered in the lighting equipment database according to the embodiment. FIG. 7 is a diagram showing an example of information registered in the suspicious person database according to the embodiment. FIG. 8 is a diagram showing an example of a process in which the information management device according to the embodiment notifies the existence and position of a suspicious person. FIG. 9 is a flowchart illustrating an example of a flow of management processing executed by the information management device according to the embodiment. FIG. 10 is a flowchart illustrating an example of a flow of notification processing executed by the information management device according to the embodiment. FIG. 11 is a diagram showing an example of variations executed by the information management device according to the embodiment.

Hereinafter, the notification device, the notification system, and the notification method according to the embodiment will be described with reference to the drawings. Configurations having the same function in the embodiment are designated by the same reference numerals, and duplicate description will be omitted. The notification device, the notification system, and the notification method described in the following embodiments are merely examples, and the embodiments are not limited. For example, in the following embodiments, the notification device, notification system and notification method can be applied to any system. The following embodiments may be appropriately combined within a consistent range.

The notification device according to the following embodiment is, for example, the information management device 10 included in the information management system 1. The information management device 10 has a detection unit 45 that detects and identifies the attributes of users in the predetermined facility based on an image taken in the predetermined facility, and a predetermined facility based on the detection result by the detection unit 45. The lighting device LD (for example, the lighting devices SL01 to SL04, the emergency light EL01) is provided with a notification unit 46 for notifying predetermined information by changing the lighting mode.

Further, the information management device 10 according to the following embodiment detects a user who satisfies a predetermined requirement for determining an attribute among the users. Then, when the information management device 10 detects a user who satisfies the predetermined requirement, the information management device 10 notifies the existence of the user who satisfies the predetermined requirement. In addition, the attribute of the user is a type related to the quality judgment of the detection target person such as a normal person or a suspicious person based on the information obtained from the image, or by detecting and discriminating a specific user from a plurality of users. It shows user-specific information and the like. In short, it does not simply detect the existence of a person by an image, but uses the information obtained by the image to select a person by an arbitrary type and perform a process to obtain the classified information. ..

Further, the information management device 10 according to the following embodiment detects the position of the user who satisfies the predetermined requirement among the users, and the detected position among the lighting equipment LDs arranged in the predetermined facility. The lighting mode of the lighting device LD at the position corresponding to is changed.

Further, the information management device 10 according to the following embodiment changes the lighting device LD that changes the lighting mode according to the detected change in the position.

Further, the information management device 10 according to the following embodiment acquires the accuracy that the user satisfies a predetermined requirement, and changes the illumination mode of the lighting device DL to the illumination mode according to the acquired accuracy.

Further, the information management device 10 according to the following embodiment changes the lighting mode of the lighting device LD in a mode that is difficult for a user who satisfies a predetermined requirement to identify.

Further, the information management device 10 according to the following embodiment changes the frequency at which the lighting device LD outputs light or the color temperature of the light output by the lighting device LD.

Further, the information management device 10 according to the following embodiment lights up the lighting device (for example, the emergency light EL01) that lights up when the power from the system power supply is stopped, among the lighting device LDs, to obtain predetermined information. Is notified.

Further, the information management device 10 according to the following embodiment changes the lighting mode of the lighting device LD at a position corresponding to the predetermined information among the lighting device LDs arranged in the predetermined facility.

Further, the notification system (for example, information management system 1) according to the following embodiment includes a lighting device LD installed in a predetermined facility and the above-mentioned information management device 10.

[Embodiment]
(Overview of information management system)
Hereinafter, an example of the information management system 1 having the information management device 10 which is an example of the management device will be described. FIG. 1 is a diagram showing an example of an information management system according to an embodiment. In the example shown in FIG. 1, the information management system 1 includes an information management device 10, a plurality of lighting devices LD (lighting device SL01, emergency light EL01, etc.), a store terminal ST, a cloud server CS, a vendor server BS, and a shop center server. Has SC. The type and number of the lighting equipment LD, the store terminal ST, the cloud server CS, the vendor server BS, and the shop center server SC connected to the information management system 1 can be arbitrarily set.

Here, the lighting device LD, the information management device 10, and the store terminal ST are installed in a predetermined facility such as a store that sells products, and a network such as a LAN (Local Area Network) in the facility (hereinafter, "" It is assumed that it is connected to the "internal network"). Further, the cloud server CS, the vendor server BS, and the shop center server SC are installed outside the predetermined facility, and are networks such as LAN and the Internet outside the predetermined facility (hereinafter, collectively referred to as "external network"). It is assumed that it is connected to. In the following description, the predetermined facilities where the internal network is arranged are collectively referred to as "stores".

Further, in the following description, it is assumed that the information management device 10 has a gateway function and has a function of communicating with a cloud server CS connected to an external network, but the embodiment is limited to this. It is not something that is done. For example, the information management system 1 may install a gateway device in a predetermined facility and relay communication between the information management device 10 and the cloud server CS via the gateway device.

The lighting device LD is a lighting device that uses a semiconductor light emitting element such as an LED (Light Emitting Diode), a fluorescent lamp, or a light bulb as a light source, is installed in a predetermined facility, and can illuminate the predetermined facility in a predetermined lighting mode. Is. For example, the lighting device SL01 is a lighting device that is installed on the ceiling and illuminates the inside of the facility by turning on the light source using the electric power supplied from the system power supply. Further, the emergency light EL01 has various storage batteries, and is an emergency light that lights up using the power charged in the storage battery when the power supply from the system power supply is stopped in an emergency such as an earthquake or a fire. is there. As the lighting device LD, any lighting device such as a ceiling light, a base light, or a downlight can be adopted.

Here, it is assumed that the lighting device LD can perform lighting in various modes according to the control from the information management device 10. For example, when the lighting equipment LD receives a signal conforming to an arbitrary control standard such as a DMX standard signal or a PWM (Pulse Width Modulation) signal from the information management device 10, the received signal can be used in any lighting mode indicated by the received signal. , It is possible to illuminate the inside of a predetermined facility.

To give a more specific example, the lighting device SL01 has a function of blinking a light source at a frequency that is invisible to the human eye under the control of the information management device 10, and has a function of blinking a light source in primary colors such as red, blue, and green. It is a lighting device that can control the color of the output light by controlling each light individually. Further, the emergency light EL01 is a lighting device capable of turning on the light source at an arbitrary timing according to the control from the information management device 10.

Further, the lighting device SL01 is a lighting device having a lighting unit L01 that illuminates the inside of a predetermined facility and a photographing unit C01 that photographs the inside of the predetermined facility. For example, FIG. 2 is a diagram showing an example of a lighting device according to an embodiment. In the example shown in FIG. 2, the lighting device SL01 includes a housing LB01, a lighting unit L01, a photographing unit C01, and a notification lamp LM01.

The lighting unit L01 is a light source included in the lighting device SL01, and is, for example, a semiconductor light emitting element such as an LED. The photographing unit C01 is a photographing device for photographing a predetermined range from the position where the lighting device SL01 is installed, and is, for example, a camera that photographs using various image pickup elements. The photographing unit C01 may capture not only a still image but also a moving image.

The notification lamp LM01 is a lamp capable of outputting light of a predetermined color according to control from the information management device 10, and performs various notifications by outputting light of a color such as green, yellow, or red, for example. be able to. The housing LB01 is a housing of the lighting device SL01, and is a communication circuit for communicating with the lighting unit L01, the photographing unit C01, the control circuit of the notification lamp LM01, the power supply circuit, and the information management device 10. Is placed.

As described above, the lighting device SL01 has a function as a lighting device and a function as a camera. When such a lighting device SL01 is installed in a store, it is not necessary to install a surveillance camera, so that the intimidating feeling in the store due to the installation of the surveillance camera can be reduced.

Returning to FIG. 1, the explanation will be continued. The store terminal ST is a terminal device used in a store, and is realized by an information processing device such as a smartphone, a tablet terminal, or a PC (Personal Computer). For example, the store terminal ST is a terminal device used by the store clerk S01, and outputs various information notified from the information management device 10.

The information management device 10 is an information processing device such as a PC or a server device, and is an information processing device installed in a store. For example, the information management device 10 analyzes the image acquired by the lighting device LD and estimates various information about the photographed user. Then, the information management device 10 detects a suspicious person in the store based on the estimation result, and when the suspicious person is detected, notifies the store terminal ST of the suspicious person's entry or the like, thereby shoplifting or the like. Such acts are prevented in advance.

To give a more specific example, the information management device 10 holds data for face recognition of a shoplifting addict, and executes face recognition of a user captured in an image acquired by the lighting device LD. Then, when the face of the shoplifting addict is authenticated, the information management device 10 displays the face photograph of the shoplifting addict and various information on the store terminal ST.

Further, the information management device 10 analyzes the behaviors of the users U01 and U02 in the store based on the image taken by the lighting device LD, and detects the user who performs the suspicious behavior based on the analysis result. For example, the information management device 10 detects behaviors that shoplifting addicts are likely to perform, such as users with suspicious behavior and users who put products in their bags. Then, when the information management device 10 detects a user who performs suspicious behavior, the information management device 10 outputs information indicating such fact and the user to the store terminal ST.

The cloud server CS is an information processing device installed in an external network and providing various cloud-related services to the information management device 10, and is realized by a server device, a cloud system, or the like. The vendor server BS is an information processing device used by a provider who provides the information management system 1 to a store or the like, that is, a vendor, and is realized by a server device, a cloud system, or the like. The shop center server SC is an information processing device used by the headquarters of a store in which the information management device 10 is installed, and is realized by a server device, a cloud system, or the like.

(Example of management process)
Here, the movements of the users U01 and U02 (hereinafter, may be collectively referred to as "user U") in the store are analyzed from the image taken by the lighting device LD, and the analysis result is used for the store design or the suspicious person. There may be a desire to use it for detection. For example, by collecting the movements of the user U who did not perform suspicious behavior as learning data and letting a model such as DNN (Deep Neural Network) or SVM (Support Vector Machine) learn the characteristics of the movements collected as learning data. , It is conceivable that there is a request to generate or update a model for determining whether or not the user is a suspicious person from the operation of the user U. Further, in order to satisfy such a demand, there is also a method of uploading an image taken by the lighting device LD and various information estimated from the image to the cloud server CS, and generating or updating a model based on the uploaded information. Conceivable.

However, the image taken by the lighting device LD and various types of information specified from the image may include personally identifiable information, that is, personal information. For example, the face of the user U may be photographed in order to detect a suspicious person in the image taken by the lighting device LD or various information estimated from the image. Uploading such personal information on the cloud server CS may not protect the personal information properly.

Therefore, the information management system 1 executes the following processing. First, the lighting device LD photographs the user U of the store by photographing the inside of the store (step S1). In such a case, the information management device 10 acquires an image taken by the lighting device LD (step S2). Then, the information management device 10 analyzes the image and estimates various information of the user U (step S3).

For example, the information management device 10 generates information indicating the face of the user U, such as the face of the user U and the feature amount of the image in which the face of the user U is captured, from the image taken by the lighting device LD. .. Further, the information management device 10 identifies the operation of the user U from the image taken by the lighting device LD, and generates information indicating the specified operation (hereinafter, may be referred to as "operation information"). .. Further, the information management device 10 generates information indicating the gender, age, height, etc. of the user U, information indicating whether or not the user is determined to be a suspicious person, and the like from the image taken by the lighting device LD.

Then, the information management device 10 classifies the estimation result into personal information and other information (step S4). For example, the information management device 10 may use information indicating the face of the user U, such as a face photograph of the user U, a feature amount of an image in which the face of the user U is captured, or information indicating the name of the user U. Is classified as personal information. Further, the information management device 10 uses information other than personal information (hereinafter, "non-personal information") for operating information, information indicating the gender, age, height, etc. of the user U, or information indicating whether or not the user is determined to be a suspicious person. It may be described as "personal information"). Then, the information management device 10 manages the personal information on the internal network, and outputs information other than the personal information to the cloud server CS on the external network (step S5).

On the other hand, the shop center server SC acquires non-personal information from the cloud server CS (step S6). Then, the shop center server SC can carry out statistical reports and store design using non-personal information (step S7). For example, the shop center server SC may create various reports showing statistics of the gender and age of the user U who uses the store. Further, the shop center server SC may design the store based on the flow line of the user U based on the operation information.

Further, the vendor server BS acquires non-personal information from the cloud server CS (step S8). Then, the vendor server BS executes an update of the system software executed by the information management device 10 in order to detect a suspicious person (step S9). For example, the vendor server BS extracts the operation information indicating the behavior of the user U who is not determined to be a suspicious person from the operation information included in the non-personal information. Then, the vendor server BS uses the extracted operation information as learning data, and generates a model for determining whether or not the user U is a suspicious person.

Further, the vendor server BS updates the system software so as to detect a suspicious person using the generated model, and registers the updated system software on the cloud server CS (step S10). In such a case, the cloud server CS reflects the updated system software on the information management device 10 (step S11). As a result, the information management device 10 detects a suspicious person by using the updated system software (step S12).

For example, the information management device 10 identifies the movement of the user U in the store from the image taken by the lighting device LD, and uses the model newly generated by the vendor server BS to identify the movement of the suspicious person. It is determined whether or not it is. Then, when the information management device 10 determines that the specified movement is the movement of a suspicious person, the information management device 10 detects the user U as a suspicious person.

Further, the information management device 10 detects a suspicious person registered in advance by using the personal information managed by the internal network. For example, the information management device 10 internally holds authentication data for performing face recognition of shoplifting addicts and the like. Further, the information management device 10 identifies the face of the user U from the image captured by the lighting device LD. Then, the information management device 10 performs face recognition of the user U using the authentication data, and detects a suspicious person.

In this way, the information management device 10 manages personal information on the internal network and outputs non-personal information on the external network. Therefore, the information management device 10 can output information (for example, operation information) that can be used for determining whether or not the user U in the store is a suspicious person while protecting personal information.

(Example of notification processing)
Here, when the information management device 10 detects a suspicious person, the information management device 10 notifies the clerk S01 of the existence of the suspicious person via the store terminal ST (step S13). However, in such a notification mode, the clerk S01 needs to keep checking the store terminal ST.

Therefore, the information management device 10 notifies the clerk S01 of the existence of the suspicious person and the position in the store where the suspicious person is present by controlling the lighting device in the vicinity of the suspicious person to a predetermined lighting mode (step S14). (Step S15).

For example, the information management device 10 detects a user U who satisfies a specific requirement, such as a user U who has performed a suspicious action or a user U who is determined to be a suspicious person by face recognition. Then, the information management device 10 notifies the clerk S01 of the presence of the detected user U and the position of the user U by changing the lighting mode of the lighting device LD.

For example, the information management device 10 may notify the presence of a suspicious person by changing the color temperature of the light output by the lighting device SL01. For example, the information management device 10 may control the lighting device SL01 so as to change the color of the light output by the lighting device SL01 from neutral white to a color such as red or yellow. Further, for example, the information management device 10 may notify the presence of a suspicious person by changing the color in which the notification lamp LM01 lights up.

Further, the information management device 10 may control the lighting device SL01 so that the lighting unit L01 blinks at a frequency higher than a predetermined frequency. That is, the information management device 10 may notify the suspicious person in a manner in which the suspicious person is difficult to identify by blinking the illumination unit L01 at a frequency high enough for the store user U to be difficult to identify. In such a case, the store clerk S01 can identify the notification of the suspicious person by causing the store terminal ST to execute an application for detecting the light blinking at a high frequency. Further, the clerk S01 may identify the notification by the lighting device SL01 by using, for example, a wearable device or the like that detects light blinking at a high frequency.

Further, since the lighting device SL01 is lit in the normal state, it is considered unlikely that a suspicious person will be noticed even if the emergency light EL01 is lit. Therefore, the information management device 10 may notify the existence of a suspicious person by turning on the lighting device LD, that is, the emergency light EL01, which lights up when the power from the system power supply is stopped.

Further, the information management device 10 detects the position of the user U satisfying a predetermined requirement, and changes the lighting mode of the lighting device LD at the position corresponding to the detected position among the lighting device LDs arranged in the store. You may let me. Further, the information management device 10 may change the lighting device that changes the lighting mode according to the change in the detected position.

For example, the information management device 10 identifies the position of the user U determined to be a suspicious person from the image analysis result. Then, the information management device 10 may change the lighting mode of the lighting device LD installed at the specified position. As a result of such processing, the information management device 10 can notify the clerk S01 of the position of the suspicious person by, for example, changing the lighting device that changes the lighting mode as the suspicious person moves.

Here, a method of detecting a suspicious person by a model that outputs the probability that the behavior of the user U is a suspicious behavior, that is, the probability that the user U is a suspicious person can be considered. Therefore, the information management device 10 may control the lighting device LD in a lighting mode according to the certainty that the user is a suspicious person.

For example, in the information management device 10, when the face of the user U01 matches the face photograph of the shoplifting addict registered in advance, or when the probability that the behavior of the user U01 is a suspicious behavior exceeds the first threshold value. , The lighting device SL01 at the position where the user U01 is located may be lit in red, or may be blinked at the first frequency. Further, when the probability that the behavior of the user U01 is suspicious behavior is equal to or less than the first threshold value and equal to or higher than the second threshold value, the information management device 10 turns the lighting device SL01 at the position where the user U01 is located in yellow. It may be turned on or blinked at a second frequency higher than the first frequency (ie, a frequency that is more difficult to identify). Further, the information management device 10 may light the lighting device SL01 in a normal lighting mode when the probability that the behavior of the user U01 is a suspicious behavior is equal to or less than the second threshold value.

In this way, the information management device 10 detects the user of the store based on the image taken in the store. Then, the information management device 10 notifies predetermined information by changing the lighting mode of the lighting device LD that illuminates the store based on the detection result. For example, when the information management device 10 detects a user U who becomes a suspicious person, the information management device 10 notifies the clerk S01 of the existence and location of the suspicious person by changing the lighting mode of the lighting device LD. Therefore, the information management device 10 can easily notify various information without having the store clerk S01 keep checking the store terminal ST.

(Example of functional configuration)
Next, an example of the functional configuration of the information management device 10 will be described with reference to FIG. FIG. 3 is a diagram showing an example of the functional configuration of the information management device according to the embodiment. As shown in FIG. 3, the information management device 10 includes an external network communication unit 21, an internal network communication unit 22, a storage unit 30, and a control unit 40.

The external network communication unit 21 is a communication device realized by, for example, a NIC (Network Interface Card) or the like. The external network communication unit 21 communicates with a network N such as the Internet by wire or wirelessly, and relays communication between the information management device 10 and the cloud server CS, the vendor server BS, and the shop center server SC. That is, the external network communication unit 21 communicates with various information processing devices on the external network.

The internal network communication unit 22 is realized by, for example, a NIC or the like, and relays communication between the information management device 10 and the lighting device LD or the store terminal ST via the network in the store, that is, the internal network.

The storage unit 30 is realized by, for example, a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or a RAM (Random Access Memory). As shown in FIG. 3, the storage unit 30 includes an external information database 31, an internal information database 32, a lighting equipment database 33, and a suspicious person database 34 (hereinafter, may be collectively referred to as “each database 31 to 34”). Remember.

Hereinafter, an example of information registered in each database 31 to 34 will be described with reference to FIGS. 4 to 7. In the external information database 31, among the information about the user U estimated from the image, information that is non-personal information, that is, external information output to the external network is registered. For example, FIG. 4 is a diagram showing an example of information registered in the external information database according to the embodiment. In the example shown in FIG. 4, the external information database 31 contains items such as "data ID (Identifier)", "acquisition date and time", "operation information", "gender information", "age information", and "height information". The information you have is registered. In addition to the information shown in FIG. 4, it is assumed that the external information database 31 is registered with various information that is non-personal information and is information about the user U estimated from the image.

Here, the "data ID" is an identifier for identifying non-personal information registered in the external information database 31. The "acquisition date and time" is information indicating the acquisition date and time of non-personal information. Further, the "motion information" is information indicating the movement of the user estimated from the image, and is, for example, motion capture data. Further, the "gender information" is information indicating the gender of the user U estimated from the image. Further, the "age information" is information indicating the age of the user U estimated from the image. Further, the "height information" is information indicating the height of the user U estimated from the image.

For example, in the example shown in FIG. 4, the data ID "data ID # 1", the acquisition date and time "date and time # 1", the operation information "operation information # 1", the gender information "male", the age information "30s", and the height. Information Information such as "170 cm to 180 cm" is registered. Such information is the information obtained by the non-personal information indicated by the data ID "data ID # 1" at the acquisition date and time "date and time # 1", and the operation indicated by the operation information "operation information # 1" is performed. Indicates that the user is a man in his thirties with a height of 170 cm to 180 cm.

In the internal information database 32, among the information about the user U estimated from the image, information that becomes personal information, that is, internal information managed by the internal network is registered. For example, FIG. 5 is a diagram showing an example of information registered in the internal information database according to the embodiment. In the example shown in FIG. 5, information having items such as "data ID", "acquisition date and time", "image information", "name", and "correspondence flag" is registered in the internal information database 32. In addition to the information shown in FIG. 5, it is assumed that various personal information is registered in the internal information database 32.

Here, the "data ID" is an identifier for identifying personal information. The "acquisition date and time" is information indicating the acquisition date and time of personal information. Further, the "image information" is an image including a user's face extracted from the image, and is, for example, an image used when performing face authentication. The "name" is the name of the user U estimated from the image. Further, the "correspondence flag" is information indicating whether or not the corresponding user has called out when he / she is a suspicious person. For example, a user who is determined to be a suspicious person is set with a response flag "1", and a user who is determined not to be a suspicious person or a user who is determined to be a suspicious person is called by a clerk. Correspondence flag "0" is set for the performed user.

For example, in the example shown in FIG. 5, information such as data ID "data ID # 1", acquisition date and time "date and time # 1", image information "image information # 1", name "unknown", and correspondence flag "0" is registered. ing. Such information is the personal information indicated by the data ID "data ID # 1", which is the information acquired on the acquisition date and time "date and time # 1", and is included in the image information "image information # 1". Indicates that the name of the user is unknown and has not been determined to be a suspicious person, or that the user has been determined to be a suspicious person but has been called out.

In the lighting equipment database 33, various information related to the lighting equipment LD such as the type and installation position of the lighting equipment LD that illuminates the store is registered. For example, FIG. 6 is a diagram showing an example of information registered in the lighting equipment database according to the embodiment. In the example shown in FIG. 6, information having items such as "lighting device ID", "type", "installation position", and "acquired data" is registered in the lighting device database 33. In addition to the information shown in FIG. 6, it is assumed that various information related to the lighting equipment is registered in the lighting equipment database 33.

Here, the "lighting device ID" is an identifier for identifying the lighting device. Further, the "type" means that the lighting device indicated by the associated "lighting device ID" is a lighting device having a photographing unit C01, that is, a lighting device with a camera, a normal lighting device, or an emergency. It is information indicating whether it is a light. The "installation position" is information indicating the installation position of the lighting equipment. Further, the "acquired data" is information indicating image data acquired by a lighting device with a camera.

For example, in the example shown in FIG. 6, the lighting device database 33 contains the lighting device ID “lighting device # 1”, the type “lighting device with camera”, the installation position “installation position # 1”, and the acquired data “image data # 1”. "Is registered in association with each other. For such information, the type of lighting device indicated by the lighting device ID "lighting device # 1" is "lighting device with camera", and it is installed at "installation position # 1", and "image data # 1" is displayed. Indicates that it has been acquired.

In the suspicious person database 34, authentication data for performing face recognition of a suspicious person is registered. For example, FIG. 7 is a diagram showing an example of information registered in the suspicious person database according to the embodiment. In the example shown in FIG. 7, information such as "suspicious person ID", "authentication data", and "name" is registered in the suspicious person database 34. In addition to the information shown in FIG. 7, it is assumed that arbitrary information about the suspicious person is registered in the suspicious person database 34.

Here, the "suspicious person ID" is an identifier for identifying a suspicious person. Further, the "authentication data" is data for authentication used when performing face authentication of a suspicious person indicated by the associated "suspicious person ID". The "name" is the name of the suspicious person indicated by the associated "suspicious person ID".

For example, in the example shown in FIG. 7, the suspicious person ID “suspicious person # 1”, the authentication data “authentication data # 1”, and the name “name # 1” are registered in association with each other. Such information indicates that the name of the suspicious person indicated by the suspicious person ID "suspicious person # 1" is "name # 1" and the authentication data for performing face recognition is "authentication data # 1". ..

Returning to FIG. 3, the explanation will be continued. The control unit 40 stores in a storage device inside the information management device 10 by an integrated circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). It is realized by executing various programs being executed with a RAM or the like as a work area.

As shown in FIG. 3, the control unit 40 includes an acquisition unit 41, an estimation unit 42, a management unit 43, an output unit 44, a detection unit 45, and a notification unit 46. The acquisition unit 41 acquires an image taken by a lighting device LD installed in a predetermined facility such as a store. For example, the acquisition unit 41 refers to the lighting device database 33 and identifies the lighting device SL01 with a camera. Then, the acquisition unit 41 communicates with the specified lighting device SL01 via the internal network, and acquires an image taken by the lighting device SL01 using the imaging unit C01. The acquisition unit 41 may acquire not only the image taken by the lighting device LD but also the image taken by various shooting devices such as a security camera installed in the store.

The estimation unit 42 estimates information such as various attributes related to the user U in the store based on the image captured in the store. For example, the estimation unit 42 identifies the range in which the user U in the store is imaged by using an arbitrary image analysis technique. Further, the estimation unit 42 identifies the range in which the face of the user U is captured from the specified range, and extracts the specified range as image information for performing face recognition. In addition, the estimation unit 42 estimates the gender and age of the user U from the face of the user U and the physical characteristics of the user U. In addition, the estimation unit 42 estimates the height of the user U from the result of comparing the size with a predetermined reference object such as a shelf arranged around the user U. In addition, the estimation unit 42 uses a technique such as motion capture to generate motion information indicating the movement of the user.

In addition, the estimation unit 42 uses the information registered in the suspicious person database 34 to authenticate the face of the photographed user U. Then, when the face of the user U matches the face of the suspicious person, the estimation unit 42 uses the name of the suspicious person as the name of the user U.

In addition, the estimation unit 42 may extract image information for performing face recognition of the user U by using images taken by a plurality of cameras in an integrated manner. For example, the estimation unit 42 synthesizes a face image of the right half of the user U taken by a certain camera and an image of the face of the left half of the user U taken by another camera to perform face recognition. You may generate an image for doing this. The image used for such composition may be an image taken at the same time or an image taken at a different time. That is, the estimation unit 42 may estimate the face image of the user U from the image taken in the store by using an arbitrary image composition technique.

The management unit 43 manages personally identifiable information among the estimated information on the internal network. For example, among the information estimated by the estimation unit 42, the management unit 43 includes personal information such as a face image and a name, motion information related to the user's behavior, and non-personal information such as gender information, age information, and height information. And classify. Then, the management unit 43 registers the personal information in the internal information database 32 and registers the non-personal information in the external information database 31.

In addition, the management unit 43 applies arbitrary information holding technology, such as granting a predetermined access right to the information registered in the internal information database 32, to obtain the information registered in the internal information database 32. Set the information so that it can be used only on the internal network. As a result of such processing, the management unit 43 can manage personal information on the internal network.

The output unit 44 outputs the estimated information that cannot identify an individual to an information processing device on the external network. For example, the output unit 44 outputs the information registered as non-personal information in the external information database 31 to the cloud server CS on the external network at a predetermined processing timing. The non-personal information output on the cloud server CS in this way is used for analyzing the behavior of the user U in the store and generating a model for determining whether or not the user U is a suspicious person. It will be used.

The detection unit 45 detects and identifies the attribute of the user U of the store based on the captured image. More specifically, the detection unit 45 is a user who is registered as a suspicious person in the suspicious person database 34, a user U who moves suspiciously, or a user who satisfies a predetermined requirement for determining an attribute. Detect. For example, the detection unit 45 performs face recognition of the user U using the information registered in the suspicious person database 34 and the face image of the user U estimated by the estimation unit 42. Then, when the face authentication of the user U is successful, the detection unit 45 determines the user U as a suspicious person.

Further, the detection unit 45 detects a suspicious person by using a model for determining whether or not the user U is a suspicious person based on the movement of the user U. For example, the detection unit 45 uses a model to determine whether or not the operation of the user U indicated by the operation information estimated by the estimation unit 42 is a suspicious operation, and when it is determined that the operation is suspicious. , The user U is determined to be a suspicious person. The detection unit 45 may use the model to acquire the probability that the user U is a suspicious person. Then, when the detection unit 45 determines that the user U is a suspicious person, the detection unit 45 sets the corresponding flag associated with the image information of the user U to "1".

When the detection unit 45 determines that the user U is a suspicious person, the detection unit 45 identifies the position of the user U based on the captured image. For example, the detection unit 45 identifies the image captured by the user U determined to be a suspicious person from the image taken by the lighting device LD in real time, and sets the installation position of the lighting device LD that has taken the specified image as a suspicious person. It is specified as the position of the user U determined to be. The detection unit 45 may specify the position of the user U by using an arbitrary image analysis technique.

The notification unit 46 notifies predetermined information by changing the lighting mode of the lighting device LD based on the detection result. More specifically, when a user satisfying a predetermined requirement such as a suspicious person is detected, the notification unit 46 notifies the existence of a user satisfying the predetermined requirement. For example, when the detection unit 45 specifies the position of the user determined to be a suspicious person, the notification unit 46 specifies the lighting device LD installed at the specified position from the lighting device database 33. Then, the notification unit 46 notifies the clerk S01 of the existence of the suspicious person and the position of the suspicious person by changing the lighting mode of the specified lighting device LD. Further, when the position detected by the detection unit 45 changes as the position of the suspicious person, the notification unit 46 changes the lighting device LD that changes the lighting mode according to the change.

The notification unit 46 may perform various notifications by controlling the lighting mode to an arbitrary mode. For example, the notification unit 46 may change the lighting mode of the lighting device LD in a manner that is difficult for a suspicious person to identify, such as changing the frequency at which the lighting device LD outputs light. Further, the notification unit 46 may notify the presence or position of a suspicious person by changing the color temperature of the light, and may notify the presence or position of a suspicious person by turning on the emergency light EL01 or the like. May be good. Further, even if the notification unit 46 notifies the clerk S01 of the probability that the user U is a suspicious person by changing the lighting mode of the lighting device LD to a lighting mode according to the accuracy acquired by the detection unit 45. Good.

For example, FIG. 8 is a diagram showing an example of a process in which the information management device according to the embodiment notifies the presence or position of a suspicious person. In the example shown in FIG. 8, an example in which lighting devices SL01 and SL02 and an emergency light EL01 are installed in a store that detects a suspicious person is described.

For example, in the example shown in FIG. 8 (A), an example in which the user U01 moves from under the lighting device SL01 to under the emergency light EL01 is described. In such a case, when the information management device 10 determines that the user U01 is a suspicious person, the lighting mode of the lighting device SL01 is set to the normal lighting mode while the user U01 is under the lighting device SL01. Is changed to a different predetermined lighting mode. Then, when the user U01 moves from under the lighting device SL01 to under the emergency light EL01, the information management device 10 changes the lighting mode of the lighting device SL01 to the normal lighting mode and also changes the lighting mode of the lighting device SL01 to the normal lighting mode and the emergency light EL01. Turn on.

Further, in the example shown in FIG. 8B, the example in which the user U01 is under the lighting device SL01 and the user U02 is under the lighting device SL02 is described. Here, in the information management device 10, the probability that the user U01 is a suspicious person is higher than the first threshold value, and the probability that the user U02 is a suspicious person is lower than the second threshold value. It is assumed that it was expensive. In such a case, the information management device 10 blinks the lighting device SL01 above the user U01 at the first frequency, and blinks the lighting device SL02 above the user U02 at the second frequency.

Here, in the example shown in FIG. 8B, it is assumed that the clerk S01 is wearing a glasses-type wearable device G01 that can visually recognize the light blinking at a high frequency in a color corresponding to the frequency. In such a case, when the clerk S01 visually recognizes the user U01, the clerk S01 visually recognizes the user U01 illuminated with the color corresponding to the first frequency as shown in FIG. 8C. Further, when the clerk S01 visually recognizes the user U02, the clerk S01 visually recognizes the user U02 illuminated with the color corresponding to the second frequency as shown in FIG. 8D. As a result, the information management device 10 provides various information such as the fact that the users U01 and U02 are suspicious persons, the positions of the suspicious users U01 and U02, and the probability that the users U01 and U02 are suspicious persons. You can report to S01.

The clerk S01 may perform any action when receiving the notification of the suspicious person. For example, the clerk S01 tries to prevent shoplifting by calling out to the suspicious person such as greetings and performing intensive monitoring of the suspicious person. Further, the information management device 10 acquires the information of the suspicious person whom the clerk S01 has spoken to via the store terminal ST. In such a case, for example, the notification unit 46 sets the corresponding flag corresponding to the suspicious person to whom the call was made to "0". As a result of such processing, the information management device 10 can realize the management of the suspicious person in the store.

(Processing flow of embodiment)
Next, the flow of each process executed by the information management device 10 will be described. FIG. 9 is a flowchart illustrating an example of a flow of management processing executed by the information management device according to the embodiment. For example, the information management device 10 acquires an image taken by the lighting device SL01 or the like (step S101). Subsequently, the information management device 10 estimates the image of the face portion and the motion information from the acquired image (step S102). Then, the information management device 10 classifies the various estimated information into personal information that can identify an individual and other non-personal information (step S103). After that, the information management device 10 manages the personal information in the internal network, uploads the non-personal information to the cloud server CS (step S104), and ends the process.

FIG. 10 is a flowchart illustrating an example of a flow of notification processing executed by the information management device according to the embodiment. For example, the information management device 10 uses the authentication data registered in the suspicious person database 34 to perform face recognition as to whether or not the user U in the store is a suspicious person (step S201). Further, the information management device 10 acquires the probability that the user U is a suspicious person by using the model (step S202).

Then, the information management device 10 determines whether or not the user U is a suspicious person or the possibility of being a suspicious person is higher than a predetermined threshold value (step S203), and the probability of being a suspicious person is higher than the predetermined threshold value. If it is determined that the value is high (step S204: Yes), the position of the suspicious person is specified (step S204), and the lighting device corresponding to the specified position is controlled in an manner according to the accuracy of the suspicious person (step S205). ), End the process. On the other hand, when the information management device 10 determines that the accuracy of the suspicious person is lower than the predetermined threshold value (step S204: No), the information management device 10 ends the process as it is.

(Variations of embodiments)
In the information management system 1 described above, the information management device 10 determines whether or not the user U in the store is a suspicious person, and if it is determined that the user U is a suspicious person, the lighting mode of the lighting device is determined. By changing it, the existence and position of the suspicious person were notified. However, the embodiment is not limited to this. Hereinafter, variations of the processing executed by the information management device 10 will be described.

(Application to logistics-related facilities)
In the above example, the information management device 10 detects a suspicious person in the store. Here, the information management device 10 may detect a user who satisfies a predetermined condition in a distribution-related facility such as a warehouse. For example, the information management device 10 detects users who handle luggage in a messy manner or users who are not improperly performing business in a logistics-related facility, and when these users are detected, a lighting device By controlling the lighting mode of the LD, a notification may be given to the administrator or the like, or a warning may be given to the user himself / herself.

Further, the information management device 10 detects a predetermined user in a predetermined facility, and changes the lighting mode of the lighting device at a position corresponding to the predetermined information to be notified to the detected user. , Predetermined information may be notified. For example, the information management device 10 detects a user who picks up the cargo and changes the lighting mode of the lighting device DL in the warehouse to detect the position where the detected user picks up the cargo. You may notify the user who has done so.

For example, FIG. 11 is a diagram showing an example of variations executed by the information management device according to the embodiment. In the example shown in FIG. 11, the user U01 wears a wearable device G01 that visualizes the blinking illumination at the first frequency, and the user U02 is a wearable device that visualizes the blinking illumination at the second frequency. I am wearing G02. Further, in the example shown in FIG. 11, it is assumed that the luggage picked up by the user U01 is arranged at the position A1 and the luggage picked up by the user U02 is arranged at the position B1. Further, in the example shown in FIG. 11, it is assumed that the lighting devices SL01 to SL04 are arranged in the warehouse.

In such a case, the information management device 10 detects the user U01 using the images taken by the lighting devices SL01 to SL04. Then, when the information management device 10 detects the user U01, that is, when it is specified that the user U01 is in the warehouse, the information management device 10 refers to a database or the like indicating the position where the luggage is arranged, and refers to the user U01. Identify the position of the luggage A1 to be picked up by. Then, the information management device 10 blinks the lighting device SL01 that illuminates the specified position at the first frequency. Further, when the information management device 10 detects the user U02, the information management device 10 specifies the position of the luggage A2 picked up by the user U02. Then, the information management device 10 blinks the lighting device SL04 that illuminates the specified position at the second frequency.

When the warehouse is illuminated in such an illumination mode, the user U01 visualizes the range AL1 illuminated by the lighting device SL01, while the other ranges are illuminated in the normal illumination mode. Seems to be taking place. As a result, the user U01 identifies that the luggage to be picked up is arranged in the range AL1 illuminated by the lighting device SL01, and at the same time, walks through the warehouse where the normal lighting is performed by the lighting devices SL02 to SL04. You can move.

Further, the user U02 visualizes the range BL1 illuminated by the lighting device SL04, while the other ranges appear to be illuminated in the normal lighting mode. As a result, the user U02 identifies that the luggage to be picked up is arranged in the range BL1 illuminated by the lighting device SL04, and at the same time, walks through the warehouse where the normal lighting is performed by the lighting devices SL01 to SL03. You can move.

The information management device 10 guides the user U01 to the position A1 by, for example, specifying the movement route from the user U01 to the position A1 and changing the lighting mode of the lighting device LD that illuminates the movement route. You may. For example, the information management device 10 may visually notify the movement path by blinking all the lighting devices LD that illuminate the specified movement path at the first frequency.

Further, the information management device 10 may detect the position of the user U01 and visually notify the movement path by sequentially changing the lighting mode of the lighting device LD on the movement path according to the detected position. Good. For example, the information management device 10 can change the lighting mode of the lighting device LD installed in the traveling direction of the user U01 among the lighting device LDs installed in the vicinity of the user U01 to change the lighting mode of the user U01. It may be guided sequentially.

(Application to any facility)
The non-personal information uploaded on the cloud server CS by the above-mentioned management process may be information used in a facility different from the facility from which the non-personal information was acquired. For example, the vendor server BS acquires operation information acquired from images taken at various facilities from the cloud server CS, and uses the acquired operation information to detect a suspicious person at each facility. May be generated.

(Lighting mode)
In the above-described processing, the lighting mode used by the information management device 10 to notify various information is only an example, and notification using any lighting mode can be adopted. For example, the information management device 10 outputs the lighting device LD in a manner recognizable by the user, such as outputting red light when the probability that the user using the store becomes a suspicious person exceeds a predetermined threshold value. You may control it.

(Other)
The information management device 10 described above is a control program in which a computer having a CPU, RAM, EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, and the like reads and reads a control program registered in a predetermined recording medium. It may be realized by executing.

Although the embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments are included in the scope of the invention described in the claims and the equivalent scope thereof, as are included in the scope and gist of the invention. Further, these embodiments can be appropriately combined as long as the processing contents do not contradict each other.

1 Information management system 10 Information management device 21 External network communication unit 22 Internal network communication unit 30 Storage unit 31 External information database 32 Internal information database 33 Lighting equipment database 34 Suspicious person database 40 Control unit 41 Acquisition unit 42 Estimating unit 43 Management unit 44 Output unit 45 Detection unit 46 Notification unit ST Store terminal CS Cloud server SC Shop center server BS Vendor server LD, SL01 to SL04 Lighting equipment LB01 Housing L01 Lighting unit C01 Imaging unit LM01 Notification lamp EL01 Emergency light

Claims (5)

With a detection unit that detects users of a predetermined facility from images taken by the imaging unit;
A notification unit that notifies the user of the position of the object by changing the lighting mode of the lighting device that illuminates the position where the object associated with the user is arranged;
A notification device including. With a detection unit that detects users of a predetermined facility from images taken by the imaging unit;
Notification that notifies the user of the movement path to the position by changing the lighting mode of the lighting device that illuminates the movement path from the user to the position where the object associated with the user is arranged. With the department;
A notification device including. The notification device according to claim 2, wherein the lighting mode of the lighting device on the movement path is sequentially changed according to the position where the detection unit detects the user. The notification unit has a database in which information indicating a position where the object is arranged is stored, and the position where the object is arranged is specified by referring to the database. 3 The notification device according to any one. With the lighting equipment installed in the designated facility;
With the notification device according to any one of claims 1 to 4.
A notification system characterized by being equipped with.

Images