JP2019179524A - Article management system - Google Patents

Abstract

To provide a system capable of appropriately grasping the location of an article in a specific space.SOLUTION: A management system 100 includes a plurality of imaging units 40, a data generation unit 63, a storage unit 61, an information search unit 66, and a reception unit 65. The imaging unit 40 is installed in an object space SP. The data generation unit 63 extracts an article OB included in imaging data D1 (an image captured by the imaging unit 40). The data generation unit 63 generates location history data D3. The location history data D3 is data for specifying the extracted article OB. The storage unit 61 stores the location history data D3 generated by the data generation unit 63. The information search unit 66 searches for information on a specific article OB in the location history data D3 stored in the storage unit 61. The reception unit 65 accepts a request regarding the article OB for which the information retrieving unit 66 performs the information search.SELECTED DRAWING: Figure 16

Description

  The present disclosure relates to an article management system.

  Conventionally, as disclosed in, for example, Patent Document 1 (Japanese Patent Laid-Open No. 2017-45373), in a target space (for example, in a company or a school), the location information of a person is grasped using images of a plurality of photographing devices. Systems that do this are known. In Patent Document 1, a person is identified by acquiring a code affixed to a hat worn by the person from an image of a photographing device.

  Here, a case where it is desired to appropriately grasp the location of an article other than a person in a specific space (for example, a case where the location of a lost item or shared equipment is desired) is assumed. In this regard, there is a concern that labor and time increase in manual search for articles. Moreover, in patent document 1, it is not grasped | ascertained about articles other than a person.

  A system capable of appropriately grasping the location of an article in a specific space is provided.

  An article management system according to a first aspect includes a plurality of photographing devices, a data generation unit, a storage unit, a search unit, and a reception unit. The photographing device is installed in the target space. A data generation part extracts the articles | goods contained in the image image | photographed with the imaging device. The data generation unit generates article data. The article data is data for specifying the extracted article. The storage unit stores the article data generated by the data generation unit. The search unit searches for information on a specific article in the article data stored in the storage unit. The reception unit receives a request regarding an article whose information is to be searched by the search unit.

  Accordingly, it is possible to generate the data (article data) related to the location of the article using the captured images of the plurality of imaging devices installed in the target space and to manage the location of the article based on the generated article data. . As a result, it is possible to easily grasp information on the article (the presence / absence of the location, the location, the location history, the related person, etc.) in the target space.

  The “image” here includes a still image and / or a moving image. The “storage unit” herein includes a main storage unit that temporarily stores article data and / or a large-capacity auxiliary storage unit that stores article data.

  The article management system according to the second aspect is the article management system according to the first aspect, and the data generation unit extracts the location of the article included in the image captured by the imaging device. The article data includes information for specifying the location of the article.

  The article management system according to the third aspect is the article management system according to the first aspect or the second aspect, and the data generation unit extracts a location time with respect to an article included in an image photographed by the photographing device. The article data includes information specifying the location time for the extracted article. Thereby, the location history of a specific article can be managed.

  An article management system according to a fourth aspect is the article management system according to any one of the first to third aspects, and the data generation unit is present in the vicinity of the article included in the image captured by the imaging device. Extract people. The article data includes information that identifies the person who has extracted the extracted article. Thereby, it is possible to manage a person (for example, an owner, a user, etc.) that is estimated to have a high degree of association with respect to each article.

  Note that the “person around” is a person existing in a predetermined range that is estimated to have a high degree of relevance with respect to the extracted article. The “predetermined range” is set according to the installation environment or the like. For example, “a person existing around” is a person existing within a radius of 1 m (predetermined range) around the article.

  An article management system according to a fifth aspect is the article management system according to any one of the first to fourth aspects, and the data generation unit includes a learning unit. The learning unit learns the characteristics of the article based on the image photographed by the photographing device. This makes it possible to extract articles with high accuracy.

  An article management system according to a sixth aspect is the article management system according to any one of the first to fifth aspects, and the storage unit stores the article data generated by the data generation unit.

  An article management system according to a seventh aspect is the article management system according to any one of the first to sixth aspects, and further includes an output unit. The output unit is a functional unit for outputting a result of information search by the search unit.

  The article management system according to the eighth aspect is the article management system according to any one of the first to seventh aspects, and any / all of the imaging devices are arranged in the air conditioner. The air conditioner is installed in the target space. By arranging the photographing device in the air conditioner, it is possible to extract articles in a wide range within the target space.

The schematic block diagram of a management system. The schematic diagram which showed an example of the installation aspect of each apparatus in a target facility. The schematic diagram which showed an example of object space. The schematic diagram which showed an example of the installation aspect of the indoor unit, remote control, and imaging | photography unit in object space. The schematic diagram which represented the structure aspect of the management apparatus roughly. FIG. 3 is a schematic diagram schematically illustrating a storage area included in a storage unit. The schematic diagram of an imaging | photography unit information table (imaging unit information). The schematic diagram of a person information table (person information). The schematic diagram of an article information table (article information). The schematic diagram of extraction data. The schematic diagram of the location history data regarding a person. The schematic diagram of the location history data regarding articles | goods. The schematic diagram which showed roughly an example of the extraction process by an extraction part. The schematic diagram of an example of display data. The schematic diagram of the other example of display data. The flowchart which showed an example of the flow of a process of a management apparatus.

  Hereinafter, a management system 100 (article management system) according to an embodiment of the present disclosure will be described. The following embodiment is a specific example, does not limit the technical scope, and can be changed as appropriate without departing from the scope of the invention.

(1) Management system 100
FIG. 1 is a schematic configuration diagram of the management system 100. FIG. 2 is a schematic diagram illustrating an example of an installation mode of each device in the target facility 1. The management system 100 individually generates data (location history data D3 to be described later) regarding the articles OB existing in a specific space (target space SP), and information management (for example, location management) of each article OB based on the generated data. , Management of owners and users, location search, etc.).

  In the present embodiment, the management system 100 is applied to the target facility 1. The target facility 1 includes a target space SP. In the present embodiment, the target facility 1 includes a plurality of target spaces SP (SP1, SP2, SP3, SP4, SP5, SP6...). The target space SP is a space where a person PS is active as shown in FIG. 3, for example, and is a space used as an office, for example. However, the present invention is not limited to this, and the target space SP may be used as, for example, a commercial facility such as a department store, a school, a factory, a hospital, or a residence. The person PS here is a person who works, learns, or lives in the target facility 1, a visitor who visits the target facility 1, or the like. In addition, the article OB here is personal items and common items of the person PS, and equipment of the target facility 1.

  The management system 100 mainly includes the air conditioning apparatus 10, a plurality of imaging units 40, and a management apparatus 60.

(1-1) Air conditioner 10
The air conditioning apparatus 10 is an apparatus that realizes air conditioning such as cooling and heating in the target space SP. The air conditioning apparatus 10 performs cooling or heating of the target space SP by performing a vapor compression refrigeration cycle in the refrigerant circuit.

  The air conditioner 10 mainly includes an outdoor unit 15 as a heat source unit, and a plurality of (6 or more in this case) indoor units 20 (20a, 20b, 20c, 20d, 20e, 20f,. ) And a plurality (here, the same number as the indoor unit 20) of remote controllers 30 (30a, 30b, 30c, 30d, 30e, 30f,...).

  In the present embodiment, the indoor unit 20a is installed in the target space SP1, the indoor unit 20b is installed in the target space SP2, the indoor unit 20c is installed in the target space SP3, and the indoor unit 20d is installed in the target space SP4. The unit 20e is installed in the target space SP5, and the indoor unit 20f is installed in the target space SP6. In the air conditioner 10, the outdoor unit 15 and each indoor unit 20 are connected by the gas communication pipe GP and the liquid communication pipe LP to form a refrigerant circuit.

(1-1-1) Outdoor unit 15
The outdoor unit 15 is installed outside the target space SP. The outdoor unit 15 mainly has a plurality of refrigerant pipes, a compressor, an outdoor heat exchanger, an expansion valve, and the like (not shown) as elements constituting the refrigerant circuit. Moreover, it has various sensors, such as a temperature sensor and a pressure sensor, and apparatuses, such as an air blower.

  The outdoor unit 15 includes an outdoor unit control unit 18 that controls operations of various actuators in the outdoor unit 15. The outdoor unit controller 18 includes a microcomputer including a memory such as a RAM and a ROM, a CPU, a communication module, various electronic components, and an electrical component. The outdoor unit controller 18 is electrically connected to various actuators and various sensors via wiring.

  The outdoor unit control unit 18 is connected to an indoor unit control unit 25 (described later) of the indoor unit 20 via a communication line cb1 and transmits / receives signals to / from each other. The outdoor unit controller 18 is connected to a wide area network NW1 including a WAN (Wide Area Network) such as the Internet via a communication line cb2, and is mutually connected to a device (for example, the server 50) connected to the wide area network NW1. Send and receive signals. The outdoor unit controller 18 constitutes an indoor unit controller 25, a remote controller 35, a server 50, and a management device 60, which will be described later.

(1-1-2) Indoor unit 20
The indoor unit 20 is a ceiling-embedded air conditioning indoor unit installed in the ceiling CI of the target space SP or a ceiling suspended type installed in the vicinity of the ceiling CI. FIG. 4 is a schematic diagram illustrating an example of an installation mode of the indoor unit 20, the remote controller 30, and the photographing unit 40 in the target space SP. In FIG. 4, in the target space SP, each indoor unit 20 is installed such that a part of the main body portion (intake port and exhaust port) is exposed from the ceiling CI. The indoor unit 20 has an indoor heat exchanger, an indoor expansion valve, and the like as elements constituting the refrigerant circuit. The indoor unit 20 includes various sensors such as a temperature sensor and a pressure sensor that detect the temperature in the target space SP and the temperature of the refrigerant.

  The indoor unit 20 includes an indoor unit control unit 25 that controls operations of various actuators (such as a blower and an expansion valve) in the indoor unit 20. The indoor unit control unit 25 includes a microcomputer including a memory such as a RAM and a ROM, a CPU, a communication module, various electronic components, and an electrical component. The indoor unit control unit 25 is electrically connected to various actuators and various sensors via wiring, and transmits and receives signals to and from each other. The indoor unit control unit 25 is connected to the outdoor unit control unit 18 or another indoor unit control unit 25 via the communication line cb1, and transmits and receives signals to and from each other. Moreover, the indoor unit control part 25 is connected to the remote control part 35 (after-mentioned) of the corresponding remote control 30 via the communication line cb3, and transmits / receives a signal mutually. Moreover, the indoor unit control part 25 is connected with the corresponding imaging | photography unit 40 via the communication line cb4, and transmits / receives a signal mutually.

  The indoor unit control unit 25 constitutes a management device 60 together with the outdoor unit control unit 18, the other indoor unit control unit 25, the remote control control unit 35, and the server 50.

(1-1-3) Remote control 30
The remote controller 30 is associated with one of the indoor units 20 and is installed on the side wall SW of the same target space SP as the corresponding indoor unit 20. Specifically, the remote controller 30a is associated with the indoor unit 20a. The remote controller 30b is associated with the indoor unit 20b, the remote controller 30c is associated with the indoor unit 20c, the remote controller 30d is associated with the indoor unit 20d, the remote controller 30e is associated with the indoor unit 20e, and the remote controller 30f is associated with the indoor unit 20c. It is associated with the unit 20f. Each remote controller 30 is a so-called wired remote control device, and is connected to a corresponding indoor unit 20 (indoor unit control unit 25) via a communication line cb3.

  Each remote controller 30 functions as an input device for inputting commands related to various settings to the air conditioner 10. The remote controller 30 also functions as a display device that displays the operating state and setting items of the air conditioner 10.

  Each remote controller 30 has a remote control unit 35 that controls the operation of the remote controller 30. The remote control control unit 35 constitutes a management device 60 together with the outdoor unit control unit 18, the indoor unit control unit 25, and the server 50.

(1-2) Imaging unit 40 (imaging device)
The management system 100 includes a plurality (here, six or more) of photographing units 40 (40a, 40b, 40c, 40d, 40e, 40f...). The photographing unit 40 is a unit that photographs the corresponding target space SP, generates and outputs data including the photographing result (photographing data D1). The photographing unit 40 is disposed in the corresponding target space SP. In the present embodiment, the photographing unit 40 is disposed in the indoor unit 20 installed on the ceiling CI (or near the ceiling) of the corresponding target space SP. That is, the photographing unit 40 is installed on the ceiling CI or near the ceiling (portion closer to the ceiling CI than the floor surface).

  In the present embodiment, the photographing unit 40a is disposed in the target space SP1 (indoor unit 20a), the photographing unit 40b is disposed in the target space SP2 (indoor unit 20b), and the photographing unit 40c is disposed in the target space SP3 (indoor unit 20c). The photographing unit 40d is disposed in the target space SP4 (indoor unit 20d), the photographing unit 40e is disposed in the target space SP5 (indoor unit 20e), and the photographing unit 40f is disposed in the target space SP6 (indoor unit 20f). ing.

  The photographing unit 40 includes a photographing unit 41, a photographing data generation unit 42, and a photographing data output unit 43. The imaging unit 41 includes a lens for imaging a predetermined range of the corresponding target space SP (for example, a fisheye lens or a fixed focus lens, but not limited to) and an imaging element. The shooting data generation unit 42 performs A / D conversion on the electrical signal output by the imaging device of the shooting unit 41, and generates shooting data D1 along a predetermined format. The shooting data D1 includes image data (moving image data) in which a predetermined range of the target space SP is represented by predetermined pixels. The imaging data output unit 43 compresses the generated imaging data D1 and outputs the compressed imaging data D1 to the management device 60 (directly the indoor unit control unit 25).

(1-3) Management device 60
The management device 60 is a computer that comprehensively manages the operation of the management system 100. In the present embodiment, the management device 60 is configured by connecting the outdoor unit control unit 18, the indoor unit control unit 25, the remote control unit 35, and the server 50 via a communication network, as shown in FIG. The

  The server 50 is a computer that constitutes the management device 60 together with the outdoor unit control unit 18, the indoor unit control unit 25, and the remote control unit 35 in the management system 100. The server 50 is arranged in a remote place away from the target space SP. The server 50 is connected to the wide area network NW1 via a communication line, and is configured to be able to communicate with the outdoor unit control unit 18, the indoor unit control unit 25, and the remote control unit 35 via the wide area network NW1.

  The management device 60 transmits / receives data to / from the photographing unit 40 and the terminal 90. The management device 60 performs processing based on the shooting data D1. More specifically, the management device 60 individually extracts the person PS and the article OB included in the shooting data D1, and generates and stores data (location history data D3) corresponding to the extracted person PS and the article OB. To do. The location history data D3 is data that specifies the location history of the extracted person PS and article OB in the target facility 1. Details of the location history data D3 will be described later.

  In addition, the management device 60 performs processing according to a command input by the user and outputs a processing result. For example, when the management device 60 receives a command (output request command) requesting to output the location history of a specific person PS or article OB, the management apparatus 60 relates to the person PS or article OB specified in the command. Based on the location history data D3, data for specifying location information (display data D4 described later) is generated and output.

(1-4) Terminal 90
The terminal 90 is an information terminal held by an administrator or a user. The terminal 90 is assumed to be a mobile terminal such as a smartphone or a tablet PC, or a personal computer such as a laptop personal computer, but may be another information processing device.

  The terminal 90 has a communication module, and is configured to be able to communicate with other units (the outdoor unit control unit 18, the indoor unit control unit 25, the remote control unit 35, and the server 50) via the wide area network NW1. Yes. The terminal 90 is connected to the wide area network NW1 by one or both of wireless communication and wired communication.

  The terminal 90 has an input unit for inputting a command. In the management system 100, the terminal 90 can function as a “command input unit” for inputting a command. The user can appropriately control the operations of the photographing unit 40 and the management device 60 by inputting commands using the terminal 90.

  The terminal 90 has a display unit that displays (outputs) information. In the management system 100, the terminal 90 can function as an “output unit” that outputs information. The user can grasp the operation status and processing result of the management system 100 through the terminal 90.

  The terminal 90 can input a command to the management apparatus 60 and display a response result of the management apparatus 60 by installing a predetermined application program.

(2) Details of Management Device 60 As shown in FIG. 1, the management device 60 mainly includes a storage unit 61, an acquisition unit 62, a data generation unit 63, a location management unit 64, a reception unit 65, an information search unit 66, and Functional units such as the output control unit 67 are included. Each functional unit is realized by operation of any or all of the constituent devices of the management device 60 (here, the outdoor unit control unit 18, each indoor unit control unit 25, each remote control unit 35, and the server 50). . The management device 60 is configured to be able to acquire time in real time independently or from another device.

(2-1) Storage unit 61
The storage unit 61 is configured by a memory such as a ROM, a RAM, a flash memory, and a hard disk included in any or all of the constituent devices of the management device 60. The storage unit 61 includes a plurality of storage areas, and has a volatile storage area for temporarily storing information and a nonvolatile storage area for storing various types of information. Specifically, as shown in FIG. 6, the storage unit 61 includes a program information storage area M1, an environment information storage area M2, a system information storage area M3, a person information storage area M4, an article information storage area M5, and shooting data. Storage areas such as a storage area M6, an extracted data storage area M7, a location history data storage area M8, an input information storage area M9, an output information storage area M10, and a feature data storage area M11 are included. Various information stored in each storage area can be updated as appropriate.

  The program information storage area M1 stores a control program that defines various processes executed by each unit of the management device 60, a communication protocol used for communication between the units, and the like.

  Information (environment information) related to the target facility 1 is stored in the environment information storage area M2. The environment information includes, for example, information that individually specifies the number, position, size, and the like of each target space SP included in the target facility 1.

  Information related to each device included in the management system 100 is stored in the system information storage area M3. For example, in the system information storage area M3, information (shooting unit information) related to the shooting unit 40 arranged in the target facility 1 is stored. The imaging unit information includes information for specifying the ID, communication address, arrangement position, arrangement mode, and the like of the imaging unit 40 arranged in the target facility 1. The photographing unit information is stored as a photographing unit information table TB1 as shown in FIG. 7, for example. In the photographing unit information table TB1 of FIG. 7, the communication address of the photographing unit 40 with the ID “0120” is “172.16. **. 01”, the arrangement position is “(target space) SP1,” and the arrangement form is as follows. It is specified that it is “built in the indoor unit 20a”.

  In the person information storage area M4, information (person information) related to the person PS registered in advance in the target facility 1 and the person PS detected in the past is stored. The person information includes information for specifying the ID (arbitrary or automatically assigned ID), name, affiliation group, and the like of each person PS. The person information is stored as a person information table TB2 as shown in FIG. 8, for example. In the person information table TB2 of FIG. 8, for example, for the person PS whose ID is “01139”, the name is “Tanaka XX”, the group to which the affiliation belongs is “G3”, and the latest location space is “(target space) SP2. It is specified that the latest location time is “2018/03/05/17: 55”.

  The article information storage area M5 stores articles OB registered in advance in the target facility 1 and information (article information) related to articles OB detected in the past. In the item information, each item OB ID (arbitrarily or automatically assigned ID), name (arbitrary or automatically assigned name), item (type), attribute, latest location information (location space, The location information, the location date and time), and information related to the latest (most recent) related person are included. The “related person” related to the article OB is a person who is estimated to have a high degree of relation regarding the extracted article OB. In the present embodiment, the related person of the article OB is a person existing around the article OB, and a predetermined range in which the degree of association is estimated to be large with respect to the extracted article OB (for example, within a radius of 1 m around the article OB). Someone who exists in The article information is stored as an article information table TB3 as shown in FIG. 9, for example. In the article information table TB3 of FIG. 9, for example, for the article OB with ID “35869912”, the name is “second notebook PC”, the item is “laptop PC”, the attribute is “equipment”, and the latest The (most recent) location space is “(target space) SP2,” the latest location location is “(192,113,65)”, and the latest location time is “2018/03/05/17: 55” It is specified that the latest related person ID is “01139”.

  In the shooting data storage area M6, shooting data D1 output from each shooting unit 40 is stored. In the shooting data storage area M6, shooting data D1 is stored for each shooting unit 40.

  The extracted data storage area M7 stores data (extracted data D2) for specifying the person PS and the article OB extracted from the shooting data D1 output from the shooting unit 40. The extracted data D2 is generated for each photographing unit 40 that is the transmission source of the photographing data D1. More specifically, the extraction data D2 is generated for each received shooting data D1. The extracted data D2 includes information for specifying the location of the extracted person PS and article OB, and information for specifying the location time of the extracted person PS and article OB. For example, as shown in FIG. 10, the extracted data D2 includes the ID of the transmission source shooting unit 40 and its shooting location, the reception date and time of the shooting data D1, the ID of the person PS and the article OB extracted from the received shooting data D1, This is information specifying the position (coordinates), the ID of a person PS (related person) related to the extracted article OB, and the like. In FIG. 10, for example, the location where the image is transmitted from the imaging unit 40 with the ID “0121” is “(target space) SP2”, and the date and time when the image was captured is “2018/03/06/17: 55”. Regarding the shooting data D1, it is specified that the ID of the extracted person PS is “01139”, “01564” and the respective positions are “(120,100,60)”, “(238,201,58)”, etc. Yes. Further, the ID of the extracted article OB is “3586912”, “5922038”, “57666598”, and the respective positions are “(192,113,65)”, “(250,240,61)”, “(125,110,69)”. ", And the ID of each related person is specified as" 01139 "," 01564 "," 01139 ", and the like.

  In the location history data storage area M8, data (location history data D3) for specifying the location history regarding the person PS or the article OB in the target facility 1 is individually stored. The location history data D3 (corresponding to “article data” described in the claims) is information for specifying the location history of the person PS or the article OB in the target facility 1. The location history data D3 is generated for each person PS or article OB. The location history data D3 is data that individually specifies the person PS or the article OB extracted from the photographing data D1, and is information that specifies the location of the extracted person PS and the article OB in time series, and is extracted. It includes information for specifying the location time in time series with respect to the person PS and the article OB.

  The location history data D3 related to the person PS includes, for example, as shown in FIG. 11, the ID of the person PS, the ID of the imaged photographing unit 40, the location space (the location where the image was taken), the location date and time (the date and time when the image was taken), and the like. It is information to include. In the location history data D3 shown in FIG. 11, for example, the location history every 5 minutes for the person PS whose ID is “01139” is specified.

  The location history data D3 related to the article OB includes, as shown in FIG. 12, for example, the ID of the article OB, the ID of the photographing unit 40 that has been photographed, the location space (the place where the photograph was taken), the location (coordinates), Date and time) and related person IDs. In the location history data D3 shown in FIG. 12, for example, a location history every 5 minutes for the article OB whose ID is “35869912” is specified. The location history specified in the location history data D3 does not necessarily have to be every 5 minutes, and the date and time related to the location history can be changed as appropriate.

  Information input to the management device 60 is stored in the input information storage area M9. For example, the command input via the terminal 90 is stored in the input information storage area M9.

  In the output information storage area M10, information output from the management device 60 to other devices is stored. For example, display data D4 (FIGS. 14 and 15) to be displayed on the terminal 90 is stored in the output information storage area M10.

  The feature data storage area M11 stores the feature data D5 in which the general features of the person PS and each article OB and the unique features of each person PS and each article OB extracted in the target space SP are individually specified. ing. The feature data D5 is created for each person PS or article OB. The “feature” here is information used to uniquely identify the person PS or the article OB. “Characteristics” of the person PS vary, and for example, the shape, size, color, or motion (motion speed, motion range, motion angle) of the part of the character PS (eg, head, toothpick, face, shoulder, arm, leg, etc.) ) Etc. Further, the “feature” of the article OB varies, for example, the shape, size, color, or operation of the article OB.

(2-2) Acquisition unit 62
The acquisition unit 62 is a functional unit that acquires the shooting data D1 output from each of the shooting units 40 and appropriately stores it in the shooting data storage area M6.

(2-3) Data generation unit 63
The data generation unit 63 is a functional unit that extracts the person PS and the article OB included in the shooting data D1 stored in the shooting data storage area M6 and generates extracted data D2. The data generation unit 63 includes an extraction unit 631 and an extraction data generation unit 633.

  The extraction unit 631 acquires the shooting data D1 stored in the shooting data storage area M6, and performs a process (extraction process) for individually extracting the person PS and the article OB included in the acquired shooting data D1. . The extraction unit 631 performs an extraction process every hour. The extraction process is performed for each photographing data D1 (that is, for each photographing unit 40 that is the transmission source of the photographing data D1).

  The extraction unit 631 includes a learning unit 632 and is configured to perform machine learning. The learning unit 632 here performs machine learning using means such as “neural network” and “deep learning”. Such learning may be so-called supervised learning or unsupervised learning.

  The extraction process by the extraction unit 631 is performed using predetermined means (including known techniques). For example, the extraction unit 631 extracts (detects and specifies) the person PS or the article OB based on the characteristic data D5 in which the characteristics of the person PS or the article OB are defined in advance. For example, the extraction unit 631 detects the person PS or the article OB by identifying the feature of the person PS or the article OB in the shooting data D1, and compares the identified feature with the feature defined in the feature data D5. Thus, the person PS or the article OB is uniquely specified.

  An example of the extraction process by the extraction unit 631 is shown in FIG. FIG. 13 shows an example in which the extraction unit 631 extracts a person PS or an article OB in the target space SP using a plurality of neural networks (N1, N2, N3, N4).

  In FIG. 13, first, the photographing data D1 is input to the first neural network N1. The first neural network N1 executes a process P1 for detecting (estimating) the distance between each element included in the captured data D1.

  Further, the result of the process P1 is input to the second neural network N2 together with the photographing data D1. The second neural network N2 executes a process P2 for detecting (estimating) the range of the person PS or the article OB included in the shooting data D1 based on the result of the process P1. When the range of the person PS or the article OB can be detected, the movement of the person PS or the article OB can be detected, and the characteristics of the person PS or the article OB can be acquired in the process P3 described later.

  Furthermore, the result of the process P2 is input to the third neural network N3 together with the result of the process P1. The third neural network N3 executes a process P3 for detecting and specifying the characteristics of the person PS and the article OB included in the shooting data D1 based on the results of the processes P1 and P2. In the process P3, the person PS or the article OB is uniquely specified based on the detected feature of the person PS or the article OB and the feature data D5 stored in the feature data storage area M11. For example, in the process P3, the similarity between each feature of the detected person PS or article OB and each feature data D5 stored in the feature data storage area M11 is calculated, and the calculated similarity is a predetermined value. The target person PS or target article OB in the feature data D5 that is equal to or greater than the threshold is extracted as a person PS or article OB that matches the detected feature, and the person PS or article OB is uniquely identified. In addition, when the feature data D5 having the feature of the detected person PS or the article OB and the feature data D5 having a similarity greater than or equal to a predetermined threshold value are not stored in the feature data storage area M11, the person PS or the article OB having the feature. Is newly generated and stored as a newly detected person PS or article OB. The feature data D5 generated as a result of the process P3 is, for example, 100-dimensional vector data.

  Further, the result of the process P2 is input to the fourth neural network N4 together with the result of the process P1. The fourth neural network N4 executes a process P4 for detecting positions (coordinates) in the target space SP of the person PS and the article OB included in the shooting data D1 based on the results of the processes P1 and P2.

  When the extraction process is performed in such a manner, the data generation unit 63 estimates the distance between each element from the shooting data D1 in the extraction process, and extracts the person PS or the article OB based on the estimated distance. I can say that.

  The learning unit 632 appropriately learns the characteristics of each person PS and the article OB using various types of information (for example, information that can be acquired via the shooting data D1 and the wide area network NW1). For example, the learning unit 632 specifically learns the characteristics of the person PS and the article OB included in the shooting data D1, and appropriately updates the corresponding characteristic data D5. As a result, fluctuations in the detection result due to changes in the characteristics of the person PS or the article OB (for example, changes in clothes and hairstyle, deterioration of the color of the object, etc.) are suppressed.

  The extraction data generation unit 633 generates extraction data D2 (FIG. 10) based on the result of the extraction process by the extraction unit 631. In the extraction data D2, the extraction data generation unit 633 extracts the ID of the extracted person PS or article OB, the detected location (location), the detected time (location time), and the related person of the extracted item OB. Incorporate information identifying PS. The extracted data generation unit 633 generates extracted data D2 for each imaging unit 40 that is the transmission source of the imaging data D1.

  The data generation unit 63 including the extraction unit 631 and the extraction data generation unit 633 as described above indicates the location and time of the person PS or the article OB included in the shooting data D1 (the image shot by the shooting unit 40). It can be said that it is a functional part to be extracted.

(2-4) Location management unit 64
The location management unit 64 is a functional unit that manages information on the person PS or the article OB existing in the target space SP based on the extracted data D2. The location management unit 64 performs processing periodically (for example, at a cycle of 5 minutes). For example, the location management unit 64 generates location history data D3 (FIGS. 11 and 12) for each extracted person PS or article OB based on each extracted data D2 stored in the extracted data storage area M7. Stored in the location history data storage area M8. For example, the location management unit 64 creates and updates the person information table TB2 and stores it in the person information storage area M4. For example, the location management unit 64 creates and updates the article information table TB3 and stores it in the article information storage area M5.

(2-5) Reception unit 65
The receiving unit 65 acquires information input to the management apparatus and stores it in the input information storage area M9. For example, the reception unit 65 acquires a command input via the terminal 90 and stores it in the input information storage area M9. The command input to the management device 60 is, for example, an output for instructing output of information related to the location of a specific person PS or article OB (information specifying the location, location, or location history during a predetermined period). Request command. In the output request command for requesting information regarding the location of the person PS, the target person PS is designated using information such as the ID, name, or group of the person PS. In the output request command for requesting information regarding the location of the article OB, the target article OB is designated using information such as the ID, name, item, and ID and name of the related person PS.

(2-6) Information search unit 66 (search unit)
The information search unit 66 executes processing based on the output request command (search request) stored in the input information storage area M9. The information search unit 66 searches for the person PS or article OB specified in the output request command by referring to each data (location history data D3, person information or article information, etc.) stored in the storage unit 61, Display data D4 indicating information on the location of the person PS or the article OB (for example, the latest location space, location, or location history) is generated as a search result.

  The display data D4 for the output request command related to the person PS is generated, for example, in a manner as shown in FIG. In the display data D4 shown in FIG. 14, the ID of the person PS specified in the output request command is “01139”, the name is “Tanaka XX”, the belonging group “G3”, and “2018/03 / The location history in the period (designated period) from “05/18: 00” to “2018/03/06/18: 00” is specified. The information search unit 66 stores the generated display data D4 in the output information storage area M10.

  The display data D4 for the output request command related to the article OB is generated in a manner as shown in FIG. 15, for example. In the display data D4 shown in FIG. 15, the ID of the article OB specified in the output request command is “35869912”, the name is “second notebook PC”, the item is “laptop PC”, and the attribute is “equipment”. , Where location information (location space, location, date and time) from “2018/03/05/18: 00” to “2018/03/06/18: 00” (designated period), and The ID of the related person is specified.

(2-7) Output control unit 67
The output control unit 67 transmits (outputs) the data stored in the output information storage area M10 to the destination device. For example, the output control unit 67 transmits the display data D4 stored in the output information storage area M10 to the terminal 90 that is the transmission source of the output request command. Thereby, the display data D4 (search result) generated by the information search unit 66 is transmitted to the terminal 90 that is the transmission source of the output request command and displayed on the terminal 90.

(3) Process Flow by Management Device 60 An example of the process flow of the management device 60 will be described below with reference to FIG. FIG. 16 is a flowchart illustrating an example of the processing flow of the management apparatus 60.

  The management device 60 executes the process as shown in steps S101 to S106 in FIG. Note that the processing flow shown in FIG. 16 can be changed as appropriate, and as long as the processing is properly performed, the order of any of the steps may be changed, or any of the steps may be executed simultaneously. However, other steps (not shown) may be added.

  In step S101, the management device 60 proceeds to step S105 when the shooting data D1 is not received (that is, when the new shooting data D1 is not stored in the storage unit 61) (that is, when NO). On the other hand, if the management device 60 has received the shooting data D1 (that is, if new shooting data D1 is stored in the storage unit 61) (that is, YES), the management device 60 proceeds to step S102.

  In step S102, the management apparatus 60 acquires the shooting data D1 from the storage unit 61, executes an extraction process for extracting the person PS and the article OB included in the acquired shooting data D1, and generates extraction data D2. Thereafter, the process proceeds to step S103.

  In step S103, when the person PS or the article OB cannot be extracted as a result of the extraction process (when the person PS or the article OB is not detected in the shooting data D1), that is, in the case of NO, the management apparatus 60 proceeds to step S105. move on. On the other hand, when the person PS or the article OB can be extracted as a result of the extraction process (when the person PS or the article OB is detected in the shooting data D1), that is, in the case of YES, the management device 60 proceeds to step S104. move on.

  In step S104, the management device 60 executes a predetermined process for the person PS or the article OB extracted in the extraction process. For example, the management device 60 generates or updates the location history data D3 regarding the person PS or the article OB extracted in the extraction process. For example, the management device 60 generates or updates the person information table TB2 regarding the person PS extracted in the extraction process. For example, the management device 60 generates or updates the article information table TB3 regarding the article OB extracted in the extraction process. In addition, for example, the management device 60 learns features regarding the person PS or the article OB extracted in the extraction process. For example, the management device 60 generates or updates the feature data D5 regarding the person PS or the article OB extracted in the extraction process. The management apparatus 60 proceeds to step S105 after appropriately performing the process.

  In step S105, when the management device 60 has not received the output request command (that is, in the case of NO), the management device 60 returns to step S101. On the other hand, when the output request command is received (that is, in the case of YES), the management device 60 proceeds to step S106.

  In step S <b> 106, the management device 60 generates display data D <b> 4 that specifies the location history of the period specified for the person PS or the article OB specified in the output request command. The management device 60 outputs the generated display data D4 to the terminal 90 that is the transmission source of the output request command. Thereby, the generated display data D4 is displayed on the terminal 90 that is the transmission source of the output request command.

(4) Features (4-1)
The management system 100 according to the embodiment extracts a plurality of imaging units 40 installed in the target space SP and an article OB included in the imaging data D1 (images captured by the imaging unit 40) to extract data D2 (extraction). The data generation unit 63 for generating the data OB), the storage unit 61 for storing the extraction data D2 generated by the data generation unit 63, and the extraction data D2 stored in the storage unit 61. An information search unit 66 that performs information search regarding the article OB, and a reception unit 65 that receives a request regarding the article OB to be searched for information by the information search unit 66 are provided.

  Thereby, the data (extraction data D2) relating to the location of the article OB is generated using the shooting data D1 (captured images) of the plurality of shooting units 40 installed in the target space SP, and the article is based on the generated extraction data D2. It is possible to manage OB. As a result, it is possible to easily grasp the information on the article OB (the presence / absence, location, location history, related person, etc.) in the target space SP.

(4-2)
In the above embodiment, the data generation unit 63 extracts the location of the article OB included in the shooting data D1, and the extracted data D2 includes information for specifying the location of the article OB. This makes it easy to manage the location history of each article OB.

(4-3)
In the above embodiment, the data generation unit 63 extracts the location time for the article OB included in the shooting data D1, and the extraction data D2 includes information for specifying the location time for the extracted article OB. Thereby, the location history of each article OB can be managed.

(4-4)
In the above-described embodiment, the data generation unit 63 extracts the related person PS (a person existing in the surroundings) with respect to the article OB included in the shooting data D1 (the image shot by the shooting unit 40), and the extracted data D2 Includes information specifying the extracted related person PS regarding the extracted article OB.

  Thereby, it is possible to manage the related person PS (a person who is estimated to have a high degree of association, such as an owner or a user) for each article OB.

(4-5)
In the embodiment described above, the data generation unit 63 includes a learning unit 632 that learns the characteristics of the article OB based on the shooting data D1 (an image shot by the shooting unit 40). Thereby, it is possible to extract the article OB with high accuracy.

(4-6)
In the above embodiment, the storage unit 61 stores the extracted data D2 generated by the data generation unit 63. Thereby, it is possible to manage the location history of the article OB for a long time.

(4-7)
The management system 100 according to the embodiment includes a terminal 90 that functions as an output unit for outputting the information search result by the information search unit 66. Thereby, the user can easily grasp the processing result of the management device 60.

(4-8)
In the management system 100 according to the above embodiment, the imaging unit 40 is disposed in the indoor unit 20 (air conditioner) installed in the target space SP. By arranging the photographing unit 40 in the indoor unit 20 installed on or near the ceiling, it is possible to extract articles OB in a wide range within the target space SP.

(5) Modifications The above embodiment can be appropriately modified as shown in the following modifications. Each modification may be applied in combination with another modification as long as no contradiction occurs.

(5-1) Modification 1
In the above embodiment, the photographing unit 40 is disposed in the ceiling embedded type indoor unit 20 embedded in the ceiling CI of the target space SP. However, the arrangement of the photographing units 40 is not necessarily limited to this, and can be changed as appropriate. For example, any / all of the imaging units 40 may be arranged in a ceiling-suspended indoor unit 20 that is suspended from the ceiling of the target space SP, or a wall-hanging type that is installed on the side wall SW of the target space SP. The indoor unit 20 may be disposed. Further, for example, any / all of the photographing units 40 are not necessarily arranged in the indoor unit 20, and may be arranged in other devices or may be installed independently.

(5-2) Modification 2
In the above-described embodiment, the example in which the extraction process is performed in the manner illustrated in FIG. 13 has been described. However, it goes without saying that the extraction process may be performed according to other modes. For example, the extraction process may be executed using means other than the neural network. For example, even if the person PS and the article OB are extracted and specified by extracting the characteristics from the shooting data D1 based on the data defining the characteristics of the person PS and the article OB registered in advance by the administrator or the like. Good. Further, the characteristics of the person PS or the article OB used in the extraction process can be changed as appropriate.

  Further, the extraction process is not necessarily performed every hour, and may be performed at a predetermined timing. For example, the extraction process may be performed periodically (for example, at a cycle of 5 minutes).

  In the extraction process, the person PS does not necessarily have to be extracted, and only the article OB may be extracted.

(5-3) Modification 3
In the above embodiment, the shooting data D1 includes image data (moving image data) in which a predetermined range of the target space SP is represented by predetermined pixels. However, the format of the photographing data D1 can be changed as appropriate according to the installation environment, design specifications, and the like. For example, the shooting data D1 may be image data (still image) representing a predetermined range of the target space SP with predetermined pixels.

(5-4) Modification 4
In the above embodiment, one photographing unit 40 is arranged in one target space SP. However, the arrangement mode of the photographing unit 40 is not necessarily limited to this, and can be changed as appropriate. For example, a plurality of photographing units 40 may be arranged in one target space SP. In such a case, the characteristics of the person PS are identified based on the respective photographing data D1 photographed by the plurality of photographing units 40. In other words, the extraction process is performed based on the shooting data D1 shot at different shooting angles in one target space SP, so that the extraction of the person PS and the identification of the person can be performed with high accuracy.

(5-5) Modification 5
In the above embodiment, the photographing unit information is generated as the photographing unit information table TB1 as shown in FIG. The shooting unit information does not necessarily have to be generated in this manner, and the shooting unit information generation mode can be changed as appropriate.

(5-6) Modification 6
In the above embodiment, the person information is generated as a person information table TB2 as shown in FIG. 8, for example. The person information does not necessarily have to be generated in such a mode, and the generation mode of the person information can be changed as appropriate. That is, the person information table TB2 includes other information (for example, information such as the dimensions of the person PS, the state of the person PS, and the relationship between the persons PS) instead of any of the information included in FIG. Also good.

(5-7) Modification 7
In the above embodiment, the article information is generated as an article information table TB3 as shown in FIG. 9, for example. The article information does not necessarily have to be generated in such a mode, and the generation mode of the article information can be changed as appropriate. That is, the item information table TB3 is replaced with any of the information included in FIG. 9 (for example, the size of the item OB, the state of the item OB, the relationship between the items OB, and the relationship between the item OB and the person PS). Information such as sex).

(5-8) Modification 8
In the above embodiment, the extraction data D2 is generated in a manner as shown in FIG. However, the extraction data D2 does not necessarily have to be generated in such a manner, and the generation mode of the extraction data D2 can be changed as appropriate. For example, the extracted data D2 may include other information instead of any of the information included in FIG.

(5-9) Modification 9
In the above embodiment, the location history data D3 is generated in a manner as shown in FIGS. However, the location history data D3 does not necessarily have to be generated in such a manner, and the generation mode of the location history data D3 can be changed as appropriate. For example, the location history data D3 may include other information instead of any of the information included in FIGS.

(5-10) Modification 10
In the above embodiment, the display data D4 is generated in a manner as shown in FIG. However, the display data D4 does not necessarily have to be generated in such a manner, and the generation mode of the display data D4 can be changed as appropriate. For example, the display data D4 may include other information instead of any of the information included in FIG. Further, for example, when the output request command requests the location of the article OB, the display data D4 may simply specify only the latest location space and location of the article OB. . For example, when the output request command is a request for the location (location) of the article OB, the state of the target space SP where the designated article OB exists is displayed on the terminal 90. The photographing data D1 of the photographing unit 40 that photographs the target space SP may be output as the display data D4.

(5-11) Modification 11
In the embodiment described above, the management device 60 extracts a person PS or an article OB from the photographic data D1, and then executes a predetermined process for the extracted person PS or article OB (step S104 in FIG. 16). Here, any of the processes in step S104 in FIG. 16 may be omitted as appropriate. For example, the management device 60 does not necessarily have to learn features regarding the person PS or the article OB extracted in the extraction process. For example, the management device 60 does not necessarily need to generate or update the feature data D5 regarding the person PS or the article OB extracted in the extraction process.

(5-12) Modification 12
In the above embodiment, the management device 60 generates the location history data D3 based on the extracted data D2 generated based on the shooting data D1, and the processing result for the output request command (search request) based on the location history data D3. (Display data D4) was generated. However, the management device 60 may generate a processing result (display data D4) for the output request command (search request) based on the extracted data D2 instead of the location history data D3. In such a case, the extracted data D2 corresponds to “article data” recited in the claims.

(5-13) Modification 13
In the above embodiment, the “related person” related to the article OB (a person estimated to have a high degree of relation regarding the extracted article OB) is a person existing around the article OB, and the degree of relevance regarding the extracted article OB. A case has been described in which the person is present in a predetermined range in which is estimated to be large. However, the setting mode of the “related person” is not necessarily limited to this, and can be appropriately changed according to the installation environment or the like.

(5-14) Modification 14
In the embodiment described above, the data generation unit 63 includes the learning unit 632 that learns the characteristics of the person PS and the article OB based on the shooting data D1. However, the data generation unit 63 does not necessarily have the learning unit 632.

(5-15) Modification 15
In the above embodiment, the terminal 90 functions as an “input unit” for inputting an output request command to the management device 60. However, the present invention is not necessarily limited thereto, and a device other than the terminal 90 may function as the “input unit”. For example, the remote controller 30, the server 50, or another device may function as the “input unit”.

  In the above embodiment, the terminal 90 functions as an “output unit” that outputs a processing result (display data D4) for the output request command. However, the present invention is not necessarily limited to this, and a device other than the terminal 90 may function as the “output unit”. For example, the remote controller 30, the server 50, or another device may function as the “output unit”.

  The information output mode in the “output unit” is not necessarily limited to data display. For example, when the “output unit” has a speaker capable of outputting sound, the processing result for the output request command may be output by outputting sound data.

(5-16) Modification 16
In the above embodiment, the management system 100 is applied in the target facility 1 including a plurality (six) target spaces SP. However, the number of target spaces SP in the target facility 1 to which the management system 100 is applied can be changed as appropriate. For example, the management system 100 may be applied to a target facility including a target space SP of 7 or more or 5 or less. For example, the management system 100 may be applied to a target facility that includes a single target space SP.

(5-17) Modification 17
In the above embodiment, between each unit (for example, between the outdoor unit controller 18 and the indoor unit controller 25, between the indoor unit controller 25 and the indoor unit controller 25, between the indoor unit controller 25 and the remote controller 35, and In the indoor unit control unit 25 and the photographing unit 40), a communication network is configured using a communication line. However, it is a matter of course that a communication network may be configured by wireless communication using radio waves or infrared rays in addition to the communication line / in place of the communication line between the units. The outdoor unit control unit 18 or the server 50 may be connected to the wide area network NW1 by wireless communication in addition to / in place of the communication line.

(5-18) Modification 18
In the above embodiment, the server 50 is configured to be able to communicate with the outdoor unit controller 18, the indoor unit controller 25, and the remote controller 35 via the wide area network NW1, but a LAN (local area network) is used. It may be configured to be able to communicate with these units.

(5-19) Modification 19
In the said embodiment, the management apparatus 60 is comprised by connecting the outdoor unit control part 18, the indoor unit control part 25, the remote control control part 35, and the server 50 via a communication network. However, the configuration mode of the management device 60 is not necessarily limited to this, and may be configured in other modes. For example, any of the outdoor unit control unit 18, the indoor unit control unit 25, the remote control unit 35, and the server 50 may be omitted as a component device of the management device 60. Further, for example, instead of / in addition to any of the outdoor unit control unit 18, the indoor unit control unit 25, the remote control unit 35, and the server 50, another device is connected via a communication network, so that the management device 60 is connected. May be configured. Further, the management device 60 does not necessarily need to be configured across the wide area network NW1, and may be configured only with devices connected to the LAN.

(6)
Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope described in the claims.

  The present disclosure can be used for an article management system.

1: target facility 10: air conditioner 15: outdoor unit 18: outdoor unit control unit 20, (20a-20f): indoor unit (air conditioner)
25: Indoor unit control unit 30, (30a-30f): Remote control 35: Remote control unit 40, (40a-40f): Shooting unit (shooting device)
41: Shooting unit 42: Shooting data generation unit 43: Shooting data output unit 50: Server 60: Management device 61: Storage unit 62: Acquisition unit 63: Data generation unit 64: Location management unit 65: Reception unit 66: Information search unit (Search part)
67: Output control unit 90: Terminal (output unit)
100: Management system (article management system)
631: Extraction unit 632: Learning unit 633: Extraction data generation unit CI: Ceiling D1: Shooting data D2: Extraction data (article data)
D3: Location history data (article data)
D4: Display data D5: Feature data GP: Gas communication pipe LP: Liquid communication pipe M1: Program information storage area M2: Environmental information storage area M3: System information storage area M4: Person information storage area M5: Article information storage area M6: Shooting data storage area M7: Extraction data storage area M8: Location history data storage area M9: Input information storage area M10: Output information storage area M11: Feature data storage area N1: First neural network N2: Second neural network N3: First 3 neural network N4: 4th neural network NW1: wide area network OB: article PS: person SP, (SP1-SP6): target space SW: sidewall TB1: photographing unit information table TB2: person information table cb1-cb4: communication line

JP 2017-45373 A

Claims (8)

A plurality of imaging devices (40) installed in the target space (SP);
A data generation unit (63) for extracting an article (OB) included in an image (D1) photographed by the photographing device and generating article data (D2, D3) for specifying the extracted article;
A storage unit (61) for storing the article data generated by the data generation unit;
A search unit (66) for performing information search on the specific product in the product data stored in the storage unit;
A receiving unit (65) for receiving a request regarding the article whose information is to be searched by the search unit;
Comprising
Article management system (100). The data generation unit extracts a location of the article included in the image captured by the imaging device,
The article data includes information for specifying a location of the article.
The article management system (100) of claim 1. The data generation unit extracts a location time with respect to the article included in the image captured by the imaging device,
The article data includes information for specifying a location time with respect to the extracted article.
The article management system (100) according to claim 1 or 2. The data generation unit extracts a person (PS) present in the surroundings with respect to the article included in the image captured by the imaging device,
The article data includes information for identifying the extracted person with respect to the extracted article.
The article management system (100) according to any one of claims 1 to 3. The data generation unit includes a learning unit (632) that learns characteristics of the article based on an image captured by the imaging device.
The article management system (100) according to any one of claims 1 to 4. The storage unit stores the article data generated by the data generation unit.
The article management system (100) according to any one of claims 1 to 5. An output unit (90) for outputting a result of the information search by the search unit;
The article management system (100) according to any one of claims 1 to 6. Any / all of the imaging devices are arranged in an air conditioner (20) installed in the target space.
The article management system (100) according to any one of claims 1 to 7.

Images