JP7174565B2 - Management device, management system, management method and program - Google Patents

Description

The present invention relates to a management device, management system, management method and program.

Based on images and sounds obtained from security devices installed in homes and shops, acquaintance information is recorded in advance to identify people who do not need to be reported, and new images or sounds of visitors are recorded. A security system is known in which, when acquired, the acquired information is compared with pre-recorded acquaintance information (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP 2016-177484 A

However, in the above security system, if a new visitor is not recorded in the acquaintance information, it is regarded as an unauthorized intruder and reported to the user terminal, and the user is requested to operate the user terminal. It was troublesome for users.

In one aspect of the present invention, a detection unit that detects a person outside the entrance to a specific area of the building, a person outside the entrance to the specific area of the building, and whether or not a specific process is required. Based on the learning result of the relationship, an estimation unit that estimates whether or not a specific process is required from the person outside the entrance detected by the detection unit A management device is provided that includes a processing unit that executes processing.

In one aspect of the present invention, there is provided a management system comprising a plurality of management devices, and a person who lives, works, or goes to school in a building in which one of the plurality of management devices is installed. However, when a building in which another management device is installed among a plurality of management devices is newly used as a residence, place of work, or commuting to school, the learning result of one management device is transferred to the other management device. A management system is provided for sharing.

In one aspect of the present invention, a detecting step of detecting a person outside the entrance to a specific area of the building, and a person outside the entrance to the specific area of the building and whether or not specific processing is required. Based on the learning result of the relationship, an estimation stage of estimating whether or not a specific process is required from the person outside the entrance detected in the detection stage, and if the estimation stage estimates that a specific process is necessary, a specific A management method is provided comprising a processing stage for performing a process.

An aspect of the present invention provides a program that causes a computer to execute the management method described above.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

FIG. 2 is a schematic diagram of the situation of the entrance, which is a common part of the condominium 30 in which the management device 100 is installed, according to the first embodiment. 1 is a block diagram of a management device 100 according to the first embodiment; FIG. FIG. 4 is a flow diagram as an example of learning according to the first embodiment; FIG. 10 is a flowchart as an example until specific processing is executed based on learning results according to the first embodiment; FIG. 10 is a flowchart as an example until specific processing is executed based on learning results according to the first embodiment; FIG. 11 is a flowchart as an example until specific processing is executed based on learning results according to the second embodiment; 12 illustrates an example computer 1200 in which aspects of the present invention may be implemented in whole or in part; FIG.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention. In addition, in the drawings, the same or similar parts may be denoted by the same reference numerals to omit redundant description.

FIG. 1 is a schematic diagram of the situation of an entrance, which is a common part of a condominium 30 in which a management device 100 is installed, according to the first embodiment. An entrance door 41 that automatically opens and closes is provided at the entrance of a condominium 30, and the entrance door 41 separates an outer area 31 and an inner area 33 in the entrance. The condominium 30 is an example of a building, the area 33 inside the entrance door 41 is an example of a specific area in the building, and the entrance door 41 is an example of an entrance to the specific area.

An operation panel 43 is installed in the area 31 outside the entrance door 41 . In the situation shown in FIG. 1, in the area 31 outside the entrance door 41, a first person 61 operating the operation panel 43, a second person 63 holding hands with the first person 61, and the first person There is a third person 65 standing in isolation at a distance from 61 and second person 63 . The first person 61, the second person 63, and the third person 65 are examples of persons outside the entrance to the specific area, and the first person 61 unlocks the entrance door 41 using the operation panel 43. The second person 63 and the third person 65 are examples of persons different from the operator.

The operation panel 43 is connected to the management device 100 via a communication network 50 by wired communication or wireless communication. The operation panel 43 has an interface operated by a resident of the condominium 30, a visitor to the condominium 30, or the like. The interface includes, for example, a touch panel operated by an operator and a camera that captures an image of the operator's face from the front. Further, the interface may collect the uttered voice of the operator and utter the uttered voice collected by the indoor intercom provided in each living room of the condominium 30 . The touch panel includes, for example, an unlock button for unlocking the entrance door 41 and a call button for inputting and calling the room number of the condominium 30 . Information input/output through the interface of the operation panel 43 is transmitted/received to/from the management device 100 .

The management device 100 includes a device body 101 and a detection unit 102 . The apparatus main body 101 may be integrated with the operation panel 43 or may be provided separately from the operation panel 43 in, for example, the janitor's room of the condominium 30 . The device main body 101 is connected to the operation panel 43 and the detection unit 102 via the communication network 50 by wired communication or wireless communication. The apparatus main body 101 further communicates with pre-registered notification destinations, for example, an indoor intercom provided in each room of the condominium 30, a communication terminal of each resident of the condominium 30, and a communication terminal of each resident of the condominium 30, via wired communication or wireless communication via the communication network 50. It may be connected to at least one of the communication equipment of the janitor's office 30, the communication equipment of the security company that manages the security of the condominium 30, or the communication equipment of the police.

The detection unit 102 detects a person in the area 31 outside the entrance door 41 . The detection unit 102 includes, for example, a camera that captures moving images. As an example, the detection unit 102 is installed on the wall surface above the entrance door 41 and captures an image of the area 31 outside the entrance door 41 . In FIG. 1, as an example, a single detection unit 102 is installed, but instead of this, a plurality of detection units that detect at least a person in the area 31 outside the entrance door 41 from a plurality of different directions. 102 may be installed. The detection unit 102 transmits captured image data to the apparatus main body 101 via the communication network 50 at any time. The image data may be still image data or moving image data. The image data includes image data in which a person in the area 31 outside the entrance door 41 appears as a subject.

The device body 101 incorporates artificial intelligence (AI). In this embodiment, the artificial intelligence of the device main body 101 learns the relationship between the person in the area 31 outside the entrance door 41 and the necessity of specific processing among a plurality of mutually different processing, and based on the learning result. , from the person in the outer region 31 detected by the detection unit 102, the necessity of a specific process among a plurality of mutually different processes is estimated. Alternatively, the artificial intelligence of the apparatus main body 101 learns the relationship between a person in the area 31 outside the entrance door 41 and whether or not a single specific process is required, and based on the learning result, performs detection. Whether or not the single specific process is necessary may be estimated from the person in the outer region 31 detected by the unit 102 .

As an example, the specific processing is performed when the device main body 101 receives an input of an unlocking operation of the entrance door 41 by the operator of the operation panel 43 from the operation panel 43 via the communication network 50. and request processing for requesting confirmation of a person different from the operator. The request processing can be changed in real time by presenting the above-mentioned information about different persons to the operator, for example, on the touch panel of the operation panel 43 after receiving the operator's input of the unlocking operation of the entrance door 41. It may also include a process of requesting confirmation of the person. Further, in the request process, after receiving the operator's input of the unlocking operation of the entrance door 41, the operator receives the information about the different person from the operator's pre-registered communication terminal or indoor intercom, for example. may include processing for requesting confirmation of a different person after the fact by presenting the . It should be noted that, for example, presenting information about different people on each device may mean displaying images of different people detected by the detection unit 102 on the display of each device, or displaying an index indicating the existence of a different person. It may include at least one of displaying on a display and uttering a sound from each device to inform that a different person is present.

Specific processing also includes warning processing of the entrance door 41 as an example. The warning process includes locking the entrance door 41, closing the entrance door 41, locking the entrance to the condominium 30 other than the entrance door 41, closing the relevant entrance, issuing a warning at the entrance door 41, and pre-registered examples of which are described above. including at least one such as notification to the notification destination. The warning on the entrance door 41 includes, for example, at least one of utterance of a warning voice from the operation panel 43 and blinking of a warning lamp provided in the area 31 outside the entrance door 41 .

Specific treatments also include, by way of example, dominant treatments. Predominant processing includes, for example, (A) unlocking of the entrance door 41, (B) opening of the entrance door 41, (C) release of alert of the entrance door 41, and (D) pre-registered notification At least one of communication for requesting remote control of at least one of (A) unlocking, (B) opening the door, and (C) canceling the alert. The pre-registered notification destination referred to here includes, for example, the communication terminal of the resident of the visiting destination called from the operation panel 43 by the operator of the operation panel 43 .

Further, when it is estimated that a specific process is required, the device body 101 executes the specific process.

By using such processing by artificial intelligence, the management device 100 can determine whether specific processing is necessary not only for a person who has entered and exited the condominium 30 but also for a person who has visited the condominium 30 for the first time. Estimate whether or not, and execute automatically.

FIG. 2 is a block diagram of the management device 100 according to the first embodiment. The management device 100 includes the detection unit 102 in addition to the device main body 101 as described above. Device body 101 includes communication section 111 , estimation section 113 , storage section 117 , and processing section 119 .

Upon receiving arbitrary information from the operation panel 43 via the communication network 50 , the communication unit 111 outputs the information to the estimation unit 113 . For example, when image data is received from the detection unit 102 via the communication network 50 , the communication unit 111 outputs the image data to the estimation unit 113 and the processing unit 119 . When arbitrary information and a control signal are input from the estimating unit 113 and the processing unit 119, the communication unit 111 transmits the control signal to either the operation panel 43 or the entrance door 41 via the communication network 50. Send any information and/or control signals.

Estimation unit 113 includes the artificial intelligence described above. The estimating unit 113 learns the relationship between the person in the area 31 outside the entrance door 41 and whether or not specific processing is required. Estimate the need for a particular process.

As an example of the above learning, the estimation unit 113 in the present embodiment also learns the relationship between the interpersonal relationship between the operator of the operation panel 43 and a person different from the operator and the necessity of the above specific processing. do. The interpersonal relationship is, for example, a relationship in which the operator is aware of entering the area 33 inside the entrance door 41 together with the different person, and a relationship in which the operator is aware of entering the area 33 inside the entrance door 41. Including non-relationships. The estimating unit 113 further determines the relationship between the interpersonal relationship between the operator and the different person and the necessity of the specific process based on the confirmation result of the different person by the operator after the request process is executed. may be learned.

As an example, if at least one of the following conditions (D) to (K) is satisfied, the estimating unit 113 moves to the area 33 inside the entrance door 41 together with a different person who operates the operation panel 43 as described above. It may be learned as a relationship in which one is conscious of entering. In addition, the situations shown in the following conditions (D) to (K) are, as an example, in addition to the detection target range by the detection unit 102, the area 33 inside the entrance door 41 of the condominium 30, the corridor, the elevator, and , a parking lot, or the like, and image data indicating the detected situation may be transmitted to the estimating unit 113 .

(D) The time during which the distance between the operator of the operation panel 43 and a different person is shorter than a predetermined length threshold is longer than the predetermined time threshold. (E) A different person entered the inner area 33 from the entrance door 41 within a time period shorter than the predetermined time threshold after the operator of the operation panel 43 entered the inner area 33 from the entrance door 41. thing. (F) The operator of the operation panel 43 is in direct or indirect contact with a different person. (G) There is a history that the operator of the operation panel 43 and a different person have entered the inner area 33 from the entrance door 41 together more than a predetermined number of times.

(H) There is a history in which the operator of the operation panel 43 and a different person have moved together in the region 33 inside the entrance door 41 for a distance equal to or greater than a predetermined distance threshold or more. thing. (I) A history in which the operator of the operation panel 43 and a different person were together in the living room of the operator or in the meeting room or the entrance hall, which are the shared parts of the condominium 30, for more than a predetermined time threshold. to be. (J) The image published by the operator of the control panel 43 on the SNS includes a person different from the operator. (K) A different person visits the operator of the operation panel 43, and the operator responds well, for example, shows a good facial expression, answers in a good tone, or invites the operator into the room. have a history.

The estimating unit 113 in this embodiment also learns the relationship between the person in the area 31 outside the entrance door 41 of the condominium 30 and whether or not the warning process is necessary. Examples of persons requiring caution processing include at least any of thieves, robbers, suspicious persons, malicious salesmen, stalkers, illegal intruders, and illegal dumpers of garbage. Such a person may be registered, for example, in the database of the security company of the condominium 30 or the database of criminals of the police as a target person requiring caution processing, and the actions taken by such a person are also recorded. may have been The estimating unit 113 also learns the relationship between the situation of a person in the area 31 outside the entrance door 41 of the condominium 30 and the necessity of the warning process. In this case, the estimating unit 113 acquires, from the above-described database, for example, a record of actions taken by a person registered as a target person requiring caution processing, and from the record, an entrance door that requires caution processing. The situation of a person in the area 31 outside 41 may be learned.

As an example, the estimating unit 113 determines that a person outside an entrance to a specific area in a building requiring caution processing, as described above, when at least one of the following conditions (L) to (N) is satisfied: It may be learned as a situation of In addition, the situations indicated by the following conditions (L) to (N) include, as an example, the range to be detected by the detection unit 102, as well as the area 33 inside the entrance door 41 of the condominium 30. Image data indicating the detected situation may be detected by a camera or the like installed at a location and transmitted to the estimation unit 113 .

(L) performing a specific behavior pattern in the condominium 30; Specific behavior patterns include, for example, repeatedly looking around in front of an entrance to a specific area in condominium 30, checking the nameplates and contents of a plurality of mailboxes, checking detection unit 102 and control panel 43 Covering the camera with a cloth, etc., leaving without performing any action such as unlocking the door after coming near the door of any room, pressing the doorbell, etc., entering a specific area in the apartment 30 Leaving without being able to unlock the door after operating the lock of the entrance, Leaving without doing anything after touching the door of an arbitrary room for more than a predetermined time threshold, Entrance to a specific area in the condominium 30 Wear clothes that are expected to break the lock of the apartment, to behave differently from the usual flow line although it is a resident of the apartment 30, and to perform a specific action such as a delivery company or a cleaning company. including at least one of the following: Note that the management device 100 recognizes that a certain person is a resident of the condominium 30 by unsupervised learning from image data detected by the detection unit 102 or a camera or the like installed at an arbitrary location in the building. Recognition may be performed by performing supervised learning together with information input by the person, and recognition is performed by registering a person related to the person, such as a parent who lives with the person, a manager of the apartment 30, etc. You may

The action of breaking the entrance lock includes, by way of example, using a picking tool, crowbar, drill, etc. against the entrance lock. For example, a resident of the condominium 30 performing an action that differs from the usual flow line is, for example, visiting a room that has been registered in advance in relation to the resident, for example, a room other than the resident's room. using a floor other than the floor registered in advance in the

(M) One or more people who live in the condominium 30 have a history of bad behavior toward people in the area 31 outside the entrance door 41 of the condominium 30 . In this embodiment, an apartment building 30 is used as an example of a building in which the management device 100 is installed. One or more persons who live, work, or go to school in the building where the building is located have a history of bad behavior toward persons outside the entrance to a specific area of the building. good too.

Note that the estimating unit 113 determines that a certain person uses the building as a residence, place of work, or school, from image data or the like detected by the detecting unit 102 or a camera or the like installed at an arbitrary location in the building. The person may be recognized by unsupervised learning, or may be recognized by performing supervised learning together with information input by the person. The person may be recognized by being registered by a boss of the company where the person works, a teacher of the school where the person goes to school, or the like.

(N) Appearing in front of the entrance to a specific area in the condominium 30 with a specific object, or a person other than the residents of the condominium 30 including garbage in front of the outer door of the garbage disposal area of the condominium 30 Show up with a litter bag. For example, the entrance to the specific area is the entrance door 41, the outer door of the bicycle parking lot that is the shared part of the apartment 30, the outer door of the parking lot, the door of each living room, the outer gate of each living room, the window of each room, and , garden gates for each room, and/or the like. Specific objects include, by way of example, picking tools, crowbars, and/or drills.

The estimating unit 113 in this embodiment also learns the relationship between the person outside the entrance to the specific area in the building and whether or not the superior processing is necessary. As an example, the estimating unit 113 determines whether an object belonging to the person outside the entrance door 41 has the same visual feature as the surface of the object belonging to the person exiting from the inner area 33 through the entrance door 41 . It may be learned that the case where it is detected from the surface is a situation that requires superior processing. Examples of objects attached to a person include at least one of clothes, bags, shoes, wristwatches, glasses, hats, and the like.

As an example, the estimating unit 113 determines that at least one of the following conditions (O) to (S) is satisfied, outside the entrance to the specific area of the building that requires superior processing. You can learn as a person. In addition, the situations shown in the following conditions (O) to (S) include, as an example, the detection target range by the detection unit 102, and similarly to the above, the area 33 inside the entrance door 41 of the condominium 30. Image data indicating the detected situation may be detected by a camera or the like installed at a location and transmitted to the estimation unit 113 .

(O) A person outside the entrance to the specific area in the condominium 30 has a history of entering the entrance more than a predetermined number of times. (P) A person outside the entrance to the specific area in the condominium 30 has a history of staying in the living room of the condominium 30 for a predetermined time threshold or longer. (Q) A person outside the entrance to a specific area in the condominium 30 is pictured together with the resident in the image that the resident of the condominium 30 publishes on the SNS. (R) A person outside the entrance to a specific area in the condominium 30 has a history of visiting a resident of the condominium 30 and being treated well by the resident. (S) At least one of belongings and clothes of a person outside the entrance to the specific area in the condominium 30 has a history of passing through the entrance a predetermined number of times or more.

The above-described various learnings by the estimating unit 113 may be started after the management device 100 is installed in the condominium 30. Before the management device 100 is installed in the condominium 30, the learning may be started at a place different from the condominium 30, for example. A certain amount of learning may be completed in advance at the manufacturing stage of the management device 100 at the factory, and after the management device 100 is installed in the condominium 30, the learning may be continued in the same manner as the prior learning. In the latter case, the management device 100 may reduce the amount and period of learning to be performed after being installed in the condominium 30 compared to the former case.

The storage unit 117 stores a program for operating each component of the management apparatus 100, and is referred to by the estimation unit 113 and the processing unit 119, for example. The storage unit 117 also includes, as an example, audio data read by at least one of the estimation unit 113 and the processing unit 119, still image data, video data, text data, residents of the condominium 30, acquaintances of the residents, people who enter and exit the condominium 30. Information on at least one of the trader and the target person who needs to be alerted, information on the above-mentioned pre-registered notification destination, information on conditions for making multiple judgments, and information on multiple processes, store at least one of

When the estimation unit 113 estimates that a specific process is necessary, the processing unit 119 executes the specific process. More specifically, when the estimation result is input from the estimation unit 113 , the processing unit 119 performs specific processing included in the estimation result by, for example, referring to the storage unit 117 . Then, a control signal based on specific processing is transmitted to at least one of the operation panel 43 and the entrance door 41 . The processing unit 119 also receives detection results such as image data indicating a person in the area 31 outside the entrance door 41 detected by the detection unit 102 via the communication unit 111 and refers to the storage unit 117 . By doing so, when it is determined that the detection result satisfies a predetermined condition, a specific exceptional process is preferentially executed.

FIG. 3 is a flow diagram as an example of learning according to the first embodiment. The learning flow of FIG. 3 is started, for example, when the management device 100 is installed in the condominium 30 and becomes operable.

The management device 100 waits until one of the operation buttons on the operation panel 43, such as the unlock button and the call button, is pressed (step S101: NO). When the operation button of the operation panel 43 is pressed and the input information corresponding to the operation is received from the operation panel 43 via the communication network 50 (step S101: YES), the management device 100 , the estimating unit 113 receives image data transmitted from the detecting unit 102 as needed, and the estimating unit 113 refers to the image data together with the input information (step S103).

The estimating unit 113 determines whether or not a plurality of people are shown in the image of the image data referred to (step S105), and when it is determined that a plurality of people are shown (step S105: YES), 50, message information requesting input of information about the operator of the operation panel 43 himself and information about interpersonal relationships with persons other than the operator is transmitted to the operation panel 43 and displayed on the touch panel of the operation panel 43. (step S107).

The operator's own information is, for example, if the operator is a resident of the condominium 30, a resident-related person such as a relative or friend of the resident of the condominium 30, a tutor, or a home care helper, or if the operator is a resident of the condominium 30 or a manager. Contains information that indicates who is expected to be a person, and any other person. In the case of a resident of the condominium 30, the information may further include information identifying the individual, such as the room number of the condominium 30 and the name.

The information on the interpersonal relationship between the operator and a different person is, for example, whether or not the operator is conscious of entering the area 33 inside the entrance door 41 with different persons. In addition, when information that includes information and is the conscious relationship is input, a different person may be a resident of the same room as the operator, a relative or friend of the operator, a tutor, a home care helper, etc. It includes information indicating at least one of a person related to the operator and a person recognized by the operator as a resident of another room. As an example, these pieces of information to be input may be listed in advance and displayed as options on the touch panel of the operation panel 43 .

The estimating unit 113 waits until both information are input from the operation panel 43 via the communication network 50 (step S109: NO), and when both information are input (step S109: YES), the input information is stored in the storage unit 117 together with the input information corresponding to the operation on the operation panel 43 and the image data referred to, and one week has passed since the learning based on the input from the operation panel 43 and the detection unit 102 was started. It is determined whether or not (step S111).

When the estimating unit 113 determines in step S105 that a plurality of persons are not shown in the image, that is, that only the operator is present in the area 31 outside the entrance door 41 (step S105: NO), the communication network 50 , to the operation panel 43, and display it on the touch panel of the operation panel 43 (step S108). The estimation unit 113 waits until the information is input from the operation panel 43 via the communication network 50 (step S110: NO), and when the information is input (step S110: YES), the input information is stored in the storage unit 117 together with the input information corresponding to the operation on the operation panel 43 and the image data referred to, and one week has passed since the learning based on the input from the operation panel 43 and the detection unit 102 was started. It is determined whether or not (step S111).

If the estimating unit 113 determines that one week has not passed in step S111 (step S111: NO), it returns to step S101, and if it determines that one week has passed (step S111: YES), the flow is finish. Note that in the repetition of steps S107 and S109: NO, and the repetition of steps S108 and S110: NO, even if it is repeated for a predetermined time threshold or more or a predetermined number of times or more, , when there is no information input from the operation panel 43, the repetition may be terminated and the flow may be terminated.

The estimating unit 113 determines whether the person in the area 31 outside the entrance door 41 of the condominium 30 is within one week from the start of the learning based on the input from the operation panel 43 and the detecting unit 102. , image data input from the detection unit 102, input information corresponding to each operation of the unlock button and the call button on the operation panel 43, operator information, and interpersonal relationship between the operator and a person other than the operator. Using at least any of the information, that is, using labeled training data, the interpersonal relationship between the operator of the operation panel 43 and a different person, the relationship between the interpersonal relationship and the necessity of request processing, and the entrance Supervised learning is performed on at least one of the relationships between the person in the area 31 outside the door 41 and the necessity of superior processing. Techniques such as deep learning, machine learning, and statistical processing are used for learning.

Examples of the above learning method include support vector machine (SVM), recurrent neural network (RNN), and Bayesian network (BN). The SVM can discriminate even unknown samples with little error by learning a relatively small number of samples. However, although the learning result can be understood to some extent, it is difficult for humans to understand, and it is difficult to understand the causal relationship of discrimination. RNN requires many training samples and takes a long time to learn. It is effective for objects that are affected by contextual relationships, such as time series, but it is difficult to understand the learning results. BN requires many training samples and takes a long time to learn. Learning results are expressed in the form of a network connecting conditional probability models, so causal relationships are easy to understand. In the learned network, known nodes and unknown nodes can be used in any combination by belief propagation.

In learning, the estimation unit 113 extracts image feature amounts using a convolution neural network (CNN). CNN is a type of deep learning neural network (DLNN), and DLNN is a three-layer neural network (NN) expanded to four or more layers, and in recent years it has become possible to accumulate a large amount of data. It is known that due to recent developments and the advancement of computers, the performance has been improved more than the new calculation methods that appeared after the NN. Among them, CNN is modeled on the visual cortex (V1) of the brain, and if it is used to improve the accuracy of image recognition in advance, image processing results similar to those of the visual cortex of the brain can be obtained. Therefore, it is a suitable method for extracting feature values of images. It should be noted that NN itself began to be actively researched around the 1980s, and by connecting multiple nodes and performing non-linear processing at each node, it is possible to create seemingly meaningless data arrays (patterns). It is a computational technique (and a data structure for it) that can assign meaningful symbols.

In this embodiment, the training data includes image data detected by the detection unit 102, input information corresponding to each operation of the unlock button and the call button on the operation panel 43, operator information, operator The training data is, for example, voice data exchanged between the intercom in the living room of the condominium 30 and the operation panel 43, or the reader of the operation panel 43. It may also include identification information of the electronic key to be read. The feature quantity indicated by the voice may include, for example, a voiceprint, utterance content, etc. The utterance content may include, for example, "I'm home", "Long time no see", "It's ○○ transportation", "○○ product introduction Can you let me When such other training data is included, methods other than CNN such as deep learning, machine learning, or statistical processing may be used as conversion methods close to brain information processing for each type of training data. For example, a self-organizing map (SOM) suitable for extracting features shown in speech, a recurrent neural network (RNN) suitable for extracting features from time-series data, and a deep neural network that can be used in the same way as an RNN. (DNN) or other technique may be used. However, when handling speech using SOM, it is difficult to handle time series with SOM alone, so it is necessary to combine it with another method. Also, when handling speech using RNN, it is possible to extract the feature quantity indicated by the speech in combination with preprocessing that is separately selected. Unlike RNN, DNN can learn the preprocessing itself.

After the estimation unit 113 performs supervised learning on the feature amounts of the person shown in the various labeled images acquired in the above learning flow, the estimation unit 113 performs various images of the person in the area 31 outside the entrance door 41. Continue to acquire training data without the above input information, operator information and interpersonal relationship information, that is, without labels, automatically acquire the feature amount of the person shown in the image, and repeat, The interpersonal relationship between the operator of the operation panel 43 and a different person, the relationship between the interpersonal relationship and the necessity of request processing, and the relationship between the person in the area 31 outside the entrance door 41 and the necessity of superior processing. Unsupervised learning is performed on at least one of the relationships. Estimating unit 113 also performs unsupervised learning on the relationship between the situation of a person in outer area 31 and whether or not the above warning process is necessary, in addition to pre-learning before management device 100 is installed in condominium 30 . conduct. As an example of unsupervised learning related to warning processing, the estimating unit 113 performs warning processing for the operator when the flow ends due to the operator not inputting information from the operation panel 43 in the above learning flow. You may learn that you need The estimating unit 113 also determines the interpersonal relationship between the operator and the different person and the necessity of the specific process based on the confirmation result of the different person by the operator after the request process is executed. can be supervised learning. The features of a person shown in an image include, for example, features such as face, facial expression, body shape, movement, walking style, gesture, clothing, accessories, glasses, belongings, and line of sight.

Unsupervised learning requires feedback from residents, managers, and the like of the condominium 30 at arbitrary timing regarding the appropriateness of the estimation results. For example, each time the estimation unit 113 estimates the necessity of specific processing from a person in the outer region 31 detected by the detection unit 102, the estimation unit 113 sends the estimation result to a pre-registered notification destination related to the person. Feedback may be obtained from the person to whom the notification is sent by confirming the estimation result and returning information indicating estimation failure when the estimation result is incorrect. In addition, in order to further save the resident's trouble, for example, the management device 100 actually executes the warning process based on the estimation result, and erroneously executes the warning process even though it was not necessary to execute the warning process. Feedback may be obtained by receiving an input for canceling the warning process from the operation panel 43, an intercom in the living room, a specific notification destination, or the like.

Based on these learning results, the estimation unit 113 estimates the necessity of specific processing from the person in the outer area 31 detected by the detection unit 102 . The estimating unit 113 uses the same method as the above learning method when estimating whether or not a specific process is necessary. The same applies to subsequent learning and estimation.

The above-mentioned "relationship" learned by the estimating unit 113 is also called a "learning model" in the technical field, and the estimating unit 113 receives input information corresponding to operations such as unlocking on the operation panel 43, It includes the regularity, pattern, etc. between these data extracted from operator information, interpersonal relationship information, and information indicating the necessity of specific processing. The "relationship" can also be said to be a correspondence relationship between input information, operator information, and interpersonal relationship information as input data, and information indicating the necessity of specific processing as output data.

FIGS. 4 and 5 are flow charts as an example until specific processing is executed based on the learning result according to the first embodiment. A and B shown in FIG. 4 follow A and B shown in FIG. 5 respectively. The description of the flow continuing from FIG. 4 to FIG. 5 starts, as an example, when the period of supervised learning described using FIG. 3 ends.

The estimating unit 113 waits until the detecting unit 102 detects a person in the area 31 outside the entrance door 41, that is, until image data indicating the person in the outside area 31 is received from the detecting unit 102 (step S201: NO), if the image data is received (step S201: YES), based on the above learning result, the situation of the person detected by the detection unit 102 shown in the image data is an example of specific processing. It is estimated whether some warning process is required (step S203).

If the estimating unit 113 estimates that the situation of the person detected by the detecting unit 102 does not require alert processing (step S203: NO), the estimating unit 113 detects the person within a predetermined time after the detection unit 102 detects the person. , whether or not any of the operation buttons on the operation panel 43, for example, the unlock button and the call button, has been pressed, that is, the input information corresponding to the operation has been received from the operation panel 43 via the communication network 50. It is determined whether or not (step S205).

If the estimating unit 113 determines that the input information has not been received within the time period (step S205: NO), the process returns to step S201, and if it determines that the input information has been received (step S205: YES). , the image data transmitted from the detection unit 102 at any time is received via the communication network 50, and the estimation unit 113 refers to the image data together with the input information (step S207).

The estimating unit 113 estimates whether or not the operator of the operation panel 43 shown in the referenced image data needs warning processing, which is an example of specific processing, based on the above learning result (step S209). ), if it is estimated that the warning process is not necessary (step S209: NO), it is subsequently determined whether or not a plurality of persons are shown in the image of the image data referred to (step S211). When the estimating unit 113 determines that a plurality of persons are shown in the image (step S211: YES), the estimating unit 113 subsequently refers to, for example, the storage unit 117 to determine whether a person different from the operator of the operation panel 43 is It is determined whether or not the person is registered as a person who is previously permitted to enter the area 33 inside the entrance door 41 (step S213).

If the estimation unit 113 determines that the different person is not permitted and registered (step S213: NO), the estimation unit 113 estimates the interpersonal relationship between the operator and the different person based on the above learning result. Here, as an example, the situation of the area 31 outside the entrance door 41 shown in FIG. 1 will be described below.

The estimation unit 113 determines that the interpersonal relationship between the first person 61 who is the operator and the second person 63 who is one of the different persons is such that the first person 61 moves to the area 33 inside the entrance door 41 together with the second person 63 . Assume that the relationship conscious to enter into, ie the second person 63 is the companion of the first person 61 . In addition, the estimating unit 113 determines that the first person 61 is inside the entrance door 41 together with the third person 65 as an interpersonal relationship between the first person 61 who is the operator and the third person 65 who is another one of the different persons. In other words, the third person 65 is a companion who tries to enter the area 33 inside the entrance door 41 in response to the unlocking operation by the first person 61. presume. For example, the tailgating person enters the condominium 30, a resident of the condominium 30 who has been permitted to enter the area 33 inside the entrance door 41 in advance, an acquaintance of the resident, a resident of the condominium 30 who has not been permitted to enter, or enters the condominium 30. This includes at least one of a visitor who has a valid reason and an unauthorized intruder who enters the condominium 30 without a valid reason.

When estimating the interpersonal relationship between the operator and a different person, the estimating unit 113 estimates the relationship between the estimated interpersonal relationship and the necessity of request processing, which is an example of specific processing, based on the above learning result. (step S215). More specifically, the estimating unit 113 estimates that the request processing is unnecessary for the relationship that the second person 63 is the companion of the first person 61, while the third person 65 is the first person When it is estimated that the request processing is necessary for the relationship of 61 companions (step S215: YES), a control signal for executing the request processing is output to the processing unit 119. FIG.

As an example of request processing, the processing unit 119 causes the touch panel of the operation panel 43 to display an image of the area 31 outside the entrance door 41 based on the control signal input from the estimation unit 113, and displays the image of the area 31 outside the entrance door 41. An operation is performed from the communication unit 111 via the communication network 50 in order to execute a process of requesting the first person 61 who is the operator to confirm in real time the third person 65 who is another person different from the operator. A control signal corresponding to the process is transmitted to the board 43 (step S217).

Based on the control signal received from the processing unit 119, the operation panel 43 causes the touch panel to display the image of the area 31 outside the entrance door 41 and the third person 65 enter the area 33 inside the entrance door 41 together. and a message requesting a decision as to whether or not to permit the operation is displayed, and when an input from the first person 61 who is the operator is received, the input information is sent to the processing unit 119 via the communication network 50. Send.

When receiving the input information from the operation panel 43, the processing unit 119 determines whether or not the input information indicates an entry permission input (step S219). If the estimation unit 113 determines that the permission input is indicated (step S219: YES), the operation button indicated in the input information received from the operation panel 43 in step S205, for example, the unlock button or the call button In order to execute the process, a control signal corresponding to the process is transmitted from the communication unit 111 to the entrance door 41 or the intercom of a specific living room via the communication network 50 (step S221), and the flow ends.

In step S203, when the estimating unit 113 estimates that the situation of the person detected by the detecting unit 102 requires caution processing based on the above learning result (step S203: YES), the estimating unit 113 executes caution processing. A control signal for this purpose is output to the processing unit 119, and the processing unit 119 executes warning processing based on the control signal (step S223), and the flow ends. For example, the situation of a person in the area 31 outside the entrance door 41 that is estimated to require caution processing is that the person is in possession of a weapon or a picking tool in the area 31 outside the entrance door 41, or that the person is The person frequently visits the condominium 30 and has a history of forcing the residents to offer vicious sales, the person looks around in front of the entrance door 41 many times, and the person has a plurality of objects in the outer area 31. This includes at least one of checking the name tag and contents of the mailbox, and covering the detection unit 102 and the camera of the operation panel 43 in the outer area 31 by the person with a cloth or the like. Further, an example of the warning process here may be to utter a voice from the operation panel 43, "What is your business?" In this case, the management device 100 can indicate to the person that it is constantly monitoring the outer area 31, and if the estimation by the artificial intelligence is incorrect and the person enters the condominium 30, the management device 100 is not allowed to enter the condominium 30. Even if the visitor has a reason, he or she does not utter a warning message, so it is possible to avoid making the visitor uncomfortable. Also, an example of the warning process here may be to utter from the operation panel 43 a voice stating, "I have judged you to be a vicious visiting salesperson. Please take over immediately." In this case, the management device 100 can prevent a situation disadvantageous to the residents, such as the person randomly calling the residents of the condominium 30 and making vicious and forced sales.

In step S209, when the estimating unit 113 estimates that the operator of the operation panel 43 shown in the referenced image data requires caution processing based on the above learning result (step S209: YES), the caution A control signal for executing processing is output to the processing unit 119, and the processing unit 119 executes warning processing based on the control signal (step S223), and the flow ends. For example, the operator who is presumed to require alert processing is a person who is being alerted as a stalker by a resident of the living room who is about to be called from the operation panel 43, and a person who performed the calling operation on the operation panel 43. , covering the camera of the operation panel 43 with a cloth, holding a photograph over the camera of the operation panel 43, intentionally shifting the standing position so as not to be reflected in the camera of the operation panel 43, Includes at least one of the following: a person wearing a hat so that their face cannot be seen. Also, an example of the warning process here may be to utter from the operation panel 43 a voice stating, "You are not permitted to visit by the person you are visiting. Please take it back." . In this case, the management device 100 can completely prevent the person from contacting the called resident. Further, an example of the alert processing here may be to utter a voice from the operation panel 43, saying, "Since your face cannot be recognized, you cannot connect to the called resident." In this case, the management device 100 can prompt the person whose face has been unwillingly hidden to redo the operation.

In step S219, when the processing unit 119 determines that the input information received from the operation panel 43 does not indicate an entry permission input (step S219: NO), it executes warning processing (step S223), The flow ends. An example of the warning process here is a process of inquiring the janitor's office of the condominium 30 to call the janitor in order to request identification of the third person 65, and having people from a security company or the police rush to the scene as soon as possible. at least one of such as processing to notify the Alternatively, the processing unit 119 may output the input information to the estimation unit 113, and the estimation unit 113 may perform supervised learning using the image data from the detection unit 102 and the input information as learning materials. The first person 61 may enter the condominium 30 by going to another entrance other than the entrance door 41 after performing the input on the operation panel 43 .

In step S219, the first person 61 cannot determine whether or not the third person 65 should not enter the area 33 inside the entrance door 41. A case is assumed in which priority is given to entering the inner area 33 rather than preventing entry into the area 33 . Therefore, the processing unit 119 not only allows the operation panel 43 to allow the third person 65 to enter the area 33 inside the entrance door 41 together, but also allows the operation panel 43 to open the entrance door 41 because the judgment cannot be made. A control signal may be sent such that an option for shortening the door time is displayed on the touch panel.

In step S211, when the estimating unit 113 determines that a plurality of persons are not shown in the image of the image data referred to and that only the first person 61 who is the operator is in the outer region 31 (step S211: NO ), since it has already been estimated based on the learning result that the first person 61 is a person who does not require caution processing in steps prior to step S211, the process proceeds to step S221. Moreover, when the estimation unit 113 determines in step S213 that a different person is permitted and registered (step S213: YES), the process similarly proceeds to step S221. Also, in step S215, the estimation unit 113 estimates, based on the learning result, that the interpersonal relationship between the operator and the person different from the operator in the area 31 outside the entrance door 41 does not require request processing (step S215: NO), and similarly proceeds to step S221. As described above, the flow continuing from FIG. 4 to FIG. 5 is repeatedly executed while the management device 100 is operating.

In step S217, as an example of the request processing, the processing unit 119 executes the processing of requesting the first person 61 who is the operator of the operation panel 43 to confirm the third person 65 in real time. Alternatively, as an example of request processing, the processing unit 119 may execute processing of requesting the first person 61 who is the operator of the operation panel 43 to confirm the third person 65 after the fact. More specifically, as an example, after step S221, the processing unit 119 connects to a communication destination registered in advance by the first person 61 who is the operator of the operation panel 43, such as a smartphone owned by the first person 61 or a first A process of requesting confirmation of the third person 65 from the first person 61 may be performed by displaying an image of the area 31 outside the entrance door 41 on a display such as an indoor intercom in the living room of the first person 61. . In this case, step S219 is omitted.

Regardless of whether the content of the request processing by the processing unit 119 is to request the first person 61 to confirm the third person 65 in real time or after the fact, the estimation unit 113 is the interpersonal relationship between the operator and the person different from the operator, and the requirement of request processing, which is a specific process, based on the confirmation result of the third person 65 who is different from the operator by the first person 61 who is the operator. You may learn the relationship between negative and negative. In this case, the processing unit 119 determines whether or not to permit the third person 65 to enter the area 33 inside the entrance door 41 together with the first person 61 to the operation panel 43 or the registered notification destination. In addition to presenting options, detailed interpersonal relationships between the first person 61 and the third person 65 are, for example, whether or not they are acquaintances. 30 other Residents may be asked to enter who they are related to, and who is a trader or the like. Note that when the processing unit 119 executes a request process for subsequently requesting confirmation of the third person 65 who is different from the first person 61 who is the operator of the operation panel 43, the estimation unit 113 Request processing for requesting confirmation of the third person 65 to the resident of the living room to which the third person 65, not the first person 61, enters the area 33 inside the entrance door 41 and heads for is sent to the processing unit 119. may be executed.

The estimating unit 113 may receive the input information from the operation panel 43 or a registered notification destination, and use such input information as a learning material to perform more detailed supervised learning. For example, if the input information obtained as feedback of request processing indicates that the first person 61 and the third person 65 are acquaintances, the estimation unit 113 It may be learned that request processing is not necessary even if the first person 61 and the third person 65 are standing apart. As an example, the estimating unit 113 also stores ranking information obtained by ranking the reliability of the third person 65 together with the image data of the third person 65 in the storage unit 117 based on the input information obtained as feedback of the request processing. After that, the ranking information is updated in the same manner each time the third person 65 is detected by the detection unit 102, and the identification necessary for the third person 65 is determined according to the reliability rank indicated in the ranking information. You may change the content of the processing of . As an example, if the third person 65 is a relative or friend of the first person 61, the third person 65 is recognized by the first person 61. If the third person 65 is a resident or a related party of another resident, then if the third person 65 is a trader recognized by the first person 61, it may be lower in this order.

In step S219, the estimating unit 113 accepts the input on the operation panel 43 on the condition that no input is made from the first person 61 who is the operator of the operation panel 43 within a predetermined time. may be terminated and the process may return to step S201. In this case, the estimating unit 113 determines that the first person 61 has left the area 31 outside the entrance door 41 by referring to the image data transmitted as needed from the detecting unit 102 before the time elapses. On the condition that the operation panel 43 receives the above input, the acceptance of the above input may be terminated immediately.

As described above, according to the management device 100 of the first embodiment, it is estimated whether specific processing is necessary not only for a person who has entered and exited the condominium 30 but also for a person who has visited the condominium 30 for the first time. Since it is executed automatically by According to the management device 100 of the first embodiment, for example, when a resident of the condominium 30 or a visitor who has a legitimate reason to enter the condominium 30 unlocks and opens the entrance door 41 of the condominium 30, the condominium 30 If a person who does not have a legitimate reason to enter is trying to enter the condominium 30 together, the intrusion can be prevented by automatically executing the above-described request processing. Further, according to the management device 100 of the first embodiment, for example, a trader who frequently visits the condominium 30 and forces the residents to engage in vicious sales, a person who repeatedly stalks the residents of the condominium 30, and an entrance of the condominium 30 For a person who is acting suspiciously in the area 31 outside the door 41, or for a person who should be on alert, such as a person who is in possession of a weapon or a picking tool, the above warning process is automatically executed. By doing so, the security of the condominium 30 can be enhanced, and there is a possibility that it can contribute to the prevention of crime and the arrest of criminals.

In the first embodiment, the entrance door 41 that separates the outer area 31 and the inner area 33 at the entrance of the condominium 30 has been mainly described as an example of an entrance to a specific area of the building. Further, in the first embodiment, mainly the operation on the operation panel 43 installed in the area 31 outside the entrance door 41 has been described as a trigger for learning and estimation in the estimation unit 113 of the management device 100 . The management apparatus 100 according to the first embodiment is provided with a device having the same functions as the operation panel 43. In addition to the entrance door 41, the management apparatus 100 can also , an outer door of a bicycle parking lot, an outer door of a parking lot, a door of each living room, an outer gate of each living room, a window of each room, a garden gate of each room, and the like. In addition, the management device 100 according to the first embodiment is provided with a device having the same function as the operation panel 43, and operates in buildings other than the condominium 30, such as detached houses, apartments, commercial facilities, And it may be applied to entrances to specific areas in public facilities and the like. When the target building is a detached house, the management device 100 according to the first embodiment is configured to operate at least one of the exterior gate, entrance door, back door, window, garden door, and parking lot door of the detached house. may be applied to

The management device 100 according to the first embodiment may be applied to an elevator of an office building when the target building is an office building that is a commercial facility. For example, assuming that a visitor who is going to visit a company located on a specific floor of an office building gets into the elevator of the office building and presses the floor button, the detection unit 102 detects that the visitor in the elevator The estimating unit 113 detects the visitor, notifies the company as an example of specific processing, displays a guide from the front of the elevator on the floor to the company, and refuses to enter the floor. Based on the learning result of at least one of the relationships between necessity and non-necessity, it may be estimated from the detected visitor that a specific process is required, and the processing unit 119 may execute the specific process. Note that the refusal to enter a floor includes, for example, at least one of disabling button operations on the floor and uttering a warning message in an elevator.

FIG. 6 is a flowchart as an example until specific processing is executed based on the learning result according to the second embodiment. The second embodiment is the first embodiment in that the management device 100 is applied to the entrance door 41 separating the outer area 31 and the inner area 33 at the entrance of the condominium 30 as an example of an entrance to a specific area in the building. Same as form. However, the second embodiment differs from the first embodiment in that the detection of a person in the area 31 outside the entrance door 41 by the detection unit 102 is used as a trigger for learning and estimation in the estimation unit 113 of the management apparatus 100. . Since the management device 100 according to the second embodiment has the same configuration as the management device 100 according to the first embodiment, the same reference numerals are used for the same components, and overlapping descriptions are omitted. The description of the flow of FIG. 6 starts, as an example, when the period of supervised learning described using FIG. 3 ends.

The estimating unit 113 waits until the detecting unit 102 detects a person in the area 31 outside the entrance door 41, that is, until image data indicating the person in the outside area 31 is received from the detecting unit 102 (step S301: NO), if the image data is received (step S301: YES), based on the above learning result, the situation of the person detected by the detection unit 102 shown in the image data is an example of specific processing. It is estimated whether some caution process is required (step S303).

When the estimating unit 113 estimates that the situation of the person detected by the detecting unit 102 does not require alert processing (step S303: NO), the estimating unit 113 continuously estimates whether or not superior processing, which is an example of specific processing, is required (step S305). As in step S209 in the flowchart of the first embodiment, the estimation unit 113 determines whether the person detected by the detection unit 102 requires caution processing between steps S303 and S305. can be estimated.

If the estimating unit 113 estimates that the person detected by the detecting unit 102 does not require superior processing (step S305: NO), the estimating unit 113 returns to step S301. Step S305: YES), a control signal for executing superior processing is output to the processing unit 119. FIG.

Based on the control signal input from the estimation unit 113, the processing unit 119 performs, as an example of superior processing, (A) unlocking the entrance door 41, (B) opening the entrance door 41, and (C) Disarming of the entrance door 41, and (D) Remote operation of at least one of (A) unlocking, (B) opening the door, and (C) disarming for the notification destination registered in advance. In order to execute at least one of the processing of contacting for requesting , the above contact is made (step S307), and the flow ends.

In step S303, when the estimating unit 113 estimates that the situation of the person detected by the detecting unit 102 requires caution processing based on the above learning result (step S303: YES), the estimating unit 113 executes caution processing. A control signal for this purpose is output to the processing unit 119, and the processing unit 119 executes warning processing based on the control signal (step S309), and the flow ends. As described above, the flow of FIG. 6 is repeatedly executed while the management device 100 is operating.

Note that in this flow, if the management device 100 estimates that the person in the area 31 outside the entrance door 41 does not need either the warning process or the superior process based on the above learning result, the warning process neither need to perform any dominant processing. In this case, by operating the operation panel 43, the person may have the management device 100 execute processing corresponding to the operation in the same manner as in the first embodiment. Authentication may be used to unlock and open the door, and call operation may be used to make a call to an indoor intercom in a specific room without using the management device 100 .

As described above, according to the management device 100 of the second embodiment, the same effects as those of the management device 100 of the first embodiment can be obtained. Further, according to the management device 100 of the second embodiment, for example, for residents of the condominium 30 and visitors who have a legitimate reason to enter the condominium 30, the above-described superior processing can be automatically executed. Convenience can be enhanced by

Note that the management device 100 of the second embodiment is applied to both the case where there are a plurality of people in the area 31 outside the entrance door 41 and the case where there is a single person. That is, when there are a plurality of people in the area 31 outside the entrance door 41, the management device 100 of the second embodiment targets all the people in the area 31 outside the entrance door 41, and performs steps S303 and It is estimated whether or not specific processing is necessary in each of step S305, and when it is estimated that at least one person in the outer area 31 needs each specific processing, the specific processing for other persons is performed. The particular process may be performed regardless of whether or not the .

More specifically, when the management device 100 of the second embodiment estimates in step S303 based on the above learning result that the situation of at least one person in the outer area 31 requires caution processing, When it is estimated that the situations of all persons in the outer area 31 do not require caution processing, the process proceeds to step S305. In step S303, the management device 100 of the second embodiment determines that at least one person in the outer area 31 needs superior processing based on the above learning result. is executed, and when it is estimated that all the persons in the outer area 31 do not require superior processing, the process returns to step S301.

Regardless of whether or not the operation panel 43 is provided, the management device 100 according to the second embodiment includes the entrance door 41, the outer door of the garbage disposal area, the outer door of the bicycle parking lot, which are common parts of the condominium 30. It may be applied to at least one of the outer door of the parking lot, the door of each living room, the outer gate of each living room, the window of each room, the garden gate of each room, etc., and other buildings other than the condominium 30 , for example at entrances to specific areas in single-family homes, apartments, commercial establishments and public facilities. If the target building is a detached house, it may be applied to at least one of the exterior gate, entrance door, back door, window, garden door, parking lot door, etc. of the detached house. For example, assuming a situation where a pet kept in a detached house is lost, the detection unit 102 detects the pet returning to the entrance of the house, and the estimation unit 113 identifies the pet as the pet. Based on the learning results of the relationship between at least one of the processes, such as notifying residents, opening the entrance door, and feeding, etc., it is estimated that a specific process is required from the detected pet. and the processing unit 119 may perform specific processing.

A management system according to the third embodiment includes a plurality of management apparatuses 100 according to at least one of the first embodiment and the second embodiment. In the management system, a person who resides, works, or goes to school in a building in which one management device 100 out of a plurality of management devices 100 is installed can use another management device 100 out of the plurality of management devices 100 When the building provided with is newly used as a residence, workplace, or school, the learning result of the one management device 100 is shared with the other management devices 100 . According to the management system according to the third embodiment, as an example, when a specific person moves from a building in which one management device 100 is installed to a building in which another management device 100 is installed, one Learning results can be handed over from the management device 100 to another management device 100 . Further, according to the management system according to the third embodiment, as an example, a person who lives in a residential building in which one management device 100 is installed changes a commercial building in which another management device 100 is installed. In the case of setting it as a place of work, the learning result can be handed over from one management device 100 to another management device 100 .

Note that, in the management system according to the third embodiment, operation information learned by a specific management device 100 is uploaded, for example, to a cloud capable of communicating with a plurality of management devices 100, and is shared with a plurality of other management devices 100. may be shared. The motion information is, for example, information about motion included in a plurality of learned conditions for estimating that a person detected by the detection unit 102 requires superior processing, or information about motion that requires warning processing. and information related to motions included in multiple conditions learned to estimate that request processing is necessary. On the other hand, personal information about a specific person learned by a specific management device 100 is not shared with a plurality of other management devices 100, but is handed over to the management device 100 of the building to which the person has moved. good too.

In any of the first to third embodiments described above, the estimating unit 113, when a certain person takes an action that is difficult to estimate as requiring caution processing for the first time, performs caution processing on the person. It may temporarily learn that an alert process may be required, ie, the person is gray, without inferring that . An action for which it is difficult to assume that caution processing is necessary is, for example, entering the inner area 33 together with the operator of the operation panel 43 without operating the operation panel 43 in the area 31 outside the entrance door 41 of the condominium 30. , a resident of the condominium 30 may visit a room other than the resident's room, walk on a floor different from the floor where the resident's room is located, or place large furniture and home appliances in the premises of the condominium 30 late at night. It is to leave the condominium 30 without owning the furniture and home appliances that have been brought in within a predetermined time, or to throw away the garbage in the dumping area of the condominium 30 by a person other than the residents of the condominium 30.例文帳に追加

In this case, the estimating unit 113 determines if the person who has learned to be gray has taken the same action or other similar action more than a predetermined number of times, or, for example, if the person has If it is not possible to confirm that the person is permitted to enter the area 33 inside the entrance door 41 by inquiring of the resident or the manager, it is possible to learn that the person needs to be alerted. good. Further, the estimating unit 113 causes the processing unit 119 to perform warning processing when, for example, the detecting unit 102 detects that the person is about to take the same action again in the area 31 outside the entrance door 41. good too.

On the other hand, the estimating unit 113 determines that the person who has learned that the color is gray can clearly determine that a superior process is necessary, for example, holding hands with the residents of the condominium 30 in the area 33 inside the entrance door 41. or, for example, when inquiring of a plurality of residents or managers of the condominium 30, it has been confirmed that the person is authorized to enter the area 33 inside the entrance door 41. Alternatively, if the person has taken an action that may require superior processing more than a predetermined number of times, it may be learned that the alert processing is unnecessary for the person. Further, the estimating unit 113 further detects that, for example, the detecting unit 102 detects that the person is about to take another action in the area 31 outside the entrance door 41 that makes it difficult to assume that caution processing is required. , the processing unit 119 may not execute the warning process.

In any of the first to third embodiments described above, the estimating unit 113 of the management device 100 estimates that the person detected by the detecting unit 102 requires caution processing, and then the person It is determined whether or not the person is registered in advance as a person who is permitted to enter inside the entrance to the specific area of the building, and if registered, the processing unit 119 is not caused to perform the warning process. may be caused to execute superior processing, and if not registered, the processing unit 119 may be caused to execute warning processing. Alternatively, the estimation unit 113 first determines whether the person detected by the detection unit 102 is registered in advance as a person who is permitted to enter the inside of the entrance to the specific area of the building. If it is registered, the processing unit 119 may be caused to execute superior processing without estimating whether or not caution processing is necessary. If not, it may be further estimated whether or not caution processing is necessary.

In any of the first to third embodiments described above, a person who is registered in advance in the management device 100 as a target person requiring alert processing is requested to present his or her identification card to the manager of the condominium 30, for example. As a condition, entry into the area 33 inside the entrance door 41 of the condominium 30 may be permitted. In addition, the management device 100 may exclude a wandering elderly person or a child from the target persons who need to be alerted. The management device 100 excludes the wandering elderly person from the target persons who need to be alerted, for example, when the wandering elderly person is a resident of the condominium 30 and is registered as having dementia or the like.

In any of the first to third embodiments described above, the detection unit 102 includes a camera that captures an image, and a person in the area 31 outside the entrance door 41 appears as a subject in the captured image data. The description has been given assuming that captured image data is transmitted to the apparatus main body 101 at any time regardless of whether or not it is. Alternatively, the detection unit 102 determines whether or not a person in the area 31 outside the entrance door 41 appears as a subject in the captured image data. Only the captured image data may be transmitted to the apparatus main body 101 . In this case, the detection unit 102 may further determine whether or not there are a plurality of people in the area 31 outside the entrance door 41, and transmit the determination result to the apparatus main body 101 together with the image data. In this case, in step S211 of the flow diagram of FIG. You may judge whether a person is reflected.

In any of the first to third embodiments described above, the estimating unit 113 has been described as continuing unsupervised learning after a certain period of supervised learning. In the unsupervised learning, it is preferable to update the images of the residents of the condominium 30 as an example of the labeled training data acquired during the supervised learning period as needed. Thereby, the estimation unit 113 can identify the same person even if the appearance of the residents of the condominium 30 changes with age.

Various embodiments of the invention may be described with reference to flowchart illustrations and block diagrams, where blocks refer to (1) steps in a process in which operations are performed or (2) devices responsible for performing the operations. may represent a section of Certain steps and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable medium, and/or processor provided with computer readable instructions stored on a computer readable medium. you can Dedicated circuitry may include digital and/or analog hardware circuitry, and may include integrated circuits (ICs) and/or discrete circuitry. Programmable circuits include logic AND, logic OR, logic XOR, logic NAND, logic NOR, and other logic operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), etc. and the like.

Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device, such that computer-readable media having instructions stored thereon may be designated in flowcharts or block diagrams. It will comprise an article of manufacture containing instructions that can be executed to create means for performing the operations described above. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), Electrically Erasable Programmable Read Only Memory (EEPROM), Static Random Access Memory (SRAM), Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD), Blu-ray (RTM) Disc, Memory Stick, Integration Circuit cards and the like may be included.

The computer readable instructions may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or object oriented programming such as Smalltalk, JAVA, C++, etc. language, and any combination of one or more programming languages, including conventional procedural programming languages, such as the "C" programming language or similar programming languages. good.

Computer readable instructions may be transferred to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN), such as a local area network (LAN), the Internet, or the like. ) and may be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

FIG. 7 illustrates an example computer 1200 in which aspects of the invention may be implemented in whole or in part. Programs installed on the computer 1200 cause the computer 1200 to function as one or more "parts" of operations or one or more "parts" of an apparatus according to embodiments of the invention, or to and/or cause computer 1200 to perform processes or steps of processes according to embodiments of the present invention. Such programs may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

Computer 1200 according to this embodiment includes CPU 1212 , RAM 1214 , graphics controller 1216 , and display device 1218 , which are interconnected by host controller 1210 . Computer 1200 also includes input/output units such as communication interface 1222 , hard disk drive 1224 , DVD-ROM drive 1226 , and IC card drive, which are connected to host controller 1210 via input/output controller 1220 . The computer also includes legacy input/output units such as ROM 1230 and keyboard 1242 , which are connected to input/output controller 1220 through input/output chip 1240 .

The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. Graphics controller 1216 takes image data generated by CPU 1212 into a frame buffer or the like provided in RAM 1214 or into itself, and causes the image data to be displayed on display device 1218 .

Communication interface 1222 communicates with other electronic devices over a network. Hard disk drive 1224 stores programs and data used by CPU 1212 within computer 1200 . DVD-ROM drive 1226 reads programs or data from DVD-ROM 1201 and provides programs or data to hard disk drive 1224 via RAM 1214 . The IC card drive reads programs and data from IC cards and/or writes programs and data to IC cards.

ROM 1230 stores therein programs that depend on the hardware of computer 1200, such as a boot program that is executed by computer 1200 upon activation. Input/output chip 1240 may also connect various input/output units to input/output controller 1220 via parallel ports, serial ports, keyboard ports, mouse ports, and the like.

A program is provided by a computer-readable storage medium such as a DVD-ROM 1201 or an IC card. The program is read from a computer-readable storage medium, installed in hard disk drive 1224 , RAM 1214 , or ROM 1230 , which are also examples of computer-readable storage medium, and executed by CPU 1212 . The information processing described within these programs is read by computer 1200 to provide coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing information operations or processing according to the use of computer 1200 .

For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded into the RAM 1214 and sends communication processing to the communication interface 1222 based on the processing described in the communication program. you can command. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214, the hard disk drive 1224, the DVD-ROM 1201, or an IC card. Data is transmitted to the network, or received data received from the network is written in a receive buffer area or the like provided on the recording medium.

In addition, the CPU 1212 causes the RAM 1214 to read all or necessary portions of files or databases stored in external recording media such as a hard disk drive 1224, a DVD-ROM drive 1226 (DVD-ROM 1201), an IC card, etc. Various types of processing may be performed on the data in RAM 1214 . CPU 1212 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium to be processed. CPU 1212 performs various types of operations on data read from RAM 1214, information processing, conditional decisions, conditional branching, unconditional branching, and information retrieval, which are described throughout this disclosure and are specified by instruction sequences of programs. Various types of processing may be performed, including /replace, etc., and the results written back to RAM 1214 . In addition, the CPU 1212 may search for information in a file in a recording medium, a database, or the like. For example, if multiple entries are stored in the recording medium, each entry having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 selects the first attribute from among the multiple entries. Searches for an entry that matches the specified condition of the attribute value of the attribute, reads the attribute value of the second attribute stored in the entry, and thereby associates it with the first attribute that satisfies the predetermined condition. an attribute value of the second attribute obtained.

Programs or software modules according to the above description may be stored in a computer readable storage medium on or near computer 1200 . Also, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the program can be transferred to the computer 1200 via a network. offer.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications and improvements can be made to the above embodiments. In addition, matters described with respect to a specific embodiment can be applied to other embodiments as long as they are not technically inconsistent. Also, each component may have features similar to other components with the same name but different reference numerals. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is particularly "before", "before etc., and it should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for the sake of convenience, it means that it is essential to carry out in this order. not a thing

30 condominium, 31 outside area, 33 inside area, 41 entrance door, 43 operation panel, 50 communication network, 61 first person, 63 second person, 65 third person, 100 management device, 101 device body, 102 detection Unit, 111 Communication Unit, 113 Estimation Unit, 117 Storage Unit, 119 Processing Unit, 1200 Computer, 1201 DVD-ROM, 1210 Host Controller, 1212 CPU, 1214 RAM, 1216 Graphic Controller, 1218 Display Device, 1220 Input/Output Controller, 1222 Communication interface, 1224 hard disk drive, 1226 DVD-ROM drive, 1230 ROM, 1240 input/output chip, 1242 keyboard

Claims (13)

a detector that detects a person outside an entrance to a specific area of a building;
Based on the learning result of the relationship between the person outside the entrance to the specific area in the building and the necessity of the specific process, the person outside the entrance detected by the detection unit determines the necessity of the specific process. an estimating unit for estimating
a processing unit that executes the specific processing when the estimating unit estimates that the specific processing is necessary,
The person outside the entrance includes an operator who performs an unlocking operation of the entrance and a person different from the operator, and the estimating unit includes an interpersonal relationship between the operator and the different person and the identifying person. learning the relationship between whether or not to process
The specific process includes a request process of requesting the operator to confirm the different person when an input of an unlocking operation of the entrance by the operator is received.
management device. The interpersonal relationship includes a relationship in which the operator is aware of entering the specific area with the different person, and a relationship in which the operator is not aware of entering the specific area.
The management device according to claim 1. The processing unit further performs different processing depending on whether the different person is registered as a person who is previously permitted to enter the specific area.
The management device according to claim 1 or 2. The processing unit further performs different processing depending on whether or not one or more persons whose residence, workplace, or school commute to the building has a history of bad behavior toward the different persons.
The management device according to any one of claims 1 to 3. The estimating unit learns the relationship between the interpersonal relationship between the operator and the different person and whether or not the specific process is necessary, based on the confirmation result of the different person by the operator.
The management device according to any one of claims 1 to 4. The specific processing includes caution processing of the entrance, and the estimation unit learns the relationship between the situation of a person outside the entrance and the necessity of the caution processing.
The management device according to any one of claims 1 to 5. The warning process includes at least one of locking the entrance, closing the entrance, issuing a warning at the entrance, and reporting to a pre-registered notification destination.
The management device according to claim 6. The estimating unit learns the situation of the person outside the entrance requiring the warning process from a record of the actions taken by the person registered as the target person requiring the warning process.
The management device according to claim 6 or 7. The estimating unit determines, as the situation of the person outside the entrance, that one or more persons who live, work, or go to school in the building behave badly toward the person outside the entrance. learning that the alert process is necessary if there is a history of
The management device according to any one of claims 6 to 8. The specific processing includes (A) unlocking of the entrance, (B) opening of the entrance, (C) release of warning of the entrance, and (D) unlocking of the notification destination registered in advance. communication for requesting remote control of at least one of opening the door and canceling the alarm, wherein the estimating unit detects a person outside the entrance and learning the relationship between the necessity of processing and
Management device according to any one of claims 1 to 9. A management system comprising a plurality of management devices according to any one of claims 1 to 10,
A person who lives, works, or goes to school in the building in which one of the plurality of management devices is installed is provided with another management device out of the plurality of management devices. sharing the learning result of the one management device with the other management device when the building is newly used as a residence, place of work, or school,
management system. A computer-implemented management method comprising:
a detection step of detecting a person outside an entrance to a particular area of a building;
Based on the learning result of the relationship between the person outside the entrance to the specific area in the building and the necessity of the specific process, the person outside the entrance detected in the detection step determines the necessity of the specific process. an estimation step of estimating
a processing step of performing the specific processing when the estimating step deduces that the specific processing is necessary,
The person outside the entrance includes an operator who performs an unlocking operation of the entrance and a person different from the operator, and the estimating step includes interpersonal relationships between the operator and the different person, and the identifying person. including learning the relationship between whether or not to process
The specific process includes a request process of requesting the operator to confirm the different person when an input of an unlocking operation of the entrance by the operator is received.
Management method. A program that causes a computer to execute the management method according to claim 12.

Images