JP2020014249A - Intercom database generation system, intercom indoor device, intercom system, generation method, and program - Google Patents

Abstract

To provide a technique of enhancing accuracy in classifying attributes of visitors while suppressing complication of processing.SOLUTION: A first acquisition section 70 acquires image data where a visitor is captured. A second acquisition section 74 acquires voice of a visitor captured in the image data. A classification section 76 classifies a visitor captured in the image data acquired by the first acquisition section 70 into one of multiple attributes regulated in accordance with reliability of visitors on the basis of at least one of an image recognition result of the image data acquired by the first acquisition section 70 and a voice recognition result of voice acquired by the second acquisition section 74. A storage execution section 82 allows a classification result by the classification section 76 to be stored in a database 36.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an intercom, and more particularly to an intercom database generation system, an intercom indoor unit, an intercom system, a generation method, and a program installed in an apartment or a detached dwelling unit.

  The intercom system automatically or manually responds to visitors to an apartment or house. The intercom outdoor unit in the intercom apparatus is provided with a camera, and by analyzing images captured by the camera, visitors are classified into types such as carriers. Further, the intercom apparatus selects and outputs response information to a visitor according to the classification result and the state of the occupant (for example, see Patent Document 1).

JP 2012-10200 A

  In order to classify a visitor's attributes, it is not sufficient to classify the image simply by analyzing an image captured by a camera. On the other hand, in order to improve the accuracy of classification, it is effective to generate a visitor database in advance, but when the database is generated by a resident's operation, processing by the resident becomes complicated.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide a technique for improving the classification accuracy of a visitor's attribute while suppressing complication of processing.

  In order to solve the above problems, a database generation system for an intercom according to an aspect of the present invention includes a first acquisition unit that acquires image data of a visitor, and a voice of the visitor that is imaged in the image data. The reliability of the visitor based on at least one of the second acquisition unit to be acquired, the image recognition result of the image data acquired by the first acquisition unit, and the speech recognition result of the speech acquired by the second acquisition unit. A classifying unit that classifies the visitor imaged in the image data acquired by the first acquiring unit into one of a plurality of attributes defined accordingly, and a storage execution unit that stores a result of the classification by the classifying unit in a database And.

  Another embodiment of the present invention relates to an intercom system. This intercom system includes an intercom outdoor device, a control device connected to the intercom outdoor device, and an intercom indoor device connected to the control device. A first acquisition unit that acquires image data of a visitor captured by one or more of an intercom outdoor device, a control device, and an intercom indoor device; and a visit that is captured by the image data acquired by the first acquisition unit. A visitor based on at least one of a second acquisition unit that acquires the voice of the visitor, an image recognition result of the image data acquired by the first acquisition unit, and a speech recognition result of the voice acquired by the second acquisition unit. A classifying unit that classifies a visitor imaged in the image data acquired by the first acquiring unit into one of a plurality of attributes defined according to the reliability of the classifying unit; and a result of the classification by the classifying unit stored in a database. And a storage execution unit for causing the storage to execute.

  Still another embodiment of the present invention relates to a production method. The method is a method of generating a database forming system for an intercom, comprising: acquiring image data of a visitor imaged; acquiring a voice of the visitor imaged in the image data; Based on at least one of the image recognition result and the voice recognition result of the voice, the visitor captured in the acquired image data is selected for one of a plurality of attributes defined according to the reliability of the visitor. Classifying; and storing the result of the classification in a database.

  It is to be noted that any combination of the above-described components and any conversion of the expression of the present invention between a method, an apparatus, a system, and the like are also effective as embodiments of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the classification precision of a visitor's attribute can be improved, suppressing complexity of a process.

FIG. 1 is a diagram illustrating a configuration of an intercom system according to a first embodiment of the present invention. FIGS. 2A and 2B are diagrams illustrating a data structure of a table held in the classification unit in FIG. FIG. 2 is a diagram illustrating a data structure of a database in FIG. 1. FIGS. 4A and 4B are diagrams showing images displayed on the display unit in FIG. 2 is a flowchart illustrating a classification procedure performed by a generation unit in FIG. 1. 2 is a flowchart illustrating a storage procedure by a generation unit in FIG. 1. 5 is a flowchart illustrating another storage procedure performed by the generation unit in FIG. 1. 2 is a flowchart illustrating a processing procedure by the intercom indoor unit in FIG. 1. FIG. 6 is a diagram illustrating a configuration of an intercom system according to a second embodiment of the present invention. FIG. 9 is a diagram illustrating a configuration of an intercom system according to a third embodiment of the present invention.

(Example 1)
Before describing the present invention in detail, an overview will be given. Embodiment 1 Embodiment 1 of the present invention relates to an intercom system installed in an apartment or a detached dwelling unit. In an intercom system, an intercom indoor device is installed in a dwelling unit, and an intercom outdoor device is installed in a lobby of a condominium or a detached entrance, and they are connected by a signal line or the like. When the visitor presses the button on the intercom outdoor unit, the intercom system outputs a ringing tone, so that the intercom system notifies the resident of the visitor's visit. The visitors include family members, friends, carriers, salesmen, and the like, and the content of the resident's response differs depending on the type of the visitor. In the present embodiment, the attribute of the visitor is specified, and a process corresponding to the attribute is executed in order to eliminate unnecessary and unpleasant trouble of the resident involved in the reception. For example, the ringing tone is changed according to the attribute, or the ringing is not performed. In addition, attribute information is superimposed on image data of a visitor and displayed. Also, use of the delivery box is permitted only to visitors included in a specific attribute.

  Here, in order to identify the attribute of the visitor, a database indicating the correspondence between the image data and the attribute of the visitor is provided, and the image data and the database obtained by capturing the newly visited visitor are provided. Are compared. A database with high accuracy generally needs to be generated by a resident, and is premised on resident's operation input. In this embodiment, in order to reduce the burden on the occupant, the attributes of the visitor are automatically classified from the past response results, and the classification result is stored in the database together with the image data of the visitor. For example, if the response result is "a resident came in front of the intercom indoor unit due to a ringing tone but did not respond", the attribute of the visitor is a person who does not need to respond. If the response result is "the resident responded but did not unlock", the visitor attribute is also a person who does not need to respond. If the response result is “the resident responds and the visitor has a keyword such as“ now ”and is unlocked”, the attribute of the visitor is family. In addition, if the response result is “there is a response from the resident, and there is a keyword such as“ delivery ”or“ XX transportation ”from the visitor and it is unlocked”, the attribute of this visitor is the carrier. is there.

  FIG. 1 shows a configuration of an intercom system 100 according to a first embodiment of the present invention. The intercom system 100 includes an intercom outdoor device 10, a control device 12, and an intercom indoor device 14. The intercom outdoor device 10 includes a call operation unit 20, a microphone 22, a speaker 24, an outdoor audio processing unit 26, an imaging device 30, an image processing unit 32, an identification processing unit 34, a database 36, an unlock processing unit 38, and an outdoor use. IF42. The intercom outdoor unit 10 is connected to the door 40. The intercom indoor unit 14 includes an indoor IF 50, an indoor audio processing unit 52, a microphone 54, a speaker 56, a display unit 58, a generation unit 60, a response unit 62, and an unlock operation unit 64. The generation unit 60 includes a first acquisition unit 70, an extraction unit 72, a second acquisition unit 74, a classification unit 76, a sensor 78, a third acquisition unit 80, and a storage execution unit 82, and the storage execution unit 82 includes a confirmation unit 84 including. The response unit 62 includes a specifying unit 90 and a processing unit 92.

  The intercom system 100 is installed in an apartment, for example. The intercom outdoor device 10 is installed in a lobby of a condominium and is also called a lobby intercom. In addition, the intercom outdoor device 10 may be a door intercom installed at the entrance of a dwelling unit, but a description of a case where the intercom outdoor device 10 is a door intercom will be omitted below. The intercom indoor unit 14 is installed in a dwelling unit and is also called a dwelling unit. The control device 12 is installed in a common space such as a machine room or a management room, and relays a signal between the intercom outdoor device 10 and the intercom indoor device 14.

  FIG. 1 shows only one intercom indoor unit 14 installed in one dwelling unit for simplicity of explanation. However, there are many dwelling units in the apartment, and the intercom indoor unit 14 is installed in each dwelling unit. Each of the plurality of intercom indoor units 14 is connected to the control device 12 and connected to the intercom outdoor device 10 via the control device 12. Therefore, the control device 12 relays between the plurality of intercom indoor devices 14 and the intercom outdoor device 10. In such a configuration, the visitor inputs the room number to the intercom outdoor device 10 and causes the intercom indoor device 14 of the visit destination to make a call.

  In the following, in order to clarify the description, the configuration of the intercom system 100 is described as (1) basic processing between the intercom outdoor apparatus 10 and the intercom indoor apparatus 14, (2) generation processing for generating a database, (3) ) A description will be given separately of a response process for responding to a visitor using a database. Actually, when a new visitor comes, both the generation processing and the response processing are executed.

(1) Basic Processing The call operation unit 20 of the intercom outdoor device 10 is a user interface for receiving a visitor's operation. Specifically, a ten-key button for inputting a room number and a call button are included. The call operation unit 20 outputs a call signal corresponding to the received operation to the outdoor IF 42. At the timing when the calling operation unit 20 outputs the calling signal, the microphone 22, the speaker 24, the outdoor sound processing unit 26, the imaging device 30, the image processing unit 32, and the like described later start processing. Therefore, the operation of the call operation unit 20 by the visitor is a trigger for starting a series of processes described below.

  The microphone 22 and the speaker 24 are user interfaces for a visitor to talk with a resident. The microphone 22 collects the voice of the visitor, converts the voice into an electric signal, and outputs the electric signal to the outdoor sound processing unit 26. The speaker 24 reproduces an electric signal from the outdoor sound processing unit 26 and outputs sound. The outdoor audio processing unit 26 receives an electric signal from the microphone 22 and converts the electric signal into a signal (hereinafter, referred to as an “audio signal”) for transmission to the intercom indoor unit 14. The outdoor sound processor 26 outputs a sound signal to the outdoor IF 42. On the other hand, the outdoor audio processing unit 26 receives the audio signal generated in the intercom indoor unit 14 from the outdoor IF 42 and converts the audio signal into an electric signal. The outdoor sound processor 26 outputs an electric signal to the speaker 24.

  The imaging device 30 is for photographing a visitor, and includes a solid-state imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor and a CCD (Charge Coupled Device) image sensor. The solid-state imaging device outputs the photoelectrically converted video signal to the image processing unit 32. The image processing unit 32 has a frame memory for storing a video signal from the imaging device 30. The image processing unit 32 performs various signal processing on the video signal stored in the frame memory. Various signal processes include, for example, an exposure control process for controlling the exposure of the video signal, a color balance control process for controlling the color balance of the video signal, a tone characteristic control process for controlling the tone characteristics of the video signal, and a video signal An outline emphasis control process for controlling the degree of outline emphasis is included. The image processing unit 32 outputs the video signal that has been subjected to the signal processing (hereinafter, referred to as “image data”) to the identification processing unit 34 and the outdoor IF 42.

  The door 40 is provided in the lobby and has a key (not shown). The key is electronically lockable / unlockable, and usually the key is locked. The unlock processing unit 38 unlocks the key of the door 40 when receiving the unlock signal generated in the intercom indoor unit 14 from the outdoor IF 42. The unlocked key is locked after a certain period of time. When the key portion of the door 40 is unlocked by the visitor, the door 40 notifies the unlocking processing unit 38 that the door has been unlocked. Upon being notified of the unlocking, the unlocking processing section 38 outputs an unlocking signal to the outdoor IF 42. The outdoor IF 42 is a communication interface for communicating a call signal, a voice signal, image data, and an unlock signal with the control device 12. The call signal and the image data are one-way communication from the outdoor IF 42 to the control device 12, and the voice signal and the unlock signal are two-way communication between the outdoor IF 42 and the control device 12.

  The indoor IF 50 of the intercom indoor unit 14 is a communication interface for communicating with the control device 12. The indoor IF 50 executes a process corresponding to the outdoor IF 42. Therefore, the call signal and the image data are one-way communication from the control device 12 to the indoor IF 50, and the voice signal and the unlock signal are two-way communication between the indoor IF 50 and the control device 12. The display unit 58 receives image data from the indoor IF 50, reproduces the image data, and displays an image, particularly an image of a visitor.

  The microphone 54 and the speaker 56 are user interfaces for a resident to talk with a visitor. The microphone 54 and the speaker 56 perform the same processing as the microphone 22 and the speaker 24. The indoor audio processing unit 52 executes a process corresponding to the outdoor audio processing unit 26, converts an electric signal received from the microphone 54 into an audio signal, outputs the audio signal to the indoor IF 50, and outputs the audio signal from the indoor IF 50. The received audio signal is converted into an electric signal and output to the speaker 56. The unlocking operation section 64 is a user interface for receiving an operation by a resident. Specifically, an unlock button and a call button are included.

  With such a configuration, the resident hears the ringing sound output from the speaker 56 of the intercom indoor unit 14, looks at the image of the visitor displayed on the display unit 58, and determines whether to respond. Here, the resident may talk with the visitor while using the microphone 54 and the speaker 56 of the intercom indoor unit 14. When the visitor is invited to the dwelling unit as a response, the resident presses the unlocking operation unit 64 to unlock the key of the door 40. The (2) generation processing and (3) response processing at that time will be described below.

(2) Generation Processing The identification processing unit 34 receives image data from the image processing unit 32. The identification processing unit 34 performs a face image extraction process of extracting a face image (face region) of a person from image data, and a feature amount of a face of the person (for example, a face organ such as an eye, a nose, and a mouth) from the extracted face image. A feature amount extraction process for extracting information corresponding to the feature of the shape) is executed. The identification processing unit 34 outputs the image data (hereinafter, also referred to as “image data”) on which these processes have been performed to the outdoor IF 42. The image data is input to the first acquisition unit 70 via the control device 12 and the indoor IF 50.

  The first obtaining unit 70 obtains image data of a visitor imaged and image data from the identification processing unit 34. The first obtaining unit 70 outputs the image data to the extracting unit 72, the classifying unit 76, and the specifying unit 90. The second acquisition unit 74 acquires a response result of the resident to the visitor captured in the image data acquired by the first acquisition unit 70. The response result is information on the result of (1) the response indicated in the basic process and (3) the response indicated by the response process performed by the resident. The response result is output from the unlocking operation unit 64 and the indoor voice processing unit 52. More specifically, the response result from the unlocking operation unit 64 indicates whether the unlocking operation has been performed or the unlocking operation has not been performed. On the other hand, the response result from the indoor voice processing unit 52 indicates whether the call operation with the visitor has been executed or the call operation with the visitor has not been executed. If the call operation has been performed, a voice signal of the call content is included in the response result. The second acquisition unit 74 outputs the response result to the extraction unit 72 and the classification unit 76.

  The extraction unit 72 inputs the image data acquired by the first acquisition unit 70 and the response result acquired by the second acquisition unit 74. The extraction unit 72 extracts a keyword related to a visitor from at least one of the image data and the response result. More specifically, the extraction unit 72 extracts a portion suitable for identifying a trader as a partial image from the image region of a person included in the image data. The locations that are suitable for identifying the trader are the emblem that displays the trader's company name, logo, etc. attached to the visitor's chest, arms, hat, etc., the luggage brought by the visitor, the visitor Face. In addition, the extraction unit 72 classifies data for identifying which trader the visitor is (hereinafter, referred to as “identification data”) for each trader and holds the data. The identification data is classified according to the type of business such as a newspaper or a carrier, for example. Here, for clarity of description, it is assumed that the identification data is an image of an emblem for identifying the carrier. The extraction unit 72 collates the partial image with the identification data to determine whether the visitor is a carrier. If the carrier is a carrier, the extraction unit 72 outputs the carrier to the classification unit 76 as a keyword.

  In parallel with such processing, the extraction unit 72 converts the voice signal into character information by executing voice recognition processing on the voice signal of the call content. In addition, the extraction unit 72 previously stores a character string for identifying a trader, a family, and the like (hereinafter, referred to as an “identification character string”). For example, the identification string for identifying the family is "Now" or "Return", and the identification string for identifying the trader, especially the carrier, is "Delivery" or "XX Transport". And so on. Note that the identification character string may be used to identify a person other than a trader and a family. The extraction unit 72 determines whether the visitor is a family or a carrier by referring to the character information and the identification character string. The extraction unit 72 outputs the family to the classification unit 76 as a keyword when the family is a family, and outputs the carrier to the classification unit 76 as a keyword when the family is a carrier.

  The sensor 78 is a human sensor for detecting whether or not a resident is present in front of the intercom indoor unit 14 after a call signal is output from the call operation unit 20. A known technique may be used for the sensor 78, and the sensing area is set so that a resident existing in front of the intercom indoor unit 14 can be detected. Note that an imaging device is provided instead of the sensor 78, and whether or not the resident is present in front of the intercom indoor unit 14 is determined depending on whether or not the image data captured by the imaging device includes the resident. May be detected. Here, the case where the resident is present in front of the intercom indoor unit 14 corresponds to the case where the resident is available, and the case where the resident is not present in front of the intercom indoor device 14 is that the resident is not available. This corresponds to the case where it is possible. The sensor 78 outputs information (hereinafter, referred to as “possible information”) regarding whether or not the resident is available to the third acquisition unit 80.

  The third obtaining unit 80 obtains possible information from the sensor 78. The third obtaining unit 80 outputs the possible information to the classifying unit 76. In addition, when the home mode and the absence mode can be set for the intercom indoor unit 14, the third obtaining unit 80 may obtain the setting information of the home mode or the absence mode. Here, the home mode is a processing mode of the intercom indoor unit 14 on the assumption that there is a resident in the dwelling unit, and the absence mode is a processing mode of the intercom indoor unit 14 on the assumption that there is no resident in the dwelling unit. 14 processing modes. In that case, the third acquisition unit 80 outputs the setting information and the possible information to the classification unit 76 in the home mode. On the other hand, in the absence mode, the third acquisition unit 80 outputs only the setting information to the classification unit 76.

  When the indoor IF 50 receives the unlock signal from the control device 12, the indoor IF 50 outputs the unlock signal to the classification unit 76. This corresponds to the case where the visitor himself unlocks, and the occurrence of such a situation is notified to the classification unit 76.

  The classification unit 76 receives the response result from the second acquisition unit 74, the image data from the first acquisition unit 70, the keyword from the extraction unit 72, the possible information from the third acquisition unit 80, and the unlock signal from the indoor IF 50. Accept. Based on these, the classification unit 76 classifies the visitor imaged in the image data acquired by the first acquisition unit 70 into one of a plurality of attributes defined according to the reliability of the visitor. . Visitor reliability indicates the reliability of a visitor to a resident. For example, if the reliability is defined in three stages of high, medium, and low, the reliability of family / friends is “high”, the reliability of the carrier is “medium”, and the Sex is "low". Note that the reliability may be defined in more than three stages, for example, four stages, and the reliability of family and friends may be separated. In that case, the trust of the family is defined to be higher than the trust of the friends. In addition, reliability may be defined for persons other than family members, friends, carriers, and unneeded persons. Here, a plurality of attributes are defined according to the reliability. Further, the reliability may be defined in two stages. When the reliability is defined in three stages, an attribute corresponding to high reliability, an attribute corresponding to medium reliability, and an attribute corresponding to low reliability are defined. Note that an attribute corresponding to high reliability may be simply referred to as “high reliability”. The same applies to attributes corresponding to medium reliability and attributes corresponding to low reliability.

  FIGS. 2A and 2B show a data structure of a table held in the classifying unit 76. In FIG. 2A, "unlocking operation available / not available" and "communication operation available / unavailable" as corresponding information, a resident state as a keyword, possible information, and "self-unlocking / unlocking" as an unlocking signal. The attribute to be selected according to "no lock" is shown. The classification unit 76 selects one of the attributes based on the received information while referring to the table.

  First, a case where the response result indicates that the unlocking operation has not been performed will be described. It is assumed that there is no unlocking. In such a situation, when the response result also indicates that the call operation with the visitor has not been executed, the classification unit 76 determines the classification according to the possible information. In other words, when the unlocking operation is not executed and the call operation is not executed, if the resident can respond, the classifying unit 76 classifies the visitor into an attribute corresponding to low reliability. On the other hand, when the unlocking operation is not executed and the call operation is not executed, if the resident cannot respond, the classifying unit 76 suspends the classification of the visitor. At this time, if a transportation-related keyword has been acquired, the classification unit 76 may classify the visitor into an attribute corresponding to the medium reliability. Whether it is classified into the attribute corresponding to the medium reliability or is reserved depends on the resident's presetting or ex post confirmation.

  Further, in the above situation, if the response result also indicates that the call operation with the visitor has been executed, the classifying unit 76 classifies the visitor into an attribute corresponding to low reliability. At this time, if a transportation-related keyword has been acquired, the classification unit 76 may classify the visitor into an attribute corresponding to the medium reliability. This corresponds to the case where the keyword includes “delivery box”, “transport company name”, “delivery item”, or the like, or when the delivery box is used immediately thereafter. On the other hand, even if the transportation-related keyword is acquired, if the classification into the attribute corresponding to the low reliability is performed, the classification unit 76 regards that the transportation-related keyword is erroneously recognized.

  On the other hand, even if the response result indicates that the unlocking operation has not been performed, if the unlocking operation is performed by itself, the classification unit 76 visits the attribute (family) corresponding to high reliability. Classify people. As described above, self-unlocking is generally feasible only by family members.

  Next, a case where the response result indicates that the unlocking operation has been executed will be described. In such a situation, if the response result also indicates that the call operation with the visitor has not been executed, the classifying unit 76 classifies the visitor into an attribute corresponding to high reliability. As described above, the high reliability includes a family and a friend, but if registered as a family, the classification unit 76 classifies the visitor toward the family among the attributes corresponding to the high reliability. I do. In addition, if it is indicated that the unlocking operation has been performed and the call operation with the visitor has not been performed, the keyword cannot be extracted, but a “transportation-related” keyword is acquired. If this is the case, the classification unit 76 regards that the transportation-related keyword has been erroneously recognized.

  Further, in the above-described situation, if the response result also indicates that the call operation with the visitor has been performed, the classification unit 76 determines the classification according to the keyword. The classification unit 76 classifies the visitor into an attribute (family) corresponding to high reliability when the keyword is a greeting from the family, and visits the attribute corresponding to medium reliability when the keyword is related to transportation. Classify people. On the other hand, when the keyword is not one of these, the classification unit 76 classifies the visitor into an attribute (friend) corresponding to high reliability.

  FIG. 2B shows the state of the indoor device instead of the state of the occupant in FIG. 2A. The state of the indoor device is specified by “absence mode / home mode” as setting information and “no access / presence” as possible information. Since the classification process in the classification unit 76 is the same as that described above, the description is omitted here. Return to FIG.

  The storage execution unit 82 receives the result of the classification by the classification unit 76. The classification result includes, for example, image data and attribute information. The storage execution unit 82 causes the database 36 to store the classification result via the indoor IF 50, the control device 12, and the outdoor IF 42. FIG. 3 shows the data structure of the database 36. As illustrated, an image data column 200, a name column 202, and an attribute column 204 are included. The image data column 200 shows image data. Since predetermined processing is performed on the image data in the identification processing unit 34, the image data column 200 may include the result of the predetermined processing. The attribute column 204 contains attribute information. The attribute information is shown in the same manner as in FIGS. The name column 202 shows the name of the visitor. The name may be registered by post confirmation, or may be omitted. Return to FIG.

  Among the information stored in the database 36, the visitors included in the attributes (family) and (friends) corresponding to high reliability differ for each dwelling unit. On the other hand, the visitors included in the attributes corresponding to the medium reliability and the attributes corresponding to the low reliability are common to all the dwelling units. Therefore, in the database 36, at least one of the attribute of the attribute corresponding to the medium reliability and the information of the attribute corresponding to the low reliability is shared among the residents living in each of the plurality of dwelling units.

  As described above, the storage execution unit 82 automatically stores the result of the classification in the classification unit 76 in the database 36. However, in order to further improve the classification accuracy of the information stored in the database 36, the following manual processing is performed. May be additionally executed. Here, two additional processes will be described. In the first additional process, when receiving the result of the classification in the classification unit 76, the storage execution unit 82 checks whether the classified visitor is already stored in the database 36. If it is stored, the same processing as before is performed. In this case, since it is not necessary to change the name and attribute stored in the database 36, new image data is additionally stored, or new image data is stored so as to be replaced with image data already stored. Or On the other hand, if there is no memory, the confirmation unit 84 confirms the result of the classification with the resident. More specifically, the confirmation unit 84 causes the display unit 58 to display the classification result via a signal line (not shown).

  FIGS. 4A and 4B show images displayed on the display unit 58. FIG. 4A is an image displayed when the classified visitor is not stored in the database 36. As shown, the image data, name, and attribute of the visitor are shown. The display unit 58 displays a Yes button 110 and a Modify button 112. When the display unit 58 is configured by a touch panel, when the resident presses the Yes button 110, the storage execution unit 82 causes the database 36 to store the result of the classification. On the other hand, when the resident presses the correction button 112, the result of the classification can be corrected. At that time, a name may be input. When the correction is completed, the storage execution unit 82 stores the result of the classification in the database 36. As described above, the storage execution unit 82 causes the database 36 to store the classification result confirmed by the confirmation unit 84. Return to FIG.

  In the second additional process, when the storage execution unit 82 receives the classification result in the classification unit 76, the storage execution unit 82 compares the classification result stored in the database 36 with the classification result for the same visitor. Confirm. If they match, the same processing as before is performed. On the other hand, if different, the confirmation unit 84 confirms with the resident whether to select one of the classification results. More specifically, the confirmation unit 84 causes the display unit 58 to display the classification result via a signal line (not shown).

  FIG. 4B is an image displayed when the classification results do not match. As shown in the figure, the result of the newly received classification is shown as “A” together with the image data. The result of the classification stored in the database 36 is indicated as "B". Further, on the display unit 58, an A selection button 114 and a B selection button 116 are displayed. When the display unit 58 is configured by a touch panel, when the resident presses the A selection button 114, the storage execution unit 82 causes the database 36 to store the newly accepted classification result. On the other hand, when the resident presses the B selection button 116, the storage execution unit 82 does not store the newly accepted classification result in the database 36, but keeps the classification result stored in the database 36 as it is. As described above, the storage execution unit 82 causes the database 36 to store the result of the classification selected by the confirmation by the confirmation unit 84. Return to FIG.

(3) Response Processing The specifying unit 90 receives the image data acquired by the first acquiring unit 70. The identification unit 90 refers to the database 36 via the indoor IF 50, the control device 12, and the outdoor IF 42. The specifying unit 90 compares the plurality of image data included in the database 36 with the image data received from the first acquisition unit 70. A known technique may be used for the image data comparison processing, and a description thereof will be omitted. The specifying unit 90 converts the image data in which the difference between the comparison results is included in the predetermined range and the image data included in the database 36 as the image data that matches the image data received from the first acquisition unit 70. select. The specifying unit 90 extracts the attribute of the visitor corresponding to the selected image data from the database 36. This corresponds to specifying the attribute of the visitor captured in the image data acquired by the first acquisition unit 70. On the other hand, when there is no image data to be selected, the specifying unit 90 specifies that there is no attribute including a visitor. The specifying unit 90 outputs the specified attribute to the processing unit 92.

  The processing unit 92 receives attribute information from the specifying unit 90. The processing unit 92 executes a process for the resident according to the received attribute. For example, the processing unit 92 changes the type of the ring tone output from the speaker 56 of the intercom indoor unit 14 according to the attribute. Note that the attribute here includes a case where there is no attribute including a visitor. Further, when the attribute corresponds to the low reliability, the processing unit 92 does not have to output the calling sound from the speaker 56 of the intercom indoor unit 14 as a sound. Further, the processing unit 92 may cause the display unit 58 to display the attribute information together with the image data. In addition, if the attribute corresponds to low reliability, the processing unit 92 refuses a visit such as “I cannot make an order due to circumstances. If you use it, a manager…” from the speaker 24 of the intercom outdoor unit 10. May be output by voice. Further, when the attribute corresponds to the medium reliability, the processing unit 92 may output a message to the effect that the use of the home delivery box is permitted from the speaker 24 of the intercom outdoor device 10 by voice.

  This configuration can be realized in terms of hardware by a CPU, a memory, or other LSI of an arbitrary computer, and is realized in software by a program or the like loaded in the memory. The functional blocks are drawn. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware alone or a combination of hardware and software.

  The program may be stored in a memory of a computer in advance, or may be provided through an electric communication line (not shown) or recorded on a removable recording medium.

  The operation of the intercom system 100 having the above configuration will be described. FIG. 5 is a flowchart illustrating a classification procedure performed by the generation unit 60. The first acquisition unit 70 acquires image data (S10). The second acquisition unit 74 acquires the response result (S12). If there is no unlocking operation (Y in S14) and the unlocking has been performed by itself (Y in S16), the classification unit 76 classifies the visitor into an attribute corresponding to high reliability (S18). If the self-unlock has not been performed (N in S16), there is no call operation (Y in S20), and if it is possible to respond (Y in S22), the classifying unit 76 sets the visitor to an attribute corresponding to low reliability. Are classified (S24). If not available (N in S22), the classification unit 76 suspends the classification (S26). If there is a call operation (N in S20), the classification unit 76 classifies the visitor into an attribute corresponding to low reliability (S28).

  If there is an unlocking operation (N in S14) and if there is no call operation (Y in S30), the classification unit 76 classifies the visitor into an attribute corresponding to high reliability (S32). If there is a call operation (N in S30) and the keyword is related to transportation (Y in S34), the classification unit 76 classifies the visitor into an attribute corresponding to the medium reliability (S36). If the keyword is not related to transportation (N in S34), the classification unit 76 classifies the visitor into an attribute corresponding to high reliability (S38).

  FIG. 6 is a flowchart illustrating a storage procedure performed by the generation unit 60. If the visitor is not stored in the database 36 (Y in S50), the confirmation unit 84 causes the display unit 58 to display a confirmation screen (S52). If the resident selects “Yes” (Y in S54), the storage execution unit 82 stores the classification result in the database 36 (S56). If the resident does not select “Yes” (N in S54), the storage execution unit 82 stores the corrected classification result in the database 36 (S58). If the visitor has been stored in the database 36 (N in S50), steps 52 and 54 are skipped, and the storage execution unit 82 stores the result of the classification in the database 36 (S56).

  FIG. 7 is a flowchart illustrating another storage procedure performed by the generation unit 60. The storage execution unit 82 compares the result of the new classification with the result of the past classification (S70). If they match (Y in S72), the storage execution unit 82 stores the result of the classification (S74). If they do not match (N in S72), the confirmation unit 84 displays a selection screen on the display unit 58 (S76). The storage execution unit 82 stores the one selected by the resident in the database 36 (S78).

  FIG. 8 is a flowchart showing a processing procedure by the intercom indoor unit 14. The first acquisition unit 70 acquires image data (S90). The specifying unit 90 specifies the attribute based on the image data while referring to the database 36 (S92). The processing unit 92 executes a process according to the specified attribute (S94).

  According to the embodiment of the present invention, a visitor is automatically classified into any one of a plurality of attributes based on a response result of a resident. Classification accuracy can be improved. In addition, since the response result of the resident is used as the classification criterion, the intention of the resident can be reflected in the classification. In addition, if the unlocking operation has not been executed, the visitor is classified into attributes corresponding to low reliability, so that the occupant should be alerted to the next visitor who did not unlock. Can be. Further, if the unlocking operation and the call operation are not executed even though the resident is available, the visitor is classified into the attribute corresponding to the low reliability, so that the classification accuracy can be improved. Further, if the unlocking operation and the call operation are not executed when the resident is not available, the classification is suspended, so that the execution of incorrect processing can be avoided. Also, if the unlocking operation has not been performed even though the call operation has been performed, the visitor is classified into attributes corresponding to low reliability, so that visitors who did not need to be unlocked can be accurately detected. Can be classified.

  In addition, if the unlocking operation is performed even though the call operation has not been performed, the visitor is classified into attributes corresponding to high reliability, so that a high reliability enough to unlock without calling. It is possible to classify visitors who have specificity with high accuracy. Further, if the call operation and the unlock operation are performed, the classification is performed according to the keyword, so that the accuracy of the classification can be improved. In addition, since the attribute corresponding to the medium reliability and the attribute corresponding to the low reliability in the database are shared among a plurality of residents, the result of the classification can be effectively used.

  Further, when the result of the classification is not stored in the database, the resident is made to check the result of the classification, and then the result of the classification is stored in the database. Therefore, the accuracy of the information stored in the database can be improved. Also, when the result of the new classification is different from the result of the classification already stored in the database, the resident is allowed to select the result of one of the classifications, and then the storage in the database is executed. The accuracy of the received information can be improved. In addition, visitors can be classified without any special operation, and the ringtone and display of the main unit and the automatic response to visitors are optimized according to the attributes, improving convenience and safety. it can.

  The outline of one embodiment of the present invention is as follows. The generation unit 60 according to an aspect of the present invention includes a first acquisition unit 70 that acquires image data of a visitor, and a resident for a visitor that is imaged with the image data acquired by the first acquisition unit 70. A second obtaining unit that obtains a response result; and a first obtaining unit that assigns one of a plurality of attributes defined according to the reliability of the visitor based on the response result obtained by the second obtaining unit to the first obtaining unit. A classification unit 76 for classifying visitors imaged in the image data acquired at 70, and a storage execution unit 82 for storing a result of the classification by the classification unit 76 in a database.

  If the visitor classified by the classification unit 76 is not stored in the database, a confirmation unit 84 for confirming the classification result to the resident may be further provided. The storage execution unit 82 may store the classification result confirmed by the confirmation unit 84 in a database.

  If the result of the classification performed by the classification unit 76 is different from the result of the classification stored in the database, the classification unit 76 may further include a confirmation unit 84 that confirms to the resident whether to select one of the classification results. The storage execution unit 82 may store the classification result selected by the check in the check unit 84 in the database.

  The response result acquired by the second acquisition unit 74 indicates that the unlocking operation has not been executed, and the classification unit 76 defines that the reliability is at least high and low when the unlocking operation is not executed. Visitors may be classified into attributes that correspond to low reliability among the attributes.

  A third acquisition unit 80 that acquires information on whether or not the resident is available may be further provided. The response result acquired by the second acquisition unit 74 indicates that the call operation with the visitor has not been executed, and the classification unit 76 has determined that the unlock operation has not been executed and the call operation has not been executed. In this case, if the third acquisition unit 80 acquires information on the availability of the resident, the visitor is classified into an attribute corresponding to low reliability, and the classification unit 76 determines that the unlock operation has not been performed. In the case where the execution is performed and the call operation is not performed, the classification may be suspended if the third acquisition unit 80 acquires information regarding that the resident is not available.

  The response result acquired by the second acquisition unit 74 indicates that the call operation with the visitor has been executed, and the classification unit 76 has performed the unlock operation and have executed the call operation. In this case, visitors may be classified into attributes corresponding to low reliability.

  The response result obtained by the second obtaining unit 74 indicates that the unlocking operation has been performed and the call operation with the visitor has not been performed or has been performed. If the call has been executed and the call operation has not been executed, the visitor is classified into the attribute corresponding to the high reliability among the reliability defined as at least high, medium, and low, and the classification unit 76 In the case where the unlocking operation has been executed and the telephone call operation has been executed, the visitor may be classified into an attribute corresponding to high reliability or medium reliability.

  An extraction unit 72 that extracts a keyword related to a visitor from at least one of the image data acquired by the first acquisition unit 70 and the response result acquired by the second acquisition unit 74 may be further provided. When the unlocking operation has been executed and the call operation has been executed, the classifying unit 76 visits the attribute corresponding to high reliability or medium reliability according to the keyword extracted by the extracting unit 72. May be classified.

  Part of the information in the database in which the result of the classification is stored by the storage execution unit 82 may be shared among the residents who live in each of the plurality of dwelling units.

  By referring to the database generated by the generation unit 60, the identification unit 90 that identifies the attribute of the visitor imaged in the image data acquired by the first acquisition unit 70 and the attribute identified by the identification unit 90 according to the attribute , A processing unit 92 that performs processing on the resident.

  Another aspect of the present invention is intercom system 100. The intercom system 100 includes an intercom outdoor device 10, a control device 12 connected to the intercom outdoor device 10, and an intercom indoor device 14 connected to the control device 12. A first acquisition unit 70 that acquires image data of a visitor captured by one or more of the intercom outdoor device 10, the control device 12, and the intercom indoor device 14, and an image data acquired by the first acquisition unit 70. A second acquisition unit 74 for acquiring a response result of the resident to the visitor being imaged, and a plurality of reception units defined in accordance with the reliability of the visitor based on the response result acquired by the second acquisition unit 74 A classification unit 76 for classifying a visitor imaged in the image data acquired by the first acquisition unit 70 in one of the attributes, and a storage execution unit 82 for storing a result of the classification in the classification unit 76 in a database. Be provided.

  Still another embodiment of the present invention relates to a method for generating a database for an intercom. The method comprises the steps of: acquiring image data of an image of a visitor; acquiring a response result of a resident to the visitor imaged by the acquired image data; and The method includes a step of classifying a visitor imaged in the acquired image data into one of a plurality of attributes defined according to the reliability of the visitor, and a step of storing a result of the classification in a database.

  Still another preferred embodiment according to the present invention relates to a program. The program includes a step of acquiring image data of a visitor imaged, a step of acquiring a resident's response result to the visitor imaged in the acquired image data, and The method includes a step of classifying a visitor imaged in the acquired image data into one of a plurality of attributes defined according to the reliability of the visitor, and a step of storing a result of the classification in a database.

(Example 2)
Until now, the intercom system 100 has been installed in an apartment house such as an apartment, and the generation unit 60 and the response unit 62 are included in the intercom indoor unit 14. Since the generation unit 60 and the response unit 62 are included in the intercom indoor unit 14, it is not necessary to transmit the occupant approach information by the sensor 78. On the other hand, the generation unit 60 and the response unit 62 can be included in the intercom outdoor device 10, the control device 12, and the cloud. Here, an example in which the generation unit 60 and the response unit 62 are included in the intercom outdoor device 10 will be described.

  FIG. 9 shows a configuration of an intercom system 100 according to the second embodiment of the present invention. The intercom outdoor device 10 includes a calling operation unit 20, a microphone 22, a speaker 24, an outdoor voice processing unit 26, an imaging device 30, an image processing unit 32, an identification processing unit 34, a database 36, an unlock processing unit 38, and image compression. And an audio compression / decompression unit 152 and a generation / response unit 154. The control device 12 includes an indoor IF 50, an indoor audio processing unit 52, a microphone 54, a speaker 56, a display unit 58, an unlock operation unit 64, a sensor 78, an image decompression unit 160, a voice compression / decompression unit 162, and a call operation unit. 164. Further, the intercom outdoor device 10 is connected to the door 40. Components indicated by the same reference numerals as those used so far perform the same processing as before. Here, the description will focus on the differences from the past.

  The image compression section 150 receives the image data from the image processing section 32 and performs an image compression process on the image data. Since a known technique may be used for the image compression processing, the description is omitted here. The image compression section 150 outputs the image data that has been subjected to the image compression processing (hereinafter also referred to as “image data”) to the outdoor IF 42.

  The audio compression / decompression unit 152 receives an audio signal from the outdoor audio processing unit 26 and performs an audio compression process on the audio signal. Since a known technique may be used for the audio compression processing, the description is omitted here. The audio compression / expansion unit 152 outputs the audio signal (hereinafter, also referred to as “audio signal”) on which the audio compression processing has been performed to the outdoor IF 42. The audio compression / expansion unit 152 receives an audio signal from the outdoor IF 42 and executes audio expansion processing on the audio signal. Since a known technique may be used for the audio decompression process, the description is omitted here. The audio compression / expansion unit 152 outputs the audio signal that has been subjected to the audio expansion process (hereinafter also referred to as an “audio signal”) to the outdoor audio processing unit 26. The generation and response unit 154 corresponds to a configuration in which the generation unit 60 and the response unit 62 are combined. Here, the generation / response unit 154 receives an uncompressed voice signal from the outdoor voice processing unit 26, receives a call signal from the call operation unit 20, and receives an unlock signal from the unlock processing unit 38. Note that the generation and response unit 154 does not include the sensor 78.

  The image decompression unit 160 receives image data from the indoor IF 50 and executes an image decompression process on the image data. Since a known technique may be used for the image decompression processing, the description is omitted here. The image decompression unit 160 outputs the image data that has undergone the image decompression processing (hereinafter, also referred to as “image data”) to the display unit 58. The audio compression / expansion unit 162 performs the same processing as that of the audio compression / expansion unit 152, and a description thereof will be omitted. The call operation unit 164 is a button or the like used when the resident talks with the visitor. Until now, it has been assumed that the lock is included in the unlocking operation unit 64, but it is separately shown here.

  According to the present embodiment, the generation / response unit is a high-cost component, and since this is included in the intercom outdoor device, the number of intercom outdoor devices is smaller than that of the intercom indoor and outdoor devices, so that the total system cost can be reduced. In addition, since the intercom outdoor unit is located in the common part, operations such as maintenance of the database and updating of operation software can be easily performed. Also, since the number is small, labor can be reduced. An image without deterioration before compression for transmission, or a large-size image before reduction can be used when the size of the captured image is different from the size of the transmitted image.

  In addition, when the generation unit 60 and the response unit 62 are included in the control device 12, the control device 12 is usually provided only once in the property. Therefore, in a property including a plurality of intercom outdoor devices 10, The total system cost can be reduced as compared with the case of processing. Further, when the generation unit 60 and the response unit 62 are included in the cloud, it is possible to execute high-precision processing using a large-scale operation. In addition, databases can be referred to each other for a plurality of properties.

(Example 3)
Up to now, the intercom system 100 has been installed in an apartment house or the like. Intercom system 100 may be installed in a detached house. In a detached house, the intercom outdoor device 10 and the intercom indoor device 14 are directly connected without the control device 12 being installed. In this case, the generation unit 60 and the response unit 62 are included in the intercom indoor unit 14 and the cloud. Here, an example in which the generation unit 60 and the response unit 62 are included in the intercom indoor unit 14 will be described.

  FIG. 10 shows a configuration of an intercom system 100 according to the third embodiment of the present invention. The intercom outdoor device 10 includes a calling operation unit 20, a microphone 22, a speaker 24, an outdoor audio processing unit 26, an imaging device 30, an image processing unit 32, and an outdoor IF 42. The intercom indoor unit 14 includes an indoor IF 50, an indoor audio processing unit 52, a microphone 54, a speaker 56, a display unit 58, a generation and response unit 154, a call operation unit 164, and a database 36. Components indicated by the same reference numerals as those used so far perform the same processing as before. Here, the intercom outdoor device 10 corresponds to an intercom slave device, and the intercom indoor device 14 corresponds to an intercom master device. As illustrated, since the generation / response unit 154 is included in the intercom indoor unit 14, the database 36 is also included in the intercom indoor unit 14.

  According to this embodiment, since the generation / response unit is included in the intercom indoor unit, the processing can be completed in one device.

  Further, when the generation unit 60 and the response unit 62 are included in the cloud, it is possible to execute high-precision processing using a large-scale operation. In addition, databases can be referred to each other for a plurality of properties.

  The present invention has been described based on the embodiments. This embodiment is an exemplification, and it is understood by those skilled in the art that various modifications can be made to the combination of each component or each processing process, and that such modifications are also within the scope of the present invention. .

  In the first to third embodiments, the intercom system 100 is installed in a condominium or a detached house. However, the present invention is not limited to this. For example, the intercom system 100 may be installed in an office or a factory. In this case, the resident in the embodiment is an employee, or the family is an employee of the office or factory. According to the present modification, the applicable range of the present embodiment can be expanded.

  In the first to third embodiments, the generation unit 60 is integrally included in the intercom outdoor device 10, the control device 12, the intercom indoor device 14, and the cloud. However, the present invention is not limited to this. For example, the components included in the generation unit 60 may be separately provided in two or more of the intercom outdoor device 10, the control device 12, the intercom indoor device 14, and the cloud. In addition, the generation unit 60 may be configured separately from the intercom outdoor device 10, the control device 12, the intercom indoor device 14, and the cloud as the intercom database generation device. According to this modification, the degree of freedom of the configuration can be improved.

  In this embodiment, when the visitor operates the call operation unit 20, both the generation process and the response process are executed. However, the present invention is not limited to this. For example, when the visitor operates the call operation unit 20, there may be a period in which only the response process is executed and the generation process is not executed. According to this modification, the processing amount can be reduced.

  Reference Signs List 10 intercom outdoor device, 12 control device, 14 intercom indoor device, 20 call operation unit, 22 microphone, 24 speaker, 26 outdoor sound processing unit, 30 imaging device, 32 image processing unit, 34 identification processing unit, 36 database, 38 unlocking processing section, 40 door, 42 outdoor IF, 50 indoor IF, 52 indoor audio processing section, 54 microphone, 56 speaker, 58 display section, 60 generation section (interphone database generation device), 62 response section , 64 unlock operation section, 70 first acquisition section, 72 extraction section, 74 second acquisition section, 76 classification section, 78 sensor, 80 third acquisition section, 82 storage execution section, 84 confirmation section, 90 identification section, 92 Processing unit, 100 intercom system.

Claims (7)

A first acquisition unit for acquiring image data of a visitor imaged,
A second acquisition unit that acquires a visitor's voice captured in the image data;
Based on at least one of the image recognition result of the image data acquired by the first acquisition unit and the speech recognition result of the audio acquired by the second acquisition unit, a plurality of images defined according to the reliability of the visitor. A classification unit that classifies a visitor imaged in the image data acquired by the first acquisition unit,
A storage execution unit that stores the result of the classification in the classification unit in a database,
A database generation system for an intercom, comprising: When the visitor classified in the classification unit is not stored in the database, the system further includes a confirmation unit configured to confirm a result of the classification to a resident,
The interphone database generation system according to claim 1, wherein the storage execution unit stores the result of the classification confirmed by the confirmation unit in the database. An extraction unit configured to extract a keyword related to a visitor from at least one of the image data acquired by the first acquisition unit and the response result acquired by the second acquisition unit;
The classification unit classifies a visitor imaged in the image data acquired by the first acquisition unit into one of a plurality of defined attributes according to the keyword extracted by the extraction unit. The interphone database generation system according to claim 1 or 2, wherein: An attribute of a visitor imaged in the image data acquired by the first acquisition unit by referring to the database generated by the interphone database generation system according to any one of claims 1 to 3. A specific part to be
A processing unit that performs a process on a resident according to the attribute specified by the specifying unit;
An intercom indoor unit comprising: Intercom outdoor equipment,
A control device connected to the intercom outdoor device,
An intercom indoor device connected to the control device,
The intercom outdoor device, the control device, by one or more of the intercom indoor device,
A first acquisition unit for acquiring image data of a visitor imaged,
A second acquisition unit that acquires a voice of a visitor captured in the image data acquired by the first acquisition unit;
Based on at least one of the image recognition result of the image data acquired by the first acquisition unit and the speech recognition result of the audio acquired by the second acquisition unit, a plurality of images defined according to the reliability of the visitor. A classification unit that classifies a visitor imaged in the image data acquired by the first acquisition unit,
A storage execution unit that stores the result of the classification in the classification unit in a database,
An intercom system comprising: A method for generating a database forming system for an intercom, comprising:
A step of acquiring image data of the image of the visitor;
Acquiring the voice of the visitor being imaged in the image data;
Based on at least one of the image recognition result of the image data and the voice recognition result of the voice, any of a plurality of attributes defined according to the reliability of the visitor, the captured image data is captured Classifying the visitors who are
Storing the result of the classification in a database;
A generation method, comprising:   A program for causing a computer to execute the generation method according to claim 6.

Images