JP6767924B2 - Systems, methods and programs - Google Patents

Description

The present embodiment relates to systems, methods and programs.

Conventionally, various communication systems such as transceivers, interphones, and mobile terminals have been developed. Conventionally, the calling device that starts a call generally inputs a dial number, an address, or the like by some means in order to determine the answering (or the other party) device.

On the other hand, recently, in the interphone system, when a plurality of other devices are installed, the priority of the other devices is determined, and when calling, the other devices are called in order of priority. It is being developed. This is to solve such a problem because when making an individual call, it may be unknown which room the other party is in, and the other party's device cannot be specified accurately.

International Publication No. 2015/125451 Japanese Unexamined Patent Publication No. 2011-23779

In recent years, devices have been developed that integrally have a communication function, a speaker, and a microphone, and can control surrounding electronic devices by the communication function. According to this device, a speaker can convey a message by voice, and a microphone can pick up and transmit the voice.

Therefore, an object of the present embodiment is to provide a system, a method, and a program that can determine the location of the other party and the other party's device that can be used by the other party, and arbitrarily switch and select the communication route and the other party's device with the other party. And.

According to the embodiment, the system includes a receiving unit, a determining unit, and a communication unit. The receiving unit receives a request for communication with the first user. The determination unit determines which device the first user exists in the vicinity of the plurality of devices. When it is determined that the first user exists in the vicinity of the first device, the communication unit establishes a communication route with the first device for communication in response to the request. When it is determined that the first user exists in the vicinity of the first device, and the mobile terminal owned by the first user is carried by the first user, the communication unit is used. A communication route with a mobile terminal owned by the first user is formed.

FIG. 1 is an explanatory diagram showing an example of a communication system of a certain family to which one embodiment is applied. FIG. 2 is a diagram showing a configuration example of the telephone device 11A as a representative. FIG. 3 is a diagram showing a part of the configuration included in the communication system. FIG. 4 is a diagram illustrating how the configuration shown in FIG. 3 is applied to the family shown in FIG. FIG. 5 is a diagram showing an example of an operation flow when the data of FIG. 13 is updated. FIG. 6 assumes that the server of FIG. 1 can recognize the data of FIGS. 12, 13, and 14, and identifies the registrant ID of the device according to which device the server communicates with. It is a flowchart explaining the method of doing. FIG. 7 is a flowchart showing an example in which the communication system of the present embodiment identifies the location of the other party of a call and forms a communication route. FIG. 8 is a flowchart showing still another example of the operation of the communication system. FIG. 9 is a diagram showing another example of a calling device or a special mobile terminal. FIG. 10 is a diagram showing a modified example of a special mobile terminal. FIG. 11 is an explanatory diagram showing an example of use when the communication system is specifically introduced in a department store, a factory, or the like. FIG. 12 is a diagram showing a registration information table of a registrant, a calling device, and a mobile terminal registered in the communication system. FIG. 13 is a diagram showing a status table used for grasping the current positions of registrants, calling devices, mobile terminals, etc. registered in this communication system. FIG. 14 is a diagram showing a connection destination determination table used when the communication route is established.

Hereinafter, embodiments will be described with reference to the drawings.

<Example of communication system in family>
FIG. 1 shows an example of a family of communication systems (information transmission environment in a region (home)) to which one embodiment is applied.

A family is composed of, for example, 12 sons, 13 daughters, 14 mothers, 15 grandfathers, and 16 fathers. The family's home shall have at least a son's room, a daughter's bedroom, a kitchen, a couple's bedroom (not shown), a living room, and a garden. Each of the families owns a one-to-one linked portable electronic device, such as a smartphone, mobile phone device, tablet PC or wearable device.

At this point, it is assumed that son 12 is in the son's room, daughter 13 is in the daughter's bedroom, mother 14 is in the kitchen, grandfather 15 is in the garden, and father 16 is abroad on a business trip.

In the room of the son 12, an electronic device (communication device (sometimes referred to as an Internet on Things (IoT) device or IoT (device))) 11A is arranged. It is assumed that the telephone device (electronic device) 11B is also arranged in the daughter's bedroom, and the telephone device (electronic device) 11C is also arranged in the kitchen. The individual communication devices 11 (A, B, C (subscripts may be omitted if no identification is required) are an interactive electronic device, THING (single (multifunctional device)), and monitoring electronic devices. It may also be referred to as a device or the like. The configuration of each communication device 11 (A, B, C) will be described with reference to FIG.

The individual communication devices 11A, 11B, and 11C can perform short-range communication with the router 40, respectively. This communication is, for example, communication via Wi-Fi (registered trademark). The router 40 can communicate with the mobile terminal by Wi-Fi. Furthermore, the router 40 can also communicate with the server 50 via the Internet 51.

The individual calling devices 11A, 11B, 11C can also communicate with mobile terminals at close range (nearby). This communication is, for example, communication by Bluetooth (registered trademark).

In the example of FIG. 1, the router 40, the communication devices 11A, 11B, 11C, and the mobile terminal 23 can each perform Wi-Fi communication. Further, the communication devices 11B and 11C can communicate with the mobile terminals 21 and 22, respectively.

Further, the son 12, the daughter 13, and the mother 14 can call the mobile terminal 24 of the father 16 via the respective calling devices 11A, 11B, 11C, or the mobile terminal. The communication route at this time is formed by the router 40, the Internet 51, the server 50, the access point 52, and the father's mobile terminal 24. On the contrary, the father 16 can also connect a line from his / her mobile terminal 24 to the calling devices 11A, 11B, 11C of the son 12, the daughter 13, and the mother 14.

FIG. 2 shows an example of the configuration of the call device 11 (A, B, C (attached with a subscript when identification is particularly required)).

The calling device 11 has a control unit 100. The control unit 100 can output audio data, and the audio data is converted to DA (digital-analog) by the audio output unit 110, the volume and the like are controlled, and the audio data is supplied to the speaker 111.

The video signal captured by the camera 121 is digitized and encoded by the video capture 122 and input to the control unit 100. The control unit 100 can control the illumination power supply 123 to control the illuminator 124. Further, the control unit 100 can acquire a nearby voice via the microphone 131 and the voice acquisition unit 132. The illuminator 124 is used to illuminate the photographing area when the surroundings are dark when the image is taken by the camera 121.

The control unit 100 can perform Wi-Fi (Local Area Network (LAN)) communication with a mobile terminal or a telephone device located in the vicinity via the transmitter / receiver 141. , Communication by short-range wireless communication (Bluetooth) can be performed with a mobile terminal in the vicinity via the transmitter / receiver 142.

Further, detection signals from various sensors 143 may be input to the control unit 100, or the control unit 100 may perform operation control (for example, on / off, characteristic switching, etc.) of the various sensors 143. The camera, microphone, and speaker can be turned off or on by the user's operation. Further, in each device 11, at least one of the camera, the speaker, and the microphone may utilize the data from the already provided one. The communication device 11 may be, for example, a PC (personal computer) with a camera or the like. Further, as a short-range wireless communication function, for example, zigbee (registered trademark), Z-Wave (registered trademark), etc. can be used.

FIG. 3 shows a part of the configuration included in the communication system. The receiving unit 301 receives a call signal input from the outside and requests communication with the first user. The first user is, for example, a user who has or is near the device 311.

The identification unit 302 identifies the first device 311 in which the first user exists in the vicinity or is owned by the first user from among a plurality of devices according to the call signal received by the receiving unit 301. Can be done. Then, the communication unit 303 establishes a communication route so that the first device 311 can perform communication according to the request according to the result specified by the specific unit 302.

The configuration shown in FIG. 3 may be built in one communication device that is independent, or may be built in the server 50 and cooperate with each device. Further, it may be shared by a plurality of debass shown in FIG. That is, as described above, it is possible to use the output (data) from an already prepared element such as a speaker or a camera.

FIG. 4 illustrates how the above configuration is applied to the family shown in FIG. Below, the speaker (son 12, sister 13, mother 14, grandfather 15, father 16) who starts the conversation (communication) and the person whom the speaker wants to talk to ((son 12, sister 13, mother 14, grandfather)) Various examples in which a communication route is formed between 15 and 16) will be described. The communication route is executed via the receiving unit 301, the specific unit 302, and the communication unit 303.

(Example 1)
In this system, the person who starts the conversation (speaking side) calls the name of the person to talk to (the other party (first user)) to the calling device located near him (for example, with mom and dad). (Call), voice output (calling voice) or ringing tone is automatically output to the calling device (first device) near the other party's location. As a method (UI (User Interface)) for specifying (designating) the person to talk to, in addition to voice instructions and authentication of video recognition results, for example, the screen of the terminal device held by the speaker is used. It may be a GUI (Graphical User Interface) to be displayed, a button displayed on the screen, or a button (hardware button) widely used in a remote controller or the like. In addition, when accepting instructions by voice, for example, various readings, such as personal names, siblings and grandparents (father, mother), names that can identify individuals from age information (brother / grandma), nicknames, etc. A replacement dictionary is prepared so that the same person can be identified for each of the above. When accepting a voice instruction, the speaker can be identified by voice recognition, and a person having an age characteristic for the speaker can be identified. For example, when the daughter 13 speaks "dad", the father 16 is specified, and when the father 16 speaks "dad", the grandfather 15 is specified.

For example, when the son 12 shown in FIG. 1 speaks "I want to talk to a mom" in the son's room, a communication route is formed between the talk device 11C in the kitchen where the mother 14 is located and the call device 11A in the children's room. A voice saying "I want to talk to my mom" is output from the call device 11C. Not connected to other bedroom or living room calling devices.

(Example 2)
In this case, whether or not the other party (for example, mom) of the call is near the call device 11C depends on the camera attached to the call device 11C or the room or kitchen where the mom is in cooperation with the call device 11C. Judgment is made by the person recognition (face recognition) function of the camera device.

(Example 3)
For example, my son 12 said, "I want to talk to my dad" in the children's room, but it cannot be detected by the calling device in the home centered on the router 40 or the Wi-Fi of the mobile terminal. It may be judged that there is no such thing. In this case, the line is automatically connected to the dad's mobile terminal 24. As shown in FIG. 1, when it is determined that the daddy 16 is not in the house, the communication routes of the router 40, the Internet 51, the server 50, the access point 52, and the mobile terminal 24 are established. In this case, even when the dad (father) 16 is abroad for work, for example, a communication route is established with the mobile terminal owned by the dad 16 via the Internet 51. If the dad 16 is abroad for work, for example, he outputs a message or voice such as "Daddy is working abroad" and carelessly goes through the Internet 51. It is also possible to prevent the establishment of a communication route.

As another example, suppose that the mother 14 calls the calling device 11C "grandpa". In the example of FIG. 1, the grandfather 15 is in the garden, and the mobile terminal 23 owned by the grandfather 15 is detected by the router 40. In this case, a communication route between the communication device 11C, the router 40, and the mobile terminal 23 is formed, and an incoming call is made to the mobile terminal 23.

(Example 4)
Whether or not the other party of the call is near the calling device installed in the home is determined by performing short-range wireless communication between the mobile terminal (device) carried by the other party of the calling and the calling device. The calling device can also determine. That is, when the device in the room detects the smartphone of the other party in the call by short-range wireless communication, the device in the room is connected.

Then, when the calling device detects the mobile terminal of the other party of the call by short-range wireless communication, the calling device on the speaking side is connected to the calling device. For example, as shown in FIG. 1, in the daughter 13's room, the call device 11B in the daughter's room detects the daughter's mobile terminal 21 nearby. In such a case, for example, when the mother 14 calls "wakame" (daughter's name) to the calling device 11C, a communication route between the calling device 11C and the calling device 11B is formed.

Further, it is determined whether or not the other party of the call is near the calling device and the mobile terminal is carried (carried) or held, for example, whether or not the acceleration sensor information of the mobile terminal is output. That is, in the example of FIG. 1, when the daughter 13 touches the mobile terminal 21 and enjoys playing a game, for example, the calling device 11B detects that the mobile terminal 21 is nearby by communication via Bluetooth.

The method of detecting the presence or absence of a visitor (or the presence or absence of a visitor) of a person or a specific person who wants to make a call (conversation) is limited to the detection of the mobile terminal device held by the other person (user) by the communication device 11. For example, recognition of a person by a camera device, acquisition of voice by a microphone (including using a speaker as a microphone when not in use), personal identification based on the characteristics of the acquired voice, presence of someone by a human sensor (heat sensitive) sensor. Any method that can detect a person, such as detection and personal identification by voice or video, may be used. However, it is preferable that pets, etc. can be registered in advance as detection exclusion targets.

(Example 5)
As in Example 4 above, the calling device and the mobile terminal near it are related, but if it is determined that the mobile terminal is not carried (grasped), the device on the speaking side is the mobile terminal. It will not be connected (if it is determined that the mobile terminal is not gripped (carried), it will not be connected to the communication device 11 in that room).

For example, in the case of FIG. 1, when the mother 14 calls "Katsuo" (the name of the son 12) to the calling device 11C, the son 12 does not own the mobile terminal, so that the communication between the calling device 11C and the calling device 11A is performed. Only the route is formed.

(Example 6)
Even when it is determined that the mobile terminal is not carried as in Example 5 above, if the mobile terminal or room is specially registered, the calling device on the speaking side is connected to the mobile terminal. ..

For example, when you are in the bedroom, you have your mobile device on the bedside, so by registering it as a specific device (it is considered to be in the room even if you do not wear it), the mobile device and caller in the bedroom It is possible to form a connection route between the calling devices of. For example, in the case of a person who cannot move freely and lightly in a room (sick person or person requiring long-term care), the mobile terminal is often left on the bedside for a long time, so it is possible to register the mobile terminal as a special device. preferable. As a condition for registration, it is preferable to exclude a time zone that can be regarded as not being in the room, such as a meal time zone or a bathing time zone.

(Example 7)
For example, in Example 1, when the connection destination is unknown (when the connection destination to which the other party can talk cannot be found), the calling device notifies the user to that effect (by voice or the like).

For example, in FIG. 1, the son 12 utters "I want to talk to Mr. Yamada", but when Mr. Yamada's information is not registered in the system, the calling device 11A replies "Mr. Yamada is not registered". ..

(Example 8)
The above example mainly describes a communication route connected from a calling device to a mobile terminal. However, there is a [call request (incoming call)] that a call comes directly to the mobile terminal from the outside. In this case, it is also possible to construct a communication route in which the mobile terminal is connected to a nearby calling device, and the owner of the mobile terminal is connected to another calling device nearby.

In FIG. 1, it is assumed that the daughter 13 moves to the kitchen, for example, and helps the mother 14 to cook. At this time, it is assumed that a call is received from the outside to the mobile terminal 21 (the mobile terminal owned by the daughter) in the bedroom of the daughter 21. In this case, since the electronic system knows the whereabouts of the daughter 13, a communication route is formed from the mobile terminal 21 to the calling device 11B, the router 40, and the calling device 11C. As another example, the mobile terminal may be charged in the bedroom, but the owner of the mobile terminal may be in a living room different from the bedroom. In this case, the incoming call to the mobile terminal in the bedroom is automatically connected to the calling device in the living room where the owner is located via the calling device in the bedroom automatically connected by short-range wireless communication, and through the calling device. You can talk to the other person.

Regardless of whether or not each person holds (carries) a mobile terminal, for example, a communication from a family member from outside the region via a communication line arrives and is linked to the mobile terminal by any communication device. When the whereabouts of the other party cannot be specified, for example, the mobile terminal owned by the other party (to be communicated) is notified (notified) that a communication is being received, for example, through a public communication network. On the other hand, for communications where the caller is unknown, for example, a message or data notifying that a call cannot be made is returned. It should be noted that the priority for notifying the other party's mobile terminal that a communication has arrived can be set in advance. For example, it is preferable to notify communication from family and friends even at midnight. On the other hand, for notifications from sources that are clearly related to work, you may reply to have them call back during working hours (usually possible work response hours). Further, as in the above (Example 6), when notifying a mobile terminal in a pre-registered place that a communication is received, it is limited to a case where it is clear that the communication is from a family member. It is also possible to apply.

<Reference table and data>
Next, the reference table and data used to dynamically construct each of the above communication routes will be described.
FIG. 12 shows a registration information table of a registrant, a calling device, and a mobile terminal registered in the above-mentioned communication system.
FIG. 13 shows a status table used for grasping the current positions of registrants, calling devices, mobile terminals, etc. registered in the above-mentioned communication system.
FIG. 14 shows a connection destination determination table used when a communication route is established as described above.
Each of the data shown in FIGS. 12 to 14 may be possessed by the server 50 shown in FIG. 1 or may be possessed by any of the communication devices shown in FIG.

First, in order for the above communication system to operate accurately, as shown in FIG. 12 (registration information table), registration of a person who uses this system, registration of a calling device to be used, and registration of a mobile terminal to be used are performed. is necessary. In addition, a surveillance camera incorporated in the communication system, an interphone, a special sensor for temperature, humidity, light, sound, etc. may be added.

In the registration information table shown in FIG. 12, identification data (ID) of father, mother, son, daughter, and grandfather who are family members are registered. In addition, the names of each registrant (father, mother, bonito (son), wakame seaweed (daughter), grandpa) are registered. In addition, each person's mobile terminal phone number, mobile terminal Bluetooth ID, mobile terminal MAC address, face data, calling device (Internet on) that is decided to contact even if the mobile terminal is not worn The address of Things (IoT) device or (sometimes called IoT) is registered.

In the situation table shown in FIG. 13, identification data (ID) of father, mother, son (bonito), daughter (wakame), and grandfather who are family members are first described. Then, the situation data indicating how each member is currently involved with the communication device or the mobile terminal is updated and described. The update time of this status data may be periodic, or may be when the other party of the call is called.

The connection destination determination table shown in FIG. 14 shows a combination of requirement data necessary for determining the location of the other party and the communication route (connection route) when the speaker calls the other party of the call. ..

Case 1 is a case where the face of the other party of the call is recognized through a certain calling device. In case 1, none of the calling devices detects the mobile terminal of the other party by short-range radio, the router does not detect the mobile terminal of the other party of the call by Wi-Fi, and the mobile terminal of the other party of the call There is no acceleration sensor information in the situation. In this case, it is determined that the other party of the call is in the room where the face-recognized call device is installed. As a result, a communication route for the face-recognized call device is established for the other party of the call called in Case 1.

In case 2, there is no face recognition, the calling device associated with the other party of the call detects the mobile terminal of the other party of the call by short-range radio, and the other party of the call wears the mobile terminal. This is an example of what is known. In this case, it is determined that the other party of the call is in the room where the calling device is installed. Then, in this case 2, a communication route is established in which the device on the speaker side is connected to the calling device installed in the room in the home.

In case 3, there is no face recognition, the calling device in a specific room associated with the other party of the call detects the mobile terminal of the other party by short-range radio, and the other party of the call wears the mobile terminal. This is an example of knowing that you are not wearing a cellphone. This example is the environment described in (Example 6) above. In this case, it is determined that the other party of the call is in a specific room. Then, in this case 3, a communication route is established in which the device on the speaker side is connected to the calling device installed in the specific room in the home.

In case 4, there is no face recognition, the calling device in a specific room associated with the other party of the call detects the mobile terminal of the other party by short-range radio, and the other party of the call wears the mobile terminal. This is an example of knowing that you are not wearing it. In this case, the other party of the call is determined to be unknown. The exception to this case 4 is the previous case 3. In this case 4, the calling device notifies the user (by voice or the like) that the other party's talkable connection destination cannot be found as described in the previous (Example 7).

In case 5, there is no face recognition, the calling device in a particular room associated with the other party does not detect the other party's mobile device via short-range radio, but the router makes a call via Wi-Fi communication. This is an example in which it is known that the other party is wearing a mobile terminal. In this case, it is determined that the other party of the call is near the house but the calling device is not nearby.

In this case 5, as described in (Example 3) above, it corresponds to the case where the mother 14 calls the calling device 11C as "grandpa". In this case, the speaker's device is connected to the mobile terminal via Wi-Fi communication.

In case 6, there is no face recognition, the calling device in a particular room associated with the other party does not detect the other party's mobile device via short-range radio, but the router makes a call via Wi-Fi communication. This is an example of the case where the mobile terminal of the other party is detected, but the mobile terminal is not worn. In this case 6, the calling device notifies the user (by voice or the like) that the other party's talkable connection destination cannot be found as described in the previous (Example 7).

In case 7, there is no face recognition, the calling device in a specific room associated with the other party does not detect the mobile terminal of the other party by short-range radio, and the router makes a call by Wi-Fi communication. This is an example of the case where the other party's mobile terminal is not detected. At this time, the communication system determines that the other party of the call is out. Then, a telephone connection is made to the mobile terminal of the other party of the call by the mobile terminal number.

As described above, the communication system can determine the communication route according to the situation of the other party of the call. For this purpose, the communication system utilizes the registration information table shown in FIG. 12, the status table shown in FIG. 13, and the connection destination determination table shown in FIG.

The registration information table shown in FIG. 12 is registered as common data in the registration information table when the communication device is installed in each room.

For example, one communication device is connected to a user's mobile terminal. As a connection method, for example, the communication device is set to the registration mode, the user's mobile terminal is set to the operation mode via Bluetooth, for example, and the ID of the communication device is input to the mobile terminal. As a result, the communication device and the mobile terminal are in a communication state. Then, "Please enter the registrant ID" is displayed on the screen of the mobile terminal. The registrant ID may be arbitrary. In this case, it is better that the family registrant IDs do not overlap. Next to the registrant ID, you can enter "the nickname, bonito, wakame seaweed, dad, mom, grandpa, etc."

At this time, the telephone number, Bluetooth ID, MAC address, etc. of the mobile terminal are automatically sent to the communication device and registered. In addition, the message "Do you want to enter face image data?" Is displayed on the screen of the mobile terminal. When the user inputs the face image data, for example, the user faces the front of the camera of the communication device and says "high" or "please", and the user's face image is acquired by the communication device. On the contrary, if the user says "No", the face image is not acquired.

Furthermore, when the communication device is in the registration mode, even if it is determined that the other party of the call does not wear the mobile terminal, the communication device that forms a communication route and contacts can be registered (specific). Is a communication device in the room). This registration reception is for establishing the communication route described in Case 3 above. In the registration mode menu, a message such as "Do you want to register a communication device in a specific room?" Is prepared. The user may select this message, enter the ID of the communication device in a specific room into the mobile terminal, and perform the confirmation operation.

In the above-mentioned communication system, the data in the status table shown in FIG. 13 is updated when the speaker calls or periodically. However, it is preferable to update the data shown in FIG. 13 each time the information on the shaking of the mobile terminal (shaking that can be determined to be carried) is received.

FIG. 5 is a diagram showing an example of an operation flow when the data of the situation table shown in FIG. 13 is updated. When the owner of the mobile terminal is carried and moved, the acceleration sensor information of the mobile terminal changes (Sa1).

This change is detected by, for example, communication between the communication device and the mobile terminal via Bluetooth, or Wi-Fi communication between the router 40 and the mobile terminal. Execution of communication via Bluetooth and execution of communication via Wi-Fi are activated by a control unit in the communication device and a control unit in the router. Whether or not the acquired acceleration sensor information of the mobile terminal is changed is used for updating the column of the acceleration sensor information in FIG. 13 above. The acceleration sensor information will indicate "wearing a mobile terminal" or "not wearing a mobile terminal". At this time, the information indicating whether the acceleration sensor information is detected by Bluetooth communication, Wi-Fi communication, or both is also updated in FIG.

That is, in the above communication process, whether or not the acceleration sensor information has changed, the ID of the communication device that communicated with the mobile terminal, or the IP address of the router that communicated with the mobile terminal can be known, so that the room in which the mobile terminal is located is known. It turns out that is not in the room. As a result, the whereabouts and status of the registrant can be updated (Sa5).

Further, the user who has registered the face image data in FIG. 12 can recognize the face by the communication device in each room in which the camera is turned on. The communication device can acquire face image data when a moving object is detected via the camera (Sa2). This communication system can compare the acquired face image data with the face image data registered in FIG. 12 and grasp who's face image data was acquired in which room.

If the acquired face image data matches the registered face image data of the registrant, it is found that the registrant is in the room of the acquired communication device. According to the result of this finding, the registrant's "location and situation can be updated (Sa6).

Further, since the communication device includes a microphone, a voice recognition function may be used. The voice data of the registrant is registered in FIG. 12, for example, in the same manner as the face image data. Then, when the communication device picks up the user's voice, it recognizes the user's voice and identifies that the user is in the room.

FIG. 6 is based on the premise that, for example, the server 40 of FIG. 1 can recognize the data of FIGS. 12, 13, and 14, and the registrant of the device depends on which device the server 40 communicates with. The method of specifying the ID is shown (Sb1).

The case of (Sb2) shown in FIG. 6 will be described. The server 50 may acquire face image data based on communication with the router 40 via any communication debus. In this case, the registered face image data corresponding to this face image data is detected. In this case, the registrant ID corresponding to the detected registered face image data is determined by referring to the registration information shown in FIG.

The case of (Sb3) shown in FIG. 6 will be described. The server 50 may detect acceleration sensor information indicating the movement of the mobile terminal based on communication via the router 40, any communication device, and any mobile terminal. In this case, with reference to FIG. 12, the Bluetooth ID used for communication between the mobile terminal and the communication device is specified, and the registrant ID corresponding to this Bluetooth ID is determined.

The case of (Sb4) shown in FIG. 6 will be described. The server 50 may communicate with the mobile terminal via the router 40 or the access point 52 and detect the acceleration sensor information of the mobile terminal. In this case, referring to FIG. 12, the registrant ID corresponding to the MAC address of the mobile terminal is determined when going through the router 40, and the registrant corresponding to the telephone number of the mobile terminal when going through the access point 52. Determine the ID.

FIG. 7 shows an example in which the communication system of the present embodiment identifies the location of the other party of a call and forms a communication route. Suppose a user calls a communication device in a room. That is, it is assumed that the communication device picks up the ringing tone (utterance). Then, the communication system performs voice recognition of the ringing tone (for example, wanting to talk to the mother) and searches for the registrant ID (mother's ID) registered in the registration information shown in FIG. 12, which is the other party of the call. (Sc1). If there is no registrant ID in FIG. 12, the called communication device (calling device) replies by voice saying "I am not a registrant" (Sc4).

When the registrant ID is detected, refer to FIG. 13 based on the registrant ID of the called party (the other party of the call), and refer to the situation of the other party of the call (are you in your room? / Are you in another room? / Are you out? Etc.). Then, based on the situation shown in FIG. 13, the communication route from the speaker to the other party of the call is specified (Sc3).

The process for identifying the communication route is carried out according to various situations. As a result of referring to FIG. 13, when it is determined that the device of the other party (registrant ID) of the call is a communication device, a communication route is formed between the communication device of the caller and the communication device of the other party of the call. (Sc5).

As a result of referring to FIG. 13, when it is determined that the device of the other party (registrant ID) of the call is a mobile terminal that makes a call via Wi-Fi communication, the communication device of the caller and the mobile terminal of the other party of the call are made. A communication route between and is formed using an IP address (Sc6).

As a result of referring to FIG. 13, when it is determined that the device of the other party (registrant ID) of the call is a mobile terminal that makes a call via the telephone number of the mobile terminal, the communication device of the caller and the mobile phone of the other party of the call are made. A communication route with the terminal is formed by using the telephone number of the mobile terminal (Sc7).

If the location of the other party in the call cannot be specified as a result of referring to FIG. 13, the communication device of the caller is controlled to execute a voice response saying "I do not know the location".

FIG. 8 is a flowchart showing still another example of the operation of the communication system. The explanation so far has mainly described an example in which a communication route is formed from a calling device to another calling device, an external mobile terminal, or the like.

Further, the communication system can output voice from a calling device or output from a mobile terminal according to the situation of the receiver (the other party of the call).

First, the communication device can detect an incoming call to the mobile terminal by short-range wireless communication (Sd1). At this time, the communication device acquires the Bluetooth ID of the mobile terminal (Sd2). Next, referring to the registration information shown in FIG. 12, the registrant ID of the owner of the mobile terminal is acquired from the acquired Bluetooth ID (Sd3).

Next, the communication system refers to the situation data shown in FIG. 13 and identifies the location of the owner of the mobile terminal based on the acquired registrant ID (Sd4). Next, it is determined whether or not a communication device that has detected the previous incoming call is installed in the specified location (Sd5). If it is found that the owner of the mobile terminal is near (room) the communication device that detected the incoming call (Sd6), the mobile terminal is connected to the communication device by communication via Bluetooth, and the mobile terminal is connected. The ringing tone and voice received in the call can be output from the communication device that has detected the incoming call (Sd7).

In step Sd6, when the owner of the mobile terminal is near a communication device different from the communication device that detected the incoming call, the communication system communicates between the communication device that detected the incoming call and the different communication device. Form a route. Then, the present communication system outputs a ringing tone and a voice received by the mobile terminal from the different communication devices (Sd8, Sd9).

When the above-mentioned communication route is formed, it is determined whether or not the incoming call is cut off (Sd10), and if the communication route is maintained, it is determined whether or not the utterance is started (Sd11), and the call is started. For example, it is determined that the call is in progress (Sd12). When it is determined that the incoming call has been cut off (Sd10), the communication ends.

In the above Sd5, when it is determined that the owner of the mobile terminal is not in the room where the communication device is arranged, the process proceeds to Sc1 in FIG. The process of FIG. 7 starts with checking the registrant ID again, determines the status (location, unknown) of the other party and secures a communication route, or if the other party is not registered, the caller and the device " Reply by voice such as "I am not a registrant".

FIG. 9 shows a calling device or mobile terminal 27 of another embodiment. The calling device or the mobile terminal 27 owns an interactive application that receives video data from the other party and transmits video data from the other party. For that purpose, it is provided with at least a display 27a, a camera 27b and a speaker 27c.

When the calling device has such a function, the speaker and the other party can talk with each other in sign language or the like.

FIG. 10 shows an example of the notification device 28. For example, when a user using a calling device as shown in FIG. 9 works in a noisy environment, he / she may not notice the incoming call. Therefore, when there is an incoming call to the calling device or the mobile terminal 27, the notification device 28 operates in conjunction with the vibration (by the oscillator), the light flash (by the light emitting unit), or (by the temperature rise mechanism). The heat, or a combination of these, can notify the user of an incoming call.

FIG. 11 shows a conceptual diagram when the electronic system and the electronic system control method are used in a department store (large-scale commercial facility), a factory, or the like. In this electronic system, the communication device 11A is pinpointed to any mobile terminal (or interphone or IP phone used mainly in a closed area) located on any floor F3, F4, F5 or F6 271a. You can specify any of 271k, 271n, and 271m to connect to the outside.

For this reason, it is possible to specify a specific area and play audio instead of broadcasting in the entire building. At department stores, you can contact employees and play BGM unique to each store. At the factory, messages such as work instructions can be sent to employees independently for each area.

Needless to say, the example shown in FIG. 11 can also be applied to, for example, stations, shopping malls, underground malls, and the like. Further, in order to detect a specific mobile terminal, it is possible to detect a floor to be broadcast in the hall or a specific block in the floor from the video or audio specified by the camera of the communication device 11A on each floor. The communication may be one-way communication as in the above-mentioned in-house broadcasting.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

11A, 11B, 11C ... Communication device, 12 ... Son (child (male)), 13 ... Daughter (child (female)), 14 ... Mother, 15 ... Grandfather, 16 ... Father, 21, 22, 23, 24 ... Mobile Terminal, 40 ... router, 50 ... server, 52 ... access point, 301 ... receiver, 302 ... partner device identification unit, 303 ... communication unit, 311, 312, 313 ... device.

Claims (14)

A receiver that receives a request for communication with the first user,
A determination unit that determines whether the first user exists in the vicinity of any of a plurality of devices,
A communication unit that establishes a communication route with the first device for communicating according to the request when it is determined that the first user exists in the vicinity of the first device is provided.
When it is determined that the first user exists in the vicinity of the first device, and the mobile terminal owned by the first user is carried by the first user, the communication unit is used. , A system that forms a communication route with a mobile terminal owned by the first user . A receiver that receives a request for communication with the first user,
A determination unit that determines whether the first user exists in the vicinity of any of a plurality of devices,
A communication unit that establishes a communication route with the first device for communicating according to the request when it is determined that the first user exists in the vicinity of the first device is provided.
Even when it is determined that the first user exists in the vicinity of the first device and the mobile terminal owned by the first user is not carried by the first user, the first user. When the mobile terminal owned by one user is located in the registered place, the communication unit forms a communication route with the mobile terminal owned by the first user . When it is determined that the first user does not exist in the vicinity of any of the plurality of devices, the communication unit establishes a communication route for the mobile terminal owned by the first user, claim 1. Alternatively, the system according to claim 2 . Claim 1 or claim 2 in which the determination unit determines whether or not the first user exists in the vicinity of the first device based on the video data acquired from the camera included in the first device. The system described in. The determination unit determines whether or not the first user exists in the vicinity of the first device based on whether or not short-range wireless communication between the mobile terminal owned by the first user and the first device is established. The system according to claim 1 or 2, which determines whether or not. The system according to claim 1 or 2 , wherein when it is determined that the device to be connected cannot be specified for communication with the first user, an output relating to the unknown connection destination is generated. If there is an incoming call to the first user of the portable terminal, wherein the communication unit generates a communication route between the first user of the portable terminal and the first device, according to claim 1 or claim 2, wherein system. The system according to claim 1 or 2 , wherein the receiving unit receives a request for communication with the first user in response to an incoming call to the mobile terminal of the first user. Receives a request for communication with the first user
From among the plurality of devices, determining whether the first user is present in the vicinity of any of the equipment,
If it is determined that the first user in the vicinity of the first device is present, the communication route between the first device for communication in accordance with the received request to establish,
When it is determined that the first user exists in the vicinity of the first device, and the mobile terminal owned by the first user is carried by the first user, the first user A method of forming a communication route with a mobile terminal owned by . Receives a request for communication with the first user
From a plurality of devices, it is determined which device has the first user in the vicinity, and
If it is determined that the first user in the vicinity of the first device is present, the communication route between the first device for communication in accordance with the received request to establish,
Even when it is determined that the first user exists in the vicinity of the first device and the mobile terminal owned by the first user is not carried by the first user, the first user. A method of forming a communication route with a mobile terminal owned by the first user when the mobile terminal owned by one user is located in a registered place . If the either of the first user in the vicinity of is determined that there is no plurality of devices, the first user to establish a communication route to the portable terminal owned by The method of claim 9 or claim 10, wherein .. On the computer
Receive a request for communication with the first user
From among a plurality of devices, it is determined which device the first user exists in the vicinity of the device.
If it is determined that the first user in the vicinity of the first device is present, to establish a communication route between the first device for communication in response to the request is received,
When it is determined that the first user exists in the vicinity of the first device, and the mobile terminal owned by the first user is carried by the first user, the first user A program that forms a communication route with a mobile terminal owned by . On the computer
Receive a request for communication with the first user
From among the plurality of devices, to determine whether the first user is present in the vicinity of any of the equipment,
If it is determined that the first user in the vicinity of the first device is present, to establish a communication route between the first device for communication in response to the request is received,
Even when it is determined that the first user exists in the vicinity of the first device and the mobile terminal owned by the first user is not carried by the first user, the first user. A program that forms a communication route with a mobile terminal owned by the first user when the mobile terminal owned by one user is located in a registered place . The program
If in the vicinity of any of the plurality of devices it is determined that the first user does not exist, the first user to establish a communication route to the portable terminal owned, according to claim 12 or claim 13, wherein program.

Images