JP2021533510A - Interaction method and equipment - Google Patents

Abstract

One or more embodiments of the present specification provide methods and devices of interaction, wherein the method comprises a step of detecting a user located within a detection area and an area of detection of interaction content. It has a step to provide to the user in the detection area and a step to display information about the target interaction object to the user in the detection area when the target interaction object of the interaction content is a part of the user in the detection area. .. [Selection diagram] Fig. 2

Description

Cross-reference to related applications This application claims the priority of Chinese Patent Application No. 201810806493.6, entitled "INTERACTION METHOD AND DEVICE" and filed on 20 July 2018. The contents of this patent document are all incorporated herein by reference.

One or more embodiments herein relate to the field of electronic technology, and more particularly to methods and devices of interaction.

With the continued development of intelligent technology, electronic devices are becoming more and more intelligent, which, to some extent, are capable of interacting with the user, and at the completion of the relevant event, the user. Can help. For example, the electronic device can complete the process of interaction with the user in the form of displaying the relevant content on the screen, playing the relevant content through audio, or something similar.

In view of this, one or more embodiments herein provide methods and devices of interaction.

In order to achieve the above-mentioned object, one or more embodiments of the present specification provide the following technical solutions.

According to a first aspect of one or more embodiments herein, a method of interaction is provided, wherein the method is:
A step to detect one or more users placed in the detection area,
Steps to provide interaction content to one or more users in the detection area,
A step of displaying information about a target interaction object to one or more users in the detection area when the target interaction object of the interaction content is part of one or more users in the detection area.
including.

According to a second aspect of one or more embodiments herein, a device of interaction is provided and the device is a device.
A detection unit configured to detect one or more users placed within the detection area, and
A delivery unit configured to serve interaction content to one or more users within the detection area.
Configured to display information about the target interaction object to one or more users in the detection area when the target interaction object in the interaction content is part of one or more users in the detection area. Display unit and
including.

FIG. 3 is a schematic architectural diagram of a system of interactions according to an exemplary embodiment. It is a flowchart of the method of interaction by one Embodiment for example. FIG. 3 is a schematic diagram of an interaction scenario according to an exemplary embodiment. It is a schematic diagram of the execution of an interaction with an internal employee according to one exemplary embodiment. It is a schematic diagram of the guide of the user position through the interaction content by one example embodiment. It is a schematic diagram of an active activation of an interaction with a user by an interacting device according to an exemplary embodiment. It is another schematic of the guide of the user position through the interaction content by one example embodiment. FIG. 3 is a schematic diagram of an interaction scenario under normal circumstances according to an exemplary embodiment. It is a schematic diagram of the adjustment of the interaction content according to the related event according to one embodiment for illustration. FIG. 6 is a schematic diagram of a speaker designation by an interacting device according to an exemplary embodiment. FIG. 3 is another schematic representation of a speaker designation by an interacting device, according to an exemplary embodiment. It is still another schematic diagram of the designation of the speaker by the device of interaction according to one embodiment for illustration. It is a schematic diagram of the designation of the utterance order of an outsider by one Embodiment for example. It is a schematic diagram of the marking of an interaction object according to one embodiment for illustration. FIG. 6 is a schematic diagram of marking of a target interaction object according to an exemplary embodiment. It is the schematic of the determination of the source user of a user voice by one Embodiment for example. It is the schematic of the determination of the source direction of an audio message by one Embodiment for example. It is a schematic diagram of the marking of the source user of the user voice by one embodiment for illustration. It is the schematic structural drawing of the apparatus by one Embodiment for example. FIG. 3 is a block diagram of an interaction device according to an exemplary embodiment.

An exemplary embodiment is described in detail herein, and the accompanying drawings show embodiments for purposes of illustration. When the following description is carried out with reference to the accompanying drawings, the same reference numerals in different attached drawings represent the same or similar elements, unless otherwise indicated. Not all of the implementations described in the following exemplary embodiments are consistent with one or more embodiments herein, and these are instead. , Which is detailed in the accompanying claims and is only an example consistent with some aspects of the devices and methods in one or more embodiments herein.

Note that in other embodiments, the corresponding method steps are not necessarily performed according to the sequences illustrated and described herein. In some other embodiments, the method may include more or fewer steps than those described herein. In addition to this, the single step described herein can, in other embodiments, be split into multiple steps for illustration purposes and are described herein. The plurality of steps can also be combined into a single step for illustration purposes in other embodiments.

In one embodiment, the solutions for interactions herein can be applied to devices of interaction. The interaction device may be an electronic device dedicated to implement the interaction function. Alternatively, instead, the interacting device may be a multifunctional electronic device having an interacting function. For example, interacting devices may include personal computers (PCs), tablet devices, notebooks, wearable devices (such as smart glasses), and the like, which are described herein. Not limited to one or more embodiments of the book. In the execution process, the interaction device can execute the interaction system to implement the interaction solution. The application program of the interacting system can be pre-installed on the interacting device and then launched and executed on the interacting device. Of course, when a technology such as HTML5 is used, the interacting system can be acquired and executed without installing an application program on the interacting device.

In one embodiment, FIG. 1 is a schematic architectural diagram of a system of interactions according to an exemplary embodiment. As shown in FIG. 1, the interaction system may include a server 11, a network 12, and an interaction device 13. In the execution process, the server 11 can execute a program on the server of the interacting system in order to implement related functions such as processing, and the interacting device 13 can present information and perform human-computer interaction. Implements related functions such as the interaction of the interaction system, thereby executing the program of the client of the interaction system to implement the interaction system through the cooperation between the server 11 and the interaction device 13. be able to.

The server 11 may be a physical server containing a stand-alone host, or the server 11 may be a virtual server carried by a host cluster. The interaction device 13 may be an electronic device dedicated to implement the interaction function. Alternatively, instead of this, the interaction device 13 may be a multifunctional electronic device having an interaction function. For example, interacting devices may include PCs, tablet devices, notebooks, wearable devices (such as smart glasses), and the like, which may be one or more of the present specification. It is not limited in the embodiment. The network 12 in which the interacting device 13 and the server 11 interact with each other may include multiple types of wired or wireless networks. In one embodiment, the network 12 can include a public switched telephone network (PSTN) and the Internet. Note that the client application program of the interacting system can be pre-installed on the interacting device and then launched and executed on the interacting device. Not surprisingly, when an online "client" in a technology such as HTML5 is used, the client can acquire and execute without installing the corresponding application program on the interacting device. ..

In one embodiment, the interaction system can be implemented on the basis of a mobile group office platform. The mobile group office platform may implement communication functions and may include, for example, review and approval events (eg, vacation requests, office supply applications, and review and approval events such as financial issues), attendance check events, etc. Used as an integrated functional platform for many other features such as handling group internal events such as task events or log events, or handling group external events such as ordering food or purchasing goods. Also, this is not limited to one or more embodiments herein. Similarly, mobile group office platforms can implement systems of interaction.

For example, mobile group office platforms include, for example, Skype for Business®, Microsoft Teams®, Yammer®, Workplace®, Slack®, WeCom®, FXiaoKe. Can be carried on instant messaging applications in related technologies, such as Enterprise Instant Messaging (EIM) applications such as (Registered Trademarks), Fetion enterpress®, or EasyChat enterpress®. .. Not surprisingly, the instant messaging feature is just one of the communication features supported by the mobile group office platform, even though the mobile group office platform further implements many of the other features mentioned above. Often, detailed description of these will be omitted herein. A "group" herein may include, but is not limited to, organizations such as businesses, schools, the military, hospitals, and public institutions.

In one embodiment, the interaction system is not limited to mobile group office platforms or similar scenarios, and is based on any other type of application, for example, a common instant messaging application. It may be implemented, which is not limited herein.

FIG. 2 is a flow chart of a method of interaction according to an exemplary embodiment. As shown in FIG. 2, the method can be applied to a device of interaction and the method can include the following steps.

Step 202: Detects one or more users located in the detection area.

In one embodiment, the interacting device has a particular detection distance and the coverage area of the detection distance is in a fan shape (or in any other shape) with a radius of 3 meters (m). ) It forms a detection area such as an area. Whether or not one or more users exist in the detection area can be determined by executing the detection in the detection area.

In one embodiment, the interacting device can detect one or more users within the detection area by any method. For example, the interacting device can determine if one or more users are present in the detection area through face detection.

Step 204: Provide the interacting content to one or more users in the detection area.

In one embodiment, the interacting device may provide the interacting content in one of a plurality of schemes or a combination thereof, which is not limited herein. For example, the interaction device can include a display screen and can display the interaction content on the display screen in order to provide the interaction content to one or more users in the detection area. In another example, the interacting device can include a speaker and interact content through audio by using the speaker to provide the interacting content to one or more users in the detection area. Can be played. In yet another example, the interacting device can include multiple indicator lights and the indicator lights on / off to provide the interacting content to one or more users in the detection area. The color or blinking method can be controlled.

In one embodiment, the detection area may include a short range detection area and a long range detection area. The short-range detection area is closer to the interacting device than the long-range detection area. In other words, "short distance" and "long distance" in the present specification are in a relative relationship. For example, a range of 0 m to 1.5 m is designated as a short-distance detection area, and a range of 1.5 m to 3 m is designated as a long-distance detection area.

In one embodiment, the interacting device can provide the interaction content to the user in the short range detection area, and the interacting device enters the short range detection area from the long range detection area. , Guidance information can be sent to users in the range detection area to guide the user. As a result, the user becomes a user in the short-distance detection area, and the interaction content is provided to the user. When the user is placed within the short range detection area, the user may optionally wish to interact with the interacting device. However, the long-distance detection area cannot provide a relatively good interaction effect due to the relatively long distance. Therefore, guidance information can be sent to the user. As a result, on the one hand, it is possible to determine if the user really wants to implement the interaction, and on the other hand, the user has a relatively good interaction after entering the short range detection area. The effect can be obtained. The interacting device may transmit guidance information by any of a plurality of methods or a combination thereof, which is not limited herein. For example, the interacting device can display guidance information on the display screen. In another example, the interacting device can reproduce the guidance information through voice by using a speaker. In yet another example, the interacting device may turn on the indicator light or blink the indicator light to guide the user into the short range detection area.

In one embodiment, in the attendance check operation, the interacting device is placed in the detection area regardless of whether the user is placed in the short range detection area or in the long range detection area. It is possible to identify a user in the detection area in order to automatically implement an attendance check operation for a user who has not been checked for attendance within the attendance check period. The user's stay duration in the long-range detection area has reached the first preset duration, or the user's stay duration in the short-range detection area has reached the second preset duration. In doing so, the interacting device can determine that the user may desire to implement the interacting behavior and, in turn, can provide the interacting content to the user. In general, the user approaches the device of interaction only when the user desires interaction with the device of interaction. Therefore, the second preset duration may be less than the first preset duration, as appropriate, in order to reduce the user's wait time.

In one embodiment, the interaction device can actively provide interaction content similar to the "greeting" behavior between users to the user in the detection area. For example, the interaction content may include "What can I do for you?" To assist the user in completing the relevant event, thereby guiding the user in the detection area. can do.

Further, the interaction device determines whether or not the user in the detection area satisfies the preset condition so that the interaction content is provided only to the user who satisfies the preset condition. You can also do it. For example, the preset condition is that the duration of stay in the long-range detection area reaches the first preset duration, and the duration of stay in the short-range detection area reaches the second preset duration. Reaching time, the user observing the device of interaction, the user facing the device of interaction, or the angle between the user's face and the device of interaction is less than a preset angle. It may include, or may include at least one of the similarities thereof, which is not limited herein.

In one embodiment, when the interaction content is associated with a related event, the interaction device adjusts the interaction content provided to the user in the detection area according to the state information of the related event. As such, the interacting device can capture the user's relevant event within the detection area. Default interaction content can be provided when there are no related events associated with the interaction content. For example, when interactive content is associated with attendance checks, interactive content about users in the detection area detected during working hours under normal circumstances will be "Leave early? (Are you?). "Sure to leave early?)". Normal if the user's relevant event in the detection area includes a sickness leave review and approval event and it is within the period in which the leave associated with the sickness leave review and approval event is requested. The interaction content can be adjusted to "are you sure to lock out?" Even during working hours under certain circumstances. In another example, when the interaction content is associated with an outsider's visit, it is obtained that the user in the detection area is an outsider and the corresponding related event is a visit booking event. In that case, the interaction content may be "Would you like to call the person you plan to visit? (Would you like to call the person you will visit?)". In the absence of a corresponding visit booking event, the interaction content may be "who will be visiting? (Please tell me who you will visit?)".

In one embodiment, the interacting device can determine the identity type of the user in the detection area and then adjust the interaction content provided to the user in the detection area according to the identity type. can. For example, the identity type may include whether the user in the detection area is an insider or an outsider of the group, the department to which the user in the detection area belongs, and the like. Often, this provides the user in the detection area with interaction content that meets the identity type.

Step 206: Display information about the target interaction object to one or more users in the detection area when the target interaction object in the interaction content is part of one or more users in the detection area. ..

In one embodiment, when there are a plurality of users in the detection area, the target interaction object may be only a part of the user (the number of parts of the user may be one or more). good). By displaying information about the target interaction object to the users in the detection area, we ensure that each user in the detection area can clearly know if they are the target interaction object for the interaction content. be able to.

In one embodiment, the image information of each user in the detection area can be acquired, and the corresponding profile image can be generated for each user. Profile images of all users can then be displayed before the interaction is implemented. When an interaction with a target interaction object is performed, only the profile image of the target interaction object is displayed and the profile image of another user is blocked, or the target interaction object The profile image corresponding to is displayed separately from the profile image corresponding to other users. For example, the separate display may include displaying the profile image of the target interaction object within the central area and displaying the profile image of another user within the edge area. In another example, the separate display includes a step of displaying the profile image of the target interaction object in a relatively large size and a step of displaying the profile image of another user in a normal or relatively small size. Can include. In yet another example, the separate display may include a step of successfully (colorfully) displaying the profile image of the target interaction object and a step of monochromeizing the profile of another user to display it. This is not limited herein.

In one embodiment, when the target interaction object of the interaction content is all users in the detection area (the number of users may be one or more), the user is relative to the interaction content. There is no need to display information about the target interaction object to the user in the detection area so that it is in focus. Not surprisingly, the implementation of the technical solutions herein is also unaffected when information about the target interaction object is displayed in this scenario.

In one embodiment, the users in the detection area may change continuously. When the target interaction object of the interaction content changes from a part of the users in the detection area to all users (for example, a user who is not the target interaction object leaves the detection area), the target interaction object The steps to display information about to the user in the detection area are from "The target interaction object of the interaction content is part of the user in the detection area" to "The target interaction object of the interaction content is in the detection area. Can be paused to complete a smooth scenario transition to "all users of."

In one embodiment, the interaction device may determine the identity information of the user acting as the target interaction object, which information can be used as information about the target interaction object, as well as. The identity information can then be displayed to the user in the detection area. For example, the interaction device may display the identity information to the user in the detection area while providing the interaction content, or the interaction device may provide the interaction content and provide the identity information. It may be displayed at different moments, which is not limited herein.

In one embodiment, the interacting device can identify a user within the detection area. For example, the interacting device may identify the user by physiological feature recognition methods such as face recognition, fingerprint recognition, iris recognition, gait recognition, or voiceprint recognition, or by any other method. Is not limited herein. When the identity of the first user acting as the target interaction object is successfully identified, the identity information may include the name of the first user (eg, name, nickname, title, or other type). .. For example, when the name of the first user is "Xiao Byi", the interacting device may display "Mr. Xiao Byi, what kind of business do you have?" To the user in the detection area. In this case, "Xiao Byi" is information about the target interaction object, and "What kind of business do you have?" Is the interaction content. When the identity of the second user acting as the target interaction object is not successfully identified, the identity information can provide visual feature description information about the second user. For example, visual feature description information includes estimated gender, estimated height, estimated age, skin color, clothing, accessories, distance to the interacting device, orientation towards the interacting device, and similar. It may contain at least one. For example, the interacting device may display "Men in black coat, what are your needs?" To users in the detection area, in this case "Wearing black coat." For "male", the estimated gender and clothing are used as information about the target interaction object, and "what is your requirement?" Is the interaction content.

In one embodiment, the interacting device can display user reference information corresponding to the user in the detection area. The interaction device may determine user criteria information corresponding to the user acting as the target interaction object, and may use that information as information about the target interaction object, and then , The interacting device can prominently display the determined user reference information to the user in the detection area. For example, the interacting device can perform image acquisition to a user in the detection area and display the acquired user image as user reference information. Thus, the interaction device may improve the visual effect of the user image corresponding to the target interaction object (eg, corresponding) to help the user in the detection area know about the target interaction object. You may circle the user image or display an arrow mark around the corresponding user image, or you may weaken the visual effect of the user image corresponding to the non-target interaction object (eg, corresponding). Block the content of the user image).

In one embodiment, the interacting device can acquire an event support request sent by a user in the detection area and can respond to the event support request to assist in completing the corresponding event. .. For example, the user in the detection area may send the interaction voice "Call employee Xiao Hei" to the interaction device, and the interaction device is subject to the user's event support request. , It may be determined that the request is for calling "employee's Xiao Hei", and "employee's Xiao Hei" can be called. As a matter of course, in addition to the voice form, the user in the detection area may send the event support request by other methods such as performing a preset body movement in the space. Is not limited herein.

In one embodiment, the interacting device can receive the response information returned by the user in the detection area in response to the interacting content. The response information includes an event support request. For example, when the interaction content is "Mr. Xiao Bai, what kind of business do you have?", The user in the detection area may answer "Please call Mr. Xiao Hei, an employee." In addition, the interacting device may determine that the event support request is a call request for "employee Xiao Hei" and may call "employee Xio Hei". The method by which the interacting device provides the interaction content is not necessarily associated with the method by which the user in the detection area returns response information. The two methods may be the same or different. This is not limited herein.

In one embodiment, when a plurality of users are present in the detection area, the interacting device can select the user as a supported object according to a preset sequence. The interacting device then sequentially requests the selected user to send the corresponding event support request. As a result, users in the detection area can send event support requests in sequence, which is due to the confusion that occurs when multiple users send event support requests at the same time. The situation where the interacting device cannot accurately obtain the event support request corresponding to the user is avoided, which helps to improve the support efficiency for the user and the success rate of the interacting device.

In one embodiment, the interacting device can perform semantic recognition on the acquired user voice to acquire an event support request, and the interacting device determines the source user of the user voice. Therefore, sound feature recognition can be performed on the user's voice. As a result, even when a plurality of users in the detection area speak at the same time, the interacting device can recognize the utterance content corresponding to each user and improve the support efficiency for the user. Can respond to corresponding event support requests. The interacting device can pre-identify the user in the detection area in order to acquire the user's identity information. When there are multiple users in the detection area, the interacting device has sound features in the acquired user voice according to the recognized sound features corresponding to the user in order to determine the source user of the user voice. Awareness can be performed. Compared to the method of performing sound feature recognition for user speech by using all sound features, the time spent on sound feature recognition can be significantly reduced.

In one embodiment, when the user in the detection area is an insider of the group, the interacting device can assist in completing the corresponding group management event in response to the event support request. When a user in the detection area is an outsider of the group, the interacting device may send an alert message to the relevant insider of the group and in establishing communication with the relevant insider of the group. The outsider of the group may be assisted, or the outsider of the group may be guided to the processing location of the visit event in response to the event support request. When the user in the detection area is an administrator, the interacting device can assist in completing the corresponding device management event in response to the event support request.

In one embodiment, the interacting device can receive and respond to a user voice transmitted by a user within the detection area. For example, the user in the detection area can actively send the user voice to the interacting device, and the user voice interacts with the daily greetings, sending the event support request to the interacting device. It is used to grant to the device, send control commands to the interacting device, or perform similar content, which is not limited herein. In another example, the user in the detection area can send the corresponding user voice to the interacting device in response to the interaction behavior implemented by the interacting device. For example, when the interaction behavior implemented by the interaction device is to inquire about whether the user in the detection area needs assistance, the user voice transmitted by the user is the required assistance. The type can be notified to the interacting device. This is not limited herein.

In one embodiment, the interacting device can perform semantic recognition on the user voice. Since the same pronunciation can correspond to multiple words or phrases and there can be distortion or noise interference when the interacting device picks up the user voice, the interacting device is relative to the user voice. After performing semantic recognition, you may get multiple semantic recognition results. The interacting device can score the semantic recognition result according to a predefined semantic recognition algorithm to obtain the corresponding confidence level. When the confidence level reaches a preset value, it signals that the corresponding semantic recognition result is reliable enough. Furthermore, if there are multiple semantic recognition results with a degree of matching with the user's voice that reach a preset value, the interacting device will allow the user to express the user's true intentions. Multiple corresponding semantic recognition result choices can be displayed to the user in the detection area for selection and accurate representation, and then the user according to the semantic recognition result corresponding to the selected semantic recognition result choice. Can respond to voice. In the selection process, the user may utter a semantic recognition result that corresponds to the semantic recognition result option that the user wants to select, or the position of the semantic recognition result option that the user wants to select. (For example, the "first thing" or the "leftmost thing") may be spoken, which is not limited herein.

In one embodiment, the interacting device can determine the source direction of the user voice and can respond to the user placed in the source direction of the user voice. In one situation, after determining the source direction of the user voice so that the interacting device can respond directly to the source direction of the user voice, for example, playing the interacting voice. By default, the interacting device directly determines that only the user transmitting the user voice is present in the source direction. In another situation, the interacting device can determine one or more users present in the direction of the source of the user voice. When a plurality of users are present, the interacting device can further determine the source user of the user voice in order to respond to the source user.

In one embodiment, the microphone array is built within the interacting device and the microphone array can be used to receive user voice. The microphone array includes a first microphone arranged on the relatively left side and a second microphone arranged on the relatively right side. The source direction of the user voice is determined according to the difference between the time points when the first microphone and the second microphone receive the user voice. For example, when the user in the detection area is located on the side close to the left side, the first microphone can receive the user voice earlier than the second microphone, and the user in the detection area can receive the user voice. When placed on the side close to the right side, the second microphone can receive the user voice earlier than the first microphone. Relevant technical solutions in existing technology may be referred to for specific solutions in how to determine the source orientation of the user voice according to the difference in time of receipt, and are therefore detailed herein. Is omitted.

In one embodiment, when there are multiple users located in the direction of the source of the user voice, the interacting device determines the source user of the user voice according to the facial movements of each of the plurality of users. Along with being able to determine (eg, facial movements are acquired through image acquisition by using a camera built within the interacting device), they can respond to the source user. The user's facial movements may include movements of one or more parts such as the cheeks, mouth, and chin, which are not limited herein. As an example, using mouth movements, there are multiple users in the direction of the source of the user's voice, but when only one user is performing the mouth opening / closing movement, that user It can be determined as the source user of the user voice. In another example, when all of the plurality of users perform the mouth opening / closing operation, but the movement of one user matches the user voice in the number of times and in the opening / closing amplitude. , The user can be determined as the source user of the user voice.

In one embodiment, when the interacting device is mounted on a wall, the user is generally only able to pass in front of the interacting device and transmit the user voice. Can be done. When the interacting device is otherwise attached, the audio messages captured by the interacting device may come from the user located in front of or behind the interacting device. The user may appear in front of or behind the interacting device. If a user is in the detection area of the interacting device and another user just speaks as they pass behind the interacting device, the interacting device is in the detection area. Can be mistaken for the user sending the user voice. Therefore, after receiving the audio message, the interacting device sends the audio message by the user in the detection area based on the source direction of the audio message and whether the user is in the detection area. It is possible to determine whether or not the user voice has been used.

For example, the microphone array is constructed in an interacting device, and the microphone array is relatively far from the detection area with a third microphone arranged in a state relatively close to the detection area. Includes a fourth microphone disposed in the state. When an audio message is received by using a microphone array, the source orientation of the audio message can be determined according to the amount by which the third and fourth microphones receive the high frequency portion of the audio message. .. Audio received by the 4th microphone because the high frequency part in the audio message is affected by the absorption of the housing of the interacting device when the source direction is relatively close to the detection area. The high frequency portion of the message is less than the high frequency portion of the audio message received by the third microphone. Of the audio message received by the third microphone, the high frequency portion of the audio message is affected by the absorption of the housing of the interacting device, even when the source direction is relatively far from the detection area. The high frequency portion is less than the high frequency portion of the audio message received by the fourth microphone. Therefore, the source orientation of the audio message can be accurately determined according to the amount by which the third and fourth microphones receive the high frequency portion of the audio message.

When the source direction is on the side relatively close to the detection area and there is a user in the detection area, the interacting device will send the audio message by the user in the detection area. It can be determined that the user voice is voiced. Instead, when the source direction is on the side relatively far from the detection area, or when the source direction is on the side relatively close to the detection area, the user is in the detection area. When not present, the interacting device can determine that the audio message is not a user voice transmitted by a user within the detection area.

In relation to the first microphone, the second microphone, the third microphone, and the fourth microphone, different microphones are discriminated as the first and second microphones in order to discriminate the microphones in the left-right direction. Note that different microphones are discriminated as the third microphone and the fourth microphone in order to discriminate the microphones in the anterior-posterior direction. However, the number of microphones actually contained in the microphone array is not limited herein. For example, the microphone array may include one or more first microphones and one or more second microphones. In another example, the microphone array may include one or more third microphones and one or more fourth microphones. In addition to this, the microphone array does not necessarily have to include four microphones when the microphone array needs to simultaneously discriminate between right-left and front-rear source directions. In other words, the first microphone, the second microphone, the third microphone, and the fourth microphone only play a role in implementing the related functions. In practice, the microphone array may include a smaller number of microphones. For example, the microphone array may include three microphones, in which case microphones 1 and 2 are arranged in the same row in the front-rear direction, but separated in the left-right direction. The microphones 3 are arranged in front of or behind the microphones 1 and 2 so that the microphones 1 to 3 form a positional relationship similar to a triangle. The microphone 1 and the microphone 2 may be used as the first microphone and the second microphone in order to discriminate the source direction of the user voice in the left-right direction, and the microphone 1 and the microphone 2 are used as the third microphone. Also, the microphone 3 may be used as a fourth microphone (microphone 3 is located behind microphone 1 and microphone 2). Alternatively, instead, the microphone 3 may be used as a third microphone, and the microphones 1 and 2 are used as a fourth microphone to discriminate the direction of the audio message in the anterior-posterior direction. May be (microphone 3 is located in front of microphone 1 and microphone 2).

In one embodiment, the interacting device detects some users in the detection area, eg, by using a camera to perform acquisition, and then by performing face detection and counting. However, this is not limited herein. When a plurality of users are present in the detection area, the interacting device can display a profile image corresponding to each user in order to represent the plurality of users. When a user enters or leaves the detection area, or the number of users changes, the profile image displayed by the interacting device can change accordingly. When the interacting device receives an audio message and determines that the audio message is from a user within the detection area, the interacting device fails to receive the user voice or is the source. Without worrying about misidentifying the user, the user determines that the interacting device successfully received the user's voice and discriminated the source user by considering changes in the profile image. As such, the interacting device can display the profile image of the source user of the user voice separately from the profile image of the other user. The source user profile image of the user voice may be displayed separately from the profile images of other users by any method, which is not limited herein. For example, the profile image of the source user of the user voice can be displayed in the central area, and the profile images of other users can be displayed in the edge area. In another example, the source user profile image of the user voice can be displayed in a relatively large size, while the profile images of the other users can be displayed in a normal or relatively small size. In yet another example, the source user profile image of the user voice may be displayed normally (colorfully), and the profile images of the other users may be monochrome and then displayed. May be done.

For ease of understanding, the technical solution of one or more embodiments of the present specification will be described by using the enterprise instant messaging application "WeCom" as an example. FIG. 3 is a schematic diagram of an interaction scenario according to an exemplary embodiment. As shown in FIG. 3, the interaction device 3 is disposed in the office of the company AA, and the interaction device 3 is the solution of the interaction herein based on the WeCom client. It is assumed that the WeCom client is running on the interacting device 3 so that the measures can be implemented.

In one embodiment, the camera 31 is mounted on the interaction device 3, and the camera 31 can form the corresponding imaging area 32 as the detection area corresponding to the interaction device 3. .. Therefore, the interacting device 3 uses the camera 31, and according to the image obtained by shooting the shooting area 32, for example, the shooting area 32 such as the user 4 entering the shooting area 32 of FIG. It is possible to determine the user who enters the. Of course, in addition to the use of the camera 31, the interacting device 3 may determine the user entering the detection area through sound detection, infrared detection, or other methods, which is described herein. Not limited to.

FIG. 4 is a schematic diagram of the execution of an interaction with an internal employee according to an exemplary embodiment. As shown in FIG. 4, the interacting device 3 can be equipped with a screen 33, which is the user image 41 corresponding to the user 4 and acquired by the camera 31. It can be configured to display. The interacting device 3 may identify the user 4 by performing facial recognition, for example, based on a facial image acquired by the camera 31, which is not limited herein. By assuming that the interacting device 3 recognizes that the user 4 is an internal employee "Xiao Byi", the corresponding identity information 42 can be displayed on the screen 33. For example, the identity information 42 may be the name "Xiao Byi" of the user 4.

During the attendance check period, the interacting device 3 can inquire about the attendance check data of the internal employee "Xiao Bai" after the user 4 recognizes that the user 4 is the internal employee "Xiao Bai", and the attendance is If it is not checked, the attendance check operation for the internal employee "Xiao Byi" can be automatically implemented. The interaction device 3 can provide the corresponding interaction content to the user 4 in order to notify the user 4 that the attendance check operation is completed. For example, the interaction content may include the label 43 shown on the screen 33, and the information contained within the label 43 is a "clock in". This informs that the type of attendance check operation is "record attendance time". The interaction content may instead be provided to the user 4 in other forms. For example, when the interaction device 3 includes the speaker 34, the speaker 34 can be used to reproduce voice information such as "Mr. Xiao Bai, the attendance time was normally recorded". can. Similarly, the interaction device 3 can implement an automatic attendance check operation for recording attendance time for other internal employees of the company AA, and the interaction device 3 is the company AA's. It is also possible to implement an automatic attendance check operation for recording leave time for employees. In the present specification, the details thereof will be omitted.

In one embodiment, the detection area of the interaction device 3 can be divided into a plurality of sub-areas according to the distance to the interaction device 3. For example, in FIG. 3, the shooting area 32 is a long-distance shooting area 321 (having a distance of 1.5 m to 3.0 m to the interacting device 3) and a short-distance shooting area 322 (0 m to 1.5 m). It has a distance to the device 3 of action). At present, if it is within the attendance check period, the interacting device 3 is irrelevant whether the user is located in the long-distance shooting area 321 or the short-distance shooting area 322. It is possible to implement an automatic attendance check operation for the user 4. If the attendance check period is within the attendance check period, but the user 4 has already checked the attendance, or if the user 4 is within another period, the interaction occurs when the user 4 is placed in the long-distance shooting area 321. By default, the device 3 temporarily passes the user 4 so that the interacting device cannot actively activate the interaction with the user 4 (ie, does not provide the interaction content to the user 4). It can be determined that it has only been done and that it has no intention of interaction. However, when the stay duration of the user 4 in the long-distance shooting area 321 reaches the first preset duration (for example, 3 seconds), the interaction device 3 is the interaction device 3. Can determine that the user 4 has an intent to interact so that may provide the interaction content to the user 4. Similarly, if the attendance check period is within the attendance check period, but the attendance has been checked, or if the user 4 is within another period, the user 4 is placed in the short-distance shooting area 322. In doing so, the interaction device 3 defaults to the user 4 so that the interaction device cannot actively initiate an interaction with the user 4 (ie, does not provide the interaction content to the user 4). It can be determined that has only passed temporarily and has no intention of interaction. However, when the stay duration of the user 4 in the short-range shooting area 322 reaches the second preset duration, the interaction device 3 is such that the interaction device causes the user to interact with the content. It can be determined that the user 4 has an intention of interaction, as can be provided to 4. Since the short-range shooting area 322 is relatively close to the interacting device 3, the behavior of the user 4 actively entering the short-range shooting area 322 is a second preset duration. However, as appropriate, the intention of the interaction may be included such that it may be less than the first preset duration. For example, the first preset duration is 3 seconds and the second preset duration is 1 second. In relatively special situations, the second preset duration may be zero, which means that when the interacting device 3 enters the short range shooting area 322, the interacting content By default, it means that the user 4 is determined to have an intent to interact so that the user 4 can be provided immediately.

When the user 4 is placed in the long-distance shooting area 321 of the interaction, the interaction device 3 and the user 4 interact to ensure effective communication and improve the interaction efficiency. The device 3 can guide the user 4 to move from the long-distance shooting area 321 to the short-distance shooting area 322 through the interaction content. For example, FIG. 5 is a schematic diagram of a guide regarding user position through interactive content according to an exemplary embodiment. As shown in FIG. 5, the interaction device 3 can display the interaction content 511 in text form within the interaction display area 51 on the screen 33. For example, the interaction content 511 "please get closer to be within 1" to guide the user 4 to move from the long-distance shooting area 321 to the short-distance shooting area 322. .5 meters) ”. In addition to the interaction content 511 in text form, the interaction device 3 uses a speaker 34 to guide the user 4 to move from the long-distance shooting area 321 to the short-distance shooting area 322. For example, "Mr. Xiao Bai, a little too far (Xiao bai, you are a little far from me)" ("Xiao Bai" is identity information, and "a little too far" is an interaction. It is possible to play interactive content in the form of audio, such as (content). In addition to this, the interacting device 3 blinks regularly to attract the user 4's attention to guide the user 4 to move from the long-distance shooting area 321 to the short-distance shooting area 322. As such, the indicator light 35 may be further controlled, which is equivalent to transmitting the interaction content to the user 4. Of course, the interacting device 3 may convey the interacting content by any of the methods such as text form, audio form, and light form, which is described herein. Not limited in the book.

When the user 4 enters the short range shooting area 322 (actively or under guidance), the interacting device 3 guides the user 4 to speak out the purpose of the user's interaction. be able to. For example, FIG. 6 is a schematic representation of the active activation of an interaction with a user by an interacting device, according to an exemplary embodiment. As shown in FIG. 6, the interaction device 3 uses the speaker 34, for example, "Mr. Xiao Bai, what kind of business do you have?" ("Xioa Bai" is the identity information. Yes, and the interaction content can be played back in the form of audio, such as "What is your requirement?" Is the interaction content). In addition to this, the interaction device 3 can also display the interaction content 512 in the form of text within the interaction display area 51. For example, the interaction content 512, in the form of audio, "try to say something like" to guide the user 4 to express the interaction purpose with respect to the interaction device 3. "this)" and "Call Zhan San (all Zhang San)" may be used.

It should be noted that the interacting device 3 does not necessarily have to guide the user 4 from the long-distance shooting area 321 to the short-distance shooting area 322. For example, the interaction device 3 can instead directly guide the user 4 in the range shooting area 321 to speak out the user's interaction objectives. For example, the interacting device 3 can further perform detection of noise from the ambient environment. When the noise level exceeds a preset value, the user 4 is first guided from the long-distance shooting area 321 to the short-distance shooting area 322, and then the user 4 sets the user's interaction purpose. You will be guided to speak. When the noise level is less than a preset value, the user 4 in the range shooting area 321 is directly guided to speak the user's interaction objectives.

In one embodiment, when User 4 is an internal employee of Company AA, by using "Mr. Xiao Bai, a little too far" shown in FIG. 5, the interaction device 3 can be The interacting device 3 may acquire the name "Xiao Bai" of the user 4 so that the user 4 can be guided from the long-distance shooting area 321 to the short-distance shooting area 322. "Is used as identity information, and" a little too far "is interactive content. In the case of an outsider of the company AA, the interacting device 3 may fail to obtain the corresponding name. Therefore, the identity information used in the interaction process may differ from that in the embodiment shown in FIG. For example, FIG. 7 is another schematic of a guide regarding user location through interactive content, according to an exemplary embodiment. As shown in FIG. 7, it is assumed that the interacting device 3 acquires the user image 71 of the user by using the camera 31, but the user is an outsider of the company AA. Yes, and the interacting device 3 fails to acquire the user's name. Therefore, when guiding the user to move from the long-distance shooting area 321 to the short-distance shooting area 322, the interaction device 3 provides the interaction content 513 in the form of text within the interaction display area 51. Can be displayed. For example, the interaction content 513 is "Please approach within 1.5 meters". Instead of this, the interacting device 3 uses a speaker 34, for example, in a voice form, such as "Hello, a little too far" (user identity information omitted). Interaction content can also be played. Instead, the interacting device 3 can also control the indicator light 35 to blink regularly to guide the user to move from the short range shooting area 322 to the long distance shooting area 321. ..

In one embodiment, the interaction device 3 can know the relevant event of the user in the detection area by accessing the WeCom server, and the interaction to be provided based on the associated event. You can change the content. For example, FIG. 8 is a schematic diagram of an interaction scenario under normal circumstances according to an exemplary embodiment. As shown in FIG. 8, the interacting device 3 detects a user placed in the shooting area 32 during working hours, and the user 4 is an internal employee of the company AA “Xiao Byi”. By assuming that the device 3 of the interaction determines that the purpose of the interaction of the user 4 is to record the leaving time, but the device of the interaction is still within the working hours. When 3 cannot find the associated event associated with the user 4's record of leaving time, the interaction content 515 shown in the interaction display area 51 says "Are you going to leave early?" to leave early?) ”. FIG. 9 is a schematic representation of the regulation of interacting content in response to related events according to an exemplary embodiment. As shown in FIG. 9, the interaction device 3 detects the user 4 arranged in the shooting area 32 during working hours, and the user 4 is an internal employee of the company AA “Xiao Byi”. By assuming that the interaction device 3 finds that the user 4 has a submitted illness leave examination and approval event, and the illness departure examination and approval event. If it is within the sick leave time approved in, the interaction content 514 shown in the interaction display area 51 is "are you sure to lock out?". It may be there.

In one embodiment, a plurality of users may exist in the shooting area 32 of the interaction device 3, and the interaction device 3 communicates with these users by using appropriate interaction content. Can be implemented. FIG. 10 is a schematic diagram of the designation of the speaker by the interacting device according to one exemplary embodiment. As shown in FIG. 10, by assuming that there are a plurality of users in the detection area 32 corresponding to the user images 81 to 82 shown on the screen 33, the interaction device. 3 may recognize the user corresponding to the user image 81 as "Xiao Byi", may recognize the user corresponding to the user image 82 as "Xiao Hei", and may recognize the user corresponding to the user image 82 as "Xiao Hei". The user's name may be displayed around as identity information. For example, the corresponding user identity information 91 shown above the user image 81 has the name "Xiao Byi", and the corresponding user identity information 92 shown above the user image 82 has The name is "Xiao Hei". Due to the limited interaction capacity of the interaction device 3, and to allow the interaction device 3 to have a clear understanding of the user's interaction objectives, the interaction device 3 is at once. , Can interact with only a portion of the user. The interaction device 3 is based on the ascending order of the distance between the user and the interaction device 3, the ascending order of the angle between the user's face and the shooting direction of the camera 31, the descending order of the user's height, and the like. , The target interaction object (part of the user) may be selected by a particular method, which is not limited herein. By assuming that the interaction device 3 is intended to interact with the user "Xiao Byi" corresponding to the user image 81, the interaction device 3 is intended to avoid misunderstandings of other users in the detection area. When the device 3 provides the interaction content, it is necessary to indicate to the user in the detection area that the target interaction object corresponding to the interaction content is the user "Xiao Byi". For example, when the interaction device 3 uses the speaker 34 to play the interaction content "what is the requirement?", The interaction content "what is the requirement?" For other users. So that it becomes clear that the target interaction object of "?" Is "Xiao Byi", so that the actual content to be played can be "Mr. Xiao Byi, what kind of business do you have?" In addition, the identity information of the user "Xiao Byi" can be reproduced.

FIG. 11 is another schematic diagram of the designation of the speaker by the interacting device according to one exemplary embodiment. As shown in FIG. 11, when a plurality of users such as the user "Xiao By" and the user "Xiao Hei" simultaneously speak the intention of each interaction, the interaction device 3 makes a sound. Due to the state of confusion, it may not be possible to accurately know the interaction objectives of each user, or the interaction device 3 may not be able to respond to the interaction objectives of multiple users at the same time. Alternatively, there are other reasons such that the interaction device 3 may provide interaction content to guide these users so that their interaction objectives are expressed in turn. For example, the interaction device 3 can display the interaction content 516 within the interaction display area 51. The interaction content 516 can include "please speak one by one". Further, in the interaction device 3, according to the above embodiment, the user's utterance order is such that the user "Xio Bai" speaks first, and the user "Xiao Hei" speaks later. Upon determination, the interaction device 3 is, in addition to this, the actual playback so that other users know that the target interaction object of the interaction content is the user "Xiao Byi". By using the speaker 34, the content can be "I can't hear clearly. Mr. Xiao Bai, please speak first" (I cannot heat clear. Xiao Bai, please spak first). Playing the identity information of the user "XiaoBai" while playing the interaction content "I can't hear clearly. xx, please speak first (I cannot hair clear. Xx, please spak first)". Can be done.

In addition to the voice forms in the embodiments shown in FIGS. 10 and 11, the utterance order of the plurality of users may be specified by a plurality of other methods. For example, FIG. 12 is yet another schematic diagram of speaker designation by an interacting device according to an exemplary embodiment. As shown in FIG. 12, in the interaction device 3, the user's utterance order is such that the user "Xiao Byi" speaks first, and the user "Xiao Hei" speaks later. Upon determining that, the interaction device 3 can mark the user image 81 corresponding to the user "Xiao Byi", for example, by adding the marking box 810 to the face area. As a result, even if the interaction content is "what do you want to do?", "Speak", and similar, the user will still see that the target interaction object of the interaction content is the user. It can be known that it is "Xiao Byi". Naturally, in the embodiment shown in FIG. 12, when the interaction device 13 displays the interaction text 517 within the interaction display area 51, the interaction text 517 is the interaction text. In addition to the interaction content "xx, please talk", the user knows that the current target interaction object is the user "Xiao Byi" so that the full content of 517 is "Mr. Xiao Bai, please talk". It further contains the identity information of the user "Xiao Byi" that can be notified to.

FIG. 13 is a schematic diagram of the utterance order of an outsider according to an exemplary embodiment. As shown in FIG. 13, the users corresponding to the user images 81 and 82 are all assumed to be outsiders of the company AA, and the interacting device 3 is of these users. Although the name cannot be obtained, the user's identity information is expressed by other methods in order to specify the target interaction object corresponding to the interaction information and to specify the target interaction object corresponding to the interaction information. be able to. For example, when the interaction device 3 determines that the target interaction object is a user corresponding to the user image 81, the user image 81 corresponds to a female user and the user image 82 is a male user. The user's identity information can be expressed through gender, such as "female (this lady)" or "male (this gentleman)". Therefore, by using the speaker 34, when the audio content played by the interacting device 3 is "I cannot hear clearly. Please speak first from the female", the inside of the shooting area 32. Both users can determine that the interacting content is "not clearly audible. xx, please speak first" and target based on the identity information "female". It can be determined that the interaction object is the user corresponding to the user image 81.

When the interaction content is provided to the user in the shooting area 32, in some situations, the user, for example, the interaction content "successfully recorded work hours" in the embodiment shown in FIG. You don't have to respond to the interaction content of. In some other situations, the user can respond to the interacting content, and the response is an event triggered by the user so that the interacting device 3 assists the user in completing the corresponding event. Can include support requests. For example, in the case of the interaction content "Record leaving time?" In the embodiment shown in FIG. 9, when the response returned by the user "Xiao Byi" is "Yes", the interaction is mutual. The action device 3 is based on semantic analysis by the user "Xiao Bai" to "attend work attendance" so that the interaction device 3 can assist in completing the "attendance check event for attendance record". It can be determined that the event support request for the "check event" has been activated. Similarly, for example, in a multi-user scenario, the interacting device 3 is returned by the female user after transmitting the voice in FIG. 13, "I can't hear clearly. Please speak first, from the female." If the response is "Call Mr. Baibai", the interaction device 3 will allow the interaction device 3 to call the user "Baibai" to assist in completing the "call event". Based on the semantic analysis, it can be determined that the user "XiaoBai" has activated the event support request for the "call event" and the object to be called is the user "Baibai".

Not surprisingly, in addition to the status of the response to the interacting content, the user within the shooting area 32 can launch the event support request directly to the interacting device 3 as well as the interacting device. 3 can support the completion of the corresponding event. This is similar to the situation of response to an interaction context and therefore, the repetition of the details is omitted herein.

In the embodiments shown in FIGS. 10-10, the interaction device 3 is an event support individually invoked by the user so that the interaction device 3 individually assists in the completion of the corresponding event. It is possible to ensure that a plurality of users in the shooting area 32 speak in turn so that the request can be determined. In one embodiment, the interacting device 3 may receive a plurality of user voices transmitted from a plurality of users at the same time, and can accurately separate the plurality of user voices based on the sound characteristics. , The interaction device 3 can simultaneously recognize the event support requests of a plurality of users and support the completion of the corresponding event, thereby greatly improving the support efficiency for the plurality of users. In addition, through sound feature recognition (eg, voiceprint recognition), the mapping relationship between each user's voice and the user in the shooting area 32 can be further determined.

In one situation, the interacting device 3 may directly compare the acquired user voice with the sound feature database, which corresponds to the acquired user voice, for example, based on the comparison result. It may include the voiceprint characteristics of all internal employees within the company AA so as to determine which internal employee to do. In addition to this, the interaction device 3 can identify the user in the shooting area 32 by other methods such as face recognition and prevent the user from pretending to be an internal employee of the company AA. , The identification result can be compared with the comparison result obtained based on the sound feature database. For example, the user voice is determined to be from the user A and the user B in the company AA, respectively, through the sound feature, but the user in the shooting area cannot be recognized as the user A based on the face recognition. Suppose it is determined to be an outsider. As a result, an outsider may pretend to be User B and may be using the recorded voice of User B. In this situation, the interacting device 3 can refuse to complete the corresponding support event and can send a warning to User B.

In another situation, the interacting device 3 first recognizes the user in the shooting area 32 as the user A and the user B in the company AA by a method such as face recognition. Can be identified. Then, when the interacting device 3 acquires two user voices, the interacting device does not compare the user voice with the other voiceprint features in the sound feature database, and which user voice is the user A. In order to determine which user voice is from user B, the user voice is simply compared with the voiceprint characteristics of user A and user B, thereby greatly improving the comparison efficiency. be able to.

In addition to this, the user in the shooting area 32 may include an administrator, so that the interacting device 3 responds to the administrator's event support request, for example, in the welcome speech content on the screen 33. It can assist in completing the corresponding device management event, such as adjusting, adjusting the volume of the speaker 34, or adjusting the range of the range shooting area 321 and the range shooting area 322.

FIG. 14 is a schematic diagram of the marking of interacting objects according to an exemplary embodiment. By using the camera 31, after photographing the imaging area 32, the interacting device 3 allows the user to interact with the interacting device 3 as well as whether the user has been detected by the interacting device 3. Detected users located within the imaging area 32 can be marked so as to clearly determine if they can act. As shown in FIG. 14, when detecting the presence of a user in the shooting area 32, the interacting device 3 generates a corresponding profile image 1401 for the user according to the shot image. At the same time, the profile image 1401 can be displayed on the screen 33. Further, even when it is detected by the interacting device 3 that another user is arranged in the shooting area 32, the profile image 1402 corresponding to that user can be displayed on the screen 33. Similarly, when yet another user enters the shooting area 32, the interacting device 3 may display the corresponding profile image on the screen 33, and is therefore detailed herein. The repetition of the above is omitted.

When the user corresponding to the profile image 1402 leaves the shooting area 32, the interacting device 3 can delete the profile image 1402 from the screen. The situation of the other user is similar to that of this user, and therefore the repetition of the details is omitted herein.

Therefore, when observing that the screen 33 includes the profile images 1401 and 1402, the corresponding user has been detected by the interaction device 3 and the interaction device 3 as an interaction object. And can be confident that it can interact with the interacting device 3. When another user wants to interact with the interacting device 3, but cannot observe the profile image corresponding to himself on the screen 33, this means that the user enters the shooting area 32. In some cases, or the user has entered the shooting area 32, but it has not been successfully detected by the interacting device 3, and the user displays the user's profile image on the screen 33. It is informed that measures such as entering or re-entering the photographing area 32 can be taken until the time when the image is taken.

FIG. 15 is a schematic diagram of marking of a target interaction object according to an exemplary embodiment. The interaction device 3 recognizes the user "Xiao Byi" and the user "Xiao Hei" respectively arranged in the shooting area 32, and the interaction device 3 targets the user "Xiao Byi" with each other. It is assumed that it is judged as an action object. As shown in FIG. 15, the interacting device 3 has a profile corresponding to the user "Xiao Bai" in the central area of the screen 33 (relatively far from the edge of the screen 33) at normal ratios. The image 1401 can be displayed, and the profile image 1402 corresponding to the user "Xiao Hei" can be displayed in the edge area of the screen 33 at a relatively small ratio. Next, when the interaction device 3 transmits the interaction voice "What is your business?" By using the speaker 34, the interaction device 3 interacts according to the display ratio and display position of the profile images 1401 and 1402. It can be determined that the target interaction object corresponding to the action speaker is not the user "Xiao Hei" corresponding to the profile image 1402, but the user "Xiao Byi" corresponding to the profile image 1401.

As a matter of course, in addition to the configuration of the display ratio and the display position of the profile image, the profile image shows other modes of adjustment in order to discriminate the profile image corresponding to the target interaction object from the profile image of the other interaction object. This can be done for the display attributes of, which can help determine the corresponding target interaction object according to the profile image. This is not limited herein.

In addition to the method of invoking the interaction with the user in the shooting area 32 by the interaction device 3, for example, the interaction device 3 is the source of the user voice so that the user's voice meets the user's requirements. The user in the photographing area 32 can also activate the interaction with the interacting device 3, such as transmitting a user voice to the interacting device 3 in response to. The user voice may be used to respond to the interaction voice transmitted by the interaction device 3, or may be actively transmitted to the interaction device 3 by a user within the imaging area 32. , This is not limited herein.

In one embodiment, the interacting device 3 needs to determine the source user of the user voice upon receiving the user voice transmitted by the user in the shooting area 32, i.e., the shooting area 32. There may be a plurality of users at the same time in the shooting area 32 so that it is necessary to detect which of the plurality of users in the group has transmitted the user voice.

For example, FIG. 16 is a schematic diagram of a determination of a source user of user voice according to an exemplary embodiment. As shown in FIG. 16, the microphone array may be constructed within the interacting device 3, the microphone array may include a microphone 36 and a microphone 37, and the microphone 36 is relative. The microphone 37 is arranged on the left side, and the microphone 37 is arranged on the relatively right side. Therefore, when a user in the shooting area 32 transmits a user voice such as "It is necessary to reserve a conference room for 15 people (Ined to reserve a conference room for 15 persons)", the microphone 36 uses the microphone 36. If the user voice is received earlier than the microphone 37, this means that the source user of the user voice is relatively close to the microphone 36 and relatively far from the microphone 37. Is informed. Therefore, it can be determined that the source user is arranged relatively close to the left side in the shooting area 32. For example, the source user can determine that he is the user "Xiao Byi" in combination with the image acquired in FIG.

Similarly, if the microphone 37 receives the user voice earlier than the microphone 36, this means that the source user of the user voice is relatively close to and relative to the microphone 36. Notifying that you are away from. Therefore, it can be determined that the source user is arranged relatively close to the right side in the shooting area 32. For example, the source user can determine that he is the user "Xiao Hei" in combination with the image acquired in FIG. Alternatively, if instead, the microphone 36 and the microphone 37 receive the user voice simultaneously or at about the same time, this means that the source user of the user voice receives the user voice at the central point between the microphone 36 and the microphone 37. It notifies that it is located, that is, it is placed directly in front of the interacting device 3. Therefore, the source user can determine that the image is arranged at the center of the shooting area 32.

In one embodiment, the user is not placed in front of the orientation of the screen 33 and the camera 31, but behind the interaction device 3 based on the location or method of installation of the interaction device 3. It may have been done. Thus, the user may clearly not be located within the shooting area 32, and the user may be located more precisely around the interacting device 3. In this situation, when the interacting device 3 receives an audio message such as "must predict a conference room for 15 people", the audio message is not necessarily transmitted by the user in the shooting area 32. It is not the user voice, but the interference voice transmitted by the user located behind the interacting device. Therefore, it is necessary to determine the source direction of the audio message in order to prevent the interference voice from being erroneously determined as the user voice. The audio message from the front can be the user voice transmitted by the user in the shooting area 32, and the audio message from the rear is the interference voice.

For example, FIG. 17 is a schematic diagram of determining the source direction of an audio message according to an exemplary embodiment. As shown in FIG. 17, the microphone array may be constructed within the interacting device 3, and the microphone array may include a microphone 36, a microphone 37, and a microphone 38. In the left-right direction (ie, the horizontal direction in FIG. 17), the microphone 36 is disposed on the left side and the microphone 37 is disposed on the right side. In the anterior-posterior direction (ie, up-downward in FIG. 17), the microphone 36 and the microphone 37 are disposed within the anterior portion of the interacting device 3 that is relatively close to the video area 32. Moreover, the microphone 38 is arranged in the rear portion of the interacting device 3 relatively away from the video area 32. All of the microphones 36 to 38 are arranged in the interacting device 3. Thus, in the case of an audio message transmitted by a user placed around the device 3 of the interaction, if the user is placed in front of the device 3 of the interaction, the audio message will be of interaction. Passing through the interacting device 3 from the front of the device 3, the high frequency portion of the audio message was absorbed by the housing portion of the interacting device 3, and the microphones 36-38 received the high frequency portion of the audio message. In doing so, the strength of the high frequency signal received by the microphone 38 located within the rear portion of the interacting device 3 is due to the absorption of the housing of the interacting device 3 by each of the microphones 36 and 37. It is designed to be less than the strength of the received high frequency signal. If the user is located behind the interacting device 3, the audio message passes through the interacting device 3 from behind the interacting device 3, and the high frequency portion of the audio message interacts. Absorbed by the housing portion of the device 3 and received by each of the microphones 36 and 37 located at the front of the interacting device 3 when the microphones 36-38 receive the high frequency portion of the audio message. The intensity of the high frequency portion is set to be less than the intensity of the high frequency signal received by the microphone 38 due to the absorption of the housing of the interacting device 3.

Therefore, it is possible to determine whether the source direction of the audio message is in front of or behind the interacting device 3 according to the amount by which the microphones 36-38 receive the high frequency portion of the audio message. When it is determined that the source direction of the audio message is behind the interacting device 3, the source user of the audio message is clearly not the user in the shooting area 32. In other words, the audio message is an interference voice. When it is determined that the source direction of the audio message is in front of the interacting device 3, the source user of the audio message may be a user in the shooting area 32. As a matter of course, further determinations can be made in combination with other conditions in order to improve accuracy and reduce the probability of erroneous determinations.

In one embodiment, image acquisition can be performed by using the camera 31 on the interacting device 3, and if a user is present in the shooting area 32, an audio message will be emitted. , Can be determined to be from the user.

In one embodiment, image acquisition can be performed by using the camera 31 on the interacting device 3, and if there are multiple users within the shooting area 32, for example. Whether the opening and closing operation of the mouth is performed during the reception processing of the audio message, and whether the moment when the operation occurs is consistent with the moment when the signal change of the audio message occurs. A user having a facial movement that matches the audio message with reference to the user's facial movement, such as, is determined as the source user of the audio message.

In one embodiment, image acquisition can be performed by using the camera 31 on the interacting device 3, and if there are multiple users in the shooting area 32, the microphone. The user corresponding to the corresponding direction is determined as the source user of the audio message by referring to whether the source direction of the audio message determined by 36 and 37 is the left side, the right side, or the middle. can do. If there are more than one user in the same direction, the user's facial actions can be referenced to select users with facial movements that match the audio message and as the source user for the audio message. It can be determined.

FIG. 18 is a schematic diagram of a user voice source user marking according to an exemplary embodiment. As shown in FIG. 18, by assuming that the user "Xiao Byi" is determined as the source user of the user voice, the user in the shooting area 32 does not generate an error in the subsequent interaction process. To ensure, the interaction device 3 displays the profile image 1401 in the original colorful mode so as to quickly determine if the interaction device 3 accurately recognizes the source user of the user voice. At the same time, the profile 1402 corresponding to other users and the like can be monochromeized for display.

In one embodiment, when the interacting device 3 recognizes the user voice, all of the unfavorable factors such as source user accents, excessive noise in the external environment, or distortions that occur in the sound pickup process. , Affects the accuracy of the semantic recognition of the interacting device 3. Therefore, in the recognition process, the interaction device 3 can score each candidate semantic recognition result. The interaction device 3 can directly discard candidate semantic recognition results whose confidence level is relatively low (eg, less than a preset score). If the number of candidate semantic recognition results with a relatively high confidence level (eg, greater than a preset score) is 1, then the candidate semantic recognition results can be used directly as the semantic recognition results. can. If the number of candidate semantic recognition results with a relatively high confidence level is greater than one, the interacting device 3 asks the source user to select and confirm, for example, the source user "Xiao Bai". Their confidence levels are relatively high, such as option 1801 "must reserve a conference room for 1.15 people" and option 1802 "must reserve a conference room for 2.45 people" in Figure 18. Candidate semantic recognition result choices corresponding to high candidate semantic recognition results can be displayed.

For example, the user "Xiao Byi" includes content such as "first", "former", or "15" to notify the interacting device 3 that option 1801 has been selected. A confirmation voice can be sent. As a result, the interacting device 3 can determine that the semantic recognition result corresponding to the user voice is "a conference room for 15 people needs to be reserved", and for example, in a related conference room. It is possible to further respond to the user's voice, such as assisting the user "Xiao Byi" in completing the reservation.

FIG. 19 is a schematic structural diagram of an apparatus according to an exemplary embodiment. Referring to FIG. 19, at the hardware level, the device includes a processor 1902, an internal bus 1904, a network interface 1906, an internal memory 1908, and a non-volatile memory 1910, and of course. However, it may further include the hardware required for other services. Processor 1902 reads the corresponding computer program from the non-volatile memory 1910 into the internal memory 1908 and then executes the computer program in order to form a device of interaction at the logic level. Of course, in addition to the software implementation, one or more embodiments herein do not preclude other implementations, such as, for example, a logical device or software-hardware combination. In other words, the execution body of the following processing procedure is not limited to the logical unit, and may be hardware or a logical device instead.

Referring to FIG. 20, in the software implementation, the interacting device is
A detection unit 2001 configured to detect one or more users placed in the detection area, and
A providing unit 2002 configured to provide interaction content to one or more users in the detection area.
When the target interaction object of the interaction content is part of one or more users in the detection area, it is configured to display information about the target interaction object to one or more users in the detection area. First display unit 2003 and
Can be included.

By arbitrary selection, the first display unit 2003
The identity information of the user who functions as the target interaction object is determined as the information about the target interaction object, and
Display identity information to one or more users in the detection area,
It is configured as follows.

By arbitrary option, the device of interaction is
It further includes an identification unit 2004 configured to identify one or more users located within the detection area, in this case.
When the identity of the first user acting as the target interaction object is successfully identified, the identity information of the first user includes the name of the first user and acts as the target interaction object. If the identity of the second user is not successfully identified, the identity information of the second user includes visual feature description information about the second user.

By arbitrary option, the device of interaction is
It further includes a second display unit 2005 configured to display user reference information corresponding to one or more users in the detection area, in this case.
The first display unit 2003 determines the user reference information corresponding to the user functioning as the target interaction object as the information regarding the target interaction object, and determines the determined user reference information as one person in the detection area or one person in the detection area. It is configured to be prominently displayed to multiple users.

By arbitrary selection, the second display unit 2005 may be
It is configured to perform image acquisition for one or more users in the detection area and display one or more acquired user images as user reference information.

By arbitrary option, the device of interaction is
When the target interaction object of the interaction content changes from a part of one or more users in the detection area to all one or more users, the information about the target interaction object is one in the detection area. Management Unit 2006, configured to pause the steps displayed to a person or multiple users,
Further includes.

By arbitrary option, the device of interaction is
A request acquisition unit 2007 configured to acquire event assistance requests sent by users in the detection area and a support unit 2008 configured to respond to event assistance requests and assist in completing the corresponding event. ,
Further includes.

By optional option, the request acquisition unit 2007
It is configured to receive the response information returned by the user in the detection area in response to the interaction content, in which case the response information includes an event support request.

By optional option, the request acquisition unit 2007
When there are multiple users in the detection area, the user is selected as the supported object according to the preset order, and the user is selected.
Request the selected users in turn to send the corresponding event support request,
It is configured as follows.

By optional option, the request acquisition unit 2007
In order to acquire the event support request, semantic recognition is executed for the acquired user voice, and
Perform sound feature recognition on the user voice to determine the source user of the user voice,
It is configured as follows.

By arbitrary option, the support unit 2008
When a user in the detection area is an insider of a group, it responds to event support requests and assists in completing the corresponding group management event.
When a user in the detection area is an outsider of the group, it sends an alert message to the relevant insider of the group to assist the outsider of the group in establishing communication with the relevant insider of the group. Or, in response to an event support request, guide outsiders of the group to the processing location of the visit event, and
When the user in the detection area is an administrator, it responds to the event support request and assists in completing the corresponding device management event.
It is configured as follows.

By arbitrary selection, the detection area includes a short-distance detection area and a long-distance detection area, and the providing unit 2002 includes.
Provides interaction content to users within the short range detection area and
Send guidance information to users in the long-distance detection area to guide them to enter the short-range detection area from the long-range detection area.
It is configured as follows.

By arbitrary option, the device of interaction is
It further includes an event acquisition unit 2009 configured to acquire related events for one or more users in the detection area, in this case.
When the interaction content is associated with a related event, the serving unit 2002 is to adjust the interaction content provided to one or more users in the detection area according to the state information of the related event. It is configured in.

By arbitrary option, the device of interaction is
Further including a determination unit 2010 configured to determine the identity type of one or more users in the detection area, in this case.
The delivery unit 2002 is configured to adjust the interaction content provided to one or more users in the detection area according to the identity type.

By arbitrary option, the device of interaction is
A voice receiving unit 2011 configured to receive user voice transmitted by a user in the detection area, and
A response unit 2012 configured to respond to user voice,
Further includes.

By optional option, the response unit 2012
Perform semantic recognition on user voice,
When the confidence level of multiple semantic recognition results reaches a preset value, multiple corresponding semantic recognition result choices are displayed to the user in the detection area.
Responds to the user's voice according to the semantic recognition result corresponding to the selected semantic recognition result choice.
It is configured as follows.

By optional option, the response unit 2012
Determine the source direction of the user voice, and
Respond to the user placed in the source direction of the user's voice,
It is configured as follows.

By optional option, the response unit 2012
In the step of receiving the user voice by using the microphone array, the microphone array has a first microphone arranged on the relatively left side and a second microphone arranged on the relatively right side. Including, steps and
A step of determining the source direction of the user voice according to the difference between the time points when the first microphone and the second microphone receive the user voice, and
The source direction of the user voice is determined by the method.

By optional option, the response unit 2012
When there are multiple users placed in the source direction of the user voice, the step of determining the source user of the user voice according to the facial movement of each of the multiple users,
Steps to respond to the source user,
By this method, it responds to the user who is placed in the direction of the source of the user voice.

By arbitrary option, the device of interaction is
An audio receiving unit 2013 configured to receive audio messages by using a microphone array, the microphone array is from a third microphone disposed relatively close to the detection area and from the detection area. An audio receiving unit, including a fourth microphone disposed relatively apart, and
A directional determination unit 2014 configured to determine the source orientation of an audio message according to the amount by which the third and fourth microphones receive a high frequency portion of the audio message.
When the source direction is relatively close to the detection area and the user is in the detection area, it is determined that the audio message is the user voice transmitted by the user in the detection area. Source determination unit 2015 configured in
Further have.

By arbitrary option, the device of interaction is
When there are multiple users in the detection area, the profile image display unit 2016 configured to display the profile image corresponding to each user, and
A separate display unit 2017 configured to display the user's voice source user's profile image separately from the other user's profile images.
Further includes.

The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with specific functionality. A typical mounting device is a computer. Computers can be in the form of PCs, laptop computers, cellular phones, camera phones, smartphones, personal digital assistants, media players, navigation devices, email transceiver devices, game consoles, tablet computers, wearable devices, or any combination thereof. It may be there.

In a normal configuration, a computer includes one or more processors (such as a CPU), input / output interfaces, network interfaces, and internal memory.

Internal memory includes volatile memory, random access memory (RAM: Random Access Memory), and / or, for example, read-only memory (ROM: Read-Only Memory) or non-volatile memory in a computer-readable medium such as a flash RAM. May include the form of. Internal memory is an example of a computer-readable medium.

Computer-readable media include permanent and non-permanent media, removable and non-removable media, which may implement storage of information by using any method or technique. The information may be computer readable instructions, data structures, program modules, or other data. Examples of computer storage media are, without limitation, phase change memory (PRAM), static random access memory (SRAM: Static Random-Access Memory), dynamic random access memory (DRAM: Dynamic Random Access Memory), and the like. Types of RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or another memory technology, compact disk read-only memory (CD-ROM: Compact) Disc Read-Only Memory), Digital Versatile Disk (DVD) or another optical storage device, cassette tape, disk storage, quantum memory, graphene-based storage medium or another magnetic storage device, or any Other non-transmitting media are included, which can be configured to store information accessible by the arithmetic unit. By definition herein, a computer-readable medium does not include a temporary computer-readable medium (temporary medium) such as a modulated data signal and a carrier wave.

It should also be noted that the terms "include", "comprise", and any other variation are meant to cover non-exclusive inclusion. Thereby, a process, method, article, or device comprising a set of elements may include not only those elements, but also other elements not explicitly listed, or the process, method, article, or device. It further includes the unique elements of the device. Without further limitation, an element defined by the phrase "include one ..." is any other identical within the process, method, article, or device that contains the element. It does not exclude the element of.

In the above, some embodiments of the present specification are described. Other embodiments are also included in the appended claims. In some situations, the actions or steps recorded in the claims can be performed in a different order than in the embodiments, and the expected results can still be achieved. In addition to this, the processes depicted in the accompanying drawings do not necessarily require a particular order or sequential order to achieve the expected results. In some implementations, multitasking and parallelism can be feasible or informative.

The terms used in one or more embodiments herein are merely those used to describe the embodiments and limit one or more embodiments herein. It wasn't intended. Also, the singular forms "one (a)", "the", and "said" as used in one or more embodiments and the accompanying claims herein are. It is intended to include the plural unless other meanings are clearly communicated in the context. It is also understood that the term "and / or" as used herein includes any and all possible combinations of one or more related listed items. I want to be.

The terms "first", "second", "third", and the like, are used in one or more embodiments herein. It may be used to describe various information, but it should be understood that such information should not be limited to these terms. These terms are only used to discriminate between the same type of information. For example, the first information may be referred to as the second information without departing from the scope of one or more embodiments herein. Similarly, the second information may also be referred to as the first information. Depending on the context, the term "if" as used herein is "when ..." or "when (upon.)". It can be interpreted as "...)" or "in response to determining".

The above description is merely an exemplary embodiment of one or more embodiments herein and is not intended to limit one or more embodiments of this specification. Any changes, equal replacements, or improvements made within the spirit and principles of one or more embodiments herein are within the scope of protection of one or more embodiments herein. And.

Claims (42)

It ’s a method of interaction,
A step to detect one or more users placed in the detection area,
The step of providing the interaction content to the one or more users in the detection area, and
When the target interaction object of the interaction content is part of the one or more users in the detection area, information about the target interaction object is given to the one or more users in the detection area. Steps to display and
How to have. The step of displaying information about the target interaction object to the one or more users in the detection area is
A step of determining the identity information of a user functioning as the target interaction object as the information regarding the target interaction object, and
A step of displaying the identity information to the one or more users in the detection area,
The method according to claim 1. Further comprising a step of identifying the one or more users located within the detection area.
When the identity of the first user functioning as the target interaction object is successfully identified, the identity information of the first user has the name of the first user and the target interaction. The method of claim 2, wherein the identity information of the second user has visual feature description information about the second user when the identity of the second user acting as an object is not successfully identified. Further having a step of displaying user reference information corresponding to the one or more users in the detection area.
The step of displaying information about the target interaction object to the one or more users in the detection area presents user reference information corresponding to the user acting as the target interaction object to the target interaction object. The method according to claim 1, further comprising a step of determining as the information regarding the above, and a step of displaying the determined user reference information prominently to the one or more users in the detection area. The step of displaying the user reference information corresponding to the one or more users in the detection area is
4. The fourth aspect of the present invention includes a step of executing image acquisition for the one or more users in the detection area, and a step of displaying one or a plurality of acquired user images as the user reference information. The method described. When the target interaction object of the interaction content changes from said portion of said one or more users in the detection area to all of said one or more users, said target interaction object. The method of claim 1, further comprising a step of suspending the step of displaying the information about the above to the one or more users in the detection area. The step of acquiring the event support request sent by the user in the detection area, and
In response to the event support request, a step to support the completion of the corresponding event, and
The method according to claim 1, further comprising. The step of acquiring the event support request transmitted by the user in the detection area is
A step of receiving response information returned by the user in the detection area in response to the interaction content, wherein the response information has the event support request.
The method according to claim 7. The step of acquiring the event support request transmitted by the user in the detection area is
When there are multiple users in the detection area, the step of selecting the user as the object to be supported according to the preset order, and
The step of sequentially requesting the selected user to send the corresponding event support request, and
The method according to claim 7. The step of acquiring the event support request transmitted by the user in the detection area is
In order to acquire the event support request, a step to be executed for the user voice for which semantic recognition has been acquired, and
A step of performing sound feature recognition on the user voice to determine the source user of the user voice,
The method according to claim 7. The step of responding to the event support request and assisting in completing the corresponding event is
When the user in the detection area is an insider of the group, a step of supporting the completion of the corresponding group management event in response to the event support request, and
When the user in the detection area is an outsider of the group, an alert message is sent to the related insider of the group to establish communication with the related insider of the group. A step of assisting the outsider or guiding the outsider of the group to the processing location of the visit event in response to the event support request.
When the user in the detection area is an administrator, a step of supporting the completion of the corresponding device management event in response to the event support request, and
The method according to claim 7. The detection area has a short-distance detection area and a long-distance detection area, and the step of providing the interaction content to the one or more users in the detection area is a step.
The step of providing the interaction content to the user in the short range detection area,
A step of transmitting guidance information to a user in the long-distance detection area in order to guide the user so as to enter the short-distance detection area from the long-distance detection area.
The method according to claim 1. Further comprising the step of acquiring the relevant event of the one or more users in the detection area.
The step of providing the interaction content to the one or more users in the detection area, according to the state information of the relevant event when the interaction content is associated with the relevant event. The method of claim 1, comprising the step of adjusting the interaction content provided to the one or more users in the detection area. Further comprising a step of determining the identity type of the one or more users in the detection area.
The step of providing the interaction content to the one or more users in the detection area adjusts the interaction content provided to the one or more users in the detection area according to the identity type. The method of claim 1, wherein the method comprises the steps to be performed. The step of receiving the user voice transmitted by the user in the detection area,
The step to respond to the user voice and
The method according to claim 1, further comprising. The step in response to the user voice
The step of performing semantic recognition on the user voice,
A step of displaying a plurality of corresponding semantic recognition result choices to the user in the detection area when the confidence level of the plurality of semantic recognition results reaches a preset value.
A step that responds to the user voice according to the semantic recognition result corresponding to the selected semantic recognition result option.
The method according to claim 15. The step in response to the user voice
The step of determining the source direction of the user voice and
A step that responds to a user placed in the source direction of the user voice, and
The method according to claim 15. The step of determining the source direction of the user voice is
In the step of receiving the user voice by using the microphone array, the microphone array includes a first microphone arranged on the relatively left side and a second microphone arranged on the relatively right side. , With steps and
A step of determining the source direction of the user voice according to the difference between the time points when the first microphone and the second microphone receive the user voice.
17. The method according to claim 17. The step in response to a user placed in the source direction of the user voice is
When there are a plurality of users arranged in the source direction of the user voice, a step of determining the source user of the user voice according to the facial movement of each of the plurality of users.
The steps to respond to the source user and
17. The method according to claim 17. By using the microphone array, in the step of receiving an audio message, the microphone array is relatively separated from the detection area with a third microphone arranged relatively close to the detection area. With a fourth microphone disposed of, and a step.
A step of determining the source direction of the audio message according to the amount by which the third microphone and the fourth microphone receive a high frequency portion in the audio message.
The audio message is transmitted by the user in the detection area when the source direction is a side portion relatively close to the detection area and a user is present in the detection area. The step to determine that it is the user's voice,
The method according to claim 15, further comprising. When there are a plurality of users in the detection area, the profile image of the source user of the user voice is separated from the step of displaying the profile image corresponding to the user and the profile image of the other user. Steps to display and
The method according to claim 15, further comprising. It ’s an interaction device,
A detection unit configured to detect one or more users located within the detection area, and
A providing unit configured to provide the interacting content to the one or more users in the detection area.
When the target interaction object of the interaction content is part of the one or more users in the detection area, information about the target interaction object is given to the one or more users in the detection area. The first display unit configured to display and
A device with. The first display unit is
The identity information of the user functioning as the target interaction object is determined as the information regarding the target interaction object, and the information is determined.
Displaying the identity information to the one or more users in the detection area.
22. The apparatus of claim 22. Further having an identification unit configured to identify the one or more users located in the detection area.
When the identity of the first user functioning as the target interaction object is successfully identified, the identity information of the first user has the name of the first user and the target interaction. 23. The device of claim 23, wherein the identity information of the second user has visual feature description information about the second user when the identity of the second user acting as an object is not successfully identified. Further having a second display unit configured to display user reference information corresponding to the one or more users in the detection area.
The first display unit determines the user reference information corresponding to the user functioning as the target interaction object as the information regarding the target interaction object, and determines the determined user reference information in the detection area. 22. The device of claim 22, which is configured to be prominently displayed to the one or more users of the above. The second display unit is
Image acquisition is performed for the one or more users in the detection area, and one or more acquired user images are displayed as the user reference information.
25. The apparatus of claim 25. With respect to the target interaction object when the target interaction object of the interaction content changes from said portion of said one or more users in the detection area to all of said one or more users. 22. The apparatus of claim 22, further comprising a management unit configured to suspend the step of displaying the information to the one or more users in the detection area. A request acquisition unit configured to acquire an event support request sent by a user in the detection area, and a request acquisition unit.
A support unit configured to respond to the event support request and assist in the completion of the corresponding event.
22. The device according to claim 22. The request acquisition unit is
28. The device of claim 28, which is configured to receive response information returned by the user in the detection area in response to the interaction content, wherein the response information is the event support request. The request acquisition unit is
When a plurality of users exist in the detection area, the user is selected as a supported object according to a preset sequence, and the user is selected.
Requesting the selected users in turn to send the corresponding event support request,
28. The apparatus of claim 28. The request acquisition unit is
In order to acquire the event support request, the semantic recognition is executed for the acquired user voice, and
Sound feature recognition is performed on the user voice to determine the source user of the user voice.
28. The apparatus of claim 28. The support unit is
When the user in the detection area is an insider of the group, it assists in completing the corresponding group management event in response to the event support request.
When the user in the detection area is an outsider of the group, an alert message is sent to the related insider of the group to establish communication with the related insider of the group. In response to the event support request, or in response to the event support request, the outsider of the group is guided to the processing place of the visit event, and
When the user in the detection area is an administrator, the device management event corresponding to the event support request is assisted in completing the corresponding device management event.
28. The apparatus of claim 28. The detection area has a short-distance detection area and a long-distance detection area, and the providing unit includes.
The interaction content is provided to the users in the short-range detection area, and
Guidance information is transmitted to the user in the long-distance detection area in order to guide the user so as to enter the short-distance detection area from the long-distance detection area.
22. The apparatus of claim 22. Further having an event acquisition unit configured to acquire the related event of the one or more users in the detection area.
The providing unit provides the interaction to the one or more users in the detection area according to the state information of the related event when the interaction content is associated with the related event. 22. The device of claim 22, which is configured to adjust the content. Further having a determination unit configured to determine the identity type of the one or more users in the detection area.
22. The device of claim 22, wherein the providing unit is configured to regulate the interacting content provided to the one or more users in the detection area according to the identity type. A voice receiving unit configured to receive user voice transmitted by a user in the detection area.
A response unit configured to respond to the user voice,
22. The device according to claim 22. The response unit is
Perform semantic recognition on the user voice and
When the confidence level of the plurality of semantic recognition results reaches a preset value, the plurality of corresponding semantic recognition result options are displayed to the user in the detection area, and the plurality of semantic recognition result options are displayed to the user in the detection area.
Responding to the user voice according to the semantic recognition result corresponding to the selected semantic recognition result choice.
36. The apparatus of claim 36. The response unit is
The source direction of the user voice is determined, and
Responding to a user placed in the source direction of the user voice,
36. The apparatus of claim 36. The response unit is
In the step of receiving the user voice by using the microphone array, the microphone array includes a first microphone arranged on the relatively left side and a second microphone arranged on the relatively right side. , With steps and
A step of determining the source direction of the user voice according to the difference between the time points when the first microphone and the second microphone receive the user voice.
38. The apparatus according to claim 38, wherein the source direction of the user voice is determined by the above-mentioned method. The response unit is
When there are a plurality of users arranged in the source direction of the user voice, a step of determining the source user of the user voice according to the facial movement of each of the plurality of users.
The steps to respond to the source user and
38. The device according to claim 38, which is arranged in the direction of the source of the user voice and responds to the user. An audio receiving unit configured to receive an audio message by using a microphone array, wherein the microphone array includes a third microphone disposed relatively close to the detection area and the detection. An audio receiver unit, comprising a fourth microphone disposed relatively away from the area.
A direction determination unit configured to determine the source direction of the audio message according to the amount by which the third microphone and the fourth microphone receive a high frequency portion in the audio message.
The audio message was transmitted by the user in the detection area when the source direction was a side portion relatively close to the detection area and the user was present in the detection area. A source determination unit configured to determine that it is a user voice,
36. The device according to claim 36. When there are a plurality of users in the detection area, a profile image display unit configured to display a profile image corresponding to the user, and a profile image display unit, respectively.
A separate display unit configured to display the profile image of the source user of the user voice separately from the profile image of the other user.
36. The device according to claim 36.

Images