JP6250852B1 - Determination program, determination apparatus, and determination method - Google Patents

Abstract

To perform session management with high usability. A determination program according to the present application causes a computer to execute a collection procedure and a determination procedure. The collection procedure collects ambient environmental sounds. The determination procedure determines whether or not the predetermined action of the user is included in one session based on the continuity of the environmental sound collected by the collection procedure. For example, the collection procedure collects ambient sound that is constantly emitted, and the determination procedure determines that a predetermined action of the user is collected in one session while the ambient sound that is regularly emitted is collected. It is determined that it is included. [Selection] Figure 1

Description

  The present invention relates to a determination program, a determination apparatus, and a determination method.

  In communication such as the Internet, a series of user actions (sessions) is acquired as session information by a server or the like. For example, the server refers to a user cookie or the like, and determines that a series of actions at a certain shopping site is performed by a certain user (in other words, a terminal device operated by the user).

  As a technique related to a session, for example, in a virtual private network connecting a client and a server, a technique for preventing resource information depletion at low cost while ensuring stable communication is known.

JP 2005-110302 A

  By the way, when a user makes a purchase at a shopping site or the like using voice recognition technology, it may be difficult to recognize a series of actions intended by the user as one session. For example, in voice recognition, when a voice uttered by a user is interrupted, the session may be closed assuming that the user's action has ended. That is, the user may be required to rephrase or be asked to log in to the shopping site again if the intended voice is not continuously emitted at once. In this case, the user has to spend time and effort to perform an unintended extra voice input. As described above, there is a current situation that session management with high usability is not performed when a speech recognition technology is used on a network.

  The present application has been made in view of the above, and an object thereof is to provide a determination program, a determination apparatus, and a determination method capable of performing session management with high usability.

  The determination program according to the present application determines whether or not a predetermined action of a user is included in one session based on a collection procedure for collecting ambient environmental sounds and a continuity of environmental sounds collected by the collection procedure. A determination procedure for determining is executed by a computer.

  According to one aspect of the embodiment, there is an effect that session management with high usability can be performed.

FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of a determination processing system according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of the determination apparatus according to the embodiment. FIG. 4 is a diagram illustrating an example of the session storage unit according to the embodiment. FIG. 5 is a diagram illustrating an example of an action table according to the embodiment. FIG. 6 is a diagram illustrating an example of an effect table according to the embodiment. FIG. 7 is a diagram illustrating a configuration example of a user terminal according to the embodiment. FIG. 8 is a flowchart illustrating a processing procedure according to the embodiment. FIG. 9 is a diagram illustrating a configuration example of a determination apparatus according to a modification. FIG. 10 is a diagram illustrating an example of a registration table according to the modification. FIG. 11 is a diagram illustrating an example of a collation table according to the modification. FIG. 12 is a flowchart illustrating a processing procedure according to the modification. FIG. 13 is a hardware configuration diagram illustrating an example of a computer that realizes the function of the determination apparatus.

  Hereinafter, modes for carrying out a determination program, a determination device, and a determination method according to the present application (hereinafter referred to as “embodiments”) will be described in detail with reference to the drawings. Note that the determination program, the determination apparatus, and the determination method according to the present application are not limited to the embodiment. In addition, the embodiments can be appropriately combined within a range that does not contradict processing contents. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

[1. Example of judgment process)
First, an example of the determination process according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a determination process according to the embodiment. FIG. 1 illustrates a flow in which the determination process according to the embodiment is performed by the determination apparatus 100 that is a server apparatus that operates according to the determination program according to the present application. More specifically, FIG. 1 shows an example in which the determination apparatus 100 according to the present application performs processing for determining whether or not a predetermined action of a user on the network is included in one session.

  In the embodiment, the session indicates a series of user actions on the network. For example, a session means a period during which a user performs an action with a predetermined purpose (intention). A session means that the series of actions are performed by the same user.

  For example, the session may be defined in a predetermined time (for example, 30 minutes) set in advance, or may be defined as a period from when the user logs in to the shopping site until the user logs off. Further, the session may be defined as a period from when the user logs in to the shopping site until a predetermined action (for example, completion of an ordering procedure for an arbitrary product) is performed. Alternatively, if a next action is performed before a predetermined time (for example, 5 minutes) has elapsed since a certain action was performed, the session is determined to be continued and the next action is performed. If not, it may be defined so that it is determined that the session has ended. Specifically, the determination apparatus 100 may determine that the session is continued when the user continues the operation on the user terminal 10 without leaving an interval of 5 minutes or more. In addition, the determination apparatus 100 may start a new session, assuming that the session is ended when there is no user operation for 5 minutes or more after the user's operation was last performed during the continuation of the session. .

  In the embodiment, the determination apparatus 100 recognizes a voice transmitted from a user and provides a predetermined service. In the example of FIG. 1, the determination apparatus 100 provides a shopping service that recognizes a transmitted voice and receives an order for a predetermined product. Such a service may be provided not by the determination device 100 but by a predetermined server (for example, a web server) that communicates with the determination device 100.

  For example, when the voice transmitted from the user includes an arbitrary product name and a word intended to be ordered, the determination apparatus 100 recognizes that the product is ordered from the user. Specifically, when the determination apparatus 100 recognizes the voice as “buy rice”, it recognizes “rice” as an arbitrary product name, and “buy”. Recognize as That is, the determination apparatus 100 recognizes that the user has ordered rice in the shopping service, and performs a procedure process related to the order. Note that various known techniques can be used for speech recognition, and thus description thereof is omitted.

  In general, in the case of an order that does not use voice recognition, a server that provides a service determines a user's session based on predetermined identification information. For example, the server that provides the service acquires the cookie information of the user U1 who has accessed the shopping page, and determines the session based on the cookie information. As a result, the server that provides the service performs actions such as logging in to a shopping site, putting rice in a cart, and ordering rice in a cart, by the user U1. It can be determined that this is a series of actions.

  On the other hand, when an order using voice recognition is performed, the server that provides the service may not be able to determine whether or not the predetermined action of the user is included in one session. For example, it is assumed that the voice is interrupted for a predetermined time due to some other action after the user says “rice”. Then, it is assumed that the user utters “Buy” after a predetermined time. In this case, it is difficult for the server providing the service to determine whether or not the utterance of the previous user and the utterance performed later are included in one session. Specifically, the server providing the service cannot recognize the product name corresponding to the command “Buy.”. For this reason, the user inputs voice again. In some cases, the user may be determined to have completed the session and asked to log in again. As described above, when the session cannot be determined appropriately, the service side may reduce the usability of the service.

  Therefore, the determination apparatus 100 according to the embodiment determines whether or not the predetermined action of the user is included in one session by the determination process according to the embodiment. Specifically, the determination apparatus 100 collects environmental sounds around the user via the user terminal 10 that is a terminal apparatus used by the user. And the determination apparatus 100 determines whether a user's predetermined | prescribed action is contained in one session based on the continuity of the collected environmental sound. Thereby, since the determination apparatus 100 appropriately determines a session based on a situation where the user is placed, session management with high usability can be performed. Hereinafter, the determination process according to the embodiment will be described along the flow with reference to FIG. 1.

  A user terminal 10 shown in FIG. 1 is an information processing terminal used by a user. In FIG. 1, the user terminal 10 is, for example, a smartphone. In the example of FIG. 1, the user terminal 10 is used by a user U1 that is an example of a user. Hereinafter, the user may be read as the user terminal 10. For example, the description “user U1 transmits voice” may actually indicate a situation “user terminal 10 used by user U1 transmits voice”.

  In the example of FIG. 1, the user terminal 10 detects ambient environmental sounds and voice input from the user U1 (step S01). For example, the user terminal 10 detects these sounds with a microphone provided therein. In the following, information related to sounds such as environmental sounds and sounds may be collectively referred to as sound information.

  In the embodiment, the environmental sound refers to a sound other than the voice input to the user terminal 10 by the user U1 with some intention. For example, an example of the environmental sound is a sound that can be background noise in a place where the user U1 is based. Specifically, the environmental sound is an operating sound of the air conditioner 60 or a sound output from the television 70 at the home 50 of the user U1. Further, the environmental sound may be a living sound such as a sound of water flowing from the water supply at home 50 or a sound of opening and closing the door. The environmental sound may be a sound related to the natural environment. For example, the environmental sound may be an external rain sound that can be detected in the home 50, a wind sound, or the like. The environmental sound may be a sound that is artificially emitted. For example, the environmental sound may be a sound when the user U1 clicks the mouse, a sound when the keyboard is pressed, or the like.

  Further, the voice input from the user U1 refers to a voice that the user U1 inputs to the user terminal 10 with some intention. In the example of FIG. 1, the voice input from the user U <b> 1 is, for example, voice that is uttered to use a shopping service provided by the determination apparatus 100 via the user terminal 10. Note that the voice input from the user U <b> 1 may be a voice for instructing the user terminal 10. For example, a voice input from the user U1 is a voice for starting the user terminal 10, a voice for instructing access to a service provided by the determination apparatus 100, or authentication of the user terminal 10 for logging in to the service ( For example, it may be voice for receiving voiceprint authentication.

  The user terminal 10 transmits the detected sound information to the determination device 100 via the network (step S02). The determination apparatus 100 collects the transmitted sound information (step S03). In the example of FIG. 1, the determination apparatus 100 collects sound information “rice...” That is a voice input of the user U <b> 1 along with environmental sounds around the user terminal 10.

  It is assumed that a predetermined time has elapsed after step S03 (step S04). For example, after the user U1 utters the voice “rice ...”, the user U1 browses a memo stored in the user terminal 10 to confirm whether there is any other product to be purchased, and thus for a predetermined time. It is assumed that (for example, several minutes) has elapsed. Further, during this time, it is assumed that the user U1 is browsing the memo displayed on the user terminal 10 without making a voice.

  Through step S04, the user terminal 10 further detects ambient environmental sound and voice input from the user U1 (step S05). In step S <b> 05, the user terminal 10 detects a voice input “buy” of the user U <b> 1. Further, it is assumed that the user terminal 10 detects environmental sounds such as a sound of operating the air conditioner 60 and a sound output from the television 70, as in steps S01 to S05.

  Similarly to step S02, the user terminal 10 transmits the detected sound information to the determination device 100 (step S06). The determination apparatus 100 collects the transmitted sound information.

  And the determination apparatus 100 determines whether a user's predetermined | prescribed action is contained in one session based on the continuity of the collected environmental sound. Specifically, the determination apparatus 100 determines a series of actions performed under a continuous environmental sound as one session (step S07). In the example of FIG. 1, the determination device 100 emits “step S01” when there is continuity between the environmental sound detected in step S01 and the environmental sound detected in step S05 via step S04. It is determined that the voice input “rice ...” and the voice input “buy.” Issued in step S05 are included in one session.

  When the determination apparatus 100 determines that the voice input is included in one session, the determination apparatus 100 recognizes a word intended for an order “buy.”, And is issued as the product name of the order in “S01”. "Rice ..." That is, the determination apparatus 100 has a predetermined time between the action of voice input performed in step S01 and the action of voice input performed in step S05, and some input is performed on the determination apparatus 100. Even if not, the session of the user U1 is maintained based on the continuity of the environmental sound.

  Although the continuity of environmental sounds will be described in detail later, for example, the determination apparatus 100 collects ambient environmental sounds that are constantly emitted and detected by the user terminal 10 every few seconds, and the collected environmental sound sounds. When the pressure, frequency, waveform or the like does not exceed a predetermined threshold, it is determined that the continuity of the environmental sound is maintained. Note that the determination apparatus 100 estimates the direction in which sound is emitted, the distance to the sound source, and the like, and determines that the continuity of the environmental sound is maintained when the estimated information does not exceed a predetermined threshold. Good.

  As described above, the determination apparatus 100 according to the embodiment collects ambient environmental sounds, and determines whether or not the predetermined action of the user U1 is included in one session based on the continuity of the collected environmental sounds. To do.

  That is, the determination apparatus 100 determines, based on the environmental sound, that the plurality of actions performed on the service using voice recognition are a series of actions performed by the same user, user U1. Specifically, even if the sound transmitted from the user U1 is interrupted in the middle, the determination device 100 may move or take some other action if the environmental sound is continuous. It is determined that it is merely waiting for an input to the user terminal 10. Moreover, the determination apparatus 100 determines that there is no change in the user who uses the user terminal 10 when the environmental sound continues after the user U1 logs in. Thus, the determination apparatus 100 can continue to maintain the session by using the environmental sound even if the user U1 does not continuously emit the intended sound at once. As a result, the user U1 can save the trouble of performing re-login due to the session being interrupted or performing re-speech input. As a result, the determination apparatus 100 can perform session management with high usability when the user U1 uses voice recognition technology on the network.

  Moreover, according to the determination apparatus 100, in order to use the environmental sound naturally detected by the user terminal 10 for processing together with the voice input of the user U1, the user U1 and the user terminal 10 are caused to perform some processing in particular. You don't need to maintain a session. That is, since the determination apparatus 100 performs processing using the environmental sound that is naturally collected when accepting the sound from the user U1, the determination apparatus 100 performs session management with high usability without imposing an extra burden on the user U1. be able to.

  Moreover, the determination apparatus 100 may combine environmental sound and existing session management processing. For example, even when the environmental sound is continuous, the determination device 100 may end the session when a relatively long time (for example, 1 hour) has elapsed with respect to the use of the service. It is also assumed that the user U1 performs voice input to the user terminal 10 while moving. In this case, the determination device 100 may determine the priority of the continuity of the voice input of the user U1 and maintain the session if the voice input of the user U1 continues while the environmental sound is not continuous. .

  Further, the determination apparatus 100 may perform content effect measurement using the continuity of environmental sounds. For example, the user U1 may browse content (for example, advertisement content by a moving image or television commercial) via the user terminal 10 or the television 70. Then, after the user U1 browses the content, the determination apparatus 100 collects information that the user U1 took some action regarding the content while the environmental sound is continuous. For example, the user U1 may tweet “I like that product” for the product advertised by the content. The determination apparatus 100 collects such favorable tweets as voice inputs together with the environmental sound detected by the user terminal 10. Then, for example, the determination apparatus 100 associates the content with a favorable tweet of the user U1 using the product name as a key. In this case, the determination apparatus 100 determines that the content has exerted an effect on the user U1.

  In general, it is difficult to prove whether or not the content is effective for the user U1 because it is difficult to prove the causal relationship. According to the determination apparatus 100, information indicating that some action has been taken on the content is collected while the environmental sound is continuous. Therefore, the content distribution is motivated to take some action on the user U1. It can be estimated with high accuracy.

  In the example of FIG. 1, an example in which the determination apparatus 100 performs the determination process according to the embodiment is shown, but more accurately, the determination process according to the embodiment is performed by a determination program executed in the determination apparatus 100. Executed. Hereinafter, the determination apparatus 100 that executes the above-described determination process according to such a determination program will be described in detail.

[2. Judgment processing system configuration]
Next, the configuration of the determination processing system 1 including the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the determination processing system 1 according to the embodiment. As illustrated in FIG. 2, the determination processing system 1 according to the embodiment includes a user terminal 10 and a determination device 100. These various devices are communicably connected via a network N by wire or wireless. The determination processing system 1 may include a plurality of user terminals 10. That is, the user may own and use not only one user terminal 10 but also a plurality of user terminals 10.

  The user terminal 10 is an information processing terminal such as a desktop PC (Personal Computer), a notebook PC, a tablet terminal, a mobile phone including a smartphone, or a PDA (Personal Digital Assistant). The user terminal 10 also includes a wearable device that is a glasses-type or watch-type information processing terminal. Furthermore, the user terminal 10 may include various smart devices having an information processing function. For example, the user terminal 10 may include a smart home appliance such as a TV (television), a refrigerator, and a vacuum cleaner, a smart vehicle such as an automobile, a drone, and a home robot. Further, the user terminal 10 may include devices that detect voice input from the user and perform predetermined processing. For example, the user terminal 10 may be a speaker or a lighting device that operates by detecting sound.

  The user terminal 10 includes a sound collection device such as a microphone, and detects voice input from the user and environmental sounds around the user terminal 10. Note that the user terminal 10 does not have a built-in microphone, but can detect a user's voice or environmental sound by being connected to the microphone by wired or wireless communication or by inputting voice information as data. Good.

  The determination apparatus 100 is a server apparatus that collects environmental sounds around the user terminal 10 and determines whether a predetermined action of the user is included in one session based on the continuity of the collected environmental sounds. . The determination apparatus 100 executes the determination process according to the embodiment by operating the determination program according to the embodiment internally.

[3. (Configuration of judgment device)
Next, the configuration of the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the determination apparatus 100 according to the embodiment. As illustrated in FIG. 3, the determination apparatus 100 includes a communication unit 110, a storage unit 120, and a control unit 130. The determination device 100 includes an input unit (for example, a keyboard and a mouse) that receives various operations from an administrator who uses the determination device 100, and a display unit (for example, a liquid crystal display) that displays various types of information. You may have.

(About the communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the user terminal 10 via the network N.

(About the storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 3, the storage unit 120 includes a session storage unit 121 and an effect measurement storage unit 122. Hereinafter, each storage unit included in the storage unit 120 will be described in order. In addition, the description regarding the overlapping item is abbreviate | omitted suitably.

(Regarding session storage unit 121)
The session storage unit 121 stores information related to the session. Here, FIG. 4 shows an example of the session storage unit 121 according to the embodiment. FIG. 4 is a diagram illustrating an example of the session storage unit 121 according to the embodiment. In the example illustrated in FIG. 4, the session storage unit 121 includes items such as “session ID”, “user ID”, “environmental sound information”, and “behavior information”. Also, the behavior information has small items such as “input means”, “collected data”, “content”, and “date and time”.

  “Session ID” is identification information for identifying a session. “User ID” is identification information for identifying a user. In the embodiment, identification information as shown in FIG. 4 may be used as a reference symbol. For example, the user identified by the user ID “U1” may be referred to as “user U1”.

  “Environmental sound information” indicates information relating to the collected environmental sound. “Behavior information” indicates a predetermined behavior of the user in the session. “Input means” indicates an input means in the user's action. Specifically, the input means indicates means (such as voice input or operation on the touch panel) used when the user inputs information to the user terminal 10. “Collected data” indicates specific data collected as information indicating user behavior. “Content” indicates the content of the user's action. “Date and time” indicates the date and time when the user's action was performed.

  In the example shown in FIG. 4, conceptual information such as “environmental sound data # 1”, “audio data # 1”, and “time # 1” is shown as information stored in the item. Actually, an audio file of an arbitrary format, a value indicating date and time, and the like are stored.

  That is, the example of the data illustrated in FIG. 4 indicates that the session SE01 identified by the session ID “SE01” includes information regarding a series of actions performed by the user U1 identified by the user ID “U1”. Yes. In session SE01, when “environmental sound data # 1” is collected as environmental sound information, user U1 performs “login” using “voice” as an input means, and is collected at that time. The data is “voice data # 1”, and the date and time when login was performed is “time # 1”.

  The user U1 subsequently makes a “request” using “voice”, and the data collected at that time is “voice data # 2”. For example, as shown in FIG. 1, this action is stored as a request for the service (that is, the determination device 100) due to the fact that the user U1 utters “rice ...”. Further, the user U1 subsequently performs “web browsing” using the “touch panel”, and the data collected at that time is “operation data # 1”. For example, as shown in FIG. 1, this behavior is stored as the behavior of the same session SE01 when the user U1 makes a certain request and is performed while the environmental sound is continuous. The user U1 subsequently makes a “request” using “voice”, and the data collected at that time is “voice data # 3”. For example, as shown in FIG. 1, this action is stored as a request for a service when the user U1 utters “Buy.” When it is performed while environmental sounds are continuous. This behavior is stored as the behavior of the same session SE01.

  In the example of FIG. 4, for the sake of explanation, an example in which environmental sound information is stored every time a user's predetermined action is performed is shown. However, the determination apparatus 100 is configured to perform every predetermined time (eg, every 3 seconds). Environmental sound information may be collected. And even if it is a case where a user's some action is not memorize | stored, as long as it determines with the continuity of environmental sound being hold | maintained, the determination apparatus 100 may be made to continue one session.

(Regarding the effect measurement storage unit 122)
The effect measurement storage unit 122 stores information related to content effect measurement. The effect measurement storage unit 122 includes an action table 123 and an effect table 124 as data tables.

(About the action table 123)
The behavior table 123 stores information regarding content distribution and user behavior after content distribution. Here, FIG. 5 shows an example of the action table 123 according to the embodiment. FIG. 5 is a diagram illustrating an example of the behavior table 123 according to the embodiment. In the example illustrated in FIG. 5, the behavior table 123 includes items such as “session ID”, “user ID”, “environmental sound information”, “distribution information”, and “behavior information”. In addition, the item of distribution information has small items such as “content ID”, “distribution date / time”, and “media”. The action information item has small items such as “input means”, “collected data”, “related content ID”, and “content”.

  “Distribution information” indicates information related to the content distributed to the user. “Content ID” indicates identification information for identifying content. “Distribution date / time” indicates the date / time when the content was distributed to the user. “Media” indicates media on which content is distributed. The media may be, for example, a web page displayed on the user terminal 10 or a television or radio other than the user terminal 10. For example, when collecting the user action history transmitted from the user terminal 10, the determination apparatus 100 collects information regarding the distributed content and the date and time when the content was distributed. When the medium is a television or radio, the determination apparatus 100 may acquire distribution information for distributing content on the television or radio from an external server in advance, or based on collected sound information. The content may be specified.

  “Behavior information” indicates information related to the user's behavior in response to the content. “Related content ID” indicates identification information of content presumed to be related to user behavior. It is assumed that common identification information is stored for the related content ID and the content ID. “Content” indicates the content of the user's action.

  The determination apparatus 100 determines whether or not the user's action is related to the distributed content using various methods. For example, when the user tweeted the product name, the determination apparatus 100 recognizes the product name and specifies content corresponding to the recognized product name (for example, advertising content for promoting the product). Then, the determination apparatus 100 determines that the user's tweet (tweet) is associated with the content. In addition, when the user orders a product, the determination device 100, when content related to the product (for example, a television commercial) is distributed to the user in the same session, Are related.

  That is, in the example of the data shown in FIG. 5, in the session SE02, the user U1 is identified by the content ID “C01” via “TV” at the timing of the time # 21 when the environmental sound data # 21 is collected. This indicates that the distribution of the content C01 to be received has been received. In addition, the user U1 indicates that the content C02 has been distributed via the television at the time # 22 when the environmental sound data # 22 is collected.

  Further, in the example of the data shown in FIG. 5, it is indicated that the user U1 has tweeted the voice data # 21 in the session SE02, and the content related to the tweet is the content C01. In addition, the user U1 makes a request (for example, purchase of a product) of the audio data # 22 to the determination apparatus 100, and the content related to the request is the content C02.

  In the above example, tweets and requests are shown as user actions, but user actions are not limited to these. For example, the determination device 100 selects, as a user's action, access to a web page of a product advertised by the content, writing about the product to an SNS (Social Networking Service), or selection of the content (touch or click). Various actions may be collected, such as having done or adding a product to the wish list.

(About the effect table 124)
The effect table 124 stores information related to content effects. Here, FIG. 6 shows an example of the effect table 124 according to the embodiment. FIG. 6 is a diagram illustrating an example of the effect table 124 according to the embodiment. In the example illustrated in FIG. 6, the effect table 124 includes items such as “content ID”, “tweet rate”, and “CVR (Conversion Rate)”.

  The “tweet rate” indicates the rate at which the user tweeted the content distributed to the user. “CVR” indicates a rate at which some benefit is provided to the content provider with respect to the content distributed to the user. The conversion corresponds to, for example, an action in which a user purchases a product advertised with content, makes an application, requests a material, or accesses a web page of a content provider.

  That is, the example of the data illustrated in FIG. 6 indicates that the tweet rate of the content C01 is “tweet rate # 1” and the CVR is “CVR # 1”.

(About the control unit 130)
Returning to FIG. 3, the description will be continued. The control unit 130 is a controller, for example, various programs (for example, determination programs) stored in a storage device inside the determination apparatus 100 by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Is implemented by using a RAM (Random Access Memory) as a work area. The control unit 130 is a controller, and is realized by an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

  As illustrated in FIG. 3, the control unit 130 includes a collection unit 131, an extraction unit 132, a determination unit 133, and a measurement unit 134, and implements or executes the information processing functions and operations described below. . Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 3, and may be another configuration as long as the information processing described below is performed. Further, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship illustrated in FIG. 3, and may be another connection relationship. Each processing unit included in the control unit 130 corresponds to each procedure executed by the determination program according to the present application. For example, the process executed by the collection unit 131 corresponds to a collection procedure that the determination program causes the determination apparatus 100 to execute. Similarly, the process executed by the extraction unit 132 corresponds to the extraction procedure that the determination program causes the determination device 100 to execute, and the process that the determination unit 133 executes corresponds to the determination procedure that the determination program causes the determination device 100 to execute. The process executed by the measurement unit 134 corresponds to a measurement procedure that the determination program causes the determination apparatus 100 to execute.

(About the collection unit 131)
The collection unit 131 collects various information. For example, the collection unit 131 collects environmental sounds around the user and the user terminal 10.

  Specifically, the collection unit 131 collects environmental sounds detected by the user terminal 10 via the network N. Note that the collection unit 131 may collect sound information transmitted by the user terminal 10 and collect environmental sounds included in the sound information, or crawl the user terminal 10 so that the user terminal 10 Sound information held in the sound information may be collected, and environmental sounds included in the sound information may be collected.

  The collection unit 131 collects ambient environmental sounds that are regularly emitted. For example, the collection unit 131 collects ambient environmental sounds in which at least one of the direction in which the sound is emitted, the distance to the sound source, the waveform of the collected sound, and the volume (sound pressure) of the collected sound is steady. More specifically, the collection unit 131 collects the operating sound of the air conditioner 60 at the user's home or the like, the sound output from the television 70, and the like as environmental sounds.

  For example, the collection unit 131 continuously collects environmental sounds every predetermined time. For example, the collection unit 131 continues to collect environmental sounds every predetermined time set by the administrator of the determination apparatus 100 (for example, every 3 seconds or 5 seconds). In this case, the collected environmental sound may be not only sound information at a certain moment but also continuous sound information of 3 seconds or 5 seconds.

  The collection unit 131 uses sound pressure level, frequency, the number of estimated sound sources (number of devices estimated to constitute the environmental sound, etc.), sound pressure, and waveform as sound information including environmental sound. Collect period etc. Note that regarding the collection of the sound information, the collection unit 131 may appropriately collect the sound information using a known analysis technique.

  Further, the collection unit 131 may collect not only the environmental sound but also sound information emitted by the user. Specifically, the collection unit 131 collects voice information issued by the user in order to use the service. Further, the collection unit 131 may intermittently collect voices uttered by the user. In this case, the determination unit 133 described later may determine that the voice collected intermittently is voice information that constitutes one intention.

  Note that the collection unit 131 may collect various information collected by a sensor included in the user terminal 10 together with the sound information. For example, the collection unit 131 collects environment information detected by the user terminal 10. The collection unit 131 may collect device information of the user terminal 10 itself, information about an external device that communicates with the user terminal 10, and the like.

  Specifically, the collection unit 131 includes, as information detected by the user terminal 10, position information indicating a position where the user terminal 10 (or a user who uses the user terminal 10) is located, a temperature around the user terminal 10. Humidity information, light information indicating the intensity of ambient light, etc. are collected. Further, the collection unit 131 may collect environmental information around the user terminal 10 based on a photograph or video taken by a camera provided in the user terminal 10. For example, the collection unit 131 collects environmental information around the user terminal 10 based on image information captured by the camera, position information included in the image information, date and time of capture, and the like.

  In addition, the collection unit 131 includes, as device information of the user terminal 10 itself, information on the CPU, OS (Operating System), memory, and the like of the user terminal 10, network functions such as an antenna, installed software, and browser software to be used , Information such as input means (for example, a microphone, a touch panel, or a fingerprint reader capable of collecting fingerprint data) included in the user terminal 10 is collected.

  The collection unit 131 may collect the operating status of the user terminal 10. For example, the collection unit 131 determines whether or not the user terminal 10 is in an activated state, and if the user terminal 10 is in an activated state, whether the screen is on / off, whether the user terminal 10 is in a moving / still state, and the like Collect information. Such information is collected by, for example, an application having a predetermined sensing function installed in the user terminal 10 and is held in the user terminal 10. Further, the collection unit 131 may collect accelerations and the like observed by the user terminal 10 as the operation status of the user terminal 10.

  In addition, the collection unit 131 collects information for identifying an external device in communication with the user terminal 10 as well as information on the external device that communicates with the user terminal 10 and the type and frequency band of established communication. To do.

  Further, the collection unit 131 may collect information related to user behavior. For example, the collection unit 131 may collect information on a service page browsed by the user using the user terminal 10, information on a query transmitted to the search service, and the like.

  The collection unit 131 may collect user personal information. For example, the collection unit 131 accepts user personal information as registration information for using a predetermined service. The collection unit 131 may collect user personal information from a service side such as a web server.

  The collection unit 131 may collect information related to content distribution. For example, the collection unit 131 collects information such as media (television, radio, etc.) to which content is distributed and content distribution date and time from an external server.

  The collection unit 131 stores the collected information in each storage unit in the storage unit 120. Further, the collection unit 131 may appropriately collect information already stored in the storage unit 120.

(About the extraction unit 132)
The extraction unit 132 extracts environmental information or sound information indicating a predetermined action of the user from the sound information collected by the collection unit 131.

  Based on a known voice analysis process, the extraction unit 132 extracts voice information indicating the user's predetermined behavior from the sound information collected from the user terminal 10. Specifically, when a voice corresponding to a word registered in advance as a word indicating the user's action is extracted, the extraction unit 132 extracts the voice as voice information indicating the user's predetermined action. .

  For example, the extraction unit 132 uses, for sound information collected from the user terminal 10, a word input by the user to use a service provided by the determination device 100 (for example, a service name provided by the determination device 100 or “Hello”). The word and the previous and next voices are extracted as voice information that the user requests to log in to the service. Alternatively, the extraction unit 132 may include words registered in the sound information collected from the user terminal 10 as words intended to order products in the service (for example, “Buy.”, “I want to buy”, “ If the user wants to order an arbitrary product, the word and the audio information before and after that are extracted.

  Note that the extraction unit 132 may determine whether or not the voice is a voice uttered by the user himself / herself based on a user's voiceprint registered in advance. Only when it is determined that the user is the user, the extraction unit 132 may extract the voice to be processed as voice information corresponding to the predetermined action of the user. Thereby, the extraction part 132 can prevent extracting accidentally the audio | voice which the person other than the user uttered as audio | voice information corresponding to a user's predetermined | prescribed action.

  Further, the extraction unit 132 extracts sound information other than the sound information extracted as information indicating the user's predetermined behavior as environmental sound. In addition, when the sound information indicating the user's predetermined action and the environmental sound are emitted simultaneously (for example, when the environmental sound is detected as background noise with respect to the sound emitted by the user), the extraction unit 132 is The sound information corresponding to the predetermined action and the sound information corresponding to the environmental sound may be separately recognized and extracted.

  In addition, the extraction unit 132 may analyze sound information and generate a predetermined learning model using a known technique when extracting voice information indicating a user's predetermined action or environmental sound. For example, the extraction unit 132 may extract sound information indicating a predetermined action of the user included in the sound information or extract environmental sound by inputting sound information into the generated model. For example, the model may be a model that learns in advance a voice utterance or the like produced by the user, or may be one in which learning is continuously performed by various learning techniques such as deep learning.

(About the determination unit 133)
The determination unit 133 determines whether or not the predetermined action of the user is included in one session based on the continuity of the environmental sound.

  For example, the determination unit 133 determines that a predetermined action of the user while ambient ambient sounds that are regularly emitted are collected is included in one session. More specifically, the determination unit 133 determines that the predetermined action of the user observed after the ambient environmental sound that is constantly emitted exceeds a predetermined threshold is not included in one session. . In other words, the determination unit 133 determines that the predetermined action of the user observed while the ambient environmental sound that is constantly emitted does not exceed the predetermined threshold is included in one session.

  Note that the predetermined threshold value in this case may include various types of information. For example, the determination unit 133 provides a predetermined threshold value for each of sound pressure, frequency, sound magnitude period, waveform, and the like, and when at least one of them is observed exceeding the threshold value, the environmental sound It may be determined that is no longer continuous. In addition, the determination unit 133 captures the environmental sound as a waveform including sound pressure, frequency, and the like, and when the environmental sound waveform observed at a certain time point and the current environmental sound waveform are dissimilar, It may be determined that the sound is no longer continuous. Note that the criteria for similarity or dissimilarity may be arbitrarily set based on known speech analysis techniques.

  In addition, the determination unit 133 may determine whether or not the intermittent sound generated by the user is a series of sound inputs based on the continuity of the environmental sound collected by the collection unit 131. That is, even if the collected sound is interrupted, the determination unit 133 determines that the sound is a series of sound inputs when the environmental sound observed along with the sound is continuous. Also good. In the example illustrated in FIG. 1, the determination unit 133 includes a voice “rice ...” uttered by the user U1 in step S01 and a voice “buy.” Uttered by the user U1 in step S05. It is determined that the input is a series of voice inputs.

  In addition, when the environmental sound continues for a predetermined time (for example, 30 minutes or 1 hour), the determination unit 133 indicates that the predetermined action of the user performed during the predetermined time is one session. It may be determined that the predetermined action of the user performed over the predetermined time is not included in one session. That is, the determination unit 133 may use not only the continuity of the environmental sound but also session management based on time.

  Further, even when the environmental sound becomes discontinuous (when the environmental sound has changed beyond a predetermined threshold), the determination unit 133 does not stop the sound while the user's voice input continues. The input may be determined to be included in one session.

  Further, the determination unit 133 may determine whether or not the sound information collected from the user terminal 10 includes information related to content distribution. The determination unit 133 may determine whether or not the content has been distributed by determining whether or not the sound information output by the content is included in the environmental sound. Alternatively, the determination unit 133 may include not only the sound information but also information indicating that the content has been distributed to the user (for example, the fact that the video content for promotion has been distributed in the service provided by the determination device 100). It may be determined that the content has been distributed to the user.

(About measuring unit 134)
The measurement unit 134 measures the effect of the content provided to the user when the determination unit 133 determines that the predetermined action of the user and the provision of the content to the user are included in one session.

  For example, the measurement unit 134 measures the effect of the content provided to the user based on whether or not the predetermined behavior of the user is an action estimated to have viewed the content. Specifically, the measurement unit 134 measures the effect of the content based on the action that the user has made some sound on the distributed content. More specifically, the measurement unit 134 presumes that the content has been viewed by the user when the content product name, a word praising the product, a word indicating interest in the product, or the like is tweeted after the content is distributed. To do. Then, the measuring unit 134 measures the effect of the content by regarding that the content has been effective by viewing the content.

  Further, the measurement unit 134 may measure the effect of the content provided to the user based on whether or not the predetermined action of the user can be a conversion related to the content. The conversion corresponds to, for example, an action in which a user purchases a product advertised with content, makes an application, requests a material, or accesses a web page of a content provider.

  The measuring unit 134 includes identification information of the content distributed to the user, the number of content distributions, the rate of tweeting for the content (tweet rate), and the rate of conversion for the content. (CVR) and the like are stored in the storage unit 120. Note that the measurement unit 134 may acquire information regarding attributes such as the sex and age of the user during the measurement process. Thereby, the measurement part 134 can also measure together about the information regarding the age, sex, etc. in which a content exhibits an effect especially, for example.

  Further, the measurement unit 134 may transmit information regarding the measured effect to the content provider. By referring to such information, the content provider can confirm whether or not the distributed content has been effective for the user.

[4. Configuration of user terminal]
Next, the configuration of the user terminal 10 according to the embodiment will be described with reference to FIG. FIG. 7 is a diagram illustrating a configuration example of the user terminal 10 according to the embodiment. As illustrated in FIG. 7, the user terminal 10 includes a communication unit 11, an input unit 12, a display unit 13, a detection unit 14, a storage unit 15, and a control unit 16.

  The communication unit 11 is connected to the network N by wire or wireless, and transmits / receives information to / from the determination device 100. For example, the communication unit 11 is realized by a NIC or the like.

  The input unit 12 is an input device that receives various operations from the user. For example, the input unit 12 is realized by an operation key or the like provided in the user terminal 10. The display unit 13 is a display device for displaying various information. For example, the display unit 13 is realized by a liquid crystal display or the like. In addition, when a touch panel is employ | adopted for the user terminal 10, a part of input part 12 and the display part 13 are integrated.

  The detection unit 14 detects various information related to the user terminal 10. Specifically, the detection unit 14 detects voice emitted by the user and environmental sounds around the user terminal 10. For example, the detection unit 14 is sound collection means such as a microphone, and when sound is input, acquires the sound as sound information.

  Further, the detection unit 14 may detect a user operation on the user terminal 10, position information of the user terminal 10, information about a device connected to the user terminal 10, an environment in the user terminal 10, and the like. Good.

  For example, the detection unit 14 detects a user operation based on information input to the input unit 12. In other words, the detection unit 14 detects that there has been an input of an operation for touching the screen to the input unit 12, an input of audio, or the like. Moreover, the detection part 14 may detect that the predetermined application was started by the user. When such an application is an application that operates the imaging device in the user terminal 10, the detection unit 14 detects that the imaging function is used by the user. Further, the detection unit 14 may detect an operation in which the user terminal 10 itself is moved based on data detected by an acceleration sensor, a gyro sensor, or the like provided in the user terminal 10.

  The detection unit 14 may detect the current position of the user terminal 10. Specifically, the detection unit 14 receives radio waves transmitted from a GPS (Global Positioning System) satellite, and obtains position information (for example, latitude and longitude) indicating the current position of the user terminal 10 based on the received radio waves. get. The position information may be acquired by an optical sensor, an infrared sensor, a magnetic sensor, or the like included in the user terminal 10.

  Further, the detection unit 14 may detect an external device connected to the user terminal 10. For example, the detection unit 14 detects the external device based on exchange of communication packets with the external device. Then, the detection unit 14 recognizes the detected external device as a terminal connected to the user terminal 10. The detection unit 14 may detect the type of connection with the external device. For example, the detection unit 14 detects whether it is connected to an external device by wire or wireless communication. Moreover, the detection part 14 may detect the communication system etc. which are used by radio | wireless communication. The detection unit 14 may detect the external device based on information acquired by a radio wave sensor that detects a radio wave emitted by the external device, an electromagnetic wave sensor that detects an electromagnetic wave, or the like.

  When the user terminal 10 is connected to an external device, the voice from the user may be detected by the external device. The external device is, for example, a home appliance having a voice assist function, and is a smart device that can communicate with the user terminal 10 and the determination device 100.

  Further, the detection unit 14 may detect the surrounding environment of the user terminal 10. The detection unit 14 uses various sensors and functions provided in the user terminal 10 to detect information about the environment. For example, the detection unit 14 includes an illuminance sensor that detects illuminance around the user terminal 10, an acceleration sensor (or gyro sensor, etc.) that detects physical movement of the user terminal 10, and humidity around the user terminal 10. A humidity sensor for detecting the magnetic field, a geomagnetic sensor for detecting a magnetic field at the location of the user terminal 10, and the like are used. And the detection part 14 detects various information using various sensors. For example, the detection unit 14 detects a noise level around the user terminal 10 and whether the surroundings of the user terminal 10 have illuminance suitable for imaging. Further, the detection unit 14 may detect surrounding environment information based on a photograph or video taken by the camera.

  The storage unit 15 stores various information. The storage unit 15 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 15 stores the sound information detected by the detection unit 14 in association with the date and time when the sound was detected.

  The control unit 16 is a controller, and is realized, for example, by executing various programs stored in a storage device inside the user terminal 10 using the RAM as a work area by a CPU, an MPU, or the like. The control unit 16 is a controller, and is realized by an integrated circuit such as an ASIC or FPGA, for example.

  As illustrated in FIG. 7, the control unit 16 includes an acquisition unit 161 and a transmission unit 162, and realizes or executes functions and operations of information processing described below. Note that the internal configuration of the control unit 16 is not limited to the configuration illustrated in FIG. 7, and may be another configuration as long as the information processing described below is performed.

  The acquisition unit 161 acquires various types of information. For example, the acquisition unit 161 acquires various types of information detected by the detection unit 14 by controlling the detection unit 14. Specifically, the acquisition unit 161 acquires sound information including environmental sounds around the user or the user terminal 10, voices uttered by the user, and the like.

  The acquisition unit 161 may acquire sound information every predetermined time. For example, the acquisition unit 161 acquires sound information by controlling the detection unit 14 described above. When there is no voice input from the user, for example, the acquisition unit 161 acquires sound information (environmental sound) having a length of every predetermined time (every 3 seconds, every 5 seconds, every 10 seconds, etc.). . Or the acquisition part 161 acquires the sound information of the length corresponding to the audio | voice which the user uttered, when the audio | voice input from a user is detected by the detection part 14. FIG. Note that the length of the sound information acquired by the acquisition unit 161 may be set by the determination device 100.

  The transmission part 162 transmits various information. For example, the transmission unit 162 transmits the sound information acquired by the acquisition unit 161 to the determination device 100 in response to a request from the determination device 100.

[5. Processing procedure)
Next, a procedure of processing performed by the determination apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 8 is a flowchart illustrating a processing procedure according to the embodiment.

  As illustrated in FIG. 8, the determination apparatus 100 collects sound information including ambient environmental sounds from the user terminal 10 (step S101). And the determination apparatus 100 extracts audio | voice information (namely, information which shows a user's predetermined action) from sound information (step S102).

  And the determination apparatus 100 determines whether the predetermined | prescribed action of the user corresponding to audio | voice information is the action performed under the continuous environmental sound (step S103). When it is determined that the predetermined action of the user is an action performed under a continuous environmental sound (step S103; Yes), the determination apparatus 100 is an action in which the predetermined action is included in one session. (Step S104). And the determination apparatus 100 repeats the process which collects sound information (environmental sound).

  On the other hand, when it is determined that the predetermined action of the user is not an action performed under a continuous environmental sound (step S103; No), the determination apparatus 100 determines that the predetermined action is an action of another session. It is determined that there is (step S105). In this case, for example, the determination apparatus 100 ends the previous session, starts a new session, and determines that the action is included in the new session.

[6. (Modification)
The processing according to the above-described embodiment may be performed in various different forms other than the above-described embodiment. Hereinafter, another embodiment (modified example) of the determination apparatus 100 or the determination processing system 1 will be described.

[6-1. Variations in session management using environmental sounds)
In the embodiment described above, the determination apparatus 100 illustrates an example in which a session is managed by determining whether or not the predetermined action of the user is an action performed under continuous environmental sound. Here, even if there is no explicit action from the user (even if the user is silent), the determination apparatus 100 performs processing such as maintaining the session by estimating that the user continues to use the service. You may go. This point will be described with reference to FIGS. In the following, for the purpose of distinction, the determination device 100 having the configuration of the modified example is referred to as a determination device 100A, but is collectively referred to as the determination device 100 when it is not particularly necessary to distinguish.

  For example, the determination apparatus 100A determines whether or not the subject of any action is the user by collating sound information for recognizing the user with environmental sound. When determining that the subject of the arbitrary action is the user, the determining apparatus 100A determines that the arbitrary action is included in one session. In this case, the arbitrary behavior includes not only an explicit behavior such as a voice request but also an behavior such as waiting while logged in to the service.

  The sound information for recognizing the user is sound information for recognizing the user himself / herself registered in advance by the user, for example. For example, when using the service provided by the determination apparatus 100A, the user registers sound information for recognizing the user on the service side.

  Specifically, the user registers in advance the environmental sound of the place (base) where he / she mainly operates the user terminal 10. Alternatively, the user registers in advance a sound that is emitted by the user and can be an environmental sound. Specifically, the user registers his / her footsteps as one of the environmental sounds. Alternatively, the user registers his / her family's voice, pet's voice, and the like as one of the environmental sounds. The registration of the sound information does not need to be explicitly performed by the user. For example, the sound information may be arbitrarily registered by the determination device 100A that collects environmental sounds. In other words, the determination apparatus 100A constantly collects environmental sounds for recognizing the user, and when such environmental sounds are observed, information for estimating that the user himself is using the service. The sound information is registered in advance.

  Then, the determination apparatus 100 </ b> A is at least one of a user's footsteps registered in advance, a voice of a person different from the user (for example, a user's family and friends), and an environmental sound of a location where the user is based. Then, it is determined whether or not the subject of any action is the user by comparing the collected environmental sound. When the collected environmental sound includes pre-registered sound information, the determination device 100A determines that the user who uses the user terminal 10 does not perform voice input from the user for a while. The session is maintained as having a high probability of being the user himself / herself. That is, when the determination apparatus 100 determines that the subject of an arbitrary action is a user, the determination apparatus 100 determines that the arbitrary action is an action included in one continuing session. As a result, the user can maintain the session while the environmental sound estimated to indicate the user himself / herself is continuously observed in the logged-in service without performing explicit voice input for a while.

  As described above, the determination apparatus 100A according to the modified example maintains the session by verifying the user's identity based on the comparison between the sound information indicating the user who is registered in advance and the environmental sound. In performing the process according to the modification, the determination apparatus 100A according to the modification has a configuration illustrated in FIG. FIG. 9 is a diagram illustrating a configuration example of a determination apparatus 100A according to a modification. As illustrated in FIG. 9, the determination apparatus 100 </ b> A further includes a collation information storage unit 125.

(About the verification information storage unit 125)
The collation information storage unit 125 stores information related to user collation. The collation information storage unit 125 includes a registration table 126 and a collation table 127 as data tables.

(Registration table 126)
The registration table 126 stores information related to registration data used for recognizing a user. Here, FIG. 10 shows an example of the registration table 126 according to the modification. FIG. 10 is a diagram illustrating an example of the registration table 126 according to the modification. In the example illustrated in FIG. 10, the registration table 126 includes items such as “user ID”, “registration information”, and “content”.

  “Registration information” indicates sound information registered as sound information for recognizing the user. “Content” indicates content related to sound information for recognizing the user. In the example of FIG. 10, conceptual information such as “registration data # 1” is shown as the registration information. However, in practice, an audio file of an arbitrary format is stored as the registration data.

  That is, in the example of data shown in FIG. 10, “registration data # 1” is registered as registration information for recognizing the user U1, and the content is “friend's voice”. Yes. Also, as registration information for recognizing the user U1, “registration data # 2”, “registration data # 3”, and “registration data # 4” are also registered. This indicates that the sound is “person's voice”, “person's footsteps”, or “room environmental sound”.

(Regarding the collation table 127)
The collation table 127 stores information related to user collation. Here, FIG. 11 shows an example of the collation table 127 according to the modification. FIG. 11 is a diagram illustrating an example of the collation table 127 according to the modification. In the example illustrated in FIG. 11, the collation table 127 includes items such as “session ID”, “user ID”, “environmental sound information”, and “collation result”.

  “Verification information” indicates a result of whether or not any registered data registered as sound information for recognizing the user is compared with the environmental sound. For example, when “◯” is stored in the collation information, this indicates that any registered data registered as sound information for recognizing the user has been collated with the environmental sound. In this case, it is presumed that an arbitrary action (for example, an action of a user who waits for using the service after logging in to the service) is performed by the user U1 himself / herself. Therefore, the determination apparatus 100A maintains the session SE03 so that the session is not terminated.

  That is, in the example of the data shown in FIG. 11, the session SE03 is a session related to the user U1, and the collected environmental sound information # 31 is any registration registered as sound information for recognizing the user. Since the collation result with the data is “◯”, it indicates that the session SE03 is maintained. In addition, since the result of collation with any registered data registered as sound information for recognizing the user is “◯” between environmental sound information # 32 and environmental sound information # 34, the session It shows that SE03 is maintained.

  As described above, the determination apparatus 100A determines whether or not the subject of any action is the user by comparing the sound information for recognizing the user with the environmental sound, and the subject is the user. Is determined, it is determined that an arbitrary action is included in one session.

  Specifically, the determination apparatus 100 </ b> A uses at least one of a user's footstep registered in advance, a voice of a person different from the user, an environmental sound at a location where the user is based, and the collected environmental sound. By collating, it is determined whether the subject of arbitrary action is a user.

  As described above, the determination apparatus 100A may perform session management by comparing the sound information registered in advance and estimated to indicate the user's identity with the environmental sound. Accordingly, the determination device 100A maintains the session without disconnecting the session because the user himself / herself continues to use the service in the same environment without receiving conscious voice input from the user. be able to. For this reason, the determination apparatus 100A can perform session management with high usability.

(Regarding processing procedure according to modification)
Next, a procedure of processing performed by the determination apparatus 100A according to the modification will be described with reference to FIG. FIG. 12 is a flowchart illustrating a processing procedure according to the modification.

  As illustrated in FIG. 12, the determination apparatus 100A registers sound information for user determination (step S201). Thereafter, the determination apparatus 100A collects sound information including ambient environmental sounds from the user terminal 10 (step S202). Then, the determination apparatus 100A extracts voice information (that is, information indicating an arbitrary action of the user) from the sound information (step S203).

  Then, the determination apparatus 100A determines whether or not the environmental sound matches the sound information registered in advance (step S204). When it is determined that the environmental sound matches the sound information registered in advance (step S204; Yes), the determination apparatus 100A determines that any action is the action of the user himself (step S205). Then, the determination apparatus 100A determines that the arbitrary action is an action included in one session (step S206). In other words, the determination apparatus 100A maintains the session without disconnecting it. Then, the determination apparatus 100A repeats the process of collecting sound information (environmental sound).

  On the other hand, when it is determined that the environmental sound and the pre-registered sound information do not match (step S204; No), the determination apparatus 100A determines that the arbitrary action is not the action of the user himself (step S207). Then, the determination apparatus 100A determines that the arbitrary action is an action included in another session (step S208). In other words, the determination apparatus 100A starts a new session without maintaining the session, and determines that the arbitrary action is included in the new session.

[6-2. Judgment program)
In the above embodiment, it has been shown that the determination program according to the present application is executed inside the determination apparatus 100. However, the determination program according to the present application may be executed inside the user terminal 10. In this case, the user terminal 10 includes each processing unit included in the determination apparatus 100 described in the above embodiment and a storage unit 15 that stores each piece of information stored in the storage unit 120.

[6-3. Number of user terminals
In the said embodiment, although the user terminal 10 showed the example which is one apparatus, the user terminal 10 is not restricted to one. For example, it is assumed that the user has a plurality of terminal devices capable of communication. In this case, the determination apparatus 100 may collect environmental sounds around the user and the user terminal 10 from a plurality of user terminals 10 used by the user.

  Note that the determination apparatus 100 does not necessarily need to acquire a global identifier that is common to other devices in identifying a plurality of user terminals 10. That is, the determination apparatus 100 only needs to acquire an identifier that can uniquely identify the user terminal 10 in the process executed in the embodiment, and does not necessarily need to acquire an identifier that is permanently determined.

[6-4. Configuration of user terminal]
In the above embodiment, the configuration example of the user terminal 10 has been described with reference to FIG. However, the user terminal 10 does not necessarily need to include all the processing units illustrated in FIG. For example, the user terminal 10 does not necessarily include the display unit 13. Further, the user terminal 10 may be separated into two or more devices to realize the configuration illustrated in FIG. For example, the user terminal 10 may be realized by two or more devices having a configuration separated into at least a voice detection device having the detection unit 14 and a communication device having at least the communication unit 11.

[6-5. (Determination of voice input)
In the said embodiment, the determination apparatus 100 showed the example which determines that the intermittent audio | voice which the user uttered is a series of audio | voice input by managing a session based on the continuity of environmental sound. Here, the determination apparatus 100 may perform various processes related to voice recognition in addition to a series of voice input and determination.

  For example, the determination apparatus 100 may specify an instruction word issued by the user based on the continuity of the environmental sound. Specifically, the determination apparatus 100 determines that the user instruction word issued while the environmental sound is continuous indicates another word issued in the same session. For example, it is assumed that the determination apparatus 100 collects the voice “Rice ...” after collecting the voice uttered by the user “rice ...”. In this case, the determination apparatus 100 recognizes the instruction word “that time” as “rice”, which is a product name issued by the user immediately before. Then, the determination apparatus 100 performs a process corresponding to the recognized voice (a process of ordering rice in this example).

  As described above, the determination apparatus 100 manages the session based on the continuity of the environmental sound, so that even when the user's voice is interrupted, the front and rear voices are set as a series of voices and the front and rear voices are set as a series. Can be handled as a conversation. Thereby, the determination apparatus 100 can improve the convenience of the user who uses a service.

[6-6. (Environmental sound)
In the above-described embodiment, an example has been described in which the determination apparatus 100 collects operating sounds of devices observed at the user's home or the like as environmental sounds. However, the example of the environmental sound is not limited to this, and the determination apparatus 100 may collect various sounds as the environmental sound.

  For example, when the user uses the user terminal 10 in a car, the determination apparatus 100 may collect car engine sounds and the like as environmental sounds. Note that when the environmental sound is collected, the determination apparatus 100 collects the distance, direction, sound pressure, frequency, and the like of the sound source that emits the environmental sound. You may make it collect as a sound. Thereby, the determination apparatus 100 can observe universal sound information as described above even when suddenly different sounds (for example, sounds emitted by persons other than the user) are mixed in the sound information. As long as the environmental sound is continuous, it may be determined. As described above, the determination apparatus 100 can implement the processing described in the above embodiment in various situations.

[7. Hardware configuration)
The determination apparatus 100 and the user terminal 10 according to the embodiment described above are realized by a computer 1000 having a configuration as shown in FIG. 13, for example. Hereinafter, the determination apparatus 100 will be described as an example. FIG. 13 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the determination apparatus 100. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores a program executed by the CPU 1100, data used by the program, and the like. The communication interface 1500 receives data from other devices via the communication network 500 (corresponding to the network N shown in FIG. 2) and sends the data to the CPU 1100, and the data generated by the CPU 1100 is transferred to other devices via the communication network 500. Send to device.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the data generated via the input / output interface 1600 to the output device.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the determination apparatus 100 according to the embodiment, the CPU 1100 of the computer 1000 executes a program loaded on the RAM 1200 (for example, the determination program according to the embodiment), thereby Realize the function. The HDD 1400 stores data in the storage unit 120. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from other devices via the communication network 500.

[8. Others]
In addition, among the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified. For example, the various types of information illustrated in each drawing is not limited to the illustrated information.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the extraction unit 132 and the determination unit 133 illustrated in FIG. 3 may be integrated. Further, for example, information stored in the storage unit 120 may be stored in a storage device provided outside via the network N.

  Further, for example, in the above-described embodiment, an example has been described in which the determination apparatus 100 performs a collection process for collecting sound information from a user, a determination process for determining a session, and a measurement process for measuring the effect of content. However, the determination device 100 described above may be separated into a collection device that performs collection processing, a determination device that performs determination processing, and a measurement device that performs measurement processing. In this case, for example, the processing by the determination device 100 according to the embodiment is realized by the determination processing system 1 including each device such as a collection device, a determination device, and a measurement device.

  Moreover, each embodiment and modification which were mentioned above can be combined suitably in the range which does not contradict a process content.

[9. effect〕
As described above, the determination program according to the embodiment is based on the collection procedure of collecting ambient environmental sounds and the continuity of the environmental sounds collected by the collection procedure. A determination procedure for determining whether or not the image is included is executed by a computer (for example, the determination apparatus 100 according to the embodiment).

  As described above, the determination program according to the embodiment has one predetermined action of the user based on the surrounding environmental sound different from the user's voice when the service is used using the voice recognition technology. Determine whether it is included in the session. Accordingly, the determination program can determine the continuity of behavior even in a service using a voice recognition technology that cannot use identification information indicating identity of the user such as cookie information. For this reason, the determination program can maintain a session without asking the user to log in again or asking for re-input of voice. As a result, the determination program can perform session management with high usability without imposing an extra burden on the user.

  In addition, the collection procedure collects ambient environmental sounds that are constantly emitted. The determination procedure determines that a predetermined action of the user while ambient ambient sounds that are regularly emitted are collected is included in one session.

  As described above, the determination program according to the embodiment collects regularly observed sounds such as operating sounds of the air conditioner 60 and the television 70 as ambient environmental sounds. For this reason, since the determination program can appropriately determine whether or not the environmental sound has changed, session management with high accuracy can be performed.

  Further, the determination procedure determines that the predetermined action of the user observed after the ambient environmental sound that is regularly emitted exceeds a predetermined threshold is not included in one session.

  As described above, the determination program according to the embodiment determines that the user's action is not included in one session when the environmental sound changes. Accordingly, the determination program can perform the session management without any sense of incongruity for the user because the user's series of actions can be completed in a natural manner that the environmental sound changes.

  The collection procedure collects ambient environmental sounds in which at least one of the direction in which the sound is emitted, the distance to the sound source, the waveform of the sound to be collected, and the volume of the sound to be collected is steady.

  Thus, the determination program according to the embodiment collects environmental sounds including various sound features. Thereby, since the determination program according to the embodiment can accurately capture changes in the environmental sound, more appropriate session management can be performed.

  Moreover, a collection procedure collects the voice which a user utters intermittently. The determination procedure determines whether or not the intermittent sound produced by the user is a series of sound inputs based on the continuity of the environmental sound collected by the collection procedure.

  As described above, the determination program according to the embodiment can perform flexible processing such as capturing a discontinuous sound as a series of sound when the environmental sound is continuous. For this reason, since the determination program reduces the opportunity of requesting the user to input voice again or the like, usability can be improved.

  In addition, the determination procedure determines that the user's predetermined action performed during the predetermined time is included in one session when the environmental sound continues beyond the predetermined time, and the predetermined time is It is determined that the predetermined action of the user performed beyond is not included in one session.

  As described above, the determination program according to the embodiment may combine existing session management using duration and the like with session management using environmental sound. Thereby, the determination program can perform session management with higher safety.

  In addition, the determination procedure determines whether or not the subject of any action is the user by comparing the sound information for recognizing the user with the environmental sound, and determines that the subject is the user Determines that any action is included in one session.

  As described above, the determination program according to the embodiment continues to use the service under the same environment without accepting conscious voice input from the user, so without intentionally disconnecting the session, Sessions may be maintained. Even by such processing, the determination program can perform session management with high usability.

  Also, the judgment procedure is to collate at least one of the user's footsteps registered in advance, the voice of a person different from the user, the environmental sound of the place where the user is based and the environmental sound collected by the collection procedure. By doing so, it is determined whether or not the subject of any action is a user.

  Thus, the determination program according to the embodiment may register sound information of various modes. Thereby, the determination program can determine the user's identity with high accuracy.

  In addition, the determination program according to the embodiment has the effect of the content provided to the user when the determination procedure determines that the predetermined action of the user and the provision of the content to the user are included in one session. The computer further executes a measurement procedure for measuring.

  As described above, the determination program according to the embodiment may perform content effect measurement using the continuity of the environmental sound. This is because a user who takes an action related to the distributed content under a situation where the environmental sound is continuous can determine that the probability of causing some action is motivated by the distribution of the content. That is why. In general, it is difficult to prove the relevance between advertising content such as commercials and the like and the action that the user tweeted to the user terminal 10. On the other hand, according to the determination program according to the embodiment, it is possible to collect user actions (tweets made and orders by voice input) performed under environmental sound that has continued from the distribution of the content. It is possible to accurately measure whether the effect has been achieved.

  Further, the measurement procedure measures the effect of the content provided to the user based on whether or not the predetermined behavior of the user is an action estimated to have viewed the content.

  As described above, the determination program according to the embodiment estimates whether or not the user has viewed the content based on, for example, the user's tweet, and measures the content effect based on the estimated information. Good. As a result, the determination program can accurately grasp whether the distributed commercial or advertising content has affected the user, so that the content effect can be more accurately compared to an index value such as a TV audience rating. Can be measured.

  Further, the measurement procedure measures the effect of the content provided to the user based on whether or not the user's predetermined action can be a conversion related to the content.

  Thus, the determination program according to the embodiment may use conversion as an element of effect measurement. The determination program can measure the effect of the content more accurately by using an element that easily shows the result as a numerical value, such as conversion.

  The embodiment of the present application has been described in detail with reference to the drawings. However, this is an exemplification, and various modifications and improvements are made based on the knowledge of those skilled in the art including the aspects described in the column of the disclosure of the invention. The present invention can be implemented in other forms.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the collection unit can be read as collection means or a collection circuit.

DESCRIPTION OF SYMBOLS 1 Determination processing system 10 User terminal 100 Determination apparatus 110 Communication part 120 Storage part 121 Session storage part 122 Effect measurement storage part 123 Action table 124 Effect table 125 Collation information storage part 126 Registration table 127 Collation table 130 Control part 131 Collection part 132 Extraction Unit 133 determination unit 134 measurement unit

Claims (13)

A collection procedure for collecting environmental sounds that are ambient sound around a user terminal, which is a terminal device used by the user, and that is emitted regularly .
The continuity of the environmental sound is determined based on whether or not the environmental sound that is regularly emitted collected by the collection procedure changes beyond a predetermined threshold, and based on the continuity of the environmental sound. a determination procedure for determining whether a predetermined action is included in a single session of the user,
The determination program characterized by causing a computer to execute. Before Symbol decision procedure,
It is determined that the predetermined action of the user is included in the one session while the environmental sound that is constantly emitted does not change beyond a predetermined threshold .
The determination program according to claim 1, wherein: The determination procedure is as follows:
It determines that the steadily emitted Ru ring Sakaion the predetermined action of the user observed after changes beyond a predetermined threshold, are not included in the one session,
The determination program according to claim 2, wherein: The collection procedure includes:
Collect ambient environmental sounds in which at least one of the direction in which the sound is emitted, the distance to the sound source, the waveform of the sound to be collected, and the volume of the sound to be collected is stationary.
The determination program according to any one of claims 1 to 3, wherein: The collection procedure includes:
The voice uttered by the user is collected intermittently, and the determination procedure is as follows:
Based on the continuity of the environmental sound collected by the collection procedure, it is determined whether or not the intermittent sound emitted by the user is a series of sound inputs.
The determination program according to any one of claims 1 to 4, wherein: The determination procedure is as follows:
When the environmental sound continues beyond a predetermined time, it is determined that the predetermined action of the user performed during the predetermined time is included in one session, and the predetermined time is exceeded. It is determined that the predetermined action of the user performed is not included in one session.
The determination program according to any one of claims 1 to 5, wherein: The determination procedure is as follows:
When comparing the sound information for recognizing the user and the environmental sound to determine whether the subject of any action is the user or not, and when determining that the subject is the user , Determining that the arbitrary action is included in one session,
The determination program according to any one of claims 1 to 6, wherein: The determination procedure is as follows:
Collating at least one of the user's footsteps registered in advance, the voice of a person different from the user, and the environmental sound of the place where the user is based with the environmental sound collected by the collection procedure To determine whether the subject of the arbitrary action is the user,
The determination program according to claim 7. A measurement procedure for measuring the effect of the content provided to the user when it is determined by the determination procedure that the predetermined action of the user and provision of the content to the user are included in one session;
The determination program according to claim 1, further causing a computer to execute. The measurement procedure is as follows:
Measuring the effect of the content provided to the user based on whether the predetermined action of the user is an action presumed to have viewed the content;
The determination program according to claim 9. The measurement procedure is as follows:
Measuring the effect of the content provided to the user based on whether the predetermined behavior of the user can be a conversion for the content;
The determination program according to claim 9 or 10, wherein: A collection unit that collects environmental sounds that are ambient sound around the user terminal, which is a terminal device used by the user, and is steadily emitted ;
The continuity of the environmental sound is determined based on whether the ambient environmental sound collected by the collection unit changes beyond a predetermined threshold, and based on the continuity of the environmental sound. , a determination section for determining whether or not a predetermined action of the user are included in one session,
A determination apparatus comprising: A determination method executed by a computer,
A collection step of collecting environmental sounds that are ambient sounds around a user terminal, which is a terminal device used by the user, and that are regularly emitted ;
The continuity of the environmental sound is determined based on whether or not the environmental sound that is constantly emitted collected by the collecting step changes beyond a predetermined threshold, and based on the continuity of the environmental sound. a determination step of determining whether or not a predetermined action is included in a single session of the user,
The determination method characterized by including.

Images