JP7504978B2 - server - Google Patents

Description

The present invention relates to a server that communicates with a speaker that has a microphone.

Smart speakers that are equipped with microphones and communication functions and allow voice control and information searches are beginning to become popular (see, for example, non-patent document 1).

https://www.is.nri.co.jp/report/short-research/2017/000213.html, retrieved May 9, 2018

Current systems including smart speakers can receive voice requests from users and process those requests. In this situation, it is desirable to create a more useful smart speaker system.

The present invention was made in consideration of these problems, and its purpose is to provide technology that allows smart speakers to be used as effective advertising media, or to further improve smart speaker systems.

One aspect of the present invention relates to a server, which includes a receiving means for receiving voice information acquired from a speaker having a microphone and a communication function via a network through the microphone of the speaker, an acquiring means for acquiring voice content not accompanied by an image in response to a distribution request in the received voice information, an authenticating means for performing user authentication based on the received voice information, a selecting means for selecting a voice advertisement not accompanied by an image from a voice advertisement storage means, and a transmitting means for transmitting the acquired voice content and the selected voice advertisement together to the speaker via the network, and the selecting means is configured to select a content of the acquired voice content and a content of the selected voice advertisement based on the content of the acquired voice content and a content of the selected voice advertisement. A candidate audio advertisement is selected using a first relevance among a plurality of relevances including a relevance to the user's speech content in the received audio information and a relevance to an attribute of the authenticated user's account, and a candidate audio advertisement is selected from the selected candidate audio advertisements using another relevance among the plurality of relevances , the first relevance or the other relevance including a relevance to the user's speech content in the received audio information, and the relevance to the user's speech content in the received audio information including a relevance to a non-latest speech content of the user's multiple utterances included in a current interaction session between the speaker and the user .

In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention between devices, methods, systems, computer programs, and recording media storing computer programs, are also valid aspects of the present invention.

The present invention provides technology that allows smart speakers to be used as effective advertising media, or further improves smart speaker systems.

1 is a schematic diagram showing a configuration of an audio advertisement delivery system according to a first embodiment; FIG. 2 is a block diagram showing the functions and configuration of the smart speaker of FIG. 1. FIG. 2 is a hardware configuration diagram of the management server of FIG. 1. FIG. 2 is a block diagram showing the functions and configuration of the management server of FIG. 1 . 5 is a data structure diagram showing an example of an audio content storage unit in FIG. 4. 5 is a data structure diagram showing an example of an audio advertisement storage unit in FIG. 4 . 5 is a data structure diagram showing an example of a voice information storage unit in FIG. 4. 5 is a data structure diagram showing an example of a user information storage unit in FIG. 4 . 5 is a data structure diagram illustrating an example of a session information storage unit in FIG. 4. 2 is a flowchart showing the flow of a series of processes in the management server of FIG. 1 . FIG. 2 is a schematic top view of a user's room. FIG. 13 is a schematic diagram showing a configuration of a voice operation system according to a third embodiment. FIG. 13 is a schematic diagram showing a configuration of a voice operation system according to a fourth embodiment. FIG. 14 is a block diagram showing the functions and configuration of the management server of FIG. 13. 15 is a data structure diagram showing an example of a user information storage unit in FIG. 14 .

In the following, identical or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and duplicate explanations will be omitted where appropriate. In addition, some parts that are not important for the explanation will be omitted in each drawing.

(First embodiment)
In the audio advertisement delivery system according to the first embodiment, a user can use the smart speaker to perform, for example, the following tasks.
- Simple research - Check the weather forecast - Listen to the news - Set alarms - Check schedules - Do calculations - Play music - Control smart home appliances.
The audio advertisement delivery system acquires the user's speech via the microphone of the smart speaker, and by performing voice recognition on the speech, understands that the user is requesting the delivery of audio content (e.g., search results, weather forecast, news, schedule, calculation results, music, etc.). The system prepares the requested audio content and delivers it to the smart speaker, and at this time, inserts the audio advertisement into the audio content to be delivered so that the audio advertisement is played before the audio content is played on the smart speaker.

This audio advertisement may be, for example, an audio advertisement that matches the audio content to be delivered, an audio advertisement based on the content of previous conversations, or an audio advertisement based on sounds around the smart speaker that were picked up by the smart speaker immediately before the delivery of the audio content.

The length of the audio advertisement may be adjusted according to the situation of the dialogue with the user or the situation around the smart speaker. For example, the length of the audio advertisement may be adjusted according to the content of the audio content to be delivered, or important parts of the audio advertisement may be extracted.

The timing of playing the audio advertisement may be determined taking into consideration that the advertising effect is higher when the user is around the smart speaker, and that the user may feel uncomfortable if the audio advertisement is output when the user is talking to the smart speaker or another user. For example, the audio advertisement may be played only when it is determined that the user is around the smart speaker. Also, the audio advertisement may not be played when the user is talking to another user or speaking to the smart speaker. In the latter case, the output of the audio advertisement may be started or resumed when the user stops speaking. Also, the audio advertisement may be output in cooperation with another electronic device, such as a television (hereinafter referred to as TV). For example, after an advertisement is played on the TV, a follow-up report may be output by voice from the smart speaker. In this case, the next TV advertisement may be muted. Alternatively, a related advertisement may be played on the TV after the audio advertisement is played on the smart speaker.

The audio advertising distribution system has a function of acquiring a voiceprint from a user's speech acquired via a smart speaker, and authenticating the user through voiceprint authentication. The audio advertising distribution system also works with other services such as web services and SNS, and can associate an authenticated user in the audio advertising distribution system with a user's account in the other service. In this case, the audio advertising distribution system may select an audio advertisement linked to the authenticated user's account for the authenticated user. For example, the audio advertising distribution system may select an audio advertisement based on information collected by the smart speaker and account attributes. The audio advertising distribution system may also update the account attributes with information collected by the smart speaker.

FIG. 1 is a schematic diagram showing the configuration of an audio advertising distribution system 2 according to a first embodiment. The audio advertising distribution system 2 includes a management server 4, a smart speaker 10, and a TV 12. The management server 4, the smart speaker 10, and the TV 12 are connected to each other so as to be able to communicate with each other via a network 6 such as the Internet. Both the smart speaker 10 and the TV 12 are installed in a room 14 of a user 8. The smart speaker 10 is a speaker having a microphone and a communication function, and is connected to the network 6 as described above, and is configured to be able to perform P2P (Peer to Peer) communication 16 with the TV 12. FIG. 1 shows an example in which the smart speaker 10 and the management server 4 communicate with each other, but there is no limit to the number of smart speakers 10, and there is no limit to the number of users 8.

The user 8 speaks a sentence to the smart speaker 10, such as "I want to hear some song by Taro Komura," "Tell me today's news," "What's the weather like tonight?", or "Tell me about Izumo Taisha Shrine," requesting the delivery of audio content. The microphone of the smart speaker 10 converts the voice spoken by the user 8 into an electrical signal, and the smart speaker 10 transmits the electrical signal obtained as a result of the conversion as an audio signal to the management server 4 via the network 6. The management server 4 performs voice recognition processing on the received audio signal to understand what kind of audio content the user is requesting to be delivered. The management server 4 generates delivery information with an audio advertisement attached to the requested audio content, and transmits it to the smart speaker 10 via the network 6. When the smart speaker 10 receives the delivery information, it first outputs the audio advertisement, and then outputs the audio content.

The smart speaker 10 may or may not have a display, but the content distributed from the management server 4 is not content that combines images and audio, such as still images or videos, but audio content without images (or content consisting of audio only). Similarly, audio advertisements are audio advertisements without images.

Figure 2 is a block diagram showing the functions and configuration of the smart speaker 10 of Figure 1. Each block shown here can be realized in hardware terms by elements and mechanical devices such as a computer CPU, and in software terms by a computer program, etc., but here we depict functional blocks realized by the cooperation of these. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be realized in various ways by combining hardware and software.

The smart speaker 10 comprises a speaker 102, a microphone 104, a communication unit 106, an input unit 108, and a processing unit 110. The communication unit 106 functions as an interface for communication with the network 6, and also functions as an interface for P2P communication 16. The input unit 108 includes physical input mechanisms such as a power button and a volume control button. The processing unit 110 controls the speaker 102, microphone 104, communication unit 106, and input unit 108 to realize various functions of the smart speaker 10.

In this embodiment, it is assumed that the microphone 104 converts the user's speech into an audio signal, the communication unit 106 transmits the audio signal to the management server 4, and the management server 4 performs voice recognition processing on the audio signal. However, the technical idea of this embodiment can also be applied to cases where at least a part of the voice recognition processing is performed in the smart speaker, where the audio content acquisition processing and audio advertisement selection processing described below are performed in the smart speaker, or where the smart speaker is a standalone device. Note that sending the results of the voice recognition performed in the smart speaker to the management server, and sending the audio signal directly from the smart speaker to the management server can both be said to send audio information corresponding to the user's speech to the management server.

Figure 3 is a hardware configuration diagram of the management server 4 in Figure 1. The management server 4 includes a memory 130, a processor 132, a communication interface 134, a display 136, and an input interface 138. Each of these elements is connected to a bus 140 and communicates with each other via the bus 140.

The memory 130 is a storage area for storing data and programs. The data and programs may be stored in the memory 130 permanently or temporarily. The processor 132 executes the programs stored in the memory 130 to realize various functions in the management server 4. The communication interface 134 is an interface for sending and receiving data to and from the outside of the management server 4. For example, the communication interface 134 includes an interface for accessing the network 6. The display 136 is a device for displaying various information, such as a liquid crystal display or an organic EL (Electroluminescence) display. The input interface 138 is a device for receiving input from a user. The input interface 138 includes, for example, a mouse, a keyboard, and a touch panel provided on the display 138.

Figure 4 is a block diagram showing the functions and configuration of the management server 4 in Figure 1. Each block shown here can be realized in hardware terms by elements and mechanical devices such as a computer CPU, and in software terms by computer programs, etc., but here we show functional blocks realized by the cooperation of these. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be realized in various ways by combining hardware and software.

The management server 4 includes an audio content storage unit 402, an audio advertisement storage unit 404, an audio information storage unit 406, a user information storage unit 408, a session information storage unit 410, an audio signal reception unit 412, a voice recognition unit 414, a user authentication unit 416, a session management unit 418, a content acquisition unit 420, an advertisement selection unit 422, an advertisement adjustment unit 424, a transmission information generation unit 426, a transmission unit 428, a timing control unit 430, and an attribute update unit 432.

Figure 5 is a data structure diagram showing an example of the audio content storage unit 402 of Figure 4. The audio content storage unit 402 stores a content ID that identifies the audio content, keywords that characterize the audio content, and audio content data in association with each other. Other metadata such as tags may be used in addition to or instead of keywords.

The data held in the audio content holding unit 402 may be data that has been generated and registered by the management server 4 in advance or upon request. When creating the audio content data, a known voice synthesis technique may be used. Alternatively, the data held in the audio content holding unit 402 may be data that the management server 4 has acquired from a server of another service in advance or upon request.

Figure 6 is a data structure diagram showing an example of the audio advertisement storage unit 404 of Figure 4. The audio advertisement storage unit 404 stores an advertisement ID that identifies an audio advertisement, keywords that characterize the audio advertisement, attributes of the audio advertisement, and audio advertisement data, in association with each other. Other metadata such as tags may be used in addition to or instead of keywords. The data stored in the audio advertisement storage unit 404 may be data received from an advertiser by the entity operating the management server 4.

Figure 7 is a data structure diagram showing an example of the voice information storage unit 406 of Figure 4. The voice information storage unit 406 stores voice information acquired via the microphone 104 of the smart speaker 10. The voice information includes the user's speech content obtained by performing voice recognition on the voice signal in the voice recognition unit 414 described below. The voice information storage unit 406 stores a user ID that identifies the user, a session ID of an interaction session between the smart speaker 10 and the user, and the speech content of the user or the system in the interaction session, in association with each other. The voice information storage unit 406 stores a system-specific ID as a user ID corresponding to the system's speech content.

Figure 8 is a data structure diagram showing an example of the user information storage unit 408 of Figure 4. The user information storage unit 408 stores a user ID, the user's voiceprint data, an account ID that identifies the user's account in other services, account attributes, and an NG ad ID that identifies an advertisement that the user finds unpleasant, in association with each other. The voiceprint data may be acquired when the user registers with the audio ad distribution system 2 for the first time.

9 is a data structure diagram showing an example of the session information storage unit 410 in FIG. 4. The session information storage unit 410 stores the state of the current dialogue session between the smart speaker 10 and the user. The session information storage unit 410 stores the user ID of the user related to the currently existing or maintained dialogue session, the session ID of the dialogue session, and the state of the dialogue session in association with each other. The state of the dialogue session is selected from three states: "Speaking", which indicates that the user is speaking; "Conversing", which indicates that the user is talking to another user; and "Waiting for speech", which indicates that the user is not speaking and is waiting for the next speech by the user or the next speech by the system. When it is determined that the dialogue session has ended, the entry related to the dialogue session is deleted from the session information storage unit 410. In other words, when the session ID of a certain dialogue session is registered in the session information storage unit 410, it is determined that the dialogue session is ongoing and that the user related to the dialogue session is in the vicinity of the smart speaker 10.

Returning to FIG. 4, the audio signal receiving unit 412 receives an audio signal representing the content of the user's speech from the smart speaker 10 via the network 6. As described above, the audio signal is an electrical signal converted from the user's speech by the microphone 104, and in particular is an electrical signal representing the waveform of the speech. The content of the speech includes questions and responses to the smart speaker 10 (or the audio advertising distribution system 2), monologue, and conversations with other users.

The voice recognition unit 414 performs a predetermined voice recognition process on the voice signal received by the voice signal receiving unit 412. The voice recognition unit 414 derives the contents of the user's speech from the voice signal by voice recognition. The voice recognition process in the voice recognition unit 414 may be realized using a known voice recognition technology that uses n-gram or a hidden Markov model.

The user authentication unit 416 extracts or acquires a voiceprint from the voice signal received by the voice signal receiving unit 412. The user authentication unit 416 performs user authentication based on the extracted voiceprint (i.e., voiceprint authentication). The user authentication unit 416 refers to the user information storage unit 408 and determines whether or not there is a voiceprint that matches the extracted voiceprint among the voiceprints stored in the user information storage unit 408. If there is a matching voiceprint, the user authentication unit 416 identifies the user ID corresponding to the voiceprint and associates the identified user ID with the speech content derived in the voice recognition unit 414. In this case, the user who made the utterance corresponding to the voice signal received by the voice signal receiving unit 412 is voiceprint authenticated by the management server 4. If there is no matching voiceprint, the user authentication unit 416 generates an output indicating no match or an unknown user. The management server 4 may start new user registration in response to this output.

The session management unit 418 manages the dialogue session between the smart speaker 10 and the user. The session management unit 418 manages the voice information storage unit 406 and the session information storage unit 410. The session management unit 418 associates a session ID that identifies the dialogue session between the smart speaker 10 and the user with the user ID and speech content associated by the user authentication unit 416, and registers them in the voice information storage unit 406.

The session management unit 418 determines the state of the current dialogue session between the smart speaker 10 and its user based on the user ID and the speech content associated by the user authentication unit 416. The session management unit 418 updates the session information storage unit 410 with the determined state. For example, when the analysis result of the speech content indicates that the speech is in progress, the session management unit 418 determines the state of the current dialogue session to be "in speech". When the analysis result of the speech content indicates that the speech is at the end, the session management unit 418 determines the state of the current dialogue session to be "waiting for speech". When the analysis result of the speech content indicates the end of the dialogue session (for example, when the speech content is a word indicating the end of the dialogue session such as "see you later" or "bye-bye"), the session management unit 418 deletes all entries having the session ID of the dialogue session from the session information storage unit 410. The session management unit 418 may delete from the session information storage unit 410 a dialogue session that has remained in the "waiting for speech" state for a predetermined period of time. Here, the state of the dialogue session is determined based on the content of the utterance, but instead of or in addition to the content of the utterance, the state of the dialogue session may be determined from image information from a separately provided camera. The camera here may be disposed in the smart speaker 10 itself, or a separate camera with a communication function may be used, or a television with a camera function or a computer with a camera function may be used separately.

When the speech content derived in the voice recognition unit 414 includes a request for distribution of voice content, the content acquisition unit 420 acquires the requested voice content from the voice content storage unit 402. For example, when the speech content is a request for distribution of music content such as "I want to hear some song by Taro Komura," the content acquisition unit 420 acquires the requested music content from the voice content storage unit 402. Alternatively, the content acquisition unit 420 may access a server of a music distribution service and acquire the requested music content together with metadata from the server. In this case, the content acquisition unit 420 may register the acquired music content and metadata in the voice content storage unit 402.

When the spoken content is a request for distribution of information content such as "Tell me today's news" or "What's the weather like tonight?", the content acquisition unit 420 acquires the requested information content from the voice content storage unit 402. Alternatively, the content acquisition unit 420 may access a server of an information distribution service and acquire the requested information content in text format together with metadata from the server. In this case, the content acquisition unit 420 may convert the acquired information content in text format into voice data by using a predetermined voice synthesis process. The content acquisition unit 420 may register the information content and metadata that have become voice data in the voice content storage unit 402. The voice synthesis process may be realized using a known voice synthesis technique.

If the spoken content is a request for distribution of search results such as "Tell me about Izumo Taisha Shrine," the content acquisition unit 420 acquires the requested search results from the audio content storage unit 402. Alternatively, the content acquisition unit 420 may access a search service server and acquire the requested search results in text format together with metadata from the server. In this case, the content acquisition unit 420 may convert the acquired text-format search results into audio data by using a predetermined voice synthesis process. The content acquisition unit 420 may register the search results and metadata that have become audio data in the audio content storage unit 402.

The advertisement selection unit 422 selects an audio advertisement to be attached to the audio content acquired by the content acquisition unit 420 from the audio advertisement storage unit 404. The criteria for selecting an audio advertisement in the advertisement selection unit 422 are any one of the following, or any combination of: (1) relevance to the content of the audio content acquired by the content acquisition unit 420; (2) relevance to the content of the user's utterance in the current interactive session between the smart speaker 10 and the user, which is stored in the audio information storage unit 406; and (3) relevance to the attributes of the authenticated user's account.

For example, in regard to (1), in response to a request for delivery of a search result of "Tell me about Izumo Taisha," the content acquisition unit 420 acquires audio data of audio content "Izumo Taisha was once...." The advertisement selection unit 422 acquires the keyword "Izumo Taisha, God, Matchmaking" corresponding to "Izumo Taisha was once..." acquired by the content acquisition unit 420 from the audio content storage unit 402. The advertisement selection unit 422 refers to the audio advertisement storage unit 404 and selects audio data of an audio advertisement "Want to travel to Izumo Taisha? Then consult ABC Travelers" that has the keyword "Izumo Taisha, Matchmaking" corresponding to the acquired keyword "Izumo Taisha, God, Matchmaking." In this way, by comparing the keywords of the audio content with the keywords of the audio advertisement, the advertisement selection unit 422 selects an audio advertisement that corresponds to the content of the audio content acquired by the content acquisition unit 420.

For example, in regard to (2), between the smart speaker 10 and the user (user) “Can I make it to the station by taxi in time?”
(Smart Speaker 10) "We'll make it."
(User) "What's the weather like tonight?"
Assume that the following dialogue is taking place. In response to a request for delivery of information content "What's the weather tonight?", the content acquisition unit 420 acquires voice data of voice content "The weather in C region tonight is a sudden rain shower, and the temperature is...". The advertisement selection unit 422 refers to the voice information storage unit 406 and identifies "Can you make it to the station by taxi?" as the user's utterance content in the current dialogue session between the smart speaker 10 and the user. The advertisement selection unit 422 extracts the keyword "station, taxi" from the identified utterance content "Can you make it to the station by taxi?" The advertisement selection unit 422 refers to the voice advertisement storage unit 404 and selects voice data of a voice advertisement "ZZZ taxi dispatch service coming soon" having the keyword "taxi, dispatch" corresponding to the extracted keyword "station, taxi". In this way, by referring to the voice information storage unit 406, the advertisement selection unit 422 can select a voice advertisement based on the user's utterance content in the current dialogue session between the smart speaker 10 and the user.

In the above example, if criterion (1) is used instead of criterion (2), the advertisement selection unit 422 acquires from the audio content storage unit 402 the keyword "C region, rain, low temperature" corresponding to "Tonight's weather in C region is showers, the temperature is..." acquired by the content acquisition unit 420. The advertisement selection unit 422 refers to the audio advertisement storage unit 404 and selects audio data for an audio advertisement saying "CB's Hyper Umbrella will last for 10 years!", which has the keyword "umbrella, rain" corresponding to the acquired keyword "C region, rain, low temperature." In this way, even if the content of the dialogue between the smart speaker 10 and the user is the same, the audio advertisement selected may differ depending on the criterion used.

For example, in regard to (3), in response to a request for delivery of information content "Tell me today's news," the content acquisition unit 420 acquires voice data of the voice content "At around 6 o'clock this morning, there was a fire in City B, Prefecture A, ...". In addition, the user who spoke "Tell me today's news" is authenticated by voiceprint authentication in the user authentication unit 416, and the user ID "B102" of the user is identified. The advertisement selection unit 422 acquires the attributes "children, male, single" of the account corresponding to the identified user ID "B102" from the user information storage unit 408. The advertisement selection unit 422 refers to the audio advertisement storage unit 404, and selects the audio data of the audio advertisement "If you come to City F, you can ride a steam locomotive," which has the attributes "children, male" corresponding to the acquired attributes "children, male, single." Furthermore, if the attributes of the account corresponding to the identified user ID "B102" are "adult, female, single," the advertisement selection unit 422 refers to the audio advertisement storage unit 404 and selects the audio data of the audio advertisement "Want to take a trip to Izumo Taisha Shrine? Then consult ABC Travel Agency," which has the attribute "single, adult" corresponding to the attributes. In this way, by comparing the attributes of the authenticated user's account with the attributes of the audio advertisement, the advertisement selection unit 422 selects the audio advertisement corresponding to the attributes of the authenticated user's account.

Alternatively, if the attributes of the account corresponding to the specified user ID "B102" are "adult, male, married", the advertisement selection unit 422 first selects four audio advertisements corresponding to the attributes as candidates: "For Christmas presents, we recommend a ring made of XX precious metal", "For fire insurance, leave it to XYZ Fire and Marine Insurance", "ZZZ taxi dispatch service that will come quickly", and "Want to go on a trip to Izumo Taisha? Then consult with ABC Travel Agency". Furthermore, the advertisement selection unit 422 acquires the keyword "A prefecture, B city, fire" corresponding to "Around 6 o'clock this morning, there was a fire in B city, A prefecture, ..." acquired by the content acquisition unit 420 from the audio content storage unit 402. The advertisement selection unit 422 selects the audio data of the audio advertisement "For fire insurance, leave it to XYZ Fire and Marine Insurance" that has the keyword "fire, fire, insurance" corresponding to the acquired keyword "A prefecture, B city, fire" from the four selected candidates. In this way, it is possible to combine criteria (1) and (3), by selecting candidates using criteria (3) and then narrowing them down using criteria (1).

For example, between the smart speaker 10 and the user, (user) "I want to listen to some songs by Taro Komura"
(Smart Speaker 10) "How about some Christmas songs?"
(User) "Okay then."
Assume that the following dialogue is taking place. In response to a music content distribution request of "I want to hear some song by Taro Komura," the content acquisition unit 420 acquires audio data of Taro Komura's Christmas song. The management server 4 inquires of the user via the smart speaker 10 whether a Christmas song is OK. When the management server 4 receives an affirmative response of "Okay then," from the user, it attaches an audio advertisement to the acquired audio data of Taro Komura's Christmas song and transmits it to the smart speaker 10. Here, the advertisement selection unit 422 acquires from the audio content storage unit 402 the keywords "Taro Komura (lyrics and composition), B anime (theme song), C movie (insert song), Christmas song, ring" corresponding to Taro Komura's Christmas song acquired by the content acquisition unit 420. The advertisement selection unit 422 refers to the audio advertisement storage unit 404 and selects two audio advertisements as candidates, "Anime B, on air from 6pm on Friday!" (keyword: "Anime B, Friday, 6pm") and "For a Christmas present, we recommend a ring made of XX precious metal" (keyword: "Christmas, present, ring"), which have keywords corresponding to the acquired keywords "Taro Komura (lyrics and composition), Anime B (theme song), Movie C (insert song), Christmas song, ring". Furthermore, the advertisement selection unit 422 acquires from the user information storage unit 408 the attributes "adult, male, married" of the account corresponding to the user ID "A101" of the user authenticated by voiceprint authentication. Of the two selected candidates, the advertisement selection unit 422 selects the audio data of the audio advertisement "For a Christmas present, we recommend a ring made of XX precious metal" which has the attribute "adult" corresponding to the acquired attribute "adult, male, married". Moreover, if the user ID of the user authenticated by voiceprint authentication is "A105", the advertisement selection unit 422 selects, from the two selected candidates, the audio data of the audio advertisement "Anime No. 2, on air from 6pm on Friday!", which has the attribute "female, child" corresponding to the acquired attribute "child, female, single". In this way, it is also possible to combine criteria (1) and (3) in such a way that candidates are selected based on criterion (1) and then narrowed down based on criterion (3).

Also possible are a combination of criteria (1) and (2), a combination of criteria (2) and (3), and a combination of the three criteria (1), (2), and (3).
Alternatively, in addition to the criteria (1), (2), and (3), the audio advertisement may be selected based on sounds around the smart speaker 10 collected by the smart speaker 10 or conversations between users. For example, if the smart speaker 10 picks up a conversation between a user and another user, such as "There's no tissue paper," or "Yeah, let's order some from an EC site," the advertisement selection unit 422 may extract the keyword "tissue paper" from the content of the conversation and select an audio advertisement promoting the extracted "tissue paper." Also, for example, if the smart speaker 10 picks up the cries of a dog or cat, the advertisement selection unit 422 may identify the keyword "dog, cat" from the cries and select an audio advertisement promoting dog food or cat food related to the identified "dog, cat."

The advertisement adjustment unit 424 determines whether to adjust the length of the audio advertisement selected by the advertisement selection unit 422. If the advertisement adjustment unit 424 determines to adjust the length of the audio advertisement, it extracts a portion (e.g., a relatively important portion) from the selected audio advertisement by applying a predetermined extraction algorithm to the audio advertisement. The advertisement adjustment unit 424 may determine whether to adjust the length of the audio advertisement based on the content of the audio content acquired by the content acquisition unit 420 and/or the state of the current interactive session between the smart speaker 10 and the user.

For example, the advertisement adjustment unit 424 may refer to the session information storage unit 410 and determine to adjust if the state of the session corresponding to the selected audio advertisement is "talking", and determine not to adjust if the state is "waiting for speech". Alternatively, the advertisement adjustment unit 424 may determine the length of the audio advertisement based on the content of the audio content acquired by the content acquisition unit 420. For example, the advertisement adjustment unit 424 may determine the length of the audio advertisement according to the playback time of the audio content. For relatively long audio content, the audio advertisement may be played multiple times. Also, for example, if the audio content is information content such as news or a weather forecast, the advertisement adjustment unit 424 may determine to adjust since there is a high probability that the user wants to obtain the desired information as quickly as possible.

Technology is known for extracting important parts of an audio advertisement, such as parts where a person is speaking, parts where a person is speaking loudly, and parts where background sounds gradually get louder, and this technology is used in hard disk recorders and the like. The specified extraction algorithm may be configured using this known technology. Note that the advertisement adjustment unit 424 may adjust the volume of the audio advertisement in addition to or instead of the length of the audio advertisement.

The transmission information generation unit 426 generates one piece of transmission information by combining the audio content acquired by the content acquisition unit 420 and the audio advertisement selected by the advertisement selection unit 422. If the length of the audio advertisement has been adjusted by the advertisement adjustment unit 424, the audio advertisement whose length has been adjusted by the advertisement adjustment unit 424 is used instead of the audio advertisement selected by the advertisement selection unit 422. The transmission information generation unit 426 configures the transmission information so that when the transmission information is received and played by the smart speaker 10, the audio advertisement is played back temporally before the audio content is played back. For example, when the transmission information includes a header, audio content, and audio advertisement, the transmission information generation unit 426 may generate the transmission information so that the header, audio advertisement, and audio content are arranged in this order.

The transmitting unit 428 transmits the transmission information generated by the transmission information generating unit 426 to the smart speaker 10 via the network 6. When the smart speaker 10 receives the transmission information via the network 6, it first plays the audio advertisement included in the transmission information, and then plays the audio content included in the transmission information. Alternatively, the timing control unit 430 described below may control the output of audio from the smart speaker 10 via the network 6. In this case, the timing control unit 430 causes the smart speaker 10 to first output the audio advertisement included in the transmission information, and then output the audio content included in the transmission information. In either case, the audio advertisement and the audio content are played back consecutively. That is, there is no other audio between the audio advertisement and the audio content. In particular, the audio advertisement is played back immediately before the audio content.

Alternatively, audio advertisements may be embedded in the middle of the audio content, or may be output after the audio content is output.

The timing control unit 430 communicates with the smart speaker 10 via the network 6 and controls the timing of audio output from the smart speaker 10. The timing control unit 430 refers to the session information storage unit 410 and determines whether or not a user is present around the smart speaker 10. If a session ID of an interactive session between the smart speaker 10 and the user is stored in the session information storage unit 410, the timing control unit 430 determines that a user is present around the smart speaker 10 or detects the presence of a user. If such a session ID is not stored in the session information storage unit 410, the timing control unit 430 determines that no user is present around the smart speaker 10.

The timing control unit 430 restricts the output of audio advertisements from the smart speaker 10 when the presence of a user is not detected around the smart speaker 10, or when the state of the dialogue session between the smart speaker 10 and the user stored in the session information storage unit 410 is "speaking" or "conversing". When the timing control unit 430 detects the presence of a user, it allows the output of audio advertisements from the smart speaker 10. When the state of the dialogue session is changed to "waiting for speech", the timing control unit 430 allows the output of audio advertisements from the smart speaker 10.

The timing control unit 430 controls the timing of the output of the audio advertisement so that the output of the TV 12 associated with the smart speaker 10 and the audio advertisement output from the smart speaker 10 are linked. For example, the timing control unit 430 controls the smart speaker 10 so that the output of the audio advertisement "This product introduced on TV is..." starts from the smart speaker 10 at the timing when the advertisement "Continue on the speaker!" from the TV 12 ends. In this case, the timing control unit 430 acquires the number of the channel currently being broadcast from the TV 12 via the network 6. The timing control unit 430 acquires the broadcast schedule in advance from the server of another service. The timing control unit 430 can specify the content, start timing, and end timing of the advertisement to be broadcast on the TV 12 from the acquired channel number and broadcast schedule. The timing control unit 430 selects an audio advertisement related to the specified content from the audio advertisement storage unit 404 and transmits it to the smart speaker 10 (which may or may not be attached to the audio content). The timing control unit 430 controls the smart speaker 10 to start outputting the transmitted audio advertisement at the end of the advertisement aired on the TV 12.

The attribute update unit 432 updates the attributes of the user's account with the voice information collected by the smart speaker 10. For example, if the account ID shown in the user information storage unit 408 in FIG. 8 is for an account of a search service, this account ID is given to a user who visits the site of the search service. The attributes of this user's account are derived from information about what this user is searching for, and are registered as attributes in the user information storage unit 408 in FIG. 8.

On the other hand, the management server 4 can grasp the user's preferences by analyzing the contents of the dialogue between the smart speaker 10 and the user, and the contents of the conversation between the user and other users. The attribute update unit 432 updates the attributes in the user information storage unit 408 with the preferences thus grasped via the smart speaker 10. The attribute update unit 432 may provide the update contents to the search service server in association with the account ID. This allows the search service to obtain user preferences that could not be obtained by the search service so far, and the browser's advertisement output can be optimized using the preferences thus obtained. Note that the user's preferences and attributes obtained via the smart speaker 10 and the preferences and attributes grasped by the search service may be retained in a manner that allows them to be distinguished from each other. This makes it possible to handle cases where audio advertisements are selected using only one of the preferences and attributes.

When an audio advertisement is output from the smart speaker 10, the attribute update unit 432 determines whether a direct or indirect expression of disinterest has been received from the user via the smart speaker 10. If the attribute update unit 432 has received an advertisement ID of the corresponding output audio advertisement, the attribute update unit 432 registers the advertisement ID of the corresponding output audio advertisement in the user information storage unit 408 as an NG advertisement ID in association with the user ID of the user. When selecting an audio advertisement, the advertisement selection unit 422 refers to the user information storage unit 408 and excludes an audio advertisement identified by an NG advertisement ID stored in association with the user ID of the authenticated user from the selection targets. The advertisement selection unit 422 may exclude an audio advertisement that is a successor to an audio advertisement identified by an NG advertisement ID from the selection targets. The advertisement selection unit 422 may exclude an audio advertisement that corresponds to or has the same attributes or keywords as the attributes or keywords of the audio advertisement identified by the NG advertisement ID from the selection targets.

The operation of the management server 4 configured as above will now be described.
10 is a flowchart showing a series of processing steps in the management server 4 of FIG. 1. The management server 4 receives a voice signal representing a request for delivery of voice content from the smart speaker 10 via the network 6 (S302). The management server 4 performs a voice recognition process on the received voice signal (S304) to identify the requested voice content. The management server 4 acquires the requested voice content from the voice content storage unit 402 or from the outside (S306). The management server 4 selects a voice advertisement from the voice advertisement storage unit 404 (S308). The management server 4 determines whether or not the length of the selected voice advertisement needs to be adjusted (S310). If it is determined that adjustment is necessary (YES in S310), the management server 4 adjusts the length of the voice advertisement (S312). The management server 4 generates transmission information based on the audio content acquired in step S306 and the audio advertisement selected in step S308 (if NO in step S310) or the audio advertisement whose length was adjusted in step S312 (if YES in step S310) (S314). The management server 4 transmits the generated transmission information to the smart speaker 10 via the network 6 (S316). The management server 4 determines whether the current timing is suitable for outputting the audio advertisement (S318). If the current timing is suitable (YES in S318), the management server 4 first causes the smart speaker 10 to output the audio advertisement (S320), and then causes the smart speaker 10 to output the audio content (S322).

In the above-described embodiment, examples of the storage unit are a hard disk and a semiconductor memory. Furthermore, based on the description in this specification, it will be understood by those skilled in the art who have read this specification that each unit can be realized by a CPU (not shown), an installed application program module, a system program module, a semiconductor memory that temporarily stores the contents of data read from a hard disk, and the like.

According to the management server 4 of this embodiment, audio advertisements are played in accordance with the playback of audio content on the smart speaker 10. This makes it possible to provide audio advertisements in accordance with the delivery of audio content. Furthermore, in this embodiment, audio advertisements are played before the playback of audio content. In this case, the likelihood that the user will hear the audio advertisement is higher than when the audio advertisement is played after the playback of audio content. This makes it possible to provide more effective advertisements.

In addition, in the management server 4 according to this embodiment, audio advertisements are selected based on the content of the audio content, the audio information obtained via the smart speaker 10, and the attributes of the authenticated user's account. Audio advertisements selected in this manner are highly likely to be in line with the user's preferences and requests. Therefore, audio advertisements that are more appealing to users can be provided.

In addition, in the management server 4 according to this embodiment, the timing of audio advertisement output from the smart speaker 10 is appropriately controlled. Therefore, it is possible to output audio advertisements that do not interfere with the user's conversation or speech. Alternatively, it is possible to provide audio advertisements in cooperation with other electronic devices such as the TV 12.

In this embodiment, in relation to the smart speaker 10 being able to acquire surrounding sounds and conversations between users, the management server 4 may determine whether or not child abuse is occurring by understanding the voice context. The management server 4 may also edit voice data related to child abuse and provide it to a specified investigative agency. The investigative agency can listen to the entire voice data.

In this embodiment, the user ID is identified by voiceprint authentication by the user authentication unit 416, and the attributes corresponding to this user ID are identified by referring to the user information storage unit 408. In this case, the management server 4 may change the manner of outputting audio content or audio advertisements depending on the user attributes. For example, if the user attribute is a child, the management server 4 may use as simple words as possible in the audio content or audio advertisements, and delete or rephrase foul language. Alternatively, if the user attribute is an elderly person, the management server 4 may increase the volume or make the pronunciation clearer in the audio content or audio advertisements.

In this embodiment, the case where the user is authenticated by the user authentication unit 416 has been described, but this is not limiting, and user authentication may not be required. In this case, the user attributes are not reflected in the selection of the audio advertisement. Here, in this embodiment, the operation of outputting the audio advertisement in S320 has been described, but in addition to outputting this audio advertisement, the management server 4 can also store this output status. Examples of output status are "the target advertisement has been played to the end," "the target advertisement has been stopped midway," and "in addition to playing the target advertisement, additional information regarding the product targeted by the advertisement has been output." "The target advertisement has been played to the end" can be realized by the smart speaker 10 reporting this to the management server 4 when it has output the target audio data to the end. In addition, both the stopping midway and the output of the additional information are controlled by the management server 4, so they can naturally be managed.

The technical idea according to the present embodiment may be expressed by the following items.
(Item 1)
a function of receiving a distribution request via a network from a speaker having a microphone and a communication function;
A function of acquiring audio content without images in response to the received distribution request;
a function of selecting an audio advertisement without an image from an audio advertisement storage means;
a computer program for causing the server to realize a function of transmitting the acquired audio content and the selected audio advertisement together to the speaker via the network.
(Item 2)
receiving a distribution request via a network from a speaker having a microphone and a communication function;
Acquiring audio content without accompanying images in response to the received distribution request;
selecting an audio advertisement without an image from an audio advertisement storage means;
transmitting the captured audio content together with the selected audio advertisement to the speaker over the network.

Second Embodiment
In the second embodiment, multiple smart speakers are placed at different positions in a real space, and each of them is connected via a network to a management server similar to the management server 4 in the first embodiment.

Figure 11 is a schematic top view of a room 202 of a user 204. In this room 202, a fixed first smart speaker 208, a fixed second smart speaker 210, a fixed third smart speaker 212, a fixed fourth smart speaker 214, a movable fifth smart speaker 216, and a TV 206 are arranged. Each smart speaker communicates with a management server via a network. Note that although five smart speakers are shown in Figure 11, there is no limit to the number of smart speakers. Each smart speaker may be installed on the wall, floor, or ceiling of the room 202.

(1) Automatic Determination of Smart Speaker Location The management server records and manages the location of each smart speaker in the room 202. This location may be registered in the management server 4 by the user 204. Alternatively, the management server may automatically determine the location of each smart speaker using the microphones and speakers of the five smart speakers.

The management server determines the relative positions between the smart speakers by having the other smart speakers 10 detect the sound output by one smart speaker. For example, when the positions of the second smart speaker 210, the third smart speaker 212, and the fourth smart speaker 214 are known and the position of the first smart speaker 208 is to be determined, the management server causes the speaker of the first smart speaker 208 to output a pulse of sound of a predetermined wavelength. The management server acquires the time at which the pulse of sound of a predetermined wavelength was received from each of the second smart speaker 210, the third smart speaker 212, and the fourth smart speaker 214. The management server calculates the propagation time of the pulse from the acquired times, and calculates the distance from the calculated propagation time and the speed of sound. The management server calculates the position of the first smart speaker 208 from each of the calculated distances and the known positions of the second, third, and fourth smart speakers 210, 212, and 214.

The fifth smart speaker 216 is, for example, a smart speaker attached to a robot, and can move by itself. When the positions of the first, second, third and fourth smart speakers 208, 210, 212, 214 are known, the management server can track the position of the fifth smart speaker 216 by the position calculation process described above. In addition, when the fifth smart speaker 216 determines the positions of other smart speakers based on its own position, it may move to a position where it is easy to receive sounds emitted by those smart speakers.

As components of the system shown in FIG. 11, smart speakers or smartphones equipped with both a microphone and a speaker are desirable, but televisions, radios, and other electrical devices that generally only have speakers can also assist with audio playback because they have speakers. Some electrical devices also have communication capabilities, which allows speaker output from multiple positions. The position of the electrical device may be notified to the management server by the user by setting it, or it may be automatically determined by the management server using the position calculation process described above or wireless communication radio waves.

(2) Use of Movable Smart Speaker In audio output, the way the target user hears the sound may change depending on the position of the speaker. Therefore, the management server may control the fifth smart speaker 216 to move to a position where the sound can be heard more appropriately.

Furthermore, TV 206 and other devices generally do not have a microphone function, and therefore cannot participate in the above-mentioned position calculation process in this state. However, if the fifth smart speaker 216 moves to the position of TV 206 and takes over the microphone function of TV 206, TV 206 will also be able to participate in the position calculation process.

(3) Voice output according to user's position When the position of each smart speaker is known, if the position of the user 204 is known, the smart speaker closest to the user 204 can be identified. When the user 204 utters a voice output request such as "Turn on the TV," the five smart speakers convert the utterance into a voice signal and transmit it to the management server. The management server performs voice recognition processing on the voice signal to understand the voice output request of the user 204. The management server identifies the smart speaker corresponding to the position of the user 204 in the room 202. In particular, the management server identifies the second smart speaker 210 closest to the position of the user 204. At this time, the management server compares the volume of the voice when the microphone of each smart speaker receives the user's voice, and identifies the second smart speaker 210 with the largest volume as the smart speaker closest to the position of the user 204. Alternatively, if the user 204 uses a smartphone, the management server can identify the position of the user 204 by acquiring the current position of the smartphone.

The management server transmits the audio accompanying the video being played on the TV 206 to the second smart speaker 210 identified as described above. In this way, the user 204 can receive the audio output of the TV 206 from the second smart speaker 210 that is closest to the user 204.

Alternatively, if the speakers of the smart speakers have directionality, the management server may control the directivity. For example, if the first, second, third and fourth smart speakers 208, 210, 212, 214 have output directionality, the management server controls each smart speaker so that the audio output of each speaker is directed toward the position of the user 204. The management server transmits audio accompanying the video streamed on the TV 206 to each smart speaker. In this case, each smart speaker outputs audio toward the user 204.

In addition, a directional smart speaker may indicate to the user the direction of the current audio output in a visible manner. For example, an arrow indicating the directionality may be displayed on the top surface of the smart speaker using an LED or the like.

According to this example, for example, when a user who has instructed a smart speaker to play content and other users are present in room 202, audio can be output for the instructed user. By adopting a configuration in which a server process is assigned to each user, the first smart speaker may be controlled to face the position of the first user, and the second smart speaker may be controlled to face the position of the second user, so that the first smart speaker and the second smart speaker output audio simultaneously. By adopting a configuration in which one smart speaker has multiple audio output devices and the server assigns a server process to each user in the server, one smart speaker may be controlled to output audio directed to the position of the first user and audio directed to the position of the second user simultaneously.

The technical idea according to the present embodiment may be expressed by the following items.
(Item 3)
A system comprising a plurality of speakers, each having a microphone and a communication function,
A system configured to determine the relative positions of speakers by having one speaker detect the sound output by another speaker.
(Item 4)
A server that communicates with a plurality of speakers via a network, each of the speakers having a microphone and a communication function, the plurality of speakers being arranged at different positions within the same real space;
The server,
means for receiving a voice output request from a user in the real space via any one of the plurality of speakers;
means for identifying a speaker corresponding to a position of the user;
and means for transmitting audio content to the identified speaker.
(Item 5)
A server that communicates with a plurality of speakers via a network, each of the speakers having a microphone and a communication function, the plurality of speakers being arranged at different positions within the same real space;
The server,
means for receiving a voice output request from a user in the real space via any one of the plurality of speakers;
and a means for controlling directivity of at least one of the plurality of speakers so that sound is output toward the position of the user.

Third Embodiment
12 is a schematic diagram showing the configuration of a voice operation system 232 according to a third embodiment. The voice operation system 232 includes a management server 234, a smart speaker 240, a TV 242, and a smartphone 248. The management server 234, the smart speaker 240, the TV 242, and the smartphone 248 are communicatively connected via a network 236 such as the Internet. The smart speaker 240 and the TV 242 are both installed in a room 244 of a user 238. The smart speaker 240 is configured to be capable of P2P communication 246 with the smartphone 248.

When operating the TV 242 by voice, the user 238 speaks a sentence expressing an operation instruction, such as "Turn on the TV," toward the smart speaker 240. The microphone of the smart speaker 240 converts the voice spoken by the user 238 into an electrical signal, and the smart speaker 240 transmits the electrical signal obtained as a result of the conversion as an audio signal to the management server 234 via the network 236. The management server 234 performs voice recognition processing on the received audio signal and understands that the user 238 is requesting to turn on the TV 242. The management server 234 generates an instruction signal to realize the requested operation, i.e., to turn on the TV 242, and transmits it to the TV 242 via the network 236. When the TV 242 receives the instruction signal via the network 236, it transitions from a power-off state to a power-on state.

In this way, control and operation via smart speaker 10 is basically performed by voice. However, if there are users other than user 238 in room 244, some users 238 may dislike voice control. Even if only user 238 is in room 244, there may be cases where user 238 wishes to avoid voice control depending on the content of the control. In such cases, voice operation system 232 makes it possible to perform text control on the system side (management server 234) of smart speaker 240 via smartphone 248.

To enable operation on the smartphone 248, the smartphone 248 downloads and installs an application dedicated to the management server 234. When the application is started on the smartphone 248, the application obtains the URL of the management server 234 from the smart speaker 240 via P2P communication 246 or a local network. The application establishes a connection with the management server 234 using the obtained URL. When a connection between the smartphone 248 and the management server 234 is established, the operation content input to the smartphone 248 is transmitted to the management server 234 through the connection. The management server 234 generates and transmits an instruction signal to realize the received operation content.

For example, when the management server 234 receives a text string "Turn on the TV" from the smartphone 248, the management server 234 analyzes the received text string and understands that the user 238 is requesting to turn on the TV 242. The management server 234 generates an instruction signal to perform the requested operation, i.e., to turn on the TV 242, and transmits the instruction signal to the TV 242 via the network 236. When the TV 242 receives the instruction signal via the network 236, it transitions from a power-off state to a power-on state.

According to the voice operation system 232 of this embodiment, the user 238 can selectively use voice operation and operation via the smartphone 248 depending on the situation.

In this embodiment, the management server 234 receives an operation instruction from the user 238 via the smart speaker 240 or the smartphone 248. However, this is not limited to this. For example, as in the first embodiment, the management server 234 may receive a request for delivery of audio content from the user 238 via the smart speaker 240 or the smartphone 248.

The technical idea according to the present embodiment may be expressed by the following items.
(Item 6)
A server that communicates with a speaker having a microphone and a communication function via a network,
means for processing requests received from a user via a microphone of said speaker;
and means for processing requests received from the user via other electronic devices in communication with the speaker.

(Fourth embodiment)
The fourth embodiment relates to layering in a management server. In this embodiment, a server process (or a server) is changed depending on the speaker. The server can recognize who is speaking by voiceprint authentication. The server process (or a server) that the target user normally uses performs the processing.

Figure 13 is a schematic diagram showing the configuration of a voice operation system 252 according to the fourth embodiment. The voice operation system 252 includes a management server 254, a smart speaker 260, and a TV 262. The management server 254, the smart speaker 260, and the TV 262 are communicatively connected via a network 256 such as the Internet. The smart speaker 260 and the TV 262 are both installed in a room 264, and there are three users in the room 264 (a first user 266, a second user 268, and a third user 270).

When operating the TV 262 by voice, the first user 266 speaks a sentence expressing an operation instruction such as "Turn on the TV" to the smart speaker 260. The microphone of the smart speaker 260 converts the voice spoken by the first user 266 into an electrical signal, and the smart speaker 260 transmits the electrical signal obtained as a result of the conversion as an audio signal to the management server 254 via the network 256. The management server 254 performs voiceprint authentication on the received audio signal to identify the first user 266. The management server 254 selects a server process corresponding to the identified first user 266, and the selected server process processes subsequent requests. When the TV 262 receives an instruction signal from the selected server process of the management server 254 via the network 236, it transitions from a power-off state to a power-on state. In the management server 254, a server process different from the server process corresponding to the first user 266 is assigned to the speech of the second user 268 or the third user 270 different from the first user 266.

Figure 14 is a block diagram showing the functions and configuration of the management server 254 in Figure 13. Each block shown here can be realized in hardware terms by elements and mechanical devices such as a computer CPU, and in software terms by computer programs, etc., but here we have depicted a functional block realized by the cooperation of these. Therefore, those skilled in the art who have read this specification will understand that these functional blocks can be realized in various ways by combining hardware and software.

The management server 254 comprises a user information storage unit 272, an audio signal receiving unit 274, a user authentication unit 276, and a server process group 278. The server process group 278 includes a number of server processes SP1, SP2, SP3, ... each assigned to a specific user. Below, a case will be described in which the server process differs for each user, but in other embodiments, the server itself may be different for each user. Different servers will naturally result in different server processes.

Figure 15 is a data structure diagram showing an example of the user information storage unit 272 of Figure 14. The user information storage unit 272 stores a user ID, voiceprint data of the user, and the ID of a server process assigned to the user, in association with each other.

Returning to FIG. 14, the voice signal receiving unit 274 receives a voice signal representing the speech content of one of the three users 266, 268, 270 from the smart speaker 260 via the network 256.

The user authentication unit 276 extracts or acquires a voiceprint from the voice signal received by the voice signal receiving unit 274. The user authentication unit 276 performs voiceprint authentication based on the extracted voiceprint. The user authentication unit 276 refers to the user information storage unit 272 and determines whether or not there is a voiceprint that matches the extracted voiceprint among the voiceprints stored in the user information storage unit 272. If there is a matching voiceprint, the user authentication unit 276 identifies the user ID and server process ID corresponding to that voiceprint. If there is no matching voiceprint, the user authentication unit 276 generates an output indicating no match or an unknown user.

When the user authentication unit 276 identifies a server process ID, a server process having the identified server process ID is started among the server processes included in the server process group 278. The started server process performs subsequent processing on the audio signal accepted by the audio signal acceptance unit 274.

Each server process included in the server process group 278 realizes the operation function of the electronic device as described in the third embodiment. In other embodiments, the server process may realize, for example, the delivery function of audio content as described in the first embodiment.

For example, if the first user 266 is a resident of the room 264, the server process of the first user 266 is given the authority to control the TV 262. If the second user 268 and the third user 270 are visitors who have come to the room 264 of the first user 266, the server processes of these users are not given the authority to control the TV 262. Therefore, when the second user 268 or the third user 270 operates the TV 262 by voice, the server process of the second user 268 or the third user 270 sends a request to operate the TV 262 to the server process of the first user 266. The server process that receives the investigation request inquires of the first user 266 whether it is OK to perform the operation, and if consent is obtained from the first user 266, executes the target operation.

Alternatively, a first user 266 who is the owner of smart speaker 260 may set permissions for a second user 268 and a third user 270 who are guests (visitors). For example, it may be possible to control a light via smart speaker 260, but not to make purchases on an e-commerce site via smart speaker 260.

According to the voice operation system 252 of this embodiment, by assigning a server process to each user, the operations that can be executed, the information that can be accessed, the authority, etc. can be made different for each user.

In this embodiment, the case has been described where the management server 254 has multiple server processes, and the management server 254 receives an audio signal, performs voiceprint authentication, and identifies the server process to be used, but this is not limited to the above. For example, when multiple servers exist, the audio signal from the smart speaker 260 may be sent to all the servers, and voiceprint authentication may be performed on each server. Alternatively, a server or server process of any one user, or any one server or server process, may receive an audio signal, extract only the audio signal of the target user, and transfer it to the target server or server process.

In this embodiment, the first user 266 who issues the operation instruction and the TV 262 to be operated are in the same room 264, but the present invention is not limited to this, and the electronic device to be operated may be remotely operated. For example, assume that the second user 268 wants to start the air conditioner in his/her own room (different from the room 264) when visiting the room 264 of the first user 266. The second user 268 speaks a sentence representing an operation instruction, "Turn on the air conditioner in my room," to the smart speaker 260. The management server 254 understands the request of the second user 268 by voiceprint authentication and voice recognition. The management server 254 transfers the request of the second user 268 to another management server connected to the smart speaker in the room of the second user 268. At this time, a voice signal is attached as authentication data.

The technical idea according to the present embodiment may be expressed by the following items.
(Item 7)
A server that communicates with a speaker having a microphone and a communication function via a network,
means for identifying a speaker by analyzing a voice signal acquired through a microphone of the speaker;
and means for performing processing of the speech signal using a server process assigned to the identified speaker.

In addition to the smart speaker, each of the above embodiments also describes the operation of linking with a television or computer connected to a network. After the smart speaker outputs an audio advertisement, additional information can also be displayed on the television or computer in response to the user's voice control. This is achieved by the management server 4 sending a URL to be displayed on the television or computer, and information indicating that a request for additional information has been made by the smart speaker or management server 4 can also be added to the parameters of this URL. This allows the system to which the request is made to know that the request used the management server 4 or smart speaker. Here, URL parameters are used as an example of a transmission method, but notification may be made by other methods.

The above describes the configuration and operation of the system according to the embodiments. These embodiments are merely examples, and those skilled in the art will understand that various modifications are possible in the combination of each component and each process, and that such modifications are also within the scope of the present invention. Combinations of the embodiments are also possible.

2 Audio advertising delivery system, 4 Management server, 6 Network, 8 User, 10 Smart speaker.

Claims (10)

a receiving means for receiving voice information acquired via a network from a speaker having a microphone and a communication function, the voice information being acquired via the microphone of the speaker;
an acquisition means for acquiring audio content not accompanied by an image in response to a distribution request in the received audio information;
an authentication means for performing user authentication based on the received voice information;
A selection means for selecting an audio advertisement without an image from the audio advertisement storage means;
a transmission means for transmitting the acquired audio content and the selected audio advertisement together to the speaker via the network;
the selection means selects a voice advertisement candidate using a first relevance among a plurality of relevances among a relevance with the content of the acquired voice content, a relevance with the content of the user's utterance in the received voice information, and a relevance with an attribute of the authenticated user's account, and selects a voice advertisement from the selected voice advertisement candidates using another relevance among the plurality of relevances ;
The first relevance or the other relevance includes a relevance to a user's speech content in the received voice information,
A server , wherein the relevance of the received voice information to the user's speech content includes relevance to non-latest speech content among a plurality of utterances of the user included in a current dialogue session between the speaker and the user . The server according to claim 1, wherein the selection means selects audio advertisement candidates using at least the relevance to the content of the acquired audio content and the relevance to the attributes of the authenticated user's account, and then selects an audio advertisement from the selected audio advertisement candidates. The server according to claim 1 or 2, wherein the selection means selects audio advertisement candidates using relevance to attributes of the authenticated user's account, and further selects an audio advertisement from the selected audio advertisement candidates using relevance to the content of the acquired audio content. The server according to any one of claims 1 to 3, wherein selecting a candidate audio advertisement or an audio advertisement using relevance to the content of the acquired audio content includes selecting a candidate audio advertisement or an audio advertisement using a keyword related to the content of the acquired audio content. The server according to claim 1, wherein the multiple associations further include associations with sounds around the speaker acquired through a microphone of the speaker or with the contents of conversations between the user and other users. a control unit that controls a timing of audio output from the speaker via the network,
The server according to any one of claims 1 to 5 , wherein the control means limits the output of audio advertisements from the speaker when the presence of a user is not detected or when it is determined that the user's conversation is continuing. a control unit that controls a timing of audio output from the speaker via the network,
6. The server according to claim 1, wherein the control means restricts audio advertisements to be output from the speaker when the presence of a user is detected. The method further includes updating means for updating an attribute of an account of the authenticated user using the received voice information,
The server according to claim 1 , wherein the selection means uses updated attributes of an authenticated user's account to select an audio advertisement. and updating means for updating attributes of an account of the authenticated user using an expression of feedback from the user in response to the audio advertisement received via the speaker;
The server according to claim 1 , wherein the selection means uses updated attributes of an authenticated user's account to select an audio advertisement. The server according to any one of claims 1 to 7, wherein the selection means, when receiving an expression of feedback from the user regarding a particular audio advertisement via the speaker indicating no interest in the particular audio advertisement, excludes audio advertisements associated with the particular audio advertisement from the selection targets.

Images