KR102355903B1 - Apparatus and method for providing contents - Google Patents

Abstract

The present disclosure relates to a method for providing content by an electronic device and an electronic device for providing the content. A method for providing content by an electronic device includes: acquiring an audio signal including a voice input of the user from a user of the electronic device; extracting an audio feature vector from the obtained audio signal; recognizing a user's voice in the audio signal by inputting the extracted audio feature vector into a voice recognition model; obtaining at least one candidate content based on the recognized user's voice; and outputting the searched at least one candidate content. may include

Description

ELECTRONIC DEVICES AND METHOD FOR PROVIDING CONTENT {APPARATUS AND METHOD FOR PROVIDING CONTENTS}

The present disclosure relates to an electronic device and method for providing content. More particularly, it relates to an electronic device and a method for providing content through an electronic device or an external device connected to the electronic device.

Korea is currently entering an aging society at a fast pace, and the number of dementia patients is increasing every year as fast as it is entering an aging society. Since the increase in the number of dementia patients can cause many economic and psychological problems for the families of the dementia patients, the dementia disease of the elderly population needs to be addressed at the social level, not the individual level. In particular, most cases of dementia develop into a dementia disease that is judged to be dementia, and when dementia occurs, it is estimated that the socio-economic cost of treating dementia patients reaches 10 trillion won.

Currently, the technologies for the treatment of dementia in the elderly are products that improve muscle strength of the elderly, or as health care products, they have been developed with a lot of emphasis on improving the health of the elderly rather than the treatment of dementia. The development of technology for treatment is required.

In addition, as the cost to be borne by the individual due to the dementia treatment cost of the elderly is greatly increased, there is a problem in that individuals have low access to the dementia treatment of themselves or their relatives. Therefore, there is a demand for the development of a technology capable of effectively treating dementia patients even at low cost.

Patent Document 1: Patent Registration Publication No. 10-1303344

According to an embodiment, an electronic device and method for providing content may be provided.

Also, according to an embodiment, an electronic device and method capable of outputting content through an external device connected to the electronic device may be provided.

According to an embodiment of the present disclosure for achieving the above-described technical problem, a method for an electronic device to provide content includes: acquiring an audio signal including a user's voice input from a user of the electronic device; extracting an audio feature vector from the obtained audio signal; recognizing a user's voice in the audio signal by inputting the extracted audio feature vector into a voice recognition model; obtaining at least one candidate content based on the recognized user's voice; and outputting the searched at least one candidate content. may include

According to an embodiment, the method for providing content by the electronic device includes transmitting information on at least one searched candidate content to an external device connected to the electronic device by wire or wirelessly; may further include.

In addition, according to another embodiment of the present disclosure for solving the above technical problem, an electronic device for providing content includes: a microphone for acquiring ambient sound of the electronic device; a speaker outputting an audio signal related to the content; a memory storing one or more instructions; and one or more processors that provide the content by executing the one or more instructions. including, wherein the at least one processor obtains an audio signal including the user's voice input from the user of the electronic device by executing the one or more instructions, and extracts an audio feature vector from the obtained audio signal; , by inputting the extracted audio feature vector into a speech recognition model, recognizes a user's voice in the audio signal, obtains at least one candidate content based on the recognized user's voice, and finds at least one candidate Content can be output.

According to an embodiment, the at least one processor may transmit information on the found at least one candidate content to an external device connected to the electronic device by wire or wirelessly.

Further, according to another embodiment of the present disclosure for solving the technical problem, the method may further include: acquiring an audio signal including a user's voice input from a user of an electronic device; extracting an audio feature vector from the obtained audio signal; recognizing a user's voice in the audio signal by inputting the extracted audio feature vector into a voice recognition model; obtaining at least one candidate content based on the recognized user's voice; and outputting the searched at least one candidate content. A computer-readable recording medium recording a program for executing the method on a computer, including a computer-readable recording medium, may be provided.

1 is a diagram schematically illustrating a process in which an electronic device provides content according to an exemplary embodiment.
2 is a flowchart of a method for providing content by an electronic device according to an exemplary embodiment.
3 is a flowchart illustrating a method for an electronic device to authenticate a user of an electronic device according to an embodiment.
4 is a diagram for describing a specific method for an electronic device to authenticate a user of the electronic device according to an exemplary embodiment.
5 is a flowchart illustrating a specific method of activating a voice recognition model by an electronic device according to an exemplary embodiment.
6 is a diagram for explaining a process of providing content by an electronic device according to another exemplary embodiment.
7 is a diagram for describing a process in which an electronic device provides content according to another exemplary embodiment.
8 is a flowchart illustrating a specific method for an electronic device to acquire at least one candidate content, according to an embodiment.
9 is a flowchart illustrating a method in which an electronic device determines a content playback mode based on an operating state of an external device and transmits the determined content playback mode and at least one candidate content together, according to an embodiment.
10 is a diagram for describing a specific method of changing a playback mode based on sensing information obtained from an external device while an electronic device is playing content in an external device, according to an embodiment.
11 is a block diagram of an electronic device that provides content according to an embodiment.
12 is a block diagram of a server connected to an electronic device according to an exemplary embodiment.
13 is a block diagram of an external device connected to an electronic device according to an exemplary embodiment.
14 is a diagram for describing a structure of an external device connected to an electronic device according to an exemplary embodiment.
15 is a diagram for describing a process in which an electronic device and an external device operate in conjunction with each other according to an exemplary embodiment.
16 is a diagram for describing a process in which an electronic device, a server, and an external device interwork with each other to provide content, according to an exemplary embodiment.
17 is a diagram for describing an operation process of an external device connected to an electronic device according to an exemplary embodiment.

Terms used in this specification will be briefly described, and the present disclosure will be described in detail.

The terms used in the present disclosure have been selected as currently widely used general terms as possible while considering the functions in the present disclosure, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the term and the contents of the present disclosure, rather than the simple name of the term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, with reference to the accompanying drawings, the embodiments of the present disclosure will be described in detail so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a diagram schematically illustrating a process in which an electronic device provides content according to an exemplary embodiment.

According to an embodiment, the electronic device 1000 may include a processor 120 , a network interface 130 , and a memory 140 . However, not all illustrated components are essential components, and the electronic device 1000 may be implemented with more components than the illustrated components, and the electronic device 1000 may be implemented with fewer components. may be For example, the electronic device 1000 may further include a microphone (not shown) for acquiring a user's voice or a speaker (not shown) for outputting an audio signal.

According to an embodiment, the memory 140 may include a content providing service model 154 for providing content based on artificial intelligence. The content providing service model 154 is a voice recognition service model

The assistant service model 152 may include a user authentication model 142 , a voice recognition activation model 144 , a voice recognition model 146 , and a content recommendation model 148 . For example, the processor 120 of the electronic device 1000 may include one of the user authentication model 142 , the voice recognition activation model 144 , the voice recognition model 146 , or the content recommendation model 148 stored in the memory 140 . By executing at least one, the content may be provided. The user authentication model 142 , the voice recognition activation model 144 , the voice recognition model 146 , and the content recommendation model 148 include one or more instructions that may be executed by the processor 120 .

For example, by executing the in-memory user authentication model 142 , the electronic device 1000 may perform a user authentication operation of confirming whether the current user of the electronic device is a registered user. Also, the electronic device 1000 may determine whether to activate the voice recognition model for providing the voice recognition service by executing the voice recognition activation model 144 . Also, the electronic device 1000 may recognize the user's voice in the audio signal by executing the voice recognition model, and provide content based on the recognized user's voice. Also, by using the content recommendation model 148 , the electronic device 1000 may provide content suitable to the user based on the recognized user voice and user information.

According to an embodiment, the electronic device 1000 may provide content based on a user input of the electronic device requesting the content. For example, the electronic device 1000 obtains a content transmission request from the external device 4000 and operation information 104 regarding an operation state of the external device, and responds to the content transmission request to receive the content information 102 may be transmitted to the external device 4000 .

According to an embodiment, the electronic device 1000 obtains an audio signal including a user's voice input or a user's keyword input, and at least based on a character string or keyword input determined by recognizing the user's voice in the obtained audio signal. You can search for one content.

The content according to the present disclosure may be media content that can be output through an output unit of an electronic device or an external device. For example, the content provided by the electronic device is audio content that can be perceived aurally, and may include audio data that can be output through a speaker. Also, the content provided by the electronic device may include video data as video content that can be visually recognized. According to another embodiment, the content provided by the electronic device may be multimedia content including both audio data and video data.

According to an embodiment, the electronic device 1000 includes a smart phone, a tablet PC, a PC, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, a GPS ( global positioning system) device, digital camera, speaker, or other mobile or non-mobile computing device. According to an embodiment, the electronic device 1000 may be connected to the server 2000 through the network 3000 . The electronic device 1000 may provide at least one content by interworking with the server 2000 connected through the network 3000 .

According to an embodiment, the network 3000 includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, It may include a satellite communication network and a combination thereof, and may include other data communication networks for connecting the electronic device 1000 and the server 2000 . In addition, the server 2000 connected to the electronic device 1000 through a network may include other computing devices or computing service providing devices capable of providing content by interworking with the electronic device.

According to an embodiment, the external device 4000 may include a processor 160 , a network interface 170 , a memory 180 , and an output unit 190 . However, not all illustrated components are essential components, and the external device 4000 may be implemented by more components than the illustrated components, and the external device 4000 is implemented by fewer components than that. it might be For example, the external device may further include a module for discharging a fragrance through at least one sensor or an external device capable of detecting pressure or a user's position.

For example, the external device 4000 may obtain information on the content from the electronic device 1000 and output the obtained information on the content through the output unit 190 . According to an embodiment, the external device 4000 is a device in which an output unit for outputting content is built-in, and may further include a pillow-type housing. The external device 4000 may be communicatively connected to another external device 5000 through the network 6000 , and may communicate via an electronic device or directly access the server 2000 .

According to an embodiment, the network 3000 includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, It may include a satellite communication network and a combination thereof, and may further include other data communication networks for connecting the external device 4000 and other external devices 5000 . The network 6000 may be the same network as the network 3000 used by the electronic device 1000 , but may be a separate network independent of the network 3000 . According to an embodiment, the external device 4000 may perform a mental and emotional treatment function for dementia patients or the elderly who are external device users by outputting the content obtained from the electronic device 1000 .

2 is a flowchart of a method for providing content by an electronic device according to an exemplary embodiment.

In S210 , the electronic device 1000 may obtain an audio signal including a user's voice input. For example, the electronic device 1000 may acquire the user's voice through at least one microphone included in the electronic device. According to another embodiment, the electronic device 1000 may obtain at least one of a voice input, a key input, a touch input, and a motion input from the user through the user input interface.

In S220, the electronic device 1000 may extract an audio feature vector from the audio signal. For example, the electronic device 1000 may extract an audio feature vector from an audio signal using a pre-trained artificial neural network. According to an embodiment, the artificial neural network may include a deep neural network (DNN), for example, a Convolutional Neural Network (CNN), a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), or an RBM. (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or deep Q-Networks, etc., but is not limited to the above-described example.

In addition, according to an embodiment, the audio feature vector includes a linear predictive coefficient, a cepstrum, a mel frequency cepstral coefficient (MFCC), or a filter bank energy for each frequency band. ) may include at least one of the values, but is not limited thereto.

In S230 , the electronic device 1000 may recognize the user's voice in the audio signal by inputting the audio feature model to the voice recognition model. For example, the speech recognition model may include an automatic speech recognition (ASR) model, a natural language understanding model, and a natural language generation model in order to recognize a user's speech. The speech recognition model may extract an audio feature vector from an input audio signal and output a character string corresponding to the extracted audio feature vector.

For example, the automatic speech recognition model may receive a user's voice and convert the inputted user's voice into text, and the natural language understanding model uses the converted text of the user's voice based on a grammatical analysis of the Intention information may be output, and a natural language generator (NLG) may output a new character string conforming to a predetermined sentence rule based on registered character string information and intention information determined by the natural language understanding model.

According to an embodiment, the speech recognition model extracts audio feature vectors in the audio learning signal, and the speech recognition model for the extracted feature vector is based on a neural network that is pre-trained to minimize the LOSS determined by comparing the output value and the correct answer value. can be a model of

According to another embodiment, before recognizing the user's voice in the audio signal using the voice recognition model, the electronic device 1000 extracts an audio feature vector from the audio signal using the voice recognition activation model, and the extracted audio It is possible to determine whether to activate the speech recognition model based on the feature vector. A specific method for the electronic device to determine whether to activate the voice recognition model will be described in detail with reference to FIG. 5 .

In S240 , the electronic device 1000 may acquire at least one candidate content based on the recognized user's voice. For example, the electronic device 1000 may determine a character string obtained from an output value of the voice recognition model as a keyword, and search for at least one candidate content from the Internet connected through a network based on the determined keyword.

According to an embodiment, when a user of the electronic device is authenticated, the electronic device 1000 may acquire user information of the authenticated user. The electronic device 1000 may acquire user-customized content by searching for candidate content based on the recognized user's voice and user information. A specific method for the electronic device 1000 to acquire at least one candidate content will be described in detail with reference to FIGS. 8 to 9 .

In S250, the electronic device 1000 may output at least one searched candidate content. For example, the electronic apparatus 1000 may directly provide the at least one searched candidate content through the output unit of the electronic apparatus, but transmit information on the searched candidate content to an external device connected to the electronic apparatus through the external device. The searched candidate content may be output.

According to another embodiment, although not shown in FIG. 2 , the electronic device 1000 executes an application for providing the content providing service, and receives a user touch input on the content through a user interface provided by the application. may obtain more. The electronic device 1000 may acquire the at least one candidate content by using a keyword obtained based on the user touch input instead of the user's voice.

3 is a flowchart illustrating a method for an electronic device to authenticate a user of an electronic device according to an embodiment.

In S320 , the electronic device 1000 may authenticate the user based on the audio feature vector for each frame of the audio signal. For example, the electronic device 1000 may pre-store a user model including a registered user feature vector in which the user's voice characteristic is reflected through the user's voice registration process. The user's voice characteristic may include at least one of a frequency characteristic, an amplitude characteristic, a tremor characteristic, and a speed characteristic of the user's voice.

After the user voice registration process is completed, the electronic device 1000 extracts a user feature vector from an audio signal newly acquired from the user and compares the extracted user feature vector with a registered user feature vector in the user model to authenticate the user. can When user authentication is completed, the electronic device 1000 may distinguish each of the plurality of users of the electronic device by using user identification information capable of identifying each user. A method in which the electronic device 1000 authenticates a user using a user model will be described in detail with reference to FIG. 4 .

In S340 , when the user is authenticated, the electronic device 1000 may obtain user information. For example, when a current user is authenticated as a user of the electronic device using the user model, the electronic device 1000 may acquire information about the user using the authenticated user identification information.

According to an embodiment of the present disclosure, the user information includes the user's biometric information, the user's hobby information, the user's interest information for the content, the user's location information, the user's health information, and a medical database connected to the electronic device. It may include at least one of information about the user's medical record that can be obtained from , or information about a content history requested by the user through the electronic device, but is not limited thereto.

4 is a diagram for describing a specific method for an electronic device to authenticate a user of the electronic device according to an exemplary embodiment.

Referring to FIG. 4 , the user authentication process of the electronic device 1000 will be divided into a user registration process 402 and a process 404 in which the electronic device provides a voice recognition service after registration.

For example, in S412 , the electronic device 1000 may obtain a first audio signal for a user registered utterance from the user A 1002 . In S414 , the electronic device 1000 may determine a first user registered feature vector from the first audio signal. For example, the electronic device 1000 extracts a first audio feature vector from the first audio signal using a pre-trained artificial neural network-based model similarly to step S220 of FIG. 2 , and the extracted first audio feature The vector may be used to determine the first user registered feature vector. According to an embodiment, the first user registered feature vector may correspond to the first audio feature vector, and may be an output value of a layer output from one layer in the artificial neural network.

In S416 , the electronic device 1000 may obtain a second audio signal for the user registered utterance from the user A 1002 . In S418 , the electronic device 1000 may determine a second user registration feature vector from the second audio signal. As described above, the electronic device 1000 may obtain a plurality of utterances from the user A 1002 and determine a plurality of user registered feature vectors from an audio signal corresponding to the obtained utterances. The electronic device 1000 may generate the user A model 419 by using the user registration feature vectors acquired a plurality of times, and store the generated user model A in a memory.

According to an embodiment, the electronic device 1000 sets a weight to user registered feature vectors acquired a plurality of times, and a single user registered feature vector is generated by weighting and summing the plurality of user registered feature vectors according to the set weight. A registered user characteristic vector may be generated by matching with the user's unique identification information, and a user A model may be generated using the generated registered user characteristic vector.

In S422 , the electronic device 1000 may obtain an audio signal from the user A 1002 after completing the user registration process. In S424 , the electronic device 1000 may extract an audio feature vector from the obtained audio signal and determine a user feature vector using the extracted audio feature vector.

In S426 , the electronic device 1000 may obtain a registered user feature vector from the pre-stored user A model 419 . In S428 , the electronic device 1000 may determine the similarity by comparing the registered user feature vector obtained from the user model A with the user feature vector. According to an embodiment, the similarity may be a similarity score indicating a degree of similarity between vectors.

In S432 , the electronic device 1000 may determine whether the similarity determined by comparing the registered user feature vector with the user feature vector is equal to or greater than a threshold. According to an embodiment, the electronic device 1000 sets a registered user feature vector and a user feature vector based on a dot product between vectors, a log likelihood between vectors, or a Euclidean distance between vectors. The degree of similarity between them can be determined as a score.

In S436 , when the determined similarity is equal to or greater than a preset threshold, the electronic device 1000 may authenticate that the speaker of the current audio signal is user A of the electronic device. In S434 , when the similarity is less than a preset threshold, the electronic device 1000 may reject the user's authentication. When the user's authentication is rejected, the electronic device 1000 may acquire a new audio signal from the user again.

5 is a flowchart illustrating a specific method of activating a voice recognition model by an electronic device according to an exemplary embodiment.

In S520, the electronic device 1000 may determine whether to activate the voice recognition model based on the audio feature vector for each frame of the obtained audio signal. For example, the electronic device 1000 does not always leave the voice recognition model in an activated state, but uses the voice recognition activation model to activate the voice recognition model only when the user is authenticated, thereby unnecessary power according to the voice recognition model activation. consumption can be reduced.

For example, the electronic device 1000 extracts an audio feature vector from an audio signal including the user's voice using an artificial neural network-based model, and inputs the extracted audio feature vector to the voice recognition activation model to generate a voice recognition model. Decide whether to activate it or not. According to another embodiment, the electronic device 1000 may determine whether to activate the voice recognition model by acquiring the user feature vector used in the user authentication process and inputting the acquired user feature vector into the voice recognition activation model. .

More specifically, the electronic device 1000 may store an activation keyword for activating the voice recognition model in advance in the memory. The electronic device 1000 obtains the audio feature vector by inputting the currently acquired feature vector of the user's audio signal to the artificial neural network-based model, and inputs the acquired audio feature vector to the voice recognition activation model to obtain the audio feature vector. It can be determined whether the corresponding string matches the activation keyword stored in memory. The electronic device 1000 determines the similarity between the string recognized from the audio signal and the activated keyword stored in advance in the memory by using the voice recognition activation model, and determines to activate the voice recognition model when the determined similarity is greater than or equal to a preset threshold, , when the determined similarity is less than the threshold, it may be determined not to activate the speech recognition model. In S540 , as the voice recognition model is activated, the electronic device 1000 may recognize the user's voice in the audio signal by inputting the audio feature vector into the voice recognition model.

6 is a diagram for explaining a process of providing content by an electronic device according to another exemplary embodiment.

In S602, the electronic device 1000 may provide a guide message for content output. For example, the electronic device 1000 may induce a user input by providing a guide message 612 related to content input on the display even before a user voice input or a keyword input is obtained from the user. The guide message provided by the electronic device 1000 may be visually provided on the display, or may be provided audibly through a speaker of the electronic device. Also, the electronic device 1000 may provide at least one of a user interface 614 for inducing a user's keyword input or a user interface 614 for inducing a user's voice input together with a guide message.

Also, according to an embodiment, the electronic device may store an application for providing content in advance in the memory. The electronic device 1000 may output a guide message through an application service provided by executing an application stored in the memory.

In S604 , the electronic device 1000 may obtain an audio signal including the user's voice. For example, when the user of the electronic device acquires a voice signal corresponding to “Play me something from Luis Armstrong”, the electronic device 1000 extracts an audio feature vector from the audio signal including the acquired voice signal, and extracts The user's voice can be recognized by inputting the obtained audio feature vector to the voice recognition model. According to an embodiment, while recognizing a user's voice, the electronic device 1000 may display "listening" or visual content to indicate that a voice recognition process is currently in progress.

In S606 , the electronic device 1000 may search for at least one candidate content according to the recognized user's voice, and output the searched candidate content on the screen of the electronic device. Referring to FIG. 6 , the electronic device 1000 may recognize Luis Armstrong among the user's voice included in the audio signal, and output the recognized Luis Armstrong album as candidate content. A detailed method for the electronic device 1000 to acquire candidate content will be described in detail with reference to FIG. 8 .

7 is a diagram for describing a process in which an electronic device provides content according to another exemplary embodiment.

In S702, the electronic device 1000 acquires user information. For example, the electronic device 1000 may acquire information of a user whose authentication has been completed by completing the user authentication process of step S320 . For example, the electronic device 1000 may acquire user interest information related to the user's hobby as the user's information.

According to an embodiment, the electronic device 1000 displays “Hobby?” As examples of hobbies and guide messages such as “Choose a hobby”, icons symbolizing each hobby and examples such as “Dancing” “Walking” “Singing” and “Swimming” can be output together. The electronic device 1000 may acquire the user's interest information based on a user's input for selecting one of the output icons.

In S704 , the electronic device 1000 may determine at least one candidate content based on user information. For example, the electronic device 1000 may provide at least one candidate content based on the user's voice recognized through the voice recognition model and the user's information. However, according to another embodiment, when interest information related to a user's hobby is input, the electronic device 1000 may provide candidate content based on the user's interest information. For example, referring to FIG. 7 , when the user selects the 'swimming' icon, the electronic device 1000 may output content such as 'ocean wave' related to swimming as candidate content.

When the user of the electronic device is a dementia patient, the electronic device 1000 according to an embodiment reproduces the reproduced candidate content based on the user's information, thereby helping to recover the memory of the dementia patient. For example, the electronic device 1000 may obtain interest information indicating that the user's hobby is 'swimming', search for content related to the sound of waves according to the obtained interest information, and provide the found content about the sound of waves. have. Accordingly, the user of the electronic device 1000 may recall past memories by listening to content related to sound waves reproduced in the electronic device. Accordingly, the electronic device 1000 according to the present disclosure may help a user with dementia by playing candidate content based on interest information obtained from the user.

8 is a flowchart illustrating a specific method for an electronic device to acquire at least one candidate content, according to an embodiment.

In S820 , the electronic device 1000 may obtain a content list from a network connected to the electronic device by using a string corresponding to the recognized user's voice as a search keyword. For example, the electronic device 1000 obtains a character string obtained by inputting an audio signal to a voice recognition model. The electronic device 1000 may access the Internet through a network and obtain a content list for candidate contents using a string obtained from the connected Internet as a search keyword.

In S840, the electronic device 1000 may determine the priority of contents in the contents list based on user information. For example, when the user's voice included in the audio signal is recognized as 'summer' or 'summer', the electronic device 1000 may determine 'summer story' or 'prince of the sea' related to 'female' or 'summer'. You can obtain a content list including songs such as 'A Midsummer Night's Dream', 'Let's Go to the Beach' and 'Swimming in the Pool'.

In addition, the electronic device 1000 accesses user information stored in advance in the electronic device, identifies that the user's hobby information is 'swimming' among the user information, and prioritizes current candidate contents by 'swimming in the pool', 'Let's go to the beach', 'Prince of the sea', and 'A Midsummer Night's Dream' can be decided in the same order.

In S860 , the electronic device 1000 may acquire some contents in the contents list as at least one candidate contents according to the determined priority. For example, when the number of currently displayable content is two, the electronic device 1000 selects 'swimming in the pool' and 'Let's go to the beach', which have a higher priority in a content list including four content, as candidate content can be obtained by The electronic apparatus 1000 may output the obtained candidate contents through the electronic apparatus or an external device connected to the electronic apparatus.

According to another embodiment, although not shown in FIG. 8 , the electronic device 1000 may obtain first operation information regarding an operation state of the electronic device or second operation information regarding an operation state of an external device connected to the electronic device. can Also, the electronic device 1000 changes the priority determined in step S840 based on at least one of the obtained first operation information and the second operation information, and selects some contents in the content list according to the changed priority as at least one candidate. It can also be obtained as content.

According to an embodiment, the first operation information may include at least one of information on an operation history of the electronic device and information on a connection state between the electronic device and the external device, and the second operation information includes the It may include at least one of information on a power supply state of an external device, location information of the external device, or an output value of a pressure sensor in the external device.

Accordingly, the electronic device 1000 can provide not only user-customized content suitable for user information, but also provide content that more closely matches the state of the device from which the content is output based on the operating state of the electronic device or the current external device. can be printed out.

9 is a flowchart illustrating a method in which an electronic device determines a content playback mode based on an operating state of an external device and transmits the determined content playback mode and at least one candidate content together, according to an embodiment.

Hereinafter, it is assumed that the external device is a pillow-type content output apparatus including a pressure sensor, a network interface, and a speaker, and the electronic apparatus 1000 outputs searched candidate content through the external device.

In S920 , the electronic device 1000 may determine at least one content reproduction mode required to reproduce the at least one candidate content based on the user information and the second operation information. In S940 , the electronic device 1000 may output information on the determined content reproduction mode together with the candidate content.

For example, the reproduction mode may indicate an operating state of a device required to reproduce content in order to reproduce content. According to an embodiment, the playback mode may include a volume parameter for content playback, a playback speed parameter for determining a content playback speed, a screen brightness parameter for determining screen brightness, and the like.

For example, the electronic device 1000 receives the output value of the pressure sensor as second operation information from the external device, and when the received output value of the pressure sensor is equal to or greater than a preset threshold, sets the content reproduction mode to the first The playback mode may be determined, and information on the first playback mode and candidate content may be transmitted together to an external device. Here, the first playback mode may include a low volume level parameter. The external device 4000 may reproduce the candidate content in the first playback mode (eg, low volume level and slow playback speed) based on the information on the candidate content and the information on the first playback mode received from the electronic device.

According to another embodiment, the electronic device 1000 receives the output value of the pressure sensor as second operation information from the external device, and when the received output value of the pressure sensor is smaller than a preset threshold, The mode may be determined as the second playback mode, and information on the second playback mode and candidate content may be transmitted together to an external device. According to an embodiment, the second playback mode may include a high volume level parameter. The external device 4000 may reproduce the candidate content in the second playback mode by using the information on the candidate content received from the electronic device and the information on the second playback mode.

10 is a diagram for describing a specific method of changing a playback mode based on sensing information obtained from an external device while an electronic device is playing content in an external device, according to an embodiment.

In S1002, the electronic device 1000 may acquire at least one candidate content. Since S1002 may correspond to process S250 in FIG. 2 , a detailed description thereof will be omitted. In S1004 , the electronic device 1000 may transmit information on at least one candidate content to the external device 4000 . For example, the electronic apparatus 1000 may transmit, to the external device 4000 , at least one of audio data or auditory data for reproducing the content as information on the candidate content. Also, the electronic device 1000 may transmit information on at least one candidate content to the external device 4000 in a streaming manner.

In S1006 , the external device 4000 may reproduce the first content using information about the content received from the electronic device. In S1008 , the external device 4000 may acquire first sensing information using at least one sensor in the external device. According to an embodiment, the first sensing information is information on an output value of a proximity sensor for sensing a location of a user of an external device or an output value of a pressure sensor for sensing a pressure intensity of a user's touch on a housing within the external device may include

In S1010 , the external device 4000 may transmit the first sensing information, identification information of the currently playing content, and information on the location of the currently playing content to the electronic device 1000 . For example, the identification information of the currently playing content may include metadata about the content, and the information on the location of the currently playing content may include synchronization data related to the currently playing content.

In S1012 , the electronic apparatus 1000 may determine the first reproduction mode based on the first sensing information received from the external device. In S1014 , the electronic device 1000 may acquire the first partial content, which is a partial content in the first content, based on the information on the location of the content. For example, the electronic apparatus 1000 identifies what content is currently being played in the external device based on identification information of the content received from the external device, and uses the information about the location of the content received from the external device. It may be identified whether any partial content in the identified content is currently being played in the external device. That is, the electronic device 1000 may identify the first partial content in the first content based on the information on the location of the currently playing content received from the external device 4000 .

In S1016 , the electronic device 1000 may transmit the first partial content and information on the first playback mode together to the external device 4000 . In S1018 , the external device 4000 may reproduce the first partial content based on the first reproduction mode. Since the method for the electronic device to determine the reproduction mode may correspond to S920, a detailed description thereof will be omitted.

In S1020, when the reproduction of the first content is completed, the external device 4000 reproduces the second content. In S1022 , the external device 4000 may acquire second sensing information using at least one sensor of the external device while playing the second content. In S1024 , the external device 4000 transmits the second sensing information, identification information of the currently playing content, and information on the location of the currently playing content to the electronic device.

In S1026, the electronic device 1000 may determine the second reproduction mode based on the second sensing information. A method for the electronic device 1000 to determine the second reproduction mode based on the second sensing information may correspond to S920, and thus a detailed description thereof will be omitted. In S1028 , the electronic device 1000 may acquire the second partial content, which is the partial content of the second content, based on the information on the location of the content received from the external device.

In S1030 , the electronic device 1000 transmits the second partial content and information on the second playback mode to the external device 4000 . In S1032 , the external device 4000 reproduces the second partial content according to the second reproduction mode based on the information about the second partial content and the second reproduction mode received from the electronic device.

As described above, the electronic device 1000 according to the present disclosure determines a playback mode based on sensing information received from the external device while content is being played in the external device 4000 , and according to the determined playback mode, the current external device 1000 . You can change the playback mode being used on the device. In addition, the electronic apparatus 1000 according to the present disclosure identifies a location where the content is reproduced in the external device 4000, so that the content can be streamed in real time by being synchronized with the order in which the content is output from the external device.

Also, although not shown in FIG. 10 , the electronic device 1000 according to the present disclosure obtains a content transmission request from an external device connected to the electronic device, and outputs at least one candidate content in response to the content transmission request of the external device. can also

Also, according to an embodiment, before a content transmission request is received from the external device 4000 , the electronic apparatus 1000 may store state information regarding the reproduction state of content previously output from the electronic apparatus. When a content reproduction request is received from the external device 4000 , the electronic device 1000 determines whether the most recently output content before the content reproduction request is received, based on the state information on the content reproduction state By identifying , content may be continuously output without interruption.

11 is a block diagram of an electronic device that provides content according to an embodiment.

According to an embodiment, the electronic device 1000 may include a processor 1400 and a memory 1402 . However, not all illustrated components are essential components, and the electronic device 1000 may be implemented with more components than the illustrated components, and the electronic device 1000 may be implemented with fewer components. may be For example, the electronic device 1000 may further include a user input interface (not shown), an output unit (not shown), or a network interface (not shown) for communicating with a server or an external device.

For example, the user input interface (not shown) may obtain a user input for the user to control the electronic device 1000 . For example, the user input interface may include a microphone and acquire an audio signal including a voice of a user of the electronic device. Also, the user input interface (not shown) may include, but is not limited to, a key pad, a touch pad, a jog switch, and the like.

The user input interface may receive an input sequence from the user according to a guide message output by the electronic device 1000 on the display. Also, the user input interface 1100 may acquire the user's voice shown in FIGS. 1 to 10 , or acquire at least one of a key input, a touch input, and a motion input from the user.

The output unit (not shown) may output an audio signal, a video signal, or a vibration signal, and the output unit (not shown) may include a display unit and a sound output unit. The display unit (not shown) includes a screen for displaying and outputting information processed by the electronic device 1000 . Also, the screen may display a response operation to a user's voice, such as a guide message and an error message, as a text image or a video image.

The sound output unit outputs audio data received from the network interface or stored in advance in the memory 1402 . For example, the sound output unit may include a speaker for outputting one or more audio data. Also, the sound output unit outputs an audio signal related to a function (eg, a guide message shown in FIG. 6 ) performed by the electronic device 1000 . The vibration motor may output a vibration signal. For example, the vibration motor may output a vibration signal together with outputs of functions performed by the electronic device 1000 (eg, a guide message illustrated in FIG. 6 ).

The network interface may include one or more components that allow the electronic device 1000 to communicate with an external device (not shown) and the server 2000 . The external device (not shown) may be a computing device such as the electronic device 1000 , a sensing device, or a speaker device capable of outputting audio data, but is not limited thereto. For example, the network interface may include a short-range communication unit, a mobile communication unit, and a broadcast receiving unit.

The processor 1400 generally controls the overall operation of the electronic device 1000 .

According to an embodiment, the processor 1400 according to the present disclosure executes programs stored in the memory 1402 to perform the functions of the electronic device 1000 described in FIGS. 1 to 10 . In addition, the processor 1400 may be composed of one or a plurality of processors, and the one or the plurality of processors is a general-purpose processor such as a CPU, AP, DSP (Digital Signal Processor), etc., a graphics-only processor such as a GPU, or artificial intelligence (AI). ) may be a dedicated processor.

According to an embodiment, when the processor 1400 is implemented as a plurality of processors or graphics-only processors or artificial intelligence-only processors, at least some of the plurality of processors or graphics-only processors or artificial intelligence-only processors include the electronic device 1000 and It may be mounted on another electronic device connected to the electronic device 1000 or a server.

For example, the processor 1400 executes programs stored in the memory 1402 , whereby the at least one processor executes the one or more instructions, so that the audio including the user's voice input from the user of the electronic device Obtaining a signal, extracting an audio feature vector from the obtained audio signal, and inputting the extracted audio feature vector into a speech recognition model, to recognize a user's voice in the audio signal, and At least one candidate content may be acquired based on the at least one candidate content, and the searched at least one candidate content may be output.

Also, the processor 1400 may transmit information on the at least one searched candidate content to an external device connected to the electronic device by wire or wirelessly by executing programs stored in the memory 1402 .

In addition, the processor 1400 may authenticate the user based on the audio feature vector for each frame of the obtained audio signal, and when the user is authenticated, obtain user information of the authenticated user.

Also, the processor 1400 determines whether to activate the speech recognition model based on the audio feature vector for each frame of the obtained audio signal, and as the speech recognition model is activated, converts the audio feature vector into a speech recognition model. By inputting into the , it is possible to recognize the user's voice in the audio signal.

In addition, the processor 1400 obtains a content list from a network connected to the electronic device by using a string corresponding to the recognized user's voice as a search keyword, and the obtained content list based on the obtained user information Priority of my contents may be determined, and some contents in the contents list may be acquired as the at least one candidate contents according to the determined priority.

In addition, the processor 1400 obtains first operation information on the operation state of the electronic device or second operation information on the operation state of an external device connected to the electronic device, and obtains the first operation information or the second operation information. The determined priority may be changed based on at least one piece of information, and some contents in the contents list may be acquired as the at least one candidate contents according to the changed priority.

In addition, the processor 1400 determines at least one content reproduction mode required to reproduce the at least one candidate content based on the user information and the second operation information, and provides information on the determined content reproduction mode. It may be output together with the at least one candidate content.

Also, the processor 1400 may obtain a content transmission request from an external device connected to the electronic device, and may output the at least one candidate content in response to the content transmission request from the external device.

The memory 1402 may store a program for processing and control of the processor 1400 , and may also store data input to or output from the electronic device 1000 . In addition, the memory 1402 includes a user authentication model for authenticating the user of the electronic device, a voice recognition activation model for determining whether to activate the voice recognition model, and a voice recognition model for providing a voice recognition service according to the user's voice. Alternatively, it may include a content recommendation model for recommending content based on the recognized user's voice and user information.

In addition, the memory 1402 stores information on weights related to the layers constituting the artificial neural network, nodes included in the layers, and the connection strength of the layers, and, when the weights in the artificial neural network are modified or updated, the modified and updated information It is possible to further store information about the weight. The memory 1402 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , may include at least one type of storage medium among optical disks.

12 is a block diagram of a server connected to an electronic device according to an exemplary embodiment.

According to an embodiment, the server 2000 may include a network interface 2100 , a database 2200 , and a processor 2300 .

The network interface 2100 may correspond to the above-described network interface (not shown) of the electronic device 1000 . For example, the network interface 2100 may obtain information about content from the electronic device 1000 . Also, the network interface 2100 may receive information about content from an external device different from the external device, or transmit information about content currently being played in the external device to another external device.

The database 2200 may correspond to the memory 1402 of the electronic device shown in FIG. 11 . For example, the database 2200 may store a program for processing and controlling the processor 2300 , and may also store data input to or output from the electronic device 1000 .

The processor 2300 typically controls the overall operation of the server 2000 . For example, the processor 2300 may control the DB 2200 and the network interface 2100 in general by executing programs stored in the DB 2200 of the server 2000 . Also, the processor 2300 may perform some of the operations of the electronic device 1000 in FIGS. 1 to 11 by executing programs stored in the DB 2100 .

13 is a block diagram of an external device connected to an electronic device according to an exemplary embodiment.

The external device 4000 is a network interface 4100, a memory 4200, a processor 4300, an output unit 4400, a power supply unit 4500, a housing 4600, a strap fastening unit 4700, a fragrance discharge unit ( 4800 ) and a sensor unit 4900 . However, not all illustrated components are essential components, and the external device 4000 may be implemented by more components than the illustrated components, and the external device 4000 may be implemented with fewer components. may be

According to an embodiment, the network interface 4100 is connected to another external device or electronic device to provide information about content, sensing information currently acquired from a sensor unit of an external device, identification information of content currently being played, or information about content being played. You can share information about your location. The network interface 4100 may correspond to a network interface of the electronic device.

The memory 4200 may store a program for processing and controlling the processor 4300 , and may store data input to or output from the external device 4000 . In addition, the memory 1402 may further store sensing information obtained from the sensor unit of the external device, identification information of the currently playing content, or information on the location of the currently playing content. The memory 4200 of the external device according to the present disclosure may correspond to the memory 1402 of the electronic device.

The processor 4300 executes the programs stored in the memory 4200, so that the network interface 4100, the memory 4200, the output unit 4400, the power supply unit 4500, the housing 4600, and the strap fastening unit within the external device. (4700), it is possible to control the overall operation of the fragrance discharge unit (4800) and the sensor unit (4900).

For example, the processor 4300 may control the network interface to transmit a content request signal, identification information of the currently playing content, and information on the location of the currently playing content to the electronic device 1000 . Also, the processor 4300 may determine a volume level, a reproduction speed, etc. required for the external device to output content, based on the information on the reproduction mode received from the electronic device 1000 .

The output unit 4400 may output an audio signal or a vibration signal, and the output unit (not shown) may include a sound output unit including at least one speaker or a diaphragm. The sound output unit outputs audio data received from the network interface or stored in advance in the memory 1402 .

The power supply unit 4500 is an external device within the network interface 4100, the memory 4200, the processor 4300, the output unit 4400, the fragrance discharge unit 4800 and the sensor unit 4900 to supply power for the operation. can For example, the power supply 4500 may be included in the external device to supply power for the operation of the external device.

Also, according to an embodiment, the power supply 4500 may wirelessly receive power for the operation of an external device from the outside. For example, the power supply unit 4500 may receive power from a wireless charging module external to an external device using either a magnetic induction method or a resonance induction method.

The housing 4600 is located outside the network interface 4100 , the memory 4200 , the processor 4300 , the output unit 4400 , the power supply unit 4500 , and the sensor unit 4900 , and fixes the internal configuration of the external device. can do it According to an embodiment, the housing 4600 of the external device may be divided into a first partial housing, a second partial housing, and a third partial housing, and the second partial housing may be located outside the speaker of the above-described output unit. .

In addition, the second partial housing may include an inner material and a fabric material attached to the outside of the inner material. According to an embodiment, the fabric positioned on the outside of the second partial housing may include a grid including a hypothetical line formed at a preset interval on the outer surface and a meridian disposed at a preset angle with the hypothetical line. In addition, the housing 4600 may be further formed with a discharge hole for discharging the fragrance in at least a portion of the housing.

The strap fastening part 4700 is formed in at least a portion of the first partial housing, and a strap for controlling power of an external device may be inserted. According to an embodiment, the strap fastening part may be located on the inner surface of the strap channel formed in at least a part of the housing to be connected to the housing. According to an embodiment, the external device 4000 may control the power supply to supply power when the strap is inserted through the strap channel in the housing, but power supply when the strap is not inserted through the strap channel in the housing. The power supply from the supply can be cut off.

The fragrance discharge unit 4800 may control the aroma body inside the external device so that the fragrance is discharged through the discharge hole formed in at least a portion of the housing. According to an embodiment, the discharge hole is spaced apart from the strap fastening part at a preset interval, and may discharge the fragrance. For example, the external device may output the content obtained from the electronic device and control the fragrance discharge unit so that the fragrance related to the obtained content is discharged through the discharge hole. Therefore, it is possible to maximize the dementia treatment effect of dementia patients using the external device.

The sensor unit 4900 may detect a state of the electronic device 1000 or a state around the electronic device 1000 , and transmit the sensed information to the processor 1300 . The sensor unit 1400 may be used to measure surrounding environment information of the external device 4000 , state information of the external device, a motion input of a user of the external device, and intensity of a user touch input touching the external device.

The external device sensor unit 4900 includes at least one of an acceleration sensor, a temperature/humidity sensor, an infrared sensor, a gyroscope sensor, a location sensor (eg, GPS), a barometric pressure sensor, a proximity sensor, or a pressure sensor. can, but is not limited thereto. For example, the sensing unit is located in a part of the housing of the external device, measures the intensity of the user's touch touching the external device, and controls the output value of the measured pressure sensor through the control of the processor 4300 to the network interface 4100 can be transmitted as

14 is a diagram for describing a structure of an external device connected to an electronic device according to an exemplary embodiment.

Referring to FIG. 14 , the external device 4000 may include a first partial housing 1422 , a second partial housing 1424 , and a third partial housing 1426 . In addition, the external device 4000 may further include a fragrance discharge hole 1434 spaced apart from the strap fastening part 1428 and the strap fastening part 1428 formed in at least a part of the first partial housing 1422 at a preset interval. can In addition, the strap fastening portion 1428 passes through the first partial housing 1422 , is coupled to the strap channel, and the strap 1432 may be moved through the strap channel. According to an embodiment, the strap 1432 may be formed of a leather ring type.

Referring to 1440 of FIG. 14 , a view from the side of the strap fastening part of the external device 4000 is shown. For example, looking at the external device from the side of the housing of the first part of the external device 4000, the strap 1432 and the fragrance discharge hole 1434 located adjacent to the strap fastening portion 4700 can be observed.

Referring to 1460 of FIG. 14 , an external device 4000 mounted on the wireless charging module 1462 is shown. The external device 4000 may receive power wirelessly from a wireless charging module located outside the external device. Referring to 1480 of FIG. 14 , the shape of the external device separated from the wireless charging module 1462 is shown. The wireless charging module 1462 may acquire energy from an external power source through a separate power channel 1482 separated from the wireless charging module.

15 is a diagram for describing a process in which an electronic device and an external device operate in conjunction with each other according to an exemplary embodiment.

In S1502, the electronic device 1000 may obtain user information. For example, the electronic device 1000 may acquire fingerprint information of a user's finger touching the display panel through a fingerprint sensor formed on the display panel of the electronic device. The electronic device 1000 may authenticate the user of the electronic device using fingerprint information. In S1506, the electronic device 1000 may output content based on the obtained user information. A method for the electronic device 1000 to output content based on user information may correspond to steps S702 to S704 in FIG. 7 , and thus a detailed description thereof will be omitted.

In S1508, the strap in the strap channel of the external device 4000 may be moved based on the user's physical input. For example, the external device transmits information on a content transmission request to the electronic device 1000 when a strip in the strap channel moves out, and when information on content is received from the electronic device 1000 , the external device transmits the content For immediate playback, power can be supplied to each component in an external device.

In S1510 , the external device 4000 may acquire sensing information through a sensor unit in the external device. For example, the external device 4000 may measure a touch intensity as a pressure sensor value when a user of the external device touches the external device using a pressure sensor.

In S1512, the external device 4000 may insert the strap back into the strap channel. For example, when the strap in the strap channel is reinserted, the external device 4000 may transmit information about the content transmission stop request to the electronic device 1000 . In S1514, the external device 4000 may be coupled to an external wireless charging module to receive power from the external wireless charging module.

16 is a diagram for describing a process in which an electronic device, a server, and an external device interwork with each other to provide content, according to an exemplary embodiment.

In S1602, the electronic device 1000 acquires a first audio signal. In S1604 , the electronic device 1000 may transmit the acquired first audio signal to the server 2000 . In S1606 , the server 2000 may extract an audio feature vector from the first audio signal using the voice recognition model stored in the server. In S1608 , the server 2000 may train a speech recognition model based on the audio feature vector.

In S1610, the electronic device 1000 may obtain a second audio signal including a user voice. In S1612 , the electronic device 1000 may transmit the acquired second audio signal to the server 2000 . In S1614 , the server 2000 may extract an audio feature vector from the second audio signal using the speech recognition model. In S1616, the server 2000 may recognize the user's voice by inputting the extracted audio feature vector into a pre-trained voice recognition model.

In S1618 , the server 2000 may transmit information on the voice recognition result obtained by using the output value of the voice recognition model to the electronic device 1000 . In S1620 , the electronic device 1000 may obtain information on the voice recognition result, and transmit a content search request to the server 2000 based on the obtained information on the voice recognition result.

In S1622, the server 2000 acquires content based on the voice recognition result. For example, the server 2000 may access the Internet through a network and obtain candidate content using a character string corresponding to a voice recognition result obtained from the electronic device as a keyword. In S1624, the server 2000 may transmit the searched candidate content to the electronic device. In S1628 , the server 2000 may transmit the searched candidate content to the electronic device while transmitting the searched candidate content to the external device 4000 . In S1630 , the electronic device 1000 may transmit the content received from the server 2000 back to the external device 4000 .

Therefore, the external device 4000 according to the present disclosure may directly receive information on the searched candidate content from the server, but it may also obtain information on the candidate content via the electronic device 1000, so that the content can be stably provided. can be printed

17 is a diagram for describing an operation process of an external device connected to an electronic device according to an exemplary embodiment.

As described above with reference to FIG. 15 , the external device 4000 may detect movement of the strap 1702 within a strap channel formed in at least a portion of the first partial housing. For example, when the strap in the strap channel moves toward the outside of the first partial housing, the external device 4000 transmits a content transmission request to the electronic device 1000 in response to detecting the movement of the strap, and The device 1000 may play the transmitted content in response to the content transmission request.

According to another embodiment, when the strap 1702 in the strap channel moves in the inner direction of the first partial housing, the external device 4000 performs playback of the currently playing content in response to detecting the movement of the strap. can stop According to an embodiment, the external device 4000 may transmit a request to stop transmission of curtains to the electronic device 1000 in response to detecting that the strap in the strap channel moves in the inner direction of the first partial housing.

According to an embodiment, the external device 4000 may further include a driving unit 1706 including at least one driving spring. According to an embodiment, the driving unit 1706 may be located in the first partial housing in the external device. The driving unit 1706 may include a driving mainspring module 1708 including a plurality of gears 1709 having a predetermined gear ratio and a spring connected to the plurality of gears.

For example, the external device 4000 may reproduce content obtained from the electronic device based on the rotation of the driving mainspring module in the driving unit. For example, a strap 1702 that moves within a strap channel of a housing of the external device 4000 may be connected to a strap line 1705 via a strap connection 1704 . The strap line 1705 may be connected to the drive unit 1706 in the first partial housing. The strap line 1705 causes the drive mainspring module 1708 in the drive unit 1706 to rotate in the first direction according to the movement of the strap connection part moving in the outward direction of the first partial housing, so that the drive mainspring module stores the driving force. can do.

According to another embodiment, when the driving force stored by the driving power supply module of the external device 4000 is equal to or greater than a predetermined threshold, content acquired from the electronic device may be reproduced based on the driving force of the driving mainspring module. According to an embodiment, when the driving mainspring module rotates based on the driving force, the movement of the strap connection part may also move in the inner direction of the first partial housing.

That is, when the user of the external device pulls the strap 1702 located outside the external device, the external device 400 connected to the electronic device 1000 according to the present disclosure is received from the electronic device or stored in the external device itself. Content can be played, and the user of the external device inserts the strap 1702 back into the first partial housing, or the strap 1702 is reinserted into the first partial housing by the driving force of the mainspring module in the driving unit of the external device. In this case, playback of the content may be stopped.

The method according to an embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present disclosure, or may be known and available to those skilled in the art of computer software.

In addition, according to the embodiment, a computer program apparatus including a recording medium storing a program for performing another method may be provided. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiment of the present disclosure has been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improved forms of the present disclosure are also provided by those skilled in the art using the basic concept of the present disclosure as defined in the following claims. belong to the scope of the right.

Claims (20)

A method for providing a content service by an electronic device, the method comprising:
obtaining an audio signal including the user's voice input from the user of the electronic device;
extracting an audio feature vector from the obtained audio signal;
authenticating the user based on the extracted audio feature vector;
obtaining user information of the authenticated user when the user is authenticated based on the authentication result of the user;
recognizing the user's voice in the audio signal by inputting the extracted audio feature vector into a voice recognition model activated based on the user's authentication result;
obtaining at least one candidate content based on the recognized user's voice; and
outputting the obtained at least one candidate content; including,
The step of obtaining the at least one candidate content includes:
obtaining a content list from a network connected to the electronic device by using a string corresponding to the recognized user's voice as a search keyword;
determining the priority of contents in the obtained contents list based on the obtained user information;
acquiring first operation information on the operation state of the electronic device or second operation information on the operation state of an external device connected to the electronic device;
changing the determined priority based on at least one of the first operation information and the second operation information;
acquiring some contents in the contents list as the at least one candidate contents according to the changed priority; A method comprising The method of claim 1, wherein the method
Before acquiring the at least one candidate content, executing an application for providing the content service;
obtaining a user touch input regarding the content through a user interface provided by the application; and
when the user touch input is obtained, obtaining the at least one candidate content based on the user touch input instead of the recognized user voice; A method further comprising: The method of claim 1, wherein the method
transmitting, after acquiring the at least one candidate content, information on the acquired at least one candidate content to an external device connected to the electronic device by wire or wirelessly; A method further comprising: delete The method of claim 1, wherein recognizing the user's voice comprises:
determining whether to activate the speech recognition model based on the audio feature vector for each frame of the obtained audio signal; and
recognizing a user's voice in the audio signal by inputting the audio feature vector into the voice recognition model as the voice recognition model is activated; A method comprising delete delete The method of claim 1, wherein outputting the at least one candidate content comprises:
determining at least one content reproduction mode required to reproduce the at least one candidate content based on the user information and the second operation information; and
outputting information on the determined content reproduction mode together with the at least one candidate content; A method comprising The method of claim 3, wherein the outputting of the at least one candidate content comprises:
obtaining a content transmission request from an external device connected to the electronic device; and
outputting the at least one candidate content in response to a content transmission request from the external device; A method comprising The method of claim 9, wherein the outputting of the at least one candidate content comprises:
before the content transmission request is obtained, storing state information about a reproduction state of the content output from the electronic device; and
outputting at least one candidate content to be output from the electronic device based on the status information when the content transmission request is obtained; A method comprising The method of claim 1, wherein the user's information is
The user's biometric information, the user's hobby information, the user's interest information for the content, the user's location information, the user's health information, and the user's medical care that may be obtained from a medical database connected to the electronic device It includes at least one of information about the record or information about a content history requested by a user through the electronic device,
The first operation information includes at least one of information on an operation history of the electronic device and information on a connection state between the electronic device and the external device,
The second operation information comprises at least one of information about a power supply state of the external device, location information of the external device, or an output value of a pressure sensor in the external device. An electronic device for providing a content service, comprising:
a microphone for acquiring ambient sound of the electronic device;
a speaker outputting an audio signal related to the content service;
a memory storing one or more instructions; and
one or more processors that provide the content by executing the one or more instructions; including,
The at least one processor by executing the one or more instructions,
Obtaining an audio signal including the user's voice input from the user of the electronic device,
extracting an audio feature vector from the obtained audio signal,
authenticating the user based on the extracted audio feature vector;
Based on the user authentication result, when the user is authenticated, obtain user information of the authenticated user,
Recognizing the user's voice in the audio signal by inputting the extracted audio feature vector to a voice recognition model activated based on the user's authentication result,
obtaining a content list from a network connected to the electronic device using a string corresponding to the user's voice as a search keyword based on the recognized user's voice;
Determine the priority of the contents in the obtained contents list based on the obtained user information,
Obtaining first operation information on the operation state of the electronic device or second operation information on the operation state of an external device connected to the electronic device,
changing the determined priority based on at least one of the first operation information and the second operation information,
acquiring some contents in the contents list as at least one candidate contents according to the changed priority,
and outputting the obtained at least one candidate content. 13. The method of claim 12, wherein the at least one processor comprises:
An electronic device for transmitting the obtained information on the at least one candidate content to an external device connected to the electronic device by wire or wirelessly. delete 13. The method of claim 12, wherein the at least one processor comprises:
determining whether to activate the speech recognition model based on the audio feature vector for each frame of the obtained audio signal;
When the voice recognition model is activated, the electronic device recognizes the user's voice in the audio signal by inputting the audio feature vector into the voice recognition model. delete delete 13. The method of claim 12, wherein the at least one processor comprises:
determining at least one content reproduction mode necessary to reproduce the at least one candidate content based on the user information and the second operation information;
and outputting information on the determined content reproduction mode together with the at least one candidate content. 14. The method of claim 13, wherein the at least one processor comprises:
obtaining a content transmission request from an external device connected to the electronic device;
and outputting the at least one candidate content in response to a content transmission request from the external device. obtaining an audio signal including the user's voice input from a user of the electronic device;
extracting an audio feature vector from the obtained audio signal;
authenticating the user based on the extracted audio feature vector;
obtaining user information of the authenticated user when the user is authenticated based on the authentication result of the user;
recognizing the user's voice in the audio signal by inputting the extracted audio feature vector to the voice recognition model, which is activated based on the user's authentication result;
obtaining at least one candidate content based on the recognized user's voice; and
outputting the obtained at least one candidate content; including,
The step of obtaining the at least one candidate content includes:
obtaining a content list from a network connected to the electronic device by using a string corresponding to the recognized user's voice as a search keyword;
determining the priority of contents in the obtained contents list based on the obtained user information;
acquiring first operation information on the operation state of the electronic device or second operation information on the operation state of an external device connected to the electronic device;
changing the determined priority based on at least one of the first operation information and the second operation information;
acquiring some contents in the contents list as the at least one candidate contents according to the changed priority; A computer-readable recording medium recording a program for executing the method on a computer, comprising a.

Images