KR20200092763A - Electronic device for processing user speech and controlling method thereof - Google Patents

Abstract

According to various embodiments of the present invention, an electronic device comprises: a microphone; a communication module including a circuit; and at least one processor. The at least one processor controls the microphone to obtain a user voice, controls the communication module to transmit the obtained user voice to a server, and controls the communication module to receive a command for performing at least one operation performed in order from the server. The at least one operation corresponds to at least part of the semantic information of the user voice and at least part of the emphasis information of the user voice, which is confirmed according to the analysis of the user voice. The electronic device is set to perform the at least one operation based on the received command. Other various embodiments of the present invention are possible.

Description

ELECTRONIC DEVICE FOR PROCESSING USER SPEECH AND CONTROLLING METHOD THEREOF}

This document relates to an electronic device that processes a user's voice and a control method thereof.

For many people living in modern times, portable digital communication devices have become an essential element. Consumers want to be provided with various high-quality services they want anytime, anywhere using portable digital communication devices.

The voice recognition service is a service that provides various content services to consumers in response to the received user voice based on a voice recognition interface implemented in portable digital communication devices. In order to provide voice recognition services, portable digital aisles include technologies that recognize and analyze human language (eg, automatic speech recognition, natural language understanding, natural language generation, machine translation, conversation system, question and answer, speech recognition/synthesis, Etc) is implemented.

In order to provide a high-quality voice recognition service to consumers, it is necessary to implement a technology for accurately identifying a user's intention from a user's voice and a technology for providing an appropriate content service corresponding to the identified user's intention.

A voice recognition service that provides various content services in response to a user voice is provided. However, since the emphasis information of the user's voice included in the user's voice (for example, at least a part of the user's voice intentionally emphasized by the user) is not identified, an appropriate content service corresponding to the user's intention in response to the emphasis information of the user's voice Could not be provided. Accordingly, in the conventional electronic devices, more tasks are performed according to a user's control to receive an appropriate content service that meets a user's intention, thereby increasing the operational burden of the electronic device.

The electronic device and the control method according to various embodiments may perform an operation of checking emphasis information of the user voice included in the user voice, and provide various content services corresponding to the highlighted emphasis information of the user voice. Accordingly, an electronic device and a method of controlling the operation burden are reduced by performing less tasks in order to provide an appropriate content service that meets a user's intention.

According to various embodiments, in an electronic device, a microphone, a communication module including a circuit, and at least one processor, wherein the at least one processor controls the microphone to acquire a user voice, and the obtained Control the communication module to transmit a user voice to a server, and control the communication module to receive a command for performing at least one operation sequentially performed from the server, wherein the at least one operation is the Corresponds to at least some of the semantic information of the user's voice and at least some of the emphasis information of the user's voice identified according to analysis of the user's voice, and performs the at least one or more operations based on the received command An electronic device configured to be provided may be provided.

According to various embodiments, a microphone, a communication module including a circuit, and at least one processor, wherein the at least one processor controls the microphone to acquire a user voice, and the acquired user voice to a server Control the communication module to deliver, and control the communication module to receive information on an answer including at least one text from the server, and information on an answer including the at least one text to the user voice Corresponding to at least a part of the meaning information of the user voice and at least a part of the emphasis information of the user voice confirmed according to the analysis of the user, and at least one or more texts in a designated manner based on the information on the received answer An electronic device, which is configured to output, may be provided.

According to various embodiments, a memory for storing instructions, and at least one processor, wherein the instructions, when executed, the at least one processor acquires a user voice, and the user voice corresponding to the user voice At least one or more operations sequentially performed by checking the emphasis information, checking the semantic information of the user voice corresponding to the user voice, and based on the emphasizing information of the identified user voice and the semantic information of the identified user voice An electronic device configured to confirm a command for performing a command and to transmit a command for performing the at least one identified operation may be provided.

According to various embodiments, a microphone, a communication module including a circuit, and at least one processor, wherein the at least one processor controls the microphone to acquire a user voice, and the acquired user voice to a server Control the communication module to deliver, control the communication module to receive a first command for performing at least one first operation performed sequentially from the server, the at least one first operation , When the emphasis information is not confirmed from the user voice, corresponds to the semantic information of the user voice identified according to the analysis of the user voice, and for performing at least one second operation sequentially performed by the server It is set to control the communication module to receive a second command (command), the at least one second operation, when the emphasis information is confirmed from the user voice, the user confirmed according to the analysis of the user voice An electronic device may be provided corresponding to the meaning information of the voice and the emphasis information of the user voice.

According to various embodiments, a solution means for solving a problem is not limited to the above-described solution means, and solution means not mentioned are provided to those skilled in the art from the present specification and the accompanying drawings. It will be clearly understood.

According to various embodiments, there is provided an electronic device and a control method thereof, which perform an operation of confirming highlighting information of a user's voice included in the user's voice, and provide various content services corresponding to the identified highlighting information of the user's voice Can. According to various embodiments of the present disclosure, an electronic device and a method of controlling the same can be provided in which less task is performed and an operation burden is reduced to provide an appropriate content service that meets a user's intention.

1 is a block diagram of an electronic device in a network environment 100 according to various embodiments.
2 is a diagram illustrating an electronic device and a server according to various embodiments of the present disclosure.
3 is a flowchart illustrating an example of an operation of an electronic device based on emphasis information of a user input (eg, user voice) according to various embodiments.
4 is a diagram illustrating an example of an operation of an electronic device based on emphasis information of a user input (eg, user voice) according to various embodiments.
5 is a flowchart illustrating another example of an operation of an electronic device according to various embodiments.
6 is a view for explaining another example of the operation of the electronic device according to various embodiments.
7 is a flowchart illustrating an example of an operation of an electronic device and a server according to various embodiments of the present disclosure.
8 is a view for explaining another example of the operation of the electronic device according to various embodiments.
9 is a flowchart illustrating an example of an operation of an electronic device and a server according to various embodiments.
10 is a view for explaining an example of the operation of the electronic device and the server according to various embodiments.
11 is a view for explaining an example of the operation of the server, according to various embodiments.
12 is a diagram illustrating an example of an operation of checking emphasis information of a user voice of a server according to various embodiments.
13 is a flowchart illustrating an example of an operation of controlling a certain alarm application of an electronic device according to various embodiments.
14 is a diagram illustrating an example of an operation of controlling a certain alarm application of an electronic device according to various embodiments.
15 is a flowchart illustrating an example of an operation of setting a state value of an electronic device according to various embodiments.
16A is a diagram illustrating an example of an operation of setting a state value of an electronic device according to various embodiments.
16B is a diagram illustrating an example of an operation of setting a state value of an electronic device according to various embodiments.
17 is a diagram illustrating another example of an operation of setting a state value of an electronic device according to various embodiments of the present disclosure.
18 is a flowchart illustrating an example of an operation of controlling a plurality of applications of an electronic device according to various embodiments.
19 is a diagram illustrating an example of an operation of controlling a plurality of applications of an electronic device according to various embodiments.
20 is a diagram illustrating an example of an operation of an electronic device, a server, and at least one external electronic device according to various embodiments of the present disclosure.
21 is a diagram illustrating an example of an electronic device, a server, and at least one external electronic device according to various embodiments of the present disclosure.
22 is a flowchart illustrating an operation of outputting information stored in an electronic device of an electronic device according to various embodiments.
23 is a diagram illustrating an operation of outputting information stored in an electronic device of an electronic device according to various embodiments.
24 is a flowchart illustrating another example of an operation of an electronic device according to various embodiments of the present disclosure.
25 is a flowchart illustrating another example of an operation of an electronic device and a server according to various embodiments.
26 is a view for explaining another example of the operation of the electronic device and the server according to various embodiments.
27 is a view for explaining an example of the operation of the server, according to various embodiments.
28 is a flowchart illustrating an operation of outputting text of an electronic device in a specified order according to various embodiments.
29 is a diagram for describing an operation of outputting text of an electronic device in a specified order according to various embodiments of the present disclosure.
30 is a flowchart illustrating an operation of performing at least one operation of an electronic device and outputting text according to various embodiments.
31 is a diagram illustrating an operation of performing at least one operation of an electronic device and outputting text according to various embodiments.
32 is a diagram illustrating an integrated intelligent system according to various embodiments of the present disclosure.
33 is a block diagram illustrating a user terminal of an integrated intelligent system according to various embodiments.
34 is a block diagram illustrating an intelligent server of an integrated intelligent system according to various embodiments.
35 is a diagram illustrating a method of generating a path rule of a path planner module according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It should be understood that the examples and terms used therein are not intended to limit the technology described in this document to specific embodiments, but include various modifications, equivalents, and/or substitutes of the examples. In connection with the description of the drawings, similar reference numerals may be used for similar elements. Singular expressions may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as “A or B” or “at least one of A and/or B” may include all possible combinations of items listed together. Expressions such as "first," "second," "first," or "second," can modify the components, regardless of order or importance, to distinguish one component from another component It is used but does not limit the components. When it is stated that one (eg, first) component is “connected (functionally or communicatively)” to another (eg, second) component or is “connected,” the component is the other It may be directly connected to the component, or may be connected through another component (eg, the third component).

In this document, "configured to (or configured)" is changed to "a suitable for", "a suitable for," or "a", depending on the situation, for example, in hardware or software. It can be used interchangeably with "made to do," "can do," or "designed to do." In some situations, the expression "a device configured to" may mean that the device "can" with other devices or parts. For example, the phrase “processors configured (or set) to perform A, B, and C” means by executing a dedicated processor (eg, an embedded processor) to perform the operation, or one or more software programs stored in the memory device. , It may mean a general-purpose processor (eg, CPU or application processor) capable of performing the corresponding operations.

Electronic devices or external electronic devices according to various embodiments of the present disclosure include, for example, a smart phone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, and a server. , PDA, portable multimedia player (PMP), MP3 player, medical device, camera, or wearable device. Wearable devices are accessories (e.g. watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-devices (HMD)), fabrics or clothing (e.g. electronic clothing), It may include at least one of a body-attached type (eg, a skin pad) or a bio-implantable circuit In some embodiments, the electronic device or electronic device is, for example, a television, a digital video disk (DVD) player. , Audio, refrigerator, air conditioner, cleaner, oven, microwave, washing machine, air purifier, set-top box, home automation control panel, security control panel, media box, game console, electronic dictionary, electronic key, camcorder, or electronic picture frame It may include at least one of.

In another embodiment, the electronic device or external electronic device includes various medical devices (eg, various portable medical measurement devices (such as a blood glucose meter, heart rate monitor, blood pressure meter, or body temperature meter), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI) resonance imaging (CT), computed tomography (CT), imaging, or ultrasound), navigation devices, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), automotive infotainment devices , Marine electronic equipment (e.g. marine navigation devices, gyro compasses, etc.), avionics, security devices, head units for vehicles, industrial or household robots, drones, financial institution ATMs, shops Point of sales (POS), or Internet of Things devices (eg, light bulbs, various sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.) Can. According to some embodiments, the electronic device or external electronic device is a furniture, building/structure, or part of a vehicle, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, Water, electricity, gas, or radio wave measurement devices, etc.). In various embodiments, the electronic device or at least one external electronic device may be flexible or a combination of two or more of the various devices described above. The electronic device or at least one external electronic device according to the embodiment of the present document is not limited to the above-described devices. In this document, the term user may refer to a person or electronic device using an electronic device or at least one external electronic device, or a device using at least one external electronic device (eg, artificial intelligence electronic device).

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1, in the network environment 100, the electronic device 101 communicates with the electronic device 102 through the first network 198 (eg, a short-range wireless communication network), or the second network 199. It may communicate with the electronic device 104 or the server 108 through (for example, a remote wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108. According to an embodiment, the electronic device 101 includes a processor 120, a memory 130, an input device 150, an audio output device 155, a display device 160, an audio module 170, a sensor module ( 176), interface 177, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196, or antenna module 197 ). In some embodiments, at least one of the components (for example, the display device 160 or the camera module 180) may be omitted or one or more other components may be added to the electronic device 101. In some embodiments, some of these components may be implemented in one integrated circuit. For example, the sensor module 176 (eg, a fingerprint sensor, an iris sensor, or an illuminance sensor) may be implemented while embedded in the display device 160 (eg, a display).

The processor 120, for example, executes software (eg, the program 140) to execute at least one other component (eg, hardware or software component) of the electronic device 101 connected to the processor 120. It can be controlled and can perform various data processing or operations. According to one embodiment, as at least part of data processing or computation, the processor 120 may receive instructions or data received from other components (eg, the sensor module 176 or the communication module 190) in the volatile memory 132. Loaded into, process instructions or data stored in volatile memory 132, and store result data in non-volatile memory 134. According to one embodiment, the processor 120 includes a main processor 121 (eg, a central processing unit or an application processor), and an auxiliary processor 123 (eg, a graphics processing unit, an image signal processor) that can be operated independently or together. , Sensor hub processor, or communication processor). Additionally or alternatively, the coprocessor 123 may be set to use lower power than the main processor 121, or to be specialized for a specified function. The coprocessor 123 may be implemented separately from the main processor 121 or as a part thereof.

The coprocessor 123 may replace, for example, the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 may be active (eg, execute an application) ) While in the state, with the main processor 121, at least one component of the components of the electronic device 101 (eg, the display device 160, the sensor module 176, or the communication module 190) It can control at least some of the functions or states associated with. According to one embodiment, the coprocessor 123 (eg, image signal processor or communication processor) may be implemented as part of other functionally relevant components (eg, camera module 180 or communication module 190). have.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176). The data may include, for example, software (eg, the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

The program 140 may be stored as software in the memory 130, and may include, for example, an operating system 142, middleware 144, or an application 146.

The input device 150 may receive commands or data to be used for components (eg, the processor 120) of the electronic device 101 from outside (eg, a user) of the electronic device 101. The input device 150 may include, for example, a microphone, mouse, or keyboard.

The audio output device 155 may output an audio signal to the outside of the electronic device 101. The audio output device 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback, and the receiver can be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from, or as part of, the speaker.

The display device 160 may visually provide information to the outside of the electronic device 101 (eg, a user). The display device 160 may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to an embodiment, the display device 160 may include a touch circuitry configured to sense a touch, or a sensor circuit (eg, a pressure sensor) configured to measure the strength of the force generated by the touch. have.

The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 acquires sound through the input device 150 or directly or wirelessly connects to the sound output device 155 or the electronic device 101 (for example, an external electronic device) Sound may be output through the electronic device 102) (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the detected state can do. According to one embodiment, the sensor module 176 includes, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biological sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used for the electronic device 101 to directly or wirelessly connect to an external electronic device (eg, the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert electrical signals into mechanical stimuli (eg, vibration or movement) or electrical stimuli that the user can perceive through tactile or motor sensations. According to one embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and videos. According to one embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101. According to one embodiment, the power management module 388 may be implemented, for example, as at least part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101. According to one embodiment, the battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishing and performing communication through the established communication channel. The communication module 190 operates independently of the processor 120 (eg, an application processor) and may include one or more communication processors supporting direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg : Local area network (LAN) communication module, or power line communication module. The corresponding communication module among these communication modules includes a first network 198 (for example, a short-range communication network such as Bluetooth, WiFi direct, or infrared data association (IrDA)) or a second network 199 (for example, a cellular network, the Internet, or It may communicate with external electronic devices through a computer network (eg, a telecommunication network such as a LAN or WAN). These various types of communication modules may be integrated into a single component (eg, a single chip), or may be implemented as a plurality of separate components (eg, multiple chips). The wireless communication module 192 uses a subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199. The electronic device 101 can be identified and authenticated.

The antenna module 197 may transmit a signal or power to the outside (eg, an external electronic device) or receive it from the outside. According to an embodiment, the antenna module 197 may include one or more antennas, from which at least one antenna suitable for a communication scheme used in a communication network, such as a first network 198 or a second network 199, For example, it may be selected by the communication module 190. The signal or power may be transmitted or received between the communication module 190 and an external electronic device through the at least one selected antenna.

At least some of the components are connected to each other via a communication method between peripheral devices (for example, a bus, a general purpose input and output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and a signal ( Ex: command or data) can be exchanged with each other.

According to one embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199. Each of the electronic devices 102 and 104 may be the same or a different type of device from the electronic device 101. According to an embodiment, all or some of the operations executed in the electronic device 101 may be performed in one or more external devices of the external electronic devices 102, 104, or 108. For example, when the electronic device 101 needs to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device 101 instead executes the function or service itself. In addition or in addition, one or more external electronic devices may be requested to perform at least a portion of the function or the service. The one or more external electronic devices receiving the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and deliver the result of the execution to the electronic device 101. The electronic device 101 may process the result, as it is or additionally, and provide it as at least part of a response to the request. To this end, cloud computing, distributed computing, or client-server computing technology can be used, for example.

Hereinafter, an example of the electronic device 101 and a control method thereof according to various embodiments will be described.

The operation of the electronic device 101 described below is described as being controlled such that the processor 120 performs the corresponding operation using at least one configuration of the electronic device 101, but may not be limited thereto. For example, instructions or computer codes that cause the operation of the electronic device 101 described below may be stored in the memory 130. The stored instructions or computer code may cause the processor 120 to process the corresponding operation using at least one component included in the electronic device 101.

2 is a diagram illustrating an electronic device 101 and a server 201 according to various embodiments.

According to various embodiments, the electronic device 101 is connected to the server 201 through a communication network, and transmits a communication module 190 to transmit user input (eg, user voice) received through the communication network to the server 201. It is possible to control and control the communication module 190 to receive the analysis result information of the user's voice from the server 201, and provide a voice recognition service based on the analysis result information of the received user's voice.

According to various embodiments, the server 201 may perform an operation associated with a user input received from the electronic device 101. For example, the server 201 may analyze the user input received from the electronic device 101 based on at least one analysis method. For example, the server 201 may check a command for performing at least one operation sequentially performed in the electronic device 101 based on the analysis of user input based on at least one analysis method. For example, the server 201 may control the communication module 190 to transmit a command for performing at least one operation performed sequentially to the electronic device 101. For example, the server 201 may check answer information on at least one text based on the analysis of user input based on at least one analysis method. For example, the server 201 may control the communication module 190 to transmit response information regarding at least one text. The operation of the server 201 will be described later in detail.

At least one operation performed in the server 201 according to various embodiments may be performed in the electronic device 101. In other words, the electronic device 101 may be implemented as a stand-alone type electronic device 101 in which the operation of the server 201 according to various embodiments is performed in the electronic device 101. .

That is, at least one operation performed in the server 201 described below may be performed in the electronic device 101.

3 is a flowchart 300 illustrating an example of an operation of an electronic device based on emphasis information of a user input (eg, user voice) according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 3, but may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 3 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 3. The operation may be performed. Hereinafter, FIG. 3 will be described with reference to FIG. 4.

4 is a diagram illustrating an example of an operation of an electronic device based on emphasis information of a user input (eg, user voice) according to various embodiments.

According to various embodiments, the electronic device may perform at least one operation according to whether emphasis information of a user input (eg, user voice) exists.

According to various embodiments, the electronic device may receive a user input in operation 301. For example, as illustrated in FIG. 4, the electronic device activates a microphone provided in the electronic device (eg, a microphone on), and inputs a voice from a user using the activated microphone (eg, “alarm at 7:00 in the morning”). Settings” user's voice). In addition, the present invention is not limited to the above description, and the electronic device may receive text input, touch input, and the like from the user.

According to various embodiments, the electronic device performs at least one first operation based on the semantic information of the user input in operation 303 or the user in operation 304 according to whether or not emphasis information of the user input exists in operation 302. At least one second operation may be performed based on the semantic information of the input and the emphasis information of the user input. For example, referring to FIG. 4, the electronic device acquires a first user voice 401 called “alarm set at 7 am” in operation 302, and semantic information of the first user voice 401 in operation 303 (eg: In response to "2018/xx/xx 7 AM alarm setting"), the operation related to the alarm function installed in the electronic device is executed, and the alarm is output as the first number at 7 AM using the executed application. You can perform the operation. Further, for example, referring to FIG. 4, the electronic device may display a second user voice 402 called “alarm setting at 7:00 in the morning!” including a user voice “alarm setting” highlighted in the user voice according to the user's intention in operation 302. ). For example, "!" may mean that the voice is loud. Without being limited to this, "!" may mean that the corresponding voice is emphasized in various ways (eg, phonetic emphasis, linguistic emphasis). The electronic device 101 may confirm whether or not emphasis is made by confirming the volume of the voice, which will be described later in more detail. In response to the semantic information of the second user voice 402 in operation 303 (for example, "2018/xx/xx 7 am alarm setting") and the emphasis information of the second user voice 402 (eg, alarm setting!) An operation related to an alarm function installed in the electronic device may be performed, and an operation of setting an alarm to be outputted at a second number of times greater than the first number of times using the executed application may be performed. In other words, even if the texts of the two user voices are the same, the electronic device may perform at least one or more operations different from the case where the emphasis information is included in the user input or the emphasis information is not included in the user input. For example, as described above, when the electronic device receives a user input (eg, the first user voice 401), the electronic device receives a user input (eg, the second user voice 402) that does not include emphasis information. In some cases, the number of alarms may be set to be greater than the number of alarms set, and some operations of at least one operation of the electronic device performed in response to a user input may be enhanced and provided.

According to various embodiments, the semantic information of the user input may be information indicating a user's intention confirmed from the user input. For example, if the user voice is "alarm setting at 7 am", the user's intention is "2018/xx/xx 7 am alarm setting", and the semantic information of the user voice may be information indicating the user intention. have. According to various embodiments, the semantic information of the user input may be identified according to a text data corresponding to the user input and a matching rule stored in the natural language recognition database according to the natural language understanding method.

According to various embodiments, the emphasis information of the user input is at least a part of the user input (eg, the second user voice 402) that is emphasized compared to another part (eg, 7:00 in the morning) according to the user's intention (eg, at 7 am). Alarm setting!). For example, when the user input is a voice input, the emphasis information of the user input is phonetic (eg, size, pitch, duration, etc.) or linguistic (eg, necessarily, fast, etc.) among the user voices input according to the user's intention. It can mean a part emphasized with the same modifier). For example, if the user input is text input, the highlighting information of the user input may be linguistically emphasized (for example, a modifier such as must, fast, etc.) among texts input according to the user's intention, or a text format (eg highlight, underline) , Bold, etc.). According to various embodiments, emphasis information of the user input may be identified based on at least one analysis method of the user input. This will be described later in detail.

Hereinafter, for convenience of description, the user input to the electronic device is described as a voice input, but descriptions of various embodiments are also applied to other types of user inputs (eg, text input, touch input, etc.). Can be.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described.

According to various embodiments, the electronic device 101 may perform at least one operation corresponding to emphasis information of the user's voice. Hereinafter, the performance of at least one operation corresponding to the emphasis information of the user's voice of the electronic device 101 will be described with reference to FIGS. 5 and 6.

5 is a flowchart 500 for explaining another example of the operation of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 5, but may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 5 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 5. The operation may be performed. Hereinafter, FIG. 5 will be described with reference to FIG. 6.

6 is a view for explaining another example of the operation of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may control the microphone to acquire a user voice in operation 501. For example, as illustrated in FIG. 5, the electronic device 101 receives a user voice (eg, “alarm setting at 7 am!”) in which at least some voice (eg, “alarm setting!”) is emphasized by the user. So you can control the microphone.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may control the communication module 190 to transmit the user voice obtained in operation 502 to the server 201.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) receives a command for performing at least one or more operations 602 sequentially performed from the server 201 in operation 503. The communication module 190 can be controlled to do so.

According to various embodiments, at least one or more operations 602 sequentially performed as illustrated in FIG. 6 (eg, a first operation 611 (eg, executing an alarm application)) and a second operation 612 (eg, : Command for performing the alarm setting at 7 o'clock in the morning multiple times) is the meaning information of the user's voice (eg, "2018/XX/XX", "7 am", "alarm setting") and the highlight information of the user's voice ( Example: "Alarm setting!") can be confirmed by the server 201. According to various embodiments, parameters corresponding to at least a portion of at least one or more operations 602 that are sequentially performed as illustrated in FIG. 6 (for example, a second operation (for example, setting an alarm multiple times at 7 am)) (E.g., number of alarm settings) may be confirmed based on emphasis information of the user's voice (e.g., "alarm setting!"). Specifically, as illustrated in FIG. 6, at least some (eg, “alarm setting”) of the meaning information of the user voice is checked based on the emphasis information of the user voice (eg, “alarm setting!”), and the meaning of the user voice The parameters included in at least some of the information (for example, the number of alarm settings) may be checked based on the highlight information of the user voice (for example, “alarm setting!”) (for example, the number of alarm setting times is set multiple times). Accordingly, at least a part of at least one or more operations 602 (eg, a second operation (eg, a plurality of seven o'clock in the morning)) based on at least some of the meaning information of the user's voice that includes a parameter set based on emphasis information of the user's voice Times alarm setting)) can be confirmed.

According to various embodiments, the semantic information of the user's voice and the emphasis information of the user's voice may be confirmed by separate analysis methods or may be confirmed together, which will be described later in detail.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may perform at least one operation 602 based on the command received in operation 504. For example, the electronic device 101 executes an alarm application based on the received command, as shown in FIG. 6, and the executed alarm application provides alarms multiple times around 7 am 2018/XX/XX The setting operation can be performed.

Hereinafter, an example of operations of the electronic device 101 related to operations 501 and 502 according to various embodiments will be described with reference to FIGS. 7 and 8. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

7 is a flowchart 700 illustrating an example of operations of the electronic device 101 and the server 201 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 7, and may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 7 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 7. The operation may be performed. Hereinafter, FIG. 7 will be described with reference to FIG. 8.

8 is a view for explaining another example of the operation of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 controls the communication module 190 to deliver the user's voice to the server 201 in real time based on the programs executed as at least part of the above-described operations 501 and 502, The display may be controlled to display information corresponding to the user's voice received from the server 201 after operations 501 and 502.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may execute a program in operation 701. According to various embodiments, the program may be an application associated with an intelligent agent (eg, a voice recognition application). The electronic device 101 executes an application associated with an intelligent agent in response to receiving a user input (eg, input through a hardware key provided in the electronic device 101, input through a touch screen, designated voice input). Can.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may control the microphone to acquire a user voice in operation 702. For example, the electronic device 101 acquires a user voice based on an application associated with the executed intelligent agent, and preprocesses the received user voice (eg, adaptive echo canceller AEC, noise suppression ( noise suppression NS), end-point detection EPD, or automatic gain control AGC. 8, the electronic device 101 may sequentially receive user voices 801, 802, and 803 (eg, "morning", "seven hours", and "alarm setting"). The sequentially received user voice illustrated in FIG. 8 is an example for convenience of description, and the electronic device 101 is a unit smaller than the unit of the received user voice illustrated in FIG. 7 (eg, “ㅇ”, “ ㅏ", "ㅊ", "ㅣ", "ㅁ", etc.) to receive user voices sequentially.

According to various embodiments, as shown in FIG. 8, the electronic device 101 (eg, at least one processor 120) sequentially corresponds to user voices 801, 802, and 803 received sequentially You can control the display to display text. At this time, a portion of the text displayed sequentially corresponding to the highlighting information of the user's voice may be highlighted and displayed on the display of the electronic device 101. Hereinafter, an example of the operation of controlling the display to sequentially display the text of the electronic device 101 will be described with reference to operations 703 to 708.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) communicates with the server 201 to transmit user voices 811, 812, and 813 sequentially received in operation 703 to the server 201. 190 can be controlled.

According to various embodiments, the server 201 may check the text corresponding to the user voice by analyzing the user voice sequentially received in operation 704. For example, the server 201 analyzes sequentially received user voices 801, 802, and 803 based on automatic speech recognition ASR technology, and sequentially receives user voices 801, 802 , 803).

According to various embodiments, the server 201 may check emphasis information of the user's voice in operation 705. For example, the server 201 may confirm “alarm setting!” as emphasis information of the user voice from the user voices 801, 802, and 803 sequentially received as illustrated in FIG. 8.

According to various embodiments, the server 201 is configured to emphasis information (eg, “alarm setting!”) of the user voice among texts identified in response to the user voices 801, 802, and 803 sequentially received in operation 706. You can check the corresponding part (eg alarm setting).

According to various embodiments, the server 201 may highlight a part of the text identified in operation 707 corresponding to the emphasis information of the user's voice. For example, when a part of the identified text corresponding to the highlight information of the user's voice is displayed on the electronic device 101, the server 201 is highlighted (eg, bold, underlined, large size compared to other text, etc.) You can highlight some of the text to be displayed.

According to various embodiments, the server 201 may include information regarding the sequentially confirmed text (eg, “morning”, “seven hours”) in operation 708 and the first confirmed text (eg, “alarm setting!”). The communication module may be controlled to transmit information on the electronic device 101.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may control the display to display text information sequentially received in operation 709 (eg, 811, 812, 813). have. The electronic device 101 may display 813 such that some of the text is emphasized.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may end the user voice acquisition in operation 710. For example, if the user voice is not received through the microphone for a designated time, the electronic device 101 may end the user voice acquisition.

As described above, according to various embodiments, since the operation performed by the server 201 may be performed by the electronic device 101, the operations performed by the server 201 described above are implemented to be performed by the electronic device 101 It may be. For example, the electronic device 101 controls the communication module 190 to receive text data from the server 201, checks the user's emphasis information in the electronic device 101, and corresponds to the checked emphasis information. The display may be controlled to check a portion of the text data and to highlight and display a portion of the confirmed text data.

Hereinafter, operation 403 according to various embodiments will be described with reference to FIGS. 9 to 12. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

9 is a flowchart 900 for describing an example of operations of the electronic device 101 and the server 201 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 9, and may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 9 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 9. The operation may be performed. Hereinafter, FIG. 9 will be described with reference to FIGS. 10 to 12.

10 is a view for explaining an example of the operation of the electronic device 101 and the server 201 according to various embodiments.

11 is a view for explaining an example of the operation of the server 201 according to various embodiments.

12 is a flowchart 1200 illustrating an example of an operation of outputting an answer of the electronic device 101 according to various embodiments.

According to various embodiments, the server 201 may analyze the user's voice received from the electronic device 101 based on at least one analysis method, and the meaning information 1104 of the user's voice and the emphasis information of the user's voice (1103) (e.g., "alarm setting!"), and sequentially performed based on the identified meaning information 1104 of the user voice and the highlighting information 1103 of the user voice (e.g., "alarm setting!") It is possible to check the command 1106 for performing at least one or more operations, and to control the communication module to deliver the confirmed command 1106 to the electronic device 101.

According to various embodiments, the server 201 may check the highlighting information 1103 of the user's voice (eg, “alarm setting!”) in operation 901. Hereinafter, operation 901 will be described with reference to FIG. 10.

According to various embodiments, the server 201 uses the linguistic analysis method or the phonetic analysis method as shown in FIG. 11 in operation 1001 to highlight information 1103 of the user's voice (eg, “alarm setting!”). ).

According to various embodiments, the emphasis information may include at least one of a user's voice that satisfies a specified phonetic condition, or at least a portion that satisfies a specified linguistic condition. The specified phonetic condition may include at least one of a specified volume size, a specified pitch height, or a specified carcinogenic length. The specified linguistic conditions may include at least one of being identified by a specified word or being repeated as a word repeated within the user's voice.

For example, the server 201 is a phonetic analysis method, and checks at least one of the volume of the user's voice, the pitch of the user's voice, or the duration for each section of the user's voice, thereby highlighting the user's voice 1103 (eg : "Alarm setting!"). For example, the server 201 may analyze the user's voice for each time period and check the highlight information 1103 (eg, “alarm setting!”). The server 201 checks the size of the volume for each time interval of the identified user voice, checks a time interval having a size equal to or larger than a specified size among the sizes of the volume for each checked time interval, and information associated with the confirmed time interval (for example: At least a part of the user's voice in the corresponding time period can be confirmed as the highlight information 1103 (eg, “alarm setting!”). In addition, the server 201 checks the height of the pitch for each time interval of the identified user voice, and checks the time interval having a pitch height equal to or higher than the designated pitch among the checked time interval pitches. The information associated with the time period may be checked as emphasis information 1103 (eg, at least a part of a user's voice in the time period). In addition, the server 201 checks the strength of each user's voice for each time period, and the intensity of the FO frequency is high among the confirmed strength for each time period or the average pitch duration (or the length of pronunciation). This long time period can be checked, and information associated with the checked time period can be checked as the highlight information 1103 (eg, “alarm setting!”). Here, the FO frequency means a fundamental frequency that is phonetically analyzed in the user's voice, and the corresponding F0 frequency can be identified based on data related to the user.

In addition, for example, the server 201, as a linguistic analysis method, may check the emphasis word included in the user's voice, and confirm the emphasis information 1103 of the user's voice (eg, “alarm setting!”). For example, the server 201 identifies words that are used repeatedly in the user's voice, or words that are used in a user's voice, or words that are emphasized linguistically, such as, for example, modifiers such as must, fast, somehow, etc. Can. In addition, the server 201 may check a word in the user's voice that is identified as a high-frequency word based on data about the user (eg, data about words frequently used by the user).

The analysis method for checking the highlight information 1103 of the user voice of the server 201 (for example, “alarm setting!”) may be stored in a database provided in the server 201.

According to various embodiments, the server 201 may check a value associated with emphasis information 1103 (eg, “alarm setting!”) of the user voice identified in operation 1002. For example, the server 201 may check the weight value assigned to the checked highlight information 1103 (eg, “alarm setting!”), and check whether the checked weight value is greater than or equal to a specified value. At this time, the server 201 may further consider user data. For example, the server 201 refers to the user data, and the voice emphasis habit of the user of the electronic device 101 (eg, the size of the volume of the voice, the height of the pitch of the voice, the intensity of each frequency of the voice, or frequently The user data related to the word to be used may be checked, and a weight value may be assigned to the highlighted information 1103 (eg, “alarm setting!”) identified based on the verified user information. For example, the server 201 confirms that the degree of emphasis is relatively high based on the verified user information (eg, the frequency of the emphasis word identified in the user's voice is determined according to the user data. A relatively high weight value can be assigned to the highlight information 1103 (eg, “alarm setting!”) that is relatively high compared to the frequency.

According to various embodiments, the server 201 determines whether a value (eg, a weight value) associated with the highlighting information 1103 (eg, “alarm setting!”) of the user voice identified in operation 1003 is greater than or equal to a specified value. Can be confirmed. If it is determined that the value associated with the emphasis information 1103 of the user's voice is greater than or equal to the specified value, the server 201 may continue to perform operation 902. Accordingly, unnecessary highlighting information 1103 (eg, “alarm setting!”) may be filtered to check the command 1106 for performing at least one operation sequentially performed based on a specified value.

According to various embodiments, the server 201 checks the semantic information 1104 of the user's voice identified in operation 1004 and at least sequentially performed based on only the semantic information 1104 of the user's voice identified in operation 1005 Commands 1106 for performing one or more operations may be identified.

According to various embodiments, the server 201 may check the semantic information 1104 of the user voice 1101 in operation 902. As shown in FIG. 11, the server 201 checks the semantic information 1104 of the user's voice by using a natural language understanding method in response to the received user's voice 1102 (for example, an alarm setting at 7:00 in the morning) Can. Specifically, the server 201 may check the semantic information 1104 of the user's voice corresponding to the user's voice 1102 based on syntactic analysis or semantic analysis. For example, the server 201 divides text corresponding to the identified user's voice based on automatic speech recognition into text in a designated unit (eg, word, phrase, morpheme, etc.), and grammatically divides the text divided into the specified unit. Can be analyzed. In addition, for example, the server 201 may semantically analyze the semantic information 1104 of the user's voice corresponding to the user's voice using semantic matching, rule matching, or formula matching. Accordingly, the server 201 includes at least one or more data 1111, 1112, and 1113 (eg, "2018/xx/xx", "7 AM", "Alarm Setting") as shown in FIG. The meaning information 1104 of the user's voice can be checked. Each of the at least one or more data 1111, 1112, 1113 may include parameters 1121, 1122, 1123, 1124. For example, as shown in FIG. 11, some data included in the user voice of “alarm setting” include “bell type”, “alarm setting number”, and “alarm duration”, “associated with “alarm setting”, “ Alarm title", and the like, at least one or more parameters 1121, 1122, 1123, 1124.

According to various embodiments, the server 201 checks at least a part of the semantic information 1104 of the user voice corresponding to the emphasis information 1103 of the user voice identified in operation 903 (eg, “alarm setting!”). Can be. For example, as illustrated in FIG. 11, the server 201 highlights the user's voice 1103 among at least one or more data 1111, 1112, and 1113 included in the meaning information 1104 of the user's voice (eg : It is possible to check at least one data (eg, “alarm setting”) corresponding to “alarm setting!”) (eg, alarm setting!). The server 201 checks the viewpoint information of the highlighted user's voice emphasis information 1103 (for example, “alarm setting!”), and highlights information 1103 (eg, “alarm setting” based on the confirmed viewpoint information). !"), at least a part of the meaning information 1104 of the user's voice. The viewpoint information may be information indicating a viewpoint during a user's speech corresponding to the emphasis information 1103 of the user's voice. For example, the server 201 at least part of the user voice corresponding to the highlight information 1103 during the user's speech based on the viewpoint information corresponding to the identified highlight information 1103 (eg, “alarm setting!”). (Eg, “alarm setting!”) may be checked, and at least a part (eg, “alarm setting”) of semantic information 1104 of the user voice corresponding to the checked time may be checked. In addition, the server 201 checks the type information of the confirmed user voice, and the semantic information 1104 of the user voice corresponding to the highlight information 1103 (eg, “alarm setting!”) based on the confirmed type information. At least some can be identified. The type information may include first emphasis information (or first emphasis information identified based on a phonetic analysis method) that satisfies the specified phonetic condition or second emphasis information (or linguistic analysis) that satisfies the specified linguistic condition. The second emphasis information identified based on the method may be included. For example, the server 201 is a part of the user's voice that is modified by the emphasis information (eg, emphasis word (eg, modifier)) according to the type of the highlighted emphasis information 1103 (eg, “alarm setting!”). And confirm at least some of the meaning information of the user's voice corresponding to the portion of the user's voice.

According to various embodiments, the server 201 may check whether the user voice emphasis information 1103 (eg, “alarm setting!”) is available in operation 904. In addition, the server 201 includes the parameters 1121 included in at least one data included in the semantic information 1104 of the user voice corresponding to the confirmed emphasis information 1103 of the user voice (eg, “alarm setting!”). , 1122, 1123, 1124 may be set based on highlighting information 1103 of the user's voice (eg, “alarm setting!”).

According to various embodiments of the present disclosure, when it is determined that the highlighting information 1103 of the user's voice (eg, “alarm setting!”) is available, the highlighting information 1103 of the user's voice confirmed in operation 905. Based on (eg, “alarm setting!”), parameters 1121, 1122, 1123, and 1124 included in at least a portion of the meaning information 1104 of the user voice may be checked. For example, the server 201 among the at least one or more parameters 1121, 1122, 1123, and 1124 included in at least a part of the semantic information 1104 of the user's voice, the highlighting information 1103 of the user's voice (for example, "alarm" Setting!"), at least one or more first parameters 1122 and 1123 to be set. For example, the server 201 provides the user data with an appropriate function enhancement method related to the semantic information 1104 of the user's voice, which is confirmed to correspond to the emphasis information 1103 of the user's voice (eg, “alarm setting!”). At least one of the at least one parameter (1121, 1122, 1123, 1124) included in the highlighting information 1103 (eg, “alarm setting!”) of the user voice confirmed based on the confirmed function enhancement method. One or more first parameters 1122 and 1123 may be identified. The server 201 may set the values of the identified at least one first parameter 1122 or 1123 to a specified value. The specified value may be a predetermined value that the values of the parameters 1122 and 1123 are set based on the emphasis information 1103 (eg, “alarm setting!”), and the emphasis information 1103 is not limited thereto (eg : “Alarm setting!”) may be a value determined based on a value (eg, a weight value).

According to various embodiments, the server 201 sequentially performs at least a portion 1113 including parameters 1122 and 1123 set in operation 906 and a semantic information 1104 of a user voice including another portion. The command 1106 for performing at least one or more operations. For example, as shown in FIG. 11, the server 201 executes an alarm application in response to semantic information 1104 of a user's voice (first operation) and a plurality of executed alarm applications before or after 7 am A command 1106 for performing an operation (second operation) of setting a conference alarm to be provided in a specified alarm size may be confirmed. At this time, the parameters 1122 and 1123 corresponding to at least a part of the command 1106 (eg, multiple alarms, a specified alarm size), as described above, highlighting information 1103 of the user's voice (eg, “alarm setting!”) It was confirmed based on.

According to various embodiments, the command 1106 may be a pass rule set. The pass rule set may include information about an app to be executed, an action to be executed in the app, and parameters required to execute the action. The server 201 uses the information of the electronic device 101 to arrange an app to be executed in the electronic device 101 and an operation to be executed in the app in an ontology or graph model form, and at least one pass. You can check the rules. The server 201 may select an optimal pass rule set from the generated at least one pass rule.

According to various embodiments, when the server 201 determines that the user's voice emphasis information 1103 (eg, “alarm setting!”) is not usable, the command 20 based on the user voice information identified in operation 907 ( 1106). In other words, the server 201 may check the command 1106 for performing a general task in which the highlight information 1103 (eg, “alarm setting!”) is not reflected.

According to various embodiments, the server 201 may control the communication module to transmit the command 1106 identified in operation 908 to the electronic device 101.

According to various embodiments, the electronic device 101 may perform at least one operation based on the command 1106 received in operation 909. For example, the electronic device 101 executes an application (eg, executes an alarm application) based on the command 1106 as illustrated in FIG. 10, and controls the executed application to perform a specified operation (eg : You can set multiple alarms to be provided at a specified alarm size before or after 7 am).

According to various embodiments, the electronic device 101 (eg, at least one processor 120) outputs information related to at least one operation performed in operation 910 (eg, controls a display or displays audio to display text) You can control the audio output device to output. Hereinafter, operation 910 of the electronic device 101 will be described with reference to FIG. 12.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the communication module to transmit result information to the server 201 after performing at least one or more operations in operation 1201 can do.

According to various embodiments, the server 201 may check text information related to at least one operation based on result information received in operation 1202.

According to various embodiments, the server 201 checks at least some of the confirmed text information corresponding to the highlighting information 1103 (eg, “alarm setting!”) of the user's voice identified in operation 1203, and confirms You can highlight the formatting of at least some of the text information (eg highlight, underline, bold, etc.).

According to various embodiments, the server 201 may control the communication module to transmit text information including at least a portion of which a form is emphasized in operation 1204 to the electronic device 101.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) is based on text information received in operation 1205, and text corresponding to the received text information as shown in FIG. 11. You can control the display to display (1107) (eg "Twice around 7 o'clock in the morning!! I have set an alarm"). At least a portion 1108 of the displayed text (eg, "twice around seven hours!!") may be highlighted.

In the above, the operation of displaying text can be applied to the audio output operation. In the above-described operation, audio information is checked instead of text information, and the electronic device 101 can control the audio output device to output the checked audio information.

Meanwhile, according to various embodiments of the present disclosure, the server 201 emphasis information 1103 of the user's voice as at least part of an operation of checking the meaning information 1104 of the user's voice without performing a separate linguistic analysis method or a phonetic analysis method ) (Eg "Alarm Settings!"). For example, when the user's voice contains the emphasis word, the server 201 emphasis information 1103 of the user's voice as at least part of the operation of checking the semantic information 1104 of the user's voice without performing a separate analysis method (Eg "Alarm settings!"). Specifically, the server 201 corresponds to the user's voice (eg, “must set alarm at 7 am”), meaning information 1104 of the user's voice (eg, “2018/xx/xx”, “7 am”), At least one piece of data corresponding to the highlighted word (eg, “must”) among the semantic information 1104 of the user's voice that is checked based on the information about the highlighted word stored in advance by checking “alarm setting” and “must”. You can check The server 201 checks at least a part (eg, “alarm setting”) of the semantic information 1104 of the user voice corresponding to the checked at least one data (eg, “must”), and as described above, at least Commands 1106 for performing one or more operations may be identified. Other redundant descriptions will be omitted.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may control a schedule alarm application so that a schedule alarm is provided a specified number of times based on emphasis information of a user's voice. Hereinafter, the operation of the electronic device 101 will be described with reference to FIGS. 13 and 14.

13 is a flowchart 1300 illustrating an example of an operation of controlling a certain alarm application of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 13, and may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 13 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 13. The operation may be performed. Hereinafter, FIG. 13 will be described with reference to FIG. 14.

14 is a diagram illustrating an example of an operation of controlling a certain alarm application of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire a user voice in operation 1301, and transfers the first user voice to the server 201 in operation 1302. In response to the transmission, the server 201 may receive a command for performing at least one operation. For example, as shown in FIG. 14(b), the electronic device 101 may obtain a user voice 1402 that says, “Please remind the post office to go find the parcel by 5 pm.” The electronic device 101 transmits the user voice 1402 to the server 201, and the first operation corresponding to the emphasis information (for example, necessarily!) included in the user voice 1402 from the server 201 (eg : Set an alarm multiple times).

According to various embodiments, the server 201 identifies a highlighted word (eg, necessarily!) as emphasis information of the user's voice based on the user's voice, and semantic information of the user's voice (eg, "2018/XX/XX") , "5 PM", "Schedule Alarm Settings"), and at least some of the meaning information of the user voice corresponding to the emphasis information of the user voice (eg, "Schedule Alarm Settings"). The server 201 is a command for performing at least one operation including a parameter related to a predetermined alarm period and a predetermined number of alarms included in at least some of the semantic information of the identified user voice, and a first operation corresponding to the parameter. By doing so, the electronic device 101 can be set to perform a predetermined alarm for a specified number of times during a specified period. Accordingly, the server 201, based on the semantic information of the user voice including at least a portion of which the parameter is set, at least one or more operations (for example, "execute a certain alarm application" "scheduled alarm at a specified number of times during a specified time period" It is possible to check the command for performing "), and to control the communication module to deliver the confirmed command to the electronic device 101.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) executes a schedule alarm application based on a command received in operation 1303, and during a time period designated by the schedule alarm application The control may be performed such that a predetermined alarm is provided at a specified number of times corresponding to the first information included in the user voice. The command is a schedule alarm application so that more schedule alarms are provided for a specified period, compared to a set number of schedule alarms set when the user voice 1401 does not include emphasis information as shown in FIG. 14(a). It is possible to set a certain number of alarms.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may display the result of the operation as text 1403 as illustrated in FIG. 14B.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) sets an operation state value (eg, audio volume) of the electronic device 101 based on emphasis information of the user's voice It is possible to adjust the set status value (for example, audio volume) based on other user voices received for a specified period.

According to various embodiments, the state value of the electronic device includes an audio volume value of the electronic device, a brightness value of the display of the electronic device, a vibration setting value of the electronic device, and the like, and is not limited to the above description. It can include all of the various possible settings.

Hereinafter, the operation of the electronic device 101 will be described with reference to FIGS. 15 and 16. Meanwhile, overlapping descriptions related to the operation of the electronic device 101 are omitted.

15 is a flowchart 1500 for describing an example of an operation of setting a state value of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 15, and may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 illustrated in FIG. 15 may be performed, or at least one or more operations may be performed. Hereinafter, FIG. 15 will be described with reference to FIG. 16.

16A is a diagram illustrating an example of an operation of setting a state value of the electronic device 101 according to various embodiments. 16B is a diagram illustrating an example of an operation of setting a state value of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire the first user voice 1601 in operation 1501, and the first user voice in operation 1502 ( 1601), the first command for performing at least one operation may be received from the server 201. For example, as illustrated in FIG. 16B, the electronic device 101 may acquire the first user voice 1601 saying “Volume louder!”. The electronic device 101 transmits the first user voice 1601 to the server 201, and from the server 201, the electronic device 101 corresponds to emphasis information (for example, loud!) included in the first user voice 1601. A command for performing at least one operation including one operation (eg, “increase the audio output value of the electronic device 101 by five steps”) may be received.

According to various embodiments, the server 201 checks emphasis information (eg, loud!) of the user's voice based on the user's voice, checks semantic information of the user's voice (for example, “set the audio volume to a large volume”), , At least some of the meaning information of the user voice corresponding to the emphasis information of the user voice (eg, “set the audio output value large”) may be checked. The server 201 electronically transmits a parameter related to the size of the state value included in at least a part of the semantic information of the identified user voice by a command for performing at least one or more operations including a first operation corresponding to the parameter. The device 101 may be set to a specified value so that the state value (eg, audio output value) is set to the first state value (eg, Volume 5 up). Accordingly, the server 201 performs at least one or more operations (for example, "increase the audio output value of the electronic device 101 by 5 steps") based on the semantic information of the user voice including at least a part of which parameters are set. The communication module may be controlled to confirm a command to be performed and to transmit the confirmed command to the electronic device 101.

According to various embodiments of the present disclosure, the electronic device 101 (eg, at least one processor 120) is based on the first command received as shown in FIG. 16B in operation 1503, and the electronic device 101 The status value (eg, an audio output value) may be set to a designated first value corresponding to emphasis information of the user's voice. As illustrated in FIG. 16A, the command compares the state value set when the user voice 1602 does not include emphasis information, and the state value of the electronic device 101 is the first value that is a higher state value. Can be set.

In addition, according to various embodiments, the electronic device 101 (eg, at least one processor 120) corresponds to a state value of the electronic device 101 corresponding to emphasis information of a user's voice, as shown in FIG. 16B. After performing at least one action to set the specified first value, text related to at least one action (eg, volume 5 up) may be displayed. Redundant descriptions related to the operation of displaying the text of the electronic device 101 are omitted.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) checks whether the second user voice 1603 is received during the designated period in operation 1504, and the second user during the designated period. In the case of acquiring the voice 1603, a second command may be received from the server 201 in response to transmitting the second user voice 1603 obtained in operation 1505 to the server 201. For example, the electronic device 101 may activate the microphone for a specified period after performing at least one operation based on the first command corresponding to the first user voice 1601. When the second user voice 1603 is received through the microphone activated for a designated period of time, the electronic device 101 may transmit the second user voice 1603 to the server 201. The server 201 performs a first operation (for example, an audio output value of the electronic device 101 is increased by 5 steps) corresponding to the emphasis information of the user voice among the at least one of the received second user voices 1603. In response to confirming the correspondence, it is possible to check at least one second operation different from the first operation based on the second user voice 1603 (eg, increasing the audio output value of the electronic device 101 by 2 steps). have. The server 201 may control the communication module to transmit the second command including the identified at least one second operation to the electronic device 101.

According to various embodiments of the present disclosure, the electronic device 101 (eg, at least one processor 120) is based on the second command received in operation 1506, and the status value of the electronic device 101 (eg, an audio output value) ) May be set to a designated second value corresponding to emphasis information of the user's voice.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may adjust a set state value (eg, audio volume) based on a user's touch input. Hereinafter, an operation of adjusting a set state value (eg, audio volume) based on a user's touch input of the electronic device 101 will be described with reference to FIG. 17.

17 is a diagram illustrating another example of an operation of setting a state value of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) displays text related to at least one operation performed by the electronic device 101 as illustrated in FIG. 17. The touch screen display may be controlled to control a touch screen display and to receive a user input for selecting a portion 1702 of the displayed text. The electronic device 101 may display information related to a plurality of status values 1703 (eg, Volume 1 up, Volume 2 up, Volume 3 up, Volume 4 up, Volume 5 up) in response to the received user input. You can control the touch screen display. The electronic device 101 receives a user input for selecting at least one of the displayed plurality of status values 1703, and based on at least one of the selected plurality of status values, the electronic device 101 (eg, an audio volume). Can be reset.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) executes a plurality of first applications corresponding to emphasis information of a user's voice among a plurality of applications installed in the electronic device 101. And, it is possible to control the executed first plurality of applications to perform at least one operation. Hereinafter, the operation of the electronic device 101 will be described with reference to FIGS. 18 and 19. Meanwhile, overlapping descriptions related to the operation of the electronic device 101 are omitted.

18 is a flowchart 1800 illustrating an example of an operation of controlling a plurality of applications of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 18, but may be performed in a different order from the order shown. Further, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 18 may be performed, or at least one or more operations may be performed. Hereinafter, FIG. 18 will be described with reference to FIG. 19.

19 is a diagram illustrating an example of an operation of controlling a plurality of applications of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire a user voice in operation 1801, and transmits the user voice to the server 201 in operation 1802. In response, the server 201 may receive a command for performing at least one operation. For example, as shown in FIG. 19(b), the electronic device 101 acquires a user voice 1902 saying “Bixby! Hurry up, somehow catch a taxi.” and receives the user voice from the server. To perform at least one operation including a first operation corresponding to the emphasis information (eg, “emergency!”, “somehow”) included in the user voice from the server 201 and transmitted to the server 201. Commands can be received.

According to various embodiments, the server 201 checks emphasis information (eg, “emergency”, “somehow”) of the user's voice based on the user's voice, and semantic information of the user's voice (eg, “current place xx Dong-xx-gil", "Taxi Call"), and at least a part (eg, "Taxi Call") of the semantic information of the user voice corresponding to the emphasis information of the user voice. The server 201 may include at least one or more operations including a first operation to execute a plurality of first applications among a plurality of applications installed on the electronic device, based on at least some of the meaning information of the user voice and the emphasis information of the user voice. The command for performing the operation can be confirmed. The server 201 uses the electronic device by a command for performing at least one or more operations including a first operation corresponding to a parameter related to execution of an application included in at least some of the semantic information of the identified user voice ( In step 101), a plurality of first applications (for example, a plurality of traffic calling applications) among a plurality of applications installed in the electronic device 101 may be executed, and set to a specified value. Accordingly, the server 201 may cause the user voice including at least a part of which parameters are set to perform at least one or more operations based on the semantic information (eg, "run all traffic call applications installed in the electronic device", "transport means") It is possible to control a communication module to confirm a command for performing the traffic control call application control) and to transmit the confirmed command to the electronic device 101.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) executes a plurality of applications installed in the electronic device based on the command received in operation 1803, and is executed in operation 1804. Each of the plurality of applications may be controlled to perform an operation to perform an operation based on the received command (for example, a taxi call to xx-dong xx-gil). The command is compared to the number of applications that are executed when emphasis information is not included in the user's voice 1901 as shown in FIG. 19(a), and a plurality of applications are executed by the electronic device 101. It can be done.

In addition, according to various embodiments, the electronic device 101 (eg, at least one processor 120) performs an operation of controlling a plurality of applications by executing a plurality of applications, as illustrated in FIG. 19B. It can display text 1902 related to the action (eg, "All connected services are connected, yes, I called a taxi"). Redundant descriptions related to the operation of displaying the text of the electronic device 101 are omitted.

According to various embodiments of the present disclosure, the electronic device 101 (eg, at least one processor 120) is not limited to at least one operation according to the above description, and other at least one operation (eg, multiple location search applications) You can run it to search for a place). Redundant explanation is omitted.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, a command for performing at least one operation based on emphasis information of a user voice is transmitted to at least one external electronic device associated with the electronic device 101, and at least one operation is performed by the electronic device ( 101) may be performed in at least one external electronic device. Hereinafter, the operation will be described with reference to FIGS. 20 and 21.

20 is a diagram illustrating an example of operations of the electronic device 101, the server 201, and at least one external electronic device 2010 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 20, and may be performed in a different order from the order shown. Further, according to various embodiments, more operations may be performed than the operations of the electronic device 101 illustrated in FIG. 20, or at least one or more operations may be performed. Hereinafter, FIG. 20 will be described with reference to FIG. 21.

21 is a diagram illustrating an example of an electronic device 101, a server 201, and at least one external electronic device 2010 according to various embodiments.

According to various embodiments, the electronic device 101 controls the microphone to acquire a user voice in operation 2001, and in response to transmitting the user voice to the server 201 in operation 2002, at least one or more from the server 201 A command for performing an operation may be received. For example, as illustrated in FIG. 18, the electronic device 101 acquires a user voice 2101 saying “Bixby, let me know the concentration of fine dust when the weather is bad in the future” and receives the received user voice from the server ( 201), and receive a command for performing at least one operation corresponding to the user voice from the server 201.

According to various embodiments, the server 201 checks the emphasis information of the user's voice and semantic information of the user's voice in operation 2003, and executes a command for performing at least one operation based on the semantic information of the user's voice in operation 2004. Can be confirmed. For example, the server 201 checks the emphasis word (eg, necessarily) as the emphasis information of the user's voice based on the user's voice 2101, and semantic information of the user's voice (eg, "deterioration of bad weather", "fine" Dust concentration check and notification", and at least one action based on the semantic information of the user's voice (for example, "Check weather information", "Check bad weather condition from confirmed weather information", "Weather bad condition check" In the case, it is possible to confirm the command including the fine dust concentration confirmation and notification").

According to various embodiments, the server 201 may control the communication module to transmit a command to the electronic device 101 in operation 2005. In response to this, the electronic device may perform at least one operation based on a command in operation 2006.

According to various embodiments, the server 201 may check at least one external electronic device 2010 associated with the electronic device 101 in response to checking the highlight information of the user voice in operation 2007. In other words, an operation of transmitting a command to the external electronic device of the server 201 may be triggered by the emphasis information of the user voice. For example, the server 201 may check at least one external electronic device 2010 associated with the electronic device 101 in response to being confirmed to include the designated emphasis information (eg, necessarily!). The specified emphasis information is to cause confirmation of at least one external electronic device 2010 associated with the electronic device 101 among emphasis information previously stored in the database (eg, linguistic/speech emphasis information). It may be 101, or emphasis information designated by the server 201.

According to various embodiments, the server 201 may check at least one external electronic device 2010 using the same communication network as the electronic device 101. The communication network may include a wireless communication network such as Wifi using an AP (acceess point), a D2D communication network such as Bluetooth, Wifi-direct, etc., and is not limited to the above description and can be used by the electronic device 101. Any communication network method can be included. For example, the server 201 may check at least one external electronic device 2010 using the same Wifi communication network as the electronic device 101.

According to various embodiments, the server 201 checks the registration information of the electronic device 101 when the electronic device 101 is the electronic device 101 registered in the server 201, and is associated with the confirmed registration information. At least one external electronic device 2010 may be identified. For example, when a user of the electronic device 101 subscribes to the server 201, the user can register at least one external electronic device 2010 associated with the electronic device 101 together with the electronic device 101. have. Accordingly, the registration information of the electronic device 101 may include information associated with at least one external electronic device 2010 associated with the electronic device 101.

According to various embodiments, the server 201 may control the communication module to transmit a command to at least one external electronic device 2010 (eg, the first external electronic device 2010) identified in operation 2008. In response to this, at least one external electronic device 2010 (eg, the first external electronic device 2010) may perform at least one or more operations based on a command in operation 2009.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) sequentially outputs information stored in the electronic device 101 corresponding to emphasis information of the user's voice, and controls the user's voice According to this, an operation of stopping outputting of stored information may be performed. Hereinafter, the operation will be described with reference to FIGS. 22 and 23.

22 is a flowchart 2200 illustrating an operation of outputting information stored in the electronic device 101 of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 22, and may be performed in a different order from the order shown. Further, according to various embodiments, more operations may be performed than the operations of the electronic device 101 illustrated in FIG. 22, or at least one or more operations may be performed. Hereinafter, FIG. 22 will be described with reference to FIG. 23.

23 is a view for explaining an operation of outputting information stored in the electronic device 101 of the electronic device 101 according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire a user voice in operation 2201, and transmits the user voice to the server 201 in operation 2202. In response, the server 201 may receive a command for performing at least one operation. For example, as shown in FIG. 23(b), the electronic device 101 acquires a user voice 2302 saying “Please tell me about Bixby's schedule today~~” and the received user voice 2302 server At least one that includes a first operation (eg, sequential output of stored information) corresponding to the emphasized word (eg, multi-~~) included in the user voice 2302 from the server 201 and transmitted to the 201. A command for performing the above operation may be received.

According to various embodiments, the server 201 checks emphasis information (eg, da~~~~) of the user's voice based on the user's voice, and semantic information of the user's voice (for example, "2018/xx/xx" , "Schedule information output"), and at least some (eg, schedule information output) of the meaning information of the user voice corresponding to the emphasis information of the user voice. The server 201 is a schedule information stored by a command for performing at least one operation including a first operation corresponding to a parameter related to application execution included in at least some of the semantic information of the identified user voice. To be output sequentially, it can be set to the specified value. Accordingly, the server 201 has at least one or more actions based on the semantic information of the user's voice including at least a portion of which parameters are set (eg, "confirming schedule information associated with 2018/xx/xx", "confirmed schedule information" It is possible to control the communication module to confirm the command for performing "sequential output") and to transmit the confirmed command to the electronic device 101.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) is based on the command received in operation 2203, and based on the received command, corresponds to the user voice stored in the electronic device. It may be controlled to perform an operation of outputting information (for example, sequentially outputting 2018/xx/xx schedule information). The command outputs more information sequentially from the electronic device 101 as compared to the number of information output when the user's voice 2301 does not include emphasis information as shown in FIG. 23(a) ( Example: You can output the information in text and the information in audio).

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may determine whether a user input is acquired during output of information corresponding to the user voice 2302 in operation 2204. For example, the electronic device 101 may activate the microphone during output of information corresponding to the user's voice. The electronic device 101 may acquire a user voice for stopping the corresponding audio output through the activated microphone. Alternatively, the electronic device may control the touch screen to receive a user touch input for stopping audio output. The electronic device 101 may stop outputting information corresponding to the user voice 2302 in response to confirming the acquisition of the user input. Alternatively, the electronic device 101 may continue to output information corresponding to the user's voice in response to the acquisition of the user input being not confirmed.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may output an answer corresponding to emphasis information of the user's voice. Hereinafter, a response output operation corresponding to the emphasis information of the user's voice of the electronic device 101 will be described with reference to FIG. 24.

24 is a flowchart 2400 illustrating another example of the operation of the electronic device 101 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 24, but may be performed in a different order from the order shown. Further, according to various embodiments, more operations than the operation of the electronic device 101 shown in FIG. 24 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 24. This can also be done.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire a user voice in operation 2401, and sends the user voice obtained in operation 2402 to the server 201. The communication module can be controlled to deliver. Operations 2401 and 2402 may be performed in the same manner as operations 501 and 502 described above, and thus duplicate descriptions are omitted.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) receives a communication module to receive answer information regarding at least one text sequentially performed from the server 201 in operation 2403. Can be controlled. According to various embodiments, the answer information including at least one text may be checked by the server 201 based on the meaning information of the user voice and emphasis information of the user voice. According to various embodiments, parameters corresponding to at least a part of the at least one text may be identified based on emphasis information of the user's voice. Accordingly, at least a part of the at least one text may be identified based on at least some of the semantic information of the user voice identified based on the emphasis information of the user voice. At least a portion of the identified at least one text may be set to be emphasized compared to other portions when output from the electronic device 101. The operation 2403 may be performed as the operation 503 described above, and thus redundant description will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may perform at least one or more operations based on the command received in operation 2404. For example, the electronic device 101 may output (eg, output through a display or output via audio) by emphasizing at least a part of at least one text based on the received answer information. According to various embodiments, emphasis of at least a portion of the text may be performed in a visual way. For example, at least a portion of the text may be shaded, displayed at a larger size compared to other portions, or emphasized by formatting such as bold/underlined. Alternatively, when text is output as audio through an audio device, it may be emphasized in an audible manner. For example, if at least a portion of the text is output as audio, it is output at a larger volume compared to the other portion, output as a different kind of audio (e.g., a different voice), or a specified audio (e.g., back ground music BGM). It can be emphasized in a manner that is output together.

In addition, in the response output operation of the electronic device 101, text may be output in response to the user's voice sequentially received, which may be performed as described above with reference to FIG. 6, and thus duplicate description will be omitted.

Hereinafter, operation 2403 according to various embodiments will be described with reference to FIGS. 25 to 27. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

25 is a flowchart 2500 illustrating another example of operations of the electronic device 101 and the server 201 according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 25, but may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 25 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 25. The operation may be performed.

Hereinafter, FIG. 25 will be described with reference to FIGS. 26 and 27.

26 is a diagram illustrating another example of operations of the electronic device 101 and the server 201 according to various embodiments.

27 is a view for explaining an example of the operation of the server 201 according to various embodiments.

According to various embodiments, the server 201 checks the emphasis information of the user's voice in operation 2501, confirms the semantic information of the user's voice in operation 2502, and the user's voice corresponding to the emphasis information of the user's voice identified in operation 2503 It is possible to check at least a part of the semantic information of and to check whether the highlight information is available in operation 2504. For example, as illustrated in FIG. 26, the server 201 corresponds to a user voice, such as “change meeting from one hour to two o'clock” transmitted from the electronic device 101, such as highlighting information of the user voice (eg, You can check the semantic information of the user's voice (for example, "Time Meeting", "Time Meeting", "Change Meeting Schedule"). In addition, operations 2501 to 2504 may be performed in the same manner as operations 901 to 904 described above, and thus duplicate descriptions are omitted.

According to various embodiments, when it is determined that the user's voice emphasis information is available, the server 201 is based on the emphasis information (eg, two o'clock) of the user's voice identified in operation 2505 as shown in FIG. 26. As a result, it is possible to check at least a part (eg, a two o'clock meeting) of the meaning information of the user's voice.

According to various embodiments, the server 201 may check answer information on at least one text based on semantic information of a user voice including at least a part different from a part identified in operation 2506. At this time, as shown in FIG. 26, the server 201 may check the answer information in comparison to text corresponding to at least a part of the semantic information of the identified user voice among at least one text compared to text corresponding to another part.

According to various embodiments of the present disclosure, when it is determined that the user's voice emphasis information is unavailable, the server 201 may check answer information related to at least one text based on the user voice information identified in operation 2507. As illustrated in FIG. 27, the server 201 may check at least a part of at least one text based on emphasis information previously stored as at least a part of operation 2507. For example, if the server 201 checks at least one text based on the user's voice (eg Bixby is currently absent. Please leave a message after the “beep-” sounds). The information (eg, “beep-”) may be checked, and at least a part (eg, “beep-”) of at least one text corresponding to the checked highlight information may be checked. The server 201 may highlight at least a part of the identified at least one text. However, the present invention is not limited to the above description, and the server 201 may check answer information related to at least one text that does not include the emphasized text corresponding to the emphasis information of the user voice based on the user voice information.

According to various embodiments, the server 201 may control the communication module to transmit answer information to the electronic device 101 in operation 2508. In operation 2509, the electronic device 101 may output at least one text (eg, output through a display or output through audio) based on the response information. For example, as illustrated in FIG. 26, the electronic device 101 highlights at least a part (eg, two o'clock) of at least one text (eg, emphasis by visual means or emphasis by audible method). Can print For example, as illustrated in FIG. 27, the electronic device 101 emphasizes at least a part (eg, “beep”) of at least one text (eg, a visual method or an aural method). Emphasis).

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may output text corresponding to emphasis information of the user's voice in a designated order. Hereinafter, the operation of outputting the text corresponding to the emphasis information of the user's voice of the electronic device 101 in a specified order will be described with reference to FIGS. 28 and 29.

28 is a flowchart 2800 illustrating an operation of outputting text of the electronic device 101 in a specified order according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 28, but may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 28 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 28. The operation may be performed. Hereinafter, FIG. 28 will be described with reference to FIG. 29.

29 is a diagram for describing an operation of outputting text of the electronic device 101 in a specified order according to various embodiments.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) controls the microphone to acquire a user voice in operation 2801, and transmits to the at least one text from the server 201 in operation 2802. You can receive information about the answers. For example, as shown in FIG. 29(b), the electronic device 101 may obtain a user voice 2901 saying "Please save a schedule. At 9:00 tomorrow, a business meeting!" The electronic device 101 delivers the user voice 2901 to the server 201, and the text highlighted from the server 201 to correspond to the emphasis information (eg, "business meeting!") included in the user voice 2901 You can receive answer information about.

According to various embodiments, the server 201 checks emphasis information (eg, a business meeting!) of the user voice based on the user voice, and semantic information of the user voice (eg, “2018/xx/xx”,” 9:00 am", "schedule storage", "business meeting"), and at least some of the meaning information of the user voice corresponding to the highlight information of the user voice (eg, "business meeting"). The server 201 may check at least one text corresponding to the meaning information of the user's voice, and highlight at least a portion of the at least one text corresponding to at least some of the identified meaning information. For example, the server 201 may check the answer information so that at least a part of the highlighted at least one text is output in a designated order. For example, when at least a portion of the highlighted at least one text is displayed through the display, the server 201 is displayed in a designated order within a sentence (eg, the earliest order excluding the word "yes," among texts) It can be done. For example, when at least a portion of the highlighted at least one text is output through the audio device, the server 201 searches the audio devices in a designated order (for example, the earliest order excluding the word "yes," among text). Output.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) is highlighted in the text output based on the received answer information as shown in FIG. 29(b) in operation 2803. At least a portion corresponding to the information may be output in a specified order (eg, displayed in a sentence in a specified order or output in audio in a specified order). Accordingly, as shown in FIG. 29(a), when the user's voice 2901 does not include emphasis information, it is displayed in a sentence in an earlier order or in a faster order compared to an order in which at least a part of the text is displayed. It can be output by voice.

Hereinafter, another example of the electronic device 101 and its control method according to various embodiments will be described. Meanwhile, hereinafter, overlapping descriptions related to the operation of the electronic device 101 will be omitted.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) includes commands for performing at least one operation corresponding to emphasis information of a user's voice, and response information regarding at least one text To receive, perform at least one or more operations based on the received command, and output text based on the received answer information. Hereinafter, an operation of performing at least one operation based on the received command of the electronic device 101 and outputting text based on the received answer information will be described with reference to FIGS. 30 and 31.

30 is a flowchart 3000 illustrating an operation of performing at least one operation of the electronic device 101 and outputting text, according to various embodiments. According to various embodiments, the operation of the electronic device 101 is not limited to the order of the operation of the electronic device 101 shown in FIG. 30, and may be performed in a different order from the order shown. In addition, according to various embodiments, more operations than the operations of the electronic device 101 shown in FIG. 30 are performed, or at least one or more operations less than those of the electronic device 101 shown in FIG. 30. The operation may be performed. Hereinafter, FIG. 30 will be described with reference to FIG. 31.

31 is a view for explaining an operation of performing at least one operation of the electronic device 101 and outputting text, according to various embodiments.

According to various embodiments, the electronic device 101 (eg, the at least one processor 120) controls the microphone to acquire a user voice in operation 3001, and transmits to the at least one text from the server 201 in operation 3002. You can receive information about answers. For example, as shown in FIG. 31(b), the electronic device 101 may obtain a user voice 3101, “Where am I here? The electronic device 101 transmits the user's voice 3201 to the server 201, and includes a first operation corresponding to the emphasis information (for example, even if it is true) of the user's voice 3201 from the server 201 Response information regarding at least one or more texts may be received, including commands for performing at least one or more operations and texts highlighted in response to the emphasis information of the user voice.

According to various embodiments, the server 201 checks emphasis information of the user's voice based on the user's voice (eg, even if it is real) and semantic information of the user's voice (eg, “current place”, “output”) , And at least a part (eg, “current place”) of semantic information of the user voice corresponding to the emphasis information of the user voice. The server 201 checks at least one text corresponding to the meaning information of the user's voice (for example, "Yes, it is the Seoul R&D Center of Samsung Electronics 33, Seongchon-gil, Seocho-gu, Seoul") It is possible to emphasize at least part of one or more texts (eg, “Seoul R&D Center, Samsung Electronics, 33 Seongchon-gil, Seocho-gu, Seoul”). In addition, the server 201 may include at least one or more operations (eg, “execute the map application”) including a first operation corresponding to at least some of the recognized semantic information (eg, “highlighting the current location on the map application”). , You can check the command for performing "Highlight the current location on the map application").

According to various embodiments, the electronic device 101 (eg, at least one processor 120) may output at least one or more output based on the received answer information as shown in FIG. 31(b) in operation 3003. At least a portion of the text 3103 corresponding to the emphasis information of the user's voice may be displayed to be emphasized.

According to various embodiments, the electronic device 101 (eg, at least one processor 120) corresponds to emphasis information of the user's voice based on the received command as shown in FIG. 31(b) in operation 3004. The display can be controlled to display the information 3104 on the screen of the executed application. For example, the electronic device 101 executes a map application based on the received command, and highlights information 3104 (for example, current location information) corresponding to the user's voice emphasis information. The display can be controlled to display on the screen. Accordingly, as illustrated in FIG. 31(a), the content more visually emphasized may be provided as compared to the case where the emphasis information is not included in the user voice 3102.

32 to 34, an integrated intelligent system to which an electronic device (for example, the electronic device 101) or an external server (for example, the external server 600) according to various embodiments of the present disclosure can be applied will be described.

32 is a diagram illustrating an integrated intelligent system according to various embodiments of the present disclosure.

Referring to FIG. 32, the integrated intelligent system 3200 may include a user terminal 3210 (eg, an electronic device 101), an intelligent server 3220, a personalization information server 3230, or a suggestion server 3240. have. At least one of the intelligent server 3220, the personalized information server 3230, or the proposal server 3240 may be included in the external server 600 described above.

The user terminal 3210 is a service required for a user through an application (or an application program) stored in the user terminal 3210 (eg, an alarm application, a message application, a photo (gallery) application, and the like). Can provide. For example, the user terminal 3210 may execute and operate other applications through an intelligent application (or voice recognition application) stored in the user terminal 3210. Through the intelligent application of the user terminal 3210, a user input for executing and executing an operation of the other application may be received. The user input may be received through, for example, a physical button, a touch pad, voice input, remote input, or the like. According to various embodiments, the user terminal 3210 may correspond to various terminal devices (or electronic devices) that can be connected to the Internet, such as a mobile phone, a smart phone, a personal digital assistant (PDA), or a notebook computer.

According to various embodiments, the user terminal 3210 may receive a user's speech as a user input. The user terminal 3210 may receive a user's utterance and generate a command to operate an application based on the user's utterance. Accordingly, the user terminal 3210 may operate the application using the command.

The intelligent server 3220 may receive a user voice input (eg, user speech) from the user terminal 3210 through a communication network and change it to text data. In another embodiment, the intelligent server 3220 may generate (or select) a path rule (eg, action sequence) based on the text data. The pass rule may include information on an action (or operation or task) for performing a function of an application or information on parameters required to execute the action. Also, the pass rule may include an order of the operation of the application. The user terminal 3210 may receive the pass rule, select an application according to the pass rule, and execute an operation included in the pass rule in the selected application.

For example, the user terminal 3210 may execute the above operation and display a screen corresponding to the state of the user terminal 3210 that executed the operation on the display. For another example, the user terminal 3210 may execute the operation and may not display the result of performing the operation on the display. The user terminal 3210 may, for example, execute a plurality of operations and display only partial results of the plurality of operations on the display. The user terminal 3210 may, for example, display only the result of executing the last sequence of operations on the display. For another example, the user terminal 3210 may receive a user input and display the result of executing the operation on the display.

The personalized information server 3230 may include a database in which user information is stored. For example, the personalization information server 3230 may receive user information (eg, context information, application execution, etc.) from the user terminal 3210 and store it in the database. The intelligent server 3220 may be used when receiving the user information from the personalized information server 3230 through a communication network to generate a pass rule for user input. According to various embodiments, the user terminal 3210 may receive user information from the personalized information server 3230 through a communication network and use it as information for managing the database.

The proposal server 3240 may include a database in which information about a function or application to be provided or function to be provided is stored in the terminal. For example, the proposal server 3240 may receive a user information of the user terminal 3210 from the personalization information server 3230 and include a database of functions that a user can use. The user terminal 3210 may receive information on the function to be provided from the proposal server 3240 through a communication network and provide information to the user.

33 is a block diagram illustrating a user terminal of an integrated intelligent system according to various embodiments.

Referring to FIG. 33, the user terminal 3210 includes an input module 3310 (eg, a microphone 288 or an input device 250), a display 3320 (eg, a display 160 or 260), a speaker 4320 ) (Eg, speaker 282), memory 3340 (eg, memory 130 or memory 230) or processor 3350 (eg, processor 120 or 210). The user terminal 3210 may further include a housing, and components of the user terminal 3210 may be seated inside the housing or may be located on the housing.

According to various embodiments, the input module 3310 may receive user input from a user. For example, the input module 3310 may receive user input from a connected external device (eg, keyboard, headset). For another example, the input module 3310 may include a touch screen (eg, a touch screen display) combined with the display 3320. For another example, the input module 3310 may include a hardware key (or a physical key) located in the user terminal 3210 (or the housing of the user terminal 3210).

According to various embodiments, the input module 3310 may include a microphone 3311 capable of receiving a user's speech as a voice signal. For example, the input module 3310 includes a speech input system and may receive a user's speech as a voice signal through the speech input system.

According to various embodiments, the display 3320 may display an image, a video, and/or an execution screen of an application. For example, the display 3320 may display a graphical user interface (GUI) of the application.

According to various embodiments, the speaker 4320 may output a voice signal. For example, the speaker 4320 may output a voice signal generated inside the user terminal 3210 to the outside.

According to various embodiments, the memory 3340 may store a plurality of applications 3331 and 3343. The plurality of applications 3331 and 3343 stored in the memory 3340 may be selected, executed, and operated according to user input.

According to various embodiments, the memory 3340 may include a database capable of storing information necessary to recognize a user input. For example, the memory 3340 may include a log database capable of storing log information. For another example, the memory 3340 may include a persona database that can store user information.

According to various embodiments, the memory 3340 stores a plurality of applications 3331 and 3343, and the plurality of applications 3331 and 3343 may be loaded and operated. For example, the plurality of applications 3331 and 132 stored in the memory 3340 may be loaded and operated by the execution manager module 3355 of the processor 3350. The plurality of applications 3331 and 3343 may include execution services 3331a and 3343a that perform functions or a plurality of operations (or unit operations) 3331b and 3343b. The execution services 3331a and 3343a are generated by the execution manager module 3335 of the processor 3350, and may execute a plurality of operations 3331b and 3343b.

According to various embodiments, when the operations 3331b and 3343b of the applications 3331 and 3343 are executed, the execution status screen according to the execution of the operations 3331b and 3343b may be displayed on the display 3320. The execution status screen may be, for example, a screen in which the operations 3331b and 3343b have been completed. The execution status screen may be, for example, a screen in which the execution of operations 3331b and 3343b is stopped (eg, when parameters required for operations 3331b and 3343b are not input). .

According to various embodiments, the execution services 3331a and 3343a may execute operations 3331b and 3343b according to a pass rule. For example, the execution services 3331a and 3343a are generated by the execution manager module 3335, receive an execution request according to the pass rule from the execution manager module 3335, and operate 341b according to the execution request. 3343b) may execute the operations of the applications 3331 and 3343. The execution services 3331a and 3343a may transmit completion information to the execution manager module 3335 when the operations 3331b and 3343b are completed.

According to various embodiments, when a plurality of operations 3331b and 3343b are executed in the applications 3331 and 3343, the plurality of operations 3331b and 3343b may be sequentially executed. When the execution of one operation (operation 1) is completed, the execution services 3331a and 3343a may open the next operation (operation 2) and transmit the completion information to the execution manager module 3335. Here, opening an arbitrary operation may be understood as transitioning the arbitrary operation to an executable state or preparing for execution of the arbitrary operation. In other words, if any operation is not opened, the operation cannot be executed. When the completion information is received, the execution manager module 3335 may transmit an execution request for the next operations 3331b and 3343b to the execution service (eg, operation 2). According to various embodiments, when a plurality of applications 3331 and 3343 are executed, the plurality of applications 3331 and 3343 may be sequentially executed. For example, when execution of the last operation of the first application 3331 is completed and the completion information is received, the execution manager module 3335 sends an execution request of the first operation of the second application 3333 to the execution service 3333a. I can send it.

According to various embodiments, when a plurality of operations 3331b and 3343b are executed in the applications 3331 and 3343, a result screen according to each execution of the plurality of operations 3331b and 3343b is displayed 3320 Can be displayed on. According to various embodiments, only a part of the plurality of result screens according to execution of the executed plurality of operations 3331b and 3343b may be displayed on the display 3320.

According to various embodiments, the memory 3340 may store an intelligent application (eg, a voice recognition application) associated with the intelligent agent 3351. The application associated with the intelligent agent 3351 may receive and process a user's speech as a voice signal. According to various embodiments, the application associated with the intelligent agent 3351 operates by a specific input (eg, input through a hardware key, input through a touch screen, specific voice input) input through the input module 3310. Can be.

According to various embodiments, the processor 3350 may control the overall operation of the user terminal 3210. For example, the processor 3350 may control the input module 3310 to receive a user input. The processor 3350 may control the display 3320 to display an image. The processor 3350 may control the speaker 4320 to output a voice signal. The processor 3350 controls the memory 3340 to load or store necessary information.

According to various embodiments, the processor 3350 may include an intelligent agent 3351, an execution manager module 3355 or an intelligent service module 3355. In various embodiments, the processor 3350 executes instructions stored in the memory 3340 to drive the intelligent agent 3351, the execution manager module 3355, or the intelligent service module 3355. Various modules mentioned in various embodiments of the present invention may be implemented in hardware or may be implemented in software. In various embodiments of the present invention, an operation performed by the intelligent agent 3351, the execution manager module 3355, or the intelligent service module 3355 may be understood as an operation performed by the processor 3350.

According to various embodiments, the intelligent agent 3351 may generate a command to operate an application based on a voice signal received as a user input. According to various embodiments, the execution manager module 3335 may receive the generated command from the intelligent agent 3351, select and execute and operate the applications 3331 and 3343 stored in the memory 3340. According to various embodiments, the intelligent service module 3355 may be used to process user input by managing user information.

The intelligent agent 3351 may transmit and process user input received through the input module 3310 to the intelligent server 3220.

According to various embodiments, the intelligent agent 3351 may preprocess the user input before transmitting the user input to the intelligent server 3220. According to various embodiments, the intelligent agent 3351 pre-processes the user input, an adaptive echo canceller (AEC) module, a noise suppression (NS) module, and an endpoint detection (end-) point detection (EPD) module or automatic gain control (AGC) module. The adaptive echo canceller may remove echo included in the user input. The noise suppression module may suppress background noise included in the user input. The end point detection module may detect the end point of the user's voice included in the user input to find a portion where the user's voice is present. The automatic gain control module may adjust the volume of the user input to be suitable for recognizing and processing the user input. According to various embodiments, the intelligent agent 3351 may include all of the preprocessing configurations for performance, but in other embodiments, the intelligent agent 3351 may include some of the preprocessing configurations to operate at low power. have.

According to various embodiments, the intelligent agent 3351 may include a wake up recognition module that recognizes a user's call. The wake-up recognition module may recognize a user's wake-up command through a voice recognition module, and activate the intelligent agent 3351 to receive a user input when the wake-up command is received. According to various embodiments, the wake-up recognition module of the intelligent agent 3351 may be implemented in a low-power processor (eg, a processor included in an audio codec). According to various embodiments, the intelligent agent 3351 may be activated according to user input through a hardware key. When the intelligent agent 3351 is activated, an intelligent application (eg, a voice recognition application) linked with the intelligent agent 3351 may be executed.

According to various embodiments, the intelligent agent 3351 may include a voice recognition module for executing user input. The voice recognition module may recognize a user input for causing an application to perform an operation. For example, the speech recognition module may include a limited user (speech) input (eg, “click”) that executes a shooting operation when the camera application is running, such as the wakeup command in the applications 3331 and 3343. The same utterance, etc.). The voice recognition module that recognizes a user input by assisting the intelligent server 3220 may recognize and process a user command that can be processed in the user terminal 3210, for example. According to various embodiments, a voice recognition module for executing user input of the intelligent agent 3351 may be implemented in an application processor.

According to various embodiments, the voice recognition module of the intelligent agent 3351 (including the voice recognition module of the wake-up module) may recognize a user input using an algorithm for recognizing voice. The algorithm used to recognize the voice may be, for example, at least one of a hidden markov model (HMM) algorithm, an artificial neural network (ANN) algorithm, or a dynamic time warping (DTW) algorithm.

According to various embodiments, the intelligent agent 3351 may convert a user's voice input into text data. According to various embodiments, the intelligent agent 3351 may transmit the user's voice to the intelligent server 3320 to receive the converted text data. Accordingly, the intelligent agent 3351 may display the text on the display 3320.

According to various embodiments, the intelligent agent 3351 may receive a pass rule transmitted from the intelligent server 3220. According to various embodiments, the intelligent agent 3351 may transmit the pass rule to the execution manager module 3335.

According to various embodiments, the intelligent agent 3351 transmits the execution result log according to the pass rule received from the intelligent server 3220 to the intelligent service module 3355, and the executed execution The result log may be accumulated and managed in the preference information of the user of the persona module (persona manager) 3355b.

According to various embodiments, the execution manager module 3353 receives the pass rules from the intelligent agent 3351 to execute the applications 3331 and 3343, and the applications 3331 and 3343 are included in the pass rules ( 3341b, 3343b). For example, the execution manager module 3335 can transmit command information for executing the operations 3331b and 3343b to the applications 3331 and 3343, and the operations 3331b and 3343b from the applications 3331 and 3343. You can receive the complete information.

According to various embodiments, the execution manager module 3335 transmits and receives command information for executing the operations 3331b and 3343b of the applications 3331 and 3343 between the intelligent agent 3351 and the applications 3331 and 3343. can do. The execution manager module 3355 binds the applications 3331 and 3343 to be executed according to the pass rule, and transmits command information of the operations 3331b and 3343b included in the pass rule to the applications 3331 and 3343. can do. For example, the execution manager module 3335 sequentially transmits the operations 3331b and 3343b included in the pass rule to the applications 3331 and 3343, thereby executing the operations 3331b and 3343b of the applications 3331 and 3343. It can be executed sequentially according to the pass rule.

According to various embodiments, the execution manager module 3335 may manage the execution state of the operations 3331b and 3343b of the applications 3331 and 3343. For example, the execution manager module 3335 may receive information on the execution status of the operations 3331b and 3343b from the applications 3331 and 3343. When the execution state of the operations 3331b and 3343b is, for example, a partial landing (for example, when a parameter required for the operations 3331b and 3343b is not input), the execution manager module 3351 May transmit the information on the stopped state to the intelligent agent 3351. The intelligent agent 3351 may request the user to input necessary information (eg, parameter information) using the received information. When the execution state of the operations 3331b and 3343b is, for example, an operation state, a utterance may be received from a user, and the execution manager module 3335 may execute the applications 3331 and 3343 and applications. Information about the execution state of (3341, 3343) may be transmitted to the intelligent agent 3351. The intelligent agent 3351 may receive parameter information of the user's utterance through the intelligent server 3220, and may transmit the received parameter information to the execution manager module 3335. The execution manager module 3335 may change the parameters of the operations 3331b and 3343b to new parameters using the received parameter information.

According to various embodiments, the execution manager module 3335 may transmit parameter information included in the pass rule to the applications 3331 and 3343. When a plurality of applications 3331 and 3343 are sequentially executed according to the pass rule, the execution manager module 3335 may transmit parameter information included in the pass rule from one application to another application.

According to various embodiments, the execution manager module 3335 may receive a plurality of pass rules. The execution manager module 3353 may select a plurality of pass rules based on the user's speech. For example, the execution manager module 3335 specifies some applications 3331 in which the user's speech will execute some operations 3331a, but does not specify other applications 3431 to execute the remaining operations 3431b. The same application 3341 (e.g., gallery application) to execute the operation 3331a (e.g., gallery application) is executed, and different applications 3343 (e.g., message application, telegram application) capable of executing the remaining operation 3343b are executed. A plurality of different pass rules may be received. The execution manager module 3335 may execute, for example, the same operations 3331b and 3343b of the plurality of pass rules (for example, the same operations 3331b and 3343b in series). When executing the same operation, the execution manager module 3335 may display a status screen on the display 3320 to select different applications 3331 and 3343 included in the plurality of pass rules, respectively.

According to various embodiments, the intelligent service module 3355 may include a context module 3355a, a persona module 3355b, or a proposal module 3355c.

The context module 3355a may collect the current state of the applications 3331 and 3343 from the applications 3331 and 3343. For example, the context module 3355a may collect the current state of the applications 3331 and 3343 by receiving context information indicating the current state of the applications 3331 and 3343.

The persona module 3355b may manage personal information of a user who uses the user terminal 3210. For example, the persona module 3355b may collect user information and performance results of the user terminal 3210 to manage personal information of the user.

The proposal module 3355c may predict a user's intention and recommend a command to the user. For example, the suggestion module 3355c may recommend a command to the user in consideration of the user's current state (eg, time, place, situation, application).

34 is a block diagram showing an intelligent server of the integrated intelligent system according to various embodiments.

Referring to FIG. 34, the intelligent server 3220 includes an automatic speech recognition (ASR) module 3410 (eg, ASR module 751), a natural language understanding (NLU) module 3420 ) (E.g. NLU module 753), path planner module 3430 (e.g. action plan manager 759), conversation manager (DM) module 3440 (e.g. DM ( 757), natural language generator (NLG) module 3450 (e.g. NLG manager 761) or text to speech (TTS) module 3460 (e.g. TTS manager 763) ).

The natural language understanding module 3420 or the path planner module 3430 of the intelligent server 3220 may generate path rules (eg, action sequences or tasks).

According to various embodiments, the automatic speech recognition (ASR) module 3410 may convert user input received from the user terminal 3210 into text data.

According to various embodiments, the automatic speech recognition module 3410 may convert user input received from the user terminal 3210 into text data. For example, the automatic speech recognition module 3410 may include a speech recognition module. The speech recognition module may include an acoustic model and a language model. For example, the acoustic model may include information related to speech, and the language model may include information on a combination of unit phoneme information and unit phoneme information. The speech recognition module may convert user speech into text data by using information related to speech and information about unit phoneme information. Information about the acoustic model and the language model may be stored in, for example, an automatic speech recognition database (ASR DB) 3411.

According to various embodiments, the natural language understanding module 3420 may grasp a user's intention by performing syntactic analysis or semantic analysis. The grammatical analysis may divide user input into grammatical units (eg, words, phrases, morphemes, etc.), and grasp what grammatical elements the divided units have. The semantic analysis may be performed using semantic matching, rule matching, or formula matching. Accordingly, the natural language understanding module 3420 may obtain a domain, an intent, or a parameter (or a slot) necessary for the user input to express the intention.

According to various embodiments, the natural language understanding module 3420 uses a matching rule divided into a domain, an intent, and a parameter (or a slot) required to grasp the intention. The user's intentions and parameters can be determined. For example, the one domain (eg, alarm) may include a plurality of intentions (eg, alarm setting, alarm release, etc.), and one intention may include a plurality of parameters (eg, time, number of repetitions, alarm sound) Etc.). The plurality of rules may include, for example, one or more essential element parameters. The matching rule may be stored in a natural language understanding database (NLU DB) 3421.

According to various embodiments, the natural language understanding module 3420 uses a linguistic feature such as a morpheme and a phrase (eg, a grammatical element) to grasp the meaning of the word extracted from the user input, and to determine the meaning of the identified word. The user's intention can be determined by matching the domain and the intention. For example, the natural language understanding module 3420 may determine the user intention by calculating how many words extracted from the user input are included in each domain and intention. According to various embodiments, the natural language understanding module 3420 may determine a parameter of a user input using a word based on grasping the intention. According to various embodiments, the natural language understanding module 3420 may determine the user's intention using the natural language recognition database 3421 in which linguistic features for identifying the intention of the user input are stored. According to another embodiment, the natural language understanding module 3420 may determine a user's intention using a personal language model (PLM). For example, the natural language understanding module 3420 may determine a user's intention using personalized information (eg, a contact list, a music list). The personalized language model may be stored in the natural language recognition database 3421, for example. According to various embodiments, the automatic speech recognition module 3410 as well as the natural language understanding module 3420 may recognize a user's voice by referring to a personalized language model stored in the natural language recognition database 3421.

According to various embodiments, the natural language understanding module 3420 may generate a pass rule based on intention and parameters of a user input. For example, the natural language understanding module 3420 may select an application to be executed based on the intention of a user input, and determine an operation to be performed in the selected application. The box salmon understanding module 3420 may generate a pass rule by determining a parameter corresponding to the determined operation. According to various embodiments, the pass rule generated by the natural language understanding module 3420 may include information about an application to be executed, an operation to be executed in the application, and parameters required to execute the operation.

According to various embodiments, the natural language understanding module 3420 may generate one pass rule or a plurality of pass rules based on the intention and parameters of the user input. For example, the natural language understanding module 3420 receives the pass rule set corresponding to the user terminal 3210 from the pass planner module 3430, and maps the intent and parameters of the user input to the received pass rule set to pass Rules can be determined.

According to another embodiment, the natural language understanding module 3420 determines an application to be executed, an operation to be executed in the application, and a parameter required to execute the operation, based on the intention and parameters of a user input, to perform one pass rule, or multiple passes You can create rules. For example, the natural language understanding module 3420 uses the information of the user terminal 3210 to display the application to be executed and the operation to be executed in the application in the form of an ontology or graph model according to the intention of user input. You can create pass rules by arranging them. The generated pass rule may be stored in a path rule database (PR DB) 3431 through the pass planner module 3430, for example. The generated pass rule may be added to the pass rule set of the database 3431.

According to various embodiments, the natural language understanding module 3420 may select at least one pass rule among the generated plurality of pass rules. For example, the natural language understanding module 3420 may select an optimal pass rule for the plurality of pass rules. For another example, the natural language understanding module 3420 may select a plurality of pass rules when only some motions are specified based on a user's speech. The natural language understanding module 3420 may determine one pass rule among the plurality of pass rules by the user's additional input.

According to various embodiments, the natural language understanding module 3420 may transmit a pass rule to the user terminal 3210 as a request for user input. For example, the natural language understanding module 3420 may transmit one pass rule corresponding to the user input to the user terminal 3210. For another example, the natural language understanding module 3420 may transmit a plurality of pass rules corresponding to user input to the user terminal 3210. The plurality of pass rules may be generated by the natural language understanding module 3420 when only some actions are specified based on, for example, user speech.

According to various embodiments, the pass planner module 3430 may select at least one pass rule among a plurality of pass rules.

According to various embodiments, the pass planner module 3430 may transmit a pass rule set including a plurality of pass rules to the natural language understanding module 3420. The plurality of pass rules of the pass rule set may be stored in a table form in the pass rule database 3431 connected to the pass planner module 3430. For example, the pass planner module 3430 transmits a pass rule set corresponding to information (eg, OS information, application information) of the user terminal 3210 received from the intelligent agent 3351 to the natural language understanding module 3420. Can. The table stored in the pass rule database 3431 may be stored, for example, for each domain or version of the domain.

According to various embodiments, the pass planner module 3430 may select one pass rule or a plurality of pass rules from the pass rule set and transmit the selected pass rule to the natural language understanding module 3420. For example, the path planner module 3430 matches a user's intention and parameters to a set of pass rules corresponding to the user terminal 3210 to select one pass rule or a plurality of pass rules to understand the natural language module 3420 Can be delivered to.

According to various embodiments, the pass planner module 3430 may generate one pass rule or a plurality of pass rules using user intention and parameters. For example, the pass planner module 3430 may generate an pass rule or a plurality of pass rules by determining an application to be executed and an operation to be performed in the application based on user intention and parameters. According to various embodiments, the pass planner module 3430 may store the generated pass rules in the pass rule database 3431.

According to various embodiments, the pass planner module 3430 may store the pass rules generated in the natural language understanding module 3420 in the pass rule database 3431. The generated pass rule may be added to the pass rule set stored in the pass rule database 3431.

According to various embodiments, the table stored in the pass rule database 3431 may include a plurality of pass rules or a plurality of pass rule sets. The plurality of pass rules or the plurality of pass rule sets may reflect a type, version, type, or characteristic of a device that performs each pass rule.

According to various embodiments, the conversation manager module 3440 may determine whether the user's intention identified by the natural language understanding module 3420 is clear. For example, the conversation manager module 3440 may determine whether the user's intention is clear based on whether the parameter information is sufficient. The conversation manager module 3440 may determine whether the parameters identified in the natural language understanding module 3420 are sufficient to perform the task. According to various embodiments, the conversation manager module 3440 may perform feedback requesting information necessary from the user when the user's intention is not clear. For example, the conversation manager module 3440 may perform feedback requesting information about parameters for grasping the user's intention.

According to various embodiments, the conversation manager module 3440 may include a content provider module. The content providing module may generate a result of performing a task corresponding to a user input when an operation can be performed based on intentions and parameters identified in the natural language understanding module 3420. According to various embodiments, the conversation manager module 3440 may transmit the result generated in the content providing module to the user terminal 3210 in response to a user input.

According to various embodiments, the natural language generation module (NLG) 3450 may change the designated information into a text form. The information changed to the text form may be in the form of natural language speech. The specified information may be, for example, information about additional input, information that guides the completion of an operation corresponding to the user input, or information that guides an additional input of the user (eg, feedback information on user input). The information changed in the text form may be transmitted to the user terminal 3210 and displayed on the display 3320, or may be transmitted to the text-to-speech module 3460 to be changed into a voice form.

According to various embodiments, the text-to-speech module 3460 may change text-type information to voice-type information. The text-to-speech module 3460 may receive text-type information from the natural language generation module 3450 and change the text-type information to voice-type information and transmit the text-type information to the user terminal 3210. The user terminal 3210 may output the voice type information to the speaker 4320.

According to various embodiments, the natural language understanding module 3420, the path planner module 3430, and the conversation manager module 3440 may be implemented as one module. For example, the natural language understanding module 3420, the path planner module 3430, and the conversation manager module 3440 may be implemented as one module to determine the user's intentions and parameters, and correspond to the determined user's intentions and parameters. You can generate a response (eg pass rule). Accordingly, the generated response may be transmitted to the user terminal 3210.

35 is a diagram illustrating a method of generating a path rule of a path planner module according to various embodiments.

Referring to FIG. 35, according to various embodiments, the natural language understanding module 3420 may divide the function of the app into unit operations A to F and store it in the pass rule database 3435. For example, the natural language understanding module 3420 uses a plurality of pass rules (A-B1-C1, A-B1-C2, A-B1-C3-DF, A-B1-C3-DEF), which are divided into unit operations. The included pass rule set may be stored in the pass rule database 3435.

According to various embodiments, the pass rule database 3435 of the pass planner module 3430 may store a pass rule set for performing an app function. The pass rule set may include a plurality of pass rules including a plurality of operations. In the plurality of pass rules, operations executed according to parameters input to each of the plurality of operations may be sequentially arranged. According to an embodiment, the plurality of pass rules may be configured in an ontology or graph model form and stored in the pass rule database 3435.

According to various embodiments, the natural language understanding module 3420 may include the plurality of pass rules A-B1-C1, A-B1-C2, A-B1-C3-DF, and A corresponding to intention and parameters of a user input. The optimal pass rule (A-B1-C3-DF) can be selected from -B1-C3-DEF).

According to various embodiments, the natural language understanding module 3420 may deliver a plurality of rules to the user terminal 3210 when there is no pass rule that perfectly matches the user input. For example, the natural language understanding module 3420 may select a pass rule (eg, A-B1) partially corresponding to the user input. The natural language understanding module 3420 includes one or more pass rules (eg, A-B1-C1, A-B1-C2, A-B1-C3) including pass rules (eg, A-B1) partially corresponding to user input. -DF, A-B1-C3-DEF) can be delivered to the user terminal 3210.

According to various embodiments, the natural language understanding module 3420 selects one of a plurality of pass rules based on the additional input of the user terminal 3210, and delivers the selected one pass rule to the user terminal 3210. Can. For example, the natural language understanding module 3420 may include a plurality of pass rules (eg, A-B1-C1, A-B1) according to a user input (eg, an input for selecting C3) additionally input from the user terminal 3210. One of the pass rules (eg, A-B1-C3-DF) among -C2, A-B1-C3-DF, and A-B1-C3-DEF) may be selected and transmitted to the user terminal 3210.

According to various embodiments of the present disclosure, the natural language understanding module 3420 is a user's intention corresponding to a user input (eg, an input for selecting C3) additionally input to the user terminal 3210 through the natural language understanding module 3420 And a parameter, and transmit the determined user's intention or parameter to the user terminal 3210. The user terminal 3210 is based on the transmitted intention or the parameter, a plurality of pass rules (eg, A-B1-C1, A-B1-C2, A-B1-C3-DF, A-B1-C3- DEF), one of the pass rules (eg, A-B1-C3-DF) can be selected.

Accordingly, the user terminal 3210 may complete the operation of the apps 3551 and 3543 by the selected one pass rule.

According to various embodiments, the natural language understanding module 3420 may generate a pass rule partially corresponding to the received user input when a user input lacking information is received by the intelligent server 3400. For example, the natural language understanding module 3420 may transmit (①) the partially corresponding pass rule to the intelligent agent 3551. The intelligent agent 3551 transmits (②) the partially corresponding pass rule to the execution manager module 353, and the execution manager module 353 can execute the first app 3541 according to the pass rule. . The execution manager module 353 may transmit (③) information about the insufficient parameter to the intelligent agent 3551 while executing the first app 3541. The intelligent agent 3551 may request additional input from the user by using the information on the insufficient parameter. When an additional input is received (④) by the user, the intelligent agent 3551 may transmit to the intelligent server 3400 for processing. The natural language understanding module 3420 may generate an additional pass rule based on the additionally input user input intention and parameter information and transmit (⑤) it to the intelligent agent 3551. The intelligent agent 3551 may execute the second app 3543 by transmitting (⑥) the pass rule to the execution manager module 353.

According to various embodiments, the natural language understanding module 3420 may transmit a user information request to the personalized information server 3300 when a user input in which some information is missing is received by the intelligent server 3400. The personalization information server 3300 may transmit information of a user who inputs a user input stored in a persona database to the natural language understanding module 3420. The natural language understanding module 3420 may use the user information to select a pass rule corresponding to a user input in which some operations are missing. Accordingly, the natural language understanding module 3420 requests additional information by requesting the missing information or receives the additional input, or passes the rule corresponding to the user input by using the user information, even if the user input with some information missing is received by the intelligent server 3400. Can decide.

According to various embodiments, in the electronic device 101, a microphone, a communication module including a circuit, and at least one processor 120, the at least one processor 120 to obtain a user voice Control the microphone, control the communication module to deliver the obtained user voice to the server 201, and receive a command for performing at least one operation sequentially performed from the server 201 To control the communication module, and the at least one or more operations correspond to at least a part of the semantic information of the user voice and at least a part of the emphasis information of the user voice identified according to the analysis of the user voice An electronic device 101 configured to perform the at least one or more operations based on the received command may be provided.

According to various embodiments, the emphasis information may include at least one of the user's voice that satisfies a specified phonetic condition, or at least a portion that satisfies a specified linguistic condition, and the specified phonetic The condition includes at least one of a size of a specified volume, a height of a specified pitch, or a length of a specified pronunciation, wherein the specified linguistic condition is confirmed by a specified word, or a repeated word within the user's voice It may include at least one of.

According to various embodiments, at least a portion of the semantic information of the user voice is identified based on at least a portion of the emphasis information of the user voice, and at least one parameter included in at least a portion of the semantic information of the user voice is the It may be set based on at least a part of the emphasis information of the user voice.

According to various embodiments, a parameter corresponding to at least a part of the at least one operation is a parameter included in at least some of the meaning information of the user voice, and the parameter included in at least some of the meaning information of the user voice May be set based on emphasis information of the user voice.

According to various embodiments, at least some of the semantic information of the user voice includes information indicating a time point of a user's speech corresponding to the user voice emphasis information of the emphasis information of the user voice or a type of emphasis information of the user voice and It is identified based on the related information, and the information related to the viewpoint is information indicating a viewpoint of the user's speech corresponding to the emphasis information of the user's voice, and the type of the emphasis information of the user's voice is a product that satisfies the specified phonetic condition It may include 1 emphasis information or second emphasis information that satisfies the specified linguistic conditions.

According to various embodiments, the at least one processor 120 executes an alarm application based on the received command as at least part of an operation of performing at least one or more operations based on the received command, The electronic device 101 is set so that an alarm is provided at a specified number of times corresponding to emphasis information of the user voice during a time period specified by the executed alarm application.

According to various embodiments, the at least one processor 120 is at least part of an operation of performing at least one operation based on the received command, and a plurality of applications installed in the electronic device 101 based on the command Among the plurality of first applications corresponding to the emphasis information of the user's voice, the plurality of first applications may be executed, and an operation corresponding to at least a portion of the at least one operation may be performed by each of the plurality of executed first applications. Can.

According to various embodiments, the at least one processor 120 may receive a first command for performing at least one operation from the server 201 in response to transmitting the user voice to the server 201. The communication module may be configured to set the status value of the electronic device 101 to a designated first value corresponding to emphasis information of the user's voice, based on the received first command.

According to various embodiments, the status value may include a setting value of the electronic device including at least one audio output parameter of the electronic device, a display brightness value of the electronic device, or a value indicating the degree of vibration of the electronic device. It can contain.

According to various embodiments, the at least one processor 120 acquires a first user voice for a specified period of time, delivers the acquired first user voice to the server 201, and from the server 201 Receiving a second command for performing at least one first operation, and at least a portion of the at least one first operation corresponds to first information of the first user voice corresponding to emphasis information of the user voice, Based on the received second command, the state value of the electronic device 101 may be further set to be set to a second value corresponding to the first information of the first user voice.

According to various embodiments, at least one external electronic device associated with the electronic device is identified by the server based on emphasis information of the user voice, and the server 201 performs the at least one operation. The command for the communication is transmitted to the at least one external electronic device 2010, and the at least one operation may be performed based on the received command by the at least one other external electronic device 2010.

According to various embodiments, the at least one processor 120 is at least part of an operation for performing at least one operation based on the received command, and is based on at least a portion of at least one operation corresponding to the emphasis information. In response, in response to acquiring user input and obtaining the user input while sequentially outputting information corresponding to the user voice stored in the electronic device and sequentially outputting information corresponding to the user voice, the It may be set to stop outputting information corresponding to the user's voice.

According to various embodiments, the at least one processor 120 is set to control the communication module to receive information on an answer including at least one text from the server 201, and the at least one text Is confirmed based on the meaning information of the user voice and the emphasis information of the user voice confirmed according to the analysis of the user voice, and may be set to output at least one text based on the received answer information. .

According to various embodiments, a microphone, a communication module including a circuit, and at least one processor 120, wherein the at least one processor 120 controls the microphone to acquire a user voice, and the acquisition The communication module controls the communication module to transmit the user voice to the server 201, and controls the communication module to receive information on an answer including at least one text from the server 201, and the at least one text Information about the answer including is confirmed based on the meaning information of the user voice and emphasis information of the user voice confirmed according to the analysis of the user voice, and based on the information on the received answer The electronic device 101, which is configured to output at least one text in a designated manner as may be provided.

According to various embodiments, the at least one processor 120 is at least part of an operation of outputting at least one text in a designated manner based on information on the answer, based on the information on the answer, and the at least One or more texts are set to be displayed on the display of the electronic device 101, and a first text corresponding to the emphasis information of the user's voice among the at least one text is displayed on another text of the displayed at least one text. In comparison, it may be visually emphasized.

According to various embodiments, the at least one processor 120 is at least part of an operation of outputting at least one text in a designated manner based on the information on the answer, and the at least one is based on the information on the answer The text above is set to be output through the audio device of the electronic device 101, and at least a part of the at least one text corresponding to the emphasis information of the user's voice is another part of the at least one text It can be output with audible emphasis.

According to various embodiments, a memory for storing instructions, and at least one processor 120, wherein the instructions, when executed, the at least one processor 120 acquires a user voice, and the user voice Check the emphasis information of the user's voice corresponding to, check the semantic information of the user's voice corresponding to the user's voice, sequentially based on the identified emphasis information of the user's voice and the semantic information of the identified user's voice An electronic device 101 configured to identify a command for performing at least one or more operations performed and to transmit a command for performing the identified at least one operation may be provided.

According to various embodiments of the present disclosure, at least one processor 120 identifies at least a portion of the meaning information of the user voice corresponding to at least a portion of the highlight information of the user voice, and displays the highlight information of the user voice. A value of at least one parameter included in at least a part of the semantic information of the identified user voice is set based on at least a part, and the meaning of the user voice includes at least a part and another part including a value of the set parameter Based on the information, it may be set to confirm a command for performing the at least one operation.

According to various embodiments, the emphasis information of the user voice may be identified using at least one of at least one acoustic analysis technique or at least one linguistic analysis technique.

According to various embodiments, the at least one acoustic analysis technique is a technique for analyzing at least one of a volume, pitch, or pronunciation length of at least a portion of the user voice, and the at least one linguistic analysis technique is the user voice Is a technique for analyzing a specific word repetition, or emphasizing adverb utterance, wherein the first information is applied to the first emphasis information identified from the user's voice or the at least one linguistic analysis technique according to the at least one acoustic analysis technique. Accordingly, it may be characterized in that the second emphasis information confirmed from the user voice.

According to various embodiments, at least one processor 120 checks answer information on at least one text based on the highlighted user voice emphasis information and the identified user voice semantic information. , May be configured to transmit answer information regarding the at least one text.

According to various embodiments, a microphone, a communication module including a circuit, and at least one processor 120, wherein the at least one processor 120 controls the microphone to acquire a user voice, and the acquisition The communication module to control the communication module to transmit the user voice to the server 201, and to receive a first command (command) for performing at least one first operation performed sequentially from the server 201 And the first operation corresponds to semantic information of the user voice identified according to the analysis of the user voice when the emphasis information is not confirmed from the user voice, and the server 201. It is set to control the communication module to receive a second command (command) for performing at least one second operation performed sequentially from the, the at least one second operation, the highlight information from the user voice is confirmed In some cases, the electronic device 101 may be provided to correspond to semantic information of the user voice and emphasis information of the user voice identified according to the analysis of the user voice.

Claims (21)

In the electronic device,
MIC;
A communication module including circuitry; And
And at least one processor; wherein the at least one processor comprises:
Control the microphone to obtain a user voice,
Control the communication module to deliver the acquired user voice to a server,
The communication module is controlled to receive a command for performing at least one operation sequentially performed from the server, and the at least one operation is the meaning of the user voice confirmed according to analysis of the user voice Corresponds to at least a portion of information and at least a portion of emphasis information of the user voice,
An electronic device configured to perform the at least one or more operations based on the received command.
According to claim 1,
The emphasis information may include at least one of the user voices that satisfies a specified phonetic condition, or at least a portion that satisfies a specified linguistic condition,
The designated phonetic condition includes at least one of a specified volume size, a specified pitch height, or a specified pronunciation length,
The specified linguistic condition includes at least one of being identified by a designated word or being repeated by a word repeated in the user's voice.
According to claim 2,
At least a part of the semantic information of the user voice is identified based on at least a part of the emphasis information of the user voice,
The at least one parameter included in at least a part of the meaning information of the user voice is set based on at least a part of the emphasis information of the user voice.
The method of claim 3,
At least some of the at least one operation corresponds to at least one parameter included in at least a part of the semantic information of the user voice, the electronic device.
The method of claim 4,
At least some of the meaning information of the user's voice is identified based on information related to the type of the user's voice emphasis information or information indicating a time point of the user's speech corresponding to the user voice emphasis information of the user voice emphasis information ,
The information related to the viewpoint is information indicating a viewpoint of user speech corresponding to the emphasis information of the user voice,
The type of emphasis information of the user voice includes first emphasis information that satisfies the specified phonetic condition or second emphasis information that satisfies the specified linguistic condition.
According to claim 1,
The electronic device emphasizing information of the user voice is identified based on linguistic analysis technology or acoustic analysis technology separately from the semantic information of the user voice.
According to claim 1,
The at least one processor is:
At least part of the operation of performing at least one or more operations based on the received command, the alarm application is executed based on the received command, and the execution is executed at a number of times corresponding to the emphasis information of the user's voice at a specified time. An electronic device that is configured to provide an alarm by an alarm application.
According to claim 1,
The at least one processor is:
As at least part of the operation of performing at least one or more operations based on the received command:
Based on the command, among the plurality of applications installed in the electronic device, the plurality of first applications corresponding to the emphasis information of the user voice are executed,
An electronic device configured to control an operation corresponding to the at least one operation to be performed by each of the plurality of executed first applications.
According to claim 1,
The at least one processor is:
As at least part of the operation of performing at least one or more operations based on the received command:
Based on the received first command, the electronic device is set to set the state value of the electronic device to a designated first value corresponding to emphasis information of the user's voice.
The method of claim 9,
The status value is a set value of the electronic device including at least one audio volume value of the electronic device, a display brightness value of the electronic device, or a value indicating the degree of vibration of the electronic device.
The method of claim 10,
The at least one processor is:
Acquire a first user voice for a specified period of time,
Deliver the acquired first user voice to the server,
A second command for performing at least one first operation from a server, and at least a portion of the at least one first operation corresponds to first information of the first user voice corresponding to emphasis information of the user voice and,
Based on the received second command, the electronic device is further set to set a state value of the electronic device to a second value corresponding to the first information of the first user voice.
According to claim 1,
At least one external electronic device associated with the electronic device is identified by the server based on the emphasis information of the user voice,
Commands for performing the at least one operation by the server is transmitted to the at least one external electronic device,
The at least one operation is performed based on the received command by the at least one other external electronic device.
According to claim 1,
The at least one processor is:
As at least part of the operation of performing at least one or more operations based on the received command:
Sequentially outputting information corresponding to the user's voice stored in the electronic device based on at least a part of at least one operation corresponding to the emphasis information,
While sequentially outputting information corresponding to the user's voice, a user input is acquired,
And in response to obtaining the user input, set to stop outputting information corresponding to the user voice.
According to claim 1,
The at least one processor is:
It is set to control the communication module to receive information on an answer including at least one text from the server, wherein the at least one text is semantic information of the user's voice and the information identified by the analysis of the user's voice Confirmed based on the emphasis information of the user's voice,
An electronic device configured to output at least one or more texts based on the received answer information.
In the electronic device,
MIC;
A communication module including circuitry; And
And at least one processor; wherein the at least one processor comprises:
Control the microphone to obtain a user voice,
Control the communication module to deliver the acquired user voice to a server,
The communication module controls the communication module to receive information on an answer including at least one text from the server, and the information on the answer including the at least one text is the user voice confirmed according to the analysis on the user voice Corresponds to at least a part of the semantic information and to at least a part of the emphasis information of the user voice,
An electronic device configured to output at least one text in a designated manner based on information on the received answer.
The method of claim 15,
The at least one processor is:
Set to display the at least one text on the display of the electronic device as at least a part of the operation of outputting at least one text in a designated manner based on the information on the received answer. Become,
An electronic device in which the first text corresponding to the emphasis information of the user's voice among the at least one text is visually emphasized compared to other text among the displayed at least one text.
The method of claim 15,
The at least one processor is:
Set to output the at least one text through the audio device of the electronic device as at least a part of the operation of outputting at least one text in a designated manner based on the information on the received answer. Become,
An electronic device in which at least a portion of the output at least one text corresponding to the emphasis information of the user's voice is audibly emphasized and output compared to another portion of the output at least one text.
A memory for storing instructions; And
And at least one processor;
The instructions, when executed, cause the at least one processor to:
To acquire a user voice,
Check the emphasis information of the user voice corresponding to the user voice,
Check the meaning information of the user voice corresponding to the user voice,
Based on the highlighted user voice emphasis information and the identified user voice semantic information, identify a command for performing at least one operation sequentially performed,
And an electronic device configured to transmit a command for performing the identified at least one operation.
The method of claim 18,
The instructions have at least one processor:
Confirming at least a portion of the semantic information of the user voice corresponding to at least a portion of the emphasis information of the user voice,
Set values of at least one parameter included in at least a portion of the semantic information of the identified user voice based on at least a portion of the emphasis information of the user voice,
The electronic device is configured to confirm a command for performing the at least one operation, based on the semantic information of the user voice including at least a part including the value of the set parameter and another part.
The method of claim 18,
The emphasis information of the user voice is identified using at least one of at least one acoustic analysis technique or at least one linguistic analysis technique,
The at least one acoustic analysis technique is a technique for analyzing at least one of volume, pitch, or pronunciation length of at least a portion of the user's voice,
The at least one linguistic analysis technique is a technique for analyzing a specific word repetition or emphasizing speech of the user's voice,
The first information may be first emphasis information identified from the user's voice according to the at least one acoustic analysis technique or second emphasis information identified from the user's voice according to the at least one linguistic analysis technique. Electronic device.
In the electronic device,
MIC;
A communication module including circuitry; And
And at least one processor; wherein the at least one processor comprises:
Control the microphone to obtain a user voice,
Control the communication module to deliver the acquired user voice to a server,
The communication module is controlled to receive a first command for performing at least one first operation sequentially performed from the server, and the at least one first operation is confirmed by highlighting information from the user voice. If not, it corresponds to the semantic information of the user voice confirmed according to the analysis of the user voice,
In order to control the communication module to receive a second command for performing at least one second operation performed sequentially from the server, the at least one second operation is confirmed by highlighting information from the user voice If possible, the electronic device corresponding to the semantic information of the user's voice and emphasis information of the user's voice identified according to the analysis of the user's voice.

Images