KR102546510B1 - Method for providing information mapped between plurality inputs and electronic device supporting the same - Google Patents

Abstract

An embodiment of the present invention includes a microphone, a touch screen display, a processor, and a memory, wherein the memory, when executed, the processor supports a first interface for supporting recording of voice input through the microphone, and Outputting an execution screen of an application including a second interface supporting user input of information, and recording voice data of a first point of view based on the first interface and the first point of view input based on the second interface Mapping and storing content data of the specified information, receiving a user input for searching for at least one voice data mapped to the specified content data based on a search word for the specified information, and responding to the user input to the specified content data. Disclosed is an electronic device storing instructions for collecting at least one piece of mapped voice data, converting the collected at least one piece of voice data into text data, and outputting the converted text data. In addition to this, various embodiments identified through the specification are possible.

Description

Method for providing information mapped between plurality of inputs and electronic device supporting the same

Various embodiments disclosed in this document relate to a technology for providing information mapped between a plurality of inputs.

Recent electronic devices support various input methods as part of oriented interaction with a user. For example, the electronic device may support an input method using a software input panel (SIP) keyboard or a stylus pen, as well as a voice-based input method according to user speech. Alternatively, the electronic device may support a multimodal interface input method that complexly operates the above-described input methods.

The electronic device may receive a multimodal interface input from a user in relation to recording information related to a specific event. For example, the electronic device may receive and record a voice input related to the event, and simultaneously receive and store content input (eg, text input, etc.) including information related to the voice. At this time, since the electronic device separately processes recording data and content data caused by the same event, user needs to check information (eg, recording information and content information) on the same event in a complex manner may not be met or may not provide convenience.

Various embodiments disclosed in this document are a method for providing mapped information between a plurality of inputs, which can map data according to a multimodal interface input according to a specified criterion and provide them in a linkage manner, and an electronic device supporting the same can provide.

An electronic device according to an embodiment includes at least one microphone, a touch screen display, at least one processor electrically connected to the at least one microphone and the touch screen display, and a memory electrically connected to the at least one processor. can include

According to an embodiment, the memory, when executed, the at least one processor, a first interface for supporting the recording (recording) of the voice input through the at least one microphone through the touch screen display and the user outputs an execution screen of a designated application including a second interface supporting content input of, and based on voice data of a first time recorded based on the first interface in the memory and the second interface Mapping and storing the input content data of the first view, receiving a user input for searching for at least one voice data mapped to the specified content data based on a search word for the specified content, and Collecting at least one voice data mapped to the designated content data in response to an input, converting the collected at least one voice data into text data, and outputting the converted text data through the touch screen display You can store instructions.

A method of providing information mapped between a plurality of inputs of an electronic device according to an embodiment includes a first interface supporting recording of voice input to the electronic device and a second interface supporting content input by a user. An operation of outputting an execution screen of a designated application including an interface, voice data of a first time recorded based on the first interface and content data of the first time input based on the second interface An operation of mapping and storing, an operation of receiving a user input for searching for at least one voice data mapped to the designated content data based on a search word for the designated content, and an operation of at least one mapped to the designated content data. It may include an operation of collecting voice data, an operation of converting the collected at least one voice data into text data, and an operation of outputting the converted text data.

According to various embodiments, by mapping recording data and content data (eg, text data, etc.) related to the same event according to a specified criterion, information related to mutually mapped data can be provided in a complex manner. there is.

In addition to this, various effects identified directly or indirectly through this document may be provided.

1 is a diagram illustrating an example of operation of an electronic device according to an embodiment.
2 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment.
3A is a diagram illustrating a response form of an electronic device to a first user input according to an embodiment.
3B is a diagram illustrating a response form of an electronic device to a second user input according to an exemplary embodiment.
3C is a diagram illustrating a response form of an electronic device to a third user input according to an embodiment.
3D is a diagram illustrating a graphic effect display form corresponding to recording and playback of an electronic device according to an exemplary embodiment.
4 is a diagram illustrating a response form of an electronic device to a fourth user input according to an exemplary embodiment.
5 is a diagram illustrating a response form to a fourth user input of an electronic device according to another embodiment.
6 is a diagram illustrating a form of a response to a fifth user input of an electronic device according to an exemplary embodiment.
7A is a diagram illustrating a response form of an electronic device to a sixth user input according to an exemplary embodiment.
7B is a diagram illustrating a response form of an electronic device to a seventh user input according to an embodiment.
7C is a diagram illustrating a specific information registration form related to processing of a seventh user input of an electronic device according to an embodiment.
8 is a diagram illustrating a method of providing information mapped between a plurality of inputs of an electronic device according to an embodiment.
9 is a diagram illustrating an electronic device in a network environment according to an embodiment.
In connection with the description of the drawings, the same reference numerals may be assigned to the same or corresponding elements.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that this is not intended to limit the present invention to the specific embodiments, and includes various modifications, equivalents, and/or alternatives of the embodiments of the present invention. In connection with the description of the drawings, like reference numerals may be used for like elements.

In this document, expressions such as "has", "may have", "includes", or "may include" refer to the presence of a corresponding feature (eg, numerical value, function, operation, or component such as a part). , which does not preclude the existence of additional features.

In this document, expressions such as "A or B", "at least one of A and/and B", or "one or more of A or/and B" may include all possible combinations of the items listed together. . For example, "A or B", "at least one of A and B", or "at least one of A or B" includes (1) at least one A, (2) at least one B, Or (3) may refer to all cases including at least one A and at least one B.

Expressions such as "first", "second", "first", or "second" as used herein may modify various elements, in any order and/or importance, and may refer to one element as another. It is used to distinguish from components, but does not limit the components. For example, a first user device and a second user device may represent different user devices regardless of order or importance. For example, without departing from the scope of rights described in this document, a first element may be called a second element, and similarly, the second element may also be renamed to the first element.

A component (e.g., a first component) is "(operatively or communicatively) coupled with/to" another component (e.g., a second component); When referred to as "connected to", it should be understood that the certain component may be directly connected to the other component or connected through another component (eg, a third component). On the other hand, when an element (eg, a first element) is referred to as being “directly connected” or “directly connected” to another element (eg, a second element), the element and the above It may be understood that other components (eg, third components) do not exist between the other components.

As used in this document, the expression "configured to" means "suitable for", "having the capacity to", depending on the situation. ", "designed to", "adapted to", "made to", or "capable of" can be used interchangeably. The term "configured (or set) to" may not necessarily mean only "specifically designed to" hardware. Instead, in some contexts, the phrase "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or set) to perform A, B, and C" may include a dedicated processor (eg, an embedded processor) to perform those operations, or one or more software programs stored in a memory device that executes By doing so, it may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe a specific embodiment, and may not be intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by a person of ordinary skill in the technical field described in this document. Among the terms used in this document, the terms defined in general dictionaries may be interpreted as having the same or similar meaning to the meaning in the context of the related art, and unless explicitly defined in this document, in an ideal or excessively formal meaning. not interpreted In some cases, even terms defined in this document cannot be interpreted to exclude the embodiments of this document.

An electronic device according to various embodiments of the present document may include, for example, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an e-book reader, Desktop PC, laptop PC, netbook computer, workstation, server, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, mobile medical device, It may include at least one of a camera or a wearable device. According to various embodiments, the wearable device is an accessory type (eg, a watch, a ring, a bracelet, an anklet, a necklace, glasses, a contact lens, or a head-mounted-device (HMD)), a fabric or a clothing integral type ( For example, it may include at least one of an electronic clothing), a body attachment type (eg, a skin pad or tattoo), or a body implant type (eg, an implantable circuit).

In some embodiments, the electronic device may be a home appliance. Home appliances include, for example, televisions, DVD players (Digital Video Disk players), audio systems, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air purifiers, set-top boxes, and home automation. Home automation control panel, security control panel, TV box (e.g. Samsung HomeSync™, Apple TV™, or Google TV™), game console (e.g. Xbox™, PlayStation™), electronics It may include at least one of a dictionary, an electronic key, a camcorder, or an electronic picture frame.

In another embodiment, the electronic device may include various types of medical devices (e.g., various portable medical measuring devices (blood glucose meter, heart rate monitor, blood pressure monitor, body temperature monitor, etc.), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), CT (computed tomography), camera, or ultrasonicator, etc.), navigation device, GNSS (Global Navigation Satellite System), EDR (event data recorder), FDR (flight data recorder), automotive infotainment ) devices, marine electronics (e.g. marine navigation systems, gyrocompasses, etc.), avionics, security devices, automotive head units, industrial or home robots, automatic teller's machines (ATMs) of financial institutions , point of sales (POS) in stores, or internet of things devices (e.g. light bulbs, sensors, electric or gas meters, sprinklers, smoke alarms, thermostats, street lights, toasters) , Exercise equipment, hot water tank, heater, boiler, etc.) may include at least one.

According to some embodiments, the electronic device may be a piece of furniture or a building/structure, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, Water, electricity, gas, radio wave measuring devices, etc.) may include at least one. In various embodiments, the electronic device may be one or a combination of more than one of the various devices described above. An electronic device according to some embodiments may be a flexible electronic device. In addition, the electronic device according to the embodiment of this document is not limited to the above-described devices, and may include new electronic devices according to technological development.

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (eg, an artificial intelligence electronic device).

1 is a diagram illustrating an example of operation of an electronic device according to an embodiment.

Referring to FIG. 1 , the electronic device 100 executes an application (eg, a note application) supporting user handwriting input based on a designated input interface in response to user control, and outputs an execution screen 133 of the application. can In one embodiment, the input interface is either a software input panel (SIP) keyboard 30 output to one area of the screen of the electronic device 100 or a stylus pen 40 mounted on the electronic device 100. may contain at least one. In this regard, the execution screen 133 of the application includes at least one menu tab 134 and/or 136 for specifying an input interface supporting the user handwriting input (or setting a user handwriting input method). ) may be provided. For example, when a user input (eg, a touch) is applied to the first menu tab 134, the electronic device 100 can output the SIP keyboard 30, and the user presses the SIP keyboard 30. You can use text input. Alternatively, when a user input (eg, touch) is applied to the second menu tab 136, the electronic device 100 can detect the signal of the stylus pen 40, and the user moves the stylus pen 40. Handwriting input can be performed. According to the foregoing, user handwriting input can be understood as encompassing text input or marking input based on the SIP keyboard 30 or the stylus pen 40 (eg, non-grammatical asterisk, underline, wave or check). In the following, the user handwriting input may be referred to as a content input.

In one embodiment, the execution screen 133 of the application may include a first interface 10 for operating a recording function installed in the electronic device 100 . The electronic device 100 records the voice generated in the operation environment of the electronic device 100 (or the voice received by the electronic device 100) in response to the user control of the first interface 10, You can save or output the stored recording data. Also, the execution screen 133 of the application may include a second interface 20 for writing arbitrary information. The electronic device 100 may receive a user's content input related to the information entry and display it in at least one area of the second interface 20 . According to an embodiment, the second interface 20 may be expanded in a vertical or horizontal scroll form according to the amount, size, or location of input content.

In an embodiment, the electronic device 100 records a voice input received in response to a user control on the first interface 10 and simultaneously receives a user's content input and displays it on the second interface 20. can For example, the electronic device 100 may perform recording or screen display by receiving a voice input and a content input related to the same event (eg, a lecture). In this operation, the electronic device 100 may map and store the recorded data and the input content data based on the same time. For example, the electronic device 100 may map data recorded at the same time and content data input to each other and store them in a database or in the form of an index.

In an embodiment, the electronic device 100 responds to a user input applied to the first interface 10 or the second interface 20, and second data mapped to first data requested or selected from the user input. can be presented visually or audibly. For example, when data recorded at a first point in time is requested from a user control of the first interface 10, the electronic device 100 responds to the request for recorded data at the first point in time and records the database or index at the same time. The content data mapped to the recording data of the first view (or input at the first view) may be visually specified with reference to . Similarly, when the content input at the second point of view is selected from the user control of the second interface 20, the electronic device 100 responds to the content selection of the second point of view and simultaneously responds to the content of the second point of view. Recorded data mapped to data (or recorded at the second point of view) may be provided visually or audibly. Hereinafter, various embodiments of providing information in response to user input based on mapping of data (eg, recording data and content data) generated or input at the same time, and an electronic device 100 implementing the same Let's take a look at the functional operation of the components.

2 is a diagram illustrating a configuration of an electronic device according to an exemplary embodiment.

Referring to FIG. 2 , the electronic device 100 may include at least one of a communication circuit 110, an input/output device 120, a memory 130, a display 140, or a processor 150. According to various embodiments, the electronic device 100 may omit at least one of the above-described components or may additionally include other components. For example, the electronic device 100 supports a power supply device (eg, battery) for supplying power to the components, and capturing an image (eg, still image or video) of an area around the electronic device 100. A camera device (eg, a front camera and/or a rear camera) or a sensor device (eg, a proximity sensor, an illuminance sensor, an acceleration sensor, a fingerprint recognition sensor, or an iris recognition sensor) that senses various information about the operating environment of the electronic device 100 etc.) and the like may be further included. Alternatively, the electronic device 100 may further include components of the electronic device 901 to be referred to through FIG. 9 described later. It may be understood that at least some of the functional operations of components of the electronic device 100 described below are implemented under the control of the processor 150 based on at least one command stored in the memory 130 .

The communication circuit 110 may support communication between the electronic device 100 and at least one external device 200 . For example, the communication circuit 110 establishes a network 300 according to a prescribed protocol with the at least one external device 200, and accesses the network 300 based on wired or wireless communication. By doing so, signals or data may be transmitted to the at least one external device 300 or signals or data may be received from the at least one external device 300 . According to various embodiments, the at least one external device 300 may include a device of the same type as the electronic device 100 (eg, a mobile device) or a device of a different type (eg, a server).

The input/output device 120 may receive or support the aforementioned voice input or content input. For example, the input/output device 120 includes at least one microphone, SIP keyboard (FIG. 1 30), a keyboard, a mouse, a touch pad or a stylus pen (40 in FIG. 1 ), etc., and signals or data according to the voice input or content input may be received or transmitted to the electronic device 100. . Alternatively, the input/output device 120 may include at least one speaker mounted in one area of the electronic device 100, and a voice generated by the electronic device 100 or received from at least one external device 200. Signals or data may be output to the outside of the electronic device 100 . In one embodiment, at least a portion (eg, at least one microphone) of the input/output device 120 is controlled to be always active (eg, always on), or a user inputs to the above-described first interface (10 in FIG. 1 ). When (eg, touch) is applied, it may be triggered and activated. In one embodiment, the input/output device 120 may include a recording module for recording voice signals or data input through the microphone.

The memory 130 may store at least one signal or data related to the operation of the electronic device 100 or store at least one command related to the functional operation of components of the electronic device 100 . For example, the memory 130 may store voice data, contents data, or recording data for the voice input based on the input/output device 120 . In one embodiment, in relation to the storage of the recording data or content data, the memory 130 may store the recording data and content data of a specific time point mapped under the control of the processor 150 in the form of a database or index. there is. In this operation, the processor 150 may include information of the specific viewpoint in a database or index as identification information for the mapped recording data and content data of the specific viewpoint. In one embodiment, the memory 130 is mounted in a preloaded form when the electronic device 100 is manufactured, or is downloaded in the form of a third party from an online market (eg, an app store). At least one application 131 may be stored. According to an embodiment, the at least one application 131 controls the functional operation (eg, recording of input voice data) of the input/output device 120 (eg, a recording module, etc.) or writes user information (eg, a recording module). : content input).

The display 140 may output various screens. For example, the display 140 outputs an operation screen of the electronic device 100 (eg, a home screen or an execution screen of the application 131 (133 in FIG. 1)) in response to user control or designated scheduling information. can Alternatively, the display 140 may display text or markings (eg, non-grammatical asterisks, underlines, waves, checks, etc.) according to user input (eg, content input) on the operating screen of the electronic device 100. . In various embodiments, the display 140 may be implemented as a touch screen display including a display panel, a cover glass, and a touch panel (or touch sensor). The display panel may output a related screen based on a designated driving signal. In this regard, the display 140 may include a display driving circuit that transmits a driving signal to the display panel. The display driving circuit may be electrically connected to the display panel and transmit a driving signal corresponding to content information received from the processor 150 to the display panel at a specified frame rate. The cover glass may be disposed above the display panel to transmit light according to a screen output of the display panel. In addition, a user input (eg, touch, drag, press, hover, etc.) by a user's body (eg, a finger) or a stylus pen ( 40 in FIG. 1 ) may be applied to at least one area of the cover glass. The touch panel detects a signal (eg, electrostatic detection, pressure-sensitive detection, infrared detection, ultrasonic detection, etc.) according to a user input (eg, touch, drag, press, hovering, etc.) and outputs an electrical signal. Information on electrical signals may be transferred to a touch controller (eg, touch IC) or processor 150 .

The processor 150 is implemented as at least one of a central processing unit, an application processor, or a communication processor, and is a component (eg, a communication circuit) of the electronic device 100 described above. 110), input/output device 120, memory 130 or display 140, etc.) can be controlled. For example, the processor 150 may be electrically or functionally connected to at least one component, transmit at least one instruction related to a functional operation to the component, or perform various calculations or data processing. In one embodiment, the processor 150 processes a plurality of data (eg, recording data according to voice input recording and content data according to content input) generated or input based on the execution of a designated application (eg, note application). It can be mapped based on a specified criterion (eg, creation or input time). In one embodiment, the processor 150 may respond to a user input applied on the execution screen or setting screen of the application with reference to the mapped data. For example, in response to a user input requesting or selecting specific data, the processor 150 may visually specify or audibly provide data mapped to the specific data while processing the request or selection. there is. Various functional operations of the processor 150 in this regard will be described through referenced drawings below.

3A, 3B, and 3C are diagrams illustrating responses to various user inputs of an electronic device according to an embodiment, and FIG. 3D is a graphic effect display form corresponding to recording and playback of an electronic device according to an embodiment. It is a drawing showing

3a, 3b, 3c, or 3d, the processor (150 in FIG. 2) of the electronic device 100 records data and contents of a specific event generated or input through execution of a designated application (eg, note application). In the state in which the data is mapped, the execution screen 135 of the application displaying the playback of the recorded data and the content data can be output according to the user's control of the application. The execution screen 135 of the application referred to in the following drawings may be output in the same type as described above, and redundant description thereof may be omitted.

Referring to FIG. 3A , the processor 150 of the electronic device 100 provides a first interface 10 that supports output (or playback) of recorded data recorded for a predetermined time range and input from a user during the predetermined time range. An execution screen 135 of an application including the second interface 20 displaying selected content (eg, text or marking (ungrammatical asterisk, underline, wave, check, etc.)) may be output. In one embodiment, the processor 150 may receive a first user input (eg, a touch) for selecting specific content from among at least one piece of content displayed on the second interface 20 . In this case, the processor 150 identifies at least one stroke to which the first user input has been applied, and unit, phrase, or morpheme content (eg, a plurality of strokes) including the identified at least one stroke. Designated graphic effects (e.g. designated color display or blinking display, etc.) can be given to the composed text (Creation of Heaven and Earth). Alternatively, the processor 150 may apply the designated graphic effect to content to which the first user input is applied (eg, marking consisting of a single stroke).

In an embodiment, the processor 150, in an operation of giving the specified graphic effect to the content according to the first user input, refers to a database or index in the memory (130 in FIG. 2 ) to the content to which the graphic effect is applied. It is possible to identify recording data mapped with. For example, the processor 150 may identify recorded data recorded at the same point of time as the first stroke input point of the content to which the graphic effect is applied. In one embodiment, the processor 150 may specify a reproduction point of the identified recorded data as an area of the first interface 10 . In this regard, the first interface 10 may include a status bar indicating a playback status of the recorded data recorded for the predetermined time period, and the processor 150 may display the status bar of the recorded data identified on the status bar. The movement bar 11 may be positioned at a playback point (eg, 00:02:47). According to an embodiment, the first interface 10 may include at least one object for controlling playback or playback stop of the recorded data recorded for the predetermined time period. When a user input (eg, touch) is applied to the playback object in a state where the movement bar 11 is located at the identified playback point (eg, 00:02:47) of the recorded data, the processor 150 generates an input/output device. (120 in FIG. 2) (eg, a speaker) may be controlled to output a voice starting from the identified recorded data.

Referring to FIG. 3B , the processor 150 may receive a second user input (eg, drag) for controlling the position of the movement bar 11 included in the first interface 10 . In this case, the processor 150 refers to the database or index frame of the memory 130 and records the recorded data corresponding to the point where the movement bar 11 is located (eg, 00:02:47) according to the second user input. can identify. The processor 150 identifies content data mapped to the identified recorded data (or input at the same time as the identified recorded data was recorded), and displays at least one of the identified recorded data displayed on the second interface 20. Among the contents, the designated graphic effect may be applied to the contents 21 corresponding to the identified contents data. When a user input (eg, touch) is applied to the playback object included in the first interface 10, the processor 150 corresponds to the playback point (eg, 00:02:47) where the movement bar 11 is located. It is possible to output audio starting from the recorded data.

Referring to FIG. 3C , in an area around the status bar included in the first interface 10, a thumbnail object 13 related to content data mapped to recording data of a specific playback point may be displayed. . In one embodiment, when a third user input (eg, touch) is applied to any one of the one or more thumbnail objects 13, the processor 150 displays the thumbnail object to which the third user input has been applied on the status bar. The movement bar 11 may be positioned at a playback point (eg, 00:57:13) corresponding to the position of (13). In addition, the processor 150 records a reproduction point (eg, 00:57:13) corresponding to the thumbnail object 13 to which the third user input is applied (or corresponding to the location of the selected thumbnail object 13). content data (mapped to data) may be identified, and the specified graphic effect may be given to content 22 corresponding to the identified content data among at least one content displayed on the second interface 20 . When a user input (eg, touch) is applied to the playback object included in the first interface 10, the processor 150 corresponds to the playback point (eg, 00:57:13) where the movement bar 11 is located. It is possible to output audio starting from the recorded data.

Referring to FIG. 3D , the processor 150 is included in the first interface 10 according to the aforementioned first user input (eg, a touch to select a specific content from among at least one content displayed on the second interface 20). A playback object on the first interface 10 in a state where the moved bar 11 is moved to a playback point (eg, 00:02:47) of recorded data mapped to content data related to the first user input A user input (eg, touch) applied to (15) may be received. In this case, the processor 150 may output audio starting from the recorded data corresponding to the reproduction point (eg, 00:02:47) where the movement bar 11 is located. In this operation, the processor 150 may refer to a database or index on the memory 130 to identify recorded data and mapped content data corresponding to a point at which the movement bar 11 moves according to the lapse of the voice output. there is. For example, when the movement bar 11 moves from the first playback point (00:02:47) to the second playback point (00:02:49) according to the passage of the audio output, the processor 150, Recording data corresponding to the second playback point (00:02:49) and mapped content data may be identified. The processor 150 may give a specified graphic effect to the first content 23a corresponding to the identified content data among at least one piece of content displayed on the second interface 20 . After giving the graphic effect to the first content 23a, the processor 150 moves the position of the moving bar 11 according to the progress of the audio output similarly to the above, at the third playback point (00:02:51). Designated graphic effects may be sequentially applied to the second content 23b corresponding to and the third content 23c corresponding to the fourth playback point (00:02:53). According to various embodiments, the configuration for sequentially applying graphic effects to the contents 23a, 23b, and 23c according to the passage of the above-described audio output is the playback point where the moving bar 11 is located in FIG. 3B or 3C. The same or similar can be applied to audio output starting from the recorded data of .

4 and 5 are diagrams illustrating a response form of an electronic device to a fourth user input according to various embodiments, and FIG. 6 illustrates a response form of an electronic device to a fifth user input according to an embodiment. it is a drawing

Referring to FIG. 4 , the processor ( 150 of FIG. 2 ) of the electronic device 100 receives a fourth user input (eg, creation of the universe) for selecting specific content (eg, the creation of the world) from among at least one piece of content displayed on the second interface 20 . : double touch) can be received. In this case, the processor 150, similar to a response to the above-described first user input (eg, a touch to select a specific content from among at least one content displayed on the second interface 20), the first interface 10 The movement bar 11 included in may be positioned as a reproduction point (eg, 00:02:47) of recorded data mapped to content data related to the fourth user input. In addition, the processor 150 refers to the database or index on the memory (130 in FIG. 2) and records the recorded data mapped to the content data related to the fourth user input (eg, the playback point corresponding to 00:02:47). Recording data (eg, recorded data corresponding to the playback point 00:02:27) recorded before a designated time (eg, 20 seconds) from the recorded data) may be identified. The processor 150 may convert the identified recorded data into text form and output the third interface 50 including the converted text 51 . In one embodiment, the processor 150 responds to a user input (eg, touch) applied to one area of the third interface 50, and outputs voice based on the recorded data before being converted into the text 51. can output

Referring to FIG. 5 , the processor 150 includes a first interface 100 for operating a recording function installed in the electronic device 100 and a second interface (not shown) for writing arbitrary information (eg, FIG. 1 ). An execution screen 133 of an application (eg, a note application) including 20 of) may be output. In one embodiment, the processor 150 uses the third menu tab 137 (eg, a more menu tab) provided as one area of the execution screen 133 to select a designated file (eg, a PDF file, an e-mail file, or may receive user input to import a message file, etc.). In this case, the processor 150 may output the fourth interface 60 according to the execution of the file to an upper layer of the second interface (not shown) included in the execution screen 133 . According to an embodiment, the fourth interface may include at least one piece of content (eg, text or image) according to information included in the file.

In one embodiment, the processor 150 records a voice input received through a microphone in response to a user control of the first interface 10, and simultaneously receives a user's content input to generate the fourth interface 60. can be displayed on the According to an embodiment, in the embodiment described with reference to FIG. 5 , the user's content input may be understood as a marking input including non-grammatical asterisks, underlines, waves, checks, and the like. The processor 150 may store the recorded recorded data and the input content data in a database or index form by mapping them based on the same point in time.

In one embodiment, the processor 150 receives a fourth user input (eg, double touch) for selecting specific marking content (eg, wave marking) from among at least one marking content displayed on the fourth interface 60. can In response to receiving the fourth user input, the processor 150 records recorded data mapped with marking content data related to the fourth user input with reference to the database or index (e.g., recorded at 00:11:07). Recording data (eg, recorded data recorded at 00:10:47) recorded before a designated time (eg, 20 seconds) from the recorded data) may be identified. The processor 150 may convert the identified recorded data into text form and output the fifth interface 70 including the converted text 71 . In one embodiment, the processor 150 performs the processing in response to a user input (eg, touch) applied to the playback object 15 or a user input (eg, touch) applied to one area of the fifth interface 70 . It is possible to output audio starting from the recorded data before being converted into text 71 .

Referring to FIG. 6 , the processor 150 may receive a fifth user input (eg, drag) for controlling the position of the movement bar 11 included in the first interface 10 . In one embodiment, the fifth user input may be understood as an input for returning the movement bar 11 from the current position to a specific playback point (eg, 00:57:13) of a previous time. In response to the fifth user input, the processor 150 maps (or records) the recorded data corresponding to the specific playback point (eg, 00:57:13) with reference to the database or index of the memory 130. It is possible to identify content data (entered at the same time as the time when the data was recorded). The processor 150 may give a specified graphic effect to content 22 corresponding to the identified content data among at least one piece of content displayed on the second interface 20 .

In one embodiment, the processor 150 receives a user input controlling the new recording from the specific playback point (eg, 00:57:13) (eg, a specified response to a playback object included in the first interface 10). long presses of more than an hour, etc.) can be received. For example, the processor 150 controls overwrite recording of the voice input received by the electronic device 100, starting from the specific playback point (eg, 00:57:13). User input can be received. In addition, the processor 150 may newly record the voice input and simultaneously receive and display the new content input from the user on the second interface 20 . In this case, the processor 150 refers to the database or index of the memory 130, and the first time point corresponding to the specific playback point (eg, 00:57:13) and the second time point receiving the new content input. At least one piece of content data input at a third viewpoint corresponding to between viewpoints may be identified. The processor 150 applies a graphic effect (eg, blur, etc.) assigned to the content 25 corresponding to the at least one content data input at the third point in time among the at least one content displayed on the second interface 20 . can be granted. In other words, the processor 150 is limited to the content 25 displayed at the third point of view, excluding the content 24 displayed before the first point of view and the content 26 displayed after the second point of view. A specified graphic effect can be given. In addition, the processor 150 may cancel the mapping between content data input at the third viewpoint and recorded data recorded at the third viewpoint on the database or index.

7A and 7B are diagrams illustrating responses to various other user inputs of an electronic device according to an embodiment, and FIG. 7C is specific information related to processing of a seventh user input of an electronic device according to an embodiment. It is a drawing showing a registration form.

Referring to FIG. 7A , the execution screen of the aforementioned application (eg, note application) may include a setting screen 138 of the application. In one embodiment, the user of the electronic device 100 searches for some of the recorded data (or content data mapped to the part of the recorded data) of at least one previously recorded recorded data through the setting screen 138 can be performed. In this regard, the setting screen 138 may support registration of a search word for searching the part of the recorded data. According to one embodiment, the search term is content (e.g., text in the form of words, phrases or morphemes, or non-grammatical markings (asterisks, underlines, waves, or checks) corresponding to content data mapped with the part of the recorded data to be searched. etc.)) can be understood. For example, the user may register at least one search word by applying an input (eg, touch) to the fourth menu tab 90 provided as one area of the setting screen 138 . The processor 150 displays at least one search word 80 registered by the user in a list form on one area of the setting screen 138, and the user selects the recorded data (or , Content data mapped to recorded data) may select a search term (eg, important). In this case, the processor 150 refers to a database or index on the memory (130 in FIG. 2 ) and maps content data including the search term (eg, important) (or inputs content data including the search term). At least one recording data (recorded at a time point) may be identified or collected. The processor 150 converts the identified or collected at least one recording data into text data, and displays the converted at least one text data 81 in a list form by switching to another setting screen 138a. can

Referring to FIG. 7B , when a user input (eg, a touch) is applied to any one of the at least one text data 81 displayed in the form of a list, the processor 150 refers to the database or index to send the corresponding text data. Content data mapped to recorded data before being converted into data may be identified. The processor 150 converts the other setting screen 138a to the execution screen 135 of the application, and among the at least one content displayed on the execution screen 135, the content 25 corresponding to the identified content data can be focused. In addition, the processor 150 moves the movement bar 11 in the first interface 10 included in the execution screen 135 to a reproduction point of the identified content data and mapped recorded data (eg, 00:57:13). ) can be placed. In one embodiment, when a user input (eg, touch) is applied to the playback object in the first interface 10, the processor 150 records the recording data corresponding to the playback point (eg, 00:57:13). It is possible to output the audio as the starting point.

Referring to FIG. 7C , the processor 150 supports registration of non-grammatical markings (eg, asterisks, underlines, waves, checks, etc.) as the search word in one area of the setting screen 138 of the application ( 95) can be provided. In one embodiment, when a user input (eg, a touch) is applied to the fifth menu tab 95, the processor 150 may switch the setting screen 138 to another setting screen 138a. . The another setting screen 138a may include, for example, a sixth interface 97 through which content input corresponding to the marking may be authorized based on a stylus pen ( 40 in FIG. 1 ). In one embodiment, a search word according to the content input applied to the sixth interface 97 may be displayed in a list form in one area of the application setting screen 138, similar to the above.

An electronic device according to various embodiments described above may include at least one microphone, a touch screen display, at least one processor electrically connected to the at least one microphone and the touch screen display, and electrically connected to the at least one processor. may contain memory.

According to various embodiments of the present disclosure, the memory may include a first interface for supporting recording of voice input through the at least one processor through the touch screen display and the at least one microphone when executed; and Outputs an execution screen of a designated application including a second interface supporting user input of content, and based on voice data at a first time recorded in the memory based on the first interface and the second interface mapping and storing the input content data of the first view, and receiving a user input for searching for at least one voice data mapped to the specified content data based on a search word for the specified content; In response to a user input, at least one voice data mapped to the designated content data is collected, the collected at least one voice data is converted into text data, and the converted text data is output through the touch screen display. You can store instructions (instructions) to do.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays a status bar indicating a playback state of voice data recorded in one region of the first interface or the recording on the status bar. An instruction for displaying at least one of moving bars indicating a reproduction point of the reproduced voice data may be further stored.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays at least one content according to the user's content input to one area of the second interface, and among the at least one content An instruction for applying a specified graphic effect to the first content in response to a specified first user input for selecting the first content may be further stored.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor, in response to the designated first user input, displays the movement bar on the status bar with first content data related to the first content. An instruction for locating the mapped reproduction point of the first voice data may be further stored.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor outputs a voice starting from a reproduction point of the first voice data, and records at least one recorded voice data after the first voice data. an instruction for identifying second content data mapped to each of the second audio data of and imparting the specified graphic effect to at least one second content related to the second content data in sequence corresponding to the audio output progress; can store more.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays at least one content according to the user's content input in one area of the second interface and determines the location of the movement bar. Identifying third content data mapped to third voice data corresponding to the position of the movement bar in response to a second user input designated for control, and third content related to the third content data among the at least one content. An instruction to give a specified graphic effect to may be further stored.

According to various embodiments of the present disclosure, the memory may, when executed, cause the at least one processor to display at least one thumbnail representing content data mapped to the recorded voice data in an area around the status bar. You can store more instructions to do.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays at least one content according to the user's content input in one area of the second interface, and among the at least one thumbnail In response to a designated third user input for selecting a first thumbnail, an instruction for applying a designated graphic effect to fourth content related to the first thumbnail among the at least one content may be further stored.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays at least one content according to the user's content input to one area of the second interface, and among the at least one content In response to a specified fourth user input for selecting the fifth content, an instruction for identifying fourth audio data recorded before a specified time from recorded data mapped to fourth content data related to the fifth content may be further stored. there is.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor converts the fourth voice data into text data, and includes the converted text data in one area of the execution screen of the application. 3 You can store more instructions to output the interface.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays a first object that controls reproduction of voice data recorded in one region of the first interface, and displays the first object in the first object. An instruction for outputting a voice starting from a reproduction point of the fourth voice data in response to a designated fifth user input may be further stored.

According to various embodiments of the present disclosure, the memory may include a software input panel (SIP) keyboard output through the display or a stylus pen mounted as one area of the electronic device by the at least one processor during execution. Receives the user's content input including at least one of text or non-grammatical marking input using at least one of the above, and displays at least one content according to the content input in one area of the second interface. You can store more instructions to do this.

According to various embodiments of the present disclosure, the memory corresponds to a designated sixth user input that, when executed, causes the at least one processor to return the movement bar to a playback point of a previous point of view to control a new recording from the previous point of view. Thus, an instruction for mapping and storing the newly recorded fifth voice data and the fifth content data input at the same point in time as the fifth voice data may be further stored.

According to various embodiments of the present disclosure, when the memory is executed, the at least one processor displays at least one content according to the user's content input in one area of the second interface, and displays the previous time and the first content. 5 instructions for identifying at least one sixth content data input at points in time between points at which content data is input, and giving a specified graphic effect to sixth content corresponding to the sixth content data among the at least one content data; can store more.

8 is a diagram illustrating a method of providing information mapped between a plurality of inputs of an electronic device according to an embodiment. The operations described with reference to FIG. 8 may be implemented under the control of a processor ( 150 of FIG. 2 ) based on at least one instruction that may be stored in a memory ( 130 of FIG. 2 ).

In operation 801, the processor 150 of the electronic device (100 in FIG. 2) may execute a designated application (eg, note application) in response to user control and output a first execution screen of the application. In one embodiment, the first execution screen may include at least one of a first interface for operating a recording function installed in the electronic device 100 and a second interface for supporting a user's content input. .

In operation 803, the processor 150 may map and store audio data recorded at a first time point and content data input at the first time point through the first execution screen. In this regard, the processor 150 may record a voice input received by the electronic device 100 in response to user control of the first interface, and simultaneously receive and display a user's content input on the second interface. there is. In this operation, the processor 150 may map the recorded voice data and the input content data based on the same time and store them in a database or index form.

In operation 805, the processor 150 may receive a user input for searching for specific voice data among at least one piece of recorded voice data. In this regard, the processor 150 outputs (or switches from the first execution screen to the second execution screen) a second execution screen (eg, a setting screen of the application) of the application in response to user control, and the 2 At least one search word registered by the user may be displayed as one area of the execution screen. In one embodiment, the search word is content corresponding to content data mapped with voice data to be searched for (eg, text in the form of words, phrases or morphemes, or non-grammatical markings (asterisks, underlines, waves, checks, etc.)). can be understood The user may perform a user input for selecting a search word corresponding to voice data to be searched from among the at least one search word displayed above.

In operation 807, the processor 150 may collect voice data corresponding to the user input. For example, the processor 150 may refer to the database or index to identify content data including a search word selected from the user input, and collect at least one voice data mapped to the identified content data.

In operation 809, the processor 150 may output at least one piece of collected voice data in text form. In this regard, the processor 150 may convert the collected at least one voice data into text data. In one embodiment, the processor 150 converts the second execution screen of the application into a third execution screen (eg, another setting screen of the application), and stores the converted text data in one area of the third execution screen. It can be output in list form.

According to an embodiment, when any one of the at least one text data displayed in the list form is selected by the user, the processor 150 refers to the database or index and converts the voice data before being converted into corresponding text data. Mapped content data can be identified. The processor 150 may switch the third execution screen to a first execution screen of an application, and focus on content corresponding to the identified content data among at least one content displayed on the first execution screen.

A method for providing information mapped between a plurality of inputs of an electronic device according to various embodiments described above includes a first interface supporting recording of voice input to the electronic device and a user's input of content. Outputting a first execution screen of a designated application including a second interface, voice data of a first time recorded based on the first interface and the first time input based on the second interface An operation of mapping and storing content data of , an operation of receiving a user input for searching for at least one voice data mapped to the specified content data based on a search word for the specified content, and mapping to the specified content data. The method may include an operation of collecting at least one piece of voice data collected, an operation of converting the collected at least one piece of voice data into text data, and an operation of outputting the converted text data.

According to various embodiments of the present disclosure, the information providing method may include an operation of outputting a second execution screen of the designated application supporting registration of the search word, and an operation of providing a menu related to the registration of the search word to one area of the second execution screen. may further include.

According to various embodiments, the operation of providing a menu related to the registration of the search word may include an operation of displaying at least one search word registered according to user control through the menu as one area of the second execution screen. .

According to various embodiments, the operation of receiving the user input may include an operation of receiving a user input for selecting one of at least one search word displayed in one area of the second execution screen.

According to various embodiments of the present disclosure, the operation of outputting the converted text data corresponds to an operation of receiving a user input for selecting any one of the at least one output text data and a user input for selecting the one or more text data. and converting the second execution screen into the first execution screen.

According to various embodiments of the present disclosure, the operation of switching to the first execution screen may include displaying at least one piece of content corresponding to content data input from a user in one area of the second interface included in the first execution screen. and focusing the content corresponding to the selected text data among the at least one content.

9 is a diagram illustrating an electronic device in a network environment according to an embodiment.

Referring to FIG. 9 , in a network environment 900, an electronic device 901 communicates with an electronic device 902 through a first network 998 (eg, short-range wireless communication) or through a second network 999 ( For example, it may communicate with the electronic device 904 or the server 908 through long-distance wireless communication. According to an embodiment, the electronic device 901 may communicate with the electronic device 904 through the server 908 . According to an embodiment, the electronic device 901 includes a processor 920, a memory 930, an input device 950, an audio output device 955, a display device 960, an audio module 970, a sensor module ( 976), interface 977, haptic module 979, camera module 980, power management module 988, battery 989, communication module 990, subscriber identification module 996, and antenna module 997 ) may be included. In some embodiments, in the electronic device 901, at least one of these components (eg, the display device 960 or the camera module 980) may be omitted or another component may be added. In some embodiments, some components are included, such as, for example, in the case of sensor module 976 (eg, a fingerprint sensor, an iris sensor, or an ambient light sensor) embedded in display device 960 (eg, display). can be integrated and implemented.

The processor 920, for example, drives software (eg, the program 940) to operate at least one other component (eg, hardware or software component) of the electronic device 901 connected to the processor 920. It can control and perform various data processing and calculations. The processor 920 loads and processes commands or data received from other components (e.g., the sensor module 976 or the communication module 990) into the volatile memory 932, and transfers the resulting data to the non-volatile memory 934. can be stored in According to one embodiment, the processor 920 is operated independently of the main processor 921 (eg, a central processing unit or application processor), and additionally or alternatively, uses less power than the main processor 921, Alternatively, an auxiliary processor 923 (eg, a graphic processing unit, an image signal processor, a sensor hub processor, or a communication processor) specialized for a designated function may be included. Here, the auxiliary processor 923 may be operated separately from or embedded with the main processor 921 .

In this case, the secondary processor 923 may, for example, take the place of the main processor 921 while the main processor 921 is in an inactive (eg sleep) state, or when the main processor 921 is active (eg sleep). : Application execution) together with the main processor 921, at least one of the components of the electronic device 901 (eg, the display device 960, the sensor module 976, or a communication module ( 990)) may control at least some of the related functions or states. According to one embodiment, the auxiliary processor 923 (eg, an image signal processor or a communication processor) is implemented as a component of other functionally related components (eg, the camera module 980 or the communication module 990). It can be. The memory 930 includes various data used by at least one component (eg, the processor 920 or the sensor module 976) of the electronic device 901, such as software (eg, the program 940). ) and input data or output data for commands related thereto. The memory 930 may include volatile memory 932 or non-volatile memory 934 .

The program 940 is software stored in the memory 930 and may include, for example, an operating system 942 , middleware 944 , or an application 946 .

The input device 950 is a device for receiving a command or data to be used in a component (eg, the processor 920) of the electronic device 901 from an outside (eg, user) of the electronic device 901, for example, For example, it may include a microphone, mouse, or keyboard.

The sound output device 955 is a device for outputting sound signals to the outside of the electronic device 901, and includes, for example, a speaker used for general purposes such as multimedia playback or recording playback and a receiver used exclusively for receiving calls. can include According to one embodiment, the receiver may be formed integrally with or separately from the speaker.

The display device 960 is a device for visually providing information to the user of the electronic device 901, and may include, for example, a display, a hologram device, or a projector and a control circuit for controlling the device. According to an embodiment, the display device 960 may include a touch circuitry or a pressure sensor capable of measuring the intensity of a touch pressure.

The audio module 970 may convert a sound and an electrical signal in both directions. According to an embodiment, the audio module 970 acquires sound through the input device 950, the audio output device 955, or an external electronic device connected to the electronic device 901 by wire or wirelessly (eg: Sound may be output through the electronic device 902 (eg, a speaker or a headphone).

The sensor module 976 may generate an electrical signal or data value corresponding to an internal operating state (eg, power or temperature) of the electronic device 901 or an external environmental state. The sensor module 976 may include, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a bio sensor, a temperature sensor, a humidity sensor, Alternatively, an illuminance sensor may be included.

The interface 977 may support a designated protocol capable of connecting with an external electronic device (eg, the electronic device 902) in a wired or wireless manner. According to one embodiment, the interface 977 may include a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 978 is a connector capable of physically connecting the electronic device 901 and an external electronic device (eg, the electronic device 902), such as an HDMI connector, a USB connector, an SD card connector, or an audio connector. (e.g. headphone connector).

The haptic module 979 may convert electrical signals into mechanical stimuli (eg, vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic senses. The haptic module 979 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 980 may capture still images and moving images. According to one embodiment, the camera module 980 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 988 is a module for managing power supplied to the electronic device 901 and may be configured as at least a part of a power management integrated circuit (PMIC).

The battery 989 is a device for supplying power to at least one component of the electronic device 901, and may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

The communication module 990 establishes a wired or wireless communication channel between the electronic device 901 and an external electronic device (eg, the electronic device 902, the electronic device 904, or the server 908), and establishes the established communication channel. It can support communication through. The communication module 990 may include one or more communication processors that support wired communication or wireless communication that are independently operated from the processor 920 (eg, an application processor). According to one embodiment, the communication module 990 is a wireless communication module 992 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 994 (eg, : A LAN (local area network) communication module or a power line communication module) is included, and the first network 998 (eg, Bluetooth, WiFi direct, or IrDA (infrared data association)) is used by using a corresponding communication module. communication network) or a second network 999 (eg, a cellular network, the Internet, or a telecommunications network such as a computer network (eg, LAN or WAN)) to communicate with the external electronic device. The various types of communication modules 990 described above may be implemented as a single chip or may be implemented as separate chips.

According to an embodiment, the wireless communication module 992 may distinguish and authenticate the electronic device 901 within a communication network using user information stored in the subscriber identification module 996.

The antenna module 997 may include one or more antennas for transmitting or receiving signals or power to the outside. According to an embodiment, the communication module 990 (eg, the wireless communication module 992) may transmit a signal to an external electronic device or receive a signal from the external electronic device through an antenna suitable for a communication method.

Some of the above components are connected to each other through a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input/output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and signal (e.g. commands or data) can be exchanged with each other.

According to an embodiment, commands or data may be transmitted or received between the electronic device 901 and the external electronic device 904 through the server 908 connected to the second network 999 . Each of the electronic devices 902 and 904 may be the same as or different from the electronic device 901 . According to an embodiment, all or part of operations executed in the electronic device 901 may be executed in one or a plurality of external electronic devices. According to an embodiment, when the electronic device 901 needs to perform a certain function or service automatically or upon request, the electronic device 901 instead of or in addition to executing the function or service by itself, At least some related functions may be requested from an external electronic device. The external electronic device receiving the request may execute the requested function or additional function and deliver the result to the electronic device 901 . The electronic device 901 may provide the requested function or service by directly or additionally processing the received result. To this end, for example, cloud computing, distributed computing, or client-server computing technology may be used.

Electronic devices according to various embodiments disclosed in this document may be devices of various types. The electronic device may include, for example, at least one of a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, and a home appliance. An electronic device according to an embodiment of this document is not limited to the aforementioned devices.

Various embodiments of this document and terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like elements. Singular expressions may include plural expressions unless the context clearly dictates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B or C" or "at least one of A, B and/or C" refer to all of the items listed together. Possible combinations may be included. Expressions such as "first", "second", "first" or "second" may modify the elements in any order or importance, and are used only to distinguish one element from another. The components are not limited. When a (e.g., first) element is referred to as being "(functionally or communicatively) coupled to" or "connected to" another (e.g., second) element, that element refers to the other (e.g., second) element. It may be directly connected to the component or connected through another component (eg, a third component).

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be an integral part or a minimum unit or part thereof that performs one or more functions. For example, the module may be composed of an application-specific integrated circuit (ASIC).

Various embodiments of this document include instructions stored in a machine-readable storage media (eg, internal memory 936 or external memory 938) that can be read by a machine (eg, computer). It may be implemented in software (eg, program 940). A device is a device capable of calling a stored command from a storage medium and operating according to the called command, and may include an electronic device (eg, the electronic device 901) according to the disclosed embodiments. When the command is executed by a processor (eg, the processor 920), the processor may perform a function corresponding to the command directly or by using other components under the control of the processor. An instruction may include code generated or executed by a compiler or interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' only means that the storage medium does not contain a signal and is tangible, but does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

According to one embodiment, the method according to various embodiments disclosed in this document may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. A computer program product may be distributed in the form of a device-readable storage medium (eg, compact disc read only memory (CD-ROM)) or online through an application store (eg, Play Store ^TM ). In the case of online distribution, at least part of the computer program product may be temporarily stored or temporarily created in a storage medium such as a manufacturer's server, an application store server, or a relay server's memory.

Each component (eg, module or program) according to various embodiments may be composed of a single object or a plurality of entities, and some of the sub-components may be omitted, or other sub-components may be various. It may be further included in the embodiment. Alternatively or additionally, some components (eg, modules or programs) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. According to various embodiments, operations performed by modules, programs, or other components may be executed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be executed in a different order, may be omitted, or other operations may be added. can

The embodiments disclosed in this document are presented for explanation and understanding of technical contents, and do not limit the scope of the present invention. Therefore, the scope of this document should be construed to include all changes or various other embodiments based on the technical spirit of the present invention.

Claims (20)

In electronic devices,
at least one microphone;
touch screen display;
at least one processor electrically connected to the at least one microphone and the touch screen display; and
A memory electrically connected to the at least one processor;
The memory, when executed, the at least one processor,
Through the touch screen display, outputting an execution screen of a designated application including a first interface supporting recording of voice input through the at least one microphone and a second interface supporting content input by a user, ,
mapping and storing, in the memory, voice data of a first time recorded based on the first interface and content data of the first time input based on the second interface;
Receiving a first user input for searching for first voice data mapped to the specified content data based on a search word for the specified content;
Selecting first audio data mapped to the designated content data in response to the first user input;
Converting the first voice data into text data;
Outputting the converted text data through the touch screen display;
outputting a moving bar indicating a time point corresponding to a playback time point of the first voice data on a status bar;
Receiving a second user input for overwrite recording from a second viewpoint through the movement bar;
and storing instructions for overwriting second voice data corresponding to the second time point with a new voice received using the at least one microphone in response to the second user input. According to claim 1,
The memory, when executed, the at least one processor,
Further storing an instruction for displaying the status bar indicating a playback state of the recorded audio data in one area of the first interface and the movement bar indicating a playback point of the recorded audio data on the status bar. electronic device. According to claim 2,
The memory, when executed, the at least one processor,
At least one content according to the user's content input is displayed in one area of the second interface, and a graphic effect assigned to the first content corresponds to a third user input for selecting the first content from among the at least one content. An electronic device further storing instructions for granting. According to claim 3,
The memory, when executed, the at least one processor,
Corresponding to the third user input, further storing an instruction for locating the movement bar on the status bar at a reproduction point of the first audio data mapped with first content data related to the first content. Device. According to claim 4,
The memory, when executed, the at least one processor,
After outputting the audio starting from the reproduction point of the first audio data, identifying second content data mapped to the second audio data recorded at a later point in time to the first audio data, and The electronic device further stores instructions for imparting the designated graphic effect to texts in a sequence corresponding to the audio output progress of the second audio data. delete According to claim 2,
The memory, when executed, the at least one processor,
The electronic device further stores an instruction for displaying at least one thumbnail representing content data mapped with the recorded voice data in a peripheral area of the status bar. According to claim 7,
The memory, when executed, the at least one processor,
At least one content according to the user's content input is displayed in one area of the second interface, and the first one of the at least one content is displayed in response to a third user input for selecting a first thumbnail from among the at least one thumbnail. 1 The electronic device further stores an instruction for imparting a specified graphic effect to fourth content related to the thumbnail. According to claim 1,
The memory, when executed, the at least one processor,
At least one content according to the user's content input is displayed in one area of the second interface, and recording mapped with the fifth content in response to a fourth user input selecting a fifth content from among the at least one content. The electronic device further stores instructions for identifying fourth audio data recorded before a specified time from the data. According to claim 9,
The memory, when executed, the at least one processor,
The electronic device further stores an instruction for converting the fourth voice data into text data and outputting a third interface including the converted text data to a region of an execution screen of the application. According to claim 10,
The memory, when executed, the at least one processor,
Displaying a first object that controls playback of recorded audio data in one area of the first interface, and starting from a playback point of the fourth audio data in response to a fifth user input applied to the first object. An electronic device that further stores an instruction for outputting a voice. According to claim 1,
The memory, when executed, the at least one processor,
Includes at least one of text or non-grammatical marking input using at least one of a software input panel (SIP) keyboard output through the display or a stylus pen mounted on one area of the electronic device The electronic device further stores an instruction for receiving the user's input of content, and displaying at least one content according to the content input in one area of the second interface. According to claim 2,
The memory, when executed, the at least one processor,
The electronic device further stores an instruction for receiving a new content input through the second interface, mapping the new voice input and the new content input, and storing the new content input as the second viewpoint while receiving the new voice. According to claim 13,
The memory, when executed, the at least one processor,
The electronic device further stores an instruction for imparting a specified graphic effect to the new content corresponding to the input of the new content. A method for providing information mapped between a plurality of inputs of an electronic device,
outputting a first execution screen of a designated application including a first interface supporting recording of voice input to the electronic device and a second interface supporting content input by a user;
mapping and storing voice data of a first time recorded based on the first interface and content data of the first time input based on the second interface;
receiving a first user input for searching for at least one voice data mapped to the specified content data based on a search word for the specified content;
selecting first audio data mapped to the designated content data;
converting the first voice data into text data;
outputting the converted text data;
outputting, on a status bar, a moving bar indicating a point in time corresponding to a point in time of reproduction of the first voice data;
receiving a second user input for overwrite recording from a second viewpoint through the moving bar; and
and overwriting second voice data corresponding to the second point in time with a new voice received using the at least one microphone in response to the second user input. According to claim 15,
outputting a second execution screen of the designated application supporting the registration of the search word; and
An operation of providing a menu related to the search word registration to one area of the second execution screen; further comprising, the information providing method. According to claim 16,
The operation of providing a menu related to the search term registration,
and displaying at least one search word registered according to user control through the menu as one area of the second execution screen. According to claim 17,
The operation of receiving the first user input,
and receiving a user input for selecting one of at least one search word displayed in one area of the second execution screen. According to claim 18,
The operation of outputting the converted text data,
receiving a third user input selecting one of the at least one output text data; and
and converting the second execution screen to the first execution screen in response to a third user input for selecting the one of text data. According to claim 19,
The operation of switching to the first execution screen,
displaying at least one piece of content corresponding to content data input from a user in one area of the second interface included in the first execution screen; and
and an operation of focusing the content corresponding to the selected text data among the at least one content.

Images