KR102245291B1 - Electronic device and method for using captured image in electronic device - Google Patents

Abstract

Various embodiments related to an electronic device and a method of using a capture screen in an electronic device have been described. According to an embodiment, in an electronic device, a display unit that displays an application screen, and when the application execution screen is captured, a captured screen image It may include a control unit that acquires execution path information capable of executing the state of and stores the captured screen image and the execution path information. In addition, various other embodiments are possible.

Description

Electronic devices and how to use captured images in electronic devices {ELECTRONIC DEVICE AND METHOD FOR USING CAPTURED IMAGE IN ELECTRONIC DEVICE}

Various embodiments of the present disclosure relate to an electronic device and a technology for using an image captured by the electronic device.

Recently, electronic devices are often being developed as portable terminals so that individuals can carry them, and functions of electronic devices are diversifying.

Accordingly, various functions for user convenience are built into the electronic device, and the electronic device may activate each function according to a corresponding user operation for each of various functions to execute the corresponding function.

For example, if the electronic device has a built-in capture function, the electronic device may capture various screens displayed on the electronic device by using the capture function and store the captured image.

However, in the conventional screen capture function, a captured image is simply stored as an image, and a method of using the captured image is not various. For example, there is a limitation in that the captured image is simply used for image processing, such as being used for copying, pasting, or using an object in the captured image.

Accordingly, according to various embodiments of the present disclosure, an electronic device capable of using a captured image in various ways and a method of using the captured image in the electronic device may be provided.

In addition, according to various embodiments of the present invention, when the captured screen image is saved, execution path information for executing the screen at the time of screen capture is stored together, and the screen at the time of screen capture is executed using the captured screen image at a later time. A capable electronic device and a method of using the captured image in the electronic device may be provided.

In addition, according to various embodiments of the present disclosure, a screen image captured from a user's electronic device and execution path information capable of executing the screen at the time of screen capture are shared with another electronic device of the same user or an electronic device of another user, Another electronic device or an electronic device of another user may also provide an electronic device capable of executing a screen at the time of screen capture using the captured screen image, and a method of using the captured image in the electronic device.

According to an embodiment of the present invention, in an electronic device, a display unit for displaying an application screen, when the application execution screen is captured, acquires execution path information for executing a state of the captured screen image, and includes the captured screen image and It may include a control unit for storing the execution path information.

In addition, according to an embodiment of the present invention, in a method of using a capture screen in an electronic device, a process of capturing an application execution screen, a process of acquiring execution path information for executing a state of the captured screen image, and the capture It may include a process of storing the generated screen image and the execution path information.

In addition, according to an embodiment of the present invention, in a storage medium storing a program using a capture screen, the program is an execution capable of executing a process of capturing an application execution screen and a state of the captured screen image by a control unit of an electronic device. A process of acquiring path information, a process of storing the captured screen image and the execution path information, and transmitting the captured screen image and the execution path information to share the captured screen image and the execution path information It may include a command that causes the process to be performed.

In addition, according to an embodiment of the present invention, the data sharing server includes a storage unit for storing data including a captured image uploaded from a first electronic device and execution path information corresponding to the captured image, wherein the data is When uploaded, a control unit for controlling the transmission of the stored data to a second electronic device accessible through the same account as the first electronic device may be included.

In addition, according to an embodiment of the present invention, the data sharing server includes a storage unit for storing data including a captured image and execution path information corresponding to the captured image, uploaded from a first electronic device, and the data Is uploaded, transmits the URL of the uploaded data to the first electronic device, and receives a data request using the URL of the uploaded data from a second electronic device that has received the URL of the data from the first electronic device, It may include a control unit that controls to transmit the uploaded data to the second electronic device.

According to various embodiments of the present disclosure, when a user captures a screen while viewing a screen according to an application execution, the entire image or at least a portion of the captured screen and execution path information capable of executing the state of the captured screen are stored together. , There is an effect that the user can execute the state of the captured screen by using the captured screen.

For example, if you select all or at least some images of the saved web page screen, the status of the corresponding web page screen can be executed. Therefore, the user launches the web browser to execute the status of the corresponding web page screen, and It can be convenient because user input operations such as entering a web page address can be omitted. In addition, if all or at least some images of the stored music playback screen are selected, the status of the music playback screen can be executed. Therefore, the user runs the music playback application to execute the status of the music playback screen, and It can be convenient because user input operations such as selecting music to be played can be omitted. In addition, if all or at least some of the saved clip screen images are selected, the state of the corresponding clip screen can be executed. Therefore, the user runs the clip application to display the clip screen and selects the clip screen to be displayed in the clip application. It can be convenient because it is possible to omit the user input operation such as.

In addition, according to various embodiments of the present disclosure, a screen image captured from a user's electronic device and execution path information capable of executing the screen at the time of screen capture are shared with another electronic device of the same user or an electronic device of another user, Another electronic device of the user or an electronic device of another user may also use the captured screen image to execute the screen at the time of screen capture.

1 is a block diagram of an electronic device according to an embodiment of the present invention
2 is a flowchart illustrating an operation of using a captured image by an electronic device according to an embodiment of the present invention
3 is a diagram for explaining an application framework and an application according to an embodiment of the present invention
4 is a flowchart showing an operation of acquiring execution path information through an application framework according to an embodiment of the present invention
5 is a view showing an example of information required to execute a state of a captured screen among information that may be included in a class object according to an embodiment of the present invention
6 is a flowchart illustrating an operation of acquiring execution path information through an application according to an embodiment of the present invention
4 is a flowchart showing an operation of acquiring execution path information through an application framework according to an embodiment of the present invention
5 is a diagram showing an example of information required to execute a state of a captured screen among information that may be included in a class object according to an embodiment of the present invention
6 is a flowchart illustrating an operation of acquiring execution path information through an application according to an embodiment of the present invention
7 is a view for explaining an example of an operation of selecting at least a portion of a captured screen according to an embodiment of the present invention
8A is a diagram illustrating a concept of capturing an application screen and acquiring execution path information of the captured screen in an electronic device according to an embodiment of the present invention
8B is a diagram illustrating a concept of operation using execution path information of a screen captured by an electronic device according to an embodiment of the present invention;
9 is a diagram illustrating an example of using a captured image while executing a web browser application in an electronic device according to an embodiment of the present invention
10 is a diagram illustrating an example of using a captured image while executing a music playback application in an electronic device according to an embodiment of the present invention
11 is a diagram illustrating an example of using a captured image while executing a clip application in an electronic device according to an embodiment of the present invention
12 is a diagram illustrating an operation of sharing an image captured by a device according to an embodiment of the present invention
13 is a diagram showing a captured image and an execution path information sharing system corresponding to the captured image according to the first embodiment of the present invention
14 is a diagram showing an operation of a captured image and an execution path information sharing system corresponding to the captured image according to the first embodiment of the present invention
15 is a diagram showing a captured image and an execution path information sharing system corresponding to the captured image according to a second embodiment of the present invention
16 is a diagram showing an operation of a captured image and an execution path information sharing system corresponding to the captured image according to the second embodiment of the present invention
17 is a diagram showing a concept of sharing a captured image and execution path information corresponding to the captured image according to a third embodiment of the present invention
18 is a diagram illustrating an operation of sharing a captured image and execution path information corresponding to the captured image between electronic devices according to a third embodiment of the present invention
19 to 22 are diagrams showing a user interface screen for sharing an image captured by an electronic device and execution path information corresponding to the captured image according to an embodiment of the present invention
23 is a diagram showing an SCC file structure according to an embodiment of the present invention
24 is a diagram showing META-INF included in an SCC file according to an embodiment of the present invention
25 is a diagram showing meta data according to an embodiment of the present invention

Terms used in various embodiments of the present invention will be briefly described, and various embodiments of the present invention will be described in detail.

Terms used in various embodiments of the present invention have been selected from general terms that are currently widely used as possible while considering functions in various embodiments of the present invention, but this is the intention or precedent of a technician working in the field, and the emergence of new technologies. It can be different depending on etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of various embodiments of the corresponding invention. Accordingly, terms used in various embodiments of the present invention should be defined based on the meaning of the term and the overall contents of the various embodiments of the present invention, not a simple name of the term.

In various embodiments of the present invention, when a part "includes" a certain component, it means that other components may be further included rather than excluding other components unless specifically stated to the contrary. In addition, terms such as "... unit" and "module" described in various embodiments of the present invention mean a unit that processes at least one function or operation, which is implemented as hardware or software, or is a combination of hardware and software. Can be implemented.

Hereinafter, various embodiments according to the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited or limited by various embodiments. The same reference numerals shown in each of the drawings may indicate members that perform substantially the same function.

Expressions such as "include" or "may include" that may be used among various embodiments of the present disclosure indicate the existence of the disclosed corresponding function, operation, or component, and include one or more additional functions, operations, or components. Do not limit. In addition, in various embodiments of the present disclosure, terms such as "include" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, It is to be understood that it does not preclude the possibility of the presence or addition of one or more other features or numbers, steps, actions, components, parts, or combinations thereof.

In various embodiments of the present disclosure, expressions such as "or" include any and all combinations of words listed together. For example, "A or B" may include A, may include B, or may include both A and B.

Among various embodiments of the present disclosure, expressions such as "first", "second", "first," or "second," may modify various elements of the present disclosure, but the corresponding elements are not limited. For example, the expressions do not limit the order and/or importance of corresponding elements. The above expressions may be used to distinguish one component from another component. For example, a first user device and a second user device are both user devices and represent different user devices. For example, without departing from the scope of the rights of various embodiments of the present disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. It should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in various embodiments of the present disclosure are only used to describe specific embodiments, and are not intended to limit the various embodiments of the present disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise.

Unless otherwise defined, all terms used herein including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and ideal or excessively formal unless explicitly defined in various embodiments of the present disclosure. It is not interpreted in meaning.

An electronic device (hereinafter also referred to as an “electronic device”) according to various embodiments of the present disclosure may be a device including a communication function. For example, electronic devices include smart phones, tablet personal computers, mobile phones, video phones, e-book readers, desktop personal computers, and laptops. Laptop personal computer (PC), netbook computer, personal digital assistant (PDA), portable multimedia player (PMP), MP3 player, mobile medical device, camera, or wearable device (e.g.: It may include at least one of a head mounted device (HMD) such as electronic glasses, an electronic clothing, an electronic bracelet, an electronic necklace, an electronic appcessory, an electronic tattoo, or a smart watch.

According to some embodiments, the electronic device may be a smart home appliance having a broadcast channel reception function. Smart home appliances, for example, include televisions, digital video disk (DVD) players, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwave ovens, washing machines, air purifiers, set top boxes, and TV boxes. (Eg, Samsung HomeSyncTM, Apple TVTM, or Google TVTM), game consoles, electronic dictionaries, electronic keys, camcorders, or electronic frames.

According to some embodiments, the electronic device includes various medical devices including a broadcast channel reception function (eg, magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT)), an imager, an ultrasound device, etc.), and a navigation device ( navigation) devices, global positioning system receivers, event data recorders (EDRs), flight data recorders (FDRs), automobile infotainment devices, marine electronic equipment (e.g. marine navigation devices and gyro compasses, etc.), aviation It may include at least one of electronic devices (avionics), security devices, or industrial or domestic robots.

According to some embodiments, the electronic device is a piece of furniture or building/structure including a broadcast channel reception function, an electronic board, an electronic signature receiving device, a projector, Alternatively, it may include at least one of various measuring devices (eg, water, electricity, gas, or radio wave measuring devices). An electronic device according to various embodiments of the present disclosure may be a combination of one or more of the aforementioned various devices. In addition, it is obvious to those skilled in the art that the electronic device according to various embodiments of the present disclosure is not limited to the above-described devices.

Hereinafter, electronic devices according to various embodiments will be described with reference to the accompanying drawings. The term user used in various embodiments may refer to a person using an electronic device or a device (eg, an artificial intelligence electronic device) using an electronic device.

1 is a block diagram of an electronic device according to an embodiment of the present invention. Referring to FIG. 1, the electronic device 100 may include a control unit 110, a communication unit 120, a storage unit 130, a touch screen 140, and an input/output unit 150.

The control unit 110 may be a central processor unit (CPU). The control unit 110 may include a digital signal processor (DSP), a ROM (not shown) in which a control program for controlling the electronic device 100 is stored, and a signal input from the outside of the electronic device 100 Alternatively, it may include a RAM (not shown) used as a storage area for storing data or for a task performed in the electronic device 100. The CPU may include single core, dual core, triple core, or quad core. The CPU, ROM and RAM can be interconnected via an internal bus.

The controller 110 may control the communication unit 120, the storage unit 130, the touch screen 140, and the input/output unit 150. That is, the controller 110 performs an overall control function for the electronic device 100 and may control a signal flow between the respective constituent units.

For example, the controller 110 may control to display an application screen according to execution of an application on the touch screen 140 according to an embodiment of the present invention. The controller 110 may perform a screen capture of an application screen displayed on the touch screen 140. For example, the controller 110 may continuously display application screens according to application execution while the application is being executed on the touch screen 140, and may capture an application screen requested to be captured from among the application screens. In addition, the controller 110 may obtain execution path information corresponding to the captured screen, and execution path information corresponding to the captured screen may include information capable of executing a state of the captured screen.

For example, the controller 110 may include an application framework and an application, execute the application using the execution path information obtained through the application framework or the application, and display screens according to the application execution, Execution path information corresponding to the captured application screen may be obtained. In addition, the controller 110 may store execution path information corresponding to the captured screen together with the entire captured screen or a partial image selected from the captured screen. When the stored image is selected, the controller 110 may detect execution path information corresponding to the selected image, and execute a process corresponding to the selected image using the execution path information.

The communication unit 120 may allow the electronic device 100 to be connected to an external device (not shown). The “external device” may include other devices (not shown), mobile phones (not shown), smart phones (not shown), tablet PCs (not shown), and servers (not shown). In addition, the communication unit 120 may allow the electronic device 100 to be connected to an external device through mobile communication using at least one or a plurality of antennas (not shown) under the control of the controller 110. The communication unit 120 includes a mobile phone (not shown), a smart phone (not shown), a tablet PC or other device (not shown) having a phone number input to the electronic device 100, and makes a voice call, a video call, and a text message. It is possible to transmit/receive wireless signals for messages (SMS) or multimedia messages (MMS). The communication unit 120 may further include at least one of a wireless LAN module (not shown) and a short-range communication module (not shown). The wireless LAN module may be connected to the Internet at a place where a wireless access point (AP) (not shown) is installed under the control of the controller 110. The wireless LAN module may support the wireless LAN standard (IEEE802.11x) of the American Institute of Electrical and Electronic Engineers (IEEE). The short-range communication module may wirelessly perform short-range communication between the electronic device 100 and the image forming apparatus (not shown) under the control of the controller 110. The short-range communication method may include Bluetooth, infrared communication (IrDA, infrared data association), and the like.

The storage unit 130 may store input/output signals or data corresponding to operations of the communication unit 120, the touch screen 140, and the input/output unit 150 under the control of the controller 110. The storage unit 130 may store a plurality of programs for controlling the electronic device 100 or the controller 110, a plurality of applications, and programs and data for executing various functions. In addition, the storage unit 150 may store a captured screen according to an exemplary embodiment of the present invention, an image of at least a portion of the captured screen, and may store a program for using the captured image mentioned in the present invention.

The term storage unit refers to a ROM (not shown), a RAM (not shown) in the control unit 110, or a memory card (not shown) installed in the terminal 100 (e.g., SD card, memory). Stick). The storage unit 150 may include a nonvolatile memory, a volatile memory, a hard disk drive (HDD), or a solid state drive (SSD).

The storage unit 130 may be a medium that can be read by a machine (eg, a computer), and the term “medium that can be read by a machine” refers to a medium that provides data to the machine so that the machine can perform a specific function. Can be defined. The machine-readable medium may be a storage medium. The storage unit 130 may include non-volatile media and volatile media. All such media must be tangible so that the instructions conveyed by the medium can be detected by a physical mechanism that reads the instructions into the machine. The machine-readable medium includes, but is not limited to, a floppy disk, a flexible disk, a hard disk, a magnetic tape, a compact disc read only memory (CD ROM), an optical disk, and a punch. Include at least one of a punch card, paper tape, RAM, Programmable Read Only Memory (PROM), Erasable PROM (EPROM), and FLASH EPROM. I can.

The input/output unit 150 may include a button 152, a microphone 154, a speaker 156, and an input unit 158. Further, the input/output unit 150 is not limited thereto, and a cursor control such as a mouse, a trackball, a joystick, or cursor direction keys provides communication with the controller 110 and controls the movement of the cursor on the touch screen 140. Can be provided for.

The microphone 154 may generate an electrical signal by receiving voices or various sounds from various sound sources under the control of the controller 110.

The speaker 156 transmits sounds corresponding to various signals (eg, wireless signals, broadcasting signals, digital audio files, digital video files, etc.) by the communication unit 120 or application execution under the control of the controller 110 to an electronic device ( 100) Can be output externally.

The input unit 158 may be a device or object inserted and stored in the electronic device 100 or configured separately, and may be withdrawn or detached from the electronic device 100 when inserted and stored inside the electronic device 100. The input unit 158 may be an input means such as a stylus pen (hereinafter referred to as a “pen”) or the like, which can be input to the touch screen 140, such as proximity or a touch, or may be a user's finger.

The touch screen 140 may be a “display unit” and may provide a graphic user interface to a user when executing various applications. For example, the touch screen 140 may receive various gesture inputs such as at least one proximity or touch through a user's body (eg, a finger including a thumb) or a touchable input unit 158, for example, a pen. The touch screen 140 may be implemented in, for example, a resistive method, a capacitive method, an infrared method, or an acoustic wave method. The touch screen 140 as described above may display screens according to application execution and captured screens under the control of the controller 110 according to an embodiment of the present invention, and may display at least a portion of the captured screen. have.

2 is a flowchart illustrating an operation of using a captured image by the electronic device 100 according to an embodiment of the present invention. Referring to FIG. 2, the electronic device 100 may capture an application screen in step 210.

For example, the electronic device 100 may capture an application screen currently being displayed in response to a screen capture request from a user while an application is being executed.

Specifically, the electronic device 100 captures the currently displayed web page screen according to a screen capture request from the user while the web browsing application is running, or plays the currently displayed music according to the screen capture request from the user while the music playback application is running. You can capture the screen. In addition, while the clip application is running, the currently displayed clip screen may be captured in response to a screen capture request from the user.

The electronic device 100 may acquire execution path information corresponding to the screen captured in step 220. The execution path information may include information capable of executing the state of the captured screen.

For example, the electronic device 100 may acquire execution path information corresponding to the captured application screen by using the application framework and the application.

3 is a diagram illustrating an application framework and an application according to an embodiment of the present invention.

Referring to FIG. 3, the application framework 310 may be a group of classes and libraries implementing the standard structure of an application program. The application framework 310 may include a number of common class objects that can be commonly used in various applications and an inner class object that can be used in a specific application. The application framework 310 may execute the application 320 and display screens according to the execution of the application on the touch screen 140. The application framework 310 may be an Android operating system-based framework or an ios operating system-based framework. It may be a framework based on an electronic device operating system not mentioned in the embodiment of the present invention.

Meanwhile, according to an embodiment of the present invention, the device 100 acquires execution path information corresponding to the application screen captured through the application framework 310 or corresponds to the application screen captured through the application 320. Execution path information can be obtained.

4 is a flowchart illustrating an operation of acquiring execution path information through the application framework 310 according to an embodiment of the present invention.

Referring to FIG. 4, the application framework 310 may determine whether an application screen is captured in step 410.

When the application screen is captured while the application is running, the application framework 310 may determine a class object corresponding to the captured screen among a number of class objects in step 420.

The application framework 310 may obtain execution path information by using information included in a class object corresponding to the screen captured in step 430.

For example, the application framework 310 may obtain execution path information by extracting information necessary to execute a state of a captured screen from among information included in a class object.

5 is a diagram illustrating an example of information required to execute a state of a captured screen among information that may be included in a class object according to an embodiment of the present invention.

Referring to FIG. 5, among information that can be included in a class object, information necessary to execute a state of a captured screen is URI, String, Boolean, byte (binary), Short, Integer, Float, Double, Date, Time, It can include information such as Class Object. In addition, any information is possible as long as it is related information that can execute the captured screen.

The URI may be URI information corresponding to a screen when the captured screen is running. String may be string information when the captured screen is running. For example, the String may include text information including a phone number name, a phone number application name, and the like when the phone number is being displayed. The Boolean may include information on the value of the determination variable (true or false) of the application. Byte (binary) information may be binary data information. The binary data information may be information obtained by binarizing information when a captured screen, such as an image file or text, is being executed. Short is a short-length byte variable information, and may include a small range of integer information when the captured screen is running. Integer is numeric information from 0 to 255, and can include numeric information when the captured screen is running. Float is decimal number information and may include decimal point number information when the captured screen was running. Double is decimal number information with a large number of digits, and can include decimal number information when the captured screen is running. Date is date information and may include date information when the captured screen was running. Time is time information and may include time information when the captured screen is running. Class Objcet is inner class object information and may include inner class object information when the captured screen is running. In addition, it is obvious that the class object can contain information that can execute any number of other captured screens.

The application framework 310 may obtain execution path information capable of executing the state of the captured screen by using the information included in the class object as described above.

Meanwhile, according to an embodiment, the electronic device 100 may obtain execution path information corresponding to an application screen captured through the application 320.

6 is a flowchart illustrating an operation of acquiring execution path information through an application 320 according to an embodiment of the present invention.

Referring to FIG. 6, the application 320 may determine whether an application screen is captured in step 610.

When the application screen is captured while the application is running, the application 320 may acquire URI information corresponding to the captured screen in step 620. The format of the URI information can be <ApplicationName>://<detailed information> (ex. smartclip://1.png).

The application 320 may transmit uniform resource identifier (URI) information obtained in response to the captured application screen in step 630 to the application framework 310 in charge of a capture function.

Referring back to FIG. 2, the electronic device 100 may select an image of at least a portion of the screen captured in step 230.

7 is a diagram illustrating an example of an operation of selecting at least a portion of a captured screen according to an embodiment of the present invention.

Referring to FIG. 7, the electronic device 100 may select the entire captured application screen image 710 as shown in FIG. 7A. In addition, the electronic device 100 may select an image 720 corresponding to a specific object from among the captured application screens as shown in FIG. 7B. In addition, the electronic device 100 may select an image in the closed curve drawn on the application screen captured by the input unit 158 such as an electronic pen or a user's finger among the captured application screens as shown in FIG. 7C. . In addition, the method of selecting at least a part of the captured screen is other methods such as an object selection method according to a drawing input (straight line, curve, etc.) that connects objects existing on the screen, and a method of directly selecting a specific object existing on the screen. Can be.

The electronic device 100 may store execution path information together with at least a partial image of the captured screen in step 240.

For example, the electronic device 100 stores execution path information along with the entire captured screen, stores execution path information together with objects in the captured screen, or by drawing input using an electronic pen or a finger on the captured screen. The execution path information can be saved together with the image of the part selected by the user.

In this case, the electronic device 100 may store the execution path information as a database (DB) or a file, or may include and store the execution path information in a metadata area in at least a portion of an image of the captured screen.

The stored image, for example, the entire image of the captured screen or at least a part of the image of the captured screen may be displayed on the captured image display screen according to a user request later. On the captured image display screen, the user can select the stored image.

When the image stored in step 250 is selected, the electronic device 100 may read execution path information stored together with the image selected in step 260.

For example, when the execution path information is stored as a DB (DataBase) or a file, the electronic device 100 may read a corresponding DB (DataBase) or a file (File). In addition, when the execution path information is stored in a metadata area within an image of at least a portion of the captured screen, the electronic device 100 may read the execution path information from the corresponding metadata area.

The electronic device 100 may execute the state of the captured screen using the read execution path information in step 260.

For example, when the state of the captured screen is a state of displaying the phone number detail information (contact detail view) screen while the phone book application is running, the electronic device 100 does not require a phone book application execution request to view detailed phone number information. The phone number detailed information screen can be displayed using the execution path information.

In addition, when the state of the captured screen is a state in which a specific web page screen is displayed while the web browser application is running, the electronic device 100 can provide execution path information without entering an execution request to the web browser and inputting an address for viewing the specific web page screen. By using, the specific web page screen can be displayed.

In addition, when the state of the captured screen is a state in which a specific music playback screen is displayed while the music playback application is running, the captured screen is captured using the execution path information without a request to execute a music playback application and a specific music playback request input. The music playback screen of the viewpoint can be displayed, and the music of the captured viewpoint can be played. In addition, according to various embodiments of the present invention, the state of the captured screen may be a state in which various applications are being executed, and the execution path information is used without an input for executing the various applications to go to the state of the captured screen. You can immediately execute the state of the captured screen.

According to an embodiment, the application 320 may be an application program that can be executed in the electronic device 100. The application 320 may be an application that performs various functions, such as a phone book application, a web browser application, a music playback application, a clip application, and a scrapbook application.

According to an embodiment, the application framework 310 may execute at least one application from among various applications, and screens according to execution of the application may be displayed on the touch screen 140. When the application execution screen is captured according to a user input during application execution, the application framework 310 acquires execution path information corresponding to at least some images of the captured application execution screen (hereinafter, also referred to as'captured image'), and Captured images and execution path information corresponding to the captured images can be saved. According to an embodiment, the captured image may be an entire image of the captured application screen, or a partial image cropped by a user's drawing input on the captured application screen.

8A is a diagram illustrating a concept of capturing an application screen and obtaining execution path information corresponding to a captured image in an electronic device according to an embodiment of the present invention. Referring to FIG. 8A, the application framework 810 may correspond to the framework layer 801, and various applications 821 to 824 may correspond to the application layer 802. According to an embodiment, various applications 821 to 824 may include a scrapbook application 821, a web browser application 822, a video service application 823, and a music playback application 824. In addition, other applications may be further included.

According to an embodiment, the application framework 810 may capture an application execution screen according to a user input 842, for example, a screen capture request input from a user, while the application execution screen is being displayed, and capture the captured image and the captured image. It is possible to obtain (844) the execution path information corresponding to the image.

According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, and a data file (music file, video file, or image) corresponding to the application execution screen. File), drawing memo information corresponding to the application execution screen, handwriting recognition result information corresponding to the application execution screen, clip location information corresponding to the application execution screen, and the like.

According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the application execution screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the application execution screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the application execution screen.

According to an embodiment, the application framework 810 may store the captured image and execution path information corresponding to the captured image, that is, metadata, in the DB 830 (or file system) in 846.

According to an embodiment, the DB 830 may be accessed by the scrapbook application 821. The scrapbook application 821 may access the DB 830 to load the captured image stored in the DB 830 and execution path information, such as metadata, corresponding to the captured image (844), and execute the captured application accordingly. You can restore the screen and make it run.

8B is a diagram illustrating a concept of restoring and executing an application execution screen captured by an electronic device according to an exemplary embodiment of the present invention.

Referring to FIG. 8B, the scrapbook application 821 may access the DB 830 to load a captured image stored in the DB 830 and execution path information, such as metadata, corresponding to the captured image (850). In addition, the scrapbook application 821 may make an execution request 854 according to execution path information. According to an embodiment, the scrapbook application 821 may request execution according to execution path information, for example, metadata.

The application framework 810 may restore and execute a corresponding application execution screen according to the execution path information (854). According to an embodiment, the application framework may restore and execute a corresponding application according to metadata among at least one or more applications.

9 is a diagram illustrating an example of using a captured image while executing a web browser application in an electronic device according to an embodiment of the present invention.

Referring to FIG. 9A, the electronic device 100 may capture a web page screen while displaying a web page screen 910 while a web browser application is running as shown in FIG. 9A.

Referring to FIG. 9B, the electronic device 100 may select all or at least some of the images on the captured web page screen 920. In (b) of FIG. 9, an example is shown that an image 930 in an oval drawn by a user is selected. On the web page screen, an image of a desired shape, such as a square, a circle, and a polygon, may be selected by the user. The electronic device 100 may store the selected image 930 and execution path information of the captured screen together.

Referring to FIG. 9C, the electronic device 100 may display an image 950 stored on the captured image display screen 940 and receive a selection of the stored image 950 from the user. In this case, the stored image 950 may be directly selected, or a button 960 requesting to execute a state of the stored image 950 may be selected.

Referring to (d) of FIG. 9, when the button 960 for requesting to execute the stored image 950 or the stored image 950 is selected, the electronic device 100 is shown in FIG. 9. As shown in (d), the state 970 of the captured screen corresponding to the stored image 950 may be executed.

10 is a diagram illustrating an example of using a captured image while executing a music playback application in an electronic device according to an embodiment of the present invention.

Referring to FIG. 10A, the electronic device 100 may capture a music playback screen while displaying the music playback screen 1010 while the music playback application is running as shown in FIG. 10A.

Referring to FIG. 10B, the electronic device 100 may select all or at least some of the images on the captured music playback screen 1020. In (b) of FIG. 10, an example of an image 1030 in a closed curve drawn by a user is selected. On the captured music playback screen, an image of a desired shape, such as a square, a circle, or a polygon, may be selected at a desired location by the user. The electronic device 100 may store the selected image 1030 and execution path information of the captured music playback screen together. For example, the selected image 1030 and the execution path information of the captured music playback screen may include URI information and music playback section information (play time) at the capture time point.

Referring to FIG. 10C, the electronic device 100 may display an image 1050 stored on the captured image display screen 1040 and may receive a selection of the stored image 1050 from the user. In this case, the stored image 1050 may be directly selected, or a button 1060 requesting to execute a state of the stored image 1050 may be selected.

Referring to (d) of FIG. 10, when a button 1060 for requesting to execute a state of the stored image 1050 or the stored image 1050 is selected, the electronic device 100 is shown in FIG. 10. As shown in (d), the state of the captured screen corresponding to the stored image 1050 may be executed. For example, music can be played at a music playback point corresponding to the captured screen.

11 is a diagram illustrating an example of using a captured image while executing a clip application in an electronic device according to an embodiment of the present invention.

Referring to FIG. 11A, the electronic device 100 may capture a clip screen while displaying a clip screen 1110 displaying clipped texts while the music clip application is running as shown in FIG. 11A. have.

Referring to FIG. 11B, the electronic device 100 may select all or at least some of the images on the captured clip screen 1120. According to an embodiment, the entire or at least part of the image may be cropped and the cropped image may be selected according to a user's drawing input on the captured clip screen 1120. In (b) of FIG. 11, an example of an image 1130 in a closed curve drawn by a user is selected. On the captured clip screen, an image of a desired shape, such as a rectangle, circle, or polygon, may be selected at a desired location by the user.

Referring to FIG. 11C, the electronic device 100 may store the selected image 1130 and execution path information of the captured clip screen together. The execution path information of the captured clip screen may include URI information such as “slookdemos://SmartClip”.

Referring to FIG. 11D, the electronic device 100 may display an image 1150 stored on the captured image display screen 1140 and may receive a selection of the stored image 1150 from the user. In this case, the stored image 1150 may be directly selected, or an icon 1160 requesting to execute a state of the stored image 1150 may be selected.

Referring to (e) of FIG. 11, when an icon 1160 requesting to execute a state of the stored image 1150 or the stored image 1150 is selected, the electronic device 100 is shown in FIG. 11. As shown in (e), the state of the captured original screen corresponding to the stored image 1150 may be executed. For example, a clip screen corresponding to the captured screen may be displayed.

According to various embodiments of the present invention as described above, when a user captures a screen while viewing a screen according to an application execution, at least a portion of the captured screen and execution path information capable of executing the state of the captured screen are stored together. By doing so, there is an effect that the user can execute the state of the captured screen by using the captured screen.

For example, if at least some images of the saved web page screen are selected, the status of the corresponding web page screen can be executed. Therefore, the user launches a web browser to execute the status of the web page screen, and It can be convenient because user input operations such as entering a page address can be omitted.

In addition, if at least some images of the stored multimedia content playback screen are selected, the state of the corresponding multimedia content playback screen can be executed, so that the user executes a multimedia content playback application to execute the state of the multimedia content playback screen, and It can be convenient because a user input operation such as selecting a multimedia content to be played back in a playback application can be omitted. Specifically, the multimedia content may include music and video, as well as various content that can be played on an electronic device. When the multimedia content is music, if at least some images of the multimedia playback screen such as stored music and video are selected, the status of the music playback screen can be executed, so the user can execute the music playback application to execute the status of the music playback screen. It may be convenient because it is possible to omit a user input operation such as selecting music to be played back in the music playback application and selecting the music to be played back.

In addition, if at least some images of the saved clip screen image are selected, the state of the clip screen can be executed, so the user executes the clip application to display the clip screen, selects the clip screen to be displayed in the clip application, etc. It can be convenient because the user input operation can be omitted.

Meanwhile, it is obvious that execution using a capture screen can be applied to various applications in addition to the above-described various embodiments.

According to various embodiments of the present disclosure, the electronic device 100 may share an image captured on an application execution screen and execution path information corresponding to the captured image.

12 is a diagram illustrating an operation of sharing an image captured on an application execution screen and execution path information corresponding to the captured image according to an embodiment of the present invention.

Referring to FIG. 12, in operation 1210, the electronic device 100 may determine whether there is a request for sharing information on an image captured from an application screen and execution path information from a user.

When there is a request to share the captured image and execution path information corresponding to the captured image, the electronic device 100 may generate data including the captured image and execution path information corresponding to the captured image in operation 1220. According to an embodiment, a data format including a captured image and execution path information corresponding to the captured image may be a scrapbook file (SCC) data format or a data format consisting of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

The electronic device 100 may transmit the captured image in operation 1230 to share data including execution path information corresponding to the captured image.

According to various embodiments of the present disclosure, the electronic device 100 may transmit data including a captured image and execution path information corresponding to the captured image to another electronic device of the same user or an electronic device of another user through a data sharing server. You can share. Accordingly, the electronic device 100 may enable another electronic device of the same user or an electronic device of another user to execute an operation at the time of capture by using the captured image.

According to an embodiment of the present invention, a data sharing server includes a storage unit for storing data including a captured image uploaded from a first electronic device and execution path information corresponding to the captured image, wherein the data is When uploaded, a control unit for controlling downloading the stored data to a second electronic device accessible through the same account as the first electronic device may be included.

13 is a diagram illustrating a captured image and an execution path information sharing system corresponding to the captured image according to the first embodiment of the present invention.

Referring to FIG. 13, a first electronic device A1 1310 transmits an image captured on an application execution screen and execution path information corresponding to the captured image through a first sharing server 1330 to a second electronic device A2 1320. You can share by synchronizing with. The first sharing server 1330 may be a server that provides synchronization between electronic devices of the same account, such as an i-cloude server, for example. The second electronic device A2 1320 may be an electronic device of the same user as the user of the first electronic device A1 1310, and the first sharing server 1330 using the same account as the first electronic device A1 1310 You can connect to.

14 is a diagram illustrating an operation of a captured image and an execution path information sharing system corresponding to the captured image according to the first embodiment of the present invention.

Referring to FIG. 14, in operation 1402, the first electronic device A1 1310 may store an image captured from an application execution screen and execution path information corresponding to the captured image as data in a predetermined format.

According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

According to an embodiment, the execution path information corresponding to the captured image may include information capable of executing a state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, and a data file (music file, video file, or image) corresponding to the application execution screen. File), drawing memo information corresponding to the application execution screen, handwriting recognition result information corresponding to the application execution screen, clip location information corresponding to the application execution screen, and the like.

According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the application execution screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the application execution screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the application execution screen.

The first electronic device A1 1310 may check a periodic data change in operation 1404. When data change is detected, the first electronic device A1 1310 may upload data to the first sharing server 1330 through the communication network 1301 in operation 1406.

The first sharing server 1330 may store data uploaded from the first electronic device A1 1310 in operation 1408.

The first sharing server 1330 may download the data to the second electronic device A2 1320 set to synchronize with the first electronic device A1 1310 in operation 1410 after storing the uploaded data.

According to an embodiment, the first sharing server 1330 may download data to the second electronic device A2 1320 in a push message method or a polling method.

In the case of the push message method, the first sharing server 1330 may notify that data is stored in the second electronic device A2 1320 using a push message, and the second electronic device A2 1320 may 1 It is possible to request data stored in the shared server 1330. The first sharing server 1330 may download data to the second electronic device A2 1320 according to the stored data request.

In the case of the polling method, the second electronic device A2 1320 may periodically request to inquire whether there is newly stored data in the first shared server 1330, and the first shared server 1330 may request the second electronic device A2 1320 When there is newly stored data according to an inquiry request from, a response message notifying that there is newly stored data may be sent, and then the newly stored data may be downloaded to the second electronic device A2 1320.

The second electronic device A2 1320 may be synchronized with the first electronic device A1 1310 and share the data by storing the data downloaded in operation 1412. According to an embodiment, the first electronic device A1 1310 captures the entire web page screen according to a user input on the web browser application execution screen, that is, the web page screen, or crops some images from the captured web page screen. At least some images of the screen of the web page that have been created may be acquired. The first electronic device A1 1310 transmits at least some images of the captured web page screen and execution path information corresponding to at least some images of the captured web page screen through the first sharing server 1330 to the second electronic device A2 ( 1320).

For example, the first electronic device A1 1310 may acquire at least some images of a web page according to an input from a user while displaying a web page screen. According to an embodiment, the first electronic device A1 1310 captures a web page screen using an application that performs a capture function according to a user input (for example, a SmartClip application) to obtain a captured web page screen image, or At least some images of the web page may be obtained by cropping some images from the captured web page screen. In the following description, the captured web page screen image or some images cropped from the captured web page screen may be described as a captured image.

The first electronic device A1 1310 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing the state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to a web page, a URL that can be restored to a web page, a text corresponding to a web page, a data file (music file, video file, or image file) corresponding to the web page, and the web It may include drawing memo information corresponding to the page, handwriting recognition result information corresponding to the web page, clip position information corresponding to the web page, and the like. According to an embodiment, the metadata may further include deep link information. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to a resource used in a web page. For example, it may be a direct link to a resource such as a video file, sound, or image related to a web page. In addition, the lip link information may be information capable of accessing a subpage in the main page of the web page.

The first electronic device A1 1310 may store the captured web page image and execution path information corresponding to the captured web page image as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

The first electronic device A1 1310 may periodically check a change in data of a DB related to a scrapbook application. When data change is detected, the first electronic device A1 1310 transmits data including the captured web page image and execution path information corresponding to the captured web page image to the first sharing server 1330 through the communication network 1301. Can be uploaded to. The first sharing server 1330 may store data uploaded from the first electronic device A1 1310.

The first sharing server 1330 stores a web page image captured in the second electronic device A2 1320 set to synchronize with the first electronic device A1 1310 after storing the uploaded data and an execution path corresponding to the captured web page image Data containing information can be downloaded.

According to an embodiment, the first sharing server 1330 is data including a web page image captured by the second electronic device A2 1320 in a push message method or a polling method and execution path information corresponding to the captured web page image. You can download it.

In the case of the push message method, the first sharing server 1330 uses the push message to generate data including a web page image captured by the second electronic device A2 1320 and execution path information corresponding to the captured web page image. The second electronic device A2 1320 requests data including the captured web page image and execution path information corresponding to the captured web page image to the first shared server 1330 according to the push message. I can. The first sharing server 1330 may download data including a web page image captured by the second electronic device A2 1320 and execution path information corresponding to the captured web page image according to a stored data request.

In the case of using the polling method, the second electronic device A2 1320 may periodically request to inquire whether there is newly stored data in the first shared server 1330, and the first shared server 1330 may request the second electronic device A2 1320 If there is newly saved data according to the inquiry request from ), a response message notifying that there is newly saved data is sent, and then data including the newly saved captured web page image and execution path information corresponding to the captured web page image are provided. 2 It can be downloaded to the electronic device A2 1320.

The second electronic device A2 1320 may be synchronized with the first electronic device A1 1310 by storing data including the downloaded captured web page image and execution path information corresponding to the captured web page image, The captured web page image and execution path information corresponding to the captured web page image can be shared. In addition, the second electronic device A2 1320 may display the captured web page image and execution path information corresponding to the captured web page image to the user, and use the execution path information corresponding to the captured web page image. It can be executed to display the web page screen.

In addition, according to an embodiment, the first electronic device A1 1310 captures a video playback service application execution screen according to a user input on a video playback service application execution screen, such as a YouTube screen, or executes the captured video playback service application. At least some images of the captured video playback service application execution screen may be obtained by cropping some images from the screen. The first electronic device A1 1310 transmits execution path information corresponding to at least some images of the captured video playback service application execution screen and at least some images of the captured video playback service application execution screen through the first sharing server 1330. It can be shared with the second electronic device A2 1320.

According to an embodiment, the first electronic device A1 1310 captures a video playback service screen using an application (for example, a SmartClip application) that performs a capture function while displaying a video playback service application execution screen, and provides a captured video playback service. A screen image may be acquired, or a partial image may be cropped from a captured video playback service screen image. Hereinafter, a captured video playback service screen image or a partial image cropped from a captured video playback service screen may be described as a captured image.

Meanwhile, the first electronic device A1 1310 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing the state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to a video playback service, a URL that can be restored to a video playback service, a text corresponding to a video playback service, and a data file (music file, video file, or image) corresponding to the video playback service. File), drawing memo information corresponding to the video playback service, handwriting recognition result information corresponding to the video playback service, clip location information corresponding to the video playback service, and the like. According to an embodiment, the metadata may further include deep link information. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the video playback service screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the video playback service screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the video playback service screen.

The first electronic device A1 1310 may store at least some images of the captured video playback service screen and execution path information corresponding to at least some images of the captured video playback service screen as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared. According to an embodiment, data in a predetermined format may be stored in a DB related to a scrapbook application.

The first electronic device A1 1310 may periodically check a change in data of a DB related to a scrapbook application. When data change is detected, the first electronic device A1 1310 transmits data including execution path information corresponding to at least some images of the captured video playback service screen and at least some images of the captured video playback service screen to the communication network 1301 ) Can be uploaded to the first sharing server 1330. The first sharing server 1330 may store data uploaded from the first electronic device A1 1310. The first sharing server 1330 stores at least some images of the video playback service screen captured on the second electronic device A2 1320 set to synchronize with the first electronic device A1 1310 after storing the uploaded data and the captured video playback service Data including execution path information corresponding to at least some images of the screen may be downloaded.

According to an embodiment, the first sharing server 1330 includes at least some images of a video playback service screen captured by the second electronic device A2 1320 in a push message method or a polling method, and at least some images of the captured video playback service screen. Data including execution path information corresponding to may be downloaded.

In the case of the push message method, the first sharing server 1330 may notify that data is stored in the second electronic device A2 1320 using the push message, and the second electronic device A2 1320 may 1 It is possible to request data stored in the shared server 1330. In response to the stored data request, the first sharing server 1330 provides at least some images of the video playback service screen captured by the second electronic device A2 1320 and execution path information corresponding to at least some images of the captured video playback service screen. You can download the data it contains.

In the case of using the polling method, the second electronic device A2 1320 may periodically request to inquire whether there is newly stored data in the first shared server 1330, and the first shared server 1330 may request the second electronic device A2 1320 If there is newly stored data according to the inquiry request from ), after sending a response message notifying that there is newly stored data, respond to at least some images of the newly saved captured video playback service screen and at least some images of the captured video playback service screen Data including the executed execution path information may be downloaded to the second electronic device A2 1320. The second electronic device A2 1320 may include at least some images of the downloaded captured video playback service screen and the captured video playback service. By storing data including execution path information corresponding to at least some images of the screen, it can be synchronized with the first electronic device A1 1310, and at least some images of the captured video playback service screen and the captured video playback service screen It is possible to share execution path information corresponding to at least some images of. In addition, the second electronic device A2 1320 may display execution path information corresponding to at least some images of the captured video playback service screen and at least some images of the captured video playback service screen to the user, and the captured video playback service A corresponding video playback service may be executed by using execution path information corresponding to at least some image areas of the screen.

In addition, according to an embodiment, the first electronic device A1 1310 may acquire at least an image of the captured music playback screen by capturing a music playback screen according to a user input from a music playback application execution screen, such as a music playback screen. According to one implementation, the entire image of the captured music playback screen may be acquired, or some images of the captured music playback screen may be acquired by cropping some images from the captured music playback screen. The first electronic device A1 1310 transmits execution path information corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen through the first sharing server 1330 to the second electronic device A2 ( 1320) and can share

According to an embodiment, the first electronic device A1 1310 may capture at least some images of a music playback screen using an application (eg, a SmartClip application) that performs a capture function according to a user input during music playback. In the following description, the entire image of the captured music playback screen or a partial image cropped from the captured music playback screen may be described as a captured image.

Meanwhile, the first electronic device A1 1310 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing a music reproduction state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to the music playback screen, a URL that can be restored to the state of the music playback screen, text corresponding to the music playback screen, and data files (music files, video files) corresponding to the music playback screen. Or image file), drawing memo information corresponding to the music playback screen, handwriting recognition result information corresponding to the music playback screen, clip location information corresponding to the music playback screen, and the like. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the music playback screen. For example, it may be a direct link to an audio file related to a music playback screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the music playback screen.

The first electronic device A1 1310 may store at least some images of the captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared. According to an embodiment, the first electronic device A1 1310 may further include an audio file corresponding to the music playback screen in data.

The first electronic device A1 1310 may periodically check a change in data of a DB related to a scrapbook application. When data change is detected, the first electronic device A1 1310 uploads at least some images of the captured music playback screen and at least some images of the captured music playback screen to the first sharing server 1330 through the communication network 1301 can do. According to an embodiment, the first electronic device A1 1310 may control data including an audio file in addition to at least some images of the captured music playback screen and at least some images of the captured music playback screen through the communication network 1301. 1 Can be uploaded to the shared server 1330. According to an embodiment, an audio file may be uploaded only when it is less than a predetermined data size. According to an embodiment, the predetermined data size may be 15 Mbyte.

The first sharing server 1330 may store data uploaded from the first electronic device A1 1310. The first sharing server 1330 stores at least some images of the music playback screen captured on the second electronic device A2 1320 set to synchronize with the first electronic device A1 1310 after storing the uploaded data and the captured music playback screen. Data including execution path information corresponding to at least some images or data including execution path information and audio files corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen may be downloaded. have.

According to an embodiment, the first sharing server 1330 corresponds to at least some images of the music playback screen captured by the second electronic device A2 1320 and at least some images of the captured music playback screen in a push message method or a polling method. Data including the executed execution path information or data including an audio file and execution path information corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen may be downloaded.

In the case of the push message method, the first sharing server 1330 may notify that data is stored in the second electronic device A2 1320 using a push message, and the second electronic device A2 1320 may 1 It is possible to request data from the shared server 1330. The first sharing server 1330 includes at least some images of the music playback screen captured by the second electronic device A2 1320 and execution path information corresponding to at least some images of the captured music playback screen according to the stored data request. Data or data including an audio file and execution path information corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen may be downloaded.

In the case of using the polling method, the second electronic device A2 1320 may periodically request to inquire whether there is newly stored data in the first shared server 1330, and the first shared server 1330 may request the second electronic device A2 1320 If there is newly saved data according to the inquiry request from ), a response message notifying that there is newly saved data is sent, and then execution corresponding to at least some images of the newly saved captured music playback screen and at least some images of the captured music playback screen Download data including path information or data including execution path information and audio files corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen to the second electronic device A2 1320 can do.

The second electronic device A2 1320 includes data including at least some images of the stored captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen, or at least some images of the captured music playback screen. Data including execution path information and audio files corresponding to at least some images of the captured music playback screen may be downloaded. The second electronic device A2 1320 includes data including at least some images of the downloaded captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen, or at least some images of the captured music playback screen And data including an audio file and execution path information corresponding to at least some images of the captured music playback screen, which may be synchronized with the first electronic device A1 1310, and at least some images of the captured music playback screen Share execution path information corresponding to at least some images of the captured music playback screen with, or share execution path information and audio files corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen. can do. In addition, the second electronic device A2 1320 may display at least some images of the captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen to the user, and may display at least some of the captured music playback screens. Corresponding music can be played using execution path information corresponding to some images. Also, the second electronic device A2 1320 may reproduce an audio file.

According to an embodiment, a data sharing server includes a storage unit for storing data including a captured image uploaded from a first electronic device and execution path information corresponding to the captured image, and the data is uploaded when the data is uploaded. Data using the URL of the uploaded data by transmitting the URL of the data to the first electronic device so that the URL of the uploaded data by the first device is transferred to the second electronic device It may include a control unit that controls to download the uploaded data to the second electronic device according to the request.

15 is a diagram illustrating a captured image and an execution path information sharing system corresponding to the captured image according to a second embodiment of the present invention.

Referring to FIG. 15, a first electronic device A 1510 may upload data including a captured image and execution path information corresponding to the captured image to the second shared server 1530 through the communication network 1501. I can. The first electronic device 1510 may receive the URL of the uploaded data from the second sharing server 1530 and provide it to the second electronic device B 1520.

The second sharing server 1530 may be, for example, a server that provides a cloud storage service such as Dropbox, and may be a server that provides a service for electronic devices to upload and download data. When data is uploaded from the first electronic device A 1510, the second sharing server 1530 may provide the ULR of the uploaded data to the first electronic device A 1510.

The second electronic device B 1520 may receive the URL of the data uploaded to the second sharing server 1530 from the first electronic device A 1510 through the communication network 1501. The second electronic device B 1520 may be an electronic device of a user different from the user of the first electronic device A 1510, and the second sharing server 1530 using an account different from the first electronic device A 1510 Data corresponding to the ULR of the uploaded data can be downloaded by accessing to. The second electronic device B 1520 may share the data with the first electronic device A 1510 by storing the downloaded data.

16 is a diagram illustrating an operation of a captured image and an execution path information sharing system corresponding to the captured image according to the second embodiment of the present invention.

Referring to FIG. 16, in operation 1602, the first electronic device A 1510 may store an image captured from an application execution screen and execution path information corresponding to the captured image. According to an embodiment, the first electronic device A 1510 may store the captured image and execution path information corresponding to the captured image as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, and a data file (music file, video file, or image) corresponding to the application execution screen. File), drawing memo information corresponding to the application execution screen, handwriting recognition result information corresponding to the application execution screen, clip location information corresponding to the application execution screen, and the like.

According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the application execution screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the application execution screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the application execution screen.

When a data upload request from the user is received in operation 1604, the first electronic device A 1510 may upload data to the second sharing server 1630 in operation 1606.

The second sharing server 1630 may store the data uploaded in operation 1608 and transmit a URL of the uploaded data to the first electronic device A 1510 so that the data uploaded in operation 1610 can be accessed.

The first electronic device A 1510 may receive a request for sharing a URL of uploaded data from a user in operation 1612. If a request to share a URL of the uploaded data is input, the first electronic device A 1510 may select a URL transmission method of the uploaded data in operation 1614. According to an embodiment, the URL transmission method of the uploaded data may be any one of a mail transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (chat on), and a WIFI direct transmission method. In addition to this, any transmission method may be possible as long as it is a transmission method capable of transmitting ULR. The first electronic device A 1510 may transmit the URL of the data uploaded by the selected transmission method to the second electronic device B 1520 in operation 1616.

When the ULR of the uploaded data is received in operation 1618, the second electronic device B 1520 may access the URL of the uploaded data and download the uploaded data in operation 1620. The second electronic device B 1520 may share the data with the first electronic device A 1510 by storing the data downloaded in operation 1622.

According to an embodiment, when the first electronic device A 1510 is a web browser application execution screen, that is, when a web page screen is captured or at least some images are cropped from the web page screen and some images of the web page screen are captured, the captured Uploading data including execution path information corresponding to the image and the captured image to the second sharing server 1530, receiving the URL of the uploaded data from the second sharing server 1530, and receiving the second electronic device B (1520). ) Can be provided. For example, the first electronic device A 1510 may capture at least some images of a web page according to an input from a user while displaying a web page screen. According to an embodiment, the first electronic device A 1510 may capture at least some images of a web page using an application that performs a capture function according to a user input (eg, a SmartClip application). Also, the first electronic device A 1510 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing the state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to a web page, a URL that can be restored to a web page, a text corresponding to a web page, a data file (music file, video file, or image file) corresponding to the web page, and the web It may include drawing memo information corresponding to the page, handwriting recognition result information corresponding to the web page, clip position information corresponding to the web page, and the like. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to a resource used in a web page. For example, it may be a direct link to a resource such as a video file, sound, or image related to a web page. In addition, the lip link information may be information capable of accessing a subpage in the main page of the web page.

The first electronic device A 1510 may store the captured web page image and execution path information corresponding to the captured web page image as data in a predetermined format.

According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared. According to an embodiment, data in a predetermined format may be stored in a DB related to a scrapbook application. The first electronic device A 1510 may receive data to be uploaded to the second sharing server 1530 from a user among data stored in a DB related to the scrapbook application. According to an embodiment, the first electronic device A 1510 may provide an option for selecting data to be uploaded. When data to be uploaded by the user is selected, the first electronic device A 1510 may upload data to the second sharing server 1530 and request a URL of the uploaded data. The second sharing server 1530 may store the uploaded data, generate a URL of the uploaded data for accessing the uploaded data, and transmit the URL to the first electronic device A 1510. The first electronic device A 1510 may receive a request for sharing a URL of uploaded data from a user. If a request for URL sharing of uploaded data is input, the first electronic device A 1510 may select a URL sharing method of the uploaded data. According to an embodiment, the URL sharing method of uploaded data may be any one of a mail transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (chat on), and a WIFI direct transmission method. In addition to this, any transmission method may be possible as long as it is a transmission method capable of transmitting a URL. The first electronic device A 1510 may transmit the URL of the data uploaded by the selected transmission method to the second electronic device B 1520. The second electronic device B 1520 may receive a message including the URL of the uploaded data. The second electronic device B 1520 may execute a web browser application to access the URL of the uploaded data, and access the URL of the uploaded data to access the captured web page image and the execution path corresponding to the captured web page image. Data containing information can be downloaded. The second electronic device B 1520 may store and execute data including the downloaded captured web page image and execution path information corresponding to the captured web page image. According to an embodiment, the second electronic device B 1520 may store data including a captured web page image downloaded to a scrapbook application DB and execution path information corresponding to the captured web page image, and through a scrapbook application. A corresponding web page screen can be displayed using execution path information corresponding to the captured web page image.

In addition, according to an embodiment, when at least some images are captured while displaying a video playback service application execution screen, for example, during a YouTube screen display, the first electronic device A 1510 includes at least some images of the captured video playback service screen and the captured video. Data including execution path information corresponding to at least some images of the playback service screen is uploaded to the second sharing server 1530 through the first sharing server 1530, and the data uploaded from the second sharing server 1530 is The URL may be received and provided to the second electronic device B 1520.

For example, the first electronic device A 1510 may capture at least some images of a video playback service screen using an application that performs a capture function according to a user input. In the following description, the captured video playback service screen image or some images cropped from the captured captured video playback service screen image may be described as a captured image.

Also, the first electronic device A1 1510 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing the state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to a video playback service, a URL that can be restored to a video playback service, a text corresponding to a video playback service, and a data file (music file, video file, or image) corresponding to the video playback service. File), drawing memo information corresponding to the video playback service, handwriting recognition result information corresponding to the video playback service, clip location information corresponding to the video playback service, and the like. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to a video used on a video playback service screen. For example, it may be a direct link to a video file related to the video playback service screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the video playback service screen.

The first electronic device A 1510 may store at least some images of the captured video playback service screen and execution path information corresponding to at least some images of the captured video playback service screen as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

The first electronic device A 1510 includes data to be uploaded to the second sharing server 1530 from a user among data stored in a DB related to a scrapbook application, that is, at least some images of a captured video playback service screen and a captured video playback service. Data including execution path information corresponding to at least some images of the screen may be selected. According to an embodiment, the first electronic device A 1510 may provide an option for selecting data to be uploaded. When data to be uploaded by the user is selected, the first electronic device A 1510 may upload data to the second sharing server 1530 and request a URL of the uploaded data. The second sharing server 1530 may store the uploaded data, generate a URL of the uploaded data for accessing the uploaded data, and transmit the URL to the first electronic device A 1510. The first electronic device A 1510 may receive a request for sharing a URL of uploaded data from a user. If a request for URL sharing of uploaded data is input, the first electronic device A 1510 may select a URL sharing method of the uploaded data. According to an embodiment, the URL sharing method of uploaded data may be any one of a mail transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (chat on), and a WIFI direct transmission method. In addition to this, any transmission method may be possible as long as it is a transmission method capable of transmitting a URL. The first electronic device A 1510 may transmit the URL of the data uploaded by the selected transmission method to the second electronic device B 1520. The second electronic device B 1520 may receive a message including the URL of the uploaded data. The second electronic device B 1520 may run a web browser to access the URL of the uploaded data, and access the URL of the uploaded data through the web browser to access at least some images of the captured video playback service screen and the captured image. Data including execution path information corresponding to at least some images of the video playback service screen may be downloaded. The second electronic device B 1520 may store and execute data including execution path information corresponding to at least some images of the downloaded captured video playback service screen and at least some images of the captured video playback service screen. . According to an embodiment, the second electronic device B 1520 may store at least some images of the captured video playback service screen downloaded to the scrapbook application DB and execution path information corresponding to at least some images of the captured video playback service screen. In addition, through a scrapbook application, the video service screen may be displayed using execution path information corresponding to at least some images of the video playback service screen.

In addition, according to an embodiment, when at least some images of the music playback screen are captured while the first electronic device A 1510 is displaying the music playback application execution screen, at least some images of the captured music playback screen and the captured music playback screen Uploading data including execution path information corresponding to at least some images of the second shared server 1530, and receiving the URL of the uploaded data from the second shared server 1530, the second electronic device B (1520) Can be provided to.

For example, the first electronic device A 1510 may capture at least some images of a music playback application execution screen using an application that performs a capture function according to a user input. In the following description, the entire image of the captured music playback screen or a partial image cropped from the captured music playback screen may be described as a captured image.

The first electronic device A 1510 may obtain execution path information corresponding to the captured image. The execution path information corresponding to the captured image may include information capable of executing the state of the captured image. According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, the metadata is a title corresponding to the music playback screen, a URL that can be restored to the music playback screen, text corresponding to the music playback screen, and a data file (music file, video file, or image) corresponding to the music playback screen. File), drawing memo information corresponding to the music playback screen, handwriting recognition result information corresponding to the music playback screen, clip location information corresponding to the music playback screen, and the like. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to a music file used on a music playback screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the music playback screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the music playback screen.

The first electronic device A 1510 may store at least some images of the captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen as data in a predetermined format. According to an embodiment, the first electronic device A 1510 captures an original audio file corresponding to the music playback screen, and includes at least some images of the captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen. Together, they can be included in the data. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared. According to an embodiment, data in a predetermined format may be stored in a DB related to a scrapbook application.

The first electronic device A 1510 includes data to be uploaded to the second sharing server 1530 from the user among data stored in the DB related to the scrapbook application, that is, at least some images of the captured music playback screen and the captured music playback screen. Data including execution path information corresponding to at least some images or data including execution path information and audio files corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen may be selected. have. According to an embodiment, the first electronic device A 1510 may provide an option for selecting data to be uploaded. When data to be uploaded by a user is selected, the first electronic device A 1510 may upload data to the second sharing server 1630 and request a URL of the uploaded data. The second sharing server 1530 may store the uploaded data, generate a URL of the uploaded data for accessing the uploaded data, and transmit the URL to the first electronic device A 1510. The first electronic device A 1510 may receive a request for sharing a URL of uploaded data from a user. If a request for URL sharing of uploaded data is input, the first electronic device A 1510 may select a URL sharing method of the uploaded data. According to an embodiment, the URL sharing method of uploaded data may be any one of a mail transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (chat on), and a WIRI direct transmission method. In addition to this, any transmission method may be possible as long as it is a transmission method capable of transmitting ULR. The first electronic device A 1510 may transmit the URL of the data uploaded by the selected transmission method to the second electronic device B 1520. The second electronic device B 1520 may receive a message including the URL of the uploaded data. The second electronic device B 1520 may execute a web browser application to access the URL of the uploaded data, and access the URL of the uploaded data through the web browser to access at least some images of the captured music playback screen and the captured image. Download data including execution path information corresponding to at least some images of the music playback screen or data including execution path information and audio files corresponding to at least some images of the captured music playback screen and at least some images of the captured music playback screen I can receive it. The second electronic device B 1520 stores data including at least some images of the downloaded captured music playback screen and execution path information corresponding to at least some images of the captured music playback screen, or data further including audio data. You can, you can do it. According to an embodiment, the second electronic device B 1520 stores data including at least some images of the captured music playback screen downloaded to the scrapbook application DB and execution path information corresponding to at least some images of the captured music playback screen. It can be stored and executed to play the corresponding music by using execution path information corresponding to at least some images of the music playback screen through a scrapbook application.

17 is a diagram illustrating a concept of sharing a captured image and execution path information corresponding to the captured image according to a third embodiment of the present invention.

Referring to FIG. 17, a first electronic device A 1710 transmits data including an image captured from an application execution screen during application execution and execution path information corresponding to the captured image through a communication network 1701 to a second electronic device. It can be transmitted directly to the device B 1720 to share. According to an embodiment, the first electronic device A 1710 converts data including a captured image and execution path information corresponding to the captured image into data in the form of an image and text as the original, and the second electronic device B 1720 ), or data including the captured image and the execution path information corresponding to the captured image may be converted into data in the form of a scrapbook file (SCC) in a certain data format and transmitted.

18 is a diagram illustrating an operation of sharing a captured image and execution path information corresponding to the captured image between electronic devices according to a third embodiment of the present invention.

Referring to FIG. 18, in operation 1802, a first electronic device A 1710 may store data including an image captured from an application execution screen and execution path information corresponding to the captured image. According to an embodiment, the first electronic device A 1710 may store the captured image and execution path information corresponding to the captured image as data in a predetermined format. According to an embodiment, the data in the predetermined format may be data in the form of a scrapbook file (SCC) or data in the form of an image and text. In addition, any data format including the captured image and execution path information corresponding to the captured image may be any data format that can be shared.

According to an embodiment, the execution path information corresponding to the captured image may include metadata. According to an embodiment, metadata is a title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, and a data file (music file, video file, or image) corresponding to the application execution screen. File), drawing memo information corresponding to the application execution screen, handwriting recognition result information corresponding to the application execution screen, clip location information corresponding to the application execution screen, and the like. According to an embodiment, the metadata may further include deep link information. The deep link information may be information that allows direct access to resources used in the application execution screen. For example, it may be a direct link to a resource such as a video file, sound, or image related to the application execution screen. In addition, the lip link information may be information capable of accessing a subpage in the main page of the application execution screen.

In operation 1804, the first electronic device A 1710 may receive a data sharing request including a captured image by a user input and execution path information corresponding to the captured image.

If a data sharing request is received from the user, the first electronic device A 1710 may select a data sharing method in operation 1806. According to an embodiment, whether the first electronic device A 1710 shares data including the captured image and execution path information corresponding to the captured image as it is, whether the captured image and execution path information corresponding to the captured image are to be shared. It is possible to determine whether to share the data including data by converting it into a certain data format. According to an embodiment, when the original is shared as it is, the original captured image (for example, an image in PNG format or an image in JPG format) and execution path information corresponding to the captured image in text format may be shared. In addition, when data is converted to a certain format and shared, the original captured image (such as PNG format image or JPG format image) and execution path information corresponding to the captured image are converted into a single file, such as a scrap data (SCC) file. To share.

The first electronic device A 1710 may select a data transmission method in operation 1808. According to an embodiment, the first electronic device A 1710 may be selected by a user to transmit data including a captured image and execution path information corresponding to the captured image through which data transmission method among a plurality of data transmission methods. I can. According to an embodiment, the data transmission method may be at least one of various transmission methods such as an email transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (Chat on), and a WIFI direct transmission method. In addition to this, any method that can transmit data may be possible.

In operation 1810, the first electronic device A 1710 may transmit data including the captured image and execution path information corresponding to the captured image to the second electronic device B 1720. According to an embodiment, the first electronic device A 1710 transmits data including an image captured by the selected data sharing method and the selected data transmission method and execution path information corresponding to the captured image to the second electronic device B 1720. Can be transferred to.

The second electronic device B 1720 receives and stores data including an image captured from the first electronic device A 1710 and execution path information corresponding to the captured image in operation 1812, thereby storing the first electronic device A ( 1710) and data including the captured image and execution path information corresponding to the captured image may be shared.

According to various embodiments of the present disclosure, the electronic device 100 may provide various user interface screens to share a captured image and execution path information corresponding to the captured image.

19 to 22 are diagrams illustrating a user interface screen for sharing an image captured by an electronic device and execution path information corresponding to the captured image according to an embodiment of the present invention.

Referring to FIG. 19, the electronic device 100 includes an image captured on a scrapbook application execution screen 1910 and execution path information corresponding to the captured image, as shown in FIG. Also referred to as'scrap data')s 1912 may be displayed. The user may select the scrap data 1914 desired to be shared among the scrap data 1912 displayed on the scrapbook application execution screen 1910.

When the scrap data to be shared by the user is selected, the electronic device 100 includes a captured image 1924 of the selected scrap data and execution path information 1922 corresponding to the captured image, as shown in FIG. 19B. A scrap data screen 1920 indicating) may be displayed. An input field 1926 for adding tag information to the scrap data may be further displayed on the scrap data screen 1920, and a share icon 1928 for requesting sharing of scrap data may be further displayed.

When the sharing icon 1920 is selected by the user, the electronic device 100 displays a file format selection window 1930 for selecting a file format for sharing scrap data as shown in FIG. 19C. I can. The file format selection window 1930 may include a file format such as a file format 1932 consisting of images and texts, a scrapbook data (SCC) format 1934, and the like. The user may select a file format of the scrap data in the file format selection window 1930.

Referring to FIG. 20, the electronic device 100 may receive a selection of a file format 2012 consisting of images and texts by a user in a file format selection window 2010 as shown in FIG. 20A. When the electronic device 100 selects a file format 2012 consisting of images and text, the electronic device 100 selects a transmission method selection window 2020 for selecting a scrap data transmission method, as shown in FIG. 20B. ) Can be displayed. In the transmission method selection window 2020, icons corresponding to various transmission methods such as an email transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (Chat on), and a WIFI direct transmission method may be displayed. The user may select a transmission method of scrap data in the transmission method selection window 2020. When the message transmission method 2022 is selected among various transmission methods, the electronic device 100 may execute the message transmission application and display the message transmission application screen 2030 as shown in FIG. 20C. The electronic device 100 includes scrap data in the attachment file area 2034 of the message transmission application screen 2030 on the message transmission application screen 2030, and receives a user's message in the message writing area 2032 to receive the scrap data. May be transmitted to an electronic device to be shared with a message transmission method.

Meanwhile, referring to FIG. 21, the electronic device 100 may receive a scrapbook data (SCC) format 2014 selected by a user in the file format selection window 2110 as shown in FIG. 21A. When the electronic device 100 selects the scrapbook data (SCC) format 2014, the electronic device 100 selects a transmission method selection window 2120 for selecting a scrap data transmission method as shown in FIG. 21B. Can be displayed. The transmission method selection window 2120 may display icons corresponding to various transmission methods such as an email transmission method, a message transmission method (SMS, MMS, etc.), a chat message transmission method (Chat on), and a WIFI direct transmission method. The user may select a transmission method of scrap data in the transmission method selection window 2120. When the mail transmission method 2122 is selected among various transmission methods, the electronic device 100 may execute the mail transmission application and display the mail transmission application screen 2130 as shown in FIG. 21C. The electronic device 100 includes scrap data in the attachment file area 2132 of the mail transmission application screen 2130 on the mail transmission application screen 2130, and receives mail contents from the user in the mail content creation area 2134. The scrap data can be transmitted to an electronic device to be shared with a mail transmission method.

Referring to FIG. 22, the electronic device 100 may share scrap data by receiving and storing scrap data from other electronic devices through various transmission methods. According to an embodiment, when the electronic device 100 receives the scrap data through the mail transmission method, a button to save the scrap data 2212 included in the attached file on the mail reception screen 2210 as shown in FIG. 22(a) You can save it by selecting (2214). When the scrap data storage button 2214 is selected, the electronic device 100 may store the scrap data 2222 through the data storage screen 2220 as shown in FIG. 22B. When the stored scrap data 2222 is selected, the electronic device 100 may display a category selection screen 2230 for selecting in which category the scrap data is to be stored, as shown in FIG. 22C. When the category 2232 (for example,'New') is selected by the user on the category selection screen 2230, the electronic device 100 displays the scrap data 2242 in the'New' category as shown in FIG. 22(d). By storing the data, the scrap data 2242 can be shared with the sending side.

According to an embodiment, the scrap data may be an SCC file.

23 is a diagram illustrating an SCC file structure according to an embodiment of the present invention. Referring to FIG. 23, the SCC file may be a file such as “fileName.SCC” (2310). The SCC file may include Mimetype (2312), appname (2314), appicon.png (2316), thumbnail.png (2318), META-INF (2320), CONTENTS (2330), and HTML (2340).

Mimetype 2312 may be information indicating the name of each application. For example, application/vnd.samsung.SCC. is used as a prefix, and the name of each app is added after that, and may be “Scrapbook: application/vnd.samsung.SCC.pinall”.

The appname 2314 is a text file without an extension and may be an application name corresponding to scrap data. The application name may be displayed in the integrated viewer.

The appicon.png 2316 may be an icon file of an application corresponding to the scrap data. The icon may be displayed in the integrated viewer.

The thumbnail.png 2318 may be a thumbnail image of scrap data. In addition to the Png file, it can also be a jpg file.

The META-INF 2320 is a location in which metadata is stored and may include metadata. Location information in which metadata is stored may be provided to a corresponding application. The metadata format can be composed of XML and JSON files. Each XML, JSON file composition and format in the file can be defined in each application.

The CONTENTS 2330 is a location in which original multimedia files are stored. For example, original files of an image, audio, and video file may be linked in an HTML file through a relative path.

The HTML 2340 may be an HTML file and resources (font, css, js) used in the HTML file so that a screen can be configured in the integrated viewer. The HTML folder may contain index.html, which is an essential file, and additionally necessary html files may be stored. HTML can comply with HTML5 conventions and can comply with CSS conventions. Also, HTML can support JavaScript.

24 is a diagram illustrating META-INF 2320 included in an SCC file according to an embodiment of the present invention. Referring to FIG. 24, the META-INF 2320 may have a JSON file structure. META-INF 2320 is a category name 2402, content start (scrap data start) 2404, content type (scrap data type) 2406, thumbnail image path 2408, captured image path 2410, The captured image width (2412), the height of the captured image (2414), the time the image was captured (2416), the captured image cropping method (2418) (e.g. 0: free curve, 1: square, 2: free curve) Automatic correction, 3. cropping by scrap booker, etc.), and meta data 2420 may be included.

25 is a diagram illustrating meta data according to an embodiment of the present invention. Referring to FIG. 26, meta data may be classified into a meta key 2510, a meta value 2520, and a meta value type 2530. The meta key 2510 may be a key indicating the type of meta data. The meta value 2520 may be a value corresponding to a key indicating the type of each meta data. The meta value type 2530 may be a data type of a key indicating the type of meta data. According to an embodiment, a key indicating the type of metadata may include title, plain_text, app_deep_link, url, file_path_audio, file_path_video, file_path_image, file_path_drawing_memo, file_path_spd, dm_hand_write_text, and location.

Title may be a title corresponding to an application execution screen corresponding to the captured image. plain_text may be text corresponding to the captured image. app_deep_link may be a URL that can be restored to an application execution screen corresponding to the captured image. The url may be url information obtained from the captured image, and file_path_audio, file_path_video, and file_path_image may be paths of original data files (music files, video files, or image files) related to the application execution screen corresponding to the captured image. file_path_drawing_memo may be drawing memo information corresponding to the captured image. file_path_spd may be drawing memo path information. dm_hand_write_text may be handwriting recognition result information corresponding to the captured image. Location may be clip location information corresponding to the captured image.

According to various embodiments of the present invention as described above, the screen image captured by the user's electronic device and the execution path information capable of executing the screen at the time of screen capture are shared with other electronic devices of the same user or electronic devices of other users. Accordingly, another electronic device of the same user or an electronic device of another user may execute the screen at the time of screen capture by using the captured screen image. Methods according to various embodiments of the present disclosure may be performed through various computer means. It may be implemented in the form of executable program instructions and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the present invention, or may be known and usable to a person having ordinary skill in the computer software field.

It will be appreciated that the method of using a captured image according to an embodiment of the present invention can be realized in the form of hardware, software, or a combination of hardware and software. Any such software may be, for example, a volatile or non-volatile storage device such as a storage device such as a ROM, or a memory device such as a RAM, memory chip, device or integrated circuit, for example, whether erasable or rewritable. Or, for example, it may be optically or magnetically recordable, such as a CD, DVD, magnetic disk or magnetic tape, and stored in a storage medium readable by a machine (eg, a computer). A method of using a captured image according to an embodiment of the present invention may be implemented by a computer or electronic device including a control unit and a memory, and the memory stores a program or programs including instructions for implementing the embodiments of the present invention. You will see that it is an example of a machine-readable storage medium suitable for. Accordingly, embodiments of the present invention include a program including a code for implementing the apparatus or method described in any claim of the present specification, and a storage medium readable by a machine (such as a computer) storing such a program. Further, such a program may be transferred electronically through any medium, such as a communication signal transmitted through a wired or wireless connection, and the present invention suitably includes equivalents thereto.

In addition, the electronic device according to the embodiments of the present invention may receive and store the program from a program providing apparatus connected by wire or wirelessly. The program providing apparatus includes a memory for storing a program including instructions for performing a method according to the use of a captured image according to various embodiments of the present invention, a communication unit for performing wired or wireless communication with the electronic device, and And a control unit that controls the request of the electronic device or the corresponding program to be automatically transmitted through the communication unit.

Claims (23)

In the electronic device,
Ministry of Communications,
Display, and
During application execution, when an application execution screen is captured, execution path information capable of executing the state of the captured screen image is obtained, the captured screen image and the execution path information are stored, and the captured screen image and the execution And a processor for transmitting data including path information to an external electronic device through the communication unit,
The application includes a music playback application,
And the execution path information includes information on a music reproduction section at a time when the application execution screen is captured during music reproduction using the music reproduction application.
The method of claim 1,
The electronic device, wherein the execution path information includes metadata.
The method of claim 2,
The meta data is
A title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, a content file corresponding to the application execution screen, drawing memo information corresponding to the application execution screen, And at least one of handwriting recognition result information corresponding to the application execution screen and clip position information corresponding to the application execution screen.
The method of claim 3, wherein the metadata further includes deep link information,
The electronic device, wherein the deep link information includes information for directly accessing a resource used in the application execution screen.
The method of claim 4,
The deep link information further includes information for accessing a subpage within a main page of the application execution screen.
delete The method of claim 1,
The electronic device, wherein the captured screen image includes an entire image of the captured screen or a partial image of the captured screen.
The method of claim 7,
The electronic device, wherein the partial image of the captured screen includes at least one of an image of an object included in the captured screen and an image included in a closed curve by a user's drawing input on the captured screen.
delete The method of claim 1,
The electronic device, characterized in that the data is data in the form of a scrapbook file (SCC) or data including images and text.
delete In the method of using a capture screen in an electronic device,
The operation of capturing the application execution screen while the application is running,
Acquiring execution path information capable of executing the state of the captured screen image,
Storing the captured screen image and the execution path information, and
And transmitting data including the captured screen image and the execution path information to an external electronic device through the communication unit,
The application includes a music playback application,
And the execution path information includes information on a music reproduction section at a time when the application execution screen is captured during music reproduction using the music reproduction application.
The method of claim 12,
The method of claim 1, wherein the execution path information includes metadata.
The method of claim 13,
The meta data is
A title corresponding to the application execution screen, a URL that can be restored to the application execution screen, text corresponding to the application execution screen, a content file corresponding to the application execution screen, drawing memo information corresponding to the application execution screen, And at least one of handwriting recognition result information corresponding to the application execution screen and clip position information corresponding to the application execution screen.
The method of claim 14, wherein the metadata further includes deep link information,
Wherein the deep link information includes information for directly accessing a resource used in the application execution screen.
The method of claim 15,
The deep link information further includes information for accessing a subpage within a main page of the application execution screen.
The method of claim 12,
And transmitting the captured screen image and the execution path information to share the captured screen image and the execution path information.
The method of claim 17, wherein the transmitting process comprises:
A process of selecting a data format of data including the captured screen image and the execution path information,
A process of selecting a transmission method of data including the captured screen image and the execution path information,
And transmitting data including the captured screen image and the execution path information according to the selected data format and the selected transmission method. The method of claim 12,
The method of claim 1, wherein the captured screen image includes an entire image of the captured screen or a partial image of the captured screen.
The method of claim 18,
The data format is a scrapbook file (SCC) data format or a data format consisting of images and text.
In a storage medium that stores a program using a capture screen,
The program causes the electronic device,
The operation of capturing the application execution screen while the application is running,
Acquiring execution path information that can execute the state of the captured screen image,
Storing the captured screen image and the execution path information, and
To perform an operation of transmitting data including the captured screen image and the execution path information to an external electronic device through the communication unit,
The application includes a music playback application,
And the execution path information includes information on a music reproduction section at a time when the application execution screen is captured during music reproduction using the music reproduction application.
In the data sharing server,
A storage unit for storing data including an image captured from an application execution screen during execution of an application uploaded from the first electronic device and execution path information corresponding to the captured image,
When the data is uploaded, it includes a processor for controlling the transmission of the stored data to a second electronic device accessible to the same account as the first electronic device,
The application includes a music playback application,
And the execution path information includes information on a music reproduction section at a time when the application execution screen is captured during music reproduction using the music reproduction application.
In the data sharing server,
A storage unit for storing data that is uploaded from the first electronic device and includes an image captured from an application execution screen during application execution and execution path information corresponding to the captured image,
When the data is uploaded, a URL of the uploaded data is transmitted to the first electronic device, and a data request using the URL of the uploaded data from a second electronic device receiving the URL of the uploaded data from the first electronic device Upon receiving the, a processor for controlling to transmit the uploaded data to the second electronic device,
The application includes a music playback application,
And the execution path information includes information on a music reproduction section at a time when the application execution screen is captured during music reproduction using the music reproduction application.

Images