KR20190126267A - Method and device for generating capture image for display windows - Google Patents

Abstract

Provided are a method and a device for generating a capture image. The method comprises the steps of: determining a first display window to be captured from among a plurality of display windows displayed on a screen; capturing data displayed on the determined first display window based on a user input; and overlapping the capture image with the first display window in a size of the first display window and displaying the capture image, wherein the capture image is generated by the capture.

Description

METHOD AND DEVICE FOR GENERATING CAPTURE IMAGE FOR DISPLAY WINDOWS}

The present invention relates to a method and a device for generating a captured image for display windows displayed on a screen, and more particularly, to generate and move a captured image among a plurality of display windows based on a received user input. It relates to a method and a device.

In a terminal device that simultaneously executes a plurality of applications, interaction between applications is possible through a multi window framework. In addition, in the case of a multi-window system, the same application may be executed through a plurality of windows at the same time.

In a conventional terminal device environment, even though a plurality of applications are executed at the same time, each of the applications is executed on the full screen, so it is not easy to simultaneously execute and manipulate the plurality of applications.

In the present invention, through the multi-window system, to move the data between a plurality of applications running at the same time to activate the interaction between the applications.

In order to solve the above technical problem, a method of generating a capture image for a display window displayed on a screen includes: determining a first display window to be captured from among a plurality of display windows displayed on the screen; Capturing data displayed on the determined first display window based on the user input; And displaying the captured image generated by the capture overlapping the first display window in the size of the first display window.

According to an embodiment for solving the above technical problem, the displaying may include displaying the captured image by overlapping each of the plurality of first display windows when the first display windows are plural.

According to an embodiment for solving the above technical problem, the displaying may include displaying a captured image on a portion of the first display window when the first display window is a full screen mode display window.

According to an embodiment for solving the above technical problem, the image capturing method may further include inserting a part or all of the displayed captured image into the second display window.

According to an embodiment for solving the above technical problem, the step of inserting a part or all of the captured image may be based on a user input of touching the captured image for more than a preset time and dragging the touch toward the second display window. And inserting part or all of the captured image into the second display window.

According to an embodiment for solving the technical problem, determining a predetermined region of the displayed captured image; And cropping an image of the determined area, and inserting a part or all of the captured image into the second display window comprises inserting the cropped image into the second display window.

According to an embodiment of the present disclosure, the determining of a predetermined area may include displaying an area selected by a user on a displayed captured image; And modifying the displayed area, wherein the captured image of the modified area is inserted into the second display window.

According to an embodiment for solving the above technical problem, the modifying the displayed area may include changing the displayed area from the captured image as the user touches the displayed area and drags the touch within a preset time from the touched time point. It is characterized by moving to the area.

According to an embodiment for solving the above technical problem, the modifying the displayed area may include changing the size of the displayed area as the user pinches or unpinches the displayed area. .

According to an embodiment for solving the above technical problem, the area selected by the user is selected based on a user's touch input drawing a closed curve on the captured image.

According to an embodiment for solving the above technical problem, the determining of the predetermined area may include determining the predetermined area as an image to be cropped as the user's touch on the predetermined area is maintained for a predetermined time or more. It is done.

According to an embodiment for solving the above technical problem, the cropped image is overlapped and displayed on the determined image having a smaller size than the image of the determined region.

According to an embodiment for solving the above technical problem, the determining of the first display window may include determining an activated window among the plurality of displayed display windows as the first display window as a predetermined button on the screen is touched. It is characterized by.

According to an embodiment for solving the above technical problem, the determining of the first display window may include determining a first display window, in response to a touch of a predetermined button on the screen, removing the active window from among the plurality of displayed display windows. Characterized in that determined.

According to an embodiment for solving the above technical problem, the inserting into the second display window may include capturing when an application corresponding to the second display window provides a function of inserting an image into a screen displayed on the second display window. The image is inserted into the second display window.

According to an embodiment for solving the above technical problem, the captured image is characterized in that the overlapped display at the same position as the position of the first display window in the size of the first display window.

The capture image generating device for solving the technical problem, the user input unit for receiving a user input to the device; A capture unit which determines a first display window to be captured from among a plurality of display windows displayed on the screen, and captures data displayed on the first display window determined based on a user input; And a display unit configured to display the captured image generated by the capture overlapping the first display window in the size of the first display window.

Provided is a computer readable recording medium having recorded thereon a program for executing a method of generating a captured image for solving the above technical problem.

Through the above-described solutions, data such as an image or text may be moved between a plurality of applications in a multi-window system. With the data being moved, the user can expect intuitive interactions between a plurality of applications.

The invention can be easily understood by the following detailed description and the accompanying drawings in which reference numerals refer to structural elements.
1 is a diagram illustrating a plurality of display windows that are overlapped and displayed on a screen according to an exemplary embodiment.
FIG. 2 is a block diagram illustrating a configuration of a device for generating a captured image for a display window according to an exemplary embodiment.
3 is a flowchart illustrating a method of generating a captured image according to an exemplary embodiment.
4 is a flowchart illustrating an example of moving a captured image, according to an exemplary embodiment.
5 is a flowchart illustrating an embodiment of moving a captured image according to an embodiment of the present invention.
6 is a diagram illustrating a process of overlapping and displaying a captured image according to one embodiment of the present invention.
7 is a diagram illustrating a process of overlapping and displaying a captured image according to an embodiment of the present invention.
8 is a diagram illustrating a process of overlapping and displaying a captured image according to an embodiment of the present invention.
9 is a diagram illustrating a process of determining a region to be captured in accordance with one embodiment of the present invention.
10 is a diagram illustrating an embodiment of the present invention for moving a captured image.
FIG. 11 is a diagram illustrating an example of modifying an area to be inserted in a captured image.
12 illustrates another embodiment of the present invention for moving a captured image to a second display window.
FIG. 13 is a diagram illustrating another embodiment of moving the captured image to the second display window. FIG.
14 is a diagram illustrating an embodiment of displaying an editing tool for correcting an inserted capture image.
FIG. 15 is a diagram illustrating another embodiment of the present invention for moving data displayed on a first display window to a second display window.
FIG. 16 is a diagram illustrating another embodiment of the present invention for moving data displayed on a first display window to a second display window.
FIG. 17 is a diagram illustrating another embodiment of the present invention for moving data displayed on a first display window to a second display window.
FIG. 18 is a diagram illustrating another embodiment of the present invention for moving data displayed on a first display window to a second display window.

In the specification, when "capturing" the data displayed on the screen, this includes not only acquiring the displayed "image or text", but also obtaining "information related to the displayed image or text." For example, when capturing the displayed image or text, uniform resource identifier (URI) information, intent information, and the like related to the image or text may be obtained together. In addition, when a part is said to "include" a certain component, this means that it may further include other components, except to exclude other components unless otherwise stated.

Hereinafter, with reference to the drawings will be described embodiments of the present invention;

1 is a diagram illustrating a plurality of display windows that are overlapped and displayed on a screen according to an exemplary embodiment.

As shown in FIG. 1, a plurality of display windows 10, 20, 30, and 40 may be overlapped and displayed on a screen according to an exemplary embodiment. The pin-up display window 10 is a window set to be displayed on the top of the screen and may always be displayed on the top of the screen when displayed with the other display windows on the screen. In addition, a separate icon 5 may be displayed together in a predetermined area of the pinup display window 10. For example, the pin-up display window 10 may be displayed as if a pin-shaped icon is inserted into the pin-up display window 10.

The full screen mode display window 20 is a window set to be displayed on the entire screen, and may be displayed on the screen in the same size as the screen size.

The partial screen mode display window 30 is a window set to be displayed on a part of a screen and may be suitable for an application that supports a window having a smaller size than the screen size. In addition, the partial screen mode display window 30 may be suitable for applications that may display the windows in duplicate, for example, an application that provides a chat, a memo, and the like.

In addition, the free size mode display window 40 may be a window that is displayed on a part of the screen and whose size can be freely adjusted by a user input. In addition, the display windows 10, 20, 30, and 40 may overlap each other, and a predetermined window may be displayed among the display windows 10, 20, 30, and 40 according to the type of application.

2 is a block diagram illustrating a configuration of a device 100 for moving a captured image between display windows according to an exemplary embodiment. The device 100 according to an exemplary embodiment includes a user input receiver 110, a capturer 120, a display 130, a controller 140, and a memory 150.

Hereinafter, a process in which the device 100 moves the captured image between the plurality of display windows by using the configuration included in the device 100 will be described.

The user input receiver 110 receives a user input to the device 100. The user may touch a predetermined position on the screen of the device 100, and the user input receiver 110 may receive a user input by a user's touch. The user input receiver 110 may receive a user input through an input tool such as a keyboard, a mouse, a stylus, or the like.

The user input receiver 110 may receive an input for capturing data displayed on a display window on the device 100. According to an embodiment of the present disclosure, the user input receiver 110 may receive an input for selecting a preset button displayed on a display window as an input for capturing data.

The user input receiver 110 may also receive an input for selecting a part of the captured image generated by the capture. For example, the user input receiver 110 may receive a touch and drag input as an input for selecting a partial region of the captured image in order to draw a closed curve on the captured image. In the above, the user input receiving unit 110 has been described as receiving a user input for capturing a screen of the display window and selecting a part of the captured image, but is not limited thereto. The user input receiver 110 may receive a user input for directly capturing a portion of the screen of the display window. For example, the user input unit 110 may receive an input of touching a predetermined point of the captured image, and an area within a predetermined range may be captured from the touched point.

As another example, the user input receiver 110 may receive an input for selecting text. The text selection input received by the user input receiver 110 may be an input for dragging some text. However, an input for selecting data received by the user input receiver 110 is not limited to the above-described embodiment. For example, the user input receiving unit 110 may receive various types of user inputs, such as receiving an input for selecting a predetermined area by selecting two or more locations on the screen.

In addition, the user input receiver 110 may receive an input for moving the captured data. The user input receiving unit 110 may receive various inputs such as touch, drag and drop, long tap, and hold in receiving an input for selecting and moving the captured data. A kind of operation can receive a user input. In addition, the user input receiver 110 may receive an input for moving some of the selected data together with an input for selecting some of the captured data. The data may include, but is not limited to, for example, images, text, and video.

In addition, the user input receiver 110 may also receive a user input for terminating the capture of data. When it is no longer necessary to move the captured data to the second display window, the user input receiver 110 may receive a user input for terminating the capture from the user.

The capturer 120 captures data displayed on the first display window. The data displayed on the first display window may be various kinds of data such as still images, moving images, or text. However, the data captured by the capture unit 120 is not limited to such an image or text, and may be any object displayed on the display window.

The capturer 120 may capture data displayed on the first display window based on the user input received by the user input receiver 110. As described above, when the user input receiver 110 receives an input for selecting a preset button displayed on the first display window, the capturer 120 may capture the displayed data.

In detail, the capturer 120 may determine a first display window to be captured from among a plurality of display windows displayed on the screen of the device 100. As the predetermined button on the screen of the device 100 is touched, the capturer 120 may determine the activated window among the plurality of display windows displayed on the screen of the device 100 as the first display window.

In addition, when a predetermined button on the screen of the device 100 is touched, the capturer 120 may determine a window excluding the activated window among the plurality of display windows displayed on the screen of the device 100 as the first display window. have. In addition, when three or more display windows are displayed on the screen of the device 100, each of the two or more display windows may be determined as the first display window.

In addition, the capture unit 120 may capture data displayed on the first display window, and determine a part or all of the captured image generated by the capture as an image to be cropped. The capturer 120 may determine a predetermined area on the captured image based on the user input, and may crop the determined image. For example, the capturer 120 may determine an image to cut out an area included in the closed curve based on a user input of drawing a closed curve on the captured image. In addition, the capturer 120 may determine an image to cut out an area within a preset range from a predetermined point touched by the user.

In addition, a predetermined area in the captured image may be selected and modified by the user, and the capture unit 120 may determine that the image of the modified area is to be cut out. As the user touches the area selected by the user and drags the touch within a preset time from the touched time point, the touched area may be moved to another area in the captured image.

In addition, the capturer 120 may crop the image of the moved region. In addition, as the user pinches or unpinches the area selected by the user, the size of the selected area may be changed, and the capture unit 120 may cut an image of the area of which the size is changed. In addition, when the touch input for the area selected and modified by the user is maintained for a predetermined time or more, the capturer 120 may determine the image to be cut out as the image to be cut out.

The capturer 120 may capture data displayed on the first display window as an image or text. In addition, the capturer 120 may select some regions of the captured data. As described above, when the user input receiver 110 receives an input for selecting a partial region of the captured screen image, the capturer 120 may select a partial region of the captured image based on the received user input. have.

In addition, when the user input receiver 110 receives an input for selecting some text, the capturer 120 may select some text of the captured text based on the received user input. The text captured by the capture unit 120 may include character data, uniform resource identifier (URI) data, intent data, or the like among displayed data.

In addition, when the data displayed on the first display window is a video, the capturer 120 may capture URI data or intent data of the video displayed based on a user input. For example, the capturer 120 may capture the Internet address of the video displayed on the first display window and / or information related to the video (video title, contents, information on the person, etc.). The title of the captured video may be inserted in the form of text in the second display window, or the Internet address of the captured video may be inserted in the form of a link in the second display window.

The display unit 130 inserts and displays the data captured by the capture unit 120 into the second display window. The captured data may be a captured image. The display 130 may insert the data captured by the capturer 120 into the second display window based on the user input received by the user input receiver 110 and display the data.

For example, when the user input receiver 110 selects captured data by touching a predetermined area of the first display window, and receives an input for moving the selected data to the second display window through drag and drop, the display is displayed. The unit 130 may insert the captured data into the second display window to display the captured data. In addition, when the user input receiving unit 110 receives an input for moving some of the captured data, the display 130 may insert some regions or some texts of the captured data into the second display window to display the input. have.

In detail, the display 130 may display the captured image generated by the capturer 120 overlapping the first display window, and insert a portion or all of the displayed captured image into the second display window to display the captured image. . In this case, the captured image may be displayed at the same size as the first display window and overlapped with the position of the first display window. However, it is not limited thereto.

In addition, the display 130 may insert and display a part of the captured image determined by the capturer 120 in the second display window. In detail, the display 130 may display a region selected by the user on the captured image based on a user input for the captured image. In addition, as the user moves the displayed area or changes the size of the displayed area, the display 130 may move or change the displayed area to display it.

In addition, when a portion of the captured image is selected and the selected region is touched for a predetermined time or more, the capturer 120 may crop the image of the touched region, and the display 130 may touch the cropped image. The image may be reduced to be smaller than the image of the region and overlapped on the touched region. In addition, when a touch on the cut image is dragged toward the second display window, the display 130 may move the cut image to the second display window.

In addition, when the application corresponding to the second display window provides a function of inserting an image into the screen displayed on the second display window, the display 130 may insert all or a part of the captured image into the second display window. Can be.

In addition, the display unit 130 inserts and displays a part or all of the captured image into the second display window based on a user input of touching the captured image for a preset time or more and dragging the touch toward the second display window. You may.

According to an embodiment of the present disclosure, when the user input receiver 110 receives a drag and drop operation, the display 130 may insert and display data at a position where the drop operation is terminated in the second display window. have. This will be described in detail with reference to FIG. 4. In addition, when the position at which the drop operation ends is a predetermined field included in the second display window, the display 130 may insert the captured data into the field at the position where the drop operation ends.

In addition, the display 130 may adjust the size of the data to be inserted on the screen based on the size of the region in which the captured data (for example, an image or text) is displayed in the second display window. That is, when the screen size of the second display window is smaller than the screen size of the captured data, the display unit 130 may reduce the size of the captured data on the screen and insert it into the second display window. Alternatively, if the size of the area on which the data is inserted and displayed is small (for example, in the case of an area corresponding to each date in the display window where a calendar is displayed), regardless of the size on the screen of the second display window itself. In addition, the display 130 may reduce the size of the data to be inserted and displayed according to the size of the corresponding area.

In addition, for example, the display 130 may insert and display the captured image or text in the second display window. In addition, the display 130 may insert the link information corresponding to the captured data together with the captured data (for example, the image and the text) to the second display window.

In addition, the display 130 may display the captured data separately from the uncaptured area. That is, the display 130 may display the captured area and the uncaptured area of the displayed data after the data is captured and before the user input receiver 110 receives the user input for terminating the capture. have. For example, the uncaptured area of the data displayed on the first display window may be displayed darker than the captured area, or at least one of the color, saturation, and brightness of the uncaptured area may be displayed.

In addition, when the data captured by the capturer 120 is information about a video, the display 130 may insert the captured video information into the second display window. That is, when the notepad application is executed in the second display window, the display 130 may insert and display the title of the video in the form of text in the second display window. In addition, the display 130 may insert and display the Internet address of the video in a link form. In addition, when the web browser application is executed in the second display window, the display 130 directly connects the Internet address to the web browser application in the second display window, so that the captured video is captured to be executed in the second display window. Video information may be inserted and displayed.

The controller 140 controls the overall operation of the device 100 and moves the data between the plurality of display windows displayed on the device 100 so as to move the data between the user input receiver 110 and the capturer 120. , The display 130, and the memory 150 are controlled.

The memory 150 stores various kinds of information for moving data between a plurality of display windows displayed on the device 100. For example, the memory 150 may store user input received by the user input receiver 110, image or text data captured by the capture unit 120, data displayed by the display 130, and the like. have. In addition, the memory 150 may also store information transmitted and received between the user input receiver 110, the capturer 120, the display 130, and the controller 140.

Hereinafter, a method of moving data between a plurality of display windows using a configuration included in the device 100 will be described with reference to FIG. 3.

3 is a flowchart illustrating a data movement method according to an embodiment of the present invention. 3 is a time series process performed by the user input receiving unit 110, the capture unit 120, the display unit 130, the control unit 140, and the memory 150 shown in FIG. 2. It consists of. Therefore, even if omitted below, the contents described above with respect to the components illustrated in FIG. 2 may be applied to the flowchart illustrated in FIG. 3.

In operation 310, the device 100 determines a first display window to be captured. The first display window may be any one of a plurality of display windows displayed on the screen of the device 100. According to an embodiment of the present disclosure, the first display window may be any one of the currently activated display windows according to a user's selection from among a plurality of windows, and according to an embodiment, may be a plurality of display windows.

For example, when a plurality of display windows are displayed, the device 100 may determine one display window that plays a video as the first display window. In this case, the device 100 may determine the corresponding display window as the first display window by an external input signal that touches an arbitrary region of one of the display windows (that is, the activated display window) on which the video is played. Alternatively, the device 100 may determine all of the display windows or the plurality of display windows displayed on the screen as the first display window except the activated display window.

In operation 320, the device 100 captures data displayed on the first display window. The device 100 may capture data displayed on the first display window based on an input of touching an area corresponding to a capture button (a button matched with a function of capturing a display window) displayed on the screen. The user input for capturing data may be not only an input for touching the capture button area but also a long tap input for touching the capture button area for a predetermined time or more.

In operation 320, the device 100 may capture data corresponding to an area except for an area corresponding to a status bar on the top or the bottom of the data displayed on the first display window. That is, the device 100 may capture only an area corresponding to an application executed in the display window. Of course, the device 100 may capture the entire area of the display window.

In operation 330, the device 100 displays the captured image on the screen. The captured image may mean an image or a still image captured in step 320 among data displayed on the first display window.

In operation 330, the captured image may be displayed on the screen in the same size as the first display window. That is, the captured image capturing data displayed on the first display window may be the same size as the first display window, and thus may be displayed at the same size as the first display window.

Meanwhile, according to the exemplary embodiment described above, when an area except the upper and / or lower status bars is captured in the first display window, the captured image may be smaller than the first display window. As such, the captured image may be displayed in a smaller size than the first display window.

In addition, with respect to the location where the captured image is displayed, the device 100 may overlap the displayed image on the first display window. That is, the device 100 may display the captured image having the same size as the first display window by overwriting the first display window. When the moving image is played in the first display window, the captured image may be a still image corresponding to the point in time when the moving image is captured, and the still image may be displayed overlapped with the moving image. (Except that the video continues to play in the first display window)

Meanwhile, according to another exemplary embodiment, in operation 330, the captured image is not completely overlapped with the first display window, but the predetermined image is not overlapped with the first display window. It may be displayed in the area of. That is, when a capture image having a smaller size than the first display window is generated with respect to the video played in the first display window, the captured image is positioned at the bottom or top of the first display window so as not to cover the video played in the first display window. Can be displayed. Accordingly, the user may check the captured image separately from the video executed in the first display window.

According to another embodiment, the device 100 may additionally generate a separate display window to display the captured image on the new display window. In this case, the new display window may be displayed in a predetermined area on the screen of the device 100.

According to an embodiment of the present disclosure, when there are a plurality of first display windows to be captured, in operation 330, the device 100 displays a plurality of captured images captured in each first display window by overlapping each first display window. You may. According to another embodiment, when a full screen mode display window is captured in which the size of the first display window to be captured is equal to the size of the screen, the device 100 may display the captured image smaller than the first display window in a portion of the first display window. Can be displayed in size.

According to steps 310 to 330 described with reference to FIG. 3, the device 100 may conveniently manage the image captured in the display window. That is, since the captured images are overlapped (or overlayed) on the display window, which is the object of capturing, the user does not need to edit the captured image through a separate display window. In addition, the captured image is displayed in the same position and the same size as the display window to be captured, it is possible to efficiently identify the captured object in terms of the user interface.

4 is a flowchart illustrating an embodiment of moving a captured image according to an embodiment of the present invention.

In operation 341, the device 100 displays the captured image on the screen. That is, step 330 described in FIG. 3 is performed. The device 100 may overlap the first display window and display the captured image in the same size as the first display window that is the target of the capture.

In operation 342, the device 100 selects a predetermined area of the captured image to move to the second display window. That is, the device 100 may determine a predetermined area for inserting into the second display window among the captured images according to the external input signal.

For example, the device 100 may determine a predetermined area to cut out according to an external input signal that draws a closed curve. For example, the device 100 may select an external shape that selects a rectangle corresponding to a predetermined size and shape. The predetermined area to be cropped may be determined according to the input signal. The predetermined size and shape may be in various forms, and may be in various forms such as circular, elliptical, as well as rectangular.

Meanwhile, the device 100 may display the determined predetermined area to be visually distinguished from other areas of the captured image. That is, the device 100 may display at least one of hue, saturation, and brightness of a predetermined area so as to be distinguished from other areas. Of course, the device 100 may apply and display a visual effect to a region other than the predetermined region without applying the visual effect to the predetermined region. For example, the device 100 may display the determined predetermined area as it is, and express all other areas in black so that the predetermined area to be inserted becomes prominent.

In operation 343, the device 100 modifies the region selected in operation 342. That is, the device 100 may modify the size and / or location of the selected area according to the user input. The size of the selected area may be modified by pinching or unpinching input, and the device 100 may newly determine the selected area according to a new input drawing a closed curve. The selected area may also be modified by a user input for selecting any one type of figure.

For example, when the user determines a predetermined area through an input of drawing a closed curve in step 342, the predetermined area may deviate from a desired location of the user. Accordingly, the user may determine a partial region of the captured image for changing the position and inserting it into the second display window while maintaining the shape of the predetermined region. In this case, the user input for changing the position of the predetermined region may be an input of touching the predetermined region and dragging the touch input within a preset time from the touched point.

In operation 344, the device 100 moves the predetermined area determined in operation 342 and operation 343 to the second display window to insert the predetermined area into the second display window. In detail, in operation 344, when the touch input for the predetermined area is maintained for a predetermined time or more, the device 100 may determine and move the image to be cut out to insert the predetermined area. That is, as described above, when the device 100 displays a predetermined area on the screen and modifies the predetermined area such as position movement and size adjustment, the user moves the predetermined area to the second display window. You can touch for more than a preset time. The device 100 may determine a predetermined region as an image to be inserted according to the user input.

According to an embodiment of the present disclosure, when a user input for determining an image to be inserted is received, the device 100 may display an image having a size smaller than a predetermined area overlapping the predetermined area. That is, when an image to be cropped is determined according to a touch input for a preset time or more, the device 100 may overlap and display the image to be cropped to a smaller size than a predetermined area so that a user can easily check the cropped image. Meanwhile, the device 100 may overlap the size of the image to be cropped to a size larger than a predetermined area.

In addition, the device 100 may display a position on the screen of the image to be cropped at a position different from the predetermined region. That is, for example, if the predetermined area having a rectangular shape is used, the device 100 may display the location of the center of the predetermined area and the location of the center of the image to be cut out differently within the predetermined range. Specifically, it looks at in detail in FIG. Accordingly, the size and position of the image to be cropped are expressed differently from the predetermined area, so that the user can easily visually check the image to be cropped.

When a predetermined area is displayed on the screen according to the above description, the device 100 moves the predetermined area to the second display window. The predetermined area to be moved may be an image cut out by the device 100 by a user input. The device 100 may touch the predetermined area for a preset time and move the predetermined area to the second display window based on a user input of dragging the touch input toward the second display window.

As described above, the device 100 may move a portion (predetermined area) of the captured image captured in the first display window, or move all of the captured image in the first display window.

In operation 345, the device 100 inserts and displays a predetermined area in the second display window. That is, the predetermined area is moved to the second display window according to the user input, and when the drag input is terminated (when the user removes the finger from the screen), the predetermined area may be inserted at the position of the second display window where the drag input is terminated. have.

Meanwhile, when the application corresponding to the second display window provides a function of inserting an image, the device 100 may insert the captured image into the second display window. In other words, if the application corresponding to the second display window is an application irrelevant to the insertion of the image, the capture image does not need to be inserted. Can be inserted.

5 is a flowchart illustrating an embodiment of moving a captured image according to an embodiment of the present invention. In operation 345 described with reference to FIG. 4, the device 100 inserts and displays a predetermined area moved to the second display window into the second display window.

In operation 361, the device 100 determines whether the application running in the second display window supports the insertion of the captured image. If yes, go to step 362; if not, go to step 363.

In operation 362, when the application corresponding to the second display window supports insertion of the captured image as in the notepad application, the device 100 inserts the predetermined area moved in operation 350 into the second display window.

In operation 363, if the insertion of the captured image is not supported, the device 100 ignores the predetermined area without inserting it even if the predetermined area is moved to the second display window.

Accordingly, the device 100 may determine whether to insert the captured image according to the type of application corresponding to the display window into which the captured image is to be inserted, and may prevent the image from being unnecessarily inserted.

6 is a diagram illustrating a process of overlapping and displaying a captured image according to one embodiment of the present invention. Three display windows 61, 62, and 63 are displayed on the screen 60 of FIG. 6A, and the device 100 displays a capture button for capturing data on a bar displayed at the bottom of the screen 60. 65).

In addition, each display window 61, 62, 63 includes a status bar for moving the display window or performing predetermined tasks such as maximizing, minimizing, and closing the display window. In FIG. 6, the identification number of the status bar 64 is shown only for the first display window 63. On the other hand, the dark representation of the status bar 64 indicates that the first display window 63 displayed on the right side of the screen 60 is the currently active display window. However, the method of expressing the activated window is not limited thereto, and may be displayed through various methods such as displayed at the bottom of the display window.

FIG. 6B illustrates a process in which the first display window 63 is captured and displayed according to an input in which the user touches or long taps the capture button 65 in FIG. 6A. That is, when the activated first display window 63 is captured according to a user input, the captured image 631 of the first display window 63 overlaps the first display window 63. According to an embodiment of the present disclosure, the captured image 631 may be displayed at the same position as that of the first display window 63.

7 is a diagram illustrating a process of overlapping and displaying a captured image according to an embodiment of the present invention. As in FIG. 6, three display windows 61, 62, and 63 are displayed on the screen 60 of FIG. 7A, and the device 100 captures data in a bar displayed at the bottom of the screen 60. The capture button 65 for display is displayed.

When an input (or a long tap input or another predetermined input) for touching the capture button 65 is received by the user, the device 100 may display the screen shown in FIG. 7B. That is, unlike FIG. 6, the device 100 may capture not only the activated first display window 63 but also all of the plurality of display windows 61, 62, and 63 displayed on the screen 60.

Each of the captured images 611, 621, and 631 is displayed overlapping with each of the first display windows 61, 62, and 63 that are to be captured. That is, the captured image 631 for the first first display window 63 is in the first first display window 63, and the captured image 611 for the second first display window 61 is in the second agent. Each display window 61 may be overlapped and displayed.

8 is a diagram illustrating a process of overlapping and displaying a captured image according to an embodiment of the present invention. In the screen 60 illustrated in FIG. 8A, a full screen mode display window 66 having the same size as the screen 60 is displayed.

When the user touches the capture button 65 to capture data, the device 100 overlaps and displays the captured image on the first display window 66 in full screen mode. In this case, unlike the content described with reference to FIGS. 6 and 7, the device 100 may display the captured image smaller than the size of the first display window 66.

That is, as shown in FIG. 8B, the captured image 661 may be displayed in a partial area of the first display window 66 in a smaller size than the first display window 66. Meanwhile, according to an exemplary embodiment, a status bar 67 indicating that the captured image 661 is overlapped and displayed in a separate display window may be displayed together with the captured image 661.

9 is a diagram illustrating a process of determining a region to be captured in accordance with one embodiment of the present invention. In FIG. 9A, the captured image 91 is displayed on the left side of the screen 90, and the second display window 92 on which the captured image 91 is inserted is displayed on the right side of the screen 90. In the second display window, a memo application into which the captured image may be inserted may be executed.

Referring to FIG. 9A, a status bar is displayed at the top of the captured image 91, and the device 100 displays buttons corresponding to various functions for determining an area to be inserted in the status bar. 9A, a closed curve button 93 for selecting a region to be inserted as a closed curve and a figure button 94 for selecting the region to be inserted according to a predetermined figure are illustrated. Meanwhile, various buttons may be further displayed on the status bar in addition to these buttons, which will be described in detail with reference to FIG. 14.

FIG. 9B illustrates a process in which the user determines the area 931 to be cut out using the closed curve button 93 in FIG. 9A. That is, the device 100 may touch the closed curve button 93 and receive a drag input on the captured image 91. Subsequently, the device 100 determines an area 931 to be cut out according to the received drag input.

FIG. 9C illustrates a rectangular cropped area 941 having a predetermined size on the captured image 91 when the user determines the cropped area 941 using the figure button 94 in FIG. 9A. Meanwhile, the device 100 may display a button for receiving an input for adjusting the size of the region 941 to be cut out. In FIG. 9C, the device 100 displays small rectangles on the top, bottom, left, and right edges of the rectangular cropped area 941, indicating that the size of the cropped area 941 can be adjusted.

10 is a diagram illustrating an embodiment of the present invention for moving a captured image.

The device 100 captures data displayed on the first display window according to a user input of touching the capture button 830 displayed on the screen 801. Next, the device 100 determines an area 802 for inserting into the second display window based on the drag input of drawing a closed curve on the captured image. As shown in FIG. 8, the device 100 may darken an area other than the area 802 to be inserted, so that the area 802 to be inserted may be clearly distinguished. Of course, the device 100 may be displayed in black so that an area other than the area 802 for insertion is completely separated.

When an input of touching the region 802 for insertion for more than a predetermined time is received, the device 100 may display the region 803 to be cut out to be moved and inserted into the second display window. As described above with reference to FIG. 3, the device 100 may display smaller than the region 802 for inserting the size of the region 803 to be cut out. Accordingly, the user can easily check the region 803 to be cut out.

In addition, as shown in FIG. 8, the device 100 is not exactly overlapped with the region 802 for inserting the position of the region 803 to be cut out, but may be displayed such that the positions of the centers of the two regions are different from each other. Can be. In other words, the device 100 may change and display the position of the image 803 to be cropped so that the user can easily find it.

FIG. 11 is a diagram illustrating an example of modifying an area to be inserted in a captured image.

In FIG. 11A, the user determines an area 901 for insertion through a touch input that draws a closed curve with respect to the captured image displayed on the screen 900. Subsequently, the device 100 receives an input of touching and moving the region 901 to be inserted within a preset time to correct the position of the region 901 to be inserted from the user.

In FIG. 11B, the device 100 displays the region 902 for insertion where the position has been changed. That is, the device 100 changes the position of the region 901 for insertion according to the user input shown in FIG. 11A and displays it. Accordingly, the user can accurately specify the position of the region to be inserted from the captured image.

FIG. 12 illustrates another embodiment of moving the captured image to the second display window. The captured image 410 overlapping the first display window is shown.

In FIG. 12A, a captured image 410 and a second display window 420 shown overlapping the first display window are shown. The captured image 410 and the second display window 420 may display a status bar on which a button for controlling various functions and operations is displayed.

In the captured image 410 displayed on the left side, as the user input receiving unit 110 receives an input for selecting the capture button 430 from the user, the capture unit 120 captures data of the first display window to display the first display. It can be displayed overlapping the window.

Subsequently, the user input receiver 110 may receive an input for selecting a partial region 415 of the captured image 410 to move to the region where the second display window 420 is displayed.

In FIG. 12B, a process of the display 130 inserting and displaying the partial region 415 in the second display window 420 based on the received user input is illustrated. According to an embodiment of the present disclosure, in FIG. 12B, the display unit 130 may enlarge the partial area 415 of the captured image 410 on the captured image 410 in consideration of the display environment of the second display window 420. It can be inserted and displayed smaller.

According to another exemplary embodiment, the display 130 may display the partial region 415 of the captured image 410 at the position where the drop operation ends, based on the drag and drop input received by the user input receiver 110. It may be inserted and displayed (425).

12A and 12B illustrate an embodiment in which all captured images are inserted into the second display window, but FIGS. 13A and 13B illustrate an embodiment in which a portion of the captured image is inserted.

13A and 13B illustrate a process of moving a partial region 515 to the second display window 520 in the captured image 510 displayed overlapping the first display window, according to an exemplary embodiment. to be.

In FIG. 13A, the user input receiving unit 110 may receive a user input of drawing a closed curve among the captured images 510 captured by the capture unit 120. Subsequently, the capturer 120 may select a partial region 515 of the captured image 510 based on the received user input. As described above, the user input for selecting the partial region 515 may be a drag input that draws a closed curve along the edge of the partial region 515. Subsequently, the user input receiver 110 may receive an input for moving the selected partial region 515 to the second display window 520.

In FIG. 13B, the display 130 inserts and displays an image of the selected partial region 515 in the second display window 520 based on a user input for moving the partial region 515 (525). Accordingly, the second display window 520 may display a partial region 525 of the captured image 510.

When the content played in the first display window is a video, the capturer 120 may capture person information about a partial area of the video. That is, the capturer 120 may capture text information (such as a person's name or personal information) about a person appearing in the selected partial region 515. In addition, when the person appearing in the selected partial area 515 is an entertainer, the capturer 120 may capture URI data regarding the homepage address of the entertainer.

Subsequently, the display 130 may display the captured video information on the second display window 520 based on the received user input. That is, the display 130 may input the personal information of the person appearing in the selected partial region 515 into the second display window 520 in the form of text. In addition, the display unit 130 may link and display the homepage address of the entertainer appearing in the selected partial region 515 directly on the second display window 520. In addition, when the Internet address of the video is captured as URI data, the display 130 may insert and display the URI data and the intent data so that the video is executed directly on the second display window 520.

14 is a diagram illustrating an embodiment of displaying an editing tool for correcting an inserted capture image. The captured image 1401 captured in the first display window is displayed on the right side of the screen 1400 of FIG. 14. As described above with reference to FIG. 6, since the captured image 1401 is overlapped and displayed on the first display window, although not explicitly illustrated in FIG. 14, the device 100 may include the first image at the bottom of the captured image 1401. You can continue to display the display window. In addition, a second display window 1402 into which the captured image 1401 is to be inserted is displayed on the left side of the screen 1400 of FIG. 14.

At the top of the captured image 1401, various functions of selecting the region 1407 to be cropped for insertion into the second display window 1402 from the captured image 1401 are displayed. In addition to the closed curve button 1403 and the figure button 1404 described above with reference to FIG. 9, the device 100 cancels the captured image 1401 and displays a cancel image 1405 and a captured image for displaying a first display window on the screen. 1401 may display a complete button 1406 for storing all.

As described above with reference to FIGS. 10 and 12, the device 100 determines an area 1407 to be cut out through a user input of touching the figure button 1404 and, when a drag and drop input is received from the user, the area 1407 to be cut out. ) May be inserted into the second display window 1402. In FIG. 14, an inserted image 1408 is shown in a second display window 1402.

Meanwhile, a memo application supporting a function of inserting an image is illustrated in the second display window 1402 of FIG. 14. When the region 1407 to be cut out from the captured image 1401 is inserted, the device 100 may display an editing tool 1409 for correcting the inserted image 1408 at a predetermined position in the second display window.

The editing tool 1409 provides various functions for copying, cutting, deleting objects, and moving the inserted image 1408 in various directions, and the user can edit the editing tool 1409. ) Can be used to modify or edit the inserted image 1408.

Also shown in FIG. 14 is a tool tool 1410 that provides various functions separately from the image 1408 inserted in the second display window 1402. That is, the device 100 may edit or decorate the inserted image 1408 through the formula search, text input, and eraser functions displayed on the tool tool 1410.

FIG. 15 is a diagram illustrating another embodiment of the present invention for moving text displayed on the first display window 610 to the second display window 620.

In FIG. 15, a notepad application is executed in the first display window 610 and a calendar application is executed in the second display window 620. The user may want to share the memo recorded for the year-end party in the calendar application.

First, the user input receiving unit 110 may receive a user input of capturing a title 615 of a year-end party schedule from a user. Accordingly, the capture unit 120 may capture the title 615 of the year-end party schedule. Since the data is moved to the calendar application executed in the second display window 520, the capture unit 120 preferably captures the title 615 of the year-end party schedule as text.

The user input receiving unit 110 may receive a user input of moving the title 615 of the captured year-end party schedule to the second display window 620 from the user. Subsequently, the display 130 may insert and display the title 615 of the captured year-end party schedule into the second display window 620 based on the received user input.

According to an embodiment of the present disclosure, when the drag and drop operation, which is a received user input, ends in the December 2 field 625 of the second display window 620, the display 130 may display the title of the captured year-end party schedule ( 615 may be inserted and displayed in the December 2 field 625 of the second display window 620.

According to another embodiment, when capturing and moving the title 615 of the year-end party schedule, the controller 140 may automatically insert the details of the year-end party schedule into the second display window 620. Accordingly, the user may confirm details of the year-end party schedule through an input for selecting the field 625 of the December 2nd.

In addition, when there is link information corresponding to the title 615 of the year-end party schedule, the controller 140 may match the link information to the title 615 inserted into the second display window 620.

FIG. 16 is a diagram illustrating another exemplary embodiment of moving text displayed on the first display window 710 to the second display window 720. A notepad application is executed in the first display window 710, and an e-mail application is executed in the second display window 720.

According to the present embodiment, the user wants to send a memo 715 related to a volunteer schedule by e-mail among a plurality of memos displayed on the memo pad application. The user input receiving unit 110 may receive a user input of selecting a memo 715 regarding a volunteer schedule from the user. As described above, the user input receiving unit 110 may receive various types of user input, such as an input dragging along an edge of the rectangular area or an input for selecting two or more vertices to determine the rectangular area.

The capturer 120 may capture a memo 715 about a volunteer schedule according to a user input. In relation to the present embodiment, since the user wants to send the captured data by e-mail, the capture unit 120 may capture a memo 715 about the volunteer schedule as an image or as text. .

Subsequently, when the user input receiving unit 110 receives an input for moving the captured data to the second display window 720, the display unit 130 inserts the captured data into the second display window 720 to display it. Can be. In this process, when the drop operation during the user input is terminated in the content field 725 of the second display window 720, the display unit 130 writes a memo 715 about the volunteer schedule to the content field 725. Can be inserted and displayed.

FIG. 17 is a diagram illustrating another embodiment of the present invention for moving a file displayed on a first display window 810 to a second display window 820.

In relation to the present embodiment, a folder search application is executed in the first display window 810 and an e-mail application is executed in the second display window 820. The user wants to send the file 10033.jpg file 815 to the e-mail from the device 100 as an attachment.

The user input receiver 110 may receive an input for selecting the photo 0033.jpg file 815 from the user. The user input receiver 110 may receive a combination of a hold operation and a drag and drop operation of pressing an area of a screen corresponding to a file for a predetermined time or more.

The capture unit 120 may capture the photo 0033.jpg file 815 based on a user input. In connection with this embodiment, the user wants to attach the photo 0033.jpg file 815 itself to an e-mail, and captures an image or plain text of the screen corresponding to the photo 0033.jpg file 815. I don't want to. Accordingly, the capture unit 120 may capture URI data corresponding to the photo 0033.jpg file 815. According to an embodiment, based on the hold input received by the user input receiver 110, the capturer 120 may capture URI data. In addition, when the user input receiver 110 receives a drag and drop operation, the capturer 120 may capture intent data to which the photo 0033.jpg file 815 is attached as the drop operation ends.

The display 130 may insert and display the captured URI data and the intent data into the second display window 820 based on the received user input. That is, as the drop operation ends in the attachment field of the second display window 820, URI data and intent data regarding the photo 0033.jpg file 815 are inserted into the second display window 820. Subsequently, the display 130 may display the photo 0033.jpg file 825 to be attached to the second display window 820.

18 is a diagram illustrating another embodiment of the present invention for moving text displayed on the first display window 920 to a plurality of display windows.

In relation to the present embodiment, a web browser application is executed in the first display window 920 and a notepad application is executed in the second display windows 910 and 930. The user wants to write some of the content of the news displayed in the web browser application to the notepad.

First, an operation process between the first display window 920 and the first second display window 910 will be described. The user input receiver 110 may receive an input for capturing the content 925 of the news displayed on the first display window 920. Accordingly, the capture unit 120 may capture the content 925 of the news as either image or text. In Fig. 13, the case of capturing as text is shown.

The capturer 120 may capture URI data and intent data related to time, place, and / or information of a website among the contents 925 of the news. Subsequently, as the user input receiver 110 receives an input for moving the captured data, the display 130 may insert the captured data into the first second display window 910 to display 915. .

Next, an operation process between the first display window 920 and the second second display window 930 will be described. The user input receiver 110 may receive an input for capturing the title 927 of the news displayed on the first display window 920. Accordingly, the capture unit 120 may capture the title 927 of the news as either an image or text, which is illustrated in FIG. 13.

The capturer 120 may capture text data, URI data, and intent data related to the title 927 of the news. Subsequently, as the user input receiver 110 receives an input for moving the captured data, the display 130 may insert the captured data into the second second display window 930 to display 935. .

Accordingly, when the user input receiving unit 110 receives an input for selecting a title 935 of the news displayed on the second second display window 930 from the user, the device 100 inserts the inserted URI data and the intent data. Through it is possible to provide the content of the news corresponding to the user.

Meanwhile, the above-described method may be written as a program executable on a computer, and may be implemented in a general-purpose digital computer which operates the program using a computer readable medium. In addition, the structure of the data used in the above-described method can be recorded on the computer-readable medium through various means. Program storage devices that may be used to describe storage devices that include executable computer code for performing the various methods of the present invention should not be understood to include transient objects, such as carrier waves or signals. do. The computer readable medium includes a storage medium such as a magnetic storage medium (eg, a ROM, a floppy disk, a hard disk, etc.), an optical reading medium (eg, a CD-ROM, a DVD, etc.).

It will be understood by those skilled in the art that embodiments of the present invention can be implemented in a modified form without departing from the essential characteristics of the above description. Therefore, the disclosed methods should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the detailed description of the invention, and all differences within the equivalent scope should be construed as being included in the scope of the present invention.

Claims (15)

In the captured image generation method for the display window displayed on the screen,
Displaying a first display window and a second display window on the screen;
Capturing data displayed on the first display window based on a user input;
Displaying an image of the captured data on the first display window;
Inserting the captured data into an application executed in the second display window based on a gesture input by a user; And
And adjusting the size of the captured data on the screen according to the screen size of the second display window and displaying the captured data on the second display window.
The method of claim 1,
The first display window is one of a plurality of display windows displayed on the screen,
The displaying of the captured data image may include displaying the captured data image by overlapping each of the plurality of display windows. The method of claim 1,
The displaying of the captured data image may include displaying the captured data image on a portion of the first display window when the first display window is a full screen mode display window. . The method of claim 1,
Inserting at least a portion of the captured data image into the second display window. The method of claim 4, wherein
Inserting at least a portion of the captured data image comprises touching at least a portion of the captured data image for a set time and dragging the touch toward the second display window based on at least a portion of the captured data image. Inserting a second image into the second display window. The method of claim 4, wherein
Determining a predetermined area of the captured data image displayed on the screen; And
Cropping the image of the determined predetermined area;
And inserting at least a portion of the captured data image into the second display window, inserting the cropped image into the second display window. The method of claim 6,
Determining the predetermined area,
Displaying an area selected by a user on the captured data image displayed on the screen; And
Modifying the displayed area;
And the captured data image of the modified area is inserted into the second display window. The method of claim 7, wherein
Modifying the displayed area,
And a user touches the displayed area and drags the touch within a predetermined time from the touched time point to move the displayed area to another area of the captured data image. The method of claim 7, wherein
Modifying the displayed area,
And changing the size of the displayed area as the user pinches or unpinches the displayed area. The method of claim 7, wherein
And a region selected by the user is selected based on a touch input of the user drawing a closed curve on the captured data image displayed on the screen. The method of claim 6,
The determining of the predetermined area may include determining that the predetermined area is an image to be cropped as a user's touch on the predetermined area is maintained for a predetermined time or more. The method of claim 1,
The determining of the first display window may include:
And as a predetermined button on the screen is touched, determining an activated window among a plurality of display windows as the first display window. The method of claim 1,
And the captured data image is overlapped at the same position as the position of the content displayed on the first display window in the same size as the first display window. A device for generating a captured image for a display window displayed on a screen,
A user input unit to receive a user input to the device;
A capture unit which determines a first display window to be captured among a plurality of display windows displayed on the screen, and captures data displayed on the determined first display window based on the user input;
A display unit configured to overlap the captured data image on the first display window; And
Including an inserting unit for inserting the captured data by the user's gesture input to the application running on the second display window,
And the display unit adjusts the size of the captured data on the screen according to the screen size of the second display window and displays the captured image on the second display window. A computer-readable recording medium having recorded thereon a program for executing the method of claim 1 by a computer.

Images