KR20110044086A - Mobile terminal and method for processing both captured image and related information - Google Patents

Abstract

PURPOSE: A method for processing both captured image and related information in a mobile terminal is provided to improve the search efficiency of the captured image using the related information. CONSTITUTION: An input unit(11) receives the selection signal of a user. A storage unit(13) receives and saves information. A display unit(17) displays an image. If a capture request signal is inputted through the input unit, a control unit(19) captures an image on the display unit. The control unit obtains the related information about an object in the captured image by performing a search. The controller stores the related information with the captured image to the storage unit.

Description

Mobile terminal and method for processing both captured image and related information

The present invention relates to a mobile terminal and a method for processing information thereof, and more particularly, to automatically capture and provide related information about a subject when capturing a still image or a video through the mobile terminal to store the captured image and the related information together. The present invention relates to a mobile terminal capable of processing the same, and a processing method of the captured image and related information thereof.

Information and communication technology using wired and wireless communication networks is growing very fast in a short period of time due to new technologies and functions that stimulate consumer purchasing. In particular, due to the development of the technology of the mobile terminal, beyond the step of performing a simple application, various applications are installed to meet the needs of users. As a result, the user can use voice information, text information, video information, MP3 (Moving Picture Experts Group (MPEG) layer 3), games, and the like anytime and anywhere through the mobile terminal.

Such mobile terminals provide a function of capturing still images by capturing photos and capturing functions during video playback such as broadcasting or a movie. The portable terminal stores the captured image in the storage unit using a file name automatically assigned or a file name input by the user. When the user calls the captured image stored in the storage unit, the portable terminal displays the captured image stored in the form of a file name or a thumbnail.

As described above, since the conventional captured image is distinguished by a file name, discrimination between captured images is inferior. Because of this, the user has to check the file name or thumbnail in order to find the capture image that the user wants, which is cumbersome and takes a lot of time to find the captured image, and thus the search efficiency is reduced. Since the captured image merely provides image information and does not provide information related to the captured image, utilization of the captured image is also reduced.

Accordingly, an object of the present invention is to provide a mobile terminal capable of improving the retrieval efficiency and utilization of the captured image, and a method of processing the captured image and related information thereof.

It is another object of the present invention to provide a mobile terminal capable of storing a captured image and related information by automatically searching for and providing related information about a subject when capturing a still image or a moving image through the portable terminal, and a captured image and related information thereof. It is to provide a processing method of.

In order to achieve the above object, the present invention provides a portable terminal comprising an input unit, a storage unit, a display unit and a control unit. The input unit receives a user's selection signal. The storage unit stores the information. The display unit displays an image. When the capture request signal is input through the input unit, the controller captures an image displayed on the display unit to obtain a captured image, and retrieves and acquires relevant information about at least one subject included in the captured image from the storage unit. The related information is stored together with the captured image in the storage unit.

According to an embodiment of the present invention, if a capture request signal is input while the image is displayed on the display unit, the mobile terminal captures the image to obtain a captured image, and the mobile terminal includes at least one subject included in the captured image. An information acquisition step of searching for and acquiring relevant information on a mobile terminal, and the mobile terminal stores the related information together with the captured image.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, in the image acquisition step, the image is an image being photographed, a web page, a web page, a swap page, a still image and a moving image. It may include at least one of the.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the information obtaining step includes the step of the mobile terminal receiving a selection signal for at least one of the subjects included in the captured image, the portable The terminal may include searching for and acquiring relevant information about the selected subject in a storage unit, and displaying, by the portable terminal, the related information on the screen together with the captured image.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the information obtaining step includes the step of extracting a subject that the user can select from a subject included in the captured image by the mobile terminal, Receiving a selection signal for at least one subject from among the extracted subjects, and searching and acquiring, by the portable terminal, related information about the selected subject in a storage unit, the portable terminal together with the captured image; It may include the step of displaying on the screen.

In the method of processing a captured image and related information of a mobile terminal according to the present invention, the extracting may further include displaying the extracted subject to be distinguished from other subjects by the portable terminal.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the related information may be displayed by being superimposed on the captured image or displayed in a PIP.

In the method of processing a captured image and related information of a mobile terminal according to the present invention, the related information may be displayed adjacent to the subject in the displaying step.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the related information may be displayed as an icon or a list in the displaying step.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the related information includes basic information related to a capture situation and information for each subject found in relation to the subject.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the storing step includes the mobile terminal displaying the related information on the screen together with the captured image, wherein the mobile terminal displays a user's selection signal. The method may include receiving and selecting the acquired related information by the input, and storing the related information selected by the portable terminal together with the captured image.

In the method for processing the captured image and the related information of the mobile terminal according to the present invention, the selection signal of the user is input in at least one of the key input, touch input, voice input, and gesture input in the step of selecting Can be.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the mobile terminal may designate at least one of the related information as a file name of the captured image in the storing step.

The method for processing a captured image and related information of a mobile terminal according to the present invention includes the steps of: displaying, by a portable terminal, a captured image stored in the storage unit when a call signal of a captured image is input, which is performed after the storing step; The method may further include extracting and displaying a captured image including the specific related information from the captured image stored in the storage unit when the search is input by inputting relevant information.

The method for processing a captured image and related information of a mobile terminal according to the present invention includes the steps of: displaying, by a portable terminal, a captured image stored in the storage unit when a call signal of a captured image is input, which is performed after the storing step; The method may further include sorting and displaying the stored captured image according to the specific related information when the related information is inputted to request alignment.

In the method for processing a captured image and related information of a mobile terminal according to the present invention, the specific related information may be related information having an element for aligning the stored captured image.

Therefore, the mobile terminal according to the present invention has an advantage of automatically storing related information together with the captured image by automatically searching for and providing related information about a subject when capturing a still image or a moving image through the mobile terminal.

Since the mobile terminal according to the present invention supports the search, alignment, and interworking with other programs using the related information stored with the captured image, it is possible to improve the search efficiency and utilization of the captured image using the related information. have.

Hereinafter, with reference to the accompanying drawings will be described in detail an embodiment of the present invention.

As shown in FIG. 1, the mobile terminal 10 according to an exemplary embodiment of the present invention captures by searching for and providing related information about a subject included in the captured image when capturing the image displayed on the display unit 17. Store relevant information with images. That is, when the capture request signal is input while the image is displayed on the display unit 17, the portable terminal 10 captures the image to obtain the captured image. The mobile terminal 10 searches for and obtains relevant information about at least one subject included in the captured image. The mobile terminal 10 also stores the relevant information together with the captured image. At this time, the mobile terminal 10 includes a digital broadcasting receiver such as a mobile communication terminal, a notebook, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital multimedia broadcasting (DMB), or a digital video broadcasting (DVB). can do.

The portable terminal 10 according to the present embodiment includes an input unit 11, a storage unit 13, a display unit 17, and a control unit 19, and further includes a camera unit 15 and a sound source processing unit 18. It may include.

The input unit 11 provides a plurality of keys for the operation of the mobile terminal 10, and generates and transmits a selection signal according to the user's key selection to the controller 19. The user may request an image display on the screen of the display unit 17 through the input unit 11 and input a capture request signal for the displayed image. In addition, the user may input a selection signal for processing the captured image and the related information through the input unit 11. In this case, the input unit 11 may be a keypad, a pointing device such as a touch pad, or an input device such as a touch screen.

The storage unit 13 stores a program necessary for controlling the operation of the mobile terminal 10 and information generated during execution of the program. The storage unit 13 stores an execution program for processing the captured image and related information. The information stored in the storage unit 13 includes internal information that is stored by default at the manufacturing stage of the mobile terminal 10 and external information that a user inputs or receives from an external terminal during use of the mobile terminal 10. do. At this time, the information is divided or duplicated in the phone book, photo album, diary or folder according to the attribute. The folder may store text information or search information. The external information may include a captured image in which related information is stored.

The camera unit 15 performs a photographing function of a subject, and in particular, acquires a surrounding image for simultaneously storing a captured image and related information. In this case, the camera unit 15 may include an image sensor, a signal processor, and an image processor. The image sensor converts the optical signal of the captured image into an analog signal. The signal processor converts an analog signal into a digital signal. The image processor may process the image signal input through the signal processor to obtain image data, and output the acquired image data through the controller 19 or the display unit 17 or store the image data in the storage unit 13.

The display unit 17 displays information stored in the storage unit 13 including various function menus executed in the mobile terminal 10. The display unit 17 displays an image under the control of the controller 19. In this case, the image may include an image captured by the camera unit 15, a web page, a swap page, a still image or a moving image. The display unit 17 may display a captured image or display related information together with the captured image under the control of the controller 19. As the display unit 17, a liquid crystal display (LCD) or a touch screen may be used. The touch screen simultaneously serves as a display device and an input device.

The sound source processing unit 18 outputs the sound source to the speaker under the control of the control unit 19, and digitizes the sound source input to the microphone and transmits it to the control unit 19. In particular, the sound source processing unit 18 may receive a user's selection signal as a voice and transmit it to the control unit 19.

The controller 19 is a microprocessor that performs the overall control operation of the mobile terminal 10. The controller 19 controls the processing of the captured image including simultaneous storage of the captured image and related information. That is, when the capture request signal is input through the input unit 11, the controller 19 captures an image displayed on the display unit 17 to obtain a captured image. The controller 19 searches for and acquires the relevant information about the at least one subject included in the captured image from the storage 13. The controller 19 stores the related information acquired together with the captured image in the storage unit 13.

In this case, at least one subject is included in the captured image. The subject may be a person, an animal, or an object. The related information may include basic information related to a capture situation and information about each subject found in relation to a subject. The basic information may include country, place, time, subject ID, weather condition, and the like. Subject-specific information may include a name, a trade name, a relationship with oneself, a recipe, an emotional state, a price, a place of purchase, a brand, and the like. For example, when the subject is a person, the information for each subject may include a name, a relationship with oneself, an emotional state, and the like. When the subject is an animal or an object, the information for each subject may include a name, a trade name, a recipe, a price, a place of purchase, a brand, and the like.

The controller 19 may obtain related information as follows. That is, the controller 19 receives a selection signal for at least one of the subjects included in the captured image through the input unit 11. The controller 19 searches for and acquires relevant information about the selected subject from the storage unit 13. The controller 19 displays the related information acquired along with the captured image on the display unit 17. That is, the controller 19 extracts the selected subject from the captured image by using the object extraction algorithm. The controller 19 may recognize the extracted subject through an image search algorithm, an image recognition algorithm, or a text recognition algorithm. The controller 19 searches for and acquires the related information about the recognized subject from the storage 13. For example, when the subject is a person, the controller 19 extracts a person picture corresponding to the subject from among the person pictures stored in the phone book, and obtains the name, phone number, e-mail address, etc. of the person matching the extracted person picture as related information. can do. In addition, the controller 19 may search for and obtain a schedule related to the person from the diary based on the acquired person's name or phone number.

In this case, the related information may be displayed by being superimposed on the captured image or displayed as a picture in picture (PIP). The related information may be displayed adjacent to the subject or in a list form. The related information may be displayed as an icon or text.

On the other hand, the control unit 19 extracts a subject that the user can select from among the subjects included in the captured image and distinguishes the extracted subject from other subjects so that the user can easily identify a subject that can be selected from among the subjects included in the captured image. Can be displayed. For example, the controller 19 may display the extracted subject by changing color or brightness.

The controller 19 may store related information together with the captured image as follows. In a state in which the relevant information is displayed on the screen together with the captured image, the controller 19 selects the related information obtained by receiving the user's selection signal. When the storage request signal is input through the input unit 11, the controller 19 stores the selected related information together with the captured image.

In this case, when the user selects the relevant information according to the user's selection signal, the controller 19 may select related information to be stored together with the captured image or delete unnecessary related information. That is, only a selection operation may be performed, only a deletion operation may be performed, or both selection and deletion operations may be performed. The user's selection signal may be input in at least one of a key input, a touch input, a voice input, and a gesture input. For example, when a user's selection signal is input by a touch input method, the touch is a selection operation and the flick is a deletion operation.

The controller 19 may designate at least one of the selected related information as a file name of the captured image. For example, when the related information includes the name and the trade name of the person, the controller 19 may designate the file name by combining the name and the trade name of the person.

As described above, since the mobile terminal 10 stores relevant information together with the captured image, the mobile terminal 10 can support the search, alignment, and interworking with other programs of the captured image.

That is, a method of searching for a captured image using related information will be described below. When the call signal of the captured image is input through the input unit 11, the controller 19 displays the captured image stored in the storage unit 13. When requesting a search by inputting specific related information through the input unit 11, the controller 19 extracts and displays the captured image including the specific related information from the storage unit 13.

A method of arranging captured images using related information is as follows. When the call signal of the captured image is input through the input unit 11, the controller 19 displays the captured image stored in the storage unit 13. When requesting an alignment by inputting specific related information through the input unit 11, the controller 19 sorts and displays the stored captured image according to the specific related information. In this case, when displaying the captured image according to the call signal of the captured image, the controller 19 may display only the captured image or display related information on the display unit 17 together with the captured image. Specific related information is related information having elements that can align stored captured images. For example, the relevant information may be country, place, time, weather condition, name, price, and the like.

The linked program may include a phone book, a photo album, a diary, a video editing, or a text editing application.

As described above, since the mobile terminal 10 automatically searches for and provides relevant information about a subject when capturing an image, the portable terminal 10 can simultaneously store the related information together with the captured image. Since the mobile terminal 10 according to the present embodiment supports searching, sorting, and interworking with other programs using captured information stored with the captured image, it is possible to improve the search efficiency and utilization of the captured image using the related information. Can be improved.

The method of processing the captured image and the related information of the mobile terminal according to the present embodiment will be described with reference to FIGS. 1 to 5 as follows. 2 is a flowchart illustrating a method of processing a captured image and related information of a mobile terminal according to an embodiment of the present invention. 3 is a detailed flowchart according to an example of the information obtaining step of FIG. 2. 4 is a detailed flowchart according to another example of the information obtaining step of FIG. 2. FIG. 5 is a detailed flowchart according to a step of storing related information together with the captured image of FIG. 2.

First, in operation S21, the mobile terminal 10 displays an image on the display unit 17 according to a user's selection signal through the input unit 11. In this case, the image may include an image captured by the camera unit 15, a web page, a swap page, a still image or a moving image.

Next, in step S23, the mobile terminal 10 determines whether a capture request signal is input. In this case, the capture request signal of the user may be input through a key input, a touch input, a voice input, or a gesture input.

When the capture request signal is not input as a result of the determination in step S23, the mobile terminal 10 maintains the display state of step S21.

When the capture request signal is input as a result of the determination in step S23, in step S25, the mobile terminal 10 captures an image and displays the captured image on the display unit 17. That is, the mobile terminal 10 captures the image displayed on the display unit 17 at the time when the capture request signal is input to obtain the captured image. The mobile terminal 10 displays the obtained captured image on the display unit 17.

Next, in operation S27, the mobile terminal 10 searches for and obtains relevant information about at least one subject included in the captured image.

In this case, an example of step S27 will be described in detail with reference to FIG. 3.

In operation S35, the mobile terminal 10 determines whether a selection signal for at least one of the subjects included in the captured image is input. In this case, the selection signal of the subject may be input through a user's key input, touch input, voice input, or gesture input.

When the selection signal is not input as a result of the determination in step S35, the mobile terminal 10 maintains the display state of the obtained captured image.

When the selection signal is inputted as a result of the determination in step S35, in operation S37, the mobile terminal 10 retrieves and acquires the relevant information about the selected subject from the storage unit 13. That is, the mobile terminal 10 extracts the selected subject from the captured image by using an object extraction algorithm. The mobile terminal 10 recognizes the extracted subject through an image search algorithm, an image recognition algorithm, a text recognition algorithm, and the like, and retrieves and obtains the related information about the recognized subject from the storage unit 13. For example, when the subject is a person, the mobile terminal 10 extracts a portrait picture corresponding to the subject from among the portrait pictures stored in the phone book and uses the name, phone number, e-mail address, etc. of the person matching the extracted portrait picture as related information. Can be obtained. In addition, the mobile terminal 10 may search for and obtain a schedule related to the person from the diary based on the acquired person's name or phone number.

In this case, step S37 may be performed as follows. That is, the mobile terminal 10 may perform step S37 whenever the selection signal for the subject is input in step S35. Alternatively, after selecting all the subjects to acquire the relevant information from the captured image in step S35, the mobile terminal 10 may receive the user's search request signal and perform step S37 on the selected subjects collectively.

In operation S39, the mobile terminal 10 displays relevant information obtained together with the captured image. In this case, the mobile terminal 10 may display the related information superimposed on the captured image or may be displayed in a PIP. The related information may be displayed adjacent to the subject or in a list form. The related information may be displayed as an icon or text.

Another example of step S27 will be described in detail with reference to FIG. 4.

First, in step S31, the mobile terminal 10 extracts a subject that the user can select from the captured image. In operation S33, the mobile terminal 10 displays the extracted subject to be distinguished from other subjects. That is, the mobile terminal 10 extracts a subject that the user can select from among the subjects included in the captured image so that the user can easily identify a subject that can be selected from among the subjects included in the captured image, and extracts the extracted subject from another subject. Can be distinguished. For example, the mobile terminal 10 may display the extracted subject differently from other subjects by changing color or brightness. Meanwhile, step S31 may be automatically performed after step S25 or may be performed according to a user's selection signal.

Since steps S35 through S39 proceed in the same manner as in FIG. 3, detailed descriptions thereof will be omitted. At this time, since the mobile terminal 10 displays the subject that can be selected by the user to be distinguished from the subject that cannot be selected, the user can easily select the subject and search time due to the selection of the subject that cannot provide related information. This minimizes the waste of resources such as batteries.

In operation S29, the mobile terminal 10 stores relevant information together with the captured image. The mobile terminal 10 will be described in detail with reference to FIG. 5.

First, in a state in which the relevant information is displayed together with the captured image on the display unit 17, the mobile terminal 10 selects the relevant information obtained by receiving the user's selection signal in step S41. That is, the user may perform an input operation of selecting relevant information to be stored together with the captured image or deleting unnecessary information. That is, only a selection operation may be performed, only a deletion operation may be performed, or both selection and deletion operations may be performed. The user's selection signal may be input in at least one of a key input, a touch input, a voice input, and a gesture input.

Next, in step S43, the mobile terminal 10 determines whether a storage request signal is input from the user.

If the storage request signal is not input as a result of the determination in step S43, the portable terminal 10 determines again whether or not the delete request signal is input in step S44. If the delete request signal is not input as a result of the determination in step S44, the mobile terminal 10 performs step S41. When the delete request signal is input as a result of the determination in step S44, the mobile terminal 10 deletes the captured image in step S49. Meanwhile, the mobile terminal 10 may display a standby screen after deleting the captured image or display an image according to step S21.

When the storage request signal is input in step S43, the mobile terminal 10 designates at least one of the selected related information as a file name of the captured image in step S45. For example, when the related information includes the name of the person and the store name, the mobile terminal 10 may designate the file name by combining the name of the person and the store name.

In operation S47, the mobile terminal 10 stores the selected related information together with the captured image in the storage unit 13.

The method of processing the captured image and the related information of the mobile terminal according to the present embodiment will be described with reference to the screen examples of FIGS. 1 and 6A to 8 as follows. 6A to 6E are diagrams illustrating screens according to a method of processing a captured image and related information of a mobile terminal according to an embodiment of the present invention. 7 is an exemplary view of a screen according to a sorting step among processing methods of a captured image and related information of a mobile terminal according to an embodiment of the present invention. FIG. 8 is an exemplary view of a screen according to a searching step of a method of processing a captured image and related information of a mobile terminal according to an exemplary embodiment of the present invention.

First, referring to FIGS. 1 and 6A to 6E, an operation of capturing an image and storing the image with related information will be described below. In this case, the display unit 17 illustrates a touch screen.

First, when a capture request signal is input from the user 60 while the image is displayed through the display unit 17 of the mobile terminal 10, the mobile terminal 10 captures the captured image 51 as shown in FIG. 6A. Is displayed on the display unit 17. Next, when the user selects a subject through the touch input in the captured image 51, the mobile terminal 10 searches for and acquires the relevant information about the selected subject in the storage unit 13. In this case, the display unit 17 may display the save and delete buttons 55 and 57 together with the captured image 51.

As illustrated in FIG. 6B, the mobile terminal 10 displays the relevant information 53 of the selected subject together with the captured image 51. In this case, the mobile terminal 10 may display the related information 53 overlapping the captured image 51. The mobile terminal 10 may display subject-specific information 53e of the related information 53 in proximity to the corresponding subject, and display basic information 53a, 53b, 53c, and 53d on the background image between the subjects. Although the mobile terminal 10 discloses an example in which the location 53a, the weather condition 53b, the time 53c, and the country 53d are displayed with the basic information 53a, 53b, 53c, 53d, the present invention is not limited thereto. no. The place 53a may be displayed in text, and the weather state 53b, the time 53c, and the country 53d may be displayed in an icon. At this time, an example in which “DD distance” is displayed as the location 53a has been disclosed.

Next, as shown in FIG. 6C, when the user flicks specific related information 53a, the mobile terminal 10 deletes the related related information as shown in FIG. 6D. For example, when the user flicks the "DD distance", the mobile terminal 10 deletes the "DD distance" displayed on the display unit 17. FIG.

As described above, the user may select and select related information 53 to be stored together with the captured image 51 through a touch input such as a flick as illustrated in FIGS. 6C and 6D.

Next, when the user touches the save button 55, the mobile terminal 10 displays a popup window 57 for confirming whether to save the captured image 51 and the selected related information 53 together, as shown in FIG. 6E. do. In this case, the mobile terminal 10 may designate "AA, BB, CC" among the selected related information as the file name 57a of the captured image. In addition, confirmation buttons 57b and 57c may be displayed together with the popup window 57 such as “yes” or “no”.

When the yes button 57b is touched, the mobile terminal 10 stores the captured image 51 and the selected related information 53 together in the storage unit 13. On the other hand, when the NO button 57c is touched, the mobile terminal 10 returns to the display state of FIG. 6D. Alternatively, the mobile terminal 10 may end the image capturing and display the standby screen on the display unit 17.

Next, referring to FIGS. 1 and 7, the sorting process of the capture image and the related information of the mobile terminal 10 according to an embodiment of the present invention will be described as follows.

As illustrated in FIG. 7A, the portable terminal 10 displays a photo album on the display unit 17 according to a user's selection signal. In this case, the photo album may include a title window 60, alignment windows 62 and 63, and a list window 65. For example, “MY PHOTO ALBUM” may be displayed on the title window 60. Alignment windows 63 and 62 are displayed below the title window 60. The alignment windows 62 and 63 include an input window 63 for inputting a character string and an alignment button 62. In the list window 65, a plurality of captured images stored in the storage unit 13 are displayed. In this case, the captured image includes a thumbnail 68 of the captured image and a file name 69 of the captured image. The captured image may be displayed in the list window 65 in a sorted or random manner according to the date, name, weather condition, and the like. When the mobile terminal 10 stores more captured images than the number of captured images that can be displayed in the list window 65 at once, the mobile terminal 10 includes a scroll bar 67 for quickly searching the captured images. 65 may be displayed on one side. In this case, the scroll bar 67 may be arranged on the screen so that the captured image may scroll in the same direction as the aligned direction. For example, when the captured image is aligned in the vertical or horizontal direction, a scroll bar 67 for scrolling the captured image in the vertical or horizontal direction may be disposed on the screen. In the present embodiment, since the captured image is displayed on the display unit 17 in the vertical direction, an example in which the scroll bar 67 is installed in the vertical direction on the right side of the display unit 17 is disclosed.

As shown in FIG. 7B, when the user inputs "Name" into the input window 63 and touches the alignment button 62, the mobile terminal 10 as shown in FIG. The captured images are sorted and displayed in order of "Name". In this case, when the captured images are sorted in the order of "name", the captured images may be sorted and displayed according to various sorting methods such as alphabetical, alphabetical or numerical order. For example, FIG. 7C discloses an example in which the mobile terminal 10 is arranged in the order of "name" but the captured images are arranged and displayed in alphabetical order.

Meanwhile, as shown in FIG. 7A, when the alignment button 62 is pressed while nothing is input to the input window 63, the mobile terminal 10 displays the alignment button (as shown in FIG. 8A). 62 may be switched to the search button 61 for display. Alternatively, the mobile terminal 10 may include a sorting menu window as shown in FIG. 7 and a search menu window as shown in FIG. 8 as sub menus of the photo album.

Next, referring to FIGS. 1 and 8, the searching of the captured image and the related information processing method of the mobile terminal 10 according to an exemplary embodiment of the present invention will be described as follows.

As illustrated in FIG. 8A, the mobile terminal 10 displays a photo album on the display unit 17 according to a user's selection signal. In this case, the photo album includes a title window 60, search windows 61 and 63, and a list window 65. For example, “MY PHOTO ALBUM” may be displayed in the title window 60. Search boxes 61 and 63 are displayed below the title window 60. The search windows 61 and 63 include an input window 63 for inputting a character string and a search button 61. The list window 65 displays a plurality of captured images stored in the storage unit 13. At this time, since the display form of the list window 65 is the same as that of Fig. 7A, detailed description thereof will be omitted.

As shown in FIG. 8 (b), when the search button 61 is touched after inputting “BB” into the input window 63, the mobile terminal 10 displays the file name as shown in FIG. 8 (c). The captured image included in "BB" is extracted and displayed. In this case, "BB" is displayed to be distinguished from other related information. For example, although the example in which the color of “BB” is shown in bold in FIG. 8 (c) has been disclosed, it may be displayed by adjusting the brightness.

And the mobile terminal 10 according to the present embodiment, as shown in Figs. 7 and 8, but disclosed an example of linking the captured image including the relevant information with the photo album, but is not limited to this. For example, the mobile terminal 10 may link the captured image including related information with a phone book, a diary, an image editing or document editing application, and the like.

On the other hand, the embodiments of the present invention disclosed in the specification and drawings are merely presented specific examples to aid understanding, and are not intended to limit the scope of the present invention. In addition to the embodiments disclosed herein, it is apparent to those skilled in the art that other modifications based on the technical idea of the present invention may be implemented.

The present invention relates to a mobile terminal and a method for processing the captured image and related information thereof, wherein the mobile terminal automatically retrieves and provides relevant information about a subject when capturing a still image or a video displayed on a display unit, thereby capturing the captured image. And related information can be stored at the same time. In addition, since the mobile terminal supports the search, sorting, and interworking with other programs using the related information stored with the captured image, it is possible to improve the search efficiency and utilization of the captured image using the related information.

1 is a block diagram illustrating a configuration of a mobile terminal for processing a captured image and related information according to an embodiment of the present invention.

2 is a flowchart illustrating a method of processing a captured image and related information of a mobile terminal according to an embodiment of the present invention.

3 is a detailed flowchart according to an example of the information obtaining step of FIG. 2.

4 is a detailed flowchart according to another example of the information obtaining step of FIG. 2.

FIG. 5 is a detailed flowchart of storing related information together with the captured image of FIG. 2.

6 is an exemplary view of a screen according to a method of processing a captured image and related information of a mobile terminal according to an embodiment of the present invention.

7 is an exemplary view of a screen according to a sorting step among processing methods of a captured image and related information of a mobile terminal according to an embodiment of the present invention.

8 is an exemplary view of a screen according to a searching step of a method of processing a captured image and related information of a mobile terminal according to an embodiment of the present invention.

Description of the Related Art [0002]

10: mobile terminal 11: input unit

13: storage unit 15: camera unit

17: display unit 18: sound source processing unit

19: control unit

Claims (16)

An input unit configured to receive a user selection signal; A storage unit for storing information; A display unit for displaying an image; When a capture request signal is input through the input unit, an image displayed on the display unit is captured to obtain a captured image, and the storage unit searches for and acquires relevant information about at least one subject included in the captured image, and captures the captured image. A controller configured to store the related information together with an image; A mobile terminal comprising a. An image acquisition step of acquiring a captured image by the mobile terminal by capturing the image if a capture request signal is input while the image is displayed on the display unit; An information obtaining step of the mobile terminal searching for and obtaining relevant information about at least one subject included in the captured image; A storage step of storing, by the portable terminal, the related information together with the captured image; Processing method of the captured image and the related information of the mobile terminal comprising a. The method of claim 2, wherein in the image acquisition step, And the at least one selected from among an image being photographed, a web page, a swap page, a still image being played, and a moving image. The method of claim 2, wherein the obtaining of the information comprises: Receiving, by the portable terminal, a selection signal for at least one of the subjects included in the captured image; The mobile terminal searching for and obtaining information related to the selected subject in a storage unit; Displaying, by the portable terminal, the related information together with the captured image on a screen; Processing method of the captured image and the related information of the mobile terminal comprising a. The method of claim 2, wherein the obtaining of the information comprises: Extracting, by the portable terminal, a subject selectable by a user from among subjects included in the captured image; Receiving, by the portable terminal, a selection signal for at least one of the extracted subjects; The mobile terminal searching for and obtaining information related to the selected subject in a storage unit; Displaying, by the portable terminal, the related information together with the captured image on a screen; Processing method of the captured image and the related information of the mobile terminal comprising a. The method of claim 5, wherein the extracting step, Displaying, by the portable terminal, the extracted subject to be distinguished from other subjects; Processing method of the captured image and the related information of the mobile terminal further comprises. The method according to claim 4 or 5, wherein in the displaying step, And the related information is displayed superimposed on the captured image or displayed in a PIP. The method according to claim 4 or 5, wherein in the displaying step, And the related information is displayed adjacent to the subject. The method according to claim 4 or 5, wherein in the displaying step, And the related information is displayed by an icon or a list. The method of claim 2, And the related information includes basic information related to a capture situation, and information about each subject searched for in relation to the subject. The method of claim 2, wherein the storing step, Displaying, by the portable terminal, the related information together with the captured image on a screen; Receiving, by the portable terminal, a selection signal of a user, and selecting the acquired related information; Storing, by the portable terminal, the related information selected together with the captured image; Processing method of the captured image and the related information of the mobile terminal comprising a. The method of claim 11, wherein in the cooking step, And a selection signal of the user is input in at least one of a key input, a touch input, a voice input, and a gesture input. The method of claim 2, wherein in the storing step, And the portable terminal designates at least one of the related information as a file name of the captured image. The method of claim 2, wherein the step is performed after the storing step, Displaying, by the portable terminal, a captured image stored in the storage unit when a call signal of the captured image is input; If the mobile terminal inputs specific related information and requests a search, extracting and displaying a captured image including the specific related information from the captured image stored in the storage unit; Processing method of the captured image and the related information of the mobile terminal further comprises. The method of claim 2, wherein the step is performed after the storing step, Displaying, by the portable terminal, a captured image stored in the storage unit when a call signal of the captured image is input; Inputting specific related information to request alignment, and displaying, by the mobile terminal, the stored captured image according to the specific related information; Processing method of the captured image and the related information of the mobile terminal further comprises. The method of claim 15, And the specific related information is related information having an element capable of aligning the stored captured image.

Images