KR102135260B1 - Electronic device and method for controlling of the same - Google Patents

Abstract

Disclosed are an electronic device and a method of controlling the electronic device. An electronic device according to an embodiment, when a specific input is received, extracts a plurality of images that are images containing text photographed within a set time range among images photographed through a camera, and extracts a plurality of images having a similarity or higher than a reference value Multiple images can be combined to create a single image.

Description

ELECTRONIC DEVICE AND METHOD FOR CONTROLLING OF THE SAME}

The present invention relates to an electronic device and a method of controlling an electronic device that integrates a plurality of highly relevant images into one image.

Terminals such as personal computers, laptops, mobile phones, etc. have various functions, such as taking pictures or videos, playing music or video files, playing games, receiving multimedia, etc. Form.

In particular, the terminal has a front or rear camera, and a photo-taking function is frequently used. Accordingly, functions of the camera module mounted on the terminal are diversified and miniaturized.

However, the camera module mounted on the terminal has a limitation that it cannot reach the performance provided by a general camera because miniaturization is an essential requirement. In particular, there is a limit to the range in which the camera magnification can be adjusted.

Accordingly, the user has the inconvenience of dividing a necessary bundle of information into multiple cuts and photographing and storing them, and loading and checking a necessary portion of multiple images.

The present invention relates to an electronic device and a control method of an electronic device that combines a plurality of photographed images satisfying specific conditions into a single image or generates related images as a single note.

An electronic device according to an embodiment of the present invention for realizing the above object is an image including a camera, and a text captured within a set time range among images photographed through the camera when a specific input is received, and It may include a control unit for extracting a plurality of images having a similarity or higher than a reference value and generating a single image by combining the extracted multiple images.

If it is determined that a part of an object included in a specific image among a plurality of extracted images is cut off, the controller filters whether a cut out portion of the object is included in another image among the extracted multiple images, and includes an image that includes a cut out portion of the object. You can combine the images so that the cropped parts of and a specific image are connected.

The control unit may reduce the size of the extracted multiple images and generate one image having a set size.

When the number of images for integrating into one image is preset, the controller may group the extracted multiple images into a set number according to the shooting time and the similarity of the images, and combine the grouped images to generate one image. .

When there are two or more generated images, the control unit may determine the stern of the generated image, and may generate one note according to the determined sequential order.

When at least one content of text or objects included in the generated image is selected, the controller may display a list of at least one application capable of searching for information related to the selected content.

When the selected content is an object, the set number of candidate objects may be extracted according to the similarity of the object shape, and the other object included in the generated image may be used to select the object corresponding to the selected content.

In addition, the control method of the electronic device according to an embodiment of the present invention for realizing the above problems is a step of receiving a specific input, when a specific input is received, photographed within a set time range among images taken through the camera The method may include extracting a plurality of images that are text-containing images and having a similarity of a captured image that is greater than or equal to a reference value, and generating a single image by combining the extracted multiple images.

Specific details of other embodiments are included in the detailed description and drawings.

According to an embodiment of the present invention, it is possible to improve the user's convenience by integrating and photographing an image shot by dividing a plurality of cuts into a single image.

Further, according to an embodiment of the present invention, continuity of an image may be secured by generating a plurality of related images as a single note.

1 is a block diagram of an electronic device related to an embodiment of the present invention.
2A is a front perspective view of an electronic device related to an embodiment of the present invention.
2B is a rear perspective view of an electronic device related to an embodiment of the present invention.
3 is a flowchart of a method of controlling an electronic device according to an embodiment of the present invention.
4 to 11 are diagrams for describing a control method of an electronic device related to an embodiment of the present invention.

The above-described objects, features and advantages of the present invention will become more apparent through the following detailed description in conjunction with the accompanying drawings. However, the present invention can be modified in various ways and can have various embodiments. Hereinafter, specific embodiments will be illustrated in the drawings and described in detail. Throughout the specification, the same reference numbers refer to the same components. In addition, if it is determined that a detailed description of known functions or configurations related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, the numbers used in the description process of the present specification (eg, first, second, etc.) are merely identification symbols for distinguishing one component from other components.

Hereinafter, electronic devices related to the present invention will be described in more detail with reference to the drawings. The suffixes "modules" and "parts" for components used in the following description are given or mixed only considering the ease of writing the specification, and do not have distinct meanings or roles in themselves.

The electronic device described in this specification may include a mobile phone, a smart phone, a laptop computer, a terminal for digital broadcasting, a personal digital assistants (PDA), a portable multimedia player (PMP), navigation, and the like.

However, those skilled in the art will readily appreciate that the configuration according to the embodiments described herein may also be applied to fixed terminals such as digital TVs and desktop computers, except when applicable only to mobile terminals.

1 is a block diagram of an electronic device related to an embodiment of the present invention.

The electronic device 100 includes a wireless communication unit 110, an audio/video (A/V) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory unit 160, An interface unit 170, a control unit 180, and a power supply unit 190 may be included. The components shown in FIG. 1 are not essential, so an electronic device having more or fewer components may be implemented.

Hereinafter, the components will be described in turn.

The wireless communication unit 110 may include one or more modules that enable wireless communication between the electronic device 100 and the wireless communication system or between the electronic device 100 and the network in which the electronic device 100 is located. For example, the wireless communication unit 110 may include a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The broadcast receiving module 111 receives a broadcast signal and/or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server that generates and transmits broadcast signals and/or broadcast-related information or a server that receives previously generated broadcast signals and/or broadcast-related information and transmits them to an electronic device. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, and may also include a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.

The broadcast-related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms. For example, it may exist in the form of an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or an Electronic Service Guide (ESG) of Digital Video BroadcastHandheld (DVBH).

The broadcast receiving module 111 receives a broadcast signal using various broadcast systems, in particular, Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Forward Link Only (MediaFLO), Digital Video BroadcastHandheld DVBH ), a digital broadcasting signal can be received using a digital broadcasting system such as ISDBT (Integrated Services Digital Broadcast Terrestrial). Of course, the broadcast receiving module 111 may be configured to be suitable for the above-described digital broadcasting system as well as other broadcasting systems that provide broadcast signals.

The broadcast signal and/or broadcast-related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 may be built in or external to the electronic device 100. Wireless Internet technology (Wireless LAN) (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access) may be used.

The short-range communication module 114 refers to a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), and ZigBee may be used as short-range communication technology.

The location information module 115 is a module for confirming or obtaining the location of the electronic device. A representative example of the location information module is a Global Position System (GPS) module. According to the present technology, the GPS module 115 calculates information on a distance at which a point (object) is separated from three or more satellites, and information on a time when the distance information is measured, and then calculates the distance information. By applying the trigonometry to, it is possible to calculate three-dimensional position information according to latitude, longitude, and altitude for a point (object) at a time. Furthermore, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another one satellite is also used. The GPS module 115 continuously calculates the current position in real time and uses it to calculate speed information.

Referring to FIG. 1, an audio/video (A/V) input unit 120 is for inputting an audio signal or a video signal, which may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display module 151.

The image frame processed by the camera 121 may be stored in the memory unit 160 or transmitted to the outside through the wireless communication unit 110. Two or more cameras 121 may be provided according to a configuration aspect of the electronic device.

The microphone 122 receives an external sound signal by a microphone in a call mode, a recording mode, or a voice recognition mode, and processes it as electrical voice data. The processed voice data may be converted into a form transmittable to a mobile communication base station through the mobile communication module 112 and output. Various noise reduction algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122.

The user input unit 130 generates input data for the user to control the operation of the electronic device. The user input unit 130 may be configured with a key pad dome switch, a touch pad (static pressure/blackout), a jog wheel, a jog switch, or the like.

The sensing unit 140 detects the current state of the electronic device 100, such as the opening/closing state of the electronic device 100, the position of the electronic device 100, the presence or absence of user contact, the orientation of the electronic device, and the acceleration/deceleration of the electronic device. By generating a sensing signal for controlling the operation of the electronic device 100. For example, when the electronic device 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. Also, a sensing function related to whether the power supply unit 190 is supplied with power or whether the interface unit 170 is coupled with an external device may be in charge. Meanwhile, the sensing unit 140 may include a posture detection sensor 141 and/or a proximity sensor.

The output unit 150 is for generating output related to vision, hearing, or tactile sense, and includes a display module 151, an audio output module 152, an alarm unit 153, and a haptic module 154. Can.

The display module 151 displays and outputs information processed by the electronic device 100. For example, when the electronic device is in a call mode, a UI (User Interface) or a GUI (Graphic User Interface) related to the call is displayed. When the electronic device 100 is in a video call mode or a photographing mode, the photographed or/and received image, UI, or GUI is displayed.

The display module 151 is a liquid crystal display, a thin film transistor liquid crystal display, an organic light emitting diode, a flexible display, or a 3D display. It may include at least one.

Some of these displays may be of a transparent type or a light transmissive type so that the outside can be seen through them. This may be called a transparent display, and a typical example of the transparent display is a transparent LCD. The rear structure of the display module 151 may also be configured as a light transmissive structure. With this structure, the user can see an object located behind the electronic device body through an area occupied by the display module 151 of the electronic device body.

Two or more display modules 151 may exist depending on the implementation form of the electronic device 100. For example, in the electronic device 100, a plurality of display units may be spaced apart from one surface or integrally disposed, or may be disposed on different surfaces.

When the display module 151 and a sensor detecting a touch operation (hereinafter referred to as a “touch sensor”) form a mutual layer structure (hereinafter, abbreviated as “touch screen”), the display module 151 is an output device. In addition, it can be used as an input device. The touch sensor may have, for example, a form of a touch film, a touch sheet, or a touch pad.

The touch sensor may be configured to convert changes in pressure applied to a specific portion of the display module 151 or capacitance generated in a specific portion of the display module 151 into an electrical input signal. The touch sensor may be configured to detect not only the touched position and area, but also pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal(s) is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 180. Accordingly, the controller 180 can know which area of the display module 151 has been touched, and the like.

Referring to FIG. 1, a proximity sensor may be disposed in an inner area of the electronic device wrapped by the touch screen or near the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object in the vicinity using mechanical force or infrared light without mechanical contact. Proximity sensors have a longer lifespan and higher utilization than contact sensors.

Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, an infrared proximity sensor, and the like.

When the touch screen is capacitive, it is configured to detect the proximity of the pointer due to a change in electric field according to the proximity of the pointer. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

Hereinafter, for convenience of description, the act of causing the pointer to be recognized as being located on the touch screen without being touched by the pointer on the touch screen is referred to as a “proximity touch”, and the touch The act of actually touching the pointer on the screen is referred to as "contact touch". The location on the touch screen that is a proximity touch with a pointer refers to a location where the pointer is perpendicular to the touch screen when the pointer is touched close.

The proximity sensor detects a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The audio output module 152 may output audio data received from the wireless communication unit 110 or stored in the memory unit 160 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, or the like. The sound output module 152 outputs sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed in the electronic device 100. The sound output module 152 may include a receiver, a speaker, and a buzzer.

The alarm unit 153 outputs a signal for notifying the occurrence of an event in the electronic device 100. Examples of events generated in electronic devices include call signal reception, message reception, key signal input, and touch input. The alarm unit 153 may output a signal for notifying the occurrence of an event by other forms, for example, vibration, in addition to a video signal or an audio signal. The video signal or the audio signal may also be output through the display module 151 or the audio output module 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 are controllable. For example, different vibrations may be synthesized and output or sequentially output.

In addition to vibration, the haptic module 154 can be used for effects caused by stimulation by pin arrays that vertically move with respect to the contact skin surface, effects caused by stimulation through air or the inhalation force of the air through a spray or intake, and stimulation through the skin surface. Various tactile effects can be generated, such as an effect, an effect caused by stimulation through contact of an electrode, an effect caused by stimulation using electrostatic force, and an effect of reproducing a feeling of cold and warm using an element capable of absorbing heat or generating heat.

The haptic module 154 may not only transmit the tactile effect through direct contact, but may also implement the haptic effect through muscle sensation such as a user's finger or arm. Two or more haptic modules 154 may be provided according to a configuration aspect of the electronic device 100.

The memory unit 160 may store a program for the operation of the controller 180, and may temporarily store input/output data (eg, a phone book, a message, a still image, a video, etc.). The memory unit 160 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory unit 160 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory, etc.), At least one of Random Access Memory (RAM), Static Random Access Memory (SRAM), ReadOnly Memory (ROM), Electrically Erasable Programmable ReadOnly Memory (EEPROM), Programmable ReadOnly Memory (PROM) magnetic memory, magnetic disk, and optical disk It may include a storage medium of the type. The electronic device 100 may operate in relation to a web storage that performs a storage function of the memory unit 160 on the Internet.

The interface unit 170 serves as a passage with all external devices connected to the electronic device 100. The interface unit 170 receives data from an external device or receives power and transmits it to each component inside the electronic device 100 or allows data inside the electronic device 100 to be transmitted to the external device. For example, wired/wireless headset port, external charger port, wired/wireless data port, memory card port, port for connecting devices equipped with an identification module, audio input/output (I/O) port, A video input/output (I/O) port, an earphone port, and the like may be included in the interface unit 170.

The identification module is a chip that stores various information for authenticating the use authority of the electronic device 100, a user identification module (UIM), a subscriber identification module (SIM), and a universal user authentication module ( Universal Subscriber Identity Module (USIM). The device provided with the identification module (hereinafter referred to as an'identification device') may be manufactured in a smart card format. Therefore, the identification device may be connected to the electronic device 100 through a port.

When the electronic device 100 is connected to an external cradle, the interface unit serves as a passage through which power from the cradle is supplied to the electronic device 100, or various command signals input from the cradle by the user to the electronic device. It can be a passage to the device. Various command signals or power inputted from the cradle may be operated as signals for recognizing that the electronic device is correctly mounted on the cradle.

The controller 180 typically controls the overall operation of the electronic device. For example, it performs related control and processing for voice calls, data communication, video calls, and the like. The controller 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the controller 180, or may be implemented separately from the controller 180.

The controller 180 may perform a pattern recognition process capable of recognizing handwriting input or picture drawing input performed on the touch screen as text and images, respectively.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power required for the operation of each component.

The various embodiments described herein can be implemented in a computer- or similar device-readable recording medium using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, microcontrollers, microprocessors, and electrical units for performing functions. 180).

According to a software implementation, embodiments such as procedures or functions can be implemented with separate software modules that perform at least one function or operation. The software code can be implemented by software applications written in the appropriate programming language. In addition, the software code may be stored in the memory unit 160 and executed by the control unit 180.

2A is a front perspective view of an electronic device related to an embodiment of the present invention.

The disclosed electronic device 100 has a bar-shaped electronic device body. However, the present invention is not limited to this, and can be applied to various structures such as a slide type, a folder type, a swing type, and a swivel type in which two or more bodies are movably coupled.

The body includes a case (casing, housing, cover, etc.) forming an exterior. In this embodiment, the case may be divided into a front case 101 and a rear case 102. Various electronic components are embedded in the space formed between the front case 101 and the rear case 102. At least one intermediate case may be additionally disposed between the front case 101 and the rear case 102.

The cases may be formed by injecting synthetic resin or may be formed to have a metal material, for example, a metal material such as stainless steel (STS) or titanium (Ti).

In the electronic device body, mainly the front case 101, a display unit 151, an audio output unit 152, a camera 121, a user input unit 130/131,132, a microphone 122, an interface 170, and the like may be arranged. Can.

The display unit 151 occupies most of the main surface of the front case 101. An acoustic output unit 152 and a camera 121 are disposed in an area adjacent to one end of both ends of the display unit 151, and a user input unit 131 and a microphone 122 are arranged in an area adjacent to the other end. The user input unit 132 and the interface 170 are disposed on side surfaces of the front case 101 and the rear case 102.

The user input unit 130 is operated to receive a command for controlling the operation of the electronic device 100, and may include a plurality of operation units 131 and 132.

The operation units 131 and 132 may also be collectively referred to as a manipulation portion, and any method may be employed as long as the user operates with a tactile feeling.

Contents input by the operation units 131 and 132 may be variously set. For example, the first operation unit 131 receives a command such as start, end, scroll, etc., and the second operation unit 132 controls the volume of the sound output from the sound output unit 152 or the display unit 151 ) To a touch recognition mode.

2B is a rear perspective view of the electronic device shown in FIG. 2A.

Referring to FIG. 2B, a camera 121 ′ may be additionally mounted on the rear of the electronic device body, that is, the rear case 102. The camera 121 ′ has a shooting direction substantially opposite to the camera 121 (see FIG. 2A ), and may be a camera having different pixels from the camera 121.

For example, the camera 121 has a low pixel so that it is easy to take a picture of the user's face in the case of a video call and transmit it to the other party, and the camera 121' may not shoot the general subject and send it immediately It is desirable to have high pixels in many cases. The cameras 121 and 121' may be installed on the body of the electronic device to be rotated or popped up.

A flash 123 and a mirror 124 are additionally disposed adjacent to the camera 121'. When the flash 123 photographs the subject with the camera 121', light is directed toward the subject. The mirror 124 allows the user to reflect on the user's own face or the like when the user wants to photograph (self-shoot) using the camera 121'.

An acoustic output unit 152' may be additionally disposed on the rear surface of the electronic device body. The audio output unit 152' may implement a stereo function together with the audio output unit 152 (see FIG. 2A), or may be used to implement a speakerphone mode during a call.

On the side of the electronic device body, an antenna 124 for receiving a broadcast signal may be additionally disposed in addition to an antenna for a call. The antenna 124 forming a part of the broadcast receiving module 111 (refer to FIG. 1) may be installed to be withdrawn from the body of the electronic device.

The electronic device body is equipped with a power supply unit 190 for supplying power to the electronic device 100. The power supply unit 190 may be built in the electronic device body, or may be configured to be detachable directly from the outside of the electronic device body.

A touch pad 135 for sensing a touch may be additionally mounted on the rear case 102. The touch pad 135 may also be configured as a light transmitting type like the display unit 151. In this case, if the display unit 151 is configured to output visual information from both sides, it is possible to recognize the visual information through the touch pad 135 as well. Information output on both sides may be controlled by the touch pad 135. Alternatively, a display is additionally mounted on the touch pad 135, and a touch screen may be disposed on the rear case 102.

The touch pad 135 operates in relation to the display unit 151 of the front case 101. The touch pad 135 may be disposed parallel to the rear of the display unit 151. The touch pad 135 may have the same or smaller size than the display unit 151.

Hereinafter, embodiments of the present invention will be described.

3 is a flowchart of a method of controlling an electronic device related to an embodiment of the present invention, and FIGS. 4 to 11 are diagrams for explaining a method of controlling the electronic device related to an embodiment of the present invention.

Referring to FIG. 3, when a specific input is received (S110 ), the controller (180 of FIG. 1) of the electronic device may extract a plurality of images satisfying a set criterion among images captured through the camera (S130 ). .

At this time, the set criterion may be selected among cases in which the user directly selects or includes an image captured within a set time range and the similarity of the captured image is greater than or equal to a reference value.

Next, the controller (180 of FIG. 1) may generate a single image by integrating the extracted multiple images according to a set criterion (S150 ).

Specifically, if it is determined that a part of an object included in a specific image among a plurality of extracted images is cut off, the controller filters whether a cut out part of the cut out object is included in another image among the extracted multiple images, and the cut out part of the object Images can be combined so that the embedded image and the cropped portion of a particular image are linked. At this time, the control unit may determine that a part of the object is cut when the boundary area of the object included in the specific image is not all included in the image.

Also, the controller may process an image to select an image corresponding to the image including the cut portion, and automatically adjust the size and position of the two images so that feature points or text commonly included in the two images are connected.

For example, when text is included in the image and the boundary between the text and the background is not included in the image, it may be determined that the object included in the image, that is, the text is cut off.

The control unit may reduce or enlarge the size of the extracted multiple images and integrate them with other images. That is, the controller may adjust the size of the images so that the magnification of the multiple images is not constant or the size of the objects included in the multiple images correspond to each other in consideration of a change in the shooting distance from the subject. In addition, the controller may automatically edit the contrast, brightness, etc. after integrating multiple images.

When the number of images for integrating into one image is preset, the control unit groups the extracted multiple images into a set number according to the shooting time and the similarity of the images, and combines each grouped image to generate a single image. can do.

In addition, when there are two or more generated images, the controller may determine the stern of the generated image and generate one note in accordance with the determined sequential order. At this time, the control unit may determine the order of the sequencing according to the shooting time of each generated image or the content content.

When at least one content of text or objects included in the generated image is selected, the control unit may display a list of at least one application capable of searching for information related to the selected content in a pop-up window, and when a specific application is selected, selected content Search information through a specific application may be displayed. In this case, if the selected content is an object, the set number of candidate objects may be extracted according to the similarity of the object shape, and another object included in the generated image may be used to select the object corresponding to the selected content among the candidate objects. .

4 is an embodiment of receiving a specific input to extract a plurality of images to be integrated in an electronic device according to an embodiment of the present invention.

Referring to FIG. 4, when a touch input for a specific icon (A) is received from the execution screen of the gallery application, the controller (180 of FIG. 1) controls to request extraction of a plurality of images to be integrated among a plurality of captured images. It can be judged by a signal.

When the controller receives an input such as a pinch-in or a pinch-out from the execution screen of the gallery application through the touch screen 151, it is formed on the body of the terminal. When an input for manipulating an operation key is received or a touch input for a virtual key is received, it may be determined as a control signal requesting extraction of a plurality of images to be integrated.

5 is an embodiment of receiving a specific input to integrate a plurality of images extracted from an electronic device according to an embodiment of the present invention.

Referring to FIG. 5, when a touch input for a specific icon B is received on a screen where a plurality of extracted images are displayed, the control unit (180 of FIG. 1) determines the control signal to request the integration of the extracted multiple images. can do.

For example, the control unit may incorporate images I1, I2, I3, I5, I6, I7, I8, I9, I10, I12, I13, from images I1, I2, I3, I4, I5, etc. displayed on the execution screen of the gallery application. I14 can be extracted and 12 extracted images can be combined into one image. At this time, when the number of images to be integrated is set in advance, the controller may combine the set number of images into one image.

That is, when the number of images to be combined is set to 4, the controller may classify 12 images into 3 groups, and combine 4 images included in each group to generate 1 image.

6A to 6L are examples of multiple images extracted from an electronic device according to an embodiment of the present invention, and FIG. 7 is generated by integrating multiple images extracted from an electronic device according to an embodiment of the present invention It is an example of one image.

6A to 6L, the control unit (180 of FIG. 1) may extract at least one image that satisfies a preset condition from among a plurality of images captured through a camera (121 of FIG. 1 ).

Specifically, the controller may extract at least one image (I1 to I3, I5 to I10, I12 to I14) that satisfies temporal continuity and similarity of the image (see FIGS. 6A to 6L).

The controller may select an image to be integrated based on the similarity between the extracted at least one image (I1 to I3, I5 to I10, I12 to I14), and integrate each image so that a common area is matched. At this time, the control unit may generate one image P1 by adjusting the size, brightness, and contrast of at least one image I1 to I3, I5 to I10, and I12 to I14 (see FIG. 7).

8 to 9 are examples in which the number of images to be integrated in the electronic device according to an embodiment of the present invention is preset.

When the number of images to be combined is preset to six, the controller (180 in FIG. 1) may classify the extracted multiple images into several groups including six images.

At this time, the image included in each group may include an area common to the image, an image connected to the cut image, or an image sequentially photographed.

Referring to FIG. 8, the control unit may select I1 to I4, I6, and I7 as the first group, and select I8, I10 to I14 as the second group.

Referring to FIG. 9, the controller may generate two new images P1 and P2 by combining six images included in each group into one.

In addition, when there are two or more generated images, the control unit may generate the generated images as one note, and display indicators C and D indicating movement of a page on one side of the generated image.

Accordingly, according to the present invention, due to a manipulation limit of a photographing magnification of a built-in electronic device, a document is divided into a plurality of images and stored, and then a plurality of images photographed in a divided form is a document (or image). It can be created by integrating with or multiple related images can be created as a single note.

10 to 11 are examples of searching for related information by selecting a part of content included in an image generated by an electronic device according to an embodiment of the present invention.

Referring to FIG. 10, the control unit (180 in FIG. 1) selects some of the text included in the generated image (eg, selects specific text through drag input) and can search for information related to the selected text The application list W1 can be recommended.

At this time. The controller may extract some of the text included in the image using optical character recognition (OCR) technology.

Referring to FIG. 11, the control unit (180 of FIG. 1) selects an object included in the generated image (eg, pinches out a selection box and selects it), and lists applications that can search for information related to the selected object (W1) can be recommended.

At this time, the controller may extract a preliminary image similar to the selected object image, and determine one of the extracted preliminary images as a selected object image using text included in the generated image.

Accordingly, according to the present invention, information on text or objects included in the generated image can be directly searched.

The method for controlling a mobile terminal according to the present invention described above may be provided by recording on a computer-readable recording medium as a program for executing on a computer.

The method for controlling a mobile terminal according to the present invention can be executed through software. When executed in software, the configuration means of the present invention are code segments that perform necessary tasks. The program or code segments can be stored on a processor readable medium or transmitted by a computer data signal combined with a carrier wave in a transmission medium or communication network.

The computer-readable recording medium includes all types of recording devices in which data readable by a computer system are stored. Examples of computer-readable recording devices include ROM, RAM, CD-ROM, DVD-ROM, DVD-RAM, magnetic tape, floppy disk, hard disk, and optical data storage device. In addition, the computer readable recording medium may be distributed over network coupled computer devices so that the computer readable code is stored and executed in a distributed manner.

The present invention described above, as those skilled in the art to which the present invention pertains, various substitutions, modifications and changes are possible within the scope of the technical spirit of the present invention, and the above-described embodiments and the attached It is not limited by the drawings. In addition, the embodiments described in this document are not limitedly applicable, and all or part of each embodiment may be selectively combined to be configured so that various modifications can be made.

100: mobile terminal 151: display module
180: control unit

Claims (8)

camera; And
A control unit determining a first image based on a preset method among a plurality of images including text photographed within a preset time range among images photographed through the camera when a specific input is received;
The control unit,
A second image including an area common to the image is determined by at least one of the top, bottom, left, and right sides of the first image and a predetermined area or more, and the second image is updated to a first image to determine the second image. An electronic device characterized in that a plurality of images having a similarity or higher than a reference value are extracted, and the extracted multiple images are combined to generate a single image.
According to claim 1,
The second image,
An image connected to an image including an object in which at least one part of at least one side of the first image is cut off is determined as the second image, but the object is cut into another image among the extracted multiple images. An electronic device characterized by filtering whether a part is included, and integrating an image such that the clipped part of the object and the clipped part of the specific image are connected.
According to claim 1,
The control unit reduces the size of the extracted multiple images, and then generates one image having a set size.
According to claim 1,
When the number of images to be integrated into the single image is preset, the controller determines the images sequentially shot with the first image as the second image, and determines the extracted multiple images of the time and image. An electronic device characterized by grouping into a set number according to similarity and combining the grouped images to generate one image.
According to claim 4,
The controller, if there are two or more generated images, determines the stern of the generated image, and generates an electronic device according to the determined sequential order with one note.
According to claim 1,
The controller is configured to display a list of at least one application capable of retrieving information related to the selected content when at least one content of text or objects included in the generated image is selected.
The method of claim 6,
When the selected content is an object, the controller extracts a set number of candidate objects according to the similarity of the object shape, and uses other information included in the generated image to correspond to the selected content among the candidate objects Electronic device, characterized in that for selecting.
delete

Images