KR101611308B1 - Method of image taking - Google Patents

Abstract

An image photographing method comprises: a step of outputting an image recognized by a camera to a display unit; a step of obtaining multiple pieces of distance information on the image; a step of selecting a first area based on the multiple pieces of distance information; a step of applying a specified effect to the selected first area or a non-selected second area; and a step of storing the image, to which the specified image effect is applied. A plurality of objects can be distinguished based on the distance information by photographing the image including the distance information using an infrared sensor such that a specific object on the image can be extracted, thereby allowing easy editing of the image.

Description

{METHOD OF IMAGE TAKING}

The present invention relates to a method of photographing an image containing distance information and having an effect on a specific area.

A terminal can be divided into a mobile terminal (mobile / portable terminal) and a stationary terminal according to whether the terminal can be moved. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal.

The functions of mobile terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, modern mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal has various functions, for example, in the form of a multimedia device having multiple functions such as photographing and photographing of a moving picture, reproduction of a music or video file, reception of a game and broadcasting, etc. .

As a main function of a multimedia device, a mobile terminal equipped with an image shooting and editing function is increasing. In particular, the camera for image shooting has become a necessary component, and the functions of the camera have been enlarged to obtain a high-quality image, and it can be shot in various ways such as shooting with continuous shooting, panoramic shooting, and adjusting brightness.

There is a demand for editing functions in the mobile terminal itself because there are many users who edit and upload the photographed images to social network sites or the like. However, until now, there is a problem that the photographing can not be performed in consideration of the effect desired by the user at the time of photographing, and the editing is limited after the photographing.

Particularly, in order to apply an effect only to a specific object, there is a problem that a user must manually extract an editable portion, and extraction is difficult even if extracted, resulting in an unnatural picture.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method of editing an image including distance information and photographing an image using distance information and including distance information.

According to another aspect of the present invention, there is provided an image processing method comprising: outputting an image recognized through a camera to a display unit; Obtaining a plurality of distance information in the image; Selecting a first area based on the plurality of distance information; Applying a designation effect to the selected first region or the non-selected second region; And storing the image to which the designation effect is applied.

The step of selecting the first area may include: setting distance information of a point at which a user's touch input is recognized in the image as a designated distance;

A point having distance information similar to the designated distance or having distance information closer to the specified distance may be selected as the first area.

The step of selecting the first area may include grouping into one area when the distance information between neighboring points has a continuous value; And when the user selects one point of the grouped area, selecting the grouped area as the first area.

Wherein the step of selecting the first region includes the steps of: outputting a state bar indicating a distance to a display unit; Selecting the designated distance by adjusting the status bar; And selecting a position having a distance information similar to the specified distance or located within the specified distance.

The adjustment of the status bar is performed through a touch-and-drag input on the status bar, and the step of applying the specifying effect comprises: detecting an end of the touch input in the status bar; Outputting a menu including a plurality of effects on the display unit; Designating an effect of one of the menus as a designation effect; And applying the designation effect to the selected first region or the non-selected second region.

The step of outputting the menu may output the menu at a point where a touch input is detected when a touch input or a drag input is detected at a point other than the status bar.

Wherein the applying the designating effect comprises: dragging one effect from the menu to the first area or the second area; And storing the image by focusing on a point where the drag is terminated.

The step of obtaining the distance information may include: outputting an infrared ray; Obtaining a time difference of the infrared ray reflected by the output infrared ray; And calculating a distance from the infrared sensor at each point on the basis of the time difference.

Outputting an infrared ray; Calculating a time difference of infrared rays reflected by reflecting the output infrared ray on the object; Calculating distance information of each object based on the time difference to generate distance information; Storing a plurality of images including the distance information successively in a time lag; Selecting a specific object from one of the plurality of images; Extracting a specific object from another image based on the distance information of the selected object; And generating an image obtained by synthesizing the rest of the images except for the specific object.

According to at least one embodiment of the present invention, a plurality of objects can be distinguished based on distance information by photographing an image including distance information using an infrared sensor, so that specific objects in an image can be extracted, There is an easy advantage.

In particular, the background can be removed or a specific object can be excluded, so that an image extracted by a user can be obtained.

Further scope of applicability of the present invention will become apparent from the following detailed description. It should be understood, however, that the detailed description and specific examples, such as the preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art.

1A is a block diagram illustrating a mobile terminal according to the present invention.
1B and 1C are conceptual diagrams illustrating an example of a mobile terminal according to the present invention in different directions.
2 is a flowchart showing an embodiment of an image capturing method according to an aspect of the present invention.
FIGS. 3 to 9 are views for explaining respective steps of the image capturing method according to FIG.
10 is a flowchart showing another embodiment of the image capturing method according to an aspect of the present invention.
FIGS. 11 and 12 are views for explaining respective steps of the image capturing method according to FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

However, it will be appreciated by those skilled in the art that the configuration according to the embodiments described herein may be applied to fixed terminals such as a digital TV, a desktop computer, a digital signage, and the like, will be.

1A to 1C are block diagrams for explaining a mobile terminal according to the present invention, and FIGS. 1B and 1C are conceptual diagrams showing an example of a mobile terminal according to the present invention in different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, and a power supply unit 190 ), And the like. The components shown in FIG. 1A are not essential for implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than the components listed above.

The wireless communication unit 110 may be connected between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and another mobile terminal 100 or between the mobile terminal 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules for connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include at least one sensor for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor 145, an infrared sensor, a finger scan sensor, an ultrasonic sensor sensor, an optical sensor (e.g., see camera 121), a microphone (see microphone 122), a battery gauge, an environmental sensor (e.g., barometer, hygrometer, thermometer, A sensor, a thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic tip module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user and may provide an output interface between the mobile terminal 100 and a user.

The interface unit 160 serves as a path to various types of external devices connected to the mobile terminal 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the mobile terminal 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 may store a plurality of application programs or applications running on the mobile terminal 100, data for operation of the mobile terminal 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. Also, at least a part of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions (e.g., telephone call receiving function, message receiving function, and calling function) of the mobile terminal 100. Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and may be operated by the control unit 180 to perform the operation (or function) of the mobile terminal.

In addition to the operations related to the application program, the control unit 180 typically controls the overall operation of the mobile terminal 100. The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 170. FIG. In addition, the controller 180 may operate at least two of the components included in the mobile terminal 100 in combination with each other for driving the application program.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the components included in the mobile terminal 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170. [

FIG. 2 is a flowchart showing an embodiment of an image capturing method according to an aspect of the present invention, and FIGS. 3 to 9 are views for explaining each step of the image capturing method according to FIG.

An infrared sensor 145 (IR sensor) is used to detect the distance from the mobile terminal 100 at each point in image shooting. The infrared sensor is a device that detects the change by detecting the reflected light by self-copying the infrared rays, and the sensitivity and accuracy are improved as compared with the conventional temperature sensor ultraviolet sensor. The infrared sensor includes an infrared ray emitting infrared ray and an infrared ray receiving unit for detecting the reflected infrared ray.

The infrared sensor 145 is oriented in the same direction as the camera 121b for acquiring an image, and can obtain distance information at each point of the image acquired by the camera 121b. More specifically, a preview of the image input to the camera 121b is output to the display unit 151 (S11). Each of the objects 10-1, 10-2, 10-3, and 10-4 includes a plurality of objects 10-1, 10-2, 10-3, and 10-4, 100, as shown in FIG.

At the same time or sequentially displaying the image, the infrared ray is emitted by using the infrared ray LED of the infrared ray sensor 145 (S12), and the infrared ray reflected by the emitted infrared ray is detected again to detect the infrared ray returning to the mobile terminal 100 (S13). The distance information from the mobile terminal 100 to each point can be obtained based on the time difference between the emitted infrared ray and the reflected infrared ray (S14).

3, the infrared ray output from the infrared ray sensor 145 emits light to the objects 10-1, 10-2, 10-3, and 10-4 within the range captured by the camera 121b . 4 (a), the infrared rays are uniformly projected on all the objects 10-1, 10-2, 10-3, and 10-4 located on the screen, and the objects 10-1, 10-2, 10-3, and 10-4 and the mobile terminal 100, the time difference between the projected time and the reflected infrared ray reception time differs. The distance information of the point where each infrared ray is emitted according to the time difference can be obtained.

The distance information included in the image may include not only information on the distance of each point in the image, but also information on which the object is divided by grouping points having similar distance information.

Since one object is within a similar distance, an area similar in distance information in the screen can be grouped as shown in (b) of FIG. 4 (1) to (5). In the case of an obliquely arranged object, for example, an object such as a wall has distance information that is different but continuous distance information between adjacent points. That is, when the distance information of neighboring points is similar or the distance values of neighboring points have continuous values, it can be determined as one object and grouped.

Referring to FIG. 5A, one of a plurality of grouped regions may be selected as a first region as shown in FIG. 4B. The plurality of grouped regions may be displayed together with the preview of the display unit 151, and the user may select one of the grouped regions to select the first region 201 (S15, see FIG. 2). In order to indicate that the first area 201 is selected, a thick line 200 or a line of another color may be displayed around the first area 201 as shown in (a).

Alternatively, the user may select an area to which the point selected through the touch input of the preview screen belongs, as the first area 201, or an area to which the selected point belongs and an area that is closer to the selected area, as the first area 201.

It is difficult to distinguish accurate boundaries if the user wants to manually cut along the object boundary, but using the distance information, it is possible to automatically extract the desired object automatically.

It is possible to edit the object separately after extracting the object, or to leave the object only as shown in FIG. 4B, and to change the background 202 to another background 202 '.

6 is a diagram showing another method of selecting the first areas 203 and 204 on the preview screen. A state bar 210 is displayed on one side of the screen and an area corresponding to the touched position is selected as the first area 203 when the user touches the status bar 210, 210, the first region 204 is changed. If one side of the status bar 210 is close to the other and the other side is far away, the first area 204 may be reduced as shown in FIG. 6B by dragging along the status bar 210 in one direction.

7, when the user touches the preview screen, the grouped area to which the touched point belongs is selected as the first area 204, and the state bar 210 that can be adjusted immediately is displayed at the touched point. The user can drag an object along the status bar to select an object with a short distance or an object with a long distance to the first area 203 or 204. [ After selecting the first area 203 or 204 by adjusting the status bar 210, the selection of the first area is terminated when the touch input is completed in the status bar 210.

When the selection of the first area 204 is completed, a designation effect to be applied to a part of the image is selected (S16). The end of selection of the first area 204 is accomplished by releasing the user in the status bar 210 or dragging the area to an area other than the status bar 210 as shown in FIG. A menu 240 including a plurality of effects that can be applied to the display unit 151 is output. The menu 240 may include only an individual name or an example image to which the effect is applied.

The menu 240 may include an effect menu 240 including a plurality of effects 220a-220d at the point where dragging ends when dragging. When the user moves his / her hand along the menu 240 and then selects one of the effects through touch input, the selected effect 220b is set as a designation effect, , But the screen can be viewed as a preview.

If a drag input is performed to the first region 204 selected in one effect 220b as shown in FIG. 9A, the effect is applied to the first region 204b, and the position 230 at which the drag input ends, And the photographed image is stored.

9B, when an effect is dragged to a region other than the first region (referred to as a second region), the selected effect is applied to the second region 205 to store the image. Also in this case, the image to which the effect is applied is stored with the position where the drag input ends as the focus.

FIG. 10 is a flowchart showing another embodiment of the image capturing method according to an aspect of the present invention, and FIGS. 11 and 12 are views for explaining each step of the image capturing method according to FIG.

Steps S21 to S23 for obtaining distance information of each point located in the image using the infrared sensor 145 are the same as S11 to S13 of the embodiment of Fig. 2 described above.

As described above, the distance information of each point may include information obtained by dividing an object by grouping if the distance information has a continuous value.

As shown in FIG. 11, a plurality of images ((a), (b), and (c)) including distance information are stored for a predetermined time (S24). The user can select the specific object 250 in one of the stored image files (a, b, c) (S25), and the method of selecting the object 250 is the same as in the above- And may be a grouped area based on distance information.

The selected object 250 is extracted from each image ((a), (b), (c)) (S26). If the same object 250 has similar distance information for each image ((a), (b), (c)), or if a grouped area exists in the remaining image that maintains a similar shape to the selected object 250, .

(A) to (c)) including the remaining area 251 except for the image of the area determined by the same object 250 as shown in FIG. 12 to synthesize the image in which the selected object 205 ' Can be extracted (c). According to the method of FIG. 10, a desired image can be obtained by removing the obstacles to be obstructed.

As described above, according to at least one embodiment of the present invention, an infrared sensor 145 can be used to photograph an image including distance information and to distinguish a plurality of objects based on distance information, So that the image can be easily edited. The background can be removed or a specific object can be excluded.

The foregoing detailed description should not be construed in all aspects as limiting and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

100: mobile terminal 110: wireless communication unit
120: Input unit
140: sensing unit 150: output unit
160: interface unit 170: memory
180: control unit 190: power supply unit
145: Infrared sensor 210: Status bar
220: Menu

Claims (9)

Outputting a recognized image through a camera to a display unit;
Obtaining a plurality of distance information in the image;
Selecting a first area based on the plurality of distance information;
Applying a designation effect to the selected first region or the non-selected second region; And
Storing the image to which the specifying effect is applied,
Wherein the selecting of the first region comprises:
Grouping into one area when the distance information between neighboring points has a continuous value;
A state bar indicating a distance is output to a display unit;
Outputting to the display unit a grouping area including a designated distance corresponding to the touch position in the status bar; And
And selecting a grouping area including a designated distance corresponding to a point at which the touch input ends on the status bar as a first area,
Wherein applying the designation effect comprises:
Outputting a menu including a plurality of effects to the display unit when the touch input is terminated in the status bar; And
And dragging the first area or the second area by touching an effect of one of the menus to apply the touched effect to a point at which dragging is completed. delete delete delete delete The method according to claim 1,
The step of outputting the menu may include:
Wherein when the touch input is detected at a point other than the status bar, the menu is output at a point where the touch input is sensed. The method according to claim 1,
Wherein the storing the image comprises:
And storing the image by focusing on the point where the drag ends. The method according to claim 1,
The step of acquiring the distance information includes:
Outputting an infrared ray;
Obtaining a time difference of the infrared ray reflected by the output infrared ray; And
And calculating a distance from the infrared sensor at each point based on the time difference. Outputting an infrared ray;
Obtaining a time difference of infrared rays returned by reflecting the output infrared ray on an object;
Calculating distance information of each object based on the time difference to generate distance information;
Storing a plurality of images including the distance information successively in a time lag;
Selecting a specific object from one of the plurality of images;
Extracting, as a specific object, an area having similar distance information or similar shape as the selected specific object in another image; And
And generating an image obtained by synthesizing the remainder excluding the specific object in each of the images,
Wherein the step of selecting the specific object comprises:
Grouping into one area when the distance information between neighboring points has a continuous value;
A state bar indicating a distance is output to a display unit;
Outputting to the display unit a grouping area including a designated distance corresponding to the touch position in the status bar;
Selecting a grouping area including a specified distance corresponding to a point at which the touch input ends on the status bar as a specific object.

Images