KR101054690B1 - Portable terminal and method for controlling portable terminal using image data - Google Patents

Abstract

PURPOSE: A device operating method is provided to specify a divided area recognized within image data as a independent use. CONSTITUTION: An image analysis unit(120) interprets an image of a selected image data. The image analyzing unit generates image analysis information including a coordinate range of recognized partitioned areas. A function performance unit(160) manages allocation information. At least one among function setting information and setting information for performance is allocated to a partition area. The function performance unit performs a processing according to the allocation information about the selected section area.

Description

Portable terminal and method for controlling portable terminal using image data}

The present invention relates to a method for operating a device using a portable terminal and image data.

Recently, operators have provided various functions in portable terminals to satisfy user's needs, and provide various services linked thereto. For example, a mobile communication terminal, which is an example of a mobile terminal, includes various additional functions such as a camera function, a digital broadcast playback function, a video call function, and a wireless Internet access function in addition to the basic voice call function.

That is, in recent years, the portable terminal has been improved to include various additional functions suitable for the user's taste, in addition to the basic function of specifying the type of the portable terminal.

As one of these additional functions, some portable terminals also include a function of enabling the user to play predetermined music using the portable terminal.

When the user selects the performance function menu, a performance screen such as a piano keyboard image is displayed on the display of the mobile terminal, and when the user selects an arbitrary key with a key button or touches a touch-sensitive screen, the corresponding keyboard is selected. It is performed in such a manner that a predetermined sound source is reproduced to correspond.

However, such a performance function according to the prior art has a limitation that can be limited to induce user interest due to the monotony that is performed only through the screen image pre-implemented to perform the function. In addition, due to this limitation, the performance provided as an additional function is a cause that can not be continuously utilized.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

The present invention is to provide a portable terminal and a device operation method using the image data so that the partition regions recognized in the image data photographed using the camera function can be specified and used as independent uses.

In addition, the present invention is to provide a portable terminal and a device operation method using the image data to enable the user's interest and interest to continue by connecting the camera function and the playing function provided in the portable terminal.

In addition, in the present invention, when the partition region recognized in the image data photographed using the camera function is set to be allocated for each of the independent operation of the user, the acquirer arbitrarily manipulates the portable terminal even when the portable terminal is lost. It is to provide a method of operating a device using a portable terminal and image data that can not be used as a security function.

Other objects of the present invention will be readily understood through the following description.

According to an aspect of the present invention, a portable terminal comprising: an input unit for receiving selection of image data from one or more image data stored in advance; An image analyzer configured to interpret image of the selected image data by an image analysis technique and generate image analysis information including coordinate ranges of one or more partitioned regions; And allocation information allocated to the selected partition area by determining at least one of the function setting information and the setting information for performance to manage the allocation information allocated to each partition area, and determining which partition area the selection information input through the input unit is for. There is provided a portable terminal including a function performing unit for performing a process according to the present invention.

The function performing unit may include a setting manager that manages the allocation information; A touch position recognizing unit recognizing coordinate information corresponding to the selection information and determining a partition area having a coordinate range including the recognized coordinate information; And a designation function execution unit to perform a process according to the allocation information allocated to the determined partition area.

The performance setting information may include one or more of instrument selection information, scale information, and pitch information, and the function performing unit may process sound source data according to the performance setting information designated for the selected partition area to be reproduced.

The portable terminal further includes a storage unit for storing formation patterns of the partition areas and allocation proposal information for the partition areas for each formation pattern, wherein the function performing unit previously stores the formation pattern corresponding to the image analysis information in the storage unit. If it is already stored, the corresponding allocation suggestion information may be displayed through the display unit.

The image data allocated with the allocation information, which is the function setting information, may be set as a menu screen by user selection.

The input unit may be a touch sensitive screen. In this case, the function performing unit may include a setting manager that manages the allocation information; A touch position recognition unit recognizing coordinate information corresponding to a user's touch position using the touch sensitive screen and determining a partition area having a coordinate range including the recognized coordinate information; And a designation function execution unit to perform a process according to the allocation information allocated to the determined partition area.

The image analysis technique may be an edge detection technique.

The allocation information may be allocated only for a portion of the plurality of partition regions identified by the image interpretation information.

According to another aspect of the present invention, an operation method using the image data performed in the mobile terminal, comprising the steps of: receiving any image data from one or more of the previously stored image data; Interpreting the image of the selected image data by an image analysis technique to generate image analysis information including coordinate ranges of one or more partitioned regions; Generating allocation information for each partition area allocated by user selection; And determining which partition area the selection information input by the user selection is for and performing a process according to the allocation information assigned to the selected partition area, wherein the allocation information is one of function setting information and setting information for performance. It may be abnormal.

The performance setting information includes one or more of instrument selection information, scale information and pitch information, and the performing of the processing includes: reproducing sound source data according to the performance setting information designated for the selected partition area. Can be.

When the formation patterns of the partition regions and the allocation proposal information for the partition regions for each formation pattern are stored in advance, the generating of the allocation information may include determining whether a formation pattern corresponding to the image analysis information is already stored in the storage unit. Determining; And displaying corresponding allocation suggestion information through the display unit if it is already stored.

Image data to which allocation information, which is the function setting information, is allocated may be set to a menu screen by user selection.

The portable terminal may include a touch sensitive screen.

The performing of the processing may include: recognizing coordinate information corresponding to a user's touch position using the touch sensitive screen; Determining a partition area of a coordinate range including the recognized coordinate information; And performing processing according to allocation information allocated to the determined partition area.

The image analysis technique may be an edge detection technique.

The allocation information may be allocated only for a portion of the plurality of partition regions identified by the image interpretation information.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention, the partitioned areas recognized in the image data photographed using the camera function can be specified and used as independent uses.

In addition, the camera function and the playing function provided in the portable terminal are also connected organically so that the user's interest can be induced and sustained.

In addition, when the user determines that the partition areas recognized in the image data photographed using the camera function are allocated for the independent operation, the learner cannot arbitrarily operate the mobile terminal even if the mobile terminal is lost. It can also be used as a security function.

1 is a view schematically showing the configuration of a portable terminal according to an embodiment of the present invention.
2 is a view schematically showing the configuration of a function performing unit according to an embodiment of the present invention.
3 is a flow chart showing a device operation method using the image data according to an embodiment of the present invention.
4 is a view conceptually illustrating a device manipulation method using image data according to an embodiment of the present invention.
5 is a flowchart illustrating a method of operating a device using image data according to another exemplary embodiment of the present invention.
6 is a view conceptually illustrating a device manipulation method using image data according to another embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Also, the terms " part, "" unit," " module, "and the like, which are described in the specification, refer to a unit for processing at least one function or operation and may be implemented by hardware or software or a combination of hardware and software .

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

In addition, in the following description of a device operation method using a portable terminal and image data according to the present invention, any functions provided in the portable terminal may be assigned to each of the partition regions according to the interpretation of the image data without limitation. For convenience of explanation, the following description will focus on the case where the partition areas are used for the performance function.

1 is a view schematically showing a configuration of a portable terminal according to an embodiment of the present invention, Figure 2 is a view showing a configuration of a function performing unit according to an embodiment of the present invention.

Referring to FIG. 1, the mobile terminal 100 includes a camera unit 110, an image analyzer 120, a storage unit 130, a display unit 140, an input unit 150, a function performing unit 160, and a speaker unit. 170 and the controller 180. The mobile terminal 100 according to the embodiment of the present invention may be implemented as any electronic device having a camera function, for example.

The camera unit 110 photographs an external subject by a user's manipulation of the input unit 150 and generates image data accordingly. The camera unit 110 may include, for example, one or more of a lens, an image sensor, an image signal processor, and the like. In addition, when the generated image data is encoded and stored in the storage 130, the camera unit 110 may further include a multimedia processor for encoding the image data or decoding the encoded image data.

The image analyzer 120 may store image data stored in the storage 130 (eg, image data generated by the camera unit 110, image data stored from an external electronic device or downloaded through a wireless Internet). The shape of the partition regions in the image corresponding to the image data and display of the display unit 140 by using a predetermined image analysis technique (for example, an edge detection technique) for the image data selected by the user. Image analysis information for analyzing one or more of coordinate ranges for specifying the area of each partition area based on the area reference point is generated (see FIGS. 4 and 6).

Image analysis techniques for interpreting points, lines, edges, etc. in an image using characteristic information of each pixel constituting the image (for example, one or more of luminance value, brightness value, brightness value, etc.) are well known to those skilled in the art. As it is obvious, a description thereof will be omitted. The image analyzer 120 may be integrated with, for example, the multimedia processor.

The storage unit 130 may include image data generated by the camera unit 110, stored in an external electronic device, or downloaded through the wireless Internet, image analysis information generated by the image analyzing unit 120, and respective partition regions. Function setting information assigned to a function menu (e.g., search for a phone number, create a short message, play a music, play a game, etc., a numeric key for entering a phone number for a phone call), perform a music play function For this purpose, one or more of the performance setting information for setting the pitch, scale, and the like for each partition area can be stored. The performance setting information may further include information on selecting an instrument for performing a music performance function.

The display unit 140 displays an operation state of the portable terminal 100, image data selected by the user, and the like.

The input unit 150 is a means for receiving manipulation information by a user, and may be implemented as one or more buttons. If the display unit 140 is implemented as a touch sensitive screen, the display unit 140 and the input unit 150 may be integrated. Hereinafter, the case where the display unit 140 is implemented as a touch sensitive screen will be described.

As illustrated in FIG. 2, the function performing unit 160 may include an area recognizer 210, a setting manager 220, a touch position recognizer 230, and a designated function executer 240.

The area recognizer 210 recognizes information on each partition area in the image with reference to the image analysis information generated by the image analyzer 120.

The setting manager 220 manages allocation information (that is, one or more of function setting information, performance setting information, etc.) assigned by the user for each partition area. The function setting information may be, for example, a function menu (for example, a phone number search, short message creation, music playback, a game, etc.) for each partition area, a function item of one or more, a numeric key for entering a phone number for a phone call, and the like. ), And the setting information for performance is information on setting of the instrument and the pitch, scale, etc. of each partition area for performing the music playing function.

Naturally, a function menu and a setting screen for the user to set an arbitrary function menu or specify setting information for each partition area may be provided in the mobile terminal 100 in advance. Since the method is a matter that can be easily derived from the general function menu and the setting screen configuration method, a description thereof will be omitted.

The touch location recognition unit 230 recognizes a point corresponding to a coordinate of a touch location touched by the user using a finger or a stylus pen, and the touch location is recognized by the area recognition unit 210. It recognizes which area of each of the partitioned areas.

The designation function execution unit 240 processes the designated function to be performed on the partition area corresponding to the touch position recognized by the touch position recognition unit 230 with reference to the allocation information managed by the setting manager 220. . If the corresponding allocation information is the function setting information, processing such as execution of a correspondingly designated function menu or input of a number for inputting a telephone number will be performed. However, if the corresponding allocation information is the setting information for the performance, the sound source data corresponding to the corresponding designated instrument, pitch and scale will be played.

Referring back to FIG. 1, the speaker unit 170 outputs a voice of the other party during a phone call or outputs sound source data according to a user's manipulation of the mobile terminal 100 as auditory information.

The controller 180 controls the operation of each component of the above-described portable terminal 100.

3 is a flowchart illustrating a device operation method using image data according to an embodiment of the present invention, and FIG. 4 is a view conceptually illustrating a device operation method using image data according to an embodiment of the present invention.

Hereinafter, referring to FIGS. 3 and 4, a method of operating a device using image data will be described by taking an example in which allocation information for each partition area is setting information for performance. The allocation information does not have to be set in all the partition regions analyzed by the image analyzer 120 or the like, and the allocation information may be set only for some partition regions.

Referring to FIG. 3, in operation 310, the image analyzer 120 reads image data selected by a user from among image data stored in the storage 130. When the corresponding image data is encoded and stored, a decoding process may be performed to interpret and display the image data through the display unit 140.

In operation 320, the image analyzer 120 may determine a shape and a display unit of the partitioned regions in the image corresponding to the image data using a predetermined image analysis technique (for example, an edge detection technique) on the read image data. Based on the display area reference point of FIG. 140, image analysis information obtained by analyzing one or more of coordinate ranges for specifying the areas of the respective partition areas is generated (see 410 of FIG. 4).

In step 330, the setting manager 220 determines whether a setting change of allocation information for each partition area is requested from the user.

If a setting change is requested, the process proceeds to step 340 where the setting manager 220 updates the performance setting information according to the pitch and scale set by the user for each partition area. Of course, the instrument setting for performing the performance function may be changed in step 340.

As illustrated at 410 of FIG. 4, a user may set or change a pitch and a scale for each partition region interpreted by the image analyzer 120 and recognized by the region recognizer 210. For example, the user can set various performance setting information such as setting a scale for the region A, a scale for the region B, and a sole scale for the region E.

However, if the setting change is not requested, the setting management unit 220 may apply the setting information for performance designated in a manner designated as a default for each partition area. The performance setting information designated as a default may be set such that, for example, the lower scale is assigned to the partition area located above and to the left. That is, the scale may be set for the region F, the scale scale is set in the region G, and the scale is designated and set in the region A.

In operation 360, the touch position recognizing unit 230 determines whether the user touches the touch-sensitive screen using a finger or a stylus pen.

If the user's touch is not detected, the process waits at step 350.

However, if the user's touch is detected, the process proceeds to step 360, and the touch position recognizing unit 230 corresponds to any region of each of the divided regions recognized by the area recognizing unit 210 as the touch position, which is the position touched by the user. Recognize if it works.

Subsequently, in step 370, the designated function execution unit 240 performs an operation process so that the designated function is performed on the partition area including the touch position.

As illustrated in FIG. 4, when the scale is assigned to the region A, the scale to the region B, and the brush scale is set to the region E, corresponding sound source data will be played according to the corresponding touch position.

Thus far, the description has been focused on the case where the allocation information for each partition area is setting information for performance.

However, the function menu can be set for each partition area as described above.

For example, when a function menu is set for each compartment, a phone number search function is assigned to area A, a short message creation function is assigned to area B, and an MP3 file playback function is assigned to area H. In this case, a function corresponding to the partition area touched by the user may be executed. As such, it is natural that the image data on which the function menu is set may be set as a function menu screen for selecting and executing a general function menu by user's designation.

Another example is a case in which a numeric key for inputting a telephone number is set for each partition area, in which a number 1 is assigned to area A, a number 2 is assigned to area B, and a call button is assigned to area H. If each number and a call button are allocated, the numbers corresponding to the partition areas touched by the user may be continuously input to perform the phone call function.

In this case, the user designates the background screen of the number input screen for a telephone call as image data of a general dial telephone or a button telephone, and sets a numeric key or the like in the partition areas for each button to enter the telephone number. You can also have a lot of fun by making it possible.

For example, a function menu or a numeric key assigned to each partition area may display text information, etc., to indicate what allocation information is allocated to the partition area.

Of course, it may be possible not to display what the allocation information about the function menu or the numeric key assigned by the user is in each of the partition areas. In addition, if the image data in which allocation information is set for each partition area is set as a menu screen by the user's selection, when the user loses the portable terminal 100, the learner who has acquired the portable terminal 100 Optionally, the operation of the portable terminal 100 is not easy, and thus may be utilized as a security purpose.

5 is a flowchart illustrating a device operation method using image data according to another embodiment of the present invention, and FIG. 6 is a view conceptually illustrating a device operation method using image data according to another embodiment of the present invention.

5 and 6 illustrate that the setting manager 220 interprets and recognizes whether the image data selected by the user is a general musical instrument (for example, a piano, etc.) or an apparatus (for example, a phone, etc.) used for a specific purpose. An example of automatically setting allocation information for each partition area according to an established method is described.

Referring to FIG. 5, in operation 510, the image analyzer 120 reads image data selected by a user from among image data stored in the storage 130. When the corresponding image data is encoded and stored, a decoding process may be performed to interpret and display the image data through the display unit 140.

In operation 520, the image analyzer 120 may determine a shape and a display unit of the partitioned regions in the image corresponding to the image data by using a predetermined analysis technique (for example, an edge detection technique) on the read image data. Based on the display area reference point of FIG. 140, image analysis information obtained by analyzing one or more of coordinate ranges for specifying the areas of the respective partition areas is generated (see 610 of FIG. 6).

In operation 530, the setting manager 220 determines whether the pattern formed by the partitioned regions is an instrument shape by referring to the image analysis information generated by the image analyzer 120. For reference, it is assumed in FIG. 6 that image data photographing a piano keyboard is selected. Of course, in operation 530, it may be further determined whether the pattern is a general device used for a specific purpose, and for the determination, the patterns and usage information for each pattern are previously stored and managed in the storage 130. Of course it can.

If the determination in step 530 does not determine the shape of the musical instrument, the flow proceeds to step 330 of FIG. 3.

However, if it is determined in step 530 that it is determined that the shape of the instrument, the setting manager 220 may proceed to step 540 and may suggest the setting information for performance to the user by using the usage information for each pattern corresponding to the recognized pattern. The proposal may be recognized by the user by being displayed on the display unit 140. For example, in the case of the piano keyboard image illustrated in FIG. 6, the performance setting information suggestion may be presented to the user by displaying the scale information corresponding to each key overlapping the performance image.

In operation 540, the configuration manager 220 determines whether approval information for the proposal is input from the user.

When the approval information is input from the user, the process proceeds to step 340 of FIG. 3 and the setting manager 220 sets allocation information for each partition area according to the proposed performance setting information.

However, if the approval information is not entered (for example, if the reject information is input), the flow proceeds to step 330 of FIG.

As described above, when image data of a general musical instrument or device having a specific function is selected, the mobile terminal 100 may propose allocation information for allowing each partition area to function in a normal manner. This allows the captured image data to function similar to an actual instrument or device.

In addition, even if image data is selected for a general instrument or device for which the function is specified, the user can set it in different ways for each compartment (for example, to play a musical note even if a particular key is actually played on the scale). Setting or setting a function menu), so that the photographed image data can function differently from a real instrument or device, causing user's fun.

It is obvious that the method for operating a device using the image data described above may be performed by an automated procedure according to a time series sequence by a software program or the like embedded in the portable terminal. Codes and code segments constituting the program can be easily inferred by a computer programmer in the art. In addition, the program is stored in a computer readable media, and read and executed by a computer to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium and a carrier wave medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the following claims And changes may be made without departing from the spirit and scope of the invention.

100: mobile terminal
110: camera unit
120: image analysis unit
130: storage unit
140: display unit
150: input unit
160: function performing unit
170: speaker unit
180:
210: area recognition unit
220: setting management unit
230: touch position recognition unit
240: designated function execution unit

Claims (18)

In a mobile terminal,
An input unit configured to receive arbitrary image data among at least one previously stored image data;
An image analyzer configured to interpret image of the selected image data by an image analysis technique and generate image analysis information including coordinate ranges of one or more partitioned regions; And
At least one of the function setting information and the performance setting information manages allocation information allocated to each partition area, and determines which partition area the selection information input through the input unit is for the allocation information assigned to the selected partition area. A portable terminal comprising a function performing unit for performing the processing according to.
The method of claim 1,
The function performing unit,
A setting manager for managing the allocation information;
A touch position recognizing unit recognizing coordinate information corresponding to the selection information and determining a partition area having a coordinate range including the recognized coordinate information; And
And a designated function execution unit to perform a process according to allocation information allocated to the determined partition area.
The method of claim 1,
The setting information for the performance includes one or more of instrument selection information, scale information and pitch information,
And the function performing unit is configured to process sound source data according to the performance setting information designated for the selected partition area to be reproduced.
The method of claim 1,
Further comprising a storage unit for storing the formation pattern of the partition areas and the allocation proposal information for the partition area for each formation pattern,
The function performing unit determines whether a formation pattern corresponding to the image analysis information is previously stored in the storage unit, and displays corresponding allocation suggestion information through the display unit if it is already stored.
And the allocation suggestion information is information set in advance such that a specific operation is performed on each partition region of each formation pattern.
The method of claim 1,
And image data to which allocation information as the function setting information is assigned is set as a menu screen by user selection.
The method of claim 1,
The input unit is a portable terminal, characterized in that the touch-sensitive screen.
The method of claim 6,
The function performing unit,
A setting manager for managing the allocation information;
A touch position recognition unit recognizing coordinate information corresponding to a user's touch position using the touch sensitive screen and determining a partition area having a coordinate range including the recognized coordinate information; And
And a designated function execution unit to perform a process according to the allocation information allocated to the determined partition area.
The method of claim 1,
The image analysis technique is a portable terminal, characterized in that the edge detection (edge detection) technique.
The method of claim 1,
And the allocation information is assigned only to a part of the plurality of partition regions identified by the image analysis information.
In the portable terminal operating method using the image data performed in the portable terminal,
Receiving arbitrary image data among one or more previously stored image data;
Interpreting the image of the selected image data by an image analysis technique to generate image analysis information including coordinate ranges of one or more partitioned regions;
Generating allocation information for each partition area allocated by user selection; And
Determining which partition area the selection information input by the user selection is for and performing processing according to allocation information allocated to the selected partition area;
And the allocation information is at least one of function setting information and setting information for performance.
The method of claim 10,
The setting information for the performance includes one or more of instrument selection information, scale information and pitch information,
The performing of the processing comprises: reproducing the sound source data according to the performance setting information designated for the selected partition area.
The method of claim 10,
When the formation pattern of the partition regions and the allocation proposal information for the partition regions for each formation pattern are stored in advance,
Generating the allocation information,
Determining whether a formation pattern corresponding to the image analysis information is previously stored in a storage unit; And
If it is stored in advance, the step of displaying the corresponding allocation proposal information via the display unit,
And the allocation suggestion information is information which is set in advance such that a specific operation is performed on each of the partition regions for each formation pattern.
The method of claim 10,
And image data to which allocation information as the function setting information is allocated is set as a menu screen by user selection.
The method of claim 10,
The portable terminal operating method of the mobile terminal using the image data, characterized in that it comprises a touch sensitive screen.
The method of claim 14,
The step of performing the process,
Recognizing coordinate information corresponding to a user's touch position using the touch sensitive screen;
Determining a partition area having a coordinate range including the recognized coordinate information; And
And performing a process according to the allocation information allocated to the determined partition area.
The method of claim 10,
The image analysis technique is an edge detection technique, the method of operating a mobile terminal using image data.
The method of claim 10,
And the allocation information is allocated only to a part of the plurality of partition regions identified by the image analysis information.
A program of instructions that can be executed by a digital processing apparatus is tangibly embodied and can be read by the digital processing apparatus in order to perform the portable terminal operating method using the image data according to any one of claims 10 to 17. Recording medium that records the program.

Images