KR20080037312A - Apparatus for controlling moving of pointer and method thereof - Google Patents

Abstract

A device and a method for controlling a pointer are provided to control the pointer on a display unit by extracting movement of an object regardless of brightness around the object without any light source and to extract the movement of a pointer without image frames. An imaging part(210) generates first image data by capturing an object. A sharpness controller(231) generates second image data by controlling sharpness of the first image data. An outline extractor(233) extracts an outline corresponding to one end of the object by generating a differential value between the first and second image data. A pointer location setting part(239) sets a pointer on a predetermined location of the outline by reflecting a moving distance of a pointer moving controller. The pointer location setting part includes a device movement direction measuring unit measuring the device moving direction predetermined on the standard coordinates system and the pointer moving controller, an outline moving direction measuring unit measuring the pointer moving direction, the pointer moving direction measurer, and a pointer moving distance measuring unit measuring a pointer moving distance corresponding to the moving distance of the object.

Description

Apparatus for controlling moving of pointer and method

1 is a diagram illustrating a case in which a pointer movement control device according to a preferred embodiment of the present invention is provided in a mobile communication terminal.

Fig. 2A is a block diagram of a pointer movement control device in accordance with a first preferred embodiment of the present invention.

2B is a diagram showing a setting method for a reference luminance value used in pointer movement control according to the first preferred embodiment of the present invention.

3 is a flowchart of a pointer movement control method according to a first preferred embodiment of the present invention.

4 is a view showing an image generated by the pointer movement control apparatus according to the first preferred embodiment of the present invention.

5 is a view showing an image generated by the pointer movement control apparatus when the background brightness is large according to the first preferred embodiment of the present invention.

6 is a view showing an image generated by the pointer movement control apparatus when the background brightness is small according to the first preferred embodiment of the present invention.

7 is a block diagram of a pointer movement control device according to a second preferred embodiment of the present invention.

8 is a flowchart illustrating a pointer movement control method according to a second preferred embodiment of the present invention.

9 is a schematic diagram showing a configuration of a pointer movement control device according to the present invention;

10 is a cross-sectional view of a moved pointer movement control device according to a preferred embodiment of the present invention.

11 is a plan view of a moved pointer movement control apparatus according to a preferred embodiment of the present invention.

12A is a plan view of a pointer movement control device moved clockwise according to a preferred embodiment of the present invention.

12B is a plan view of a pointer movement control device moved counterclockwise according to a preferred embodiment of the present invention.

FIG. 13 is a diagram illustrating a method of calculating a movement displacement of a pointer according to the movement of a subject illustrated in FIG. 12B.

14 is a flow chart related to a method for calculating a movement displacement of a pointer according to a preferred embodiment of the present invention.

<Explanation of symbols for main parts of the drawings>

210: imaging unit 213: filter unit

216: lens unit 219: image sensor unit

230: image processing unit 231: sharpness adjustment unit

233: outline extraction unit 235: gamma correction unit

237: center determination unit 239: pointer position setting unit

240: output unit

The present invention relates to a pointer control method and apparatus, and more particularly, to a pointer control method and apparatus using a movement of a subject.

In general, a display device is a device that converts an electrical signal into an image signal to output. Types of such display devices include a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display panel (PDP), and an organic light emitting diode (OLED) display device. Each display device emits light of R, G, and B corresponding to an image signal to implement colors corresponding to the image.

In addition, with the rapid development of electronics and communication engineering, users can use various devices such as internet access, video communication and video message transmission, digital music listening and satellite broadcasting by using digital home appliances or mobile communication terminals such as computers and TVs. You can enjoy the function. In order to enjoy these various functions, efficient keypad input for selection is desired.

 Here, with the development of semiconductor technology, it is possible to supply a camera imaging device at a low price, so that a computer and a mobile communication terminal equipped with the same can be supplied to easily access electronic devices equipped with a camera imaging device.

Also, with the development of computer image processing technology and image recognition technology, various application fields using the same have been developed. As one of various application fields, a technology for transferring information using a camera imaging device as an input means is being developed.

However, additional keys for menu selection such as direction keys and selection keys by selecting a menu function using an existing keypad (for example, call button and wireless internet access button for a mobile communication terminal and file playback for an MP3 player). Buttons and file selection buttons). Therefore, when manufacturing a digital home appliance or a mobile communication terminal, it is necessary to work on the mold, and for this reason, there is a problem in that the manufacturing process of the electronic equipment becomes complicated and the manufacturing cost thereof increases.

An object of the present invention is to provide a method and an apparatus for controlling a pointer movement, which can control a pointer on a display by extracting a movement of a subject regardless of brightness around a subject.

In addition, the present invention is to provide a pointer movement control method and apparatus that do not require a separate light source to illuminate the subject in order to extract the movement of the subject.

In addition, the present invention is to provide a pointer movement control method and apparatus therefor that do not require a plurality of image frames to extract the movement of the subject.

In addition, the present invention is to provide a pointer movement control method and apparatus capable of controlling a pointer on a display using a camera imaging device.

 In addition, the present invention can omit a part of the existing keypad to increase the spatial utilization of the electronic device, and to provide a pointer movement control method and apparatus capable of miniaturization of the electronic device.

In addition, the present invention is to provide a pointer movement control method and apparatus that can simplify the manufacturing process and reduce the manufacturing cost of the electronic device manufacturing.

In addition, the present invention is to provide a pointer movement control method and apparatus that can maximize the utilization of the components by making it possible to use the camera provided universally.

In addition, the present invention is to provide a pointer movement control device and method that can adjust the position of the pointer movement control device according to the user's use environment.

The present invention also provides a pointer movement control method and apparatus capable of controlling an optional function such as a menu movement or a volume value in a device having no display, as well as a pointer represented on a display.

Technical problems other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, a method of controlling a pointer movement according to a movement of a subject by a pointer movement control device which is moved in a predetermined direction with respect to a preset reference coordinate system, includes: Generating one image data; (b) generating second image data by adjusting the sharpness of the first image data; (c) extracting a difference value between the first image data and the second image data and extracting a predetermined outline corresponding to one end of the subject; And (d) positioning the pointer at a specific position in the outline by reflecting the degree of movement of the pointer movement control device.

Here, the step (d) includes the steps of measuring the device movement direction which is the degree of inclination of the reference coordinate system and the device coordinate system preset in the pointer movement control device; Measuring a movement direction of a specific position in an outline which is a movement direction of the outline based on the device coordinate system; Measuring a pointer movement direction in which the device movement direction is corrected in the outline movement direction; Measuring a pointer movement distance corresponding to a movement distance of the subject; And positioning the pointer at a position corresponding to the pointer movement direction and the pointer movement distance.

Here, the device movement direction is an angle measured by the degree of inclination of the pointer movement control device in the counterclockwise direction with respect to the reference coordinate system, and the outline movement direction is the counterclockwise direction with respect to the device coordinate system. An inclination degree is measured, and the pointer movement direction may be a value obtained by adding the device movement direction counterclockwise from the outline movement direction.

The step (c) may further include gamma correcting the difference value.

Here, a difference value having a luminance value greater than a reference luminance value may be extracted to extract a predetermined outline corresponding to one end of the subject.

Here, the sharpness of the first image data may be adjusted by increasing at least one of color contrast, brightness contrast, and saturation contrast of pixels of the image of the subject.

Here, the specific position in the outline may be the center of gravity of the outline.

Here, the center of gravity of the outline may be calculated by the following equation.

은 무게 중심, M은 상기 외곽선을 나타내는 픽셀의 휘도값을 모두 더한 값, m_i는 상기 외곽선을 나타내는 각 픽셀 휘도값,

는 상기 외곽선을 나타내는 픽셀의 좌표값임. At this time,

Is the center of gravity, M is the sum of the luminance values of the pixels representing the outline, m _i is the luminance value of each pixel representing the outline,

Is the coordinate value of the pixel representing the outline.

Here, the center of gravity of the outline may be calculated by the following equation.

은 무게 중심, n은 상기 외곽선을 나타내는 픽셀의 개수,

는 상기 외곽선을 나타내는 픽셀의 좌표값임. At this time,

Is the center of gravity, n is the number of pixels representing the outline,

Is the coordinate value of the pixel representing the outline.

Here, the predetermined outline may be curved, and the coordinate value corresponding to the specific position may be a coordinate value of a center point determined by the curved outline.

Here, step (b) includes calculating a distance between each pixel representing the curved outline; And extracting a coordinate value of the center point by using a median value of two pixels having the largest distance of each pixel, and calculating a specific position included in the outline.

Here, the step (a) may be to capture the subject by using light reflected from the light source included in the pointer movement control device reflected on the subject.

Here, the light irradiated from the light source unit may be infrared rays.

Here, the infrared rays may pass through an infrared filter included in the pointer movement control device.

According to another aspect of the present invention, a pointer movement control device which is moved in a predetermined direction on the basis of a preset reference coordinate system and controls the movement of the pointer according to the movement of the subject, wherein the first image data is generated by imaging the subject. An imaging unit; A sharpness controller configured to generate second image data by adjusting the sharpness of the first image data captured by the image pickup unit; An outline extracting unit which generates a difference value between the first image data and the second image data and extracts a predetermined outline corresponding to one end of the subject; And a pointer position setting unit which positions the pointer at a specific position within the outline by reflecting the degree of movement of the pointer movement control device.

The pointer position setting unit may further include: a device movement direction measuring unit configured to measure a device movement direction in which the reference coordinate system and the device coordinate system preset in the pointer movement control device are inclined; An outline moving direction measuring unit measuring an outline moving direction which is a moving direction of the outline based on the device coordinate system; A pointer moving direction measuring unit measuring a pointer moving direction in which the device moving direction is corrected in the outline moving direction; And a pointer movement distance measuring unit measuring a pointer movement distance corresponding to the movement distance of the subject.

Here, the device movement direction measuring unit measures the device movement direction of the degree that the pointer movement control device is inclined counterclockwise with respect to the reference coordinate system, and the outline movement direction measuring unit is based on the device coordinate system The outline movement direction, which is the degree of inclination of the outline in a clockwise direction, is measured, and the pointer movement direction measuring unit may measure the pointer movement direction by adding the device movement direction counterclockwise in the outline movement direction.

The pointer movement control device may further include a gamma correction unit configured to gamma correct a predetermined outline extracted by the outline extracting unit.

The outline extracting unit may extract a predetermined outline corresponding to one end of the subject by extracting a difference value having a luminance value greater than a reference luminance value.

The sharpness controller may adjust the sharpness of the first image data by increasing at least one of color contrast, brightness contrast, and saturation contrast of the pixels of the first image data.

Here, the pointer movement control device may further include a center determination unit for obtaining coordinates corresponding to a specific position included in the predetermined outline.

Here, the specific position included in the outline may be the center of gravity of the outline.

Here, the center of gravity of the outline may be calculated by the following equation.

은 무게 중심, M은 상기 외곽선을 나타내는 픽셀의 휘도값을 모두 더한 값, m_i는 상기 외곽선을 나타내는 각 픽셀 휘도값,

는 상기 외곽선을 나타내는 픽셀의 좌표값임.At this time,,

Is the center of gravity, M is the sum of the luminance values of the pixels representing the outline, m _i is the luminance value of each pixel representing the outline,

Is the coordinate value of the pixel representing the outline.

Here, the center of gravity of the outline may be calculated by the following equation.

은 무게 중심, n은 상기 외곽선을 나타내는 픽셀의 개수,

는 상기 외곽선을 나타내는 픽셀의 좌표값임.At this time,

Is the center of gravity, n is the number of pixels representing the outline,

Is the coordinate value of the pixel representing the outline.

Here, the predetermined outline may be curved, and the coordinate value corresponding to the specific position may be a coordinate value of a center point determined by the curved outline.

Here, the coordinate value of the center point may be a median value of two pixels having the largest distance of each pixel representing the curved outline.

Here, the subject may be in non-contact with the imaging unit.

Here, the pointer movement control device may be provided around the imaging unit, and may further include a light source unit for irradiating light onto the subject.

Here, the light irradiated from the light source unit may be infrared rays.

Here, the pointer movement control device may further include an infrared filter for passing only the infrared light emitted from the light source unit.

According to another embodiment of the present invention, in a method of controlling a pointer movement according to a movement of a subject by a pointer movement control device which is moved in a predetermined direction with respect to a preset reference coordinate system, (a) the light source unit is activated. Photographing the subject in a state to generate first image data; (b) photographing the subject and generating second image data while the light source unit is inactive; (c) generating third image data by comparing the first image data with the second image data; (d) calculating a center point of the photographed subject by analyzing the third image data by reflecting the degree of movement of the pointer movement control device; And (e) positioning the pointer at a coordinate corresponding to the center point.

The third image data may be data obtained by subtracting the luminance value of the second image data corresponding to each pixel from the luminance value of the first image data.

Here, the center point may be the center of gravity of the subject.

Here, the center of gravity of the subject may be calculated by analyzing the third image data, extracting a reference pixel corresponding to a preset reference boundary value, and using the following equation.

은 무게 중심, M은 상기 기준 픽셀의 휘도값을 모두 더한 값, m_i는 상기 기준 픽셀 휘도값,

는 상기 기준 픽셀의 좌표값임. At this time,

Is the center of gravity, M is the sum of the luminance values of the reference pixel, m _i is the reference pixel luminance value,

Is the coordinate value of the reference pixel.

Here, the center of gravity of the subject may be calculated by analyzing the third image data, extracting a reference pixel corresponding to a preset reference boundary value, and using the following equation.

은 무게 중심, n은 상기 기준 픽셀의 개수,

는 상기 기준 픽셀의 좌표값임. At this time,

Is the center of gravity, n is the number of the reference pixel,

Is the coordinate value of the reference pixel.

According to another embodiment of the present invention, a pointer movement control device which is moved in a predetermined direction with respect to a preset reference coordinate system and controls the movement of the pointer according to the movement of the subject, wherein the light source unit is turned on at a predetermined time interval. ; An image sensor unit configured to photograph the subject while the light source unit is activated to generate first image data, and to generate second image data by photographing the subject while the light source unit is deactivated; A subject extraction unit configured to generate third image data by comparing the first image data and the second image data; A center point determination unit which calculates a center point of the photographed subject by analyzing the third image data by reflecting the degree of movement of the pointer movement control device; And a pointer position setting unit for positioning the pointer at a coordinate corresponding to the center point.

The third image data may be data obtained by subtracting the luminance value of the second image data corresponding to each pixel from the luminance value of the first image data.

Here, the center point of the subject may be the center of gravity of the subject.

Here, the center of gravity of the subject may be calculated by the following equation after extracting a reference pixel corresponding to a preset reference boundary value by analyzing third image data by the center determination unit.

은 무게 중심, M은 상기 기준 픽셀의 휘도값을 모두 더한 값, m_i는 상기 기준 픽셀 휘도값,

는 상기 기준 픽셀의 좌표값임. At this time,

Is the center of gravity, M is the sum of the luminance values of the reference pixel, m _i is the reference pixel luminance value,

Is the coordinate value of the reference pixel.

Here, the center of gravity of the subject may be calculated by the following equation after extracting a reference pixel corresponding to a preset reference boundary value by analyzing third image data by the center determination unit.

은 무게 중심, n은 상기 기준 픽셀의 개수,

는 상기 기준 픽셀의 좌표값임. At this time,

Is the center of gravity, n is the number of the reference pixel,

Is the coordinate value of the reference pixel.

Hereinafter, a preferred embodiment of the pointer movement control method and apparatus according to the present invention will be described in detail with reference to the accompanying drawings, and in the following description with reference to the accompanying drawings, the same reference numerals regardless of the reference numerals. And duplicate description thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the pointer described in the present invention means not only a pointer expressed on a display, but also all functions that perform an optional function such as moving between menus or numerical values of a volume in a device without a display. That is, the present invention is not only a method of controlling a pointer on a display in a case where a pointer is provided on the display (for example, a computer, a mobile terminal, a PDA, etc.) and an apparatus (eg, a radio) that is not equipped with a display. , MP3 player, etc.) may be applied to a method of controlling movement / selection between menus / functions. Hereinafter, for convenience of description, the display of the mobile communication terminal will be described.

1 is a diagram illustrating a case in which a pointer movement control device according to an embodiment of the present invention is provided in a folder type mobile communication terminal. Referring to FIG. 1, the mobile communication terminal 100 includes an image capturing unit 110, a light source unit 120 (1, 120 (2)), and a keypad unit 130. User finger) 140 is shown.

The light source units 120 (1) and 120 (2) irradiate predetermined light (eg, fingertips) 140 to a characteristic part of the human body. Number and positions of the light source parts 120 (1) and 120 (2) so that the light emitted from the light source parts 120 (1) and 120 (2) can be reflected from the subject 140 and enter the imaging part 110. Can be determined. For example, the light source units 120 (1) and 120 (2) may be formed around the imaging unit 110, and the number of the light source units 120 (1) and 120 (2) may be implemented as two as shown in FIG. 1. Of course, this is not limited. In addition, the position of the subject 140 may be a position that reflects the light emitted from the light sources 120 (1) and 120 (2), and may be in contact with or not in contact with the imaging unit 110 or the mobile communication terminal 100. Can be (no contact). In addition, according to another embodiment, the separate light source unit 120 (1), 120 (2) is not provided, and irradiated by the display of the mobile communication terminal 100 and then reflected by the subject 140 to capture the imaging unit 110. The present invention may be performed by using light incident on the?

The light irradiated from the light sources 120 (1) and 120 (2) is reflected from the subject 140 and continuously enters the imaging unit 110, and the imaging unit 110 generates an image of the subject 140. do. The generated image of the subject is image-processed to extract the movement of the subject 140, and controls the movement of the pointer displayed on the display provided in the mobile communication terminal 100 corresponding to the extracted movement of the subject 140. Here, the pointer displayed on the display may be a pointer or a cursor having a predetermined shape (arrow shape, etc.), and is not separately displayed, but selection of buttons, numbers, figures, etc. displayed on the display (including shadows). ) May include various forms that move on the display to perform a function.

In addition, according to another embodiment, a pointer represented by a specific luminance component in each frame is interpreted according to the movement of the extracted subject 140, and a characteristic vector may be interpreted as a movement trajectory of the pointer in successive frames. In addition, the stored menu function may be executed to correspond to the analyzed characteristic vector. For example, when the movement of the extracted subject 140 is grasped up and down, left and right, or in a specific form (for example, a circle, a triangle, a square, etc.), a function preset in response to the movement may be executed. That is, according to an embodiment of the present invention, the movement of the extracted subject 140 may control the movement of the cursor on the display, or may extract a predetermined motion vector and execute a predetermined menu.

In this case, the menu may be differently determined by a separate function button (or a combination of button inputs). For example, it may be set to perform different functions according to the movement of the extracted subject 140 in the state where the * button is pressed and the movement of the extracted subject 140 in the state where the # button is pressed.

In addition, the keypad 130 is implemented in various ways such as a button method or a touch switch, so that the user can input numbers, letters, special symbols, and the like.

In the above description, a general drawing of a pointer movement control apparatus has been described. Hereinafter, a coordinate value corresponding to a specific position of one end of the subject 140 using image data extracted from the subject 140 will be described with reference to the accompanying drawings. A method and apparatus for mapping the pointer to a pointer will be described. Embodiments according to the present invention can be broadly classified into two types. First, a method of extracting an outline of one end of a subject 140 from a frame having different clarity by adjusting the sharpness of image data generated by capturing the subject 140. And second, a method of extracting a center point of the subject 140 by comparing a plurality of different image data obtained by capturing the subject 140 at predetermined time intervals. It demonstrates in order below.

2A is a block diagram of a pointer movement control device according to a first embodiment of the present invention.

Referring to FIG. 2A, the pointer movement control apparatus 200 may include an imaging unit 210, an image processing unit 230, and a display unit 240, and the imaging unit 210 may include a filter unit 213 and a lens. The image processing unit 230 may include a sharpness adjusting unit 231, an outline extracting unit 233, a gamma correction unit 235, and a center determining unit 237. And a pointer position setting unit 239.

Light incident from the subject may be irradiated to the image sensor unit 219 via the filter unit 213 and the lens unit 216. The filter unit 213 passes only the light of a specific wavelength so that the movement of the subject can be accurately measured. That is, when the pointer movement control apparatus according to the embodiment of the present invention includes the light source unit 211, the light source unit 211 may output light having a specific wavelength, and the light is reflected from the subject to filter unit ( By passing through 213, it is possible to block light of other wavelengths that may act as ambient noise. For example, the light source unit 211 may emit infrared light, and the filter unit 213 may perform a function of allowing only infrared light to pass therethrough. Here, the infrared rays that can be used in the present invention may be a light having a relatively small proportion in the sunlight, for example, the wavelength of the infrared light is 900 ~ 980nm, 1080 ~ a wavelength of light having a relatively small amount of light in the solar spectrum 1180nm, 1325 ~ 1425nm, 1800 ~ 2000nm and so on.

Here, although the case in which the filter unit 213 is included has been described, it is obvious that the present invention can be applied using all the light irradiated from the subject without including the filter unit 213. The lens unit 216 may perform a function of focusing the light passing through the filter unit 213 to the image sensor unit 219.

The image sensor unit 219 may be a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor. A complementary metal oxide semiconductor (CMOS) image sensor is composed of a plurality of unit pixels including a photo diode, and the signal output from the sensor is a digital electrical signal, which unit pixel is driven by a control circuit and signal processing. . A charge coupled device (CCD) image sensor includes a plurality of MOS capacitors, and is operated by outputting an analog electric signal by moving charges (carriers) to the MOS capacitors.

The image sensor unit 219 has a plurality of pixels capable of sensing light. Therefore, by detecting a position where light corresponding to the coordinate value at which the subject is located, the coordinate value can be calculated to move the pointer of the display unit 240.

The image processor 230 extracts coordinate values corresponding to the movement of the subject from the image data of the subject recognized by the image sensor 219. In detail, the sharpness controller 231 adjusts the sharpness of the first image data of the photographed subject to generate second image data different from the first image data. That is, the sharpness controller 231 may adjust the sharpness by increasing at least one of color contrast, brightness contrast, and saturation contrast of pixels of the first image data of the photographed subject.

Here, the sharpness of the image indicates the clarity of the contrast boundary portion of the image, and when the sharpness of the image of the subject increases, the clarity of the boundary portion increases, so that a predetermined amount of the image of the subject is taken. There is an advantage in that it is easy to grasp the outline (or edge) of only the subject. That is, the sharpness may be increased by increasing any one of color contrast, brightness contrast, and saturation contrast with respect to pixels having high sharpness. Here, the sharpness of the image of the subject may be expressed differently according to the depth of field of the lens unit 216, that is, within the range in which the image of the subject formed by the lens unit 216 can be clearly seen. In general, the outline of the subject located at a distance similar to the focal length of the lens unit 216 has a high sharpness, but the background located far from the focal length of the lens unit 216 has a low sharpness.

In the case of using the sharpness adjusting unit 231 according to the first embodiment of the present invention, the outline of the subject located at a distance similar to the focal length of the lens unit 216 will further increase the sharpness, but the focal length of the lens unit 216 is increased. Backgrounds that are far away from each other may have less sharpness. Accordingly, since the sharpness of the subject is increased by the sharpness adjusting unit 231 than the background, it may be easier to grasp and use it.

The outline extractor 233 extracts the difference between the first image data generated by photographing the subject and the second image data processed by the subject, and extracts the outline of the subject. That is, a difference value between the second image data having high sharpness generated by the sharpness controller 231 and the first image data of the photographed subject may represent an outline of the subject. The outline of the extracted subject may be an outline of one end of the subject adjacent to the image pickup unit 210.

Here, an outline corresponding to one end of the subject may be extracted by extracting a difference value having a luminance value greater than a preset threshold (or reference luminance value). The reference luminance value may remove noise generated by light entering the imaging unit 210 from a background other than the subject. That is, since the sharpness of the second image data generated by the sharpness controller 231 may be adjusted not only for the image data of the subject but also for the background image data, only the image data having a difference value greater than or equal to the reference luminance value is recognized as an outline. The efficiency of subject recognition can be improved. Here, the reference luminance value may be fixed or may vary according to the number of pixels having a predetermined luminance value or more, which will be described in detail with reference to FIG. 2B.

The gamma correction unit 235 gamma-corrects image data of the subject. Here, gamma of an image is generally a slope of a straight line when the input / output characteristic is displayed by positive and large contractions in a photoelectric conversion system or an all-optical conversion system such as a television, a camera, a receiver, a fax transceiver, and the like. . That is, the television displays the logarithmic ratio between the camera output voltage for the object luminance and the cathode ray tube luminance for the image input unit voltage. Since gamma correction according to an exemplary embodiment of the present invention may change the luminance value of each pixel, only a portion of the sharpness of the subject is displayed, and a portion of the remaining sharpness remains black. Therefore, since only the outline of the predetermined subject is displayed brightly in the image of the subject, it may be determined whether or not the subject is moved.

The center determiner 237 may track a predetermined outline from the gamma corrected image and extract a coordinate value determined by the predetermined outline. Here, the predetermined outline may have a variety of forms, for example, may be a circular outline, the coordinate value determined by the predetermined outline is a center point determined by the circular outline, that is, the coordinates of the center of the circle Can be a value. In general, since a subject used for extracting a motion may be one end of a user's finger, the user's motion may be calculated by extracting a circular outline that is one end of the finger.

The center determiner 237 calculates specific coordinates located inside the figure formed by the outline of the object extracted by the gamma corrector 235. There may be a variety of methods for calculating the specific coordinates, for example, it is possible to extract the coordinates of the center of gravity or the center point.

The center of gravity, which is a specific coordinate located inside the figure formed by the outline of the subject, may be calculated by the following equation.

(1)

(One)

(2)

(2)

(3)

(3)

Here, R is the center of gravity, M is the sum of the luminance values of pixels representing the outline of the subject, m _i is the luminance value of each pixel representing the outline of the subject, and r _i is the coordinate value of the pixel representing the outline. The subscripts x and y indicate the values corresponding to the X and Y axes, respectively, and Equation (3) is a vector representation of Equation (1) and (2).

In addition, when the center determination unit 237 determines the center point of the subject by the above-described equation, since the luminance values of the reference pixels must all be reflected, the amount of calculation increases, thereby reducing the overall function of the pointer movement control apparatus. May occur. Therefore, according to another embodiment, the center of gravity, which is a specific coordinate located inside the figure formed by the outline of the subject, may also be calculated by the following equation.

(4)

(4)

(5)

(5)

(6)

(6)

Here, R is the center of gravity, ri is the coordinate value of the pixel representing the outline, n is the number of pixels representing the outline. Subscripts x and y represent values corresponding to the X and Y axes, respectively, and Equation (6) is a vector representation of Equation (4) and (5). That is, the center determination unit 237 may determine the center point of the subject without reflecting the luminance value of the pixel. Since the luminance value of each pixel will not be significantly different, the center determination unit 237 may calculate the average value of the coordinate values of the pixels representing the outline as the center point of the subject without reflecting the same.

Alternatively, when the outline of the subject is curved, the radius of curvature may be calculated and the coordinate value of the center point corresponding to the radius of curvature may be extracted. If the predetermined outline is a rectangle, the coordinate value where the diagonal of the rectangle meets may be a coordinate value determined by the predetermined outline. In the case of another polygon, the coordinate value where two or more lines connecting vertices facing each other are met May be For example, the distance between each pixel representing the curved outline may be calculated, and the median value of the two pixels having the largest calculated distance of each pixel may be used as the coordinate value of the center point. In another example, the intermediate value may be represented by coordinate values for the X and Y axes in the case of a two-dimensional display, and the intermediate value between the two pixels having the largest difference in the coordinate values for the X axis or the Y axis. Can also be used as the coordinate of the center point.

The pointer position setting unit 239 controls the pointer of the display unit 240 to be positioned at the calculated specific coordinates. That is, the position of the pointer is specified in a specific coordinate, for example, a coordinate value of the center of gravity or the center point with respect to the outline of the subject. In addition, in a device without the display unit 240, a function such as selecting a menu or increasing a volume may be performed. That is, when the calculated specific coordinates are located above the preset pixels, the user may select a previous menu of the currently selected menu or, when the calculated specific coordinates are located to the right of the preset pixels, increase the volume. have.

Hereinafter, the case where the predetermined outline is circular will be described.

2B is a diagram illustrating a method for setting a reference luminance value used in pointer movement control according to the first embodiment of the present invention. The first reference luminance value 250, the time point 253 of the first reference pixel number, the first luminance lower limit value 255, the luminance range a corresponding to the first reference pixel number, and the second reference luminance value 260. The luminance range b corresponding to the viewpoint 263 of the second reference pixel number, the second luminance lower limit 265, and the second reference pixel number is illustrated. Here, the first or the second is used to describe two embodiments in which the number of reference pixels is different, and will be omitted and collectively described below.

The reference luminance value 250 or 260 is provided to compare the luminance value of each pixel to extract the outline of the subject. Hereinafter, it is assumed that the resolution of the image sensor unit 219 is 640 * 480 (horizontal * vertical), and the luminance value of each pixel is configured to be divided from 0 to 255. When the luminance value is '0', it will be black because it is the darkest case, and when the luminance value is '255', it will be expressed as white because it is the brightest case.

In addition, referring to FIG. 2B, the reference luminance value 250 or 260 may vary according to the number of pixels having a predetermined luminance value or more. The outline extracting unit 233 may set the number of reference pixels and extract a pixel having a high luminance value in order to extract pixels corresponding to the outline of the subject. For example, when the number of reference pixels is set in consideration of the amount of calculation, 10000 pixels are extracted from the histogram stored in the buffer prepared in advance in order of increasing luminance, and the pixels corresponding to the 10000th pixels (starting point of the reference pixel count 253). , 263) or a smaller value may be the reference luminance value 250 or 260. Therefore, luminance ranges a and b corresponding to the number of reference pixels may be determined. Here, since the luminance value of each pixel is stored in the buffer, the reference luminance value may be calculated by counting the number of pixels in order of decreasing luminance value. That is, if the total number of pixels is stored in the buffer, the reference luminance value may be calculated by extracting the luminance values in the order in which the luminance values are smaller by the total number of pixels minus the number of reference pixels.

Here, if the number of reference pixels is too large, most pixels will be extracted to the outline of the subject, and vice versa, very few pixels will be extracted to the outline of the subject. If the reference luminance value is too small (i.e., the number of reference pixels is too large), pixels irrelevant to the outline of the subject (for example, pixels for the background part) may be extracted, and thus lower luminance values (255, 265). If the reference luminance value becomes smaller than this by setting the lower limit luminance values (255, 265) can be set as the reference luminance value. In addition, when the reference luminance value is too large (that is, when the reference pixel number is too small), the minimum number of pixels constituting the outline of the subject may not be extracted, and in this case, the upper limit of luminance (not shown) may also be set. . Here, the lower limit values 255 and 265 and the upper limit value of the luminance may be generally given one by one. Referring to FIG. 2B, when the luminance lower limit values 255 and 265 change, the reference luminance values 250 and 260 change. The case is illustrated.

For example, if the number of reference pixels is 10000 and the lower luminance limits (255, 265) are 50, each pixel is analyzed and 10000 pixels are extracted in the order of the higher luminance values, and the greater of the luminance value and the 50th pixel is 10000. Is set to the reference luminance value. On the contrary, assuming that the number of reference pixels is 10000 and the upper limit of luminance is 200, 10000 pixels are extracted in order of increasing luminance, and the luminance value of the 10000th pixel and the smaller of 200 are set as the reference luminance values.

Further, according to another embodiment, the reference luminance value may be fixed to a specific value. For example, a pixel having a reference luminance value of 100 and above can be recognized as an outline of a subject. It goes without saying that the above-described luminance lower limit value and the luminance upper limit value can be applied even when the reference luminance value is fixed.

In addition, although the controller for controlling the operation of the pointer movement control apparatus 200 is not provided separately, the pointer movement control apparatus 200 includes the controller for controlling the overall operation (not shown). It can be apparent to those skilled in the art that can be provided separately. In addition, the pointer movement control apparatus 200 may be controlled by a main controller (not shown) for controlling the overall operation of the electronic device including the pointer movement control apparatus 200 without having a separate controller. Self-explanatory

3 is a flowchart of a pointer movement control method according to a first embodiment of the present invention.

In operation S310, the imaging unit photographs an image including a subject to generate first image data. In operation S320, sharpness of the image of the photographed subject is adjusted to generate second image data. Here, the sharpness control may vary according to the depth of the image. That is, in the case of the subject outline positioned near the focal length of the lens unit, the sharpness may be greatly increased as compared to the background image.

In operation S330, a difference value between the first image data and the second image data may be calculated, and in operation S340, gamma correction may be performed on the calculated difference value to increase the luminance value of the object outline. Therefore, the outline of the subject may be clear from the gamma corrected difference value. Here, the difference value is calculated for every pixel, and the following steps are performed individually for each pixel.

In operation S350, it is checked whether a difference value between the first image data and the second image data is greater than a preset reference luminance value. When the difference value is larger than the preset reference luminance value, these pixels are extracted with an outline corresponding to one end of the subject. However, if the difference value is smaller than the preset reference luminance value, it is checked whether the difference value for the next pixel is larger than the preset reference luminance value. If the difference value for the pixels is smaller than the preset reference luminance value, the first step is performed. Return to the previous step can be repeated again. As described above, the reference luminance value may be set corresponding to the fixed or reference pixel number.

In operation S360, an outline corresponding to one end of the subject is extracted from a coordinate value in which a difference value between the first image data and the second image data is larger than a preset reference luminance value.

In operation S370, the image center point is recognized using the outline of the subject. That is, the above-described center of gravity may be obtained or the radius of curvature may be calculated from the extracted outline of the subject and the coordinate value of the center point corresponding to the radius of curvature may be extracted. Alternatively, the distance between each pixel representing the circular outline formed by the outline of the subject is calculated without separately calculating the radius of curvature, and the coordinate value of the current center point is obtained by using the median value of the two pixels having the largest calculated distance of each pixel. You can also extract In addition, the coordinate values of the current center point may be extracted by various preset methods.

In operation S380, the pointer of the display unit is moved in response to the extracted coordinates of the center point.

4 is a view for explaining an operation of performing a sharpness adjustment and gamma correction by the pointer movement control apparatus according to the first embodiment of the present invention. Referring to FIG. 4, the original image 410, the sharpness-adjusted image 420, the gamma corrected image 430, the extracted center point 440, and the circular outline 450 are shown.

One end of the user's finger, which is the subject in the original image 410, has a round shape. Then, as the sharpness is adjusted according to the depth, the outline of the subject (user's finger) located at a position similar to the focal length as shown in the sharpness-adjusted image 420 is increased, and the background that is not is relatively small. Then, if the difference between the original image 410 and the sharpness-adjusted image 420 is obtained and the gamma value is increased as a whole, only the outline of one end of the user's finger having a high sharpness is shown in the gamma-corrected image 430. Therefore, the coordinate value of the center point described above can be extracted from the circular outline which is the outline of the subject, and the coordinate value of the center point from which the pointer of the display unit is extracted can be set.

5 is a view showing an image generated by the pointer movement control apparatus when the background brightness according to the first embodiment of the present invention is large, and FIG. 6 is a shape of a subject according to the first embodiment of the present invention. If it is not circular, it is a figure which shows the image produced by the pointer movement control apparatus. 5 and 6, original images 510 and 610, sharpness adjusted images 520 and 620, gamma corrected images 530 and 630, extracted center points and circular outlines 540 and 640 are shown. do.

Referring to FIG. 5, the outline of the subject may be extracted regardless of the brightness of the background. In other words, when adjusting the sharpness, not only the contour of the finger as the subject but also a background light source formed in a straight line may be a problem. However, as a result of the experiment, it was found that there is no problem in extracting the contour of the finger because the influence of the background image is small.

In addition, referring to FIG. 6, even when the subject, which is a finger, is not circular, that is, when the outline is a semicircle, the outline of the subject, which is a boundary value, may be extracted. Therefore, according to the exemplary embodiment of the present invention, even if only one image frame is used, the depth of the image may be used to determine whether the subject is moved and the coordinate values of the movement position.

7 is a block diagram of a pointer movement control device according to a second embodiment of the present invention.

Referring to FIG. 7, the pointer movement control apparatus 700 according to the second exemplary embodiment of the present invention includes a filter unit 712, a light source unit 714, a lens unit 716, and an image sensor unit 718. An image processor 730 and a display unit 740 may include an image capturer 710, an object extractor 731, a center determiner 732, and a pointer position setter 734.

The operation of each component described above will be described below. However, the operations of the components of the image capturing unit 710 and the display unit 740 except for the light source unit 714 are described above, and thus detailed descriptions of these operations will be omitted.

The light source unit 714 may output light having a specific wavelength, and the light is reflected from the subject and passes through the filter unit 713 to emit light having a different wavelength that may act as ambient noise. There is an effect that can be blocked. In particular, the light source unit 714 may be turned on at a predetermined time interval under the control of the main controller (not shown). At this time, the image sensor unit 718 captures the first image data by capturing the subject 140 while the light source unit 714 outputs light, and captures the subject 140 while the light source unit 714 does not output light. Each of the second image data may be generated and output to the image processor 730. In this case, the light source unit 714 may be turned on at a predetermined time interval under the control of the main controller (not shown), and the image sensor unit 718 may immediately turn on the light source unit 714 immediately after generating the first image data. When deactivated, second image data may be generated. In addition, the main control unit (not shown) is a control unit for controlling the overall operation of the electronic device (for example, portable terminal, laptop, etc.) including the pointer movement control apparatus 700 according to the third embodiment of the present invention It may be a control unit provided separately in the imaging unit 710 may be a component for controlling only the light source unit 714.

The subject extractor 731 generates third image data by comparing the first image data and the second image data received from the image capturer 710. For example, the third image data may be data obtained by subtracting the luminance value of the second image data corresponding to each pixel from the luminance value of the first image data. That is, since the first image data is image data captured while the light source unit 714 is activated, the subject 140 photographed at a position close to the light source unit 714 will have a high luminance value, and at a position far from the light source unit 714. The luminance value of the imaged background portion will be relatively small. In addition, since the second image data is image data captured while the light source unit 714 is inactivated, the difference between the luminance value of the subject 140 and the background portion is smaller than that of the first image data. In addition, since the background portions of the first image data and the second image data are both located far from the light source 714 and the subject, the difference in luminance value will be very small. Therefore, if the luminance value of the second image data corresponding to each pixel is subtracted from the luminance value of the first image data, the luminance value of the background portion except for the subject 140 will be almost 'zero'. Only the luminance value of the portion corresponding to the subject 140 will exist. In this manner, the subject extractor 731 may generate third image data obtained by extracting only the subject 140 from the captured image data.

The center determining unit 732 analyzes the third image data generated by the object extracting unit 731 to determine the center point of the subject 140 (hereinafter, referred to as a reference pixel). ). For example, it is assumed that the resolution of the image sensor unit 718 is 640 * 480 (horizontal * vertical), and luminance values of each pixel are divided by 0 to 255. If the luminance value is '0', the brightness is '0', so it will be black, and if the luminance value is '255', it will be represented as white because it is the brightest case. If the preset reference boundary value is '10, 000 'for extracting the reference pixel, the center determination unit 732 may extract 10,000 reference pixels in the order of the pixels having the highest luminance value from the third image data. In this case, if the reference boundary value is too large, most pixels of the third image data will be extracted as the reference pixel, and if the reference boundary value is too small, only a portion of the third image data will be extracted as the reference pixel. In this case, the ratio of pixels (ie, noise pixels) to the background part irrelevant to the reference pixel other than the imaged image of the object 140 will be increased. That is, if the reference boundary value is too large or too small, the ratio of the noise pixels is high, so the center determination unit 732 may have difficulty in determining the exact position of the photographed subject 140. Therefore, the upper limit value and / or the lower limit value may be set so that the reference boundary value can ignore the ratio of the noise pixel. For example, assume that the reference boundary value is '10, 000 'and the lower limit value is' 50'. In this case, the center determination unit 732 may extract 10,000 reference pixels in the order of the pixels having the highest luminance from only the pixels having the luminance greater than 50 in the third image data. That is, the center determination unit 732 analyzes each pixel of the third image data and extracts 10,000 reference pixels in order of increasing luminance value, and among them, a pixel having a luminance value of less than 50 (or less) is determined from the reference pixel. Can be excluded. In addition, since the subject 140 will generally be a user's finger, if the luminance value is too large, it is likely to correspond to a noise pixel. Therefore, assuming that the reference boundary value is '10, 000 'and the upper limit value is' 200', the center determination unit 732 targets only pixels whose luminance value is less than (or less than) 200 in the third image data. The reference pixel can be extracted.

In addition, the reference boundary value may be a predetermined luminance value instead of the number of pixels having a large luminance value. That is, the center determination unit 732 may analyze the third image data to extract all pixels that exceed the reference boundary value as the reference pixels. For example, when the reference boundary value is set to '150', the center determination unit 732 may extract all pixels whose luminance value exceeds '150' from the third image data as the reference pixel. In this case as well, as described above, if the reference boundary value is too large or too small, the ratio of the noise pixels in the reference pixel will be high, so it is obvious that the upper limit value and / or the lower limit value for the reference boundary value can be set.

In addition, the center determination unit 732 analyzes the reference pixel to determine the center point of the subject 140. In this case, the center point may be the center of gravity of the subject 140. For example, the center of gravity, which is a specific coordinate located inside the subject 140, may be calculated by the following equation.

(7)

(7)

(8)

(8)

(9)

(9)

Here, R is the center of gravity, M is the sum of the luminance values of the reference pixels of the subject, m _i is the luminance value of each reference pixel of the subject, r _i is the coordinate value of the reference pixel. The subscripts x and y represent the values corresponding to the X and Y axes, respectively, and Equation (9) is a vector representation of Equations (7) and (8).

However, when the center judging unit 732 pins the center point of the subject 140 by the above-described equation, all of the luminance values of the reference pixels must be reflected, thereby increasing the amount of computation, thereby increasing the overall amount of the pointer movement control apparatus 700. Problems may occur that degrade functionality. Therefore, the center point of the subject 140 may be determined by the following equation.

(10)

10

(11)

(11)

(12)

(12)

Where R is the center of gravity, ri is the coordinate value of the reference pixel, and n is the number of reference pixels. Subscripts x and y represent values corresponding to the X and Y axes, respectively, and Equation (12) is a vector representation of Equation (10) and Equation (11).

That is, the center determination unit 732 may determine the center point of the subject 140 without reflecting the luminance value of the reference pixel. Since the luminance value of each reference pixel does not have a large difference, the center determination unit 732 may calculate the average value of the coordinate values of the reference pixel as the center point of the subject 140 without reflecting it. For example, when the luminance value of the reference pixel is divided into '1' and the luminance value of the remaining pixels are divided into '0', the third image data may be white in the reference pixel (that is, the portion where the subject 140 is captured). Will be displayed and the rest will be black. Therefore, the center of gravity of the reference pixel (that is, the subject 140) displayed in white may be obtained without reflecting the luminance value of each pixel by the above equations (10 to 12).

In addition, when the outline of the reference pixel is curved, the center determination unit 732 may calculate a radius of curvature and extract a coordinate value of a center point corresponding to the radius of curvature. The method for calculating this is described above with reference to FIG. 2, and a detailed description thereof will be omitted herein.

The pointer position setting unit 734 controls the pointer of the display unit 740 to be located at the calculated specific coordinates. That is, the position of the pointer is specified to a specific coordinate, for example, a coordinate value of a center point (for example, a center of gravity) of the subject 140.

8 is a flowchart of a pointer movement control method according to a second preferred embodiment of the present invention.

Hereinafter, a method of controlling pointer movement of a display according to a second exemplary embodiment of the present invention will be described with reference to FIG. 8.

In operation S810, the image sensor unit 718 photographs the subject 140 while the light source unit 714 is activated, that is, outputs light, generates first image data, and outputs the first image data to the image processor 730.

In operation S820, the image sensor unit 718 photographs the subject 140 while the light source unit 714 is inactive, that is, does not output light, generates second image data, and outputs the second image data to the image processor 730. .

In this case, the light source unit 714 may be turned on at a predetermined time interval under the control of the main controller (not shown), and the image sensor unit 718 may immediately turn on the light source unit 714 immediately after generating the first image data. When deactivated, second image data may be generated.

In operation S830, the object extractor 731 generates third image data by comparing the first image data and the second image data received from the imaging unit 710. A method of generating third image data has been described above with reference to FIG. 7, and a detailed description thereof will be omitted herein.

In operation S840, the center determiner 732 analyzes the third image data generated by the object extractor 731 to determine the center point of the subject 140 (hereinafter, referred to as “reference”). Pixels').

In operation S850, the center determination unit 732 extracts a coordinate value of the reference pixel, and in operation S870, the center determination unit 732 calculates a center point of the subject 140. The method for calculating the center point of the subject 140 has been described above with reference to FIG. 10, and a detailed description thereof will be omitted herein.

In operation S870, the pointer position setting unit 734 controls the pointer to be positioned at the position of the display unit 740 corresponding to the calculated center point of the subject 140. That is, the position of the pointer is specified to the coordinate value of the center point (for example, the center of gravity) of the subject 140.

9 is a schematic diagram showing a configuration of a pointer movement control apparatus according to the present invention.

Referring to FIG. 9, when the subject 140 moves in a predetermined direction on the light transmitting unit 910 through which light may pass, the image sensor unit 930 may move the subject 140. Photographing is performed continuously and information corresponding to the movement of the subject 140 is input by a controller (not shown). For example, the light emitted from the light source unit 950 is reflected from the user's finger, which is the subject 140, and is incident on the image sensor unit 930, and the image sensor unit 930 is light reflected by the subject 140. Can be received to generate corresponding image data. Accordingly, the movement of the subject is recognized by the controller (not shown), so that the pointer is desired by the user on the display unit of the electronic device (eg, a mobile communication terminal, a laptop, etc.) equipped with the pointer movement control device 990. You can move it to a location.

When the electronic device equipped with the pointer movement control device 990 is a mobile communication terminal, the user grips the mobile communication terminal with one hand and moves the pointer to a desired position using the thumb of the other hand. . However, the user's finger is generally not positioned at the center of the light transmitting part 910, that is, the center of the image sensor part 930, and as shown in FIG. 9, slightly below the center of the image sensor part 930. It will often be located. In this case, since the image sensor 930 only photographs a part of the subject 140, it may be difficult to accurately recognize the movement of the subject 140.

In order to solve such a problem, a wide-angle lens capable of increasing an imaging angle of the image sensor unit 930 may be used. However, the wide-angle lens has a problem of causing an increase in the overall price of the optical pointing device because the unit price is high, and the wide-angle lens may cause image distortion since the wide-angle lens is able to take a wider range when shooting at the same distance.

Hereinafter, the structure of the pointer movement control apparatus for solving the above-mentioned problem is demonstrated.

10 is a cross-sectional view of the moved pointer movement control apparatus according to a preferred embodiment of the present invention, Figure 11 is a plan view of a moved pointer movement control apparatus according to a preferred embodiment of the present invention.

Referring to FIG. 10, the pointer movement control apparatus 1000 according to the second embodiment of the configuration of the pointer movement control apparatus includes an image sensing unit 1070 including an image sensor unit 1030, a lens unit 1050, and a light source unit 1060. ), A housing 1090 for accommodating the image capturing unit 1070, a control unit for processing the image data generated by the light transmitting unit 1010 and the image capturing unit 1070 provided on the housing 1090 so that information is input. (1099). In addition, on the light emitting unit 1010 is shown a subject 140 for extracting the movement to input information. Hereinafter, each component is demonstrated in detail. However, the description of the image capturing unit 1070 (that is, the image sensor unit 1030, the lens unit 1050, the light source unit 1060, etc.) has been described above.

The light transmitting unit 1010 is exposed to the outside, and the subject (for example, the user's finger) 140 is positioned on the upper portion thereof. The light emitted from the light source unit 1060 passes through the light projecting unit 1010, is reflected by the subject 140, and then enters the image sensor unit 1030 through the light projecting unit 1010 and the lens unit 1050. The light transmitting unit 1010 may be manufactured using glass or a transparent plastic having light transparency.

The light transmitting unit 1010 may have a circular shape and have an X axis passing through a center thereof. The center line Y of the image capturing unit 1070 including the image sensor unit 1030 and the lens unit 1050 is spaced apart from the X axis at a predetermined interval. The direction in which the image capturing unit 1070 is spaced apart from the X axis is downward from the center of the light transmitting unit 1010 as shown in FIG. 11. Therefore, even if the subject 140 is slightly below the center of the light emitter 1010, the image capturing unit 1070 can capture the subject 140 more clearly.

In a portable terminal such as a mobile communication terminal or a PDA (Personal Digital Assistant), the user holds the terminal with one or both hands and uses the terminal while holding a terminal at a predetermined inclination angle with respect to the stretching direction of the user. The user's vision also stares at the terminal at an angle. Therefore, even if the user visually recognizes that the center of the subject 140, such as his finger or a ballpoint pen, is located at the center of the light-emitting unit 1010, the subject 140 is actually lower side of the light-emitting unit 1010 as shown in FIG. 10. It tends to be used in the direction of orientation. Therefore, the pointer movement control apparatus 1000 according to the second embodiment of the configuration of the pointer movement control apparatus moves the image capturing unit 1070 downward to move the image capturing unit 1070 to the center (Y axis) of the image capturing unit 1070. It is preferable to make the image of the subject 140 more clearly by making the center of the subject 140 as close as possible.

Rotating protrusions 1091 may be formed along the circumference of the light transmitting part 1010. The rotation protrusion 1091 may be inserted into the guide groove 1095 of the cover 1093 to allow rotation of the light transmitting unit 1010 and the housing 1090. And an adjustment knob 1020 may be provided on the upper surface of the light transmitting unit 1010. Accordingly, the user may rotate the light transmitting unit 1010, the housing 1090, and the imaging unit 1070 accommodated inside the housing 1090 at a desired angle by using the adjustment knob 1020. By adjusting the adjusting knob 1020 in the opposite direction to the image capturing unit 1070 with respect to the center line (X axis) of the light transmitting part 1010, the adjusting knob 1020 prevents the image capturing of the image capturing unit 1070 from being acquired. The user's operation can be made easy.

Although not shown in the drawings, a locking protrusion (not shown) and a locking groove (not shown) into which the locking protrusion may be inserted may be formed at portions that face each other in the rotating protrusion 1091 and the guide groove 1095. Therefore, when the user rotates the light emitting unit 1010 by using the adjustment knob 1020, the locking projection is rotated to be caught in the locking groove at regular intervals to feel a sense of rotation. In addition, the projections and the locking grooves may be formed at a predetermined angle, for example, at intervals of 15 °, thereby allowing the user to rotate the light projector 1010 and the imaging unit 1070 by a desired angle.

In addition, although not shown, the light transmitting unit 1010 may be equipped with an infrared filter. In this case, an infrared LED or the like may be used as the light source unit 1060 so that the image sensor unit 1030 receives only the light reflected from the object 140 among the light emitted from the light source unit 1060, and other light may be filtered by the infrared filter. .

The illustrated cover 1093 may correspond to one surface of the display unit or the main body of the electronic device equipped with the pointer movement control apparatus 1000.

The controller 1099 is connected to the image capturing unit 1070 to receive image data generated by the image capturing unit 1070 and process the image so that the pointer is displayed on the display. The controller 1099 may be configured as a separate block from the image sensor unit 1030, but may also be a block formed integrally with the image sensor unit 1030. The controller 1099 may be the image processor 230 and / or 730 described with reference to FIGS. 2 and / or 7.

12A and 12B are plan views of a pointer movement control device moved in a constant direction according to a preferred embodiment of the present invention.

Referring to FIG. 12A, when the subject 140 is the thumb of the user's left hand, the subject 140 is generally positioned on the left side with respect to the longitudinal axis a passing through the center of the light-transmitting unit and positioned below the horizontal axis b. Therefore, in order to allow the image capturer 1070 to more easily capture an image of the subject 140, the user uses the adjustment knob 1020 to adjust the light emitter 1010 and the image capturer 1070 to the vertical axis a. Rotate clockwise. As a result, the rotated image sensor unit 1030-1 may capture more portions of the subject 140 than the image sensor unit 1030 before rotation.

Referring to FIG. 12B, when the subject 140 is the thumb of the right hand, the subject 140 is generally positioned on the right side with respect to the vertical axis a passing through the center of the light-transmitting part and positioned below the horizontal axis b. Therefore, in order to allow the image capturer 1070 to more easily capture an image of the subject 140, the user uses the adjustment knob 1020 to adjust the light emitter 1010 and the image capturer 1070 to the vertical axis a. Rotate counterclockwise. As a result, the rotated image sensor unit 1030-2 may capture more portions of the subject 140 than the image sensor unit 1030 before rotation.

12A to 12B, the user rotates the image sensor unit 1030 or the like according to the position of the subject 140, but the image sensor unit 1030 or the like is rotated at a predetermined angle with respect to the vertical axis a. Can be fixed at That is, when the pointer movement control apparatus 1000 is mounted on the portable terminal and the user is left-handed, the image sensor unit 1030 may be fixed while being rotated in the clockwise direction with respect to the vertical axis a as shown in FIG. 12A. In the case of a right-handed user, the image sensor unit 1030 may be fixed while being rotated counterclockwise with respect to the vertical axis b as shown in FIG. 12B. As such, when the light transmitting unit 1010 and the housing 1090 including the image sensor unit 1030 are fixed with respect to the cover 1093, it is necessary to have a configuration such as the rotation protrusion 1020 and the guide groove 1095. There will be no.

Hereinafter, a pointer movement control method by a pointer movement control apparatus moved in a counterclockwise direction according to a preferred embodiment of the present invention described with reference to FIG. 12B will be described.

FIG. 13 is a diagram illustrating a method of calculating a movement displacement of a pointer according to the movement of the subject illustrated in FIG. 12B. Hereinafter, a description will be given focusing on the components shown in FIG. 2A described above.

만큼)로 시계 반대 방향으로 회전되어 있다. 즉, 이미지 센서부(1030-2)의 중심을 영점으로 하는 장치 좌표계(도 13에서는 수직축이 c축, 수평축이 d축임)는 기준 좌표계와 일정한 각도(도 13의 경우

만큼)로 시계 반대 방 향으로 회전되어 있다. 기준 좌표계와 장치 좌표계의 수직축이 동일(즉, a축과 c축이 동일)한 경우에는 이미지 처리부(230)가 피사체(140)의 이동 방향과 포인터(160-1, 160-2 등, 이하 '160'으로 통칭함)의 이동 방향이 동일하도록 포인터(160)의 이동을 제어할 수 있을 것이다. Referring to FIG. 13, the image sensor unit 1030-2 has a reference coordinate system having a center of the light transmitting unit 1010 as a zero point (a vertical axis is an a axis and a horizontal axis is an b axis in FIG. 13) and a constant angle (in the case of FIG.

Counterclockwise). That is, the device coordinate system having the center of the image sensor unit 1030-2 as a zero point (the vertical axis is the c-axis and the horizontal axis is the d-axis in FIG. 13) has a constant angle with the reference coordinate system (in the case of FIG. 13).

Rotate counterclockwise. When the vertical axes of the reference coordinate system and the device coordinate system are the same (that is, the a-axis and the c-axis are the same), the image processor 230 may move the subject 140 and the pointers 160-1, 160-2, etc. The movement of the pointer 160 may be controlled to have the same moving direction.

만큼의 방향으로 이동하였으나, 이미지 처리부(230)에서는 장치 좌표계의 수평축(즉, d축)을 기준으로 시계 방향으로

만큼의 방향으로 이동한 것으로 판단할 수 있다. 즉, 포인터(160)가 160-1의 위치에 있는 경우에 사용자는 160-2의 위치에 포인터(160)를 위치시키고자 하였으나 포인터는 160-3의 위치에 위치될 수 있다. 따라서, 이미지 센서부(1030-2)가 기준 좌표계를 기준으로 회전된 정도를 보정해야 사용자가 원하는 위치로 포인터를 위치시킬 수 있을 것이다.However, this is not the case when the device coordinate system is rotated at a predetermined angle with the reference coordinate system as shown in FIG. That is, the user looks at the pointer 160 displayed on the display 150 and moves the subjects 1340-1, 1340-2, etc. (hereinafter referred to as '1340') to move the pointer 160 to a desired position. On the contrary, since the image processor 230 analyzes image data of the image of the subject 140 and determines the moving direction of the subject 1340, the image processor 210 is different from the actual moving direction. For example, as shown in FIG. 13, when the subject is moved (ie, moved from 1340-1 to 1340-2), the center of the subject is substantially based on the horizontal axis (ie, the b axis) of the reference coordinate system. Counterclockwise

Although moved in the direction, the image processing unit 230 in the clockwise direction relative to the horizontal axis (that is, the d-axis) of the device coordinate system

It can be judged that it has moved in the same direction. That is, when the pointer 160 is located at the location of 160-1, the user attempts to place the pointer 160 at the location of 160-2, but the pointer may be located at the location of 160-3. Therefore, the image sensor unit 1030-2 may correct the degree of rotation based on the reference coordinate system so that the user may position the pointer to a desired position.

Hereinafter, a method of correcting this will be described with reference to FIG. 14.

14 is a flowchart illustrating a pointer movement control method according to a third embodiment of the apparatus for controlling a pointer movement according to the present invention.

Referring to FIG. 14, steps S310 to S360 are the same as or similar to the steps S310 to S360 described above with reference to FIG. 3, and thus a detailed description thereof will be omitted. In addition, it is assumed that the operation to be described below is an operation performed by the image processor 230 included in the pointer movement control apparatus 200 according to the third embodiment of the configuration of the pointer movement control apparatus. For example, the operation may be performed by the pointer position setting unit 239 included in the image processor 230. Of course, it may be performed by a separately provided unit (for example, a controller) instead of the pointer position setting unit 239. Hereinafter, the pointer position setting unit 239 includes a device movement direction measuring unit (not shown) and a device coordinate system for measuring a device movement direction, which is a degree in which the reference coordinate system and the device coordinate system preset in the pointer movement control device 200 are inclined. An outline moving direction measuring unit (not shown) for measuring the outline moving direction, which is a moving direction of the outline as a reference, and a pointer moving direction measuring unit (not shown) for measuring the pointer moving direction correcting the device moving direction in the outline moving direction And a pointer movement distance measuring unit measuring a pointer movement distance corresponding to the movement distance of the subject. At this time, the device movement direction measuring unit (not shown) measures the device movement direction, which is the degree to which the pointer movement control device is tilted in the counterclockwise direction with respect to the reference coordinate system, and the outline movement direction measuring unit is based on the device coordinate system. The outline movement direction, which is the degree of inclination of the outline in a clockwise direction, is measured, and the pointer movement direction measuring unit may measure the pointer movement direction by adding the device movement direction counterclockwise in the outline movement direction.

를 측정할 수 있다.In step S1410, the device moving direction measuring unit measures the device moving direction, which is the degree by which the image sensor unit 1030-2 is rotated based on the reference coordinate system. For example, referring to FIG. 13, the image sensor unit 1030-2 rotates in a counterclockwise direction with respect to the reference coordinate system.

Can be measured.

를 측정할 수 있다. 여기에서는 이해의 편의를 도모하기 위하여 외곽선 이동 방향을 시계 방향으로 측정하였으나, 시계 반대 방향으로 외곽선 이동 방향을 측정하여도 동일한 결과임은 자명하다. 즉, 도 13에서 시계 반대 방향으로 외곽선 이동 방향을 측정하면

이므로 시계 방향으로 측정한 경우와 부호만 상이할 뿐이다.In operation S1420, the outline movement direction measuring unit calculates the center of gravity of the object 1340 described above with reference to FIGS. 2 and / or 7, or calculates a radius of curvature from the outline of the extracted object 1340 and the corresponding radius of curvature. Measure the direction of outline movement by extracting the coordinate value of the center point corresponding to. For example, referring to FIG. 13, the degree of rotation in the clockwise direction with respect to the horizontal axis (that is, the d-axis) of the device coordinate system is shown.

Can be measured. Here, the direction of movement of the outline is measured in the clockwise direction for convenience of understanding, but the same result is apparent even if the direction of movement of the outline is measured in the counterclockwise direction. That is, if the outline movement direction in the counterclockwise direction in Figure 13 is measured

Therefore, only the sign is different from the case measured in the clockwise direction.

)을 측정한 다. 예를 들어, 포인터 이동 방향은

에서

를 시계 반대 방향으로 더한 값일 수 있다. 즉 포인터 이동 방향은

일 수 있다. 이때, 외곽선 이동 방향은 장치 좌표계의 수평축(즉, d축)을 기준으로 시계 반대 방향으로 회전한 정도이다. In step S1430, the pointer movement direction measuring unit considers the device movement direction measured in the movement direction of the measured center point of the outline (the pointer movement direction (

) For example, the pointer movement direction

in

Can be added in the counterclockwise direction. That is, the direction of pointer movement

Can be. In this case, the outline movement direction is the degree rotated counterclockwise with respect to the horizontal axis (ie, the d axis) of the device coordinate system.

In operation S1440, the pointer movement distance measuring unit measures the distance that the subject has moved (that is, the distance 'l' at which the center of the subject 1340 has moved in FIG. 13) and correspondingly measures the pointer movement distance, which is the distance the pointer moves. Measure and extract the coordinates of the center point to which the pointer will move.

In operation S380, the pointer of the display unit is moved in response to the extracted coordinates of the center point.

Other specific components for the pointer movement control device according to an embodiment of the present invention, common platform technology such as embedded system, O / S, communication standardization, interface standardization technology such as I / O interface, actuator, battery, camera, sensor Detailed description of the parts standardization technology, etc. will be omitted.

The pointer movement control method according to the embodiment of the present invention described above may be performed in combination with a predetermined apparatus, for example, a mobile communication terminal after being stored in a recording medium. Here, the recording medium may be a magnetic or optical recording medium such as a video tape, a CD, a VCD, a DVD, or a database of a client or server computer built offline or online.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

As described above, the method and the apparatus for controlling the pointer movement according to the present invention have the effect of controlling the pointer on the display by extracting the movement of the subject regardless of the brightness around the subject.

In addition, the pointer movement control method and apparatus according to the present invention has an effect that does not require a separate light source to illuminate the subject in order to extract the movement of the subject.

In addition, the pointer movement control method and apparatus according to the present invention has the effect that a plurality of image frames are not required to extract the movement of the subject.

In addition, the pointer movement control method and the apparatus according to the present invention has the effect of controlling the pointer on the display using the camera imaging device.

In addition, the pointer movement control method and apparatus according to the present invention can omit a part of an existing keypad, thereby increasing spatial utilization of the electronic device, and miniaturizing the electronic device.

In addition, the pointer movement control method and apparatus according to the present invention has the effect of simplifying the manufacturing process and manufacturing cost of the electronic device.

In addition, the pointer movement control method and apparatus according to the present invention has the effect of maximizing component utilization by making it possible to universally use the provided camera.

In addition, the pointer movement control method and apparatus according to the present invention has an effect that can provide a pointer movement control apparatus and method that can adjust the position of the pointer movement control apparatus according to the user's use environment.

Although the above has been described with reference to a preferred embodiment of the present invention, those of ordinary skill in the art to the present invention without departing from the spirit and scope of the present invention and equivalents thereof described in the claims below It will be understood that various modifications and changes can be made.

Claims (24)

A method of controlling a pointer movement according to a movement of a subject by a pointer movement control device which is moved in a predetermined direction based on a preset reference coordinate system, (a) photographing the subject to generate first image data; (b) generating second image data by adjusting the sharpness of the first image data; (c) extracting a difference value between the first image data and the second image data and extracting a predetermined outline corresponding to one end of the subject; And and (d) positioning the pointer at a specific position within the outline by reflecting the degree of movement of the pointer movement control device. The method of claim 1, Step (d) is Measuring a device movement direction in which the reference coordinate system and the device coordinate system preset in the pointer movement control device are inclined; Measuring a movement direction of a specific position in an outline which is a movement direction of the outline based on the device coordinate system; Measuring a pointer movement direction in which the device movement direction is corrected in the outline movement direction; Measuring a pointer movement distance corresponding to a movement distance of the subject; And And positioning the pointer at a position corresponding to the pointer movement direction and the pointer movement distance. The method of claim 2, The device movement direction is an angle measured by the degree of tilt of the pointer movement control device in a counterclockwise direction with respect to the reference coordinate system, and the outline movement direction is inclined in the counterclockwise direction relative to the device coordinate system. And a degree of measuring the degree, and the pointer movement direction is a value obtained by adding the device movement direction counterclockwise from the outline movement direction. The method of claim 1, Step (c) is And gamma correcting the difference value. The method according to claim 1 or 4, And extracting a difference value having a luminance value greater than a reference luminance value to extract a predetermined outline corresponding to one end of the subject. The method of claim 1, In step (b), And controlling the sharpness of the first image data by increasing at least one of color contrast, brightness contrast, and saturation contrast of pixels of the image of the subject. The method of claim 1, The specific position in the outline is a pointer movement control method, characterized in that the center of gravity of the outline. The method of claim 7, wherein The center of gravity of the outline is calculated by the following equation.

here,

Is the center of gravity, M is the sum of the luminance values of the pixels representing the outline, m _i is the luminance value of each pixel representing the outline,

Is the coordinate value of the pixel representing the outline. The method of claim 7, wherein The center of gravity of the outline is calculated by the following equation.

here,

Is the center of gravity, n is the number of pixels representing the outline,

Is the coordinate value of the pixel representing the outline. The method of claim 1, The predetermined outline is curved, and a coordinate value corresponding to the specific position is a coordinate value of a center point determined by the curved outline. The method of claim 10, Step (b) is Calculating a distance between each pixel representing the curved outline; And Extracting a coordinate value of the center point by using a median value of two pixels having the largest distance of each pixel, and calculating a specific position included in the outline. . The method of claim 1, Step (a) is, And photographing the subject by using light reflected from the light source included in the pointer movement control device and reflected on the subject. The method of claim 12, The light emitted from the light source unit is a pointer movement control method, characterized in that the infrared. The method of claim 13, And the infrared rays pass through an infrared filter included in the pointer movement control device. In the pointer movement control device which is moved in a predetermined direction with respect to a preset reference coordinate system and controls the movement of the pointer according to the movement of the subject, An imaging unit configured to image the subject to generate first image data; A sharpness controller configured to generate second image data by adjusting the sharpness of the first image data captured by the image pickup unit; An outline extracting unit which generates a difference value between the first image data and the second image data and extracts a predetermined outline corresponding to one end of the subject; And And a pointer position setting unit for positioning the pointer at a specific position within the outline by reflecting the degree of movement of the pointer movement control device. The method of claim 15, The pointer position setting unit A device movement direction measuring unit for measuring a device movement direction in which the reference coordinate system and the device coordinate system preset in the pointer movement control device are inclined; An outline moving direction measuring unit measuring an outline moving direction which is a moving direction of the outline based on the device coordinate system; A pointer moving direction measuring unit measuring a pointer moving direction in which the device moving direction is corrected in the outline moving direction; And And a pointer movement distance measuring unit measuring a pointer movement distance corresponding to the movement distance of the subject. The method of claim 16, The device movement direction measuring unit measures the device movement direction, which is the degree to which the pointer movement control device is inclined counterclockwise based on the reference coordinate system, The outline movement direction measuring unit measures an outline movement direction, which is an extent of inclination of the outline in a counterclockwise direction based on the device coordinate system, The pointer movement direction measuring unit measures the pointer movement direction by adding the device movement direction counterclockwise from the outline movement direction. The method of claim 15, And a gamma correction unit configured to gamma correct a predetermined outline extracted by the outline extracting unit. The method of claim 15, And a center determination unit for obtaining coordinates corresponding to a specific position included in the predetermined outline. The method of claim 15, And a light source unit provided around the image pickup unit and irradiating light to the subject. The method of claim 20, The light emitted from the light source unit pointer movement control device, characterized in that the infrared. The method of claim 21, And an infrared filter for passing only the infrared light emitted from the light source unit. A method of controlling a pointer movement according to a movement of a subject by a pointer movement control device which is moved in a predetermined direction based on a preset reference coordinate system,  (a) photographing the subject to generate first image data while the light source unit is activated; (b) photographing the subject and generating second image data while the light source unit is inactive; (c) generating third image data by comparing the first image data with the second image data; (d) calculating a center point of the photographed subject by analyzing the third image data by reflecting the degree of movement of the pointer movement control device; And (e) positioning the pointer at a coordinate corresponding to the center point. In the pointer movement control device which is moved in a predetermined direction with respect to a preset reference coordinate system and controls the movement of the pointer according to the movement of the subject, A light source unit which is turned on at a predetermined time interval; An image sensor unit configured to photograph the subject while the light source unit is activated to generate first image data, and to generate second image data by photographing the subject while the light source unit is deactivated; A subject extraction unit configured to generate third image data by comparing the first image data and the second image data; A center point determination unit which calculates a center point of the photographed subject by analyzing the third image data by reflecting the degree of movement of the pointer movement control device; And And a pointer position setting unit for positioning the pointer at a coordinate corresponding to the center point.

Images