WO2004015440A2

WO2004015440A2 - A device for making 3d movement data by detecting an object in 3d space by camera

Info

Publication number: WO2004015440A2
Application number: PCT/KR2003/001447
Authority: WO
Inventors: Woncheol Jeong
Original assignee: Jung Wonchul
Priority date: 2002-08-13
Filing date: 2003-07-22
Publication date: 2004-02-19
Also published as: AU2003248486A8; WO2004015440A3; AU2003248486A1

Abstract

This invention is the device that measures the change of position in 3-dimensional space. For that that,-this invention consists of the particle moved by gravity, the body including the particle, the camera that recognize the position of the particle, the equipment that store the data of pressure, the button equipment that select the function, the equipment that operate the position change in 3D throughout the angle measured, the pressure or the input of button, and the equipment that transmits the effect of data operated. In this invention, the device calibrates the angle by sensing the position of the particle produced by tilting or acceleration and calibrates the position change in horizontal plane by converting the angle into the distance, calibrates the position change in 3D by calibrating the movement in vertical direction throughout the change of input data in the case that the device go up or down. This invention is the device that measures the position change in 3-dimensional space. To realize that, this invention consists of the ball which is moved by gravity, the body which the ball can be filled in, the camera which recognizes the position of the ball, the memory device which stores the pressure data, the buttons which select the functions, the main processing unit which calculates the position change in 3D space by converting from angle, pressure or input button, the device which transmits the output result to outer device. In this invention, the device recognizes the position change of the ball which is moved by tilting or accelerating. It calibrates the leaned angle, gets the position change data in horizontal plane by converting the angle into length data and gets the vertical position change by change of pressure data. And finally this invention gets the position change data in 3D space.

Description

[Description]

[Name of the invention]

A device for making 3D movement data by detecting an object in 3D space by-

5 camera

[Technical Field]

This invention is the device and the system that measures the relative movement in 3D

10 space. Especially, this invention has an inclination to measure the movement by detecting the

ball movement by using optical method mainly

[Background Art]

1.5 The existing device that measures the movement includes an usual 2D mouse for PC .

The existing mouse measures the movement by detecting the rolling every unit time or using

beam sensor. And there are also the touch-pad, touch-pen used in notebook and PDA in the

existing device.

20 The movement of such existing devices is restricted in 2D space, therefore the acquired

data from such existing devices are limited in 2 dimensional. [Disclosure of Invention]

This invention has an intention to get the position data on 2D space by using the

movement data ofthe ball in tilted equipment, by using the vertical movement data by using

the movement ofthe ball and the pressure data in moving up or down to get the movement

data on 3D space in conclusion.

[Brief Description of Drawings]

FI is a drawing that shows the example ofthe device that measures the movement in

3D space in this invention

F2 is a cubic drawing that shows the example ofthe device that measures the movement

in 3D space in this invention.

F3 is a side drawing that shows the example ofthe device that measures the movement

in 3D space in this invention.

F4 is a front drawing that shows the example ofthe device that measures the movement

in 3D space in this invention.

F5 is a system flowchart that shows the steps that measure the movement in 3D space in

this invention. F6 is a drawing that defines the direction ofthe device in this invention.

F7 is a cubic drawing ofthe equipment that produces the position change by using the

hemisphere frame in this invention.

F8 is a side drawing ofthe equipment that produces the position change by using the

hemisphere frame in this invention.

F9-1 is a side drawing ofthe leaned equipment that produces the position change by

using the hemisphere frame in this invention.

F9-2 is a front drawing ofthe leaned equipment that produces the position change by

using the hemisphere frame in this invention.

F10 is a side drawing ofthe equipment leaned and moved down, which produces the

position change by using the hemisphere frame in this invention.

FI 1 is a block diagram that shows the example without the equipment that produces the

position change in this invention.

F12 is an arrangement diagram ofthe inside of main box in this invention included in the example.

100: the frame that includes the ball

110: the ball in the frame

120: the liquid (or gas) that is filled up in the frame

200: camera

300: the main processing device that includes control part for operating the data near

the speed of recognition.

400: the memory device

500: the electronic device

510: the connection part ofthe electronic device

600: the button device

610 - 650: the button

700: the main body

710: the folding part

720: the tightening part

730: the main box

900: the pressure sensor

[Best mode for carrying out the invention] This invention has the feature that consists of 6 devices, which are the device that

produces the position change, the camera device that measures the position(sensing device 1) ,

the pressure sensor device that measures the position change in vertical direction(sensing

device 2) , the memory device that stores the position data from the camera device and

pressure sensor device , the main processing device that operates the data from the memory

device and converts into the relative position change data in 3D, the device that transmits the

data into outer device.

Also the measure procedure has the 4 steps below. The 1st step is to measure the position

change ofthe ball which is moved by gravity or inertia. The 2nd step is to calculate the

distance from detecting the position between sampling time. The 3rd step is to recognize the

position change in vertical direction. The 4th step is to store the measured data and converted

data. The 5th step is to operate the position change in vertical direction by using the stored

data. The 6th step is to operate the position change in vertical direction by using the stored

data.

The next contents will explain details through referring to the example of this invention.

Dl shows elements which compose the device for measuring the position change. The

frame(lOO) is the device that produces the position change ofthe ball by gravity or inertia.

The frame shapes a hemisphere or a polyhedron and it can be filled with liquid or gas. The

ball in the frame(lOO) can be detected by camera device(200) or other optical methods. The main processing device(300) senses the position of particle and converts (the position) into

the numerical data. The position data in vertical direction is measured by the pressure

device(900). This data is corrected through the filter device and stored to the memory

device(400). The main processing device(300) calculates the position change in horizontal

plane by using the position data from the camera(200) on every sampling time. At the same

time, the main processing device operates the angle between the 1st direction and the 3rd

direction and between the 1st direction and the 2nd direction. And it detects the acceleration

in vertical direction (the 1st direction). Following time, the main processing device gets the

change ofthe acceleration data and calculates the position change. The device to

transmit(800) transmits the 3D position change data to the outside device.

D5 shows the steps to measure the position change. The measurement procedure consists

of 6 steps. The 1st step is to measure the position ofthe particle(SlOl) , the 2nd step is to

calculate the position change and the angle through the data from 1st step(S102) , the 3rd step

is to measure the pressure(S103) , the 4th is step to store the measured pressure data(S104) ,

the 5th step is to operate the position change in horizontal plane through the stored data(S105)

and the 6th step is to operate the position change in vertical direction through the stored

data(S106).

The below is the particular explanation of elements' function in respective steps.

The housing (100) can be realized in various shape like hemisphere, sphere or cubic. The shape of frame(lOO) can be referred to below example in the case ofthe hemisphere shape.

The camera device(200) is fixed to the main body near the frame(lOO) and that device

consists ofthe 1st camera which is set up in vertical direction to measure the horizontal

position change ofthe ball and the 2nd camera which is set up in horizontal direction to

measure vertical position change ofthe ball If the camera is analog type, the camera is

connected to the main processing device through the suitable conversion device. If the camera

is digital type, the camera connected to the main processing device directly or through the

suitable conversion device. The picture from these cameras is sent to the main processing

device and the main processing device detects the position where the pattern ofthe ball exists

in the image from the camera(S101). The main processing device compares the ball's position

ofthe current image with that ofthe last image and calculates the moving distance ofthe ball.

(SI 02) The pressure data from the pressure sensor set up in vertical is transmitted to the main

processing device (SI 03). The operated data is stored to the memory device in the form of

numerical and translatable data (SI 04), and the position change in horizontal direction can be

extracted through these data (S 105). These position data in horizontal data generates the

position data in vertical direction through the other operations and the input data(S106).

The method to store the digitalized data is as follows. (The main processing device(300)

samples the picture from the camera on every sampling time.) If the difference of new data

and old data is below some value (for example, 10%), the main processing device stores the

new data (sampling data) in the storage device. If not, the new data is recognized as an error

data and the main processing device continuously compares with the next sampling data which is stored in storage device. In this way, every data on every sampling time is stored in

memory device.

To measure the position change, the processing device (300) uses the data stored in the

storage device (400). The steps to measure the position change in the processing device(300)

are divided into one step to calculate the position change in horizontal plane(S105) and

another step to calculate the position change in vertical direction(S106).

The processing device (300) calculates the position change on horizontal plane as

follows (SI 04). The force in vertical direction is gravity force and the particle is moved by

gravity force with the angle of frame. Therefore the camera device, set up with the main body

in vertical direction, catches the movement ofthe particle in horizontal direction. The changed

distance and angle toward each direction can be calculated through the triangle functions.

(The particular formula is appeared as below example). In the same way, we can calculate the

angle through the 2nd direction and the 3rd direction (horizontal direction).

The position change in vertical direction can be caught in the 2nd camera that is set up in

horizontal direction and the pressure sensor that is set up the below of frame. These data can

show the position change by comparing with gravity known after initializing. The formula can

be various if it needs. For example, we can know the pressure change value on pure

acceleration through subtracting the pressure data without any acceleration from the raw

pressure data. Then, we can know the relative movement in vertical direction by multiplying the pressure change with the suitable function with θ .

Using up and down buttons(650,620) in this invention is another example to make a

relative movement data. The main processing device the position change according to the

period of pressing button or the frequency of pressing button. In this case, the device does not

measure the actual moved distance but produce the position change data intentionally. It can

be fitted by intention to control the outside device by using this invention.

We can know the relative position change in 3D space from adding the above result. The

position data can transfer to the outside device through the device to transmit (800). The way

to transmit can be variable in accordance with the kinds of industry. The outside device

including the receiving device can be used in various according to the cases of need.

There can be a difficulty to lean or move this invention continuously, therefore the range

can be limited. But, it can be solved by adding a pause button(610). For example, after

moving this device for measuring the movement to boundary we can save the current position

by pressing the button (610) and make an initial state while the button is being pressed (610).

After releasing the button (610), we can measure the position change continuously. The

control system can be modified to use easily according to the purposes of business men.

The device in D2 or D4 is the one ofthe applications in this invention and can be fitted

on the fingers and wrist. Also it can be used to acquire the movement data of hand or to control the outside device by moving the hand. The elements that are installed in the main

box(730) consist ofthe device to produce the pressure change, the device to sense , the

storage device , the processing device and the device to transmit.

Following explanations are about the stage to measure the position ofthe ball(SlOl) and

the stage to calculate the position change in vertical direction.(S106). In the case of changing

the housing shape it makes differences ofthe physical formula under the same principle.

Therefore the following formulas can be applied only in the each following examples. Each

direction is defined as D6, r is the radius of hemisphere, g is the gravity acceleration, θ is

the leaning angle between Z axis and the device, φ is the leaning between X,Y axis and the

device, x_x is the distance moved in the 2nd direction, and x_y is the distance moved in the 3rd

direction.

The Example 1 :

D9-1 and D9-2 are the side and plane drawings ofthe housing in this example. The

camera devices, used in this example, are placed on the upper part of frame in vertical

direction and on the side part of frame in horizontal direction.

The distance, in the stage to measure the distance ofthe particle (SI 02), is shown as

below. The gravity direction is always downward. Therefore the camera placed in horizontal

direction shows the picture like D9-1 and the camera placed in vertical direction shows the

picture like D9-2. In this situation, the distance between the center ofthe particle and the center of hemisphere is x. Then θ and φ are obtained through the pattern recognition,

{ The formula 1 } x = rx sin6> x. = rx sin#x cosζ_»

Xy = rx sin#x sinζ_>

The Example 2:

If the housing is forced into down way, the particle is separated from the housing by

inertia. Therefore we can know that the 2nd camera gets the picture like D10. At this time, the

distance between the center ofthe particle and the center ofthe hemisphere is r' and the

distance moved in the 1st direction is z.

{The formula 2} x = r'x sin# x. = r'xsin#xcosζZ> x_y = r'x sin θ x sin φ z = r -r

The Example 3:

If the housing is forced into upper direction, the particle is moved into the lowest point

ofthe housing. Therefore we can know that the 2nd camera gets the picture like D9-1 or D9-2.

But the distance moved in the 1st direction is different from the distance in Example 1. At this

time, the distance between the center ofthe particle and the center of hemisphere is r', the

distance in the 1st direction is z , P is the pressure data from the pressure sensor , c is a

constant with the leaning and f is a suitable function . {The formula 3} x = r'xsin6'

Xx = r 'xsin#xcos#> ^ r 'xsinøxsin^. z = f(P-c,θ)x(r-r')

[ Industrial Applicability ]

The moving of this device for measuring the movement in 3D is not limited in 2D,

therefore there is no limit of space to use and there is an effectiveness to know the movement

of 3D space.

The outer device can modify the data that are provided by the device for the movement

of 3D space according to the need of usage. Thus the applications ofthe device for measuring

the movement of 3D space can be very various. For example it can be used as a PC mouse, a

remote controller for controlling the specific device and a motion tracer attached to a moving

thing in 3D space. In addition the motion capture device or the attitude control device for a

flying thing.

Claims

[ Claims ]

[Claim 1]

The device for making the 3D movement comprising:

The housing device to produce said position change at Z(the axis of gravity),

X,Y(two axis crossing at right angle on the plane perpendicular to gravity) axis by tilting

or up-down moving;

The camera device to measure said position change in said housing device;

The sensor device to measure said position change data in gravity direction from

said housing device;

The memory device to store the data of said position change in said housing

device;

The button device to transmit said signal from the buttons including the up and

down button into said main processing device;

The main processing device that converts the data from said storage device into

the relative position data in 3D space;

The device to transmit the data from said processing device;

The device to consist of upper devices for measuring the movement in 3D space.

[Claim 2]

The device for making the 3D movement of Claim 1 comprising: The device for making the 3D movement , wherein the particle in said

housing device is sphere type and solid.

[Claim 3]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein said picture data

that transmit from the camera to said main processing device is digitalized.

[Claim 4]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein said housing device is

hemisphere type and solid.

[Claim 5]

The device for making the 3D movement of Claim 4 comprising:

The device for making the 3D movement , wherem said inside ofthe housing device

is filled with liquid.

[Claim 6] The device for making the 3D movement of Claim 4 comprising:

The device for making the 3D movement , wherein said inside ofthe housing

device is filled with gas.

[Claim 7]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein it has the device to produce

the pressure change;

The device to produce the pressure change consists ofthe below surface

and the upper surface;

The below surface is square and the upper surface consists of 4 sides that

liave the same length of a latus ofthe above square and 4 arcs that are same-shaped

and have the center that exists in out of said device to produce the pressure change;

The above 4 sides and 4arcs are connected by turns to make the closed diagram;

The below surface and upper surface have the same center. 4 sides ofthe below

surface and the upper surface are in parallel;

The area between the upper surface and below surface consists ofthe 1st area toward

the below surface and the 2nd area toward the upper surface;

The height ofthe 2nd area is the same as that ofthe square side ofthe below surface;

The shape of a section in the 2 area is the same as that ofthe below surface; The shape of a section in the 1st area is similar to that ofthe upper surface;

Each length of 4 sides that consist of a section in the 1st area is all same and these

4 sides are placed in a plane. As it is nearer toward the upper part ofthe arctic, the length

ofthe arctic is longer. Then there is no arctic area in the below ofthe 1st area;

The arctic area in the upper ofthe 1st area is converged to the below ofthe 2nd area;

And there exist sphere-typed solid inside ofthe device to produce the pressure change;

[Claim 8]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein said position change,

through inertia and gravity, is obtained by measuring the data from said pressure sensor.

[Claim 9]

The process to measure the movement in 3D space comprising the steps of:

a) measuring the position change of said particle by acceleration with gravity or inertia;

b) calculating the distance change by detecting the position change at the step a);

c) detecting the movement in vertical direction;

d) storing the measured data;

e) calculating the leaning angle between Z and X, Y direction and to convert

the angle data into distance data by using suitable formula; f) calculating the relative position change by using the converted data from

the inertia force in vertical direction;

[Claim 10]

The step f) of Claim 9 comprising the steps of:

a) comparing the stored data with the current data after sampling the

measured data per regular time;

b) storing the current data in the case that the difference between the stored data and

current data is less than certain value and to ignore the current data in the case that the

difference between the stored data and current data is more than certain value.

[Claim 11]

The process to measure the movement in 3D space of Claim 9 comprising the steps of:

a) measuring the position change ofthe particle by inertia force per regular sampling time;

b) calculating the change of acceleration from the sensed data;

c) judging the direction change after analyzing the change of acceleration during the 2nd

sampling time which is longer than the 1st sampling time;

d) calculating the position change by integrating the acceleration during

the 2nd sampling time.

[Claim 12]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein the method for detecting

the position ofthe ball is to use the infrared sensors.

[Claim 13]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein the method of detecting

the position of said housing is to use said 1st camera that is set up in horizontal direction

and said 2nd camera that is set up in vertical direction.

[Claim 14]

The device for making the 3D movement of Claim 1 comprising:

The device for making the 3D movement , wherein the method of detecting

the position ofthe ball is to use the lens for reducing the distortion ofthe picture data.