CN101248410A - Device and method for recognizing movement mode - Google Patents
Device and method for recognizing movement mode Download PDFInfo
- Publication number
- CN101248410A CN101248410A CNA2006800307850A CN200680030785A CN101248410A CN 101248410 A CN101248410 A CN 101248410A CN A2006800307850 A CNA2006800307850 A CN A2006800307850A CN 200680030785 A CN200680030785 A CN 200680030785A CN 101248410 A CN101248410 A CN 101248410A
- Authority
- CN
- China
- Prior art keywords
- motion vector
- equipment
- motor pattern
- pattern
- predefine
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Studio Devices (AREA)
- Processing Or Creating Images (AREA)
Abstract
A device for converting digital images taken along a predefined movement pattern into a control signal. The device comprises an image input device having a connection to a image sensor. The image input device is adapted to receive the digital images captured during the displacement of the image sensor and determine whether motion measured therein corresponds to any prestored movement pattern, that is to say to a predefined gesture. The device further comprises a movement pattern converter to convert the recognized pattern into the control signal.
Description
Technical field
The present invention relates to be used for motor pattern is converted to the equipment and the method for control signal, and more specifically but exclusively non-, relate to and be used to utilize Flame Image Process motor pattern to be converted to the equipment and the method for control signal.
Background technology
The rapid microminiaturization of complex electronic circuit and the appearance of high-resolution display device have greatly increased quantity and the diversity based on the equipment of portable processor.The equipment based on portable processor like this comprises handheld computer, mobile phone, pager and other portable communications and calculates solution.In addition, processing power, data storage capacities, communication speed and the battery life based on the equipment of portable processor just continues development with the step of quickening.
Aforesaid each based in the equipment of portable processor is integrated with usually and allows the user to control the man-machine interface (MMI) of its running.Yet, MMI need be adjusted into the small size of portable autonomous device.The traditional MMI based in the equipment of portable processor like this is microcomputer keyboard or keypad, and it allows the user to import the data that have such as the text representation of telephone number, contact name, word processor content or the like.
The another kind of MMI that typically is used at the PDA(Personal Digital Assistant) input text data is a touch-screen.Such equipment is integrated with usually and usually leaves near the PDA or the form of a stroke or a combination of strokes pointing device on it.In use, form of a stroke or a combination of strokes pointing device is acted on touch-screen on the PDA so that the user can elect and be mutual with PDA equipment.Mobile phone and based on using the High Resolution LCD touch-screen in the equipment of portable processor.In order to determine the touched position of display screen, generally use condenser type and resistance-type touching sensitive layer.They are based on the transparent conductor such as indium-tin-oxide film.Yet the deficiency of using such MMI is their fancy prices and limited transparency (it has reduced picture quality), particularly to modern High Resolution LCD display screen.
Aforesaid MMI allow the user by such as the brief symbol of input such as character press emulation or true keyboard on the icon of expectation or the difference in functionality that letter controls relevant device.Because each MMI is adjusted to the size of microminiaturized equipment, the user may have to concentrate to find relevant P cock or with the form of a stroke or a combination of strokes pointing device touch off character of expectation down.Further, because the restriction of physics, MMI is configured so that once to move or button is pressed and just born results or incident limited in one's ability.
The rapid microminiaturization of complex electronic circuit has caused imageing sensor is integrated in the equipment based on portable processor recently.PDA, mobile phone and laptop computer are integrated with the camera that is used to catch static and video image.One of reason that such product is popular is such as complementary metal oxide semiconductor (CMOS) (CMOS) or the imageing sensor price of charge-coupled image sensor (CDD) and the attenuating of size.
Imageing sensor integrated opened up allow the user with based on the mutual other possibility of the equipment of portable processor.The mobile phone of known integrated camera for example uses the moving interface engine, and it is by for example making the control judicial convenience of equipment with control such as the function of scroll through menus to equipment input direction or movement instruction via imageing sensor.
Yet these use the demonstration only can be used for travelling or to be used to control such as the target pointer of cursor between the different menu of equipment.Direction instruction can not be used as input and use the character that keypad or keyboard import or the basis of other predefine symbol usually.In addition, can't import the input formed by the direction instruction sequence as single instruction.
Therefore, extensively recognize the equipment and the method that need completely without the MMI of above-mentioned restriction, and it will have superiority very.
Summary of the invention
According to an aspect of the present invention, provide a kind of the having of the complicated displacement of predefine of being used to discern to the equipment of the connection of imageing sensor.Described equipment comprises the motion vector detection module, and it is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor.Described motion vector detection module is arranged to the current motion vector that produces described imageing sensor according to described digital picture.Described equipment further comprises mixing module, and it is configured to receive current motion vector.Described mixing module is configured to the continuous motion sequence vector during current motion vector and the described displacement is mixed.Described equipment further comprises: the motor pattern knowledge base, and it is adapted to be the memory module sample record, a kind of predefine motor pattern of each record representative; And the motor pattern matching module, it is adapted to be between one of described pattern sample record and described continuous motion sequence vector and mates.
Preferably, described predefine motor pattern is the nonlinear motion pattern.
Preferably, described motor pattern matching module is configured to determine according to described coupling whether described digital picture is to take along each predefine motor pattern approx.
Preferably, described mixing module is adapted for and adds new record to described pattern sample record, produces described new record according to described digital picture.
Preferably, described equipment further comprises user interface, and described user interface is configured to allow the user to start and stop catching of described digital picture.
Preferably, described equipment further comprises output device, and it is adapted to be the output control signal.
More preferably, the following group of described control signal representative at least one of them: be used for the text input character, start described equipment at least a function, have the security signature of plane mode, at least a function that has the security signature of spatial model and stop described equipment.
More preferably, each in the described pattern sample record comprises the motion vector array.
More preferably, described mixing module is adapted to be described continuous motion sequence vector is converted to normalization continuous motion sequence vector, and each normalization motion vector of described normalization sequence and each vector of described motion vector array have unified length.
Preferably, described equipment be following group one of them: mobile phone, PDA(Personal Digital Assistant) and laptop computer.
Preferably, described imageing sensor be following group one of them: complementary metal oxide semiconductor (CMOS) (CMOS) sensor and charge-coupled device (CCD) sensor.
Preferably, described digital picture is by following group of one of them expression: colourity-saturation degree-purity (HSV) color coordinate, CIE L*a*b* (CIELAB) color coordinate, YCbCr color coordinate and R-G-B (RGB) color coordinate.
Preferably, described connection is configured to allow user and described equipment to control described imageing sensor motion dividually.
More preferably, described connection comprise following connection at least one of them: RS-232 connects, Ethernet connects, USB (universal serial bus) (USB) connects, cellular transceiver connects, 1394 live wires connect, USB2 connects, bluetooth connects and IR connects.
Preferably, described imageing sensor firmly is couple to described equipment.
More preferably, described equipment further comprises communication interface modules, and described communication interface modules is adapted to be to remote equipment and sends described control signal.
According to another aspect of the present invention, provide a kind of equipment that is used for shot digital images during the displacement of imageing sensor is converted to control signal.Described equipment comprises the image input device that has to the connection of imageing sensor.Described image input device is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor.Described equipment further comprises: the knowledge base of predefine motor pattern; And the motor pattern converter, be associated with described knowledge base and described image input device, and be adapted to be according to the coupling between one of described displacement and described predefine motor pattern described digital picture is converted to described control signal.
According to another aspect of the present invention, provide a kind of method that is used to discern the displacement pattern.Said method comprising the steps of: a) be received in the digital picture of catching during the displacement of equipment of integrated image sensor; B) according to described digital picture identification exercise data sequence; And c) the predetermined motor pattern with described exercise data sequence and storage mates.
Preferably, described method further comprises the step d) that produces control signal according to described coupling.
Preferably, by described exercise data sequence and pattern sample record are mated to come execution in step c) coupling, each record in the described pattern sample record is related with different predefine motor patterns.
Preferably, described predefine motor pattern be following group one of them: the predefine profile of character, symbol, gesture, the security signature that has the security signature of plane mode and have spatial model.
Preferably, described exercise data comprises the information about the motion during the continuous time fragment of the equipment of described integrated image sensor.
Preferably, described exercise data sequence is the motion vector array.
More preferably, each record in the described pattern sample comprises the motion vector array, wherein during described coupling the corresponding motion vector in each motion vector in the described exercise data sequence and the described pattern sample record is mated.
More preferably, during described coupling according to be stored in mate the motion vector in the predefine motor pattern quantity with the motion vector segmentation in the described exercise data sequence.
More preferably, described method further is included between step a) and the step b) by in the described motion vector array each is come the step of the described exercise data sequence of normalization divided by its length.
More preferably, described method further comprises the step that described motion vector sequence is divided into the equal length fragment of predetermined quantity, and each in the described fragment is represented the directional data of the corresponding digital image in the described sequence.
Preferably, described method further is included between step a) and the step b) step that the color coordinate of the pixel of described digital picture is converted to the greyscale color coordinate.
More preferably, described method comprises that further the application program to described equipment sends the step of described control signal.
More preferably, described application program is on described equipment.
More preferably, described application program is on remote equipment.
Preferably, described method further is included in the step of storing at least a predefine motor pattern before the step a).
More preferably, described storage comprises the step of the digital picture that the equipment that is received in described integrated image sensor is caught during one of them the displacement of described at least a predefine motor pattern approx.
More preferably, described storage comprises the step that receives described at least a predefine motor pattern from external source.
More preferably, described external source is a personal computer.
According to another aspect of the present invention, provide a kind of mobile phone that is used to discern the complicated displacement of predefine with imageing sensor.Described mobile phone comprises: the motion vector detection module, it is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor, and described motion vector detection module is arranged to the current motion vector that produces described imageing sensor according to described digital picture; And the motor pattern knowledge base, be used to store a plurality of user definition motor patterns.Described mobile phone further comprises the motor pattern matching module, and it is adapted to be between one of described a plurality of user definition motor pattern records and continuous motion sequence vector and mates.
Preferably, by the described motor pattern of the user definition of described mobile phone.
According to another aspect of the present invention, provide a kind of method that is used to carry out the function of mobile hand-held device.Described method comprises: a) be received in described mobile hand-held device and continue to use the digital picture of catching during the displacement of family definition motor pattern approx; B) discern described user definition motor pattern according to described digital picture; And c) carries out function according to described user definition motor pattern.
Preferably, described method further is included between step b) and the step c) step that the predefine motor pattern with described user definition motor pattern and storage mates, and wherein comes completing steps c according to described coupling) execution.
Unless write exactly in addition, whole scientific and technical terms used herein all have the general identical meaning of understanding with one skilled in the art of the present invention.Here the data that provides, method and example only are indicative, are not intended to be used for restriction.
The realization of method and system of the present invention relates to manually, automatically or with its combination carries out or finish some selected task or step.In addition, the actual equipment and the device of the preferred embodiment of method and apparatus according to the invention can be realized several selected steps by hardware or by the software on any operating system of any firmware or its combination.For example, as hardware, selected step of the present invention can be embodied as chip or circuit.As software, selected step of the present invention can be embodied as a plurality of software instructions that utilize any appropriate operating system to carry out by computing machine.Under any circumstance, the selected step of method and apparatus of the present invention can be described as by carrying out such as the data processor of the computing platform that is used to carry out a plurality of instructions.
Description of drawings
Here describe the present invention with reference to the accompanying drawings only as illustration.Now at length specifically with reference to accompanying drawing, it is emphasized that, the details that illustrates is only as illustration and only be used for the purpose of the explanatory argumentation of the preferred embodiments of the present invention, to provide these details and be in order to provide to believe it is the most helpful and understandable description to principle of the present invention and notion.For this reason, do not attempt to show the more detailed CONSTRUCTED SPECIFICATION of the details more required than basic comprehension of the present invention, it is can several forms of the present invention specific in practice that description taken together with the accompanying drawings makes that how those skilled in the art understand.
In the accompanying drawing:
Fig. 1 be according to a preferred embodiment of the invention be used for will be converted to the synoptic diagram of the equipment of control signal along path shot digital images sequence;
Fig. 2 is being combined and representing with the exemplary figure of one group of continuous motion vector forming motor pattern according to a preferred embodiment of the invention;
Fig. 3 is the simplified flow chart that is used for digital image sequence is converted to four step processes of control signal according to a preferred embodiment of the invention;
Fig. 4 A is the diagrammatic representation of two continuous number images of being caught by conversion equipment according to a preferred embodiment of the invention;
Fig. 4 B is first the diagrammatic representation in two digital pictures of Fig. 4 A;
Fig. 4 C is second diagrammatic representation in two digital pictures of Fig. 4 A;
Fig. 4 D is the exemplary schematic representation of the deviation between the locations of pixels of matching area at the first digital picture place of Fig. 4 A and the second digital picture place;
Fig. 4 E is the process flow diagram that the generation of motion vector sequence according to a preferred embodiment of the invention is shown;
On behalf of the exemplary figure of the motion vector sequence of certain motor pattern, Fig. 5 A represent;
Fig. 5 B represents according to the exemplary figure of the normalization motion vector sequence of the motion vector sequence generation of Fig. 5 A;
Fig. 5 C is the diagrammatic representation with exemplary character of the common pattern of the normalization motion vector sequence of the motion vector sequence of Fig. 5 A and Fig. 5 B;
Fig. 6 A is the simplified flow chart of process that is used to produce normalization motion vector sequence according to a preferred embodiment of the invention;
Fig. 6 B is another simplified flow chart of process that is used to produce normalization motion vector sequence according to a preferred embodiment of the invention;
Fig. 7 is that the exemplary figure of one group of exemplary normalization motion vector and one group of corresponding motion vector is represented;
Fig. 8 A is according to the normalization motion vector sequence that produces as the motor pattern shot digital images sequence of numeral " 3 " along shape and the diagrammatic representation that is configured to respectively a pair of pattern sample record of mating as the motor pattern of " 3 " and " 6 " with shape;
Fig. 8 B describes two tables of the similarity grade point that all comprises the different mode sample record relevant with the motor pattern shown in Fig. 8 A separately; And
Fig. 9 describe according to a preferred embodiment of the invention mobile phone with and during motor pattern being recorded in pattern sample knowledge base (repository) and some demonstrations during utilizing the motor pattern that has write down.
Embodiment
Current embodiment comprises equipment and the method that is used for when identifying predetermined motion path digital picture being converted to control signal corresponding.
By reference accompanying drawing and appended explanation, can understand principle and operation better according to apparatus and method of the present invention.
Before at least one embodiment of the present invention is explained in detail, should be appreciated that to the invention is not restricted to be applied in the following description and set forth or the details of structure illustrated in the accompanying drawings and the layout of assembly.The present invention can have other embodiment or put into practice in many ways or finish.And, should be appreciated that the word and the term that adopt are to be used for purpose of description here, should not be taken as is restriction.
One embodiment of the present of invention are a kind of equipment that is used for current digital image sequence is converted to control signal.This equipment comprises the image inputting and transferring equipment that has to the connection of imageing sensor.This image inputting and transferring equipment is adapted to be the digital image sequence of catching during the displacement that is received in imageing sensor to discern any motor pattern.This conversion equipment further comprises the motor pattern converter, and it is adapted to be the motor pattern that will be discerned and is converted to control signal.
An alternative embodiment of the invention is a kind of method that is used for motor pattern is converted to control signal.This method comprises several steps.The first step, capture digital image sequence during the displacement of the equipment of integrated image sensor.This equipment can be approx according to posture or random motion and displacement.The digital image sequence that analysis was caught between the moving period of the equipment of integrated image sensor is to discern completed motor pattern.One of posture record of the predefine motor pattern discerned and a plurality of storages is in advance mated, and wherein each record comprises the sample of different predefine motor patterns.After finding coupling, produce control signal corresponding.
Conversion equipment can be understood as integrated or is connected to any equipment based on processor of imageing sensor, such as PDA, mobile phone, radio hand-held equipment or laptop computer.
Control signal can be understood as stimulation, input or the symbol that starts certain action, such as the signal or the interference that are input to such as the functional unit of telecommunication apparatus, processor or computer program.
Motor pattern or posture can be understood as object and move along any of track who follows approx or accurately such as the profile of certain shape of the shape on character shape, symbol shape, lines shape, spatial form or the object.Motor pattern also can be understood as object and moves or the typical motion of limbs during carrying out known activity along any of track of known gesture.
Referring now to Fig. 1, it describes demonstration equipment 1 according to an embodiment of the invention, and it is used for changing along with cam movement the shot digital images sequence, to determine whether camera moves along predefine motor pattern or posture.The image input module 4 that conversion equipment 1 is adapted to be via appointment receives digital image sequence 2 from imageing sensor 3.Preferably, imageing sensor 3 is cmos sensor or ccd sensor.Preferably, imageing sensor 3 is couple to conversion equipment.In such embodiments, in conversion equipment shooting digital pictures sequence during the displacement of predefine motor pattern.The mobile phone of integrated camera is the example of such embodiment.
In another embodiment of the present invention, via wireless connections or prolongation cable the image inputting and transferring equipment is connected to imageing sensor.In such embodiments, imageing sensor during the candidate displacement of predefine motor pattern with conversion equipment shooting digital pictures sequence discretely.The class example of such embodiment is mobile phone or the PDA with the independent unit communication that comprises the imageing sensor that is couple to bluetooth transmitter.In such embodiments, via adjusted wireless connections digital image sequence is sent to conversion equipment.Preferably, this independent unit has the form of a stroke or a combination of strokes or annular shape so that by the user with its displacement.
Preferably, this conversion equipment comprises user interface, and it is configured to allow the user to start and stop catching of digital image sequence.For example, in mobile phone, can be by catching by pressing the specified button on the keypad or starting by the simple command group that the designated button on the shell of pressing mobile phone is sent.In order to start and to stop catching of digital image sequence, the user can pin button, and perhaps selectively, he can press the button once to catch with startup and press the button once more with termination and catch.That is to say the beginning of posture defined by the user and end.
Referring now to Fig. 2, it is to be combined diagrammatic representation with one group of motion vector 100 in succession forming motor pattern 101.Each motion vector 100 of motor pattern 101 is represented the motion of conversion equipment during fragment sometime.
As mentioned above, motion vector maker 7 and motor pattern converter 5 are used for digital image sequence 2 is converted to control signal 6.Preferably, conversion equipment 1 is configured to a plurality of different predefine motor patterns are converted to each different control signals.In such embodiments, the user controls conversion equipment 1 along the track 102 of following the profile of predefine motor pattern approx and moves, with the information that sends about the control signal sequence of a certain association to motor pattern converter 5.Preferably, in order to discern predefine motor pattern and it is related with certain control signal, motion vector maker 7 utilizes Flame Image Process.
Referring now to Fig. 3, it is the process flow diagram that is used for digital image sequence is converted to four step processes of control signal that illustrates according to a preferred embodiment of the invention.In first step 201, the motor pattern converter receives digital image sequence.In one embodiment of the invention, each digital picture comprises a plurality of colour elements, and each pixel comprises colouring information.Each color of pixel information is used to Flame Image Process.
In step 202, produce the motion vector sequence by the motor pattern converter.As mentioned above, this digital image sequence is taken along the predefine motor pattern.Take along which kind of predefine motor pattern in order to discern this digital image sequence, need to calculate individually the motion vector of forming the predefine motor pattern.Preferably, come each motion vector in the calculation of motion vectors sequence by the motion vector maker based on the difference between two digital pictures in succession.
Referring now to Fig. 4 A, 4B, 4C and 4D, it describes the zone of partly catching, and this zone is described by two digital pictures 300,301 in succession.Fig. 4 A describes two digital pictures in succession being used to calculate one of motion vector that conversion equipment determines that motor pattern is used, as mentioned above.Motion vector based on the estimation of deviation conversion equipment 302 between the position of first digital picture 300 and second digital picture 301.Fig. 4 A describes first digital picture 300 and second digital picture 301 of being caught by the conversion equipment with the direction of motion shown in numeral 303.Because described digital picture belongs to same digital image sequence, they have identical size and can utilize same coordinate system reference.Preferably, use initial point to be positioned at the coordinate system in the lower left corner of frame.
For the deviation between the position of calculating digital picture 300 and 301, preferably in two images, all discern the reference area of describing similar pattern or zone.Fig. 4 B describes first digital picture 300 and corresponding reference area 305.Fig. 4 C describes second digital picture 301 and corresponding reference area 306. Reference area 305 and 306 is described the same part of the image of catching in image 300 and 301.
Preferably, in order to discern the zone of second digital picture 301 of mating, use block matching algorithm with the zone of first digital picture 300.In one embodiment of the invention, use exemplary absolute difference summation (SAD) algorithm to select matching area in the border of digital picture 301.Similarity in certain zone of using SAD to handle to quantize first digital picture 300 and second digital picture 301 between the different matching area candidate.Utilize the result who quantizes to determine matching area.Also can use other algorithm of having developed the computation complexity that is used to reduce the piece matching treatment.For example, can use absolute difference partial summation (PSAD), absolute difference average (MAD), three step search (TSS), improvement motion estimation algorithm (MMEA) or independent orthogonal searching algorithm (IOSA) to select the matching area of second digital picture.SAD, PSAD, MAD, TSS, MMEA and IOSA algorithm are known, thereby not for a more detailed description here.
Fig. 4 D describes the deviation between the position of corresponding pixel 308 in the matching area of certain the regional pixel 307 of first digital picture and second digital picture.This deviation reflects the motion vector that its direction is opposite with the direction of motion of conversion equipment.Preferably, this deviation chart is shown one group of two numerical value (that is, Mx and My), the change in that it represents certain pixel 307 in first digital picture and second digital picture between the coordinate of corresponding pixel 308.
Should be noted that it only is exemplary processes that the motion vector of describing among Fig. 4 detects processing.Also can use many other motion vectors that comprise non-SAD method to detect and handle and calculate, to produce motion vector according to two digital pictures in succession.
Preferably, for each consecutive image in the digital image sequence to estimating motion vector.Thereby, be used to estimate two motion vectors in succession except that first and each digital picture last in the sequence.Except that first with last digital picture, other each digital picture is used to as second digital picture of seeking matching area therein, and then as using matching area therein as first digital picture of the benchmark of piece matching treatment, as mentioned above.Preferably, calculating and storage comprise the array of the motion vectors of all having estimated as motor pattern.Preferably, each record in this array is one group of two numerical value (that is, Mx and My), and it represents the deviation between the pixel of matching area of two digital pictures in succession.
Referring now to Fig. 4 E, it is the process flow diagram that the generation of motion vector sequence according to a preferred embodiment of the invention is shown.In first step that produces the motion vector sequence, shown in 351, quality threshold is set.The quality of the estimation that this quality threshold reflection is such, certain motion vector that will be higher than it is defined as the effective exercise vector.This ability helps the integrated many algorithm known that are used for the validity of the estimating motion vector value motion vector value of before motion vector (preferably about).Service property (quality) threshold value in the step after a while of this processing is shown in 356.
In step subsequently, shown in 352, initialization motion vector array.With the motion vector array configurations is to store the not motion vector of determined number.Preferably, dynamically define array in the mode that guarantees dirigibility.Then, this processing enters loitering phase, does not write down motion vector therein till receiving enabling signal, shown in 353.Preferably, the user need press the predefine button to produce such enabling signal.After receiving enabling signal, shown in 360 and 361, retrieval present frame and next frame of catching are to allow calculation of motion vectors.Preferably use present frame and next frame of catching as calculation of motion vectors as described in according to Fig. 4 A-4D.As mentioned above, because this process iterates, be per two frame calculation of motion vectors in succession preferably, shown in 354 by the motion vector maker.
355, use the quality threshold that in step before, has calculated now.If the quality grade of the motion vector that calculates is lower than quality threshold, the motion vector indication conversion equipment that perhaps calculates does not have displacement, shown in 356, then calculates another motion vector.Yet if the quality grade of the motion vector that calculates is higher than quality threshold, and the motion vector that calculates indication conversion equipment has displacement, adds the motion vector that calculates to the motion vector array, shown in 357.In step 358 subsequently, survey whether receive termination signal.In one embodiment of the invention, the user can produce termination signal by pressing designated button.If the termination signal of not receiving is then calculated another motion vector.
As mentioned above, preferably calculate the additional motion vector according to back one and the difference that is used as between the additional frame of after a while frame in two present frames.Thereby shown in 361, if do not detect the reception of termination signal, back one that then delivers in two present frames is used for calculating next motion vector with next frame of catching.Shown in Fig. 4 E was further, obtaining of motion vector sequence was the cyclic process of calculating with repetitive mode.
Described in Fig. 4 E, the time durations receiving between enabling signal and the termination signal adds motion vector to the motion vector array in the round-robin mode.Because the user preferably has the ability that produces these signals, as mentioned above, so during the user can define between the some time that is used to write down certain motor pattern.This process finishes when generation comprises the motion vector array of continuous motion sequence vector, shown in 359.Certain motor pattern that the sequence representative that is produced is carried out during the displacement of conversion equipment, as mentioned above.
Referring again to Fig. 2, it describes the motion vector sequence 100 based on the real-time digital image of taking along desired trajectory 102 now.Every line 104 representatives between two points 103 in succession and conversion equipment at a time between the motion vector that calculates of motion association during the fragment.Fig. 2 describes the preferred embodiments of the present invention, wherein along the predefine track taking moving pattern 101 with character ' S ' shape.
Now referring again to Fig. 3.As mentioned above, produce the motion vector sequence according to digital image sequence.Next step shown in 203, is that one of motion vector sequence and a plurality of predefine motor patterns are mated.According to producing the motion vector sequence along certain motor pattern shot digital images sequence.The user can utilize conversion equipment to import multiple motor pattern.Every kind of motor pattern is used to produce different control signals.In one embodiment of the invention, the motor pattern converter comprises pattern sample knowledge base, and it comprises the pattern sample record, and each pattern sample record is related with control signal corresponding.Preferably, the quantity of pattern sample record is variable and can be adjusted by the user.Each pattern sample record preferably includes the array of the individual motion vector of predefine quantity (preferably, 24).Preferably, if obtain the spatial movement sequence vector, as mentioned above, then each pattern sample record comprises the spatial movement sequence vector.Motion vector preferably has unified length, and it is the unit of can be used as preferably.Thereby motion vector only reflects the direction change along relevant motor pattern.Arrange the motion vector of each array continuously according to certain predefine motor pattern.Such motor pattern can shape such as common character, common mark, common symbol or such as the individually mark of configuration of signature, and is as described below.
Referring now to Fig. 5 A, 5B and 5C, it describes the motion vector sequence of determining certain motor pattern 101, normalization motion vector sequence 500 and the exemplary character 501 with unified length respectively.The same among the diagrammatic representation of the motion vector sequence of Fig. 5 A and top Fig. 2.
As mentioned above, in one embodiment of the invention, the motor pattern converter comprises pattern sample knowledge base.Such embodiment allows the motor pattern converter that motion vector sequence and a plurality of pattern sample record obtained are mated, and wherein each pattern sample record comprises different predefine motor patterns.Yet different with the pattern sample record that comprises the predefine motion vector sequence with unified length, the motion vector sequence of acquisition comprises the not motion vector with different length of determined number, as shown in Figure 2.In one embodiment of the invention, before the record with sequence and pattern sample knowledge base compares, collect the motion vector of the sequence of obtaining and the normalized vector that its operation is preferably had unified length with formation.
Referring now to Fig. 6 A, its be illustrate according to one exemplary embodiment of the present invention be used to produce normalization motion vector sequence five the step processes process flow diagram.As following further as described in, represent the record of motion vector sequence of some motor pattern to mate motion vector sequence and storage.These records have the motion vector that comprises certain motor pattern of predefine quantity.For the record with motion vector sequence and storage mates, need according to the motion vector that is stored in the predefine quantity in these records the segmentation of motion vector sequence.Such segmentation allows matching treatment, as described in greater detail.
At first step 601, receive the motion vector sequence of describing motor pattern as mentioned above.Preferably, the sequence of obtaining is the array with a plurality of records, and each record comprises the motion vector with length-specific.
In step 602 subsequently, estimate the length of motor pattern.The length of the motion vector by sequence that composition is obtained is sued for peace and is finished this estimation.By estimating the length of motor pattern, can determine the length of the motion vector that length is unified, shown in 603.Preferably with the total length of the sequence obtained predefine numerical value divided by the quantity of the required motion vector of representative.As mentioned above, each record of pattern sample comprises the array of the motion vector with predefine quantity, for example as directed 24.In following step, shown in 604, before itself and each pattern sample are mated, the motion vector sequence that obtains is divided into after a while by the unified fragment of the length of normalized motion vector.The quantity of the motion vector that length is unified is suitable with the quantity that is stored in the motion vector in the pattern sample.During handling, the sequence of obtaining is divided into the unified motion vector of length of similar quantity.Preferably, the fragment that is divided into the predefine quantity that is expressed as motion vector by the length with motor pattern is calculated the length of the unified motion vector of each length, as mentioned above.The length that storage computation obtains is to allow that being segmented into of motion vector sequence had the group of equal length, as mentioned above.
As mentioned above, in step 604, after the total length of the sequence motion vector of estimating to obtain, the motion vector sequence of obtaining can be divided into the fragment of continuous normalization motion vector according to the total length of sequence.
Referring now to Fig. 6 B, it is the process flow diagram that illustrates in greater detail according to the normalization motion vector production process described in Fig. 6 A of one exemplary embodiment of the present invention.Among Fig. 6 B, j represents the sequence number (consecutive order) of current normalization motion vector, i represents the sequence number of current motion vector, E represents the quantity of current motion vector, L represents the total length of current motion vector, the quantity that M represents the normalization motion vector (for example, 24), N represents the normalization motion vector, V represents current motion vector, A represents the unified length of the unified motion vector of length, and C representative forms the current length of the motion vector fragment of the unified motion vector of current length, and poor between the length of the unified length (A) of F representative and the normalization motion vector (N (j)) of working as pre-treatment.
Shown in 651, initial reception preferably is stored in the motion vector sequence of the description motor pattern in the array (V (i)).Preferably, the sequence of obtaining is the array with a plurality of records, and each record comprises the motion vector with length-specific.After receiving the motion vector sequence, estimate the length of motor pattern.As described in 657, the length of the motion vector by sequence that composition is obtained is sued for peace and is finished length and estimate as step 652.To shown in 656, finish summation as step 654, in the single variable (L) of wherein length of total movement vector being sued for peace together with the process that iterates.Shown in 657, after the length of estimating motor pattern, with the total length (L) of the sequence obtained predefine numerical value (M) divided by the required number of motion vectors of representative.The result of this derivation (A) be according to its as step 662 as described in 669 with the segmentation of motion vector sequence to form the unified length of normalization motion vector.Now, to shown in 659, be reset at the different variable that uses in the normalization motion vector production process as step 658.Then, to shown in 671, during handling, son produces the normalization motion vector as step 660 according to the unified length of in step 657, determining (A).This son is handled and iterated, and allows to produce continuous normalization motion vector, and is as described below.During each iteration that this son is handled, the initialization representative is when the variable of the motion vector of pre-treatment.Preferably, initialization is as the normalized vector N (j) of pre-treatment and the variable C that represents its current length, shown in 661 and 660 difference.Then, to shown in 665, add a plurality of continuous motion vectors as step 662 to form normalization motion vector N (j) when pre-treatment.Add continuous motion vector constantly, as long as their length sum is no more than unified length A, shown in 662.Thereby if connection continuous motion vector subsequently, then the fragment when pre-treatment is expanded the unified length (A) that surpasses estimation.In order to be no more than the unified length (A) of estimation, utilize the part of continuous motion vector FV (i) to supply the length of the continuous motion vector that has connected of forming this fragment and the breach (A-C) between the normalization length, shown in step 666 and 667.Shown in 668, the remainder of subsequently continuous motion vector is added to first vectorial N (j+1) of composition normalized vector subsequently, it is processed during iterative process (j+1) subsequently.Preferably, shown in 669, after the length or its part of adding last motion vector, (N (j) is normalized to uniform units size (N (j)=N (j)/Length (N (j))), as described further below with new fragment.Shown in 671, step 660 is handled to of the iteration shown in 671 and is continued till last motion vector being distributed to last fragment.Thereby, shown in 672, during normalization motion vector production process, produce the normalized vector sequence, be preferably array.Different with the motion vector of the motion vector sequence of obtaining, the normalization motion vector of normalization motion vector sequence has unified length.After normalization motion vector production process has finished, can directly there be potential possible pattern sample record to compare normalization motion vector sequence with a plurality of.Fig. 5 B describes the diagrammatic representation of such normalization motion vector sequence.
During normalization motion vector production process, with each motion vector by coming normalization divided by motion vector length (N (j)=N (j)/Length (N (j))).Particularly, preferably according to the definition of following equation normalized this part:
Wherein (x ', y ') represent the normalization motion vector, and (x, y) representative unified motion vector of length before it is by normalization.After motion vector was as mentioned above by normalization, the difference between the motion vector of they and pattern sample record preferably only was reflected as angular deviation.In this example as described in following and Fig. 4 E vector is normalized to unit length (1).
Referring now to Fig. 8 A and 8B, it describes the figure and the numeric representation of three normalization motion vector sequences according to a preferred embodiment of the invention respectively.As mentioned above, this motion vector sequence can directly have potential possible pattern sample record directly to compare with a plurality of normalization motion vector sequence by normalization.
As mentioned above, each pattern sample record comprises the normalization motion vector of predefine quantity.
Preferably, for assess certain pattern sample record whether with normalization motion vector sequences match, calculate one group of relevance degree.Each relevance degree representative reflection normalization motion vector in this group (x ', y ') and pattern sample record (Px, the similarity grade point of the similarity between corresponding motion vector Py) assessed.Because preferably in an identical manner with ambilateral motion vector normalization, the difference between the motion vector only reflects the angular deviation between them.Preferably calculate similarity grade point (SLV) as relevance degree.Preferably according to following equation definition SLV:
SLV＝(x′·Px)+(y′·Py)
(x ', y ') and pattern sample record because motion vector (Px, Py) all by normalization, the similarity grade point is between-X and X, and wherein X represents the highest similarity grade, and-X represents minimum similarity grade.X preferably equals 1.Fig. 7 is the diagrammatic representation of one group of exemplary normalization motion vector 800 and corresponding motion vector 801.As described, similarity grade point 802 is corresponding with the angular deviation between the corresponding separately motion vector 801 of each normalization motion vector 800 and its.
As mentioned above, the specific pattern motion of motion vector sequence definition.Arrange motion vector to be reflected in conversion equipment sequential order of shot digital images sequence during the displacement of certain track.Any motor pattern track is along one or more line, shown in the reference numerals 102 of Fig. 2.Thereby, in order to produce the digital image sequence stream of following certain motor pattern, can come the described conversion equipment of displacement along the track of this motor pattern.
Preferably, will define the similarity grade point summation of the total movement vector of certain pattern sample record, to provide the matching score of the similarity grade that reflects the pattern sample record relevant with normalization motion vector sequence.
Fig. 8 A describes the normalization motion vector sequence that produces as the motor pattern 702 shot digital images sequences of " 3 " according to along shape.This figure further describes two example mode sample record that are configured to normalization motion vector sequences match.An example mode sample record is arranged to the motor pattern 701 of shape as " 3 ", and another example mode sample record is arranged to the motor pattern 700 of shape as " 6 ".
Fig. 8 B describe all comprise separately relevant different pattern sample with normalization motion vector motor pattern 702 two tables 703 and 704 of similarity grade point.The table 704 on the left side comprises that configuration is used for the similarity grade point of shape as the pattern sample record of the motor pattern 700 of " 6 ", and the table 703 on the right comprises that configuration is used for the similarity grade point of shape as the pattern sample record of the motor pattern 701 of " 3 ".Obviously because this normalization motion vector sequence represent the motor pattern of shape as " 3 ", its vector with dispose the vectorial similar of the pattern sample record that is used for motor pattern 701.The matching score that disposes the pattern sample record 703 that is used for motor pattern fully is higher than the matching score of another pattern sample record 704 that is used for motor pattern.
In one embodiment of the invention, calculate whole matching score of all mode sample record.Based on the calculating of matching score, the pattern sample record of selecting to have the highest matching score is as the coupling to this normalization motion vector sequence.In another embodiment, predefine matching threshold.If the pattern sample record has the matching score that is higher than matching threshold, then select this pattern sample record, and no longer calculate the matching score of other pattern sample record.
Also matching threshold can be used for as the means of avoiding erroneous matching.As mentioned above, select to have the pattern sample record of the highest matching score as coupling to this normalization motion vector sequence.By realizing such embodiment, may only, it select random motion pattern undefined or that define approx with any pattern sample as coupling to described normalization motion vector sequence because having the highest matching score.Thereby, for fear of such erroneous matching, can use the matching threshold of determining minimum matching score.Preferably, the error message that motor pattern converter output indication has made a mistake, and preferably export the matching score or the type of makeing mistakes.
As mentioned above, the motor pattern converter comprises pattern sample knowledge base.Each pattern sample record is configured to and corresponding motor pattern coupling as mentioned above.Each pattern sample record is associated with control signal.In one embodiment of the invention, pattern sample knowledge base comprise be configured to keyboard commonly used on the pattern sample record of motor pattern coupling of available alphabet.In such embodiments, can use the control signal of the control signal of pattern sample as the keypad or the keyboard of conversion equipment.Conversion equipment produces the notice of incident along the displacement of motor pattern track, and it is sent to current active program on the storer of conversion equipment.
When computer memory motion vector sequence, as mentioned above, can the package space motor pattern to produce control signal.In such embodiments, each pattern sample record is configured to as mentioned above and the spatial movement pattern match.
In one embodiment of the invention, conversion equipment further comprises communication interface modules, and it helps sending described control signal to named place of destination.For example, can use the honeycomb transmitter to transmit control signal to remote equipment.Also can use other transmitter to transmit control signal such as radio frequency (RF) transmitter.For example, can use Wi-Fi or other transmitter to transmit control signal based on the wireless lan (wlan) standard of IEEE 802.11 standards.Can use the bluetooth standard that is used for the short distance digital transmission communication protocol as RF communication.Also conversion equipment can be wiredly connected to another remote equipment.This wired connection preferably provides wired serial communication.Serial communication can comprise that RS-232 connects, Ethernet connects, USB (universal serial bus) (USB) connects, cellular transceiver connects, live wire connects, USB2 connects, bluetooth connects or IR connects.Preferably, can use USB or USB2 to connect, provide electric current to conversion equipment as power supply.
Yet owing to select control signal according to the coupling between normalization motion vector sequence and the predefined pattern sample, conversion equipment can't be discerned the motor pattern that can not be represented by one of pattern sample record.
Referring now to Fig. 9, its describe mobile phone with and the synoptic diagram of some demonstrations during mode of learning, it relates to record in the pattern sample knowledge base motor pattern and the processing that utilizes the motor pattern that has write down.In one embodiment of the invention, the motor pattern converter comprises motor pattern unit.This motor pattern unit is configured to allow the record of user's configuration mode sample knowledge base during mode of learning.This mode of learning is similar with the gesture recognition process in many aspects.The pattern sample record be configured to along individually the configuration motor pattern shot digital images sequences match.Finish during the mode of learning configuration in the mode identical to the pattern sample record with producing normalization motion vector sequence.This production process and Fig. 4 E, Fig. 6 A and Fig. 6 B describe relatively.
Preferably, the user is by catching by along new motor pattern displacement conversion equipment and shot digital images sequence or by coming one of configuration mode sample record with pattern sample record and different telephony features are related again.
During mode of learning, be converted to the corresponding motion vector sequence that is stored in the pattern sample record by the digital image sequence of will catch and finish configuration.As mentioned above, produce corresponding motion vector sequence in the mode identical with producing normalization motion vector sequence.The user further is associated relevant control signal with the pattern sample record.Fig. 9 describes exemplary learning process, and wherein conversion equipment is the mobile phone with integrated camera 900 that is in the mode of learning.Among Fig. 9, the user uses motor pattern unit to start the control signal of counter session for ' C ' shape motor pattern 901 configurations impelling mobile phone.Shown in 902, in order to assign ' C ' shape motor pattern 901, the user controls mobile phone along the motion of ' C ' shape motion path.Then, shown in 903, mobile phone screen 904 permits a user to the menu that this motor pattern is assigned certain function.In this step, new pattern sample record is assigned and is related with ' C ' shape motor pattern.In step subsequently, shown in 905, mobile phone screen 904 shows that indication receives the affirmation message of new pattern sample record.This new pattern sample record allow user shown in 905 and 906 by repeating the C posture and the sample posture of storage relatively as previously discussed, use the shortcut of ' C ' shape motor pattern 901 as the calculator function that is used to visit mobile phone.Preferably will represent the motion vector sequence of the record of motor pattern to be converted to normalization motion vector sequence with unified length.This transfer process is preferably identical with aforesaid transfer process.Should be noted that also and any other telephony feature can be associated with any pattern sample record.For example, the control signal with order call number is associated with one of pattern sample record.
As mentioned above, the reception of motion vector sequence can be attended by the order of identification posture.It starts matching process with the match pattern sample record.The pattern sample record can be associated with any function of phone.Thereby any function that can carry out phone is as the reaction relevant with finishing of posture.During aforesaid mode of learning, such motion vector sequence or posture can be associated with any function of phone.
Preferably, use the motor pattern unit predefine motor pattern of related certain pattern sample record again.The user for example can be associated as ' S ' shape predefine motor pattern that is used for indication typewriting input alphabet ' S ' shutdown command of indication conversion equipment shutdown again.
Preferably, conversion equipment is couple to display device and Observation Blocks.This Observation Blocks is configured to the display device output image user interface (GUI) to the explicit user inputting interface.Preferably, this user's inputting interface allows the user that new pattern sample record is associated with certain control signal.Select related control signal by the function of utilizing one of input of user's inputting interface or selection conversion equipment or position application program thereon.Preferably, user's inputting interface allows the user to come the input motion pattern by the MMI that uses and conversion equipment is integrated, is connected thereby carry out interface with the motor pattern converter.The user can use MMI to draw new motor pattern and it is stored as the pattern sample record.As everyone knows, be connected to the allowing user's input information and make one's options of conversion equipment such as the pointing device commonly used of roller and such as the text input device commonly used of keypad.Means based on the graphical user interface displays interface are known, thereby will not be described in detail here.
Preferably, can upgrade or interpolation pattern sample record from external source.In a preferred exemplary, special applications allows the user to utilize the main frame computing unit such as personal computer to produce or the edit pattern sample record.This special applications preferably includes the draw and edit device, and it is configured to allow the user to draw motor pattern and it is stored as the pattern sample record.Preferably utilize conventional mouse to finish drawing.Preferably the pattern sample record with storage sends to mobile phone, to upgrade the pattern sample record that is stored in as mentioned above in its local knowledge base.
In one embodiment of the invention, the user uses motor pattern unit to import security signature.Such security signature can be used to check the individual to receive the qualification of certain categorization information.Preferably with security signature with help some data of access or allow the control signal of some limitation function to be associated.For example, in one embodiment of the invention, conversion equipment is the mobile phone with integrated camera, and security signature is used to the qualification of verifying that the user transfers to or answers the call.This security signature can be space or plane motion pattern.
Be not hard to predict, during the term of validity of this patent, will develop many relevant equipment and system, and (particularly term control signal, wireless and imageing sensor) scope of the term here is intended to a priori comprise all such emerging technologies.
Should be appreciated that for clear and some feature of the present invention that describe under the background of the embodiment that separates also can be combined among the single embodiment provides.On the contrary, for simplicity and the of the present invention various features of describing under the background of single embodiment also can provide dividually or with any suitable sub-portfolio.
Though described the present invention, be not difficult to find out that many other selections, modification and variant will be conspicuous to those skilled in the art in conjunction with its specific embodiment.Thereby, be intended to comprise the spirit that falls into appended claims and whole such other selection, modification and the variant within the broad range.By reference the whole of whole publications, patent and patented claim that mention in this instructions are herein incorporated, promptly reach to be designated as by reference particularly and individually and be herein incorporated as the publication that each is independent, patent or patented claim.In addition, quoting or confirm should not be read as and admit that such list of references can be used as prior art of the present invention any list of references among the application.
Claims (40)
1. one kind is used to discern the having to the equipment of the connection of imageing sensor of the complicated displacement of predefine, comprising:
The motion vector detection module, it is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor, and described motion vector detection module is arranged to the current motion vector that produces described imageing sensor according to described digital picture;
Mixing module, it is configured to receive current motion vector.Described mixing module is configured to the continuous motion sequence vector during current motion vector and the described displacement is mixed;
The motor pattern knowledge base, it is adapted to be a plurality of pattern sample record of storage, a kind of predefine motor pattern of each record representative; And
The motor pattern matching module, it is adapted to be between one of described a plurality of pattern sample record and described continuous motion sequence vector and mates.
2. equipment as claimed in claim 1, wherein said predefine motor pattern is the nonlinear motion pattern.
3. equipment as claimed in claim 1, wherein said motor pattern matching module are configured to determine according to described coupling whether described digital picture is to take along each predefine motor pattern approx.
4. equipment as claimed in claim 1, wherein said mixing module are adapted for and add new record to described a plurality of pattern sample record, and described new record produces according to described digital picture.
5. equipment as claimed in claim 1 further comprises user interface, and described user interface is configured to allow the user to start and stop catching of described digital picture.
6. equipment as claimed in claim 1 further comprises output device, and it is adapted to be the control signal of output based on described coupling.
7. equipment as claimed in claim 5, the following group of wherein said control signal representative at least one of them: telephony feature, be used for the text input character, start described equipment at least a function, have the security signature of plane mode, at least a function that has the security signature of spatial model and stop described image input device.
8. equipment as claimed in claim 3, each in wherein said a plurality of pattern sample record comprises the motion vector array.
9. equipment as claimed in claim 7, wherein said mixing module is adapted to be described continuous motion sequence vector is converted to normalization continuous motion sequence vector, and each normalization motion vector of described normalization sequence and each vector of described motion vector array have unified length.
10. equipment as claimed in claim 1, wherein said equipment be following group one of them: mobile phone, personal digital assistant PDA and laptop computer.
11. equipment as claimed in claim 1, wherein said imageing sensor be following group one of them: complementary metal oxide semiconductor (CMOS) cmos sensor and charge coupled device ccd sensor.
12. equipment as claimed in claim 1, wherein said digital picture are by following group of one of them expression: colourity-saturation degree-purity HSV color coordinate, CIE L*a*b*CIELAB color coordinate, YCbCr color coordinate and R-G-B RGB color coordinate.
13. equipment as claimed in claim 1, wherein said connection are configured to allow user and described equipment to control described imageing sensor motion dividually.
14. equipment as claimed in claim 12, wherein said connection comprise following connection at least one of them: RS-232 connects, Ethernet connects, general-purpose serial bus USB connects, cellular transceiver connects, live wire connects, USB2 connects, bluetooth connects and IR connects.
15. equipment as claimed in claim 1 wherein firmly is couple to described equipment with described imageing sensor.
16. equipment as claimed in claim 5 further comprises communication interface modules, described communication interface modules is adapted to be to remote equipment and sends described control signal.
17. an equipment that is used for shot digital images during the displacement of imageing sensor is converted to control signal comprises:
Have to the image input device of the connection of imageing sensor, described image input device is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor;
The knowledge base of predefine motor pattern; And
The motor pattern converter is associated with described knowledge base and described image input device, and is adapted to be according to the coupling between one of described displacement and described predefine motor pattern described digital picture is converted to described control signal.
18. a method that is used to discern the displacement pattern comprises:
A) be received in the digital picture of catching during the displacement of equipment of integrated image sensor;
B) according to described digital picture identification exercise data sequence; And
C) the predetermined motor pattern with described exercise data sequence and storage mates.
19. method as claimed in claim 17 further comprises the step d) that produces control signal according to described coupling.
20. method as claimed in claim 17 is wherein by mating to come execution in step c with described exercise data sequence and a plurality of pattern sample record) coupling, each record in the described pattern sample record is related with different predefine motor patterns.
21. method as claimed in claim 17, wherein said predefine motor pattern be following group one of them: the predefine profile of character, symbol, gesture, the security signature that has the security signature of plane mode and have spatial model.
22. method as claimed in claim 17, wherein said exercise data comprise the information about the motion during the continuous time fragment of the equipment of described integrated image sensor.
23. method as claimed in claim 17, wherein said exercise data sequence is the motion vector array.
24. method as claimed in claim 22, each record in wherein said a plurality of pattern sample comprises the motion vector array, wherein during described coupling the corresponding motion vector in each motion vector in the described exercise data sequence and the described pattern sample record is mated.
25. method as claimed in claim 23, wherein during described coupling according to be stored in mate the motion vector in the predefine motor pattern quantity with the motion vector segmentation in the described exercise data sequence.
26. method as claimed in claim 22 further is included between step a) and the step b) by in the described motion vector array each is come the step of the described exercise data sequence of normalization divided by its length.
27. method as claimed in claim 25 further comprises the step that described exercise data sequence vector is divided into the equal length fragment of predetermined quantity, each in the described fragment is represented the directional data of the corresponding digital image in the described sequence.
28. method as claimed in claim 17 further is included between step a) and the step b) step that the color coordinate of the pixel of described digital picture is converted to the greyscale color coordinate.
29. method as claimed in claim 18 comprises that further the application program to described equipment sends the step of described control signal.
30. method as claimed in claim 28, wherein said application program is on described equipment.
31. method as claimed in claim 28, wherein said application program is on remote equipment.
32. method as claimed in claim 17 further is included in the step of storing at least a predefine motor pattern before the step a).
33. method as claimed in claim 31, wherein said storage comprise the step of the digital picture that the equipment that is received in described integrated image sensor is caught during one of them the displacement of described at least a predefine motor pattern approx.
34. method as claimed in claim 31, wherein said storage comprises the step that receives described at least a predefine motor pattern from external source.
35. method as claimed in claim 33, wherein said external source is a personal computer.
36. the mobile phone with imageing sensor that is used to discern the complicated displacement of predefine comprises:
The motion vector detection module, it is adapted to be the digital picture of catching during the displacement that is received in described imageing sensor, and described motion vector detection module is arranged to the current motion vector that produces described imageing sensor according to described digital picture;
The motor pattern knowledge base is used to store a plurality of user definition motor patterns; And
The motor pattern matching module, it is adapted to be between one of described a plurality of user definition motor pattern records and continuous motion sequence vector and mates.
37. mobile phone as claimed in claim 35 is wherein by the described motor pattern of the user definition of described mobile phone.
38. a method that is used to carry out the function of mobile hand-held device comprises:
A) be received in the digital picture of catching during the described mobile hand-held device displacement;
B) discern described user definition motor pattern according to described digital picture; And
C) carry out described function according to described user definition motor pattern.
39. method as claimed in claim 37 further is included between step b) and the step c) step that the predefine motor pattern with described user definition motor pattern and storage mates, and wherein comes completing steps c according to described coupling) execution.
40. one kind is used to discern the having to the equipment of the connection of imageing sensor of the complicated displacement of predefine, comprises:
Motion detection block is associated with described imageing sensor, is used for coming the complicated displacement of detection and Identification according to the output from described imageing sensor;
Output module is used to export the control signal based on described identification; And
The functional module of described equipment is used to utilize described control signal to operate the function of described equipment.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US70979405P | 2005-08-22 | 2005-08-22 | |
| US60/709,794 | 2005-08-22 | ||
| US11/503,116 | 2006-08-14 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101248410A true CN101248410A (en) | 2008-08-20 |
Family
ID=39947932
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2006800307545A Expired - Fee Related CN101248409B (en) | 2005-08-22 | 2006-08-22 | A displacement and tilt detection method for a portable autonomous device having an integrated image sensor and a device therefor |
| CN2006800306580A Expired - Fee Related CN101268434B (en) | 2005-08-22 | 2006-08-22 | An autonomous handheld device and operaiton method for controlling the same |
| CNA2006800307850A Pending CN101248410A (en) | 2005-08-22 | 2006-08-22 | Device and method for recognizing movement mode |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2006800307545A Expired - Fee Related CN101248409B (en) | 2005-08-22 | 2006-08-22 | A displacement and tilt detection method for a portable autonomous device having an integrated image sensor and a device therefor |
| CN2006800306580A Expired - Fee Related CN101268434B (en) | 2005-08-22 | 2006-08-22 | An autonomous handheld device and operaiton method for controlling the same |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN101248409B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012174759A1 (en) * | 2011-06-23 | 2012-12-27 | 深圳雷柏科技股份有限公司 | Touch input method and apparatus for mode switching |
| CN103019530A (en) * | 2013-01-04 | 2013-04-03 | 深圳市中兴移动通信有限公司 | Menu indexing method and device based on gesture recognition and terminal thereof |
| CN103678417A (en) * | 2012-09-25 | 2014-03-26 | 华为技术有限公司 | Man-machine interaction data processing method and device |
| CN104759087A (en) * | 2015-04-07 | 2015-07-08 | 张锐明 | Linear movement training method and device |
| TWI552025B (en) * | 2014-05-14 | 2016-10-01 | 瑞尼斯股份有限公司 | Pattern matching method and touch integrated circuit |
| CN107529027A (en) * | 2016-06-21 | 2017-12-29 | 手持产品公司 | Dual mode image sensor and the method using dual mode image sensor |
| CN108833812A (en) * | 2018-06-25 | 2018-11-16 | 广东工业大学 | A kind of imaging sensor and Image Motional Information processing method |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5156972B2 (en) | 2009-07-07 | 2013-03-06 | Ｓｍｃ株式会社 | Position measuring apparatus and position measuring method |
| CN102918476B (en) * | 2010-02-05 | 2017-03-08 | 索尼电脑娱乐公司 | For the controller using position, direction or action and calculation procedure interfaces |
| KR101883376B1 (en) * | 2011-11-22 | 2018-07-31 | 엘지전자 주식회사 | Mobile terminal and control method thereof |
| CN103020616B (en) * | 2012-11-30 | 2017-02-08 | 天津三星电子有限公司 | Method for detecting space position of handheld device and handheld device |
| CN103854296B (en) * | 2012-12-06 | 2017-10-03 | 腾讯科技（深圳）有限公司 | The method and device that control pigment is tinted |
| ES2755764T3 (en) | 2013-12-13 | 2020-04-23 | Huawei Device Co Ltd | Method and terminal to acquire a panoramic image |
| US10234990B2 (en) * | 2015-09-29 | 2019-03-19 | Microchip Technology Incorporated | Mapping of position measurements to objects using a movement model |
| CN107092430B (en) | 2016-02-18 | 2020-03-24 | 纬创资通（中山）有限公司 | Space drawing scoring method, device and system for scoring space drawing |
| CN105913436A (en) * | 2016-04-13 | 2016-08-31 | 乐视控股（北京）有限公司 | Wind information determining method and mobile terminal |
| CN107678818B (en) * | 2017-09-22 | 2019-11-22 | 维沃移动通信有限公司 | A kind of user interface control method and mobile terminal |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100265721B1 (en) * | 1997-03-31 | 2000-09-15 | 윤종용 | Method for estimating the motion of pictures using 2-D triangle-patch wireframe model |
| KR100252080B1 (en) * | 1997-10-10 | 2000-04-15 | 윤종용 | Apparatus for stabilizing video signals through revising the motion of the video signals using bit plane matching and a stabilizing method therefor |
| SE511516C2 (en) * | 1997-12-23 | 1999-10-11 | Ericsson Telefon Ab L M | Handheld display unit and method for displaying screens |
| DE10028138A1 (en) * | 2000-06-07 | 2001-12-20 | Siemens Ag | Mobile communications device acts as pen with incorporated image sensor unit for detection and storage of writing movements |
| JP2003008693A (en) * | 2001-06-21 | 2003-01-10 | Nec Access Technica Ltd | Portable telephone |
| JP4022090B2 (en) * | 2002-03-27 | 2007-12-12 | 富士通株式会社 | Finger movement detection method and detection apparatus |
| US7940844B2 (en) * | 2002-06-18 | 2011-05-10 | Qualcomm Incorporated | Video encoding and decoding techniques |
| KR100547792B1 (en) * | 2002-10-24 | 2006-01-31 | 삼성전자주식회사 | Wireless communication terminal that can change the image file of the background screen and display method of the image file of the background screen using the same |
-
2006
- 2006-08-22 CN CN2006800307545A patent/CN101248409B/en not_active Expired - Fee Related
- 2006-08-22 CN CN2006800306580A patent/CN101268434B/en not_active Expired - Fee Related
- 2006-08-22 CN CNA2006800307850A patent/CN101248410A/en active Pending
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012174759A1 (en) * | 2011-06-23 | 2012-12-27 | 深圳雷柏科技股份有限公司 | Touch input method and apparatus for mode switching |
| CN103678417A (en) * | 2012-09-25 | 2014-03-26 | 华为技术有限公司 | Man-machine interaction data processing method and device |
| CN103678417B (en) * | 2012-09-25 | 2017-11-24 | 华为技术有限公司 | Human-machine interaction data treating method and apparatus |
| CN103019530A (en) * | 2013-01-04 | 2013-04-03 | 深圳市中兴移动通信有限公司 | Menu indexing method and device based on gesture recognition and terminal thereof |
| CN103019530B (en) * | 2013-01-04 | 2016-09-28 | 深圳市中兴移动通信有限公司 | The index method of menu based on gesture identification, device and terminal thereof |
| TWI552025B (en) * | 2014-05-14 | 2016-10-01 | 瑞尼斯股份有限公司 | Pattern matching method and touch integrated circuit |
| CN104759087A (en) * | 2015-04-07 | 2015-07-08 | 张锐明 | Linear movement training method and device |
| CN104759087B (en) * | 2015-04-07 | 2017-01-18 | 张锐明 | Linear movement training method and device |
| CN107529027A (en) * | 2016-06-21 | 2017-12-29 | 手持产品公司 | Dual mode image sensor and the method using dual mode image sensor |
| CN108833812A (en) * | 2018-06-25 | 2018-11-16 | 广东工业大学 | A kind of imaging sensor and Image Motional Information processing method |
| CN108833812B (en) * | 2018-06-25 | 2020-08-11 | 广东工业大学 | Image sensor and image dynamic information processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101268434A (en) | 2008-09-17 |
| CN101248409A (en) | 2008-08-20 |
| CN101268434B (en) | 2010-06-16 |
| CN101248409B (en) | 2012-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101248410A (en) | Device and method for recognizing movement mode | |
| US7809214B2 (en) | Device and a method for identifying movement patterns | |
| CN110321965B (en) | Training method of object re-recognition model, and object re-recognition method and device | |
| US20070067745A1 (en) | Autonomous handheld device having a drawing tool | |
| WO2020013577A1 (en) | Electronic device and method for predicting user actions on ubiquitous devices | |
| US8577100B2 (en) | Remote input method using fingerprint recognition sensor | |
| CN111027528A (en) | Language identification method and device, terminal equipment and computer readable storage medium | |
| KR20110085728A (en) | Apparatus and method for recognzing structure area in portable terminal | |
| CN111209812A (en) | Target face picture extraction method and device and terminal equipment | |
| KR100648161B1 (en) | System and Method For Collaborative Handwriting Input | |
| US20230093983A1 (en) | Control method and device, terminal and storage medium | |
| US20210158031A1 (en) | Gesture Recognition Method, and Electronic Device and Storage Medium | |
| CN200997112Y (en) | Inputting device for electronic apparatus | |
| CN110796096B (en) | Training method, device, equipment and medium for gesture recognition model | |
| CN110443852B (en) | Image positioning method and related device | |
| CN111782041A (en) | Typing method and device, equipment and storage medium | |
| CN117234405A (en) | Information input method and device, electronic equipment and storage medium | |
| US9224590B2 (en) | Finger biometric sensor including sliding motion and static positioning based biometric data generation and related methods | |
| CN101655738B (en) | Method for operating electronic device and control device therefor | |
| CN103034333A (en) | Gesture recognition device and gesture recognition method | |
| CN107844242B (en) | Display method of mobile terminal and mobile terminal | |
| CN111062258A (en) | Text region identification method and device, terminal equipment and readable storage medium | |
| JP6321204B2 (en) | Product search device and product search method | |
| Dudhapachare et al. | Voice Guided, Gesture Controlled Virtual Mouse | |
| CN115209197B (en) | Image processing method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Open date: 20080820 |