CN102866803A - Method and device for operating and controlling virtual center console of Blind-operation-supported automobile by gestures - Google Patents
Method and device for operating and controlling virtual center console of Blind-operation-supported automobile by gestures Download PDFInfo
- Publication number
- CN102866803A CN102866803A CN2012103127984A CN201210312798A CN102866803A CN 102866803 A CN102866803 A CN 102866803A CN 2012103127984 A CN2012103127984 A CN 2012103127984A CN 201210312798 A CN201210312798 A CN 201210312798A CN 102866803 A CN102866803 A CN 102866803A
- Authority
- CN
- China
- Prior art keywords
- gesture
- sign language
- module
- touch
- screen
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a method and a device for operating and controlling a virtual center console of a blind-operation-supported automobile by gestures. By the method and the device, a user can conveniently use gestures to perform corresponding operation and control on a center console touch screen during driving without transferring road driving sight line, and by an automobile blind operation function, the shortcoming of searching complex and various mechanical buttons in the conventional driving room to control the conventional automobile is well overcome; and meanwhile, the virtual center console integrates a gesture language library module, a gesture analysis and processing module, a gesture language recognition module, a gesture language library management module, a gesture operation and control execution module and a gesture operation and control security mechanism management module, and because of cooperative work of the modules, gesture language operation and control has the characteristics that the gesture language operation and control is easy and convenient to use, gesture operation and control windows are independent, gesture language meaning is unique, a gesture language can be modified, an operation and control mechanism is safe and the like, and convenient, rapid, friendly and safe operation and control experiences of the automobile center console can be brought to the user according to specific requirements.
Description
Technical field
The present invention relates to auto electronic control technology, especially a kind of gesture control method and device that is applied to the virtual console of automobile.
Background technology
Along with developing rapidly and popularization and application of automotive electronics, increasing control function is arranged in the automobile.In pilothouse, all be covered with function control knob miscellaneous around central control board and the bearing circle.These diversified control knobs really provide to the user and have more controlled the selection right in the car, but bring impact also for simultaneously driver's operation ease and complexity, so that the driver is in carrying out the driving driving procedure of high security, also to decontrol bearing circle go to stretch out one's hand in addition turn one's head want sight line leave that driving road goes to see or the longer time go to search the position of wanting the function button place of carrying out, such behavior will inevitably disperse driver's notice, causes larger hidden danger to traffic safety undoubtedly.The driver when driving, sight line is left driving road and is removed to search and press, rotate or stir these a feast for the eyes function buttons, has really affected the security of driving, the operation control when giving driving is also made troubles.
Now, along with the development of auto industry and the raising of computer level, for the automobile man-machine interaction is integrated more requirements have been proposed.Automobile is controlled the new trend that becomes era development by touch-screen.Touch screen technology is a kind of novel man-machine interaction input mode, compares with traditional key-press input mode, and touch-screen is inputted more intuitive and convenient.Touch-screen is comprised of the detection part that is installed in the lcd screen front and touch screen controller.When finger or other object touch the graphic button that is installed on the display front end touch-screen, the position that touches is detected by the touch-screen detection part, after detecting user touch location, send its contact coordinate information to touch screen controller, touch screen controller is analyzed the order of rear execution corresponding operating to touch information.Therefore, can replace traditional mechanical button control panel in the automobile cab with touch-screen, and produce vivid figure manipulation effect by the liquid crystal display picture of touch-screen.
Yet, although existing automobile touch screen has improved the aesthetic property at the key controlled interface of automobile cab console and mutual friendly, but still rest on operations such as the simple touch-control clicks of tradition to come automobile is controlled, the driver is in the driving driving procedure, if operate by touch-screen, still need to allow sight line leave driving road and transfer to the figure button of checking above the touch-screen, come click keys to carry out the feature operation of wanting.Therefore, allow driver's sight line in the driving driving procedure leave driving road and go to click executable operations, still has larger potential safety hazard, do not need pilot's line of vision to leave driving road from changing in essence traditional mechanical button control panel, and go the deficiency of checking that the button position just can operate.
Summary of the invention
Purpose of the present invention will solve the deficiency that present above-mentioned automobile center console touch screen technology exists just, and provide a kind of automobile virtual console gesture control method and device of supporting blind operation, so that the driver can divert sb.'s attention the console touch-screen of checking pilothouse, touch-screen is carried out gesture operation can carry out corresponding function and only use, realize the blind operating function to the virtual console touch-screen of automobile.
In order to solve the problems of the technologies described above, technical scheme of the present invention is as follows:
A kind of virtual console gesture of automobile actuation means of supporting blind operation comprises that console touch-screen, sign language library module, sign language database management module, gesture analysis processing module, sign language identification module, gesture are controlled execution module and gesture is controlled the security mechanism management module;
Described console touch-screen adopts capacitive touch screen, adopts the touch screen controller of programmable system on chip framework, can communicate and collaborative work with other peripheral hardwares, and can customized demand obtain gesture information;
Described sign language library module has been stored sign language through definition and has been mentioned and sign language automobile center console manipulation function one to one;
Described sign language database management module is used for the sign language of management sign language library module;
Described gesture analysis processing module is used for the gesture that described console touch-screen detects is carried out analyzing and processing, obtains then to call the sign language identification module behind the complete sign language;
Described sign language identification module is analyzed identification to the gesture information that obtains, and judges the matching similarity of sign language in its gesture path and the sign language library module, if reach certain threshold value, then can think this gesture; Described gesture is controlled the gesture information that execution module identifies according to described sign language identification module, inquire about its corresponding automobile operation, transmit control signal to electrical control unit system or vehicle mounted electrical apparatus control system by CAN, LIN bus, corresponding automobile component is controlled, carried out corresponding operating;
Described gesture is controlled the security mechanism management module and is used for the gesture bookkeeping of described sign language database management module is carried out safety verification.
Further, when the sign language database management module manages operation to the sign language in the gesture language library, need gesture to control the security mechanism management module and carry out safety verification, after only having safety verification to pass through, could manage operation to the gesture language library, after through safety certification, when wanting to add new gesture operation, at first get access to the gesture operation of typing, and can be to analyzing identification, judge this sign language whether with existing sign language library module in the sign language stored higher duplication similarity degree is arranged, if similarity some in the new sign language that preparation is added and the existing sign language reaches more than the threshold value of setting, can think that then the sign language conflict repeats, add unsuccessful, need in addition again input, until the requirement of satisfying the similarity collision detection, after the similarity collision detection, select again current automobile center console manipulation function optional and that do not used by other sign languages, at last new gesture and its corresponding manipulation function are added in the sign language library module.
A kind of virtual console gesture of automobile control method of supporting blind operation comprises the steps
31) the console touch-screen detects and obtains gesture information;
32) the gesture analysis processing module is used for the gesture that described console touch-screen detects is carried out analyzing and processing, obtains then to call the sign language identification module behind the complete sign language;
33) described sign language identification module is analyzed identification to the gesture information that obtains, and judges the matching similarity of sign language in its gesture path and the sign language library module, if reach certain threshold value, then can think this gesture; Described gesture is controlled the gesture information that execution module identifies according to described sign language identification module, inquire about its corresponding automobile operation, transmit control signal to electrical control unit system or vehicle mounted electrical apparatus control system by CAN, LIN bus, corresponding automobile component is controlled, carried out corresponding operating;
Described sign language library module has been stored sign language through definition and has been mentioned and sign language automobile center console manipulation function one to one;
34) the sign language database management module is used for the sign language of management sign language library module, after controlling security mechanism management module safety verification through gesture, and can be to the gesture bookkeeping in the described sign language database management module.
Further, the gesture analysis disposal route of gesture analysis processing module comprises the steps:
41) the console touch-screen detects touch in one's hands, waits for that the gesture input is complete, obtains complete gesture information;
42) call the sign language identification module;
43) the gesture analysis processing module judges whether the gesture operation that detects is conventional touch screen operation;
44) if conventional touch screen operation, allocating conventional gesture processing module then according to the concrete window in current place, is carried out the corresponding conventional gesture operation to it;
Does 45) then the sign language identification module determine whether sign language in the sign language library module if not conventional touch screen operation?
46) if not sign language in the sign language library module is not then carried out any instruction of controlling;
47) if sign language in the sign language library module then calls gesture and controls execution module, carry out feature operation corresponding to sign language that defines in the sign language library module.
Further, in described step 41) in this console touch-screen when having detected hand and touched, according to the stand-by period that arranges, wait for that complete gesture input finishes.
Further, in step 43) in the gesture analysis processing module sign language storehouse gesture operation of conventional touch-screen gesture operation and definition is treated with a certain discrimination, the operation that the graphical window to the current demonstration of console touch-screen carries out for conventional touch-screen gesture operation, this gesture is the gesture operation of window correlativity, irrelevant for the operation of the sign language in the sign language library module and current place window interface, use these sign languages can carry out corresponding automobile at any window and control order.
Further, for having detected the gesture touch-control from the console touch-screen, comprise the steps: to the disposal route of this gesture being called the sign language identification module
71) when the console touch-screen detects finger touch, console touch-screen detection part sends look-at-me with the interruption form to the gesture analysis processing module;
72) after the gesture analysis processing module receives look-at-me, call corresponding interruption and process function;
73) initiation gesture is controlled language incoming timing device;
74) arrive when timer time, then obtain all gesture informations of current console touch-screen;
75) all gesture informations of current console touch-screen carried out analyzing and processing;
76) each gesture information is called respectively the sign language identification module, after it is identified, carry out corresponding operating.
Beneficial effect of the present invention is: the present invention can make things convenient for the user in driving procedure, need not shift the driving path sight line, comes automobile is carried out corresponding controlling by oarsman's gesture on the console touch-screen.This blind operating function to automobile has well remedied and has searched the deficiency that complicated and diversified mechanical key is controlled in traditional pilothouse.Simultaneously, the virtual console of the present invention is integrated sign language library module, the gesture analysis processing module, the sign language identification module, the sign language database management module, gesture is controlled execution module, and gesture is controlled the security mechanism management module.The associated treatment work of modules so that its sign language control have simple ease for use, gesture is controlled the window independence, gesture semantic uniqueness, sign language alterability, control the characteristics such as made safe.Can be according to real needs, for the user provide convenience, fast, friendly, safe automobile center console controls experience.
Description of drawings
Fig. 1 is the hierarchical chart of each functional module in apparatus of the present invention system;
Fig. 2 is the connection layout between apparatus of the present invention and electrical control unit system and vehicle mounted electrical apparatus control system;
Fig. 3 be apparatus of the present invention system self-defining sign language storehouse and its corresponding carry out feature operation;
Fig. 4 is the process flow diagram of processing the sign language input in apparatus of the present invention system;
Fig. 5 is the process flow diagram of gesture analysis processing module in apparatus of the present invention system;
Fig. 6 is the treatment step process flow diagram of the virtual console sign language of this automobile base management system;
Fig. 7 is that the virtual console sign language of this automobile administration module interpolation gesture is controlled process flow diagram.
Embodiment
The present invention is described further below in conjunction with the drawings and specific embodiments.
As shown in Figure 1, a kind of virtual console gesture of automobile actuation means of supporting blind operation, this console (center controlling platform) device comprises the console touch-screen, the sign language library module, the sign language database management module, gesture analysis processing module, sign language identification module, gesture is controlled execution module, and gesture is controlled the security mechanism management module.
The console touch-screen adopts capacitive touch screen, and the two or more points on the surface in contact are identified in this touch-screen support simultaneously, also namely support the Multi-contact location recognition.The core of multi-touch system is the capacitive sensing that a pair of adjacent electrode forms.When a conductor such as finger during near these electrodes, the electric capacity between two electrodes will increase, and then detects by microcontroller, thereby identifies touching position information.
This console touch-screen adopts the touch screen controller of programmable system on chip framework, can conveniently induce the touch situation of the single finger of user and a plurality of fingers, and can communicate and collaborative work with other peripheral hardwares, have good design flexibility and configurability, thereby conveniently obtain gesture information according to customized demand.
The sign language library module has defined the sign language of a cover towards the operation of automobile cab console, can pre-defined one feature operation of overlapping sign language storehouse and its corresponding virtual console of automobile of carrying out in this virtual console internal system.
This sign language library module has following features:
(a) manipulation function corresponding to sign language that defines in this sign language library module generally is driver's feature operation relatively more commonly used, and sign language is not to have covered all feature operations of automobile cab console.
(b) operation of the common function of the sign language in this sign language library module and the virtual console of automobile is one-to-one relationship.
(c) sign language that defines in this sign language library module is to a kind of expansion of existing conventional touch-screen gesture operation commonly used (such as clicking, long pressing slides page up and down, amplify dwindle etc.) and additional.The sign language of definition does not conflict mutually with the gesture operation that existing touch-screen is commonly used in this sign language library module.
(d) gesture operation that defines in this sign language library module can be carried out at any graphic interface window of virtual console touch-screen, does not need to turn back to the corresponding function window interface and could carry out the corresponding function operation.
(e) this sign language has the advantages that the semantic uniqueness of gesture and gesture are controlled the window independence.Wherein the semantic the Uniqueness of gesture is controlled the security mechanism management module by gesture and is processed, and gesture is controlled window independence problem and processed by the gesture analysis processing module.
The sign language database management module supports that the user manages operation to the sign language in the gesture language library module, and such as the operation that increases self-defined gesture and its correspondence, the deletion gesture is revised the operation of sign language and its correspondence etc.Simultaneously, when sign language is made amendment, need gesture to control the security mechanism management module and carry out safety verification.After only having safety verification to pass through, could make amendment to the gesture language library.
When the user through safety certification after, when wanting to add new gesture operation, at first get access to the gesture operation of user's typing, and can be to analyzing identification, judge this gesture whether with existing sign language library module in gesture higher duplication similarity degree is arranged, if some similarities reach the threshold value of setting above (calculating of gesture curve similarity have method and the algorithm of a lot of comparative maturities in the new gesture that the user prepares to add and the existing gesture, calculating for the gesture similarity can utilize conventional method), can think that then the gesture conflict repeats, add unsuccessful, need in addition again input of user, until the requirement of satisfying the collision detection of gesture similarity, after the collision detection of gesture similarity, select again current automobile center console manipulation function optional and that do not used by other sign languages, at last new gesture and its corresponding manipulation function are added in the sign language library module to getting final product.
The gesture analysis processing module mainly is responsible for the gesture that the console touch-screen detects is carried out analyzing and processing, and the gesture analysis process method step of this module is as follows:
(1) touch-screen detects touch in one's hands, waits for that the input of user's gesture is complete, obtains complete gesture information;
(2) call the sign language identification module;
Does (3) the gesture analysis processing module judge that the gesture operation that detects is conventional touch screen operation?
(4) if then allocating conventional gesture processing module according to the concrete window in current place, is carried out the corresponding conventional gesture operation to it.
(5) if not, then determine whether language in the sign language storehouse?
(6) if not, then do not carry out any instruction of controlling.
(7) if then call gesture and control execution module, feature operation corresponding to sign language that defines in the execution sign language storehouse.
In described step (1): this virtual console touch-screen is not just to call the sign language identification module at once having detected the hand touch, because the user is carrying out the gesture operations such as some multi-point touch, therefore after will waiting for that the complete input of user is finished, just the complete gesture information that obtains is analyzed identification.Otherwise the parsing identification error that may cause the gesture semanteme.The time of waiting for the input of user's gesture can according to circumstances arrange.
In described step (3): this gesture analysis processing module is treated the sign language storehouse gesture operation of conventional touch-screen gesture operation and definition with a certain discrimination.Be the operation that the graphical window to the current demonstration of console touch-screen carries out for conventional touch-screen gesture operation (such as clicking, long pressing slides page up and down, amplify dwindle etc.), so these gestures are gesture operations of window correlativity.And the gesture operation in the sign language library module and current place window interface are irrelevant, use these gestures can carry out corresponding automobile at any window and control order, so the gesture operation in the sign language storehouse is the window independence.
For having detected the gesture touch-control from the console touch-screen, as follows to the process method step of this gesture being called the sign language identification module:
(11) when the console touch-screen detects finger touch, the touch-screen detection part sends look-at-me with the interruption form to the gesture analysis processing module;
(12) after the gesture analysis processing module receives look-at-me, call corresponding interruption and process function;
(13) initiation gesture is controlled language incoming timing device;
(14) arrive when timer time, then obtain all gesture informations of current touch-screen;
(15) all gesture informations of current touch-screen carried out analyzing and processing;
(16) each gesture information is called respectively the sign language identification module, after it is identified, carry out corresponding operating.
The sign language identification module is analyzed identification to the complete gesture information of touch-screen that obtains, and judges the matching similarity of sign language in its gesture path and the sign language storehouse, if reach certain threshold value, then can think this gesture.Search the corresponding automobile operation of carrying out of this gesture in the sign language storehouse.
Gesture is controlled the gesture information that execution module identifies according to the sign language identification module, inquire about its corresponding automobile operation, this virtual console transmits control signal to electrical control unit system (BCM) or vehicle mounted electrical apparatus control system by CAN, LIN bus, corresponding automobile component is controlled, carried out corresponding operating.
Gesture is controlled the security mechanism management module and mainly is responsible for the gesture retouching operation in the gesture language library administration module is carried out safety verification.The characteristics of this module are as follows:
(a) when using this virtual console touch-screen gesture control system first, need car owner elder generation its finger print information of typing, can one of typing point or a plurality of finger, after typing is finished, when needing later on the gesture language library made amendment, just need first by finger print identification verification.If need later on to add the new finger print information of management, the finger print identification verification that arranges before also needing at first to pass through.
(b) only have car owner user by after the finger print identifying identity, just can self-defined interpolation gesture and the operation of its correspondence or sign language deleted, revises.Guarantee the security in this console touch-screen sign language storehouse.
(c) gesture is controlled the security mechanism management module gesture similarity collision detection function.Newly-increased gesture for user's input, system accounting calculates that there has been the similarity of gesture in all in current input gesture and the system, only have all gesture similarities all to be lower than setting threshold, can think that just this gesture is new gesture, add it to existing sign language storehouse, otherwise new gesture is added unsuccessful.Guaranteed like this uniqueness of sign language.
As shown in Figure 2, the invention provides a kind of virtual console gesture of automobile actuation means of supporting blind operation, the virtual console touch-screen of this automobile has a touch-screen press-key lock, when pressing this button, the console touch-screen can be pinned, this moment, console was in holding state, did not support gesture operation; Click again the console press-key lock, then recover the gesture touch and control.
This virtual console links to each other by electrical control unit system (BCM) or the vehicle mounted electrical apparatus control system of LIN/CAN bus and automobile, controls automobile execution corresponding operating by sending the corresponding signal of controlling.
Simultaneously, this console touch-screen adopts the touch screen controller of programmable system on chip framework, this console the inside is integrated sign language library module, the gesture analysis processing module, the sign language identification module, the sign language database management module, gesture is controlled execution module, and gesture is controlled the security mechanism management module.
The hierarchical structure of each functional module as shown in Figure 1 in this touch screen controller.The user can carry out two large generic operations by this touch-screen: a class mainly is the operation that the user carries out the sign language in the native system sign language storehouse, mainly by the sign language library module, sign language database management module and gesture are controlled security mechanism management module etc. and are responsible for; Another kind of mainly is the automobile control operation that the user utilizes sign language to carry out, and mainly by the gesture analysis processing module, sign language identification module and gesture are controlled execution module etc. and be responsible for.
As a selectivity case study on implementation of the present invention, as shown in Figure 3, this virtual console internal system can be pre-defined the feature operation of a cover sign language storehouse and its corresponding virtual console of automobile of carrying out.This sign language can be a stroke gesture, two stroke gestures or Multi strokes gesture, and suitably defines relevant gesture according to the meaning of concrete operations, makes things convenient for the user to remember use.
Such as opening automotive window (window), control with the gesture of its English initial w; Open boot (pack), control with the gesture of its English initial p; Hood unblank (suo) is controlled with the gesture of the initial s of its Chinese phonetic alphabet; Open glass sunroof, carry out that according to the shape " mouth " in skylight gesture is controlled etc.These also can according to user's concrete use habit, be changed by the sign language database management module.
One. the car owner logins and uses virtual console touch-screen system process first:
Basic step when car owner user uses the virtual console gesture control system of this automobile first is as follows:
(1) car owner's typing fingerprint creates car owner's finger print information, enters the virtual console gesture control system of this automobile.
(2) car owner is at certain gesture operation of console touch-screen input, according to processing sign language input operation entire flow of the present invention as shown in Figure 4.
(3) according to the gesture control method of this system, the gesture of input is called the gesture analysis processing module, its processing procedure as shown in Figure 5.
As a selectivity case study on implementation of the present invention, below explanation operates a concrete case process the user of the gesture controlling equipment that uses the virtual console of this automobile:
(a) car owner is in the console touch-screen input gesture of sliding left, and according to the control method of this system, this gesture of system identification is conventional touch screen operation gesture, then carries out corresponding next page operations of translating into.
(b) car owner is when the clockwise C of console touch-screen input, this gesture of system identification is unconventional touch-screen gesture operation, this sign language is arranged in the sign language storehouse simultaneously, therefore call gesture and control execution module execution corresponding operating, namely open air-conditioning.
(c) this moment, the car owner was at the clockwise triangle of console touch-screen input, turned on the room light lamp.
(d) this moment, the car owner was when the clockwise circle of console touch-screen input, turned on radio.
(e) this moment, the car owner during wedge angle, heightened air-conditioner temperature on console touch-screen input is from left to right clockwise.
(f) this moment, the car owner was when the input of console touch-screen is square clockwise, then glass sunroof was opened.
In this case operating process, gesture operation need to could not carried out corresponding gesture operation in respective window, namely need to could not regulate air-conditioner temperature at console touch-screen air-conditioning control interface, as long as it is that corresponding gesture is controlled that system detects, namely can carry out corresponding operating at once, irrelevant with current place window interface.Therefore, this process has embodied the virtual console of this automobile system and has had the advantages that gesture is controlled window independence and semantic uniqueness.
Two. car owner's management correcting sign language storehouse process:
The car owner can manage the sign language storehouse of this system, supports the user to carry out such as the operation that increases self-defined gesture and its correspondence, and the deletion gesture is revised the operation of sign language and its correspondence etc.
The treatment step of this sign language base management system is as follows:
(1) user logins sign language library management interface, carries out authentication;
(2) after authentication is passed through, the operation that user selection manages gesture;
(3) if selection is " adding new gesture ", then need the new gesture of typing, new gesture is carried out similarity detect, after detection was passed through, the automobile of selecting corresponding newly gesture to carry out operated, and it is joined in the sign language storehouse.
(4) if selection is " revising gesture and its respective operations ", then needing further to select is that " gesture that retouching operation is corresponding " or selection " are revised operation corresponding to gesture ";
(a) if select " gesture that retouching operation is corresponding ", then need the new gesture of typing, new gesture is carried out similarity detect, after detection is passed through, can successfully revise the new gesture of this operational correspondence.
(b) if select " revising operation corresponding to gesture ", then select the operation of the available execution of present automobile, modification is saved in the sign language storehouse gets final product.
(5) if selection is " the deletion gesture is controlled ", then need to select the gesture of deletion, its deletion is got final product.
The treatment step process flow diagram of the virtual console sign language of this automobile base management system as shown in Figure 6.
As a selectivity case study on implementation of the present invention, below explanation is for adding the concrete steps that new sign language is controlled:
(a) at first call gesture and control the security mechanism management module, user identity is verified;
(b) virtual console touch-screen typing user fingerprints information;
Is (c) user fingerprints information complementary in the finger print information of judging user's typing and the database?
(d) if not, then subscriber authentication failure.
(e) if then the user is in the newly-increased gesture of console touch-screen input;
(f) newly-increased gesture information is analyzed identification;
(g) calculate the similarity that increases all gestures in gesture and the console touch-screen newly;
(h) judge whether that the gesture similarity reaches the threshold value of setting?
(i) if having, then newly-increased gesture is added unsuccessful.
(j) if do not have, then select automobile operation corresponding to newly-increased gesture;
(k) newly-increased gesture and the operation of its corresponding automobile are added in the sign language storehouse.
The virtual console sign language of this automobile administration module adds gesture and controls detailed process as shown in Figure 7.
Can find out that by above example the present invention can make things convenient for the car owner to carry out blind operation in driving procedure, comes automobile is carried out corresponding controlling by oarsman's gesture on the console touch-screen.Simple ease for use, window independence, the management correcting in sign language storehouse and the characteristics such as security mechanism wherein of the operation of its sign language, can provide convenience for the car owner, fast, the automobile center console of safety controls experience.
The virtual console gesture of the automobile control method of the blind operation of support provided by the invention, can be widely used in all kinds of automobile middle control bench control systems, sign language, optional automobile operation and gesture identification method can be determined different customizations according to real needs in its sign language storehouse.
The above only is preferred implementation of the present invention; should be pointed out that for those skilled in the art, without departing from the inventive concept of the premise; can also make some improvements and modifications, these improvements and modifications also should be considered as in the protection domain of the present invention.
Claims (7)
1. the virtual console gesture of automobile actuation means of supporting blind operation, it is characterized in that, comprise that console touch-screen, sign language library module, sign language database management module, gesture analysis processing module, sign language identification module, gesture are controlled execution module and gesture is controlled the security mechanism management module;
Described console touch-screen adopts capacitive touch screen, uses the touch screen controller of programmable system on chip framework, can communicate and collaborative work with other peripheral hardwares, and can obtain gesture information according to customized demand;
Described sign language library module has been stored sign language through definition and has been mentioned and sign language automobile center console manipulation function one to one;
Described sign language database management module is used for the sign language of management sign language library module;
Described gesture analysis processing module is used for the gesture that described console touch-screen detects is carried out analyzing and processing, obtains then to call the sign language identification module behind the complete sign language;
Described sign language identification module is analyzed identification to the gesture information that obtains, and judges the matching similarity of sign language in its gesture path and the sign language library module, if reach certain threshold value, then can think this gesture; Described gesture is controlled the gesture information that execution module identifies according to described sign language identification module, inquire about its corresponding automobile operation, transmit control signal to electrical control unit system or vehicle mounted electrical apparatus control system by CAN, LIN bus, corresponding automobile component is controlled, carried out corresponding operating;
Described gesture is controlled the security mechanism management module and is used for the gesture bookkeeping of described sign language database management module is carried out safety verification.
2. a kind of virtual console gesture of automobile actuation means of supporting blind operation according to claim 1, it is characterized in that, when the sign language database management module manages operation to the sign language in the gesture language library, need gesture to control the security mechanism management module and carry out safety verification, after only having safety verification to pass through, could manage operation to the gesture language library, after through safety certification, when wanting to add new gesture operation, at first get access to the gesture operation of typing, and can be to analyzing identification, judge this sign language whether with existing sign language library module in the sign language stored higher duplication similarity degree is arranged, if similarity some in the new sign language that preparation is added and the existing sign language reaches more than the threshold value of setting, can think that then the sign language conflict repeats, add unsuccessful, need in addition again input, until the requirement of satisfying the similarity collision detection, after the similarity collision detection, select again current automobile center console manipulation function optional and that do not used by other sign languages, at last new gesture and its corresponding manipulation function are added in the sign language library module.
3. the virtual console gesture of an automobile control method of supporting blind operation is characterized in that, comprises the steps
31) the console touch-screen detects and obtains gesture information;
32) the gesture analysis processing module is used for the gesture that described console touch-screen detects is carried out analyzing and processing, obtains then to call the sign language identification module behind the complete sign language;
33) described sign language identification module is analyzed identification to the gesture information that obtains, and judges the matching similarity of sign language in its gesture path and the sign language library module, if reach certain threshold value, then can think this gesture; Described gesture is controlled the gesture information that execution module identifies according to described sign language identification module, inquire about its corresponding automobile operation, transmit control signal to electrical control unit system or vehicle mounted electrical apparatus control system by CAN, LIN bus, corresponding automobile component is controlled, carried out corresponding operating;
Described sign language library module has been stored sign language through definition and has been mentioned and sign language automobile center console manipulation function one to one;
34) the sign language database management module is used for the sign language of management sign language library module, after controlling security mechanism management module safety verification through gesture, and can be to the gesture bookkeeping in the described sign language database management module.
4. a kind of virtual console gesture of automobile control method of supporting blind operation according to claim 3 is characterized in that the gesture analysis disposal route of gesture analysis processing module comprises the steps:
41) the console touch-screen detects touch in one's hands, waits for that the gesture input is complete, obtains complete gesture information;
42) call the sign language identification module;
43) the gesture analysis processing module judges whether the gesture operation that detects is conventional touch screen operation;
44) if conventional touch screen operation, allocating conventional gesture processing module then according to the concrete window in current place, is carried out the corresponding conventional gesture operation to it;
Does 45) then the sign language identification module determine whether sign language in the sign language library module if not conventional touch screen operation?
46) if not sign language in the sign language library module is not then carried out any instruction of controlling;
47) if sign language in the sign language library module then calls gesture and controls execution module, carry out feature operation corresponding to sign language that defines in the sign language library module.
5. a kind of virtual console gesture of automobile control method of supporting blind operation according to claim 4, it is characterized in that, in described step 41) in this console touch-screen when having detected hand and touched, according to the stand-by period that the incoming timing device arranges, wait for that complete gesture input finishes.
6. a kind of virtual console gesture of automobile control method of supporting blind operation according to claim 4, it is characterized in that, in step 43) in the gesture analysis processing module sign language storehouse gesture operation of conventional touch-screen gesture operation and definition is treated with a certain discrimination, the operation that the graphical window to the current demonstration of console touch-screen carries out for conventional touch-screen gesture operation, this gesture is the gesture operation of window correlativity, irrelevant for the operation of the sign language in the sign language library module and current place window interface, use these sign languages can carry out corresponding automobile at any window and control order.
7. a kind of virtual console gesture of automobile control method of supporting blind operation according to claim 4, it is characterized in that, for having detected the gesture touch-control from the console touch-screen, comprise the steps: to the disposal route of this gesture being called the sign language identification module
71) when the console touch-screen detects finger touch, console touch-screen detection part sends look-at-me with the interruption form to the gesture analysis processing module;
72) after the gesture analysis processing module receives look-at-me, call corresponding interruption and process function;
73) initiation gesture is controlled language incoming timing device;
74) arrive when timer time, then obtain all gesture informations of current console touch-screen;
75) all gesture informations of current console touch-screen carried out analyzing and processing;
76) each gesture information is called respectively the sign language identification module, after it is identified, carry out corresponding operating.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210312798.4A CN102866803B (en) | 2012-08-30 | 2012-08-30 | A kind ofly support the virtual console gesture control method of the automobile of blind operation and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210312798.4A CN102866803B (en) | 2012-08-30 | 2012-08-30 | A kind ofly support the virtual console gesture control method of the automobile of blind operation and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102866803A true CN102866803A (en) | 2013-01-09 |
| CN102866803B CN102866803B (en) | 2016-02-17 |
Family
ID=47445700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210312798.4A Active CN102866803B (en) | 2012-08-30 | 2012-08-30 | A kind ofly support the virtual console gesture control method of the automobile of blind operation and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102866803B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657075A (en) * | 2015-01-29 | 2015-05-27 | 刘波 | Gesture control device and method for automobile electric appliance |
| CN104932808A (en) * | 2014-03-21 | 2015-09-23 | 宏碁股份有限公司 | Touch control display device and operation method therefor |
| CN105117147A (en) * | 2015-07-24 | 2015-12-02 | 上海修源网络科技有限公司 | Method and apparatus for manipulating vehicle-mounted operating system based on gesture and vehicle-mounted device |
| CN105760096A (en) * | 2016-01-04 | 2016-07-13 | 钟林 | Automobile center console direction gesture control method and device supporting blind operation |
| WO2017035792A1 (en) * | 2015-09-01 | 2017-03-09 | 深圳好视网络科技有限公司 | Gesture-based channel changing method and remote control |
| CN106527674A (en) * | 2015-09-14 | 2017-03-22 | 上海羽视澄蓝信息科技有限公司 | Human-computer interaction method, equipment and system for vehicle-mounted monocular camera |
| CN106598349A (en) * | 2017-02-28 | 2017-04-26 | 上海蔚来汽车有限公司 | Vehicle-mounted touch display device and control method thereof |
| CN106891724A (en) * | 2017-01-22 | 2017-06-27 | 斑马信息科技有限公司 | Automobile master control screen system and method |
| WO2017124454A1 (en) * | 2016-01-23 | 2017-07-27 | 曹晟 | Gesture-based method for calling instruction for system, and operating apparatus |
| WO2017124451A1 (en) * | 2016-01-23 | 2017-07-27 | 曹晟 | Gesture-based method for calling instruction for vehicle-mounted system, and operating apparatus |
| US9804776B2 (en) | 2015-04-14 | 2017-10-31 | Hon Hai Precision Industry Co., Ltd. | Window control system and control method thereof |
| CN107958203A (en) * | 2016-10-18 | 2018-04-24 | 普瑞有限公司 | Man-machine interface with steering wheel and the fingerprint sensor being attached on steering wheel |
| US10216358B2 (en) | 2014-01-28 | 2019-02-26 | Acer Incorporated | Touch display apparatus and operating method thereof |
| CN109410691A (en) * | 2018-12-17 | 2019-03-01 | 深圳市中智仿真科技有限公司 | A kind of automobile of gesture control function drives training analog machine |
| CN109669529A (en) * | 2017-10-16 | 2019-04-23 | 上汽通用汽车有限公司 | Vehicle-mounted man-machine interactive system |
| CN110928411A (en) * | 2019-11-18 | 2020-03-27 | 珠海格力电器股份有限公司 | AR-based interaction method and device, storage medium and electronic equipment |
| CN111045590A (en) * | 2019-12-04 | 2020-04-21 | 广州小鹏汽车科技有限公司 | Method and system for vehicle-mounted key simulation, storage medium and vehicle |
| CN111309153A (en) * | 2020-03-25 | 2020-06-19 | 北京百度网讯科技有限公司 | Control method and device for man-machine interaction, electronic equipment and storage medium |
| US10832699B1 (en) | 2019-12-05 | 2020-11-10 | Toyota Motor North America, Inc. | Impact media sharing |
| CN112363665A (en) * | 2020-10-30 | 2021-02-12 | 东风汽车有限公司 | Automobile touch screen control method, electronic equipment and storage medium |
| US11107355B2 (en) | 2019-12-05 | 2021-08-31 | Toyota Motor North America, Inc. | Transport dangerous driving reporting |
| CN113548061A (en) * | 2021-06-25 | 2021-10-26 | 北京百度网讯科技有限公司 | Man-machine interaction method and device, electronic equipment and storage medium |
| US11308800B2 (en) | 2019-12-05 | 2022-04-19 | Toyota Motor North America, Inc. | Transport impact reporting based on sound levels |
| CN114458123A (en) * | 2022-04-12 | 2022-05-10 | 国汽智控(北京)科技有限公司 | Vehicle window control method, device and equipment |
| CN114697448A (en) * | 2022-03-29 | 2022-07-01 | 南京点明软件科技有限公司 | Mobile phone input method applied to blind person operation |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11873000B2 (en) | 2020-02-18 | 2024-01-16 | Toyota Motor North America, Inc. | Gesture detection for transport control |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200619982A (en) * | 2004-08-31 | 2006-06-16 | Ibm | Touch gesture based interface for motor vehicle |
| CN101546233A (en) * | 2009-05-05 | 2009-09-30 | 上海华勤通讯技术有限公司 | Identification and operation method of touch screen interface gestures |
| JP2009301300A (en) * | 2008-06-12 | 2009-12-24 | Tokai Rika Co Ltd | Input device |
| JP2009298285A (en) * | 2008-06-12 | 2009-12-24 | Tokai Rika Co Ltd | Input device |
| CN101854496A (en) * | 2010-04-28 | 2010-10-06 | 青岛海信电器股份有限公司 | Television set and control method and remote controller thereof |
| CN102520876A (en) * | 2012-01-04 | 2012-06-27 | 北京飞漫软件技术有限公司 | Method for operating intelligent touch screen equipment |
-
2012
- 2012-08-30 CN CN201210312798.4A patent/CN102866803B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200619982A (en) * | 2004-08-31 | 2006-06-16 | Ibm | Touch gesture based interface for motor vehicle |
| JP2009301300A (en) * | 2008-06-12 | 2009-12-24 | Tokai Rika Co Ltd | Input device |
| JP2009298285A (en) * | 2008-06-12 | 2009-12-24 | Tokai Rika Co Ltd | Input device |
| CN101546233A (en) * | 2009-05-05 | 2009-09-30 | 上海华勤通讯技术有限公司 | Identification and operation method of touch screen interface gestures |
| CN101854496A (en) * | 2010-04-28 | 2010-10-06 | 青岛海信电器股份有限公司 | Television set and control method and remote controller thereof |
| CN102520876A (en) * | 2012-01-04 | 2012-06-27 | 北京飞漫软件技术有限公司 | Method for operating intelligent touch screen equipment |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10216358B2 (en) | 2014-01-28 | 2019-02-26 | Acer Incorporated | Touch display apparatus and operating method thereof |
| CN104932808B (en) * | 2014-03-21 | 2018-09-11 | 宏碁股份有限公司 | Touch control display apparatus and its operating method |
| CN104932808A (en) * | 2014-03-21 | 2015-09-23 | 宏碁股份有限公司 | Touch control display device and operation method therefor |
| CN104657075A (en) * | 2015-01-29 | 2015-05-27 | 刘波 | Gesture control device and method for automobile electric appliance |
| US9804776B2 (en) | 2015-04-14 | 2017-10-31 | Hon Hai Precision Industry Co., Ltd. | Window control system and control method thereof |
| CN105117147A (en) * | 2015-07-24 | 2015-12-02 | 上海修源网络科技有限公司 | Method and apparatus for manipulating vehicle-mounted operating system based on gesture and vehicle-mounted device |
| WO2017035792A1 (en) * | 2015-09-01 | 2017-03-09 | 深圳好视网络科技有限公司 | Gesture-based channel changing method and remote control |
| CN106527674A (en) * | 2015-09-14 | 2017-03-22 | 上海羽视澄蓝信息科技有限公司 | Human-computer interaction method, equipment and system for vehicle-mounted monocular camera |
| CN105760096A (en) * | 2016-01-04 | 2016-07-13 | 钟林 | Automobile center console direction gesture control method and device supporting blind operation |
| WO2017124454A1 (en) * | 2016-01-23 | 2017-07-27 | 曹晟 | Gesture-based method for calling instruction for system, and operating apparatus |
| WO2017124451A1 (en) * | 2016-01-23 | 2017-07-27 | 曹晟 | Gesture-based method for calling instruction for vehicle-mounted system, and operating apparatus |
| CN107958203B (en) * | 2016-10-18 | 2022-04-05 | 普瑞有限公司 | Human-machine interface with steering wheel and fingerprint sensor attached to steering wheel |
| CN107958203A (en) * | 2016-10-18 | 2018-04-24 | 普瑞有限公司 | Man-machine interface with steering wheel and the fingerprint sensor being attached on steering wheel |
| CN106891724A (en) * | 2017-01-22 | 2017-06-27 | 斑马信息科技有限公司 | Automobile master control screen system and method |
| CN106598349A (en) * | 2017-02-28 | 2017-04-26 | 上海蔚来汽车有限公司 | Vehicle-mounted touch display device and control method thereof |
| CN109669529A (en) * | 2017-10-16 | 2019-04-23 | 上汽通用汽车有限公司 | Vehicle-mounted man-machine interactive system |
| CN109410691A (en) * | 2018-12-17 | 2019-03-01 | 深圳市中智仿真科技有限公司 | A kind of automobile of gesture control function drives training analog machine |
| CN110928411B (en) * | 2019-11-18 | 2021-03-26 | 珠海格力电器股份有限公司 | AR-based interaction method and device, storage medium and electronic equipment |
| CN110928411A (en) * | 2019-11-18 | 2020-03-27 | 珠海格力电器股份有限公司 | AR-based interaction method and device, storage medium and electronic equipment |
| CN111045590A (en) * | 2019-12-04 | 2020-04-21 | 广州小鹏汽车科技有限公司 | Method and system for vehicle-mounted key simulation, storage medium and vehicle |
| US11107355B2 (en) | 2019-12-05 | 2021-08-31 | Toyota Motor North America, Inc. | Transport dangerous driving reporting |
| US10832699B1 (en) | 2019-12-05 | 2020-11-10 | Toyota Motor North America, Inc. | Impact media sharing |
| US11308800B2 (en) | 2019-12-05 | 2022-04-19 | Toyota Motor North America, Inc. | Transport impact reporting based on sound levels |
| US11328737B2 (en) | 2019-12-05 | 2022-05-10 | Toyota Motor North America, Inc. | Impact media sharing |
| CN111309153A (en) * | 2020-03-25 | 2020-06-19 | 北京百度网讯科技有限公司 | Control method and device for man-machine interaction, electronic equipment and storage medium |
| CN111309153B (en) * | 2020-03-25 | 2024-04-09 | 北京百度网讯科技有限公司 | Man-machine interaction control method and device, electronic equipment and storage medium |
| CN112363665A (en) * | 2020-10-30 | 2021-02-12 | 东风汽车有限公司 | Automobile touch screen control method, electronic equipment and storage medium |
| CN112363665B (en) * | 2020-10-30 | 2022-05-27 | 东风汽车有限公司 | Automobile touch screen control method, electronic equipment and storage medium |
| CN113548061A (en) * | 2021-06-25 | 2021-10-26 | 北京百度网讯科技有限公司 | Man-machine interaction method and device, electronic equipment and storage medium |
| CN114697448A (en) * | 2022-03-29 | 2022-07-01 | 南京点明软件科技有限公司 | Mobile phone input method applied to blind person operation |
| CN114458123A (en) * | 2022-04-12 | 2022-05-10 | 国汽智控(北京)科技有限公司 | Vehicle window control method, device and equipment |
| CN114458123B (en) * | 2022-04-12 | 2022-06-14 | 国汽智控(北京)科技有限公司 | Vehicle window control method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102866803B (en) | 2016-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102866803B (en) | A kind ofly support the virtual console gesture control method of the automobile of blind operation and device | |
| US8519960B2 (en) | Method and apparatus for switching of KVM switch ports using gestures on a touch panel | |
| CN103353828B (en) | The method and device of function is write and is wiped in a kind of switching on the touchscreen | |
| US20140191977A1 (en) | Touchpad operational mode | |
| US20150199125A1 (en) | Displaying an application image on two or more displays | |
| CN102855081A (en) | Apparatus and method for providing web browser interface using gesture in device | |
| CN102902471A (en) | Input interface switching method and input interface switching device | |
| US10146432B2 (en) | Method for operating an operator control device of a motor vehicle in different operator control modes, operator control device and motor vehicle | |
| CN102262504A (en) | User interaction gestures with virtual keyboard | |
| CN103970283B (en) | Providing device and method for virtual keyboard operated with two hands | |
| CN103425424B (en) | Handwriting input word selection system and method | |
| CN103186339A (en) | Input method and electronic equipment provided with virtual keyboard | |
| US9361022B2 (en) | Character input apparatus | |
| CN104859561A (en) | Information processing method and device of vehicle-mounted terminal for vehicle | |
| CN105528169A (en) | A touch screen apparatus and a method for operating the same | |
| CN104657075A (en) | Gesture control device and method for automobile electric appliance | |
| CN108762661B (en) | Vehicle interior interaction method and system and vehicle comprising system | |
| EP4137380A1 (en) | Human-computer interaction method and apparatus, and electronic device and storage medium | |
| CN103051695B (en) | A kind of mobile phone that utilizes carries out the method and system of information input to vehicular applications software | |
| CN106681626A (en) | Intelligent vehicular navigation operation method | |
| US20130234997A1 (en) | Input processing apparatus, input processing program, and input processing method | |
| CN104020989A (en) | Control method and system based on remote application | |
| CN103809793A (en) | Information processing method and electronic device | |
| CN106293431A (en) | Handwriting input recognition methods, system and electronic installation | |
| CN107407976B (en) | Operation equipment, motor vehicle and the method for running operation equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |