CN102700479A - Hand judgment method and judgment device, and operation device of vehicle-mounted equipment of motor vehicle - Google Patents
Hand judgment method and judgment device, and operation device of vehicle-mounted equipment of motor vehicle Download PDFInfo
- Publication number
- CN102700479A CN102700479A CN2012100304934A CN201210030493A CN102700479A CN 102700479 A CN102700479 A CN 102700479A CN 2012100304934 A CN2012100304934 A CN 2012100304934A CN 201210030493 A CN201210030493 A CN 201210030493A CN 102700479 A CN102700479 A CN 102700479A
- Authority
- CN
- China
- Prior art keywords
- hand
- unit
- sign
- white
- brightness
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
- B60R16/027—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems between relatively movable parts of the vehicle, e.g. between steering wheel and column
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention provides a hand judgment method and a judgment device, and an operation device of vehicle-mounted equipment of a motor vehicle, thus a hand of an operator and a steering wheel can be reliably two-valued into white and black to enhance judgment precision of the hand. In the first process, making a luminance column diagram of presumed areas (RC, RL, RR) of the steering wheel (S) when a mark (M1) is covered by the hand (H); in a second process, arranging a threshold value for judging white and black between two peak parts of the luminance column diagram; and in a third process, judging the area with the height larger than the threshold as an area of white. Thus, as the luminance of the hand (H) is larger than that of the steering wheel (S), the steering wheel (S) can be reliably two-valued into black and the hand (H) can be two-valued into a proper threshold of white, the white area can be judged as the hand (H), and judgment precision can be greatly enhanced.
Description
Technical field
The present invention relates to a kind of judgment means and the handling device that uses the mobile unit in the self-propelled vehicle of judgment means of this hand of determination methods and hand of hand; The determination methods of this hand and the judgment means of hand are: the image that will utilize image unit that the steering handwheel that is provided with a plurality of signs is taken to obtain turns to white and black according to the brightness two-value, and is hand with white region decision.
Background technology
In the past, and the known gesture distance type handling device that possesses like lower component: CCD (chargecoupleddeice: image unit such as photographic camera charge-coupled device), it reads position, the action of the driver's on the steering handwheel hand; And the operating order unit, it forms and can export the operational order to mobile unit according to position, the action of the hand that is read by this image unit.
For example, in following patent documentation 1, put down in writing gesture based on the operator, be the position of hand and the action of hand) output is to the content of the operational order of mobile unit.
In addition, in following patent documentation 2, put down in writing following content:, thereby avoid the work of gesture remote controller to open by mistake the beginning along with the driver covers the through hole of being located at steering hardware and begins the work of gesture remote controller with hand.
Patent documentation 1: No. 3941786 communique of Japanese Patent
Patent documentation 2: TOHKEMY 2006-312347 communique
Yet; In existing this gesture distance type handling device; According to the image that photographs by image unit; Based on pre-set fixing threshold value, the operator's of the steering handwheel of brightness ratio lower (secretly) and brightness ratio higher (bright) hand two-value is turned to black (steering handwheel) and (hand) in vain, and be hand with the white region decision more than the predetermined area according to the white and black binary image that obtains thus.
Yet; At the dash area of hand entering light, perhaps the operator has under the such situation of dark coloured gloves, for the binary image of the fixing threshold value of existing employing; Exist hand with the operator turn to the steering handwheel two-value black, thereby can't judge the situation of selling reliably.In addition; On the contrary, impinge upon on the steering handwheel and send under the situation of white light in direct sunlight, for the binary image of the fixing threshold value of existing employing; Exist hand with the operator turn to the steering handwheel two-value white, thereby can't judge the situation of selling reliably.
Summary of the invention
The present invention makes in view of above-mentioned situation just, thus its purpose be with operator's hand and steering handwheel reliably two-value turn to the white and black judgement precision that improves hand.
In order to reach above-mentioned purpose; Invention according to the first aspect record; Propose a kind of determination methods of hand, the determination methods of said hand will utilize image unit that the steering handwheel that is provided with sign is taken and the image that obtains turns to whitely and black according to the brightness two-value, and is hand with white region decision; It is characterized in that the determination methods of said hand comprises following operation: first operation of the brightness histogram of the presumptive area of the said steering handwheel when making said sign and being covered by hand; Between two peak portions of said brightness histogram, set second operation of the threshold value be used to judge white and black brightness; And the region decision that the said threshold value of brightness ratio is high is the 3rd operation in white zone.
In addition; According to the invention of second aspect record, on the basis of the structure of first aspect, a kind of determination methods of hand is proposed; It is characterized in that the presumptive area of said steering handwheel is included in hand and covers zone that said when sign cover by hand usually and the zone that is not covered by hand.
In addition; According to the invention of third aspect record, a kind of judgment means of hand is proposed, the judgment means of said hand will utilize image unit that the steering handwheel that is provided with sign is taken and the image that obtains turns to white and black according to the brightness two-value; And be hand with white region decision; It is characterized in that the judgment means of this hand possesses: brightness histogram is made the unit, the brightness histogram of the presumptive area of the said steering handwheel when said brightness histogram is made the unit and made said sign and covered by hand; Peak portion judging unit, two peak portions that said peak portion judgment unit judges goes out said brightness histogram; The threshold value that is used to judge white and black brightness is set in threshold setting unit, said threshold setting unit between said two peak portions; The binary image converting unit, said binary image converting unit is white and black binary image based on said threshold value with said image transitions; And the hand judging unit, said hand judging unit is a hand with the white region decision in the said binary image.
In addition; According to the invention of fourth aspect record, on the basis of the structure of the third aspect, a kind of judgment means of hand is proposed; It is characterized in that the presumptive area of said steering handwheel is included in hand and covers zone that said when sign cover by hand usually and the zone that is not covered by hand.
In addition; Invention according to the record of the 5th aspect; A kind of handling device of mobile unit of self-propelled vehicle is proposed; The handling device of the mobile unit of said self-propelled vehicle adopts the judgment means of the hand of the third aspect or fourth aspect record, it is characterized in that the handling device of the mobile unit of this self-propelled vehicle possesses: judging unit; Said judging unit can be based on the photographed data from said image unit output, and whether any that each sign is judged in a plurality of signs that comprise said sign individually is covered by hand; And the operating order unit, the operational order to mobile unit is exported based on the judged result of said judging unit in said operating order unit.
In addition, the audio frequency apparatus AU of embodiment, video equipment DVD and in-vehicle navigation apparatus CN are corresponding to mobile unit of the present invention, and the camera C A of embodiment is corresponding to image unit of the present invention.
According to the structure of first aspect, in first operation, the brightness histogram of the presumptive area of the steering handwheel when the making sign is covered by hand; In second operation, between two peak portions of brightness histogram, set the threshold value that is used to judge white and black brightness; In the 3rd operation, the region decision that the brightness ratio threshold value is high is white zone.Thus,, just can set can be reliably the steering handwheel two-value be turned to and deceive and the hand two-value is turned to white suitable threshold, and be hand, thereby can improve the judgement precision significantly white region decision as long as the brightness of hand is brighter relatively with respect to the brightness of steering handwheel.
In addition; Structure according to second aspect; The presumptive area of steering handwheel is included in hand and covers zone that when sign cover by hand usually and the zone that is not covered by hand, therefore, in brightness histogram, can form two peak portions reliably; And set threshold value accurately, thereby can further improve the judgement precision of hand according to this brightness histogram.
In addition; Structure according to the third aspect; The brightness histogram of the presumptive area of the steering handwheel when brightness histogram making unit making sign is covered by hand, peak portion judgment unit judges goes out two peak portions of brightness histogram, and the threshold value that is used to judge white and black brightness is set in the threshold setting unit between two peak portions; The binary image converting unit is white and black binary image based on threshold value with image transitions, and the hand judging unit is a hand with the white region decision in the binary image.Thus,, just can set can be reliably the steering handwheel two-value be turned to and deceive and the hand two-value is turned to white suitable threshold, and be hand, thereby can improve the judgement precision significantly white region decision as long as the brightness of hand is brighter relatively with respect to the brightness of steering handwheel.
In addition; Structure according to fourth aspect; The presumptive area of steering handwheel is included in hand and covers zone that when sign cover by hand usually and the zone that is not covered by hand, therefore, in brightness histogram, can form two peak portions reliably; And set threshold value accurately, thereby can further improve the judgement precision of hand according to this brightness histogram.
In addition; Structure according to the 5th aspect; Judging unit is based on the photographed data from image unit output, and whether any that each sign is judged in a plurality of signs that comprise sign individually is covered by hand, and the operating order unit is based on the operational order of this judged result output to mobile unit; Therefore; Not only can be based in the image unit situation whether each sign covered by operator's hand and export operational order exactly, and tell operator's operation order easily, can also prevent the maloperation that the identification error by the operator produces effectively mobile unit.And the decision logic in the operating order unit is fairly simple, not only help cost and reduce, and calculation process can promptly be carried out.
Description of drawings
Fig. 1 is the synoptic map that illustrates in the compartment of self-propelled vehicle.
Fig. 2 is near the front view driver's side viewing console.
Fig. 3 is the amplification view along the 3-3 line of Fig. 2.
Fig. 4 is the block diagram of control system.
Fig. 5 illustrates the operating order unit briefly from the instruction diagram of standby mode to the blocked operation process of first mode of operation switching.
Fig. 6 illustrates the operating order unit briefly from the instruction diagram of first mode of operation to the blocked operation process of second mode of operation switching.
Even thereby Fig. 7 illustrates to be used for removing driving process to the restriction of the operation of the mobile unit instruction diagram that also can carry out manually operated operating process to mobile unit of the co-driver beyond the driver in the process of moving briefly.
Fig. 8 is the block diagram that the logic of judging hand is shown.
Fig. 9 is the figure that an example of brightness histogram is shown.
Figure 10 illustrates to be used to carry out the substep synoptic map that volume is regulated the control step of operation.
Figure 11 is the substep synoptic map that the control step that is used to carry out the broadcast source selection operation is shown.
Figure 12 is the substep synoptic map that the control step of the selection operation that is used to carry out album name, bent name is shown.
Figure 13 illustrates the substep synoptic map that is used to carry out to the control step of the branching operation of sound retrieval mode shifts.
Label declaration
AU: audio frequency apparatus (mobile unit);
C1: judging unit;
C2: operating order unit;
CA: photographic camera (image unit);
CN: in-vehicle navigation apparatus (mobile unit);
DVD: video equipment (mobile unit);
H: hand;
M1: sign;
M1～M7: a plurality of signs;
P1: brightness histogram is made the unit;
P2: peak portion judging unit;
P3: threshold setting unit;
P4: binary image converting unit;
P5: hand judging unit;
S: steering handwheel.
The specific embodiment
Below, based on Fig. 1～Figure 13 embodiment of the present invention is described.
At first; In Fig. 1～Fig. 3; On the instrument carrier panel I in compartment; Except various metering devices such as speed gauge, also be provided with used for vehicle-carrying equipment such as audio frequency apparatus AU, video equipment DVD, in-vehicle navigation apparatus CN display part L, be used for the operating portion 1 of these mobile units of M/C AU, DVD, CN etc., also the face side at said various metering devices is equipped with the steering handwheel S that is carried out steering operation by the operator.
Steering handwheel S possesses: the Sa of central hub portion, and itself and steering shaft (not shown) link; The Sm of wheel rim portion, it is circularly around the Sa of this central hub portion; And three disc portion Sb1～Sb3, they are with linking up between this Sm of wheel rim portion and the Sa of central hub portion.Horn cover 2 adheres to the outside face of the Sa of central hub portion with the mode that can load and unload, is equipped with the first sign M1 of the H word shape that designing treatment crosses at the central outside face of this horn cover 2.
In addition, be equipped with the spoke cover 3 of the outside face that covers them with the mode that can load and unload at said disc portion Sb1～Sb3 attractive in appearancely.This spoke cover 3 becomes with the cooresponding expansion shape of disc portion Sb1～Sb3 through the metal plate shape that will have flexible sheet in illustrated example and constitutes, and is provided with the lining cement or the adhesive sheet of pasting usefulness at the back side of this spoke cover 3.When this spoke cover 3 being installed to disc portion Sb1～Sb3; The outside face of disc portion Sb1～Sb3 is rolled and be pasted on to this cover 3 with the mode of the outside face of copying disc portion Sb1～Sb3, and will cover 3 free ora terminalis 3e and bend to the inside as illustrated in fig. 3 and be fixed in disc portion Sb1～Sb3.
In said spoke cover 3 and the cooresponding part of the disc portion Sb1 left side, respectively through engaging or printing has set up that the second sign M2 of delegation～(in illustrated example, M2 is equivalent to the volume sign to 4th sign M4; M3 is equivalent to the broadcast source sign, and M4 is equivalent to selection marker), in addition; In spoke cover 3 and the cooresponding part of the disc portion Sb3 right side; (in illustrated example, M5 is equivalent to the speech sign, and M6 is equivalent to upwards sign to have set up the 5th sign M5～the 7th sign M7 through bonding or printing respectively; M7 is equivalent to downward sign), and they are arranged in the serration in the inner part with the 5th sign M5.
Appropriate location in the compartment for leaning on driver's the slightly tiltedly top wall portion at rear, is provided with as taking the camera C A of the monochrome of the image unit that respectively indicates M1～M7 in illustrated example.This camera C A can identify the photographed data of said situation about covering clearly to electronic control package ECU (with reference to figure 4) output when the driver covers arbitrary sign M1～M7.
The color of preferred steering handwheel S is the dead color system near black; And the preferred first sign M1～the 7th sign M7 surrounds the unregistered land youngster by white edge to form, so that can easily identify the first sign M1～the 7th sign M7 and driver's hand H according to the photographed data of camera C A.
As shown in Figure 4; Electronic control package ECU is located at the appropriate location in the compartment, the inside of for example instrument carrier panel I; This electronic control package ECU possesses: judging unit C1; This judging unit C1 can be to being analyzed by the photographed data of camera C A output, and based on this analysis data pin to each indicate M1～M7 individually judgement symbol M1～M7 whether covered; Operating order unit C2, this operating order unit C2 exports the operation instruction signal of conduct to the operational order of mobile unit (being audio frequency apparatus AU, video equipment DVD, in-vehicle navigation apparatus CN) based on the judged result of said judging unit C1; The C3 of signal output portion, the C3 of this signal output portion converts into the output signal of said operating order unit C2 to the drive signal of said mobile unit and exports mobile unit AU, DVD, CN to; And diagnosis unit C4, these diagnosis unit C4 can identify occupant's hand H based on the photographed data by camera C A output.
Said camera C A is connected in judging unit C1 and diagnosis unit C4 input side separately.In addition, the input side of C2 is connected with judging unit C1 and diagnosis unit C4 outgoing side and exterior said operating portion 1, rotation angle sensor SE1 and car speed sensor SE2 separately respectively in the operating order unit.In addition, the outgoing side of operating order unit C2 is connected with the input side of the C3 of signal output portion, and is connected with said mobile unit AU, DVD, CN at the outgoing side of the C3 of this signal output portion.
Operating order unit C2 can optionally take said exterior operating portion 1 to carry out the standby mode of input operation and to forbid any pattern in first, second mode of operation of this input operation mobile unit AU, DVD, CN; Under said standby mode; Except the first sign M1 and the sign (promptly second sign M2～four sign M4, six sign M6 and seven indicate M7) of the 5th sign the M5 even quilt is covered; Operating order unit C2 does not respond yet, and does not promptly send operational order.
In addition; When operating order unit C2 is in first or second mode of operation, this operating order unit C2 along with vehicle reach pre-set state (in illustrated example for through the state of the operation of the rudder angle more than the predetermined angular or in the schedule time speed of a motor vehicle take place state jumpy, or arbitrary mode of operation in switching to first, second mode of operation after shelve the operation that does not cover the arbitrary sign among the second sign M2～7th sign M7 more than the certain hour state, or no matter be in first mode of operation or be in the state that has carried out covering once more the first sign M1 in second mode of operation) automatically switch to said standby mode.
As shown in Figure 8, diagnosis unit C4 possesses brightness histogram and makes unit P1, the judging unit P2 of peak portion, threshold setting unit P3, binary image converting unit P4 and hand judging unit P5.Camera C A is connected in brightness histogram and makes unit P1 and binary image converting unit P4; Brightness histogram is made unit P1 and is connected in the judging unit P2 of peak portion; The judging unit P2 of peak portion is connected in threshold setting unit P3; Threshold setting unit P3 is connected in binary image converting unit P4, and binary image converting unit P4 is connected in hand judging unit P5.
As shown in Figure 2, be set with the first area RC of central authorities and second area RL and the 3rd region R R that is positioned at the RC left and right sides, first area at steering handwheel S.First area RC and second area RL and the 3rd region R R are the zone of virtual property, and first area RC is set at the position that covers the first sign M1, and second area RL and the 3rd region R R are set at the position of leaving preset distance with respect to first area RC.In other words, first area RC is set at the position that is covered when the driver will cover the first sign M1 with hand, and second area RL and the 3rd region R R are set at the position that is difficult to cover when the driver will cover the first sign M1 with hand.
Get back to Fig. 8; The brightness histogram of diagnosis unit C4 is made unit P1 and from the image of camera C A, is extracted and said first area RC, second area RL and the cooresponding pixel of the 3rd region R R; And they are arranged according to brightness, thereby produce brightness histogram shown in Figure 9.When the driver will cover the first sign M1 with hand; The major part of first area RC is covered by the hand of light tone; Therefore the brightness of its pixel is in higher trend; The major part of second area RL and the 3rd region R R is not covered by the hand of light tone, and the horn cover 2 of dark coloured steering handwheel S exposes, so its brightness is in than low tendency.Consequently, in brightness histogram, with the higher zone of the cooresponding brightness of the pixel of first area RC a peak portion is appearring, and, another peak portion appears in the lower zone of the cooresponding brightness of pixel with second area RL, the 3rd region R R.
The judging unit P2 of peak portion judges said two peak portions, and threshold setting unit P3 sets the threshold value of the brightness that is used for binaryzation between said two peak portions.This threshold value is the value with variations such as the environment in compartment; For example under the darker situation in compartment, the brightness globality of the pixel of first～the 3rd region R C, RL, RR ground reduces, so threshold value is to lower (darker) side shifting; Under the brighter situation in compartment; The brightness globality ground of the pixel of first～the 3rd region R C, RL, RR raises, so threshold value is to higher (brighter) side shifting
Whole pixel regions of the image that binary image converting unit P4 will be photographed by camera C A according to said threshold value separately; Will the pixel two-value brighter than threshold value turn to white and will the pixel two-value darker than threshold value turn to black, thereby with the image transitions of the camera C A binary image that white and black these two colors constitute of serving as reasons.And hand judging unit P5 is judged as part in the white zone of binary image, that surround with the area more than the predetermined value driver's hand H.
Like this; Utilize brightness histogram to make unit P1 and be produced on first area RC and second, third region R L of the steering handwheel S of the first sign M1 when being covered, the brightness histogram of RR by hand H; Utilize the judging unit P2 of peak portion to judge two peak portions of brightness histogram; Utilize threshold setting unit P3 between two peak portions, to set the threshold value that is used to judge white and black brightness, utilizing binary image converting unit P4 is white and black binary image based on threshold value with image transitions, utilizes hand judging unit P5 that the white region decision more than the predetermined area in the binary image is hand; Therefore; As long as the brightness of hand H is higher relatively with respect to the brightness of steering handwheel S, just can set can be reliably steering handwheel S two-value be turned to and deceive and hand H two-value is turned to white suitable threshold, can improve the judgement precision significantly.
And C2 is under the state of standby mode in the operating order unit, and judging unit C1 is based on the photographed data of camera C A; The profile (circle) that shown in Fig. 5 (A), identifies steering wheel main body Sm is with as the monitored object scope; Then shown in Fig. 5 (B), identify the first sign M1 of central authorities, therefore, in this identifying; As Fig. 5 (C) the driver that is shown in hand H cover first the sign M1 situation under; Judging unit C1 can identify this situation rapidly reliably, and through this recognition result being outputed to operating order unit C2, thereby this operating order unit C2 switches to first mode of operation.
In addition; C2 is under the state of first or second mode of operation in the operating order unit; Judging unit C1 identifies the second sign M2～the 7th sign M7 of disc portion Sb1, Sb2 shown in Fig. 6 (A); Therefore, in this identifying, for example as Fig. 6 (B) the driver that is shown in cover under a plurality of second signs situation of (promptly the second sign M2～5th indicates M5) with hand H; Judging unit C1 can identify this situation rapidly reliably, and operating order unit C2 switches to second mode of operation based on this recognition result.And then; Under the state of second mode of operation; Judging unit C1 can identify shown in Fig. 6 (C) such driver rapidly reliably and cover a plurality of the 3rd signs situation of (promptly the 6th sign M6, the 7th indicates M7) with hand H, and this recognition result is outputed to operating order unit C2.
And then no matter which kind of pattern operating order unit C2 is in, and diagnosis unit C4 is based on the photographed data of camera C A, and the profile that shown in Fig. 7 (A), identifies steering wheel main body Sm is with as the monitored object scope.And; Operating order unit C2 constitutes: C2 is judged as self-propelled vehicle based on the detection signal of car speed sensor SE2 and is in and goes when middle in the operating order unit; Go out on the Sm of wheel rim portion, to have under the situation of two hand H in that diagnosis unit C4 is unidentified; It is invalid that operating order unit C2 makes from least a portion operation signal of said operating portion 1; Have under the situation of two hand H on the Sm of wheel rim portion and identify shown in Fig. 7 (B) at diagnosis unit C4, operating order unit C2 does not make said operation signal invalid.
Therefore, same with in the past in the process of moving, from the viewpoint of safe driving, it is invalid to make manually operated at least a portion of the operating portion 1 of mobile unit AU, DVD, CN.Yet; Even if in the process of moving, recognizing on the Sm of wheel rim portion at diagnosis unit C4 has under the situation of two hand H, even exported operation signal from said operating portion 1; Also can be seen as by the operation of the occupant beyond the driver; So operating order unit C2 does not make said operation signal invalid, thereby make the occupant's beyond the driver efficient in operation.Thus, even self-propelled vehicle is in the process of moving, as long as the driver holds steering handwheel S with two hand H, the occupant beyond the driver just can operate mobile unit AU, DVD, CN, has improved easy-to-use, comfort feature.
When C2 is in standby mode in the operating order unit; Operating order unit C2 is judged as the first sign M1 situation about being covered and switches to first mode of operation based on judging unit C1; Under this first mode of operation, can come at random to select the operand in the mobile unit (in illustrated example, to be " speech " of the sound retrieval pattern of " volume ", " broadcast source " of selecting video and audio frequency, " selections " of selecting album name and bent name, selection in-vehicle navigation apparatus through covering among the second sign M2～5th sign M5 one.
In addition; When C2 is in first mode of operation in the operating order unit; Operating order unit C2 is judged as second sign M2～5 based on judging unit C1 and indicates the situation that any quilt among the M5 covers and switch to second mode of operation; Under this second mode of operation, can be through covering one that among the M7 one of the 6th sign M6 and the 7th sign selects in a plurality of operating parameters relevant with said operand.Said operating parameter is meant the content of operation that should more specifically carry out selection operation about said operand operator; In illustrated example; Selecting under the situation of " volume " through covering the second sign M2; Cover the 6th sign M6 corresponding to " volume rising ", in addition, cover the 7th sign M7 corresponding to " volume reduction ".
In addition,, cover the 6th sign M6, in addition, cover the 7th sign M7 corresponding to " other direction rolls successively " corresponding to " direction is rolled successively " of closing video/audio having selected under the situation of " broadcast source " through covering the 3rd sign M3.
And then; Selecting under the situation of " selection " through covering the 4th sign M4; Cover " direction successively roll " of the 6th sign M6 for a long time (for example 2 seconds), in addition, cover to the short time " direction successively roll " of the 6th sign M6 corresponding to bent name corresponding to album name; In addition; Cover " other direction successively roll " of the 7th sign M7 for a long time (for example 2 seconds), in addition, cover to the short time " other direction successively roll " of the 7th sign M7 corresponding to bent name corresponding to album name.
Then, with reference to Figure 10～Figure 13, the control step of said handling device is described more particularly.
[1] volume adjustment operation
When C2 is in said standby mode in the operating order unit; Shown in the step S10 of Figure 10; Display part L is common in-vehicle navigation apparatus picture; From this state, when the operator covers first sign during M1 (step S11), the prompt tone of " please operate " occurs and emphatically occur next answering the demonstration (step S12) of the sign M2～M5 of selection operation in picture the latter half.So; When covering the second sign M2 (step S13); The prompt tone of " volume " occurs and the sign M6 of selection operation, the demonstration (step S14) of M7 emphatically occur next answering in picture the latter half; Then, current volume occurs and show (step S15) through after the schedule time (for example 3 seconds).So, when covering among the M7 any one of the 6th sign M6 and the 7th sign (step S16), carry out the volume adjustment accordingly with the sign that is covered, this adjusted volume demonstration (step S17) appears.
[2] broadcast source selection operation
As illustrated in fig. 11; Step S20～S22 is identical with said step S10～S12; From the state of this step S22, when covering the 3rd sign M3 (step S23), the prompt tone of " selection broadcast source " occurs and the sign M6 of selection operation, the demonstration (step S24) of M7 emphatically occur next answering in picture the latter half; Then, current broadcast source occurs and show (selecting to be shown as " closing " in the drawings) (step S25) through after the schedule time (for example 3 seconds).So; When covering among the M7 any one of the 6th sign M6 and the 7th sign (step S26); Carry out the direction of closing video/audio or the rolling successively of other direction accordingly with the sign that is covered, and the broadcast source demonstration (step S27) after the prompt tone of " broadcast source selects 0000 " occurring and this selections occurring.
[3] selection operation of album name, bent name
As illustrated in fig. 12; Step S30～S32 is identical with said step S10～S12; From the state of this step S32, the prompt tone of " selecting special edition masterpiece in music name " when covering the 4th sign M4 (step S33), occurs and the sign M6 of selection operation, the demonstration (step S34) of M7 emphatically occur next answering in picture the latter half; Through after the schedule time (for example 3 seconds), current album name, bent name broadcast source demonstration (step S35) appear then.So; Cover any one words among the 6th sign M6 and the 7th sign M7 for a long time (for example two seconds); A direction of execution album name is rolled successively or other direction rolls (step S36) successively, in addition, covers to the short time any one words among the 6th sign M6 and the 7th sign M7; A direction of the bent name of execution is rolled successively or other direction rolls (step S36) successively, the demonstration (step S37) of the album name that in said execution, occurs reselecting, bent name.After occurring showing, shelve the schedule time (for example 5 seconds) after, the album name of reselecting, bent name have just been confirmed.
[4] transfer to the branching operation of sound retrieval pattern
Shown in figure 13; Step S40～S42 is identical with said step S10～S12; As operator (step S43) when the state of this step S42 covers the 5th sign M5, the prompt tone of " sound retrieval " occurs and can make in-vehicle navigation apparatus CN be transferred to existing known sound retrieval pattern.
This embodiment according to above explanation; Can be owing to have based on photographed data by camera C A output; Each sign M1～M7 is judged judging unit C1 that whether a plurality of sign M1～M7 are covered respectively and based on the judged result output of this judging unit C1 operating order unit C2 to the operational order (operation instruction signal) of mobile unit AU, DVD, CN; Therefore; Not only can be based among the camera C A situation whether each sign M1～M7 covered by operator's hand H and export operational order exactly mobile unit AU, DVD, CN; And tell operator's control step easily, can also prevent maloperation effectively by operator's identification error generation.
And the decision logic among this operating order unit C2 is fairly simple, has not only realized the cost reduction, and calculation process also can promptly be carried out.In addition; Said a plurality of sign M1～M7 is located at and is mounted parts (horn cover 2, spoke cover 3 for loading and unloading with respect to steering handwheel S in illustrated example) in the compartment in; Need not expressly to run through the through hole that usefulness is read in setting at these covers 2,3 and steering handwheel S; Not only follow-up selection is corresponding to being easier to; And the degree of freedom that the position is set that respectively indicates M1～M7 and camera C A also becomes than higher, and then the driver who grips steering handwheel S can cover the operation of sign M1～M7 easily and promptly through the action of hand.
In addition; The operator only just can switch to first mode of operation from standby mode rapidly and reliably with operating order unit C2 through covering the first sign M; And then under this first mode of operation, only just can be rapidly and select the operand among mobile unit AU, DVD, the CN reliably through covering among a plurality of sign M2～M5 one.In addition; When C2 is in said second mode of operation in the operating order unit; The operator is only through covering among the M7 one of the 6th sign M6 and the 7th sign, just can be rapidly and reliably selection a plurality of operating parameters relevant with operand, should appointment be in the content of operation.
In addition; When operating order unit C2 is in any one mode of operation, this operating order unit C2 all can along with vehicle reach pre-set state (in illustrated example for through the state of the operation of the rudder angle more than the predetermined angular or in the schedule time speed of a motor vehicle take place state jumpy, or arbitrary mode of operation in switching to first, second mode of operation after shelve the state that do not cover the second sign M2～7th sign M7 more than the certain hour, or no matter be in first mode of operation or be in the state that has carried out covering once more the first sign M1 in second mode of operation) automatically switch to standby mode.Thus; Should be absorbed in when curve driving, during brish acceleration etc. under the special situation such as motoring condition of driving, shelving under the situation more than the certain hour and then hope to turn back at once under the situation of standby mode the driver; Can make operating order unit C2 automatically turn back to standby mode, therefore can be with because of operator's maloperation the situation of operating order unit C2 output command being prevented trouble before it happens reliably in these cases.
More than, embodiment of the present invention has been described, yet the present invention being not limited to above-mentioned embodiment, the present invention ground that can not break away from claims record carries out various design modifications.
For example, in embodiment, be set at these three zones of first area RC and second area RL and the 3rd region R R and leave each other, make them become a continuous zone yet also can be set at.The setting means in zone is arbitrarily, and a part must comprise the first sign M1 but need make wherein.
In addition; In embodiment, show situation about can operate audio frequency apparatus AU, video equipment DVD, in-vehicle navigation apparatus CN etc. through handling device of the present invention, but in the present invention; Operand also can be vehicle-borne other equipment, for example interior environment system, TV etc. such as air-conditioning.
In addition; In embodiment; Show operating order unit C2 and the judged result that sign M1～M7 is covered is exported the situation to the operational order (operation instruction signal) of mobile unit AU, DVD, CN based on judging unit C1; Yet also can be in the present invention, based on the decision logic of judging unit C1 after to the result combinations of the identification of the sign judged result that covered of M1～M7 and diagnosis unit C4 adversary H exported the operational order (operation instruction signal) to mobile unit AU, DVD, CN.In this case, diagnosis unit C4 only is identified as the body part (head, arm etc.) beyond the hand H with hand H, therefore can as prior art, not make decision logic complicated.
In addition, in embodiment, the white portion more than the predetermined area being judged as hand, is that situation more than one is as the Rule of judgment of hand but also can the center from white portion be radially the projection of extending.
Claims (5)
1. the determination methods of a hand, the determination methods of said hand will utilize image unit (CA) that the steering handwheel (S) that is provided with sign (M1) is taken and the image that obtains turns to whitely and black according to the brightness two-value, and is hand (H) with white region decision, it is characterized in that,
The determination methods of said hand comprises following operation:
Make said sign (M1) by the operation of the brightness histogram of the presumptive area of the said steering handwheel (S) of hand (H) when covering;
Between two peak portions of said brightness histogram, set the operation of the threshold value be used to judge white and black brightness; And
The region decision that the said threshold value of brightness ratio is high is the operation in white zone.
2. the determination methods of hand according to claim 1 is characterized in that,
Usually zone that is covered by hand (H) when the presumptive area of said steering handwheel (S) is included in and covers said sign (M1) with hand (H) and the zone that is not covered by hand (H).
3. the judgment means of a hand, the judgment means of said hand will utilize image unit (CA) that the steering handwheel (S) that is provided with sign (M1) is taken and the image that obtains turns to whitely and black according to the brightness two-value, and is hand (H) with white region decision, it is characterized in that,
The judgment means of this hand possesses:
Brightness histogram is made unit (P1), and said brightness histogram is made unit (P1) and made said sign (M1) by the brightness histogram of the presumptive area of the said steering handwheel (S) of hand (H) when covering;
Peak portion judging unit (P2), said peak portion's judging unit (P2) are judged two peak portions of said brightness histogram;
The threshold value that is used to judge white and black brightness is set in threshold setting unit (P3), said threshold setting unit (P3) between said two peak portions;
Binary image converting unit (P4), said binary image converting unit (P4) is white and black binary image based on said threshold value with said image transitions; And
Hand judging unit (P5), said hand judging unit (P5) is hand (H) with the white region decision in the said binary image.
4. the judgment means of hand according to claim 3 is characterized in that,
Usually zone that is covered by hand (H) when the presumptive area of said steering handwheel (S) is included in and covers said sign (M1) with hand (H) and the zone that is not covered by hand (H).
5. the handling device of the mobile unit of a self-propelled vehicle, the handling device of the mobile unit of said self-propelled vehicle adopts the judgment means of claim 3 or the described hand of claim 4, and the handling device of the mobile unit of said self-propelled vehicle is characterised in that,
The handling device of the mobile unit of this self-propelled vehicle possesses:
Judging unit (C1); Said judging unit (C1) can be based on the photographed data from said image unit (CA) output, and (M1～M7) judges a plurality of signs of comprising said sign (M1) individually, and (whether any among the M1～M7) is covered by hand (H) to each sign; And
Operating order unit (C2), the operational order to mobile unit (AU, DVD, CN) is exported based on the judged result of said judging unit (C1) in said operating order unit (C2).
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2011069627A JP5330444B2 (en) | 2011-03-28 | 2011-03-28 | Hand determination method and determination device in an in-vehicle device operating device of an automobile |
| JP2011-069627 | 2011-03-28 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102700479A true CN102700479A (en) | 2012-10-03 |
| CN102700479B CN102700479B (en) | 2015-09-02 |
Family
ID=46893655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210030493.4A Expired - Fee Related CN102700479B (en) | 2011-03-28 | 2012-02-10 | The determination methods of hand and judgment means |
Country Status (4)
| Country | Link |
|---|---|
| JP (1) | JP5330444B2 (en) |
| KR (1) | KR101293313B1 (en) |
| CN (1) | CN102700479B (en) |
| TW (1) | TW201239676A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104267802A (en) * | 2014-08-29 | 2015-01-07 | 福州瑞芯微电子有限公司 | Human-computer interactive virtual touch device, system and method |
| CN105739679A (en) * | 2014-12-31 | 2016-07-06 | 哈曼国际工业有限公司 | Steering wheel control system |
| CN106061817A (en) * | 2014-02-18 | 2016-10-26 | 捷豹路虎有限公司 | Autonomous driving system and method for same |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5548643B2 (en) * | 2011-03-28 | 2014-07-16 | 株式会社ホンダアクセス | Operation device for in-vehicle equipment in automobile |
| JP5593339B2 (en) | 2012-02-07 | 2014-09-24 | 日本システムウエア株式会社 | Gesture recognition device using a steering wheel of an automobile, hand recognition method and program thereof |
| CN103777741B (en) * | 2012-10-19 | 2017-08-01 | 原相科技股份有限公司 | The gesture identification and system followed the trail of based on object |
| CN102982655B (en) * | 2012-12-06 | 2014-10-22 | 南京邮电大学 | Early warning method of cell phone platform for car driving fatigue |
| GB2541514B (en) * | 2014-02-18 | 2018-01-10 | Jaguar Land Rover Ltd | Autonomous driving system and method for same |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6873731B2 (en) * | 2000-02-09 | 2005-03-29 | Fujitsu Limited | Image processing method and image processing device |
| CN1661621A (en) * | 2004-02-26 | 2005-08-31 | 三菱扶桑卡客车公司 | Gesture switching gear |
| JP2005321948A (en) * | 2004-05-07 | 2005-11-17 | Fujitsu Ten Ltd | Operation input device |
| CN101764922A (en) * | 2009-08-03 | 2010-06-30 | 北京智安邦科技有限公司 | Method and device for adaptive generation of luminance threshold |
| CN101370086B (en) * | 2007-08-17 | 2011-02-16 | 索尼株式会社 | Image forming device and method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4264601B2 (en) * | 1998-10-21 | 2009-05-20 | ソニー株式会社 | Information input device, information input method, and game device |
| JP3624768B2 (en) * | 1999-12-24 | 2005-03-02 | 株式会社豊田自動織機 | Parallel parking assistance device |
| JP2005301669A (en) * | 2004-04-12 | 2005-10-27 | Seiko Epson Corp | Information processor and information processing program |
| JP5228439B2 (en) * | 2007-10-22 | 2013-07-03 | 三菱電機株式会社 | Operation input device |
-
2011
- 2011-03-28 JP JP2011069627A patent/JP5330444B2/en not_active Expired - Fee Related
- 2011-07-14 TW TW100125006A patent/TW201239676A/en unknown
- 2011-08-02 KR KR1020110077016A patent/KR101293313B1/en not_active IP Right Cessation
-
2012
- 2012-02-10 CN CN201210030493.4A patent/CN102700479B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6873731B2 (en) * | 2000-02-09 | 2005-03-29 | Fujitsu Limited | Image processing method and image processing device |
| CN1661621A (en) * | 2004-02-26 | 2005-08-31 | 三菱扶桑卡客车公司 | Gesture switching gear |
| JP2005321948A (en) * | 2004-05-07 | 2005-11-17 | Fujitsu Ten Ltd | Operation input device |
| CN101370086B (en) * | 2007-08-17 | 2011-02-16 | 索尼株式会社 | Image forming device and method |
| CN101764922A (en) * | 2009-08-03 | 2010-06-30 | 北京智安邦科技有限公司 | Method and device for adaptive generation of luminance threshold |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106061817A (en) * | 2014-02-18 | 2016-10-26 | 捷豹路虎有限公司 | Autonomous driving system and method for same |
| CN104267802A (en) * | 2014-08-29 | 2015-01-07 | 福州瑞芯微电子有限公司 | Human-computer interactive virtual touch device, system and method |
| CN105739679A (en) * | 2014-12-31 | 2016-07-06 | 哈曼国际工业有限公司 | Steering wheel control system |
| CN105739679B (en) * | 2014-12-31 | 2021-02-19 | 哈曼国际工业有限公司 | Steering wheel control system |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20120109964A (en) | 2012-10-09 |
| TW201239676A (en) | 2012-10-01 |
| CN102700479B (en) | 2015-09-02 |
| JP5330444B2 (en) | 2013-10-30 |
| KR101293313B1 (en) | 2013-08-06 |
| JP2012201301A (en) | 2012-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102700479A (en) | Hand judgment method and judgment device, and operation device of vehicle-mounted equipment of motor vehicle | |
| CN102107630B (en) | Operation apparatus for on-board devices in automobile | |
| EP3556612B1 (en) | Terminal and method for controlling terminal | |
| EP3093210B1 (en) | In-vehicle apparatus and vehicle | |
| CN101432166B (en) | Operating device | |
| CN101676152B (en) | Vehicle surroundings monitoring apparatus | |
| US10484651B2 (en) | Image display control device | |
| CN109969189A (en) | Vehicle | |
| US6223103B1 (en) | Driver display with highlighted images | |
| CN115180019B (en) | Control method and device for display screen on steering wheel, medium and vehicle | |
| US20220089029A1 (en) | Infotainment device for a vehicle | |
| JPH10264739A (en) | Monitor for vehicle | |
| JP5342588B2 (en) | Operation device for in-vehicle equipment in automobile | |
| JP5681542B2 (en) | Operation device for in-vehicle equipment in automobile | |
| JP2008044475A (en) | Vehicular periphery monitoring device | |
| JP5681543B2 (en) | Operation device for in-vehicle equipment in automobile | |
| JPH11325234A (en) | Control device for vehicle | |
| JP5450492B2 (en) | Operation device for in-vehicle equipment in automobile | |
| JP5484384B2 (en) | Operation device for in-vehicle equipment in automobile | |
| US9600097B2 (en) | On-vehicle device operation apparatus and on-vehicle device operation method | |
| CN114877064B (en) | Vehicle gear shifting method and device and vehicle | |
| JP5450495B2 (en) | Hand determination method, hand determination device, and operation device for in-vehicle device in automobile | |
| JP2012201306A (en) | Operating device for in-vehicle equipment of automobile | |
| JP2012201303A (en) | Operating device for in-vehicle equipment of automobile | |
| JP5548643B2 (en) | Operation device for in-vehicle equipment in automobile |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150902 Termination date: 20170210 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |