JP5330444B2 - Hand determination method and determination device in an in-vehicle device operating device of an automobile - Google Patents

Abstract

The present invention provides a hand judgment method and a judgment device, and an operation device of vehicle-mounted equipment of a motor vehicle, thus a hand of an operator and a steering wheel can be reliably two-valued into white and black to enhance judgment precision of the hand. In the first process, making a luminance column diagram of presumed areas (RC, RL, RR) of the steering wheel (S) when a mark (M1) is covered by the hand (H); in a second process, arranging a threshold value for judging white and black between two peak parts of the luminance column diagram; and in a third process, judging the area with the height larger than the threshold as an area of white. Thus, as the luminance of the hand (H) is larger than that of the steering wheel (S), the steering wheel (S) can be reliably two-valued into black and the hand (H) can be two-valued into a proper threshold of white, the white area can be judged as the hand (H), and judgment precision can be greatly enhanced.

Description

The present invention relates to an in-vehicle device operating apparatus for an automobile, in which an image obtained by imaging a steering handle provided with a plurality of marks is binarized into white and black by brightness and a white area is determined to be a hand . It relates to the determination equipment determination method and hand hand.

  Conventionally, an image pickup means such as a CCD camera for reading the position and movement of an occupant's hand on the steering handle, and an operation command that can output an operation command for on-vehicle equipment based on the position and movement of the hand read by the image pickup means There is known a gesture remote control type operating device provided with means.

  For example, Patent Literature 1 below describes an operator's gesture, that is, an operation command that outputs an operation command for an in-vehicle device based on a hand position or a hand movement.

  Further, in Patent Document 2 below, the operation of the gesture remote controller is prevented from being erroneously started by starting the operation of the gesture remote controller in response to the through hole provided in the steering wheel being blocked by the driver's hand. Is described.

Japanese Patent No. 3914786 JP 2006-31347 A

  By the way, in this type of conventional gesture remote control type operating device, a steering handle having a relatively low brightness (dark) and an operator's hand having a relatively high brightness (bright) are used based on an image captured by the imaging means. Then, binarization into black (steering handle) and white (hand) based on a fixed threshold value set in advance, and a white area of a predetermined area or more is handed out from the white / black binarized image obtained thereby. It was determined that it was.

  However, when the hand enters a shadowed part of the light or the operator wears a dark glove, the binarized image using the conventional fixed threshold value is not compatible with the operator's hand and the steering handle. Are both binarized to black, and the hand may not be determined reliably. On the other hand, when the steering handle is exposed to direct sunlight and shines white, in the conventional binarized image using a fixed threshold, both the operator's hand and the steering handle are binarized to white. As a result, the hand could not be determined with certainty.

  The present invention has been made in view of the above-described circumstances, and an object of the present invention is to accurately binarize an operator's hand and a steering handle in white and black to improve hand determination accuracy.

In order to achieve the above object, according to the first aspect of the present invention, any one of a plurality of marks provided on the steering handle is hidden by a hand based on imaging data output from an in- vehicle imaging means. Determination means capable of individually determining whether or not the mark has been made, and an operation command means for outputting an operation command for the in-vehicle device based on a determination result by the determination means, and the operation command means is in a standby state When it is determined that the specific mark of the plurality of marks is hidden by hand, the operation command means is used in an in-vehicle device operation apparatus for an automobile configured to switch to an operation mode. A method for determining a hand by binarizing an image captured by the imaging unit into white and black according to brightness, and determining that a white region is a hand ,
The operation command means is in the standby state for a brightness histogram of a predetermined area of the steering wheel when the specific mark is hidden by hand, and usually includes an area hidden by the hand and an area not hidden by the hand. A step of sometimes creating, a step of setting a lightness threshold value for determining white / black between two peaks of the lightness histogram when the operation command means is in the standby state, and brightness, characterized in that it comprises a and a step of determining a high area of the white region, Ru determination method of the hand in the vehicle equipment operation apparatus of an automobile have been proposed.

According to the invention described in claim 2, in addition to the configuration of claim 1, the color of the steering wheel is a dark color system close to black, and the specific mark is obtained by bordering black with white. There is proposed a method for determining a hand in an in-vehicle device operating device of an automobile characterized by the above.

According to the invention described in claim 3, based on the imaging data output from the vehicle-mounted imaging means, it is determined whether any of the plurality of marks provided on the steering handle is hidden by the hand. A plurality of determination means each capable of being individually determined; and an operation command means for outputting an operation command for an in-vehicle device based on a determination result by the determination means, and when the operation command means is in a standby state, When it is determined that the specific mark of the individual marks is hidden by hand, the operation command means is attached to an in-vehicle device operating device of the automobile that is switched to the operation mode, and the imaging means picks up the image. A hand determination device that binarizes white and black according to brightness and determines a white region as a hand ,
The operation command means is in the standby state for a brightness histogram of a predetermined area of the steering wheel when the specific mark is hidden by hand, and usually includes an area hidden by the hand and an area not hidden by the hand. and brightness histogram generation means for generating when said a crest portion determining means for determining two crests of the brightness histogram, the threshold of determining lightness of white and black between the two ridges, the operation command means Threshold setting means for setting when the camera is in the standby state, binary image conversion means for converting the image into a white / black binary image based on the threshold, and white of the binary image There is proposed a hand determination device in an in-vehicle device operating device of an automobile, characterized by comprising hand determination means for determining that an area is a hand.

According to a fourth aspect of the present invention, in addition to the configuration of the third aspect, the color of the steering wheel is a dark color system close to black, and the specific mark is obtained by bordering black with white. characterized in that, Ru is the proposed determination device in-vehicle equipment operation apparatus for an automobile according to claim 3.

  The audio AU, the video DVD, and the car navigation CN of the embodiment correspond to the in-vehicle device of the present invention, and the camera CA of the embodiment corresponds to the imaging unit of the present invention.

According to the configuration of the first aspect, in the first step, the brightness histogram of the predetermined region of the steering wheel when the mark is hidden by hand is created when the operation command means is in the standby state, and in the second step A brightness threshold value for determining white / black between the two peaks of the brightness histogram is set when the operation command means is in a standby state, and in the third step, an area having a brightness higher than the threshold value is set to white It is determined that the area. Therefore, if the lightness of the hand is relatively bright with respect to the lightness of the steering wheel, an appropriate threshold value can be set so that the steering wheel can be reliably binarized to black and the hand to white. By determining that it is, it becomes possible to greatly improve the determination accuracy . In particular, the predetermined area of the steering wheel includes an area that is normally hidden by the hand and an area that is not hidden when the specific mark is hidden by hand, so that it is possible to reliably form two peaks in the brightness histogram, An accurate threshold value can be set from the brightness histogram to further improve the hand determination accuracy.

In addition, according to the configurations of claims 2 and 4, the steering handle is a dark color system close to black, and the specific mark is a black border with white, so that the specific mark and the driver's hand are easy. Can be recognized.

According to the configuration of claim 3, the brightness histogram creating means creates a brightness histogram of a predetermined area of the steering wheel when the mark is hidden by hand when the operation command means is in a standby state. The determination means determines the two peaks of the brightness histogram, and the threshold setting means sets the brightness threshold for determining white / black between the two peaks when the operation command means is in a standby state. The binarized image converting means converts the image into a white / black binarized image based on the threshold value, and the hand determining means determines that the white area in the binarized image is a hand. Therefore, if the lightness of the hand is relatively bright with respect to the lightness of the steering wheel, an appropriate threshold value can be set so that the steering wheel can be reliably binarized to black and the hand to white. By determining that it is, it becomes possible to greatly improve the determination accuracy . In particular, the predetermined area of the steering wheel includes an area that is normally hidden by the hand and an area that is not hidden when the specific mark is hidden by hand, so that it is possible to reliably form two peaks in the brightness histogram, An accurate threshold value can be set from the brightness histogram to further improve the hand determination accuracy .

Schematic which shows the vehicle interior of a motor vehicle. The front view which looked at the instrument panel vicinity from the driver | operator side. FIG. 3 is an enlarged sectional view taken along line 3-3 in FIG. 2. The block diagram of a control system. Explanatory drawing which shows simply the switching operation process from the standby mode of an operation command means to the 1st operation mode. Explanatory drawing which shows simply the switching operation process from the 1st operation mode of the operation command means to the 2nd operation mode. Explanatory drawing which simplifies the operation process for canceling | releasing control of operation in vehicle with respect to vehicle equipment, and enabling manual operation to vehicle equipment by the passengers other than a driver | operator, even if it is drive | working. The block diagram which shows the logic which determines a hand. The figure which shows an example of a brightness histogram. The step schematic diagram which shows the operation procedure for performing volume adjustment operation. The step schematic diagram which shows the operation procedure for performing source selection operation. The step schematic diagram which shows the operation procedure for performing selection operation of an album name and a song name. The step schematic diagram which shows the operation procedure for performing transfer operation to voice search mode.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

  First, in FIG. 1 to FIG. 3, the instrument panel I in the passenger compartment includes various instruments such as a speedometer, a display unit L for in-vehicle devices such as an audio AU, a video DVD, and a car navigation CN, An operation unit 1 and the like for manually operating the devices AU, DVD, and CN are provided, and a steering handle S that is steered by the driver is disposed on the front side of the various instruments.

  The steering handle S includes a central hub portion Sa connected to a steering shaft (not shown), a rim portion Sm that surrounds the central hub portion Sa in an annular shape, and a connection between the rim portion Sm and the central hub portion Sa. Three spoke portions Sb1 to Sb3. A horn cover 2 is detachably attached to the outer surface of the central hub portion Sa, and a design-processed first H-shaped mark M1 is attached to the central outer surface of the horn cover 2.

  The spoke portions Sb1 to Sb3 are detachably mounted with a spoke cover 3 that covers the outer surfaces of the spoke portions Sb1 to Sb3. In the illustrated example, the spoke cover 3 is formed by forming a flexible sheet-like metal plate into a developed shape corresponding to the spoke portions Sb1 to Sb3, and an adhesive or adhesive for pasting is provided on the back surface thereof. A sheet is provided. When the spoke cover 3 is attached to the spoke portions Sb1 to Sb3, the cover 3 is wound and pasted so as to follow the outer surfaces of the spoke portions Sb1 to Sb3, and the free edge 3e of the cover 3 is attached to the spoke portion 3 as shown in FIG. As shown in FIG. 4, the spoke portions Sb1 to Sb3 are fixed to be bent inward.

  In the spoke cover 3, the second to fourth marks M2 to M4 (in the example shown, M2 corresponds to the volume mark, M3 corresponds to the source mark, and M4 corresponds to the select mark) on the portion corresponding to the left spoke portion Sb1. It is attached in a row by bonding or printing, and the spoke cover 3 corresponding to the right spoke portion Sb3 has fifth to seventh marks M5 to M7 (in the example shown, M5 is a voice mark and M6 is up). A mark, M7 corresponds to a down mark) is attached in a zigzag arrangement with the fifth mark M5 inward by adhesion or printing.

  A monochrome camera CA serving as an imaging unit capable of imaging each of the marks M1 to M7 is installed at a suitable position in the passenger compartment, in the illustrated example, at a ceiling slightly behind the driver. When the driver hides any of the marks M <b> 1 to M <b> 7, the camera CA outputs imaging data that makes it possible to clearly recognize that the mark is hidden to the electronic control unit ECU (see FIG. 4).

  In order that the first to seventh marks M1 to M7 and the driver's hand H can be easily recognized from the imaging data of the camera CA, the color of the steering wheel S is preferably a dark color system close to black. The 7 marks M1 to M7 are preferably black with white borders.

  As shown in FIG. 4, the electronic control unit ECU is installed in a suitable position in the vehicle interior, for example, inside the instrument panel I, which analyzes the imaging data output from the camera CA and analyzes the analysis data. Based on the determination means C1 that can individually determine whether the marks M1 to M7 are hidden for each of the marks M1 to M7, and on-vehicle equipment (that is, audio AU, video DVD, Operation command means C2 for outputting an operation command signal as an operation command for the car navigation CN), and a signal to be output to the in-vehicle equipment AU, DVD, CN by converting the output signal of the operation instruction means C2 into a drive signal for the in-vehicle equipment An output unit C3 and a recognition unit C4 capable of recognizing the occupant's hand H based on the imaging data output from the camera CA are provided.

  The camera CA is connected to each input side of the determination unit C1 and the recognition unit C4. Further, the output side of the determination means C1 and the recognition means C4, and the external operation section 1, the steering angle sensor SE1, and the vehicle speed sensor SE2 are connected to the input side of the operation command means C2. Furthermore, the output side of the operation command means C2 is connected to the input side of the signal output unit C3, and the in-vehicle devices AU, DVD, CN are connected to the output side of the signal output unit C3.

  The operation command means C2 selects either a standby mode in which the operation input of the in-vehicle equipment AU, DVD, CN can be performed by the external operation unit 1 or the first and second operation modes in which the operation input is prohibited. In the standby mode, marks other than the first mark M1 and the fifth mark M5 (that is, the second to fourth marks M2 to M4, the sixth mark M6, and the seventh mark M7) are hidden. However, the operation command means C2 does not respond, that is, no operation command is issued.

  Further, the operation command means C2 is a state in which the vehicle is set in a preset state (in the illustrated example, a steering angle operation greater than a predetermined angle or a predetermined angle when the operation command means C2 is in the first or second operation mode. There is no operation for concealing any of the second to seventh marks M2 to M7 in a state where there has been a sudden change in vehicle speed in time or for a certain period of time after switching to the first or second operation mode. , When the first mark M1 is hidden again despite being in the first or second operation mode), the mode is automatically switched to the standby mode. .

  As shown in FIG. 8, the recognition unit C4 includes a brightness histogram creation unit P1, a peak determination unit P2, a threshold setting unit P3, a binarized image conversion unit P4, and a hand determination unit P5. A camera CA is connected to the lightness histogram creation means P1 and the binarized image conversion means P4, the lightness histogram creation means P1 is connected to the peak determination means P2, and the peak determination means P2 is connected to the threshold setting means P3. The threshold setting means P3 is connected to the binarized image converting means P4, and the binarized image converting means P4 is connected to the hand determining means P5.

  As shown in FIG. 2, the steering wheel S is set with a central first region RC and second and third regions RL and RR located on the left and right sides thereof. The first region RC and the second and third regions RL and RR are all virtual regions, and the first region RC is set at a position covering the first mark M1, and the second and third regions RL and RR are set. Is set at a position spaced a predetermined distance from the first region RC. In other words, the first region RC is set to a position that is covered when the driver tries to hide the first mark M1 with his / her hand, and the second and third regions RL and RR are set to the first mark M1 by the driver. It is set to a position where it is difficult to cover when trying to hide with a hand.

Returning to FIG. 8, the lightness histogram creation means P1 of the recognition means C4 extracts pixels corresponding to the first region RC and the second and third regions RL and RR from the image of the camera CA, and arranges them for each lightness. Thus, the brightness histogram shown in FIG. 9 is created when the operation command means C2 is in the standby mode . When the driver tries to hide the first mark M1 with his hand, most of the first region RC is covered with light-colored hands, so the brightness of the pixel tends to be high, and the second and third regions RL , RR is not covered with light-colored hands, and the horn cover 2 of the dark-colored steering handle S is exposed, so that the lightness tends to be low. As a result, in the lightness histogram, one peak portion is generated in the high lightness region corresponding to the pixels in the first region RC, and other lightness regions corresponding to the pixels in the second and third regions RL and RR are displayed. One mountain part of will occur.

The peak determining means P2 determines the two peaks, the threshold setting means P3 sets a lightness threshold for binarization between the two peaks , and the operation command means C2 is in the standby mode. It is set to. This threshold value is a value that changes depending on the environment of the passenger compartment. For example, when the passenger compartment is dark, the brightness of the pixels in the first to third regions RC, RL, RR is reduced overall, so the threshold value is When moving to a lower (darker) side and the vehicle interior is brighter, the brightness of the pixels in the first to third regions RC, RL, RR is increased overall, so that the threshold value is moved to a higher (brighter) side. become.

  The binarized image conversion means P4 sorts all the pixels of the image captured by the camera CA according to the threshold value, binarizes pixels brighter than the threshold value, and binarizes pixels darker than the threshold value to black. The CA image is converted into a binary image composed of two colors of white and black. Then, the hand determination unit P5 determines that the portion of the white area of the binarized image that is gathered with an area of a predetermined value or more is the driver's hand H.

  Thus, the brightness histogram creating means P1 creates brightness histograms of the first region RC and the second and third regions RL and RR of the steering wheel S when the first mark M1 is hidden by the hand H. The part determination unit P2 determines two peaks of the brightness histogram, and the threshold setting unit P3 sets a brightness threshold for determining white / black between the two peaks, and the binarized image conversion unit P4. Thus, the image is converted into a white / black binarized image based on the threshold value, and the hand determination means P5 determines that a white area of a predetermined area or more in the binarized image is the hand H. If the brightness of the hand H is relatively high with respect to the brightness of S, it is possible to set an appropriate threshold value that can reliably binarize the steering wheel S to black and the hand H to white. Can be increased.

  Thus, when the operation command means C2 is in the standby mode, the determination means C1 recognizes the contour (circle) of the steering body Sm as shown in FIG. 5 (A) based on the imaging data of the camera CA. Since the center first mark M1 is recognized as shown in FIG. 5B, the first mark M1 is recognized during the recognition as shown in FIG. 5C. When it is hidden by the driver's hand H, it can be recognized quickly and reliably, and the recognition result is output to the operation command means C2, whereby the means C2 is switched to the first operation mode.

  In addition, when the operation command means C2 is in the first or second operation mode, the determination means C1 displays the second to seventh marks M2 to M7 of the spoke portions Sb1 and Sb2, as shown in FIG. During the recognition, for example, a plurality of second marks (that is, the second to fifth marks M2 to M5) are hidden by the driver's hand H as shown in FIG. If this happens, this can be recognized quickly and reliably, and the operation command means C2 switches to the second operation mode based on the recognition result. Further, in the second operation mode, a plurality of third marks (namely, sixth and seventh marks M6 and M7) are hidden by the driver's hand H as shown in FIG. This can be quickly and reliably recognized, and the recognition result is output to the operation command means C2.

  Further, the recognizing unit C4 recognizes the contour of the steering body Sm as a monitoring target range as shown in FIG. 7A based on the imaging data of the camera CA, regardless of the mode of the operation command unit C2. Yes. The operation command means C2 does not recognize that the recognition means C4 has two hands H on the rim portion Sm when it determines that the vehicle is running based on the detection signal of the vehicle speed sensor SE2. In this case, at least a part of the operation signal from the operation unit 1 is invalidated, but the recognition means C4 recognizes that there are two hands H on the rim portion Sm as shown in FIG. In some cases, the operation signal is not invalidated.

  Therefore, during traveling, at least a part of manual operation to the operation unit 1 of the in-vehicle devices AU, DVD, CN can be invalidated from the viewpoint of safe driving as in the past. However, if the recognition means C4 recognizes that there are two hands H on the rim portion Sm even when the vehicle is traveling, even if an operation signal is output from the operation unit 1, it is determined by an occupant other than the driver. Since it can be considered that the vehicle has been operated, the operation command means C2 does not invalidate the operation signal, and enables operations by passengers other than the driver. As a result, even when the vehicle is running, if the driver holds the steering wheel S with both hands H, passengers other than the driver can operate the in-vehicle devices AU, DVD, CN, which is convenient and convenient. Improves.

  The operation command means C2 switches to the first operation mode based on the determination by the determination means C1 that the first mark M1 is hidden when the operation command means C2 is in the standby mode. By obscuring one of the fifth marks M2 to M5, an operation target in the in-vehicle device (in the illustrated example, “volume”, “source” for selecting video / audio, “select” for selecting album name / song name, “Voice” for selecting the voice search mode of the car navigation system can be arbitrarily selected.

  Further, the operation command means C2 switches to the second operation mode when the determination means C1 determines that any of the second to fifth marks M2 to M5 is hidden when it is in the first operation mode. Instead, in the second operation mode, one of the plurality of operation parameters related to the operation object can be selected by hiding one of the sixth and seventh marks M6 and M7. The operation parameter means an operation content to be selected and operated more specifically by the operator in relation to the operation target. In the illustrated example, “volume” is selected because the second mark M2 is hidden. In such a case, hiding the sixth mark M6 corresponds to “volume up” and hiding the seventh mark M7 corresponds to “volume down”.

  Further, when the “source” is selected because the third mark M3 is hidden, hiding the sixth mark M6 is “one-way forward” of OFF / video / audio, and the seventh mark M7 is hidden. Corresponds to “forward feed in other directions”.

  Further, when “Select” is selected because the fourth mark M4 is hidden, the sixth mark M6 is hidden for a long time (for example, 2 seconds) by “one-way forward” in the album name, and the sixth mark Concealing M6 shortly means “one-way forward” of the song name, hiding the seventh mark M7 long (for example, 2 seconds), concealing the album name “forward-forwarding”, and concealing the seventh mark M7 shortly Corresponds to the song title “forward in the other direction”.

Next, the operation procedure of the operating device will be described more specifically with reference to FIGS.
[1] Volume adjustment operation When the operation command means C2 is in the standby mode, the display portion L is a normal car navigation screen as shown in step S10 of FIG. When M1 is concealed (step S11), the display of the marks M2 to M5 to be selected next is highlighted in the lower half of the screen together with the announcement “please operate” (step S12). If the second mark M2 is hidden (step S13), the display of the marks M6 and M7 to be selected next is highlighted in the lower half of the screen together with the announcement of “volume” (step S14), and then for a predetermined time (for example, After 3 seconds, the current volume display appears (step S15). Therefore, when either the sixth mark M6 or the seventh mark M7 is hidden (step S16), the volume adjustment is executed corresponding to the hidden mark, and the volume display after the adjustment appears (step S17).
[2] Source Selection Operation As shown in FIG. 11, steps S20 to S22 are the same as steps S10 to S12. When the third mark M3 is hidden from the state of step S22 (step S23), “source” The display of the marks M6 and M7 to be selected next is highlighted in the lower half of the screen together with the “selection” announcement (step S24), and when a predetermined time (for example, 3 seconds) elapses, the current source display (in the figure, “OFF” selection display) appears (step S25). Therefore, when either the sixth mark M6 or the seventh mark M7 is hidden (step S26), OFF video / audio is forwarded in one direction or the other direction corresponding to the hidden mark, and “source selection 〇〇 A source display after the selection appears together with the announcement “OO” (step S27).
[3] Album Name / Song Title Selection Operation As shown in FIG. 12, steps S30 to S32 are the same as steps S10 to S12. If the fourth mark M4 is hidden from the state of step S32 (step S33). ), Along with the announcement of “album name / song name selection”, the display of the marks M6 and M7 to be selected next is highlighted in the lower half of the screen (step S34), and then when a predetermined time (for example, 3 seconds) has passed, The album name / song title source display appears at (Step S35). Therefore, if one of the sixth mark M6 and the seventh mark M7 is hidden for a long time (for example, 2 seconds), the album name is forwarded in one direction or forward in the other direction (step S36), and the sixth mark M6 or If any of the seventh marks M7 is briefly hidden, one-way forward or other-direction forward of the song title is executed (step S36), and the newly selected album name / song title display appears along with the execution (step S36). Step S37). If the display is left for a predetermined time (for example, 5 seconds), the newly selected album name / song title is confirmed.
[4] Operation for Shifting to Voice Search Mode As shown in FIG. 13, steps S40 to S42 are the same as steps S10 to S12. When the operator hides the fifth mark M5 from the state of step S42. (Step S43) Along with the announcement of “voice search”, the car navigation CN can be shifted to a conventionally known voice search mode.

  According to the present embodiment described above, the determination unit C1 that can individually determine for each mark whether or not the plurality of marks M1 to M7 are hidden based on the imaging data output from the camera CA, and the determination Since the operation command means C2 for outputting an operation command (operation command signal) for the in-vehicle devices AU, DVD, CN based on the determination result by the means C1, each mark M1 to M7 is set by the operator's hand H with the camera CA. In addition to accurately outputting operation commands for in-vehicle devices AU, DVD, and CN based on whether or not they are hidden from the user, the operator's operation procedure is easy to understand, and erroneous operation due to a difference in operator's recognition can be effectively prevented. Become.

  In addition, the determination logic in the operation command means C2 is relatively simple, so that cost savings can be achieved and calculation processing can be performed quickly. The plurality of marks M1 to M7 are provided on a to-be-attached member (a horn cover 2 and a spoke cover 3 that can be attached to and detached from the steering wheel S in the illustrated example) and is read by the covers 2 and 3 and the steering wheel S. It is not necessary to make a special through-hole, so that it is relatively easy to respond to the option of retrofitting, and the degree of freedom of the installation positions of the marks M1 to M7 and the camera CA is also relatively high, and further the steering. The driver with S can easily and quickly perform the operation of hiding the marks M1 to M7 at hand.

  In addition, the operator can quickly and surely switch the operation command means C2 from the standby mode to the first operation mode by simply hiding the first mark M, and in the first operation mode, a plurality of By simply hiding one of the marks M2 to M5, it is possible to quickly and reliably select an operation target in the in-vehicle device AU, DVD, CN. Further, when the operation command means C2 is in the second operation mode, the operator simply hides one of the sixth and seventh marks M6 and M7, and specifies a plurality of operation parameters to be specified in relation to the operation target. That is, one of the operation contents can be selected quickly and reliably.

  By the way, the operation command means C2 is a state in which the vehicle is set in advance when the means is in one of the operation modes (in the illustrated example, the steering angle is operated more than a predetermined angle, or the vehicle speed changes rapidly within a predetermined time). Or a state in which the second to seventh marks M2 to M7 are left undisturbed for a certain period of time after switching to the first or second operation mode, or the first or second The operation automatically switches to the standby mode in accordance with the operation in which the first mark M1 is hidden again despite the operation mode. As a result, in special driving conditions that should be devoted to driving, such as when driving on a curve or sudden acceleration, or when the driver wants to return to standby mode immediately after leaving it for a certain period of time. In addition, since the operation command means C2 can be automatically returned to the standby mode, it is possible to reliably prevent the operation command means C2 from outputting a command due to an erroneous operation by the operator in those cases.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various design changes can be made without departing from the present invention described in the claims. Is possible.

  For example, in the embodiment, the first region RC and the second and third regions RL and RR are set so as to be separated from each other, but are set so that they are one continuous region. You may do it. The method of setting the area is arbitrary, but the first mark M1 must be included in a part of the area.

  In the embodiments, the audio AU, the video DVD, the car navigation CN, etc. are operated by the operation device of the present invention. However, in the present invention, the operation object is an air conditioner such as an air conditioner such as an air conditioner. It may be a device, a television or the like.

  In the embodiment, the operation command means C2 outputs an operation command (operation command signal) for the in-vehicle devices AU, DVD, CN based on the determination result of the determination means C1 that the marks M1 to M7 are hidden. In the present invention, based on the determination logic that combines the determination result of the determination unit C1 that the marks M1 to M7 are hidden and the result of the recognition unit C4 recognizing the hand H. You may make it output the operation command (operation command signal) with respect to vehicle equipment AU, DVD, CN. Even in this case, since the recognition unit C4 only identifies the hand H from a body part (head, arm, etc.) other than the hand H, the determination logic is not complicated as in the prior art.

  In the embodiment, a white region having a predetermined area or more is determined to be a hand, but the hand determination condition may be one or more protrusions extending radially from the center position of the white region.

AU audio (vehicle equipment)
C1 determination means C2 operation command means CA camera (imaging means)
CN car navigation (in-vehicle equipment)
DVD video (on-vehicle equipment)
H 1st mark M1 to M7 as specific mark M1 Mark P1 Lightness histogram creation means P2 Mountain determination means P3 Threshold setting means P4 Binary image conversion means P5 Hand determination means S Steering handle

Claims (4)

Based on the imaging data output from the vehicle-mounted imaging means (CA), it is determined whether any of the plurality of marks (M1 to M7) provided on the steering handle (S) is hidden by the hand (H). Determination means (C1) that can be individually determined for each mark (M1 to M7), and operation command means for outputting an operation command for the in-vehicle device (AU, DVD, CN) based on the determination result by the determination means (C1) (C2) and when the operation command means (C2) is in a standby state, it is determined that the specific mark (M1) among the plurality of marks (M1 to M7) is hidden by hand. Then, the operation command means (C2) is used in an in-vehicle device operating device for an automobile that is switched to an operation mode,
A method for determining a hand that binarizes an image captured by the imaging means (CA) into white and black according to lightness and determines that a white region is a hand (H) ,
A brightness histogram of a predetermined area of the steering wheel (S) when the specific mark (M1) is hidden by the hand (H), and including an area that is normally hidden by the hand (H) and an area that is not hidden by the hand (H) Are created when the operation command means (C2) is in the standby state, and a brightness threshold value for determining white / black between two peaks of the brightness histogram is defined as the operation command means (C2 ) is a step of setting when in the standby state, characterized in that it comprises a step of determining an area brightness is higher than the threshold value as the region of white, of the hand in-vehicle equipment operation apparatus of an automobile Judgment method. The in-vehicle device according to claim 1, wherein the steering handle (S) has a dark color system close to black, and the specific mark (M1) has a black border with white. A method for determining a hand in an operating device. Based on the imaging data output from the vehicle-mounted imaging means (CA), it is determined whether any of the plurality of marks (M1 to M7) provided on the steering handle (S) is hidden by the hand (H). Determination means (C1) that can be individually determined for each mark (M1 to M7), and operation command means for outputting an operation command for the in-vehicle device (AU, DVD, CN) based on the determination result by the determination means (C1) (C2) and when the operation command means (C2) is in a standby state, it is determined that the specific mark (M1) among the plurality of marks (M1 to M7) is hidden by hand. Then, the operation command means (C2) is attached to the in-vehicle device operation device of the automobile that is switched to the operation mode,
A hand determination device that binarizes an image captured by the image capturing unit (CA) into white and black according to brightness, and determines that a white region is a hand (H) ,
A brightness histogram of a predetermined area of the steering wheel (S) when the specific mark (M1) is hidden by the hand (H), and including an area that is normally hidden by the hand (H) and an area that is not hidden by the hand (H) Brightness histogram creation means (P1) created when the operation command means (C2) is in the standby state, peak determination means (P2) for determining two peaks of the brightness histogram, A threshold value setting means (P3) for setting a lightness threshold value for determining white / black between two peaks when the operation command means (C2) is in the standby state; and the image based on the threshold value. A binarized image converting means (P4) for converting to a white / black binarized image and a hand determining means (P5) for determining that the white area in the binarized image is a hand (H). and wherein the in-vehicle of the automobile Hand determination apparatus of vessel operating device. The in-vehicle device according to claim 3, wherein the color of the steering handle (S) is a dark color system close to black, and the specific mark (M1) is obtained by bordering black with white. Hand determination device in an operation device.

Images