JP2005084830A - Equipment controller and equipment control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately and quickly grasp operation contents intended by an operator, and to precisely perform control of equipment following the intentions of an operator. <P>SOLUTION: As calibration, the image of the finger preliminarily presented with a regular shape by an operator is picked up, and the finger area with another shape is predicted, based on the finger area measured from the image, and these informations are stored as calibration data. When actual equipment control is performed, the finger area measured from the image of the finger presented by the operation is compared with the calibration data, the shape of the finger presented by the operator is specified, and the equipment is controlled, based on operation input information(number) corresponding to this. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to, for example, a device control apparatus and a device control method for recognizing the shape or the like of a finger presented by an operator and controlling the device according to the recognized shape or the like of the finger.

  Conventionally, as a technology that enables an operator to control equipment without directly operating an input device such as an operation switch, the operator can recognize the shape of a finger presented by the operator and grasp the operation content intended by the operator. And the technique which controls an apparatus according to it is known (for example, refer patent document 1).

In the technique described in Patent Document 1, a finger held in front of the camera by the operator is imaged by the camera, the finger part in the image is identified by image recognition processing, and the shape of the finger part is grasped in advance. The shape of the finger of the operator is recognized by determining which of the several shape patterns is matched.
JP 2003-131785 A

  However, in the conventional technique described in Patent Document 1 described above, the actual finger shape is recognized by comparing the finger shape specified by the image recognition processing with the shape pattern that is grasped in advance. In some cases, it is not always possible to accurately recognize the finger shape of the finger. For example, a state in which the index finger and the middle finger are brought into contact with each other and stretched may be erroneously recognized as a state in which only the index finger is stretched. When the finger shape is erroneously recognized in this way, there arises a problem that the control of the device is executed with contents different from the intention of the operator.

  In addition, it is thought that it is possible to improve the recognition accuracy if a large number of shape patterns are grasped in advance in all situations, but in this case, each time hand shape recognition processing is performed. In addition, it is necessary to compare the finger shape specified by the image recognition process with a large number of shape patterns, which causes a problem that the processing load and processing time required for each recognition process increase.

  The present invention was devised to solve the above-described problems of the prior art, and is capable of accurately and quickly grasping the contents of the operation intended by the operator so as to meet the intention of the operator. It is an object of the present invention to provide a device control apparatus and a device control method capable of accurately performing the above control.

  The device control apparatus according to the present invention includes an imaging unit, a feature amount measuring unit, a feature amount predicting unit, a shape specifying unit, and a control unit.

  The imaging means captures an image of the imaging target with a part of the human body presented by the operator as the imaging target. The feature amount measuring unit measures the feature amount of the imaging target from the image captured by the imaging unit.

  The feature amount predicting unit predicts the feature amount when the imaging target is presented in another shape based on the feature amount measured from the image of the imaging target presented in the prescribed shape. In addition, the shape specifying unit is configured such that when the imaging target is presented in an arbitrary shape, the image is captured by the imaging unit, and the feature amount is measured from the image by the feature amount measuring unit. Based on the measured feature amount and the feature amount predicted by the feature amount prediction means, the shape of the imaging target is specified. And a control means controls an apparatus based on the operation input information corresponding to the shape of the imaging target specified by the shape specification means.

  The device control method according to the present invention includes the following first to seventh steps.

  In this device control method, first, in a first step, a part of a human body presented in a prescribed shape by an operator is imaged as an imaging target. Then, in the second step, the feature quantity of the imaging target is measured from the image captured in the first step. Next, in the third step, the feature amount when the imaging target is presented in another shape is predicted based on the feature amount measured in the second step.

  In the fourth step, a part of the human body presented in an arbitrary shape by the operator is imaged as an imaging target. Then, in the fifth step, the feature quantity of the imaging target is measured from the image captured in the fourth step. Next, in the sixth step, the shape of the imaging target imaged in the fourth step is specified based on the feature quantity measured in the fifth step and the feature quantity predicted in the third step. To do. In the seventh step, the device is controlled based on the operation input information corresponding to the shape of the imaging target specified in the sixth step.

  According to the present invention, when a part of the human body is presented in an arbitrary shape by the operator, the feature amount measured from the image at that time, and the feature amount information indicating the feature amount predicted in advance , The shape of a part of the human body presented by the operator is specified, and the device is controlled based on the corresponding operation input information, so that the operation content intended by the operator can be grasped accurately and quickly. Thus, it is possible to accurately control the device in accordance with the operator's intention. In addition, the prediction of the feature amount allows the operator to present a part of the human body in a prescribed shape, and based on the feature amount measured from the image at that time, the feature amount of the part of the human body in another shape Since it is performed by the method of predicting, it is possible to realize high-precision device control with a simple operation without forcing the operator to perform complicated operations.

  Hereinafter, an example in which the present invention is applied to an in-vehicle device control apparatus that controls various in-vehicle devices mounted on a vehicle will be described in detail with reference to the drawings as the best mode for carrying out the present invention.

  First, the configuration of an in-vehicle device control apparatus to which the present invention is applied will be described with reference to FIG. This in-vehicle device control apparatus collectively controls various in-vehicle devices such as a navigation device, a hands-free telephone, an audio device, an air conditioner device, etc. mounted on a vehicle. For example, the operation content intended by the operator is grasped from the shape of the finger presented by the operator, and the device is controlled accordingly.

  As shown in FIG. 1, this in-vehicle device control apparatus includes a finger support base (imaging target support means) 1, a far-infrared camera (imaging means) 2, an area measurement section (feature quantity measurement means) 3, a timing instruction section (notification) Means) 4, calibration switch (prediction process start switch) 5, finger area prediction unit (feature amount prediction unit) 6, calibration data storage unit (feature amount information storage unit) 7, information determination unit (shape specifying unit) 8 The device control unit (control means) 9 is provided.

  The finger support 1 is a table on which the operator's fingers to be imaged are placed. The finger support 1 supports the operator's fingers at a predetermined position, and has a function of keeping the distance between the far-infrared camera 2 and the operator's fingers at a predetermined distance. In order to make it easier for the driver to place his / her fingers, the driver is installed, for example, in a center console section inside the vehicle.

  In addition, the finger support base 1 is provided with an information input start switch 10, and when the operator presses the information input start switch 10, the operator's finger placed on the finger support base 1. Imaging of an image by the far-infrared camera 2 for imaging a finger is started.

  The far-infrared camera 2 captures an image of an operator's finger placed on the finger support 1 as an imaging target, and is fixed at a predetermined position inside the vehicle. The far-infrared camera 2 receives far-infrared light reflected by a subject with an image sensor and outputs an image corresponding to the intensity. Here, since the intensity of the far-infrared light reflected by the subject depends on the temperature of the subject, the image captured by the far-infrared camera 2 is an image reflecting the temperature difference of the subject.

  The far-infrared camera 2 captures an operator's fingers placed on the finger support 1 after a predetermined time has elapsed since the information input start switch 10 was pressed by the operator. An image captured by the far-infrared camera 2 is sent to the area measuring unit 3.

  The area measuring unit 3 analyzes the image captured by the far-infrared camera 2, extracts a portion corresponding to the operator's finger from this image, and measures the area of the portion corresponding to the finger. As described above, the image captured by the far-infrared camera 2 is captured as an intensity distribution of pixel values reflecting the temperature difference of the subject. Here, in the in-vehicle device control apparatus to which the present invention is applied, what is imaged by the far-infrared camera 2 is an operator's fingers placed on the finger support base 1, and the background of the finger support base 1 Since there is a clear temperature difference between the finger of the operator to be imaged, the difference between the pixel values reflecting these temperature differences causes the finger of the operator in the image captured by the far-infrared camera 2. The corresponding part can be easily extracted.

  From the above viewpoint, the area measurement unit 3 performs binarization processing on an image captured by the far-infrared camera 2 using a predetermined threshold (for example, a pixel value corresponding to 36 ° C.), thereby far-infrared. A portion corresponding to the finger of the operator is extracted from the image captured by the camera 2. Then, the area of the portion corresponding to the extracted finger is measured as the feature amount of the finger of the operator who is the imaging target.

  The timing instruction unit 4 notifies the operator of the imaging timing of the far-infrared camera 2 by, for example, a beep sound generated by a buzzer (not shown). That is, after the information input start switch 10 is pressed by the operator, the timing instructing unit 4 beeps at the timing when the far-infrared camera 2 actually images the finger of the operator placed on the finger support 1. Etc. are output. Thereby, the operator can recognize the imaging timing by the far-infrared camera 2 and can take measures such as stopping the movement of fingers during imaging.

  The timing instruction unit 4 also has a function of announcing an instruction for presenting fingers to the operator from a speaker (not shown). That is, the timing instruction unit 4 is preliminarily defined as a finger shape for performing calibration when the calibration switch 5 and the information input start switch 10 are pressed when performing calibration described in detail later. An instruction to present a finger in a different shape, such as an announcement “Please present 0” or “Please present 5.” is given. In addition, when the timing input unit 4 requests the operator to present a finger to actually control the device, when the information input start switch 10 is pressed, the timing instructing unit 4 is arbitrarily set according to the operator's intention. An instruction to present a finger in the form of, for example, “Please present your number” is given. As described above, when the instruction for presenting the finger is announced by the timing instruction unit 4, the operator may present the finger according to the announcement, and the burden on the operator is reduced.

  The calibration switch 5 is a switch for starting calibration. Here, the calibration in the in-vehicle device control apparatus to which the present invention is applied means that the finger shape presented by the operator when actually controlling the device can be easily and accurately specified in advance. Then, the finger is presented in a prescribed shape, captured by the infrared camera 2, the area measurement unit 3 measures the area of the finger part from the image, and the finger area prediction unit 6 based on the measured value This refers to a series of processes for predicting the area of the finger portion when another finger shape is presented, and storing each of the hand shapes that can be presented and their areas in the calibration data storage unit 7 as calibration data. . Such calibration is performed, for example, when the calibration switch 5 is pressed as a trigger when the on-vehicle device control apparatus to which the present invention is applied or when the operator is replaced with another operator. Is done.

  In this example, device control is realized by sequentially selecting items from a hierarchical operation menu, and each item can be selected with a number from 0 to 5 in each operation menu. It will be explained on the assumption. Therefore, the finger shape that can be presented in this in-vehicle device control device is a goo shape in which all fingers are tied to designate the number “0”, and a shape in which one finger is designated to designate the number “1”. , A shape in which two fingers for specifying the number “2” are extended, a shape in which three fingers for specifying the number “3” are extended, and four fingers in order to specify the number “4” are extended The shape is a par shape in which all fingers are opened to designate the number “5”.

  The finger area prediction unit 6 predicts the area of the finger when the operator's finger is presented in another shape based on the area of the finger part measured from the image of the finger of the prescribed shape when performing calibration. It is. In general, the area of the finger portion in the image captured by the far-infrared camera 2 is such that when a goo shape is presented, when a shape with one finger extended is presented, when a shape with two fingers extended is presented, When a shape obtained by extending three fingers is presented, when a shape obtained by stretching four fingers is presented, the number increases monotonously in the order in which the par shape is presented. Therefore, by measuring the area of a typical shape among these finger shapes in advance and predicting a function of area change, and applying this to each finger shape, the finger is presented in each possible finger shape. It is possible to predict the finger area at the time.

  In the finger area prediction unit 6, from the above viewpoint, for example, based on the finger area when the goo shape is presented as the prescribed shape and the finger area when the par shape is presented as the prescribed shape, The area of the finger with the shape of one finger extended, the area of the finger with the shape of two fingers extended, the area of the finger with the shape of three fingers extended, and the shape of the fingers extended with four fingers The finger area is predicted.

  The calibration data storage unit 7 calculates the finger area of the prescribed shape measured at the time of calibration and the finger area of another shape predicted by the finger area prediction unit 6 based on the finger area of the prescribed shape. Each is stored as calibration data (feature information) in association with the shape of the finger. Specifically, for example, when calibration is performed, the finger area S0 when the goo shape is presented as the prescribed shape and the finger area S5 when the par shape is presented as the prescribed shape are respectively the area measurement units. 3 and based on these finger areas S0 and S5, a finger area S1 with one finger extended, a finger area S2 with two fingers extended, and a shape with three fingers extended When the finger area prediction unit 6 predicts the finger area S3 in the shape of four fingers and the finger area S4 in a shape in which four fingers are extended, the calibration data storage unit 7 stores these finger areas S0, S1, and S4. S2, S3, S4 and S5 are associated with each finger shape and stored as calibration data.

  The information determination unit 8 identifies the finger shape by matching the area of the finger presented in an arbitrary shape by the operator with the calibration data when the device is actually controlled after calibration. Then, the operation input information corresponding to it is determined. That is, when actually controlling the device, the operator presents a finger in an arbitrary shape corresponding to the number of the item in order to designate a desired item from the operation menu. Then, an image of the operator's finger is picked up by the far-infrared camera 2, and the area measurement unit 3 measures the area of the finger part from the image. At this time, the information determination unit 8 reads the calibration data from the calibration data storage unit 7, compares the finger area measured by the area measurement unit 3 with the calibration data, and is presented by the operator. Specify finger shape.

  Specifically, when the area S of the finger presented in an arbitrary shape by the operator is measured by the area measurement unit 3, the information determination unit 8 determines that the measured finger area S is S0−ΔS ≦ S. <S0 + ΔS, S1-ΔS ≦ S <S1 + ΔS, S2-ΔS ≦ S <S2 + ΔS, S3-ΔS ≦ S <S3 + ΔS, S4-ΔS ≦ S <S4 + ΔS, S5-ΔS ≦ S <S5 + ΔS By determining, the shape of the finger presented by the operator is specified. Note that ΔS is a constant set to absorb the deviation between the value predicted from the calibration data and the actual value. Here, if it is determined that the finger area S measured by the area measurement unit 3 belongs to the range of S1−ΔS ≦ S <S1 + ΔS, the information determination unit 8 displays the finger presented by the operator. It is determined that the shape is a shape with one finger extended, and it is determined that the operation input information corresponding to this, that is, the number “1” is designated.

  The device control unit 9 controls various in-vehicle devices mounted on the vehicle based on operation input information corresponding to the finger shape of the operator specified by the information determination unit 8. That is, control commands for controlling various in-vehicle devices are assigned to the operation input information (for example, numbers 0 to 5) handled by the in-vehicle device control apparatus to which the present invention is applied. When the operation input information according to the operator's intention is determined by the information determination unit 8, the control command corresponding to the operation input information is recognized, and various in-vehicle devices are controlled according to the control command.

  Next, the operation of the in-vehicle device control apparatus to which the present invention configured as described above is applied will be specifically described with reference to FIGS. FIG. 2 is a flowchart showing an example of a specific processing flow when performing calibration, and FIG. 3 shows an example of a specific processing flow when actually controlling the in-vehicle device. It is a flowchart. FIG. 4 is a diagram illustrating a transition example of the operation screen displayed when the control of the in-vehicle device is executed.

  First, the operation when performing calibration will be described. As shown in FIG. 2, the calibration is triggered by the press of the calibration switch 5 by the operator (step S1-1). When the operator places his / her finger on the finger support 1 and depresses the information input start switch 10 (step S1-2), the timing instruction unit 4 sets “0” after a predetermined time (for example, 1 second). Please present me "message (step S1-3). In response to this, the operator makes the fingers placed on the finger support base 1 goo-shaped.

  The timing instruction unit 4 outputs a beep sound when a predetermined time (for example, 1 second) elapses after announcing the message “Please present 0” (step S1-4). Then, at the same time as the beep sound by the timing instruction unit 4 ends, the far-infrared camera 2 captures an image of the operator's finger placed on the finger support 1 (step S1-5). Then, this image is sent to the area measuring unit 3.

  When the image captured by the far-infrared camera 2 is input, the area measuring unit 3 performs a binarization process using a predetermined threshold (for example, a pixel value corresponding to 36 ° C.) on the image, A portion corresponding to the finger of the operator is extracted from the image (step S1-6), and the area of the finger portion is measured (step S1-7). With the above processing, the area S0 of the goo-shaped finger presented by the operator is measured.

  When the goo-shaped finger area S0 is measured, the timing instruction unit 4 next announces a message “Please show 5” (step S1-8). In response to this, the operator puts the finger placed on the finger support base 1 into a par shape. Thereafter, when a predetermined time (for example, 1 second) elapses, the timing instruction unit 4 outputs a beep sound (step S1-9), and at the same time as the beep sound ends, the far-infrared camera 2 is placed on the finger support base 1. An image of the finger of the operator placed on the camera is taken (step S1-10). Then, the image is sent to the area measuring unit 3.

  When the image captured by the far-infrared camera 2 is input, the area measuring unit 3 performs a binarization process on the image to display a portion corresponding to the finger of the operator, as in step S1-6. Extraction is performed (step S1-11), and the area of the finger portion is measured (step S1-12). With the above processing, the area S5 of the par-shaped finger presented by the operator is measured.

  When the measurement of the goo-shaped finger area S0 and the par-shaped finger area S5 is completed, the finger-area predicting unit 6 then determines the finger based on the goo-shaped finger area S0 and the par-shaped finger area S5. Finger area S1 in the shape of extending one finger, finger area S2 in the shape of extending two fingers, finger area S3 in the shape of extending three fingers, and in the shape of extending four fingers The finger area S4 is predicted (step S1-13). Information on the finger areas S0, S1, S2, S3, S4, and S5 in each shape is associated with each finger shape and stored in the calibration data storage unit 7 as calibration data (Step S1). S1-14).

  Finally, the timing instruction unit 4 announces a message “Calibration is completed” (step S1-15), and a series of processes at the time of performing calibration is completed. When the calibration is completed, the process proceeds to a process for actually controlling the in-vehicle device.

  Next, taking as an example a case where the destination setting of the navigation device is performed from registered location information registered in advance, an operation when actually controlling the in-vehicle device will be specifically described.

  In this case, as shown in FIG. 3, first, when the operator places his / her finger on the finger support 1 and depresses the information input start switch 10 (step S2-1), the control of the in-vehicle device is triggered by this. The process to be executed is started, and an operation screen of the highest hierarchy as shown in FIG. 4A, for example, is displayed (step S2-2). In this operation screen, the operation items of “Destination setting”, “Map display”, “Hands-free phone”, “Audio operation”, “Air conditioner operation”, and the number “1” associated with these operation items are displayed. ~ "5" are displayed, and by specifying these numbers, the corresponding operation items can be selected.

  When the operation screen of the highest hierarchy as shown in FIG. 4A is displayed, at the same time, the timing instructing unit 4 announces a message “please present a number” (step S2-3). ). In response to this, the operator puts the finger placed on the finger support base 1 into a shape indicating a number corresponding to the operation item to be selected. Here, since the destination of the navigation device is set, the operator assumes a finger shape with one finger extended in order to designate the number “1” corresponding to the destination.

  The timing instructing unit 4 outputs a beep sound when a predetermined time (for example, 1 second) elapses after announcing the message “please show the number” (step S2-4). Then, at the same time as the beep sound by the timing instruction unit 4 is finished, the far-infrared camera 2 captures an image of the operator's finger placed on the finger support 1 (step S2-5). Then, this image is sent to the area measuring unit 3.

  When the image captured by the far-infrared camera 2 is input, the area measuring unit 3 performs a binarization process on the image and performs processing corresponding to the operator's finger as in the calibration. Extraction is performed (step S2-6), and the area of the finger portion is measured (step S2-7). With the above processing, the finger-shaped area S presented by the operator is measured. Here, since the operator has a finger shape with one finger extended, the area measurement unit 3 measures the finger area.

  When the area S of the finger shape presented by the operator is measured, the information determination unit 8 then calculates the measured finger area S and the calibration data stored in the calibration data storage unit 7. In comparison, the finger shape presented by the operator is specified (step S2-8), and the number (operation input information) corresponding to the finger shape is determined (step S2-9). Here, since the area of the finger shape obtained by the operator extending one finger is measured by the area measuring unit 3, the finger shape presented by the operator by the information determining unit 8 is a shape obtained by extending one finger. It is determined that the operation input information corresponding to this, that is, the number “1” is designated.

  When the information determination unit 8 determines a number (operation input information) corresponding to the finger shape presented by the operator, an operation corresponding to the number is executed. At this time, if there is a lower level operation screen than the displayed operation screen, the operation corresponding to the number is a display of the lower level operation screen, and therefore there is a lower level operation screen. (Step S2-10), if there is a lower level operation screen, the process returns to step S2-2 to display the operation screen corresponding to the number. Thereafter, the same processing is repeatedly performed. After that, when the number corresponding to the finger shape presented by the operator is determined in the state where the lowest-level operation screen is displayed, the device control unit 9 performs various in-vehicle devices according to the control command corresponding to the number. The control is executed (step S2-11).

  Here, first, the number “1” is determined from the finger shape presented by the operator in the state where the highest-level operation screen shown in FIG. Returning, an operation screen listing the operation items related to the destination setting as shown in FIG. 4B is displayed. In this operation screen, the operation items related to destination settings such as “Return to home”, “Search from registered location”, “Search from recent destination”, “Search from facility”, “Search from phone number”, etc. Numbers “1” to “5” associated with each operation item are displayed, and by designating these numbers, the corresponding operation item can be selected.

  Thereafter, when it is determined by the processing from step S2-3 to step S2-9 that the number corresponding to the finger shape presented by the operator is “2”, the process returns to step S2-2 again, and further lower-order. As a hierarchical operation screen, an operation screen as shown in FIG. 4C, that is, an operation screen displaying a list of registered locations registered in advance is displayed. In this operation screen, items such as “Home”, “Company”, “XX Department Store”, “XX Restaurant” are displayed as pre-registered registration locations, and to search for registration locations on other pages Items such as “previous page” and “next page” are displayed. Also, numbers “0” to “5” associated with these items are displayed, and by designating these numbers, the corresponding items can be selected. Of these items, “previous page” and “next page” may not be displayed while the vehicle is running, and may be selected only when the vehicle is stopped.

  Thereafter, when it is determined by the processing from step S2-3 to step S2-9 that the number corresponding to the finger shape presented by the operator is “4”, the process returns to step S2-2 again, and FIG. As shown in FIG. 4D, a map around “XX restaurant”, which is a registered place corresponding to the number “4”, is displayed, and a destination setting menu is displayed on this map as shown in FIG. The superimposed lowermost operation screen is displayed. In this lowermost operation screen, each operation item related to the destination setting such as “go here”, “call here”, “view information”, and the number “1” associated with each operation item. ~ "3" is displayed, and by specifying these numbers, the corresponding operation items can be selected.

  Thereafter, when it is determined that the number corresponding to the finger shape presented by the operator is “1” by the processing from step S2-3 to step S2-9, the process proceeds to step S2-11, and the lowest layer is displayed. The device corresponding to the number “1” on the operation screen, that is, the destination setting of the navigation device is executed, and the route guidance is started after the route search.

  As described above, according to the in-vehicle device control apparatus to which the present invention is applied, the calibration for predicting the finger area of another shape from the finger area of the predetermined shape is executed in advance, and the device is actually controlled. In this case, the finger shape presented by the operator is specified using the calibration data obtained by the calibration, and device control corresponding to this is performed. Moreover, it can be performed with high accuracy, and the operation content intended by the operator can be grasped accurately and quickly, and the device can be controlled with high accuracy according to the operator's intention.

  In particular, the operator's finger shape is specified by measuring the area of the finger part from an image obtained by imaging the operator's finger and performing the determination based on the area of the finger part. There is a possibility that the pattern recognition based on the shape appearing in the image cannot be accurately identified, such as identifying a state where “2” is indicated by extending the contact and a state where “1” is indicated by extending only the index finger. A certain shape can be reliably identified, and the shape can be stably identified even when the direction in which the finger is presented changes.

  In addition, the calibration is based on the area of the finger presented in a representative shape (for example, goo shape or par shape) out of several possible finger shapes that can be presented. Therefore, it is possible to realize highly accurate device control with a simple operation without forcing the operator to perform complicated operations.

  Furthermore, since the far-infrared camera 2 is used to capture the finger presented by the operator, the area of the finger is appropriately extracted from the captured image by simple image processing such as binarization processing of the image. In addition, it is possible to appropriately extract the finger part and measure the finger area without depending on the surrounding brightness.

  As mentioned above, as a best mode for carrying out the present invention, a specific example of the in-vehicle device control apparatus to which the present invention is applied has been described. However, the present invention is not limited to the above-described example, and various Variations are possible.

  For example, in the above-described example, the finger presented by the operator is set as the imaging target, the area of the finger is measured as the feature amount of the imaging target, and the shape of the imaging target is specified based on the measured area. Although the control of the equipment along the line is performed, another part of the operator's body may be set as the imaging target, or another feature amount reflecting the shape of the imaging target may be measured.

  In the above-described example, the finger presented by the operator is imaged using the far-infrared camera 2, but other visible light such as a CCD (Charge Coupled Device) camera is used for imaging. A camera may be used. In this case, if the color of the finger support base 1 is set to a color that is significantly different from the color of the human body, it is possible to appropriately extract the finger portion by paying attention to the color component of the image.

  Further, in the above-described example, when the calibration is performed, the timing instruction unit 4 gives an instruction to present a finger in a prescribed shape, and when there is an announcement “Please present 0”, the operator Presents a goo-shaped finger and captures the image, and when an announcement such as “Please present 5” is made, the operator presents a par-shaped finger and captures the image. However, the far-infrared camera 2 automatically captures images at two timings with a predetermined interval without giving an instruction from the timing instruction unit 4 as described above, and according to these timings. The operator may be presented with a goo-shaped finger or a par-shaped finger.

  Further, in the above-described example, at the time of performing calibration, the operator is presented with a gou-shaped finger and a par-shaped finger as prescribed shapes, but other finger shapes to be presented for calibration are other shapes. May be defined. Further, either the goo shape or the par shape, or only one other shape may be presented, and the finger area of the other shape may be predicted based on the one shape. When only one shape is presented, it is desirable that the timing instruction unit 4 make an announcement for designating the shape so that the operator can recognize what shape the finger should present.

  Further, the example in which the present invention is applied to the in-vehicle device control apparatus that controls various in-vehicle devices mounted on the vehicle has been described above, but the present invention is not limited to the above example and is presented by the operator. The present invention can be effectively applied to any device control apparatus having a configuration in which a part of the human body is specified to determine the operation content intended by the operator and the device is controlled based on the determination.

It is a block diagram which shows the structure of the vehicle equipment control apparatus to which this invention is applied. It is a figure explaining operation | movement of the vehicle equipment control apparatus to which this invention is applied, and is a flowchart which shows an example of the flow of the concrete process at the time of implementing a calibration. It is a figure explaining operation | movement of the vehicle equipment control apparatus to which this invention is applied, and is a flowchart which shows an example of the flow of a specific process at the time of actually performing control of a vehicle equipment. It is a figure which shows the example of a transition of the operation screen displayed when performing control of vehicle equipment, (a) is a figure which shows the example of a display of the highest level operation screen, (b) is the operation item regarding destination setting. The figure which shows the example of a display of the enumerated operation screen, (c) is the figure which shows the example of a display of the operation screen for searching a registration place, (d) is the figure which shows the example of a display of the map around the selected registration place, e) is a diagram showing a display example of a lowermost operation screen in which a destination setting menu is superimposed on a map.

Explanation of symbols

1 Finger support base (supporting means for imaging)
2 Far-infrared camera (imaging means)
3 Area measuring unit (feature quantity measuring means)
4 Timing instruction section (notification means)
5 Calibration switch (Prediction process start switch)
6 finger area prediction unit (feature amount prediction means)
7 Calibration data storage (feature information storage means)
8 Information judgment part (shape identification means)
9 Equipment control section (control means)

Claims (11)

Imaging means for imaging a part of the human body presented by the operator as an imaging target, and an image of the imaging target;
Feature quantity measuring means for measuring a feature quantity of the imaging target from an image taken by the imaging means;
Feature amount prediction means for predicting a feature amount when the imaging target is presented in another shape based on a feature amount measured from the image of the imaging target presented in a prescribed shape;
When the imaging target is presented in an arbitrary shape, the shape of the imaging target is specified based on the feature quantity measured by the feature quantity measurement unit and the feature quantity predicted by the feature quantity prediction unit Shape identifying means to perform,
A device control apparatus comprising: control means for controlling a device based on operation input information corresponding to the shape of the imaging target specified by the shape specifying means. A feature amount information storage unit that stores the feature amount predicted by the feature amount prediction unit in association with the shape of the imaging target and stores it as feature amount information;
The shape specifying means is based on the feature quantity measured by the feature quantity measuring means when the imaging target is presented in an arbitrary shape and the feature quantity information stored in the feature quantity information storage means. The device control apparatus according to claim 1, wherein the shape of the imaging target is specified.   The apparatus control apparatus according to claim 1, wherein the imaging target is a hand presented by an operator.   The device control apparatus according to claim 3, wherein the feature amount is an area of a hand presented by an operator.   The feature amount predicting means includes an area of a hand in a goo shape measured from an image captured when a goo shape in which all fingers are tied and a par in which all fingers are open. Based on the area of the hand in the par shape measured from the image captured when the shape was presented, the area of the hand with the shape of one finger extended and the hand with the shape of the finger extended with two fingers The device control apparatus according to claim 4, wherein the area, the area of the hand in a shape with three fingers extended, and the area of the hand in a shape with four fingers extended are predicted.   6. The apparatus control apparatus according to claim 1, wherein an infrared camera is used as the imaging unit.   The imaging target support means for supporting the imaging target at a predetermined position and fixing the distance between the imaging means and the imaging target, further comprising: Equipment control equipment.   The device control apparatus according to claim 1, further comprising a notification unit that causes an operator to recognize an imaging timing of the imaging unit.   The notification means instructs the operator to present a part of the human body to be imaged in a prescribed shape or to present a part of the human body to be imaged in an arbitrary shape. The device control apparatus according to claim 8, wherein notification is performed.   The device control apparatus according to claim 1, further comprising a prediction process start switch for starting a feature amount prediction process by the feature amount prediction unit. A first step of imaging a part of a human body presented in a prescribed shape by an operator as an imaging target;
A second step of measuring a feature quantity of the imaging target from the image captured in the first step;
A third step of predicting a feature value when the imaging target is presented in another shape based on the feature value measured in the second step;
A fourth step of imaging a part of a human body presented in an arbitrary shape by an operator as an imaging target;
A fifth step of measuring a feature quantity of the imaging target from the image captured in the fourth step;
A sixth step of identifying the shape of the imaging target imaged in the fourth step based on the feature quantity measured in the fifth step and the feature quantity predicted in the third step; ,
A device control method comprising: a seventh step of controlling the device based on the operation input information corresponding to the shape of the imaging target specified in the sixth step.

Images