JP2009187232A - Onboard operation object extraction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an onboard operation object extraction device for satisfactorily securing operability even when an operation panel face is made small. <P>SOLUTION: This operation object extraction device 10 (onboard operation object extraction device) includes a function for remotely operating a car navigation device NV (onboard equipment), wherein the fingertip image of a driver D (operator) is displayed so as to be superimposed by a liquid crystal panel 1a (display part) of a display device 1. The operation object extraction part 10 includes: a camera 11; an illumination device 12; a memory 13 (storage means); the operation panel 14; a hand detection sensor 15; a photographing control part 16 (control means, photographing control means); and an image creation control part 17 (control means, image creation control means). The photographing control part 16 and the image creation control part 17 generate the fingertip image so that the area occupied by the fingertip F of a driver D can be made smaller than the area of the face of the operation panel 14. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an in-vehicle operation object extraction device, and more particularly to an in-vehicle operation object extraction device that can superimpose and display an extracted operation object on a display unit of an in-vehicle device such as a car navigation device.

  As this type of in-vehicle operation object extraction device, for example, as described in Patent Document 1 below, an operation panel is provided that can be remotely operated corresponding to the display unit of the in-vehicle device, and is on the operation panel surface. There is known a technique in which an operator's fingertip is photographed with a camera and the photographed fingertip image is superimposed and displayed on a display unit. In this vehicle-mounted operating object extraction device, the lighting device and the camera are installed on the opposite side of the operator panel from the operator's fingertip, and the operator's fingertip image and the lighting device are turned off when the lighting device is turned on. The operator's fingertip image in the state is differentiated. According to this on-vehicle operating object extraction device, since the fingertip image is extracted based on the difference image, it is possible to extract the fingertip image that is not affected by external light.

JP 2007-272596 A

  Incidentally, since the operator wants to reduce the amount of operation at hand as much as possible, it is desired to reduce the size of the operation panel surface. By reducing the operation panel surface in this way, there is also an advantage that the operation object extraction device itself can be reduced. However, if the operation panel surface (approximately equal to the photographing range) is reduced, for example, as shown in FIG. 20, the area occupied by the operator's fingertips increases with respect to the area of the operation panel surface. For this reason, the operator's fingertips are greatly displayed on the display unit of the in-vehicle device, and there is a problem that the operability is impaired, for example, it becomes difficult for the operator to understand the operation.

  The present invention has been made to address the above-described problems, and an object of the present invention is to provide an in-vehicle operation object extraction device that can ensure good operability even if the operation panel surface is reduced. There is.

  In order to achieve the above object, the present invention includes an operation panel that can be remotely operated in correspondence with a display unit of an in-vehicle device, photographs an operator's fingertip on the operation panel surface with a camera, and shoots the image In the on-vehicle operation object extraction device that displays the fingertip image superimposed on the display unit, the apparatus includes a control unit that generates the fingertip image so that the area occupied by the fingertip of the operator is smaller than the area of the operation panel surface. Features. Note that the fingertip image superimposed on the display unit is preferably approximately the same size as the fingertip of the actual operator, and is smaller than the size of each button provided on the display unit (stretching the two buttons). It is desirable that

  In this in-vehicle operation object extraction device, the fingertip image is generated by the control means so that the area occupied by the operator's fingertip is smaller than the area of the operation panel surface. For this reason, since the fingertip image of the operator displayed in a superimposed manner on the display unit can be reduced, good operability can be ensured even if the operation panel surface is reduced.

  In carrying out the present invention, a plurality of illumination means for illuminating the operator's fingertips at different angles is provided, and the control means is a photographing control means for causing the camera to photograph the operator's fingertips with different lighting modes of the plurality of illumination means. And image creation control means for creating a fingertip image to be displayed on the display unit based on at least the first fingertip image and the second fingertip image captured in a state where the lighting modes of the plurality of illumination means are different. It is good to make it the structure provided with these. In this case, for example, the camera is arranged so as to be able to photograph a surface of the operator's fingertip that is opposite to the surface that is exposed to external light, and the plurality of illumination means is a left-side illumination means that illuminates the operator's fingertip from the lower left side And the right illumination means that illuminates from the lower right side, and the imaging control means causes the camera to capture the first fingertip image with only the left illumination means turned on, and only the right illumination means is lit. In this state, the camera is set to capture the second fingertip image, and the image creation control means is set to extract the common part by comparing the first fingertip image and the second fingertip image. It is preferable that

  According to this, the plurality of illumination units are set to different lighting modes by the imaging control unit, and at least the first fingertip image and the second fingertip image are captured by the camera. Then, the image creation control means creates a fingertip image to be displayed on the display unit based on the first fingertip image and the second fingertip image. Since the first fingertip image and the second fingertip image are photographed with a plurality of lighting means having different lighting modes, the high-luminance part and the low-part part of the fingertips are different in both fingertip images. . Therefore, for example, when it is set to remove a portion having a low luminance value from each of the first fingertip image and the second fingertip image and to extract only a portion having a high luminance value common to both fingertip images, the fingertip image Can be slimmed (reduced). In addition, when the illumination unit is in a lighting state, the illumination image is reflected on the operation panel. However, when the common part between the first fingertip image and the second fingertip image is set to be extracted, the illumination image is displayed. Since the position of the illumination image reflected on the operation panel is different for each means, there is no common part in the illumination image, and the possibility that the illumination image is extracted as a fingertip image can be satisfactorily eliminated.

  Further, the image creation control means may binarize the first fingertip image and the second fingertip image on the basis of a predetermined luminance value, respectively, and extract a common part of these binarized images. Accordingly, the first fingertip image and the second fingertip image can be acquired by performing simple image processing.

  The imaging control means may set the imaging timing of the camera to a predetermined cycle, and turn on the left illumination means and the right illumination means alternately in accordance with the imaging timing. According to this, since the first fingertip image and the second fingertip image can be sequentially acquired by performing the imaging control so that the lighting timing of the left illumination means and the right illumination means matches the imaging timing, the imaging control is performed. The lighting control of the lighting means by the means becomes easy.

  In implementing the present invention, the camera is arranged so as to be able to photograph the surface of the operator's fingertip that is opposite to the surface exposed to external light, and the plurality of illumination means illuminate the operator's fingertip from the lower left side. The left side illumination unit and the right side illumination unit that illuminates from the lower right side are configured. The imaging control unit causes the camera to capture the first fingertip image with only the left side illumination unit turned on, and only the right side illumination unit is provided. Is set so that the camera can take a second fingertip image while the left illumination means and the right illumination means are turned off, and the camera can take a third fingertip image. May create a fingertip image to be displayed on the display unit based on the first fingertip image, the second fingertip image, and the third fingertip image.

  According to this, the first fingertip image is photographed by the camera while only the left illumination means is lit by the photographing control means, and the second fingertip image is photographed by the camera while only the right illumination means is lit. A third fingertip image is captured by the camera with the left illumination means and the right illumination means turned off. Then, the image creation control means creates a fingertip image to be displayed on the display unit based on the first fingertip image, the second fingertip image, and the third fingertip image. Since the first fingertip image and the third fingertip image are photographed corresponding to the lighting state and the extinguishing state of the left illumination means, respectively, the first fingertip image and the third fingertip image are obtained. If it is used, it is possible to acquire a fingertip image with a clear outline based on the first fingertip image (third fingertip image) regardless of whether the external light illuminance is strong or weak. Similarly, since the second fingertip image and the third fingertip image are photographed corresponding to the lighting state and the extinguishing state of the right illumination means, respectively, the second fingertip image and the third fingertip image are taken. If an image is used, it is possible to acquire a fingertip image with a clear outline based on the second fingertip image (third fingertip image) regardless of whether the external light illuminance is strong or weak. Thereby, it is possible to create a slim fingertip image based on both fingertip images with clear outlines.

  In this case, the image creation control means calculates the first binary value by subtracting the luminance values of the corresponding portions of the first fingertip image and the third fingertip image and binarizing the predetermined luminance value as a reference. The second binarized difference image is acquired, the luminance values of the corresponding portions of the second fingertip image and the third fingertip image are differentiated, and binarized based on a predetermined luminance value An image is acquired, and a common part is extracted by comparing the first binarized difference image and the second binarized difference image. According to this, a 1st binarized difference image and a 2nd binarized difference image can be acquired by performing simple image processing.

  The imaging control means sets the imaging timing of the camera to a predetermined cycle, and the imaging timing is set to one of the left illumination means and the right illumination means, either the left illumination means or the right illumination means. It is good to set so that it may correspond sequentially with the other lighting state and the light extinction state of both the left side illumination means and the right side illumination means. According to this, lighting control of the illumination means by the imaging control means becomes easy.

  Further, the imaging control means sets the imaging timing of the camera to a predetermined cycle, and the imaging timing is set to one of the left illumination means and the right illumination means, one extinguished state, the left illumination means and the right illumination. It is good to set so that it may be made to correspond to the other lighting state and the other light extinction state sequentially among the means. According to this, since the first binarized difference image and the second binarized difference image obtained per unit time can be increased, even when the operator's finger moves quickly, The fingertip can be smoothly displayed on the display unit.

a. First Embodiment Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall view schematically showing a car navigation device NV provided with an on-vehicle operating object extraction device according to the present invention. This car navigation device NV includes a display device 1 and an operation object extraction device 10.

  The display device 1 (display unit) includes a liquid crystal panel 1a, and is disposed in a front part of a vehicle compartment that can be viewed from the driver D. The display device 1 displays a vehicle symbol for navigation on a map and informs the driver D of various information such as road traffic information.

  The operation object extraction device 10 has a function of remotely operating the car navigation device NV, and displays a fingertip image of the driver D (operator) on the liquid crystal panel 1a of the display device 1 in a superimposed manner. As shown in FIGS. 2 and 3, the operation object extraction device 10 includes a camera 11 (imaging device), an illumination device 12, a memory 13 (storage means), an operation panel 14, and a hand detection sensor 15 (operation object detection means). The imaging control unit 16 and the image creation control unit 17 are configured.

  The camera 11 is capable of capturing an infrared image, and includes an imaging device for capturing a visible image, a reflecting mirror that reflects infrared light, a lens for condensing infrared light, and visible infrared light. For example, it is composed of a light wavelength conversion element using a nonlinear optical material that converts light. The camera 11 is disposed on the lower side of the operation panel 14 so as to photograph a surface of the fingertip F of the driver D on the side opposite to the surface on which external light strikes. The camera 11 is focused so that the captured fingertip image is approximately the same size as the actual fingertip.

  The illuminating device 12 can emit infrared light, for example, as an infrared LED, and illuminates the fingertip F of the driver D from the lower left side and a light source that illuminates from the lower right side. 12b (right illumination means). FIG. 3 shows an example in which three light sources 12a and 12b are arranged. Returning to FIG. 2, the memory 13 stores captured image data by the camera 11 and can store a plurality of captured image data.

  The operation panel 14 is for the driver D to perform an operation input with the fingertip F, and is incorporated in the center console box. The operation panel 14 is made of a material that transmits infrared rays, such as an acrylic plate, and the area of the panel surface is less than half of the area of the liquid crystal panel 1a surface of the display device 1 (for example, 60: 150). Ratio).

  The hand detection sensor 15 is composed of, for example, a microswitch or a pressure-sensitive sheet, and can be pushed down by the palm of the driver D or the wrist while the fingertip F of the driver D is placed on the operation panel 14. Is provided in the vicinity of the operation panel 14. The hand detection sensor 15 outputs an ON signal to the imaging control unit 16 when the hand of the driver A is detected.

  The photographing control unit 16 (control means, photographing control means) functions to turn on and off the light sources 12a and 12b of the illumination device 12 and to perform photographing control of the camera 11, and is not shown in the figure. , RAM, I / O, etc. are used as main components. The imaging control unit 16 repeatedly executes the imaging control program of FIG. 4 stored in the ROM or the like based on the ON signal from the hand detection sensor 15.

  The image creation control unit 17 (control means, image creation control means) fulfills the function of creating a fingertip image based on the captured image data of the camera 11 stored in the memory 13, and a CPU and ROM not shown in the figure. , RAM, I / O, etc. are used as main components. The image creation control unit 17 repeatedly executes the image processing program of FIG. 5 stored in the ROM or the like.

  Next, the operation of the first embodiment configured as described above will be described. When the occupant operates the ignition key to turn on the ignition switch, the imaging control unit 16 repeatedly executes the imaging control program of FIG. 4 every predetermined short time.

  The photographing control program is started to be executed in step S10. If the fingertip F of the driver D is not placed on the operation panel 14, the sensor signal from the hand detection sensor 15 is off (step S11: No), and thus this imaging control program is temporarily terminated in step S18.

  On the other hand, when the fingertip F of the driver D is placed on the operation panel 14 and the palm or wrist of the driver D is detected by the hand detection sensor 15, the sensor signal from the hand detection sensor 15 is turned on (step S11). : Yes), the process after step S12 is executed. In the processing after step S12, as shown in FIG. 6, the shooting control unit 16 sets the shooting timing of the camera 11 to a predetermined period, and alternately turns on the light source 12a and the light source 12b according to the shooting timing. .

  Specifically, as shown in FIG. 6 and FIG. 7A, first, only the light source 12a is turned on (step S12) so as to match the shooting timing of the camera 11, and the camera 11 is turned on only with the light source 12a. The fingertip F is photographed (step S13). The captured image data at this time is stored in a predetermined area of the memory 13 as a left-side light source lighting image (first fingertip image). Next, the light source 12a is turned off (step S14), and as shown in FIG. 6 and FIG. 7B, only the light source 12b is turned on (step S15), and the fingertip F is put on the camera 11 with only the light source 12b turned on. Photographing is performed (step S16). The captured image data at this time is stored in a predetermined area of the memory 13 as an image when the right light source is turned on (second fingertip image). After the process of step S16, the light source 12b is turned off (step S17), and this shooting control program is temporarily ended in step S18. Thereafter, if the sensor signal from the hand detection sensor 15 continues to be on, the processes of steps S12 to S17 are repeatedly executed.

  In parallel with the execution of the shooting control program of FIG. 4 by the shooting control unit 16, the image creation control unit 17 repeatedly executes the image processing program of FIG. 5 every predetermined short time. The execution of this image processing program is started in step S20. First, the left-side light source lighting image stored in the memory 13 is acquired (step S21), and the acquired left-side light source lighting image is binarized (step S22). Specifically, as shown in FIG. 8A, the high-intensity portion SL1 of the fingertip F illuminated by the light source 12a, the low-luminance portion SL2 shadowed to the right of the fingertip F, and the background of the fingertip F Only the high brightness portion SL1 is extracted from the portion SL3.

  Next, the right-side light source lighting image stored in the memory 13 is acquired (step S23), and the acquired right-side light source lighting image is binarized (step S24). Specifically, as shown in FIG. 8B, the high brightness portion SR1 of the fingertip F illuminated by the light source 12b, the low brightness portion SR2 shadowed to the left of the fingertip F, and the background of the fingertip F Only the high brightness portion SR1 is extracted from the portion SR3. Finally, as shown in FIG. 8C, the common area SC of both the high brightness portions SL1 and SR1 is extracted (step S25). After the processing in step S25, the image processing program is terminated in step S26.

  According to the first embodiment, by performing simple image processing as shown in FIG. 5, for example, as shown in FIGS. 9 and 10, a fingertip image in which both the light sources 12a and 12b are turned on is obtained. The fingertip image can be slimmed (reduced) compared to the binarized extraction area S0.

  When both the light sources 12a and 12b are turned on as shown in FIG. 9, the illumination images LL and LR of both the light sources 12a and 12b are reflected on the operation panel 14 as shown in FIG. Since the illumination images LL and LR have high luminance values, the conventional method may cause the illumination images LL and LR to be extracted as fingertip images. According to the first embodiment, FIG. And as shown to Fig.13 (a), only the light source 12b is lit as shown in the image at the time of the left side light source lighting acquired when only the light source 12a lights up, and FIG.12 (b) and FIG.13 (b) Since the positions of the illumination images LL and LR reflected in the operation panel 14 are different from the right-side light source lighting image acquired when the illumination light source is turned on, a common part does not occur between the illumination images LL and LR, and step S25 in FIG. With this process, it is possible to satisfactorily eliminate the possibility that the illumination images LL and LR from the light sources 12a and 12b are extracted as the fingertip image F (see FIG. 13C).

b. Second Embodiment In the first embodiment, the imaging control unit 16 executes the imaging control program of FIG. 4 and the image creation control unit 17 executes the image processing program of FIG. Thus, the imaging control unit 16 may execute the imaging control program of FIG. 14, and the image creation control unit 17 may execute the image processing program of FIG. Other configurations are the same as those in the first embodiment.

  In the second embodiment, in addition to the fingertip images when the light sources 12a and 12b are turned on, the fingertip images when both the light sources 12a and 12b are turned off are acquired. First, the photographing control program in FIG. 14 will be described.

  The execution of this photographing control program is started in step S30. When the palm or wrist of the driver D is detected by the hand detection sensor 15, the sensor signal from the hand detection sensor 15 is turned on (step S31: Yes), and the processing after step S32 is executed. In the processing after step S32, as shown in FIG. 16, the imaging control unit 16 sets the imaging timing of the camera 11 to a predetermined period, and turns on the light source 12a and the light source 12b alternately according to the imaging timing. .

  Specifically, as shown in FIGS. 7A and 16, first, only the light source 12a is turned on (step S32) so as to match the shooting timing of the camera 11, and the camera 11 is turned on only with the light source 12a. Then, the fingertip F is photographed (step S33). The captured image data at this time is stored in a predetermined area of the memory 13 as a left-side light source lighting image (first fingertip image). Next, the light source 12a is turned off (step S34), and as shown in FIGS. 7B and 16, only the light source 12b is turned on (step S35), and the fingertip F is put on the camera 11 with only the light source 12b turned on. Photographing is performed (step S36). The captured image data at this time is stored in a predetermined area of the memory 13 as an image when the right light source is turned on (second fingertip image). Next, the light source 12b is turned off (step S37), and the fingertip F is photographed by the camera 11 with the light sources 12a and 12b turned off (step S38). The captured image data at this time is stored in a predetermined area of the memory 13 as an image when the two light sources are turned off (third fingertip image). After the process of step S38, the photographing control program is temporarily ended in step S39. Thereafter, if the state in which the sensor signal from the hand detection sensor 15 is on continues, the processes of steps S31 to S38 are repeatedly executed.

  In parallel with the execution of the shooting control program of FIG. 14 by the shooting control unit 16, the image creation control unit 17 repeatedly executes the image processing program of FIG. 15 every predetermined short time. The execution of the image processing program is started in step S40, the left light source on-image stored in the memory 13 is acquired (step S41), and the both light source off-image stored in the memory 13 is acquired (step S41). Step S42).

  Next, the luminance values of the corresponding parts of the left-side light source on-image acquired in step S41 and the both-light source off-image acquired in step S42 are differentiated (the absolute value of the difference is taken). Then, the difference image is binarized using a predetermined luminance value as a reference, and a left-side binarized difference image (first binarized difference image) is acquired (step S43).

  Similarly, an image when the right light source is turned on stored in the memory 13 is acquired (step S44), and an image when both light sources are turned off stored in the memory 13 is acquired (step S45). The luminance values of the corresponding parts of the right light source on-image acquired in step S44 and the both light source off-images acquired in step S45 are differentiated, and the difference image is binarized based on a predetermined luminance value. A right-side binarized difference image (second binarized difference image) is acquired (step S46). Finally, a common area of both binarized difference images is extracted (step S47). After the processing in step S47, the image processing program ends in step S48.

  According to the second embodiment, a fingertip image (left side) with a clear outline can be acquired by the process of step S43 regardless of whether the external light illuminance is strong or weak, and a fingertip with a clear outline by the process of step S46. An image (right side) can be acquired. As a result, it is possible to create a slim fingertip image based on both fingertip images with clear outlines.

c. Third Embodiment In the second embodiment, the shooting control unit 16 executes the shooting control program of FIG. 14, so that the left binarized difference image or the right binarized difference image is obtained in three shooting steps by the camera 11. Although it is configured to acquire (see FIG. 16), the imaging control unit 16 executes the imaging control program of FIG. 17, and as shown in FIG. You may comprise so that an image or a right side binarized difference image may be acquired.

  The shooting control program of FIG. 17 adds the process of step S131 between the process of step S34 (light-off control of the light source 12a) and the process of step S35 (light-on control of the light source 12b) in the shooting control program of FIG. This is different from the shooting control program of FIG. That is, in the shooting control program of FIG. 14, the image when both the light sources 12a and 12b are turned off is acquired only by the process of step S38. However, in the shooting control program of FIG. An image when both the light sources 12a and 12b are turned off is acquired after the light is turned off until the light source 12b is turned on.

  According to the third embodiment, since the left side binarized difference image and the right side binarized difference image obtained per unit time increase compared to the second embodiment, the movement of the finger of the driver D is increased. Even in an early case, the fingertip can be smoothly displayed on the liquid crystal panel 1a of the display device 1.

(Modified embodiment)
The timing for obtaining the images when each light source is turned on and the images when both light sources are turned off is not limited to the mode shown in FIG. 18, but may be the mode shown in FIG. 19, for example. According to this, using the sequentially acquired images when the light sources are turned on or when both light sources are turned off and the images when both light sources are turned off or the images when each light source is turned on acquired immediately in the next shooting step, Since the image or the right-side binarized difference image can be acquired, the fingertip can be displayed more smoothly on the liquid crystal panel 1a of the display device 1.

  In the first to third embodiments, a binarized image is compared to create a display image. However, a non-binarized image is compared to create a display image. You may make it do.

  Moreover, in the said 1st thru | or 3rd embodiment etc., although the light sources 12a and 12b of the illuminating device 12 are arrange | positioned in the left-right side lower position of the fingertip F, in addition to or instead of this, for example, the front-rear side lower position of the fingertip F You may arrange in. According to this, a further reduced fingertip image can be created.

1 is an overall view schematically showing a car navigation device NV provided with an operation object extraction device according to the present invention. FIG. 2 is a block diagram illustrating the operation object extraction device in FIG. 1. Explanatory drawing which shows the state in which the driver | operator's fingertip was mounted in the operation panel of FIG. The flowchart which shows the imaging | photography control program which concerns on 1st Embodiment of this invention and is performed by the imaging | photography control part of FIG. The flowchart which shows the image processing program which concerns on 1st Embodiment of this invention and is performed by the image creation control part of FIG. The time chart which concerns on 1st Embodiment of this invention and shows the relationship between lighting / extinction of each light source of the illuminating device of FIG. 2, and the imaging | photography timing of a camera. (A) is explanatory drawing which shows the state which only the left light source turned on. (B) is explanatory drawing which shows the state which only the light source of the right side lighted. (A) is explanatory drawing which shows an image at the time of the left light source lighting. (B) is explanatory drawing which shows an image at the time of right light source lighting. (C) is explanatory drawing which shows the extraction image of the common area | region of (a), (b). Explanatory drawing which shows the state which both the light sources lighted. Explanatory drawing which shows an image at the time of both light source lighting. Explanatory drawing which shows the illumination image reflected on an operation panel in the state of FIG. (A) is explanatory drawing which shows the state which only the left light source turned on. (B) is explanatory drawing which shows the state which only the light source of the right side lighted. (A) is explanatory drawing which shows the illumination image acquired when only the left side light source lights. (B) is explanatory drawing which shows the illumination image acquired when only the right side light source lights. (C) is explanatory drawing which shows a display image when the common part of (a) and (b) is extracted. The flowchart which shows the imaging | photography control program which concerns on 2nd Embodiment of this invention and is performed by the imaging | photography control part of FIG. The flowchart which shows the image processing program which concerns on 2nd Embodiment of this invention and is performed by the image creation control part of FIG. The time chart which concerns on 2nd Embodiment of this invention and shows the relationship between lighting / extinction of each light source of the illuminating device of FIG. 2, and the imaging | photography timing of a camera. The flowchart which shows the imaging | photography control program which concerns on 3rd Embodiment of this invention and is performed by the imaging | photography control part of FIG. The time chart which concerns on 3rd Embodiment of this invention, and shows the relationship between lighting / extinction of each light source of the illuminating device of FIG. 2, and the imaging | photography timing of a camera. The time chart which concerns on modification of 3rd Embodiment and shows the relationship between lighting / extinction of each light source of the illuminating device of FIG. 2, and the imaging | photography timing of a camera. Explanatory drawing showing the problem in a prior art.

Explanation of symbols

NV car navigation device 1 Display device (display unit)
DESCRIPTION OF SYMBOLS 1a Liquid crystal panel 10 Operation object extraction apparatus 11 Camera 12 Illumination apparatus 12a, 12b Light source 13 Memory 14 Operation panel 15 Hand detection sensor 16 Imaging | photography control part (a control means, imaging | photography control means)
17 Image creation control unit (control means, image creation control means)

Claims (9)

An operation panel is arranged corresponding to the display unit of the in-vehicle device so that it can be remotely operated, the operator's fingertip on the operation panel surface is photographed with a camera, and the photographed fingertip image is superimposed and displayed on the display unit In-vehicle operating object extraction device
An in-vehicle operation object extraction device comprising control means for generating the fingertip image so that an area occupied by an operator's fingertip is smaller than an area of the operation panel surface. A plurality of illumination means for illuminating the fingertip of the operator at different angles,
The control means is a photographing control means for causing the camera to photograph the fingertip of the operator with different lighting modes of the plurality of illumination means
Image creation control means for creating a fingertip image to be displayed on the display unit based on at least a first fingertip image and a second fingertip image captured in a state in which the lighting modes of the plurality of illumination means are different; ,
An in-vehicle operating object extraction device according to claim 1, comprising:   The camera is arranged so as to be capable of photographing a surface of the operator's fingertip opposite to the surface exposed to external light, and the plurality of illumination means includes a left-side illumination means for illuminating the operator's fingertip from the lower left side, and a right-hand side And the right side illumination means that illuminates from below. The photographing control means causes the camera to photograph the first fingertip image with only the left side illumination means turned on, and only the right side illumination means is lit. The second fingertip image is set to be captured by the camera in a state where the first fingertip image is compared with the second fingertip image. The in-vehicle operation object extraction device according to claim 2, wherein the operation object extraction device is set so as to extract.   The said image creation control means binarizes said 1st fingertip image and said 2nd fingertip image, respectively with a predetermined | prescribed brightness value as a reference | standard, The common part of these binarized images is extracted. In-vehicle operation object extraction device.   The in-vehicle use according to claim 3 or 4, wherein the photographing control means sets the photographing timing of the camera to a predetermined cycle, and alternately turns on the left illumination means and the right illumination means in accordance with the photographing timing. Operating object extraction device.   The camera is arranged so as to be capable of photographing a surface of the operator's fingertip opposite to the surface exposed to external light, and the plurality of illumination means includes a left-side illumination means for illuminating the operator's fingertip from the lower left side, and a right-hand side Right side illumination means that illuminates from below, and the imaging control means causes the camera to capture the first fingertip image with only the left side illumination means turned on, and only the right side illumination means is provided. It is set so that the camera takes the second fingertip image in a lighted state, and the camera takes the third fingertip image in a state in which the left illumination means and the right illumination means are turned off. The image creation control means creates a fingertip image to be displayed on the display unit based on the first fingertip image, the second fingertip image, and the third fingertip image. Car Use operation object extracting apparatus.   The image creation control means calculates a first binary value by subtracting a luminance value between corresponding portions of the first fingertip image and the third fingertip image and binarizing with a predetermined luminance value as a reference A second binary image obtained by obtaining a difference image, calculating a difference between luminance values of corresponding portions in the second fingertip image and the third fingertip image, and binarizing with a predetermined luminance value as a reference The in-vehicle operating object extraction device according to claim 6, wherein a digitized difference image is acquired and a common part is extracted by comparing the first binarized difference image and the second binarized difference image.   The photographing control means sets the photographing timing of the camera at a predetermined cycle, and the photographing timing is set to a lighting state of one of the left illumination means and the right illumination means, the left illumination means and the right illumination. The in-vehicle operating object extraction device according to claim 6 or 7, wherein the on-vehicle operating object extraction device is set so as to sequentially match one of the other lighting states and both the left illumination unit and the right illumination unit.   The photographing control means sets the photographing timing of the camera at a predetermined period, and the photographing timing is set to one of the left illumination means and the right illumination means, the one extinguished state, the left side The in-vehicle operating object extraction device according to claim 6 or 7, wherein the on-vehicle operating object extraction device is set so as to sequentially match the other lighting state and the other lighting state of the lighting unit and the right side lighting unit.

Images