JP2002133401A - Operator-discriminating method and operator- discriminating device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operator discriminating method and an operator- discriminating device capable of discriminating the occupant sitting on which seat of a vehicle has operated a controller, etc., of an on-vehicle device. SOLUTION: An inner mirror for the vehicle is provided with a CMOS camera 46 via which the arm of the occupant operating the controller 38 is imaged. Image data obtained by the image is subjected to enhancing, erasing and extracting processing by an image processing control part 42, as well as discriminating processing. With these processes pixels estimated as the arm of the occupant on a front passenger seat among the constituting the image data are left, and when the remaining pixels are discriminated as the arm of the occupant on the front passenger seat, the occupant operating the controller 38 is discriminated as the occupant on the front passenger seat.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to, for example, an operator judgment for judging from which side of a driver or a passenger seat an operation is performed in various devices such as a car navigation device mounted on a vehicle. The present invention relates to a method and an operator determination device.

[0002]

2. Description of the Related Art With the increase in the number of functions of audio devices and car navigation devices mounted on vehicles, it is difficult to provide independent switches for selecting the functions of the above devices on a control panel or the like. Has become. For this reason, in recent years, a virtual function selection button is displayed on the screen of the monitor TV for image output, and similarly, a desired function selection button is selected while moving a cursor or the like displayed on the screen of the monitor TV with the controller. This allows the function to be executed. Furthermore, when performing a so-called hierarchical operation of selecting a detailed function belonging to the function selected in this manner, the image of the function selection button displayed on the monitor TV screen is switched to the desired function. There is also an apparatus that displays a function selection button corresponding to various detailed functions to which the function belongs and executes the same by selecting a desired detailed function selection button while moving a cursor or the like with a controller.

[0003] By the way, in a configuration in which the function selection buttons displayed on the monitor television screen are changed by such a hierarchical operation, a reliable operation cannot be performed unless the monitor television screen is watched. . In consideration of such points, it is preferable that the operation by the driver while the vehicle is running is restricted, and the development of an audio device, a car navigation device, and the like to which an operation restriction function is added while the vehicle is running has been promoted. I have.

[0004]

However, the operation restriction during the running of the vehicle should basically be applied only to the driver. Therefore, if the operation restriction function as described above is added, for example, the operation by the passenger in the passenger seat becomes impossible.

The present invention has been made in consideration of the above facts and provides an operator determining method and an operator capable of determining which position of a vehicle is operated by an occupant of a vehicle-mounted device such as an audio device or a car navigation device. The purpose is to obtain a determination device.

[0006]

According to a first aspect of the present invention, an occupant on either side of an operating body provided on a vehicle operates the operating body on an extension of the operating body. An imaging method for imaging the arm of the occupant operating the operated object by an imaging unit from a predetermined position at which the extended object of the operated object can be imaged,
A determination process of determining from which side of the arm the boundary extends from the operated body based on the shape of the outline of the arm captured by the imaging process. And

[0007] In the operator determination method having the above structure, an image pickup means capable of picking up an image extending from the operated body is provided.
The image of the arm of the vehicle occupant operating the operated body is captured by the imaging means (imaging processing).

Here, if the driver's seat is provided on one side and the passenger's seat is provided on the other side of the above-mentioned extension, the occupant tries to operate the operated body from the driver's seat side. If you do, the driver's arm of the driver's seat
The shape of the contour line is such that the entire arm or the portion of the arm from the shoulder to the elbow is inclined in the direction toward the operated body with respect to the passenger seat direction.

On the other hand, when the occupant tries to operate the operated body from the passenger seat side, the shape of the contour of the occupant on the passenger seat side to be operated is the entire arm or the shoulder of the arm. From the elbow to the elbow in a direction inclined toward the operated body with respect to the driver's seat direction.

In the determination process, the shape (angle) of the contour line of the arm is determined to reliably determine whether the operated operation is performed by the driver's seat-side occupant or the passenger's seat-side occupant. Is done. Further, for example, by adopting this determination method for various devices such as a car navigation device and an audio device mounted on a vehicle, different operations are performed when operated from the driver seat side and when operated from the passenger seat side. Restrictions can be made.

According to a second aspect of the present invention, in the operator judging method according to the first aspect, a numerical value indicating the luminance of each pixel forming the image data picked up by the image pickup means is substituted into an emphasis calculation expression. A high-luminance pixel is used as an emphasis pixel, and emphasis processing for emphasizing a difference from other low-luminance pixels, and a pseudo contour formed by a set of the emphasis pixels, from one side bordering the extension. Erases the emphasized pixels constituting the pseudo contour line directed in the erase candidate direction corresponding to the direction inclined from the extension to the operated body side with respect to the direction toward the other side, and other than the erased emphasized pixels Erasing processing that leaves the emphasized pixel as a residual pixel, and, among the pseudo contours composed of the remaining pixels, all remaining pixels that constitute a pseudo contour having characteristics corresponding to the arm of the occupant are extracted pixels. Pixel An extraction process for extracting the pseudo contour from among the extracted contours based on at least one of the position, the length, and the direction of the pseudo contour formed by the extracted pixels. It is characterized in that it is determined whether or not it is the contour of the arm of the occupant on the other side that borders on the extension.

In the operator determination method having the above configuration, each pixel constituting the image data obtained by the imaging process is subjected to the enhancement process. In this emphasis processing, a numerical value indicating the luminance of each pixel constituting the captured image data is substituted into the emphasis calculation equation, whereby the numerical value of a pixel having a high luminance is further increased and the numerical value of a pixel having a low luminance is conversely calculated. Are comparable or even smaller. Therefore, pixels forming an image in this state are pixels whose luminance was originally high become emphasis pixels, and an image formed by this emphasis pixel is replaced with an image formed by another pixel (a pixel whose luminance was originally low). It will be clearer than that. It is presumed that this set of emphasized pixels basically becomes a pseudo contour corresponding to the contour of the seat such as the occupant's arm, the driver's seat, and the passenger's seat.

Further, an erasing process is performed on these emphasized pixels. In the erasing process, a candidate erasing direction corresponding to a direction inclined from the above-mentioned extension to the object to be operated with respect to the direction from one side to the other with respect to the extension from the object to be operated in each emphasized pixel. The emphasized pixels forming the false contour line directed toward are erased, and the other emphasized pixels are left as remaining pixels.

If the driver's seat is provided on one side and the passenger's seat is provided on the other side of the above-mentioned extension, the occupant tries to operate the operated body from the driver's seat side. If you do, the driver's arm of the driver's seat
The entire arm or the part of the arm from the shoulder to the elbow will be directed in a direction inclined toward the operated body with respect to the passenger seat direction. Naturally, the contour of the arm in this case is a deletion candidate corresponding to this direction. You will be facing the direction. Therefore, it is very likely that the pseudo contour line formed by the emphasized pixels deleted by the deletion process is the contour line of the arm of the driver's seat occupant (in other words, the pseudo contour line formed by the remaining pixels). Is basically assumed to correspond to any one of the contour lines of the arm of the passenger seat, the driver's seat, and the passenger seat).

Further, in this state, extraction processing is performed on the remaining pixels. In this extraction processing, a numerical value indicating the luminance of each remaining pixel is substituted into the extraction operation expression, or a numerical value indicating the luminance of each remaining pixel is compared with numerical data of a preset comparison table. Here, the contours of the seats such as the driver's seat and the passenger's seat and other structures installed in the vehicle cabin basically do not change, or even if they do change, the difference is small or changes in a certain pattern. Therefore, it is possible to assume in advance what kind of residual pixel the pseudo contour line indicates the above-mentioned structure. Therefore, as a result of the above calculation or comparison, the remaining pixels that cannot be regarded as the remaining pixels constituting the pseudo contour corresponding to the preset contour of the structure are extracted as the extracted pixels. As a result, the pseudo contour formed by the extracted pixels is estimated as a pseudo contour corresponding to the occupant's arm.

Further, at least one of the position, length, and direction of the pseudo contour line constituted by these extracted pixels is the other side of the extension from the operated body described above in the determination processing. It is determined whether the position, length, and direction of the arm of the occupant correspond to any one of the above.

By performing the above processing, for example, when the operated body is provided between the driver's seat and the passenger seat, the operated operation is performed by the driver's seat occupant or the passenger seat side. It is reliably determined whether the operation has been performed by the occupant.

According to a third aspect of the present invention, in the operator determining method according to the second aspect, a pixel adjacent to any one of the emphasized pixels in a direction along the extension and the operated pixel When the pixel adjacent to any one of the erasure candidate direction and the opposite direction with respect to the body direction was the emphasized pixel, the erasure processing was performed by setting any one of the pixels as a remaining pixel. It is characterized by:

According to the operator determination method having the above-described configuration, in the erasing process, a pixel adjacent to any one of the emphasized pixels in a direction corresponding to a direction along an extension from the operated body is the emphasized pixel. Further, when a pixel adjacent to any one of the pixels in the erasing direction and the opposite direction is an enhancement pixel, any one of the pixels is determined as a remaining pixel.

That is, as described above, when the occupant on one side of the extension from the operated object attempts to operate the operated object, the outline of the arm of the occupant is displayed on the entire arm or the arm. The part corresponding to the region from the shoulder to the elbow faces the erasure candidate direction.

Therefore, when the pseudo contour line corresponds to the contour line of the arm of the occupant located on one side of the extension from the operated body as a boundary, the emphasis pixels constituting this pseudo contour line are as described above. The possibility of satisfying the condition for determining the remaining pixels in the present invention is extremely low. This ensures that the emphasized pixels forming the pseudo contour corresponding to the arm of the occupant located on one side of the extension from the operated body are selected from the emphasized pixels constituting the plurality of pseudo contours. As a result, in the final determination process, the pseudo contour corresponding to the occupant's arm located on one side of the extension from the operated body is set as a boundary on the extension from the operated body. The possibility of erroneously determining a false contour line corresponding to the arm of the occupant located on the other side can be eliminated or extremely reduced.

According to a fourth aspect of the present invention, the operator determining device is provided at a predetermined position where an image of an extension of the object to be operated provided on the vehicle can be taken, and the object to be operated is viewed from any side of the extension. Imaging means for imaging the arm of the occupant operating the arm, and the arm extending from any side with respect to the extension of the operated body based on the shape of the outline of the arm imaged by the imaging means. Determination means for determining whether

According to the operator judging device having the above structure, the image pickup means is provided at a position where the image can be taken on an extension of the object to be operated provided on the vehicle. The occupant's arm is imaged.

Here, if the driver's seat is provided on one side and the passenger's seat is provided on the other side of the above-mentioned extension, the occupant attempts to operate the operated body from the driver's seat side. If you do, the driver's arm of the driver's seat
The shape of the contour line is such that the entire arm or the portion of the arm from the shoulder to the elbow is inclined in the direction toward the operated body with respect to the passenger seat direction.

On the other hand, when the occupant attempts to operate the operated body from the passenger seat side, the shape of the contour of the occupant on the passenger seat side to be operated is the entire arm or the shoulder of the arm. From the elbow to the elbow in a direction inclined toward the operated body with respect to the driver's seat direction.

Therefore, the determining means reliably determines whether this arm is, for example, the driver's arm or the passenger's arm, based on the inclination of the shape of the contour line of the image data obtained by the imaging means. Is done. Thus, for example, by adopting the present determination method for various devices such as a car navigation device and an audio device mounted on a vehicle, there is a difference between when operated from the driver's seat side and when operated from the passenger seat side. Operation restriction can be performed.

As described above, the present invention is configured to include the image pickup means and the determination means. For example, a CPU or the like of a computer may include any one of these components in one member or an electric element. (In other words, these components need not be independent members or elements).

The operator determination device does not need to be a single independent device. For example, when the functions of the present invention are added to a car navigation device or an audio device for a vehicle, these devices may be used. A car navigation device and an audio device are also used as the operator determination device.

According to a fifth aspect of the present invention, in the operator judging apparatus according to the fourth aspect, a numerical value indicating the luminance of each pixel forming the image data picked up by the image pickup means is substituted into an emphasis calculation expression. Emphasizing means for emphasizing a pixel having a high luminance as an emphasized pixel, and emphasizing a difference from other pixels having a low luminance, and a pseudo contour formed by a set of the emphasized pixels, from one side bordering the extension. Erases the emphasized pixels constituting the pseudo contour line directed in the erase candidate direction corresponding to the direction inclined from the extension to the operated body side with respect to the direction toward the other side, and other than the erased emphasized pixels Erasing means for leaving the emphasized pixel as a residual pixel, and, among the false contour lines composed of the remaining pixels, all remaining pixels constituting the false contour line having characteristics corresponding to the arm of the occupant as extracted pixels. Pixel Extracting means for extracting the pseudo contour from the extracted contour based on at least one of the position, length, and direction of the pseudo contour formed by the extracted pixels. It is characterized in that it is determined whether or not it is the contour of the arm of the occupant on the other side that borders on the extension.

According to the operator judging device having the above structure, each pixel constituting the image data obtained by the imaging means is emphasized by the emphasizing means. In the emphasizing means, a numerical value indicating the luminance of each pixel is substituted into a previously set emphasis calculation expression. As a result, the numerical value of a pixel having a high luminance is further increased, and the numerical value of a pixel having a low luminance is substantially the same or smaller. Therefore, pixels forming an image in this state are pixels whose luminance was originally high become emphasis pixels, and an image formed by this emphasis pixel is replaced with an image formed by another pixel (a pixel whose luminance was originally low). It will be clearer than that. It is presumed that this set of emphasized pixels basically becomes a pseudo contour line corresponding to at least one contour line of a structure installed in a vehicle including a passenger's arm, a driver's seat, a passenger seat, and the like.

Further, among these emphasized pixels, the direction corresponds to a direction inclined from the above extension to the object to be operated with respect to the direction from one side to the other with respect to the extension from the object to be operated. The emphasizing pixels forming the pseudo contour line facing the erasure candidate direction are erased by the erasing means, and the other emphasis pixels are left as remaining pixels.

Here, if the driver's seat is provided on one side and the passenger's seat is provided on the other side on the above-mentioned extension, the occupant tries to operate the operated body from the driver's seat side. If you do, the driver's arm of the driver's seat
The entire arm or the part of the arm from the shoulder to the elbow will be directed in a direction inclined toward the operated body with respect to the passenger seat direction. Naturally, the contour of the arm in this case is a deletion candidate corresponding to this direction. You will be facing the direction. Therefore, it is very likely that the pseudo contour line formed by the emphasized pixels deleted by the deletion process is the contour line of the arm of the driver's seat occupant (in other words, the pseudo contour line formed by the remaining pixels). Is basically assumed to correspond to any one of the contour lines of the arm of the passenger seat, the driver's seat, and the passenger seat).

Next, for these remaining pixels, a numerical value indicating the luminance of each remaining pixel is substituted into an extraction operation formula set in advance in the extracting means, or the numerical data of the preset comparison table and the luminance of each remaining pixel are substituted. The numerical values shown are compared.
Here, the outline of the structure such as the seat described above does not basically change, or even if it changes, the difference is small or changes only in a fixed pattern. It can be assumed in advance whether a pseudo contour line composed of pixels indicates a structure. Therefore,
As a result of the above calculation or comparison, the remaining pixels which cannot be regarded as the remaining pixels constituting the pseudo contour corresponding to the preset contour of the structure are extracted as the extracted pixels. As a result, the pseudo contour formed by the extracted pixels is estimated as a pseudo contour corresponding to the occupant's arm.

Further, at least one of the position, the length and the direction of the pseudo contour line constituted by these extracted pixels is the arm of the occupant on the other side bordering on the above-mentioned extension from the operated body. It is determined by the determination means whether or not the position, the length, and the direction correspond to any one of the above.

As described above, in the operator determination device,
It can be reliably determined that the operated body has been operated by the occupant on either side of the extension from the operated body.

As described above, the present invention includes the emphasizing means, the erasing means, and the extracting means. For example, one of the components such as a CPU of a computer or an electric element May have the function of an arbitrary plurality of components (in other words, these components need not be independent members or elements).

According to a sixth aspect of the present invention, in the operator judging device according to the fifth aspect, a control section of a predetermined device that operates based on an operation signal from the operated body and the operated body. And a restricting means connected to the determining means for restricting the operation of the predetermined device based on a result of the determination by the determining means.

In the operator determining apparatus having the above-described structure, the control unit of a predetermined device (for example, a car navigation device) that operates based on an operation signal from the operated body when the operated body is operated, and the operated body The limiting means intervenes at least logically, and an operation signal from the operated body is sent to the limiting means once and then sent to the control unit. Here, the limiting unit is connected to the determining unit, and the determination result of the determining unit is sent to the limiting unit, and based on the received determination result, regardless of the content of the operation signal from the operated body, The operation is restricted by the restriction means.

Accordingly, even when the same operation is performed by the above-described predetermined device, the operation is performed by the operation of the occupant on one side or the other side with respect to the extension from the operated body. Can be prevented.

As an example of the relationship between the result of the judgment by the judging means and the limitation of the operation by the restricting means, the judging result from the judging means by the restricting means and the operation signal from the operated body are obtained in any state. Inquiring of a preset restriction pattern as to whether or not to execute the operation restriction is performed, and the restriction means is made to execute the restriction of the content of the corresponding restriction pattern.

[0041]

FIG. 1 shows a car navigation apparatus 10 employing an operator determination method according to an embodiment of the present invention. FIG. 2 is a perspective view of the interior of a vehicle 12 equipped with a car navigation device 10) having a part of the function, as viewed from substantially the rear left side of the vehicle. FIG. Indicated by

As shown in these figures, the car navigation device 10 includes a device main body 14. FIG.
As shown in FIG. 1, the apparatus main body 14 is housed in a housing portion 18 formed on an instrument panel 16 of the vehicle 12. As shown in FIG. 2, the apparatus main body 14 includes a navigation unit 20 and an image processing unit 22. The navigation unit 20 includes a navigation control unit 24. The navigation control unit 24 includes a CPU and the like, and is connected to the navigation storage medium 26. Various data and functional programs such as map information, a search program, and an output image processing program are stored in the navigation storage medium 26 in advance.
The data and the function program stored in the CPU are read and executed.

The navigation control unit 24 is connected to a monitor unit 28. As shown in FIG. 1, the monitor unit 28 is integrated with the apparatus main body 14, and the screen surface is exposed in a state where the screen faces substantially toward the rear of the vehicle. The above-described navigation control unit 24 displays an image on the screen of the monitor unit 28 based on the program or data read from the navigation storage medium 26 and executed, and the navigation control unit 24 displays the image on the screen of the monitor unit 28. Thus, programs and data executed by the occupants of the vehicle 12 (for example, the occupant 32 on the driver's seat 30 side and the occupant 36 on the passenger seat 34 side shown in FIGS. 3 and 4) can be confirmed.

The navigation control section 24 is connected to a controller 38 as an operated body. 1, 3, and 4
As shown in FIG. 3, the controller 38 is provided substantially in front of a console box 40 provided between the driver's seat 30 and the passenger seat 34 of the vehicle 12, and the occupant 32 (hereinafter referred to as “driving”) on the driver's seat 30 side. When the occupant 32 on the seat 30 side is simply referred to as “occupant 32” or the occupant 36 on the passenger seat 34 side (hereinafter, the occupant 36 on the passenger seat 34 side is simply referred to as “occupant 36”), the controller 38 is operated. 3
8 is sent to the navigation control unit 24, and the navigation control unit 24 basically executes a process according to the operation signal.

On the other hand, as shown in FIG. 2, the image processing unit 22 includes an image processing control unit 42 as an emphasizing unit, an erasing unit, an extracting unit, a determining unit, and a limiting unit.
The image processing unit 22 is also configured by a CPU or the like, like the navigation control unit 24, and is connected to the image processing storage medium 44. The image processing storage medium 44 stores in advance various programs such as an image processing program, a determination program, and a restriction program, which will be described later. The image processing control unit 42 executes each program stored in the image processing storage medium 44. Read and execute.

The image processing control unit 42 is connected to the navigation control unit 24 and serves as an image pickup means.
MOS (CMOS: phase correction metal oxide semiconductor) camera 4
6 and a lamp 48. As shown in FIG. 1, the CMOS camera 46 is provided integrally with an inner mirror 52 attached to the interior of the vehicle 12, and as shown in FIG. 3 and FIG.
8, the lens direction and the like are set so that the front passenger seat 34 can be imaged (photographed) closer to the front passenger seat 34 than the center between the driver seat 30 and the front passenger seat 34 in front of the console box 40. An image is taken based on an image signal from the control unit 42. In the present embodiment, the CMOS camera 46 is used as the imaging means, but the imaging means is not limited to this. A so-called CCD camera or the like can be used as the imaging means. Further, an infrared camera capable of detecting infrared rays emitted from a human body may be used.

On the other hand, although not shown in FIG. 1, the lamp 48 is located near the CMOS camera 46 and the controller 3.
For example, it is provided at a position where the portion to be imaged by the CMOS camera 46 can be appropriately illuminated, such as in the vicinity of 8, and illuminates the imaging range of the CMOS camera 46 based on a projection signal from the image processing control unit 42.

As shown in FIG. 2, the image processing control section 42 is connected to a vehicle speed sensor 52 provided at an appropriate position of the vehicle 12. The vehicle speed sensor 52 detects the speed of the vehicle 12 and transmits a speed signal that is an electric signal corresponding to the detected speed of the vehicle 12 to the image processing control unit 42.

In this embodiment, each of the navigation unit 20 and the image processing unit 22 has a control unit (ie,
The configuration includes the navigation control unit 24 and the image processing control unit 42) and the storage medium (that is, the navigation storage medium 26 and the image processing storage medium 44).
For example, the functions of the image processing control unit 42 are added to the navigation control unit 24 so that the navigation control unit 24 is used only as the navigation control unit 24, or the image processing storage medium 44 is used.
The data and the program in the storage medium 26 for navigation may be stored in advance, and the navigation control unit 24 may read the data and the program from the storage medium 44 for image processing.

Next, the operation and effect of this embodiment will be described.

In the car navigation device 10, when the occupant 32 or the occupant 36 extends the arm 54 or the arm 56 toward the controller 38 and operates the car navigation device 10 by touching the controller 38,
An operation signal is sent from the controller 38 to the navigation control unit 24. The navigation control unit 24 reads a predetermined program or data from the navigation storage medium 26 or changes an image or the like displayed on the screen of the monitor unit 28 based on an operation signal from the controller 38.

The navigation control unit 24 that has received the operation signal from the controller 38 transmits a confirmation signal, which is an electric signal corresponding to the operation signal, to the image processing control unit 42.
The image processing control unit 42 that has received the confirmation signal from the navigation control unit 24 reads the image processing program from the image processing storage medium 44. Hereinafter, description will be made with reference to the schematic flowchart of the image processing program shown in FIG.

Step 1 for reading the image processing program
The image processing control unit 42 that started the program at 00
Reads the signal V from the vehicle speed sensor 52 in step 102, and pre-stores the image processing storage medium 44 in step 104.
Is compared with the speed data Vs for comparison read together with the image processing program. According to the comparison, if the vehicle 12 is not at or above the predetermined speed (Vs = 0 <V in the case of determining whether or not the vehicle is running), step 1 is performed.
Proceeding to 06, the system waits for a confirmation signal from the navigation control unit 24.

On the other hand, when the vehicle 12 is traveling at a predetermined speed or higher, the image processing control unit 42
8 and transmits an image signal synchronized with the light emission signal to the CMOS camera 46.

The lamp 48 that has received the light projection signal illuminates the above-described imaging range between the driver's seat 30 and the passenger's seat 34 singly, intermittently, or continuously. The CMOS camera 46 having received the image captures the above-described imaging range. Here, the imaging signal is transmitted from the image processing control unit 42 by receiving the operation signal from the controller 38 by the navigation control unit 24 and transmitting a confirmation signal accompanying the operation signal from the navigation control unit 24 to the image processing control unit 42. Things. Therefore, when the CMOS camera 46 receives the imaging signal, the controller 38 is operated by the arm 54 extended by the occupant 32, or is operated by the arm 56 extended by the occupant 36. Therefore, the arm 54 of the occupant 32 or the arm 56 of the occupant 36 operating the controller 38 is captured in the image captured by the CMOS camera 46.

The CMOS camera 4 thus obtained
The image data captured by 6 is sent to the image processing control unit 42. In step 110, the image processing control unit 42 receives the image data. Next, in step 112, the image data in the processing range S indicated by a dashed line is substantially located at the center of the imaging range illustrated in FIG. Enhancement processing is performed on the pixels constituting the data.

That is, as shown in FIG. 6, the pixels constituting the image data are arranged in a matrix.
Here, assuming that a numerical value indicating the luminance of each pixel is X, the numerical value X of the luminance of each pixel is substituted into Expression (1) that forms the emphasis calculation expression. Yi, j = A (Xi−1, j + Xi, j−1 + Xi + 1, j + Xi, j + 1) −B · Xi, j (1) where A <B and A = 1 in general B = 4 is frequently used. Since equation (1) is a so-called Laplacian filter generally used in edge detection of an image, a detailed description is omitted. Further, a numerical value Xi, j indicating the luminance of each pixel is converted from a numerical value X
The value Yi, j obtained by the above equation (1) corresponding to i, j is subtracted to calculate the value Zi, j of the emphasized luminance. Zi, j = Xi, j-Yi, j (2) As a result of the above processing, the value of the emphasis luminance Zi, j is further increased in a pixel having originally high luminance, and is enhanced in a pixel having low luminance originally. The brightness value Zi, j becomes smaller. As a result, a pixel having originally high luminance becomes an emphasized pixel, and an image formed by this emphasized pixel has a large difference from other pixels (pixels having originally low luminance).

Here, in general, since the luminance change is relatively large in the edge portion (contour portion) of the image, by performing the above processing, the pixel of the edge portion becomes an emphasized pixel and the edge portion becomes clear (FIG. 10 (A), (B)). Thus, the edge portion of the image captured by the CMOS camera 46, that is, the arm 54 of the occupant 32 and the arm 56 of the occupant 36
Will be indicated by a pseudo outline formed by the emphasized pixels.

Next, at step 114, an erasing process is performed on the emphasized image obtained by the above-described emphasizing process. In this erasing process, the pixel Ki, j-1 adjacent to the right side of the vehicle 12 on the side corresponding to the rightward direction of the vehicle 12 corresponds to the leftward direction of the vehicle 12 with respect to the emphasized pixel Ki, j whose luminance has been further increased by the above-described enhancing process. The pixel Ki, j + 1 adjacent on the side to perform, and the pixel Ki-j adjacent to the pixel Ki, j-1 on the side corresponding to the forward direction of the vehicle 12.
It is determined whether all of the pixels Ki + 1, j + 1 adjacent to the rear side of the vehicle 12 with respect to the pixel j, j−1 and the pixel Ki, j + 1 are emphasized pixels. Is not an emphasized pixel, the pixel Ki, j is deleted.

Hereinafter, with respect to the emphasized pixels forming the pseudo contours corresponding to the arms 54 of the occupant 32 and the arms 56 of the occupant 36, there is a high possibility that the pixels will be erased, or they will remain as remaining pixels without being erased. Explain whether the possibility is high.

When the occupant 32 operates the controller 38, the direction of the entire arm 54 or the portion extending from the shoulder to the elbow is inclined leftward with respect to the front side of the vehicle 12. Therefore, as shown in FIG. 10A, at least a part of the false contour line L <b> 1 corresponding to the arm 54 of the occupant 32 (that is, a part from the shoulder to the elbow) is leftward with respect to the front side of the vehicle 12. It becomes a continuous line along the inclined direction.

As described above, in the case where the pseudo contour line L1 is a line continuing along the direction inclined leftward with respect to the front side of the vehicle 12, the pseudo contour line L1 moves rightward of the vehicle 12 with respect to the emphasized pixel Ki, j. The pixel Ki, j-1 adjacent on the corresponding side and the pixel Ki, j + 1 adjacent on the side corresponding to the left direction of the vehicle 12 are emphasized pixels, but the pixel Ki, j-1 of the vehicle 12 is Pixels Ki-1, j-1 and Ki, j + 1 adjacent on the side corresponding to the forward direction
Pixel K adjacent to the rear side of the vehicle 12
It is difficult for i + 1 and j + 1 to become emphasized pixels.
-1 and a pixel Ki + 1, j-1 adjacent on the side corresponding to the backward direction of the vehicle 12 with respect to the pixel Ki-1, j-1 and a pixel Ki-1 adjacent on the side corresponding to the front of the vehicle 12 with the pixel Ki, j + 1. , j + 1 tend to be emphasized pixels.

Therefore, the emphasized pixels forming the pseudo contour line L1 corresponding to the arm 54 of the occupant 32 do not satisfy the above-described conditions, and are very likely to be erased by this erasing process.

On the other hand, as shown in FIGS. 8 and 9, when the occupant 36 operates the controller 38, the direction of the entire arm 56 or the portion from the shoulder to the elbow is directed to the front side of the vehicle 12. Turn right in the direction of inclination. Therefore, as shown in FIG. 10 (B), at least a part of the pseudo contour line L2 corresponding to the arm 56 of the occupant 36 (that is, a portion from the shoulder to the elbow) is rightward with respect to the front side of the vehicle 12. It becomes a continuous line along the inclined direction.

As described above, in the case where the pseudo contour line L2 is a continuous line along the direction inclined rightward with respect to the front side of the vehicle 12, the pseudo contour line L2 moves rightward of the vehicle 12 with respect to the emphasized pixel Ki, j. The pixel Ki, j-1 adjacent on the corresponding side and the pixel Ki, j + 1 adjacent on the side corresponding to the left direction of the vehicle 12 are emphasized pixels. Of pixels Ki-1, j-1 and Ki, j + adjacent on the side corresponding to
Pixels Ki + 1 and j + 1 adjacent to the rear side of the vehicle 12 on the side corresponding to the rear of the vehicle 12 are also likely to be emphasized pixels.

Therefore, there is a very high possibility that the emphasized pixels constituting the pseudo contour line corresponding to the arm 56 of the occupant 36 are not erased even in the erasing process and remain as remaining pixels.

As described above, the pseudo contour L1 corresponding to the arm 54 of the occupant 32 operating the controller 38
Is deleted, but the pseudo contour L2 corresponding to the arm 56 of the occupant 36 operating the controller 38 is not deleted.

In this state, if the pseudo contour formed by the emphasized pixels after the emphasizing process is
Alternatively, if only the arm 56 of the occupant 36 is used, the emphasizing pixels forming the pseudo outline corresponding to the arm 54 of the occupant 32 are erased by the above-described erasing process. It is also possible to determine that the line is the arm 56 of the occupant 36. However, vehicle 12
The driver's seat 30, the passenger's seat 34, the console box 40, and the like are inside the room, and the emphasized pixels forming a part of these pseudo contour lines may remain as residual pixels even after the erasing process. Therefore, although there is a very high possibility that the pseudo contour line composed of the remaining pixels after the above-described erasing process includes the one corresponding to the arm 56 of the occupant 36, it cannot be determined immediately after the erasing process.

For this reason, from step 116 to step 1
At 20, an extraction process and a determination process are performed on the pseudo contour constituted by the remaining pixels.

In step 116, first, FIG.
(A) and (B) are subjected to the eight-neighbor labeling process on the remaining pixels constituting each pseudo contour line.
Residual pixels connected in the vicinity sense are grouped. Next, at least one of the length, the position, the inclination, and the like is detected for each of the group of remaining pixels (that is, the pseudo contour) grouped in step 118. Further, in step 120, step 1
Based on the detection result at 18, it is determined whether or not each of the aggregates of the remaining pixels can be estimated as the pseudo contour of the arm 56 of the occupant 36, and the remaining pixels of the aggregate that can be estimated are extracted pixels as shown in FIG. The extraction is performed as shown in ()), and the flow proceeds to the determination processing after step 122. In FIG. 12B, only extracted pixels are shown for simplicity.
(In (A), there are no extracted pixels.) However, since the above-described labeling process makes it possible to distinguish between the extracted pixels and the remaining pixels other than the extracted pixels, the pixels other than the extracted pixels may be left.

In step 122, the processing range S is substantially 1: 2: 1 along a direction generally corresponding to the front-back direction of the vehicle 12.
Is divided into three. Next, in step 124, a set of extracted pixels existing in each of the divided ranges is compared with pattern data previously stored in the image processing storage medium 44 and read together with the image processing program. In this pattern data, when the arm 56 of the occupant 36 of the passenger seat 34 operates the controller 38, the position and direction of the arm 56 in each of the divided portions of the processing range S are set as a pattern. The pattern of the pattern data and the direction of the set of extracted pixels are compared for each divided portion of the processing range S, and the set of extracted pixels that is considered to match the pattern of the pattern data is sent from the passenger seat 34 to the controller 34. Arm 5 of crew 36 who operated 38
6 is determined to be a pseudo contour line.

As described above, in the present embodiment,
Although the division ratio of the processing range S is set to 1: 2: 1 generally along the front-rear direction of the vehicle 12, the division condition of the processing range S is not limited to this, and the division ratio of another condition may be set. It does not matter if it is not divided. That is, in the present embodiment, as shown in FIGS.
The occupant 3 operating the controller 38 in the processing range S
This is done in consideration of ensuring that the elbow portion of the sixth arm 56 is housed in one divided portion. Therefore, depending on various conditions such as the width of the processing range S and the aspect ratio, a more preferable division condition than the above condition may be found.

If it is determined in step 124 that the set of extracted pixels (that is, the pseudo contour) is the arm 56 of the occupant 36, the process waits again for a confirmation signal via step 108. On the other hand, at step 124, the aggregate of the extracted pixels (that is, the pseudo outline) is not the arm 56 of the occupant 36. That is, if it is determined that the controller 38 has been operated by a person other than the occupant 36 (in other words, the occupant on the driver's seat 30 side), a cancel signal is transmitted to the navigation control unit 24 in step 126 and the process proceeds to step 108 to confirm again. Wait for signal. Upon receiving the cancel signal from the image processing control unit 42, the navigation control unit 24 cancels the operation signal and does not perform an operation based on the operation signal including the operation of the monitor unit 28. Thereby, the operation of the controller 38 by the occupant 32 can be restricted, and the operation of the controller 38 by the occupant 36 can be permitted.

In this embodiment, the controller 3
Since the operation 8 is performed by non-contact without performing contact with the occupants 32 and 36 or applying a load when performing one of the occupants 32 and 36, the smooth operation of the occupants 32 and 36 on the controller 38 is performed. It does not hinder.

[0075]

As described above, according to the present invention, it is possible to determine on which side of the extension of the operated body the boundary has been operated by the occupant.

[Brief description of the drawings]

FIG. 1 is a perspective view of a vehicle equipped with a car navigation device having an operator determination device according to an embodiment of the present invention as a part of its functions.

FIG. 2 is a block diagram schematically showing a configuration of a car navigation device having an operator determination device according to an embodiment of the present invention as a part of its function.

FIG. 3 is a bird's-eye view showing an imaging range of an imaging unit in a state where a driver on a driver's seat side operates an operated body.

FIG. 4 is a bird's-eye view showing an imaging range of an imaging unit in a state where a passenger on the passenger's seat side operates an operated body.

FIG. 5 is a schematic flowchart of an image processing program.

FIG. 6 is a bird's-eye view showing one mode of the arm direction and the bent state when the passenger's seat side occupant operates the operated body.

FIG. 7 is a bird's-eye view showing another aspect of the arm direction and the bent state when the operated body is operated by the passenger on the passenger seat side. You.

FIG. 8 is a diagram illustrating a relationship between a pixel to be symmetrical in enhancement processing and pixels around the pixel;

FIG. 9 is a diagram illustrating a relationship between a pixel to be subjected to erasing processing and pixels around the pixel.

10A and 10B are diagrams illustrating a state of an image of an imaging unit after an enhancement process, wherein FIG. 10A illustrates a state in which a driver on the driver's seat side operates an operated body, and FIG. Shows the state of operating the body.

11A and 11B are diagrams illustrating a state of an image of an imaging unit after an erasing process, in which FIG. 11A illustrates a state in which a driver on the driver seat side operates an operated body, and FIG. Shows the state of operating the body.

12A and 12B are diagrams illustrating a state of an image of the imaging unit after the extraction processing, in which FIG. 12A illustrates a state in which a driver on the driver seat side operates an object to be operated, and FIG. Shows the state of operating the body.

13A and 13B are diagrams illustrating a state in which an image is divided at a predetermined ratio in a determination process, wherein FIG. 13A illustrates a state in which an occupant on the driver's seat side operates an operated body, and FIG. This shows a state in which the operation tool is operated.

[Explanation of symbols]

Reference Signs List 10 car navigation device 12 vehicle 30 driver's seat 32 occupant 34 passenger's seat 36 occupant 38 controller (operated body) 42 image processing control unit (enhancing means, erasing means, extracting means, determining means, limiting means) 46 CMOS camera (imaging means) ) 54 arms 56 arms

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H04N 7/18 H04N 7/18 K 5L096 (72) Inventor Michimasa Ito 3-260 Toyota, Oguchi-machi, Oguchi-machi, Niwa-gun, Aichi Prefecture Stock F-term (reference) in the Tokai Rika Electric Works, Ltd. FA06 FA64 FA67 FA69 GA19 JA11

Claims (6)

[Claims] 1. An operator determining method for determining which side of the boundary of an operated object provided on a vehicle has been operated by an occupant on both sides of the boundary, the method comprising: An imaging process of imaging the occupant's arm operating the operated object by an imaging unit from a predetermined position where the extension of the operation body can be imaged, based on a shape of a contour line of the arm imaged by the imaging process A determination process for determining from which side of the extension of the arm the boundary extends from the operated body. 2. The method according to claim 1, wherein a numerical value indicating the luminance of each pixel forming the image data picked up by the image pickup means is substituted into an emphasis calculation expression, so that a pixel having high luminance is set as an emphasized pixel, and a difference from other pixels having low luminance is set. Emphasizing processing, and of the pseudo contours formed by the set of emphasized pixels, from the extension side to the operated body side in the direction from one side bounding the extension to the other side. An erasing process for erasing the emphasized pixels constituting the pseudo contour line directed to the erasure candidate direction corresponding to the inclined direction, and leaving the emphasized pixels other than the erased emphasized pixels as remaining pixels; and Extraction processing of extracting, from among all remaining pixels, remaining pixels forming a pseudo contour having a feature corresponding to the occupant's arm, among the remaining pixels, among the pseudo contours. , Based on at least one of the position, length, and direction of the pseudo contour formed by the extracted pixels, whether the pseudo contour is the contour of the arm of the occupant on the other side bordering the extension. The method according to claim 1, wherein it is determined whether or not the operator is present. 3. A pixel adjacent to any one of the emphasized pixels in a direction along the extension, and any one of the pixels in the direction of the erasure candidate with respect to the direction of the operated body and in a direction opposite thereto. The method according to claim 2, wherein, when a pixel adjacent to one of the pixels is the emphasized pixel, setting the one of the pixels as a remaining pixel is the erasing process. 4. An image of an arm of an occupant operating the object to be operated, which is provided at a predetermined position capable of imaging an extension of the object to be operated provided on the vehicle from any side of the extension. Imaging means; and determination means for determining from which side the arm extends from the extension of the operated body based on the shape of the outline of the arm imaged by the imaging means, An operator determination device provided. 5. A method according to claim 1, wherein a value indicating the luminance of each pixel forming the image data picked up by the image pickup means is substituted into an emphasis calculation expression, so that a pixel having a high luminance is set as an emphasized pixel, and a difference from other pixels having a low luminance is set. Emphasizing means for emphasizing, from among the pseudo contours formed by the set of emphasizing pixels, from the extension side to the operated body side with respect to the direction from one side bounding the extension to the other side. Erasing means for erasing the emphasized pixels constituting the pseudo contour line directed to the erasure candidate direction corresponding to the inclined direction, and leaving the emphasized pixels other than the erased emphasized pixels as remaining pixels, and the remaining pixels. Extracting means for extracting, from among all remaining pixels, remaining pixels constituting a pseudo contour having characteristics corresponding to the occupant's arm, among the remaining pixels. , Based on at least one of the position, length, and direction of the pseudo contour formed by the extracted pixels, whether the pseudo contour is the contour of the arm of the occupant on the other side bordering the extension. The operator determination device according to claim 4, wherein the determination is performed. 6. A determination unit that is interposed between a control unit of a predetermined device that operates based on an operation signal from the operated body and the operated body and is connected to the determining unit, and a determination result of the determining unit. 6. The apparatus according to claim 5, further comprising a restriction unit configured to restrict the operation of the predetermined device based on the condition.