JP2010211614A - Operation supporting device and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To support operations more suitably to a user. <P>SOLUTION: An imaging control part 131 controls an imaging part 125 to image a user operating a vehicle loaded with a vehicle operation supporting device 100 and generate the image of the user. A user authentication part 132 refers to registration information 141 stored in a storage part 123, and authenticates an object (user) included in the image generated by the control by the imaging control part 131. An explanation necessity determination part 133 determines the necessity of an explanation on the basis of the authentication result of the user by the user authentication part 132. An operation explanation part 134 controls an output part 122 for instance and explains the operation for which it is determined that the explanation is needed in the explanation necessity determination part 133. This invention is applicable to the vehicle operation supporting device for instance. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an operation support apparatus, method, and program, and more particularly, to an operation support apparatus, method, and program that can perform operation support more suitable for a user.

  2. Description of the Related Art Conventionally, there is a support device that performs support such as providing guidance and notification to a user who operates a vehicle such as an automobile (for example, see Patent Document 1).

  For example, Patent Document 1 proposes an apparatus that detects an occupant's impotence from an operation state when an operation target of a vehicle is operated by an occupant, and performs operation support when the imperial feeling is felt.

JP 2008-195231 A

  However, in the case of the method described in Patent Document 1, the operation support is performed only after the operation target of the vehicle is operated and an unfamiliar operation is detected. Therefore, the operation support is performed only after the occupant has stopped once. There was a fear that I could not. That is, this operation support is performed based on the operation content, and not based on the circumstances of the individual passenger.

  The present invention has been proposed in view of such circumstances, and by performing user authentication, it is possible to perform operation support more suitable for the user in consideration of user information other than the operation content. To do.

  One aspect of the present invention includes an imaging unit that captures an image of a user of a predetermined apparatus, a storage unit that stores information for each user, an image of the user obtained by imaging by the imaging unit, and the storage unit. Is it necessary to explain the operation of the apparatus to the user based on the authentication means for authenticating the user based on the stored information for each user and the authentication result by the authentication means? An operation support apparatus including a determination unit that determines whether or not, and an explanation unit that performs the description when the determination unit determines that the description is necessary.

  In the first aspect of the present invention, a user of a predetermined apparatus is imaged, information for each user is stored, based on the user image obtained by imaging, and the stored information for each user, The user is authenticated, and based on the authentication result, it is determined whether or not it is necessary to explain the operation of the apparatus to the user.

  Therefore, operation support more suitable for the user can be performed.

  For example, the imaging unit includes an imaging unit, the storage unit includes, for example, a storage unit, the authentication unit includes, for example, a user authentication unit, and the determination unit includes, for example, an explanation necessity determination unit. The explanation means is constituted by an operation explanation section, for example.

  The determination unit may determine that it is necessary to explain the operation of the device to the user when the user who has been imaged by the imaging unit uses the device for the first time.

  In addition, the determination unit determines that it is necessary to explain the operation of the device to the user when the user captured by the imaging unit has not used the device for a predetermined period or longer. be able to.

  Furthermore, the determination unit can determine that it is necessary to explain the operation of the apparatus to the user when there is an operation that has not been performed by the user imaged by the imaging unit.

  In addition, when the user imaged by the imaging unit is not the owner of the device, the determination unit can determine that it is necessary to explain the operation of the device to the user.

  Further, the determination unit needs to explain the operation of the device to the user when the operation method of the device owned by the user captured by the imaging unit is different from the operation method of the device. Can be determined.

  Thus, operation support more suitable for the user can be performed.

  The information processing apparatus may further include selection means for selecting contents to be explained according to a situation from a plurality of contents included in the explanation determined to be explained by the judging means.

  Thereby, operation support more suitable for the user can be performed according to the situation.

  The selection means is constituted by an explanation content selection unit, for example.

  The apparatus further comprises a device state acquisition unit that acquires information regarding the state of the device, and the selection unit selects the content to be described based on the information regarding the state of the device acquired by the device state acquisition unit. Can do.

  Thereby, operation assistance more suitable for the user can be performed according to the state of the apparatus.

  The device state acquisition unit is configured by, for example, a vehicle state acquisition unit.

  Content that further includes position acquisition means for acquiring information on the position or movement path of the device, and the selection means performs the description based on information on the position or movement path of the device acquired by the position acquisition means Can be selected.

  Thereby, operation support more suitable for the user can be performed according to the position of the apparatus or the movement route.

  A position acquisition means is comprised by the vehicle position acquisition part, for example.

  The apparatus further comprises user status acquisition means for acquiring information related to the user status, and the selection means selects the content to be explained based on the information related to the user status acquired by the user status acquisition means. Can do.

  Thereby, operation support more suitable for the user can be performed according to the user's state.

  The user status acquisition unit is configured by, for example, a user status acquisition unit.

  The apparatus further comprises external environment acquisition means for acquiring information related to the external environment of the apparatus, and the selection means selects the content to be described based on information related to the external environment of the apparatus acquired by the external environment acquisition means can do.

  Thereby, operation support more suitable for the user can be performed according to the external environment.

  The external environment acquisition unit is configured by, for example, an external environment acquisition unit.

  The apparatus may further comprise confirmation means for confirming whether or not to perform the explanation determined to be necessary by the determination means.

  Thereby, generation | occurrence | production of the situation which gives a user discomfort can be suppressed.

  The confirmation unit is configured by, for example, an explanation confirmation unit.

  According to another aspect of the present invention, the imaging unit images a user of a predetermined apparatus, and the authentication unit is based on the image of the user obtained by imaging and information for each user stored in the storage unit. The user is authenticated, and the determination means determines whether or not it is necessary to explain the operation of the device to the user based on the authentication result, and the explanation means needs the explanation. If it is determined, the operation support method performs the above description.

  Therefore, operation support more suitable for the user can be performed.

  One aspect of the present invention further includes a computer, an imaging unit that images a user of a predetermined device, a storage unit that stores information for each user, an image of the user obtained by imaging by the imaging unit, and Based on the information for each user stored in the storage means, the authentication means for authenticating the user, and the user to explain the operation of the device based on the authentication result by the authentication means. A determination means for determining whether or not it is necessary, and a program for functioning as an explanation means for performing the explanation when the judgment means judges that the explanation is necessary.

  Therefore, operation support more suitable for the user can be performed.

  According to the present invention, operation can be supported. In particular, operation support more suitable for the user can be performed.

It is a block diagram which shows the structural example of the vehicle operation assistance apparatus to which this invention is applied. It is a figure explaining the installation example of a vehicle operation assistance apparatus. It is a figure explaining the example of the content of registration information. It is a flowchart explaining the example of the flow of an operation assistance process. It is a flowchart explaining the example of the flow of a description necessity determination process. It is a block diagram which shows the other structural example of the vehicle operation assistance apparatus to which this invention is applied. It is a figure explaining the example of the content of description. It is a figure explaining the example of selection of the contents of explanation based on a vehicle state. It is a figure explaining the example of selection of the contents of explanation based on a vehicle position or a movement route. It is a figure explaining the example of selection of the contents of explanation based on a user state. It is a figure explaining the example of selection of the contents of explanation based on an external environment. It is a flowchart explaining the other example of the flow of an operation assistance process. It is a flowchart explaining the example of the flow of an explanation confirmation process. It is a flowchart explaining the example of the flow of description content selection processing. It is a figure explaining the structural example of the personal computer to which this invention is applied.

  FIG. 1 is a block diagram illustrating a configuration example of a vehicle operation support device to which the present invention is applied.

  The vehicle operation support device 100 shown in FIG. 1 is mounted on a vehicle such as an automobile, for example, to provide information to a user who operates the vehicle, so that the user can operate more appropriately. It is a device that supports user operations. This operation may include not only the driving operation of the vehicle but also the operation of equipment mounted on the vehicle such as lighting, air conditioning, or navigation system. The provided information may be any information as long as it relates to the operation, and may include, for example, data such as speed and temperature, guidance on the operation method and operation timing, or a warning. .

  As shown in FIG. 1, the vehicle operation support apparatus 100 includes a control unit 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, an input unit 121, an output unit 122, a storage unit 123, a communication unit 124, And an imaging unit 125.

  The control unit 111 is configured by a microcomputer including, for example, a CPU (Central Processing Unit) (not shown). The control unit 111 executes various processes according to a program stored in the ROM 112 or a program loaded from the storage unit 123 to the RAM 113. The RAM 113 also appropriately stores data necessary for the control unit 111 to execute various processes.

  The ROM 112 has a non-rewritable storage area, and stores software programs and data written at the time of factory shipment, for example. The RAM 113 has a rewritable storage area and temporarily holds a software program being executed and data being processed.

  The input unit 121 includes various input devices that accept user operations and information, such as buttons, switches, touch pads, and microphones. When the input unit 121 receives information such as a user operation or voice, the input unit 121 supplies the information to the control unit 111.

  The output unit 122 includes an output device such as a speaker, an LED (Light Emitting Diode), a monitor, or an output terminal. The output unit 122 outputs information for output (for example, data, sound, or image) supplied from the control unit 111 using an appropriate output device (including sound output and image display).

  The storage unit 123 has a rewritable nonvolatile storage area such as a flash memory or a hard disk. The storage unit 123 stores, for example, registration information 141 including information related to users registered in advance, and supplies the registration information 141 to the control unit 111 as needed.

  The communication unit 124 has a communication interface of a predetermined standard such as a modem, and performs wired or wireless communication processing via an arbitrary network including, for example, the Internet or a LAN (Local Area Network).

  The imaging unit 125 includes, for example, a predetermined imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, images a subject, and supplies the obtained image data to the control unit 111. .

  The control part 111 controls the input part 121 thru | or the imaging part 125, and makes a user provide the information which assists operation of a vehicle. The control unit 111 includes, for example, an imaging control unit 131, a user authentication unit 132, an explanation necessity determination unit 133, an operation explanation unit 134, and a registration information update unit 135 as functional blocks. These functional blocks indicate functions realized by the control unit 111 executing a program or the like.

  The imaging control unit 131 controls the imaging unit 125 to capture an image of a user who operates a vehicle on which the vehicle operation support device 100 is mounted, and generate an image of the user.

  The user authentication unit 132 refers to the registration information 141 stored in the storage unit 123 and authenticates a subject (user) included in the image generated by being controlled by the imaging control unit 131.

  The explanation necessity determination unit 133 determines the necessity of explanation based on the user authentication result by the user authentication unit 132.

  For example, the operation explanation unit 134 controls the output unit 122 to explain the operation that is determined to be necessary for explanation by the explanation necessity judgment unit 133.

  The registration information update unit 135 updates the registration information 141 stored in the storage unit 123.

  The vehicle operation support device 100 is installed at an arbitrary position of the vehicle, for example, in an instrument panel of an automobile. Further, for example, a part of the configuration of the vehicle operation support device 100 such as the imaging unit 125 and the output unit 122 may be configured as a separate body in a state where information can be exchanged with the control unit 111.

  FIG. 2 is a diagram illustrating an installation example of the vehicle operation support device 100.

  As shown in FIG. 2A, the imaging unit 125 images, for example, the vicinity of a face of a user (for example, a vehicle driver) operating the steering wheel 151 on a steering wheel 151 of a vehicle on which the vehicle operation support device 100 is mounted. Installed in a possible posture. Of course, for example, the imaging unit 125 may be installed at a position other than this, such as the vicinity of the passenger seat or the vicinity of the rear seat. For example, the imaging unit 125 may be able to capture a user located in the passenger seat or the rear seat. Further, the imaging unit 125 may be configured by a plurality of cameras or the like, and the user may be captured at a plurality of positions.

  Further, as shown in FIG. 2B, the output unit 122 may be provided with a speaker 161 that outputs a voice guidance such as “the refueling port of this car is on the left side”. The speaker 161 is installed, for example, in a predetermined position in the vehicle compartment where the user can easily hear the sound output from the speaker 161.

  FIG. 3 is a diagram for explaining an example of the contents of the registration information 141 stored in the storage unit 123.

  As shown in FIG. 3, in the registration information 141, for example, information such as a driver, the number of times of boarding, the time since the last boarding, the owner, the owned vehicle type, and the operation history are managed for each user (driver). .

  The item “driver” is identification information of each user registered in advance. For example, an image indicating the user such as a user face image is also registered. The item “number of times of boarding” indicates the number of times the user has boarded the vehicle so far. The item “time since last boarding” indicates an elapsed time since the last boarding of the user. The item “owner” indicates whether or not the user is the owner of this vehicle. The item “owned vehicle type” indicates the type of vehicle owned by the user. Each information of the item “owner” and the item “owned vehicle type” is registered, for example, at the time of user registration (when the item “driver” is generated).

  The item “operation history” indicates past operation contents of the user. For example, all the operable items (for example, A to Z) of the vehicle are listed in advance, and the identification information of the item executed by the user among the operable items is reflected in the item “operation history”.

  In the case of the example in FIG. 3, the number of times that the driver “Mr. A” has boarded is “20”, the elapsed time since the last boarding is “one day”, and “Mr. A” is the owner of this vehicle. (YES), the owned vehicle type is “AAA”, and the operation history is “A, B,..., Z” (that is, all operations have been executed).

  The registration information 141 only needs to include at least the items “boarding count” and “time since last boarding”, and other items are optional. In other words, items other than those described above may be included in the registration information 141, and some or all of the items of “owner”, “owned vehicle type”, and “operation history” may be omitted. Good.

  Next, an example of the flow of operation support processing executed by the vehicle operation support apparatus 100 of FIG. 1 will be described with reference to the flowchart of FIG.

  For example, when the operation support process is started by turning on the power to the vehicle operation support apparatus 100 or the like, the imaging control unit 131 acquires user information in step S101. For example, the imaging control unit 131 controls the imaging unit 125 to image a user (for example, the vicinity of the user's face), and acquires the user's image as user information.

  In step S102, the user authentication unit 132 authenticates the user using the user information (user image) obtained in step S101 and the registration information 141 registered in the storage unit 123. In step S103, the explanation necessity determination unit 133 performs explanation necessity determination according to the user based on the user authentication result.

In step S104, the operation explanation unit 134 determines whether or not an explanation of the operation method is necessary based on the result of the explanation necessity determination process. If it is determined that there is a need for explanation, the process proceeds to step S105. In step S <b> 105, the operation explanation unit 134 explains necessary operations. After the process of step S105 ends, the process proceeds to step S106.
If it is determined in step S104 that the description of the operation method is not necessary, the process of step S105 is omitted, and the process proceeds to step S106.

  In step S106, the registration information update unit 135 updates the contents of the registration information 141 as appropriate. After the process of step S106 is completed, the operation support process is terminated.

  Next, an example of the explanation necessity determination process executed in step S103 of FIG. 4 will be described with reference to the flowchart of FIG.

  When the explanation necessity determination process is started, in step S121, the explanation necessity determination unit 133 determines whether or not the user is a first boarding (use for the first time) based on the registration information 141. For example, if the user who is the subject of the captured image obtained by the control of the imaging control unit 131 is not registered in the registration information 141 and it is determined that this user is the first boarding, the process proceeds to step S122. In step S122, the explanation necessity determination unit 133 sets that an explanation of the operation for the first boarding user is necessary. After the process of step S122 is completed, the explanation necessity determination process is ended, the process returns to step S103 of FIG. 4, and the processes after step S104 are executed.

  In step S121, for example, when it is determined that the user who is the subject of the captured image obtained by the control of the imaging control unit 131 is registered in the registration information 141 and is not the first boarding user, the process proceeds to step S123. Proceed to

  In step S <b> 123, the explanation necessity determination unit 133 determines whether or not a predetermined period or more has elapsed since the last boarding (the previous use) based on the registration information 141. If it is determined that the predetermined period or more has elapsed, the process proceeds to step S124. In step S <b> 124, the explanation necessity determination unit 133 sets that an explanation of operation for a user who has not boarded for a long time is necessary. After the process in step S124 is completed, the explanation necessity determination process is ended, the process returns to step S103 in FIG. 4, and the processes after step S104 are executed.

  In step S123, for example, when it is determined that a predetermined period or more has not elapsed since the last boarding, the process proceeds to step S125.

  In step S125, the necessity determination unit 133 determines whether there is an operation that has not been performed based on the registration information 141. For example, if it is determined that there is an operation (unexecuted operation) that is not registered in the item “operation history” of the registration information 141 and there is an operation that has not been performed by the user, the process proceeds to step S126. In step S <b> 126, the explanation necessity determination unit 133 sets that an explanation of an operation that has not been performed is necessary. After the process of step S126 is completed, the explanation necessity determination process is ended, the process returns to step S103 of FIG. 4, and the processes after step S104 are executed.

  Further, in step S125, for example, it is determined that all the operations that can be performed are registered in the item “operation history” of the registration information 141, and there are no operations that have not been performed by the user (unimplemented operations). If so, the process proceeds to step S127.

  In step S127, the explanation necessity determination unit 133 determines whether the user is not the owner based on the registration information 141. For example, if the value of the item “owner” of the registration information 141 is “YES” and it is determined that the user is the owner, the process proceeds to step S128. In step S128, the explanation necessity determination unit 133 sets that explanation of operation is not necessary. After the process of step S128 is completed, the explanation necessity determination process is ended, the process returns to step S103 of FIG. 4, and the processes after step S104 are executed.

  In step S127, for example, if the value of the item “owner” in the registration information 141 is “NO” and it is determined that the user is not the owner, the process proceeds to step S129. In step S <b> 129, the explanation necessity determination unit 133 determines whether or not a car owned by the user is registered based on the registration information 141. For example, if the vehicle type is registered in the item “owned vehicle type” of the registration information 141 and it is determined that the vehicle owned by the user is registered, the process proceeds to step S130.

  In step S <b> 130, the explanation necessity determination unit 133 determines whether the operation method is different from the car owned by the user based on the registration information 141. For example, the operation that can be performed by the vehicle type registered in the item “owned vehicle type” of the registration information 141 is completely the same as the operation that can be performed by the vehicle, and the operation method is not different from the vehicle owned by the user. When it determines, it returns to step S128 and the process after it is repeated.

  In step S130, for example, an operation that can be performed by the vehicle type registered in the item “owned vehicle type” of the registration information 141 does not completely match an operation that can be performed by the vehicle. The process proceeds to step S131. In step S131, the explanation necessity determination unit 133 sets that an explanation of the operation is necessary for an operation that does not match. After the process of step S131 is completed, the explanation necessity determination process is ended, the process returns to step S103 of FIG. 4, and the processes after step S104 are executed.

  Furthermore, when it is determined in step S129 that the car owned by the user is not registered, the process proceeds to step S132. In step S132, the explanation necessity determination unit 133 sets that an explanation of operation for a general non-owner user is necessary. After the process of step S132 is completed, the explanation necessity determination process is ended, the process returns to step S103 of FIG. 4, and the processes after step S104 are executed.

  As described above, the vehicle operation support device 100 determines the necessity for the operation explanation based on the user information (registration information 141) registered in advance, and performs the setting. Only can be done properly. That is, the vehicle operation support device 100 can perform operation support more suitable for the user based on information such as the number of times each user has boarded and the time since the last boarding. In addition, the vehicle operation support device 100 can perform operation support more suitable for the user based on other information such as “owner”, “owned vehicle type”, or “operation history”, for example.

That is, the vehicle operation support apparatus 100 can suppress an operation that the user is not familiar with. Thereby, the user who receives operation assistance can operate the vehicle more easily.
The

  Further, it may be confirmed with the user whether or not an operation explanation is necessary. Further, for example, the content of the operation description may be changed (selected) according to the external environment, the user's state, or the like.

  FIG. 6 is a block diagram showing another configuration example of the vehicle operation support apparatus to which the present invention is applied.

  A vehicle operation support device 200 shown in FIG. 6 basically has the same configuration as that of the vehicle operation support device 100 of FIG. 1, but includes a control unit 211 instead of the control unit 111 of FIG.

  In addition to the imaging control unit 131 to the registration information update unit 135 that are functional blocks included in the control unit 111 of FIG. 1, the control unit 211 further includes a description confirmation unit 221, a description content selection unit 222, and an external environment. It has the acquisition part 231, the vehicle state acquisition part 232, the vehicle position acquisition part 233, and the user state acquisition part 234.

  The explanation confirmation unit 221 controls, for example, the input unit 121 and the output unit 122, and confirms, for example, with the user whether or not to perform the explanation that is set to be implemented in the explanation necessity determination unit 133. Note that the confirmation by the explanation confirmation unit 221 may be performed by a user other than the user.

  The description content selection unit 222 further selects content from descriptions set as necessary to be executed based on various information supplied from the external environment acquisition unit 231 to the user status acquisition unit 234.

  The external environment acquisition unit 231 acquires information about the external environment by controlling the input unit 121 and the like, for example. For example, the vehicle state acquisition unit 232 controls the input unit 121 and acquires information related to the vehicle state. The vehicle position acquisition part 233 controls the input part 121 etc., for example, and acquires the information regarding a vehicle position. The user state acquisition unit 234 acquires information on the user state based on the user's captured image acquired by the imaging control unit 131, for example. The external environment acquisition unit 231 to the user state acquisition unit 234 supply the acquired information to the explanation content selection unit 222, respectively.

  Next, description contents and selection thereof will be described.

  FIG. 7 is a diagram for explaining a content example of the explanation. The contents of the operation explanation include items in the table 251 shown in FIG. 7 such as “door opening / closing method” and “door glass opening / closing method”. Of course, other matters may be described, or some of the items in the table 251 may be omitted.

  The explanation content selection unit 222 selects which of the contents shown in the table 251 is to be explained based on various information supplied from the external environment acquisition unit 231 to the user state acquisition unit 234.

  FIG. 8 is a diagram for explaining a selection example of the explanation content based on the vehicle state. The contents of the vehicle state acquired by the vehicle state acquisition unit 232 include, for example, “engine state”, “shift position”, “vehicle speed” and the like as shown in the table 261 of FIG. 8A. The vehicle state acquisition unit 232 acquires information on these vehicle states, for example, by controlling various sensors of the input unit 121 and supplies the information to the explanation content selection unit 222.

  Of course, contents other than the items in the table 261 may be included, or some of the items in the table 261 may be omitted.

  The description content is selected based on each content of the vehicle state. For example, the description is selected as shown in Table 262 in FIG. 8B depending on whether the vehicle state is “when the engine is stopped”, “when the engine is operating”, or “when the vehicle is broken”. The In Table 262, white circles indicate the contents selected according to each vehicle state. For example, when the vehicle state is “when the engine is stopped”, “door opening / closing method”, “seat handling method”, and the like are selected. In this way, the selection pattern of the explanation content is determined in advance for each vehicle state, and the explanation content selection unit 222 has the explanation content according to the vehicle state supplied from the vehicle state acquisition unit 232 according to the selection pattern. Make a selection. The selected content is explained by the operation explanation unit 134.

  Of course, a selection pattern other than the example in Table 262 may be applied to each vehicle state.

  FIG. 9 is a diagram for explaining an example of selection of explanation contents based on a vehicle position or a movement route. As the contents of the vehicle position or movement route acquired by the vehicle position acquisition unit 233, as shown in the table 271 of FIG. 9A, for example, “parking lot”, “gas station”, “via a general road”, etc. is there. The vehicle position acquisition unit 233 acquires information on these vehicle states by controlling various sensors of the input unit 121 or acquiring information obtained by the navigation function from the RAM 113 or the storage unit 123, for example. It supplies to the content selection part 222.

  Of course, contents other than the items in the table 271 may be included, or some of the items in the table 271 may be omitted.

  The description content is selected based on each content of the vehicle position or the movement route. For example, the vehicle position is “parking lot”, “parking lot gate”, “gas station”, “drive-through”, “general road”, or “highway” The contents of the description are selected as shown in Table 272 of FIG. 9B depending on whether the travel route is “via a general road” or “via a highway”. In Table 272, white circles indicate contents selected depending on the vehicle position or the movement route. For example, when the vehicle position is “parking lot”, “door opening / closing method”, “seat handling method”, and the like are selected. As described above, the selection pattern of the description content is determined in advance for each vehicle position or movement route, and the description content selection unit 222 supplies the vehicle position or the movement route supplied from the vehicle position acquisition unit 233 according to the selection pattern. The contents of explanation are selected according to. The selected content is explained by the operation explanation unit 134.

  Of course, a selection pattern other than the example in Table 272 may be applied to each vehicle position or movement route.

  FIG. 10 is a diagram for explaining an example of selection of explanation content based on the user state. The contents of the user status acquired by the user status acquisition unit 234 include, for example, “seat position”, “face orientation / line of sight”, “age”, and the like, as shown in the table 281 of FIG. 10A. For example, the user status acquisition unit 234 controls various sensors of the input unit 121, analyzes a user image obtained by the imaging control unit 131, or registration information regarding the user registered in the RAM 113 or the storage unit 123. To obtain information on the status of these users and supply the information to the explanation content selection unit 222.

  Of course, contents other than the items in the table 281 may be included, or some of the items in the table 281 may be omitted.

  The description content is selected based on each content of the user state. For example, depending on whether the user's “seat position” is “driver's seat”, “passenger seat”, or “rear seat”, the description is selected as shown in Table 282 in FIG. 10B. . In Table 282, white circles indicate contents selected depending on the user state. For example, when the user's “seat position” is “driver's seat”, “door opening / closing method”, “door glass opening / closing method”, “seat handling method”, and “handle and mirror adjustment method” Etc. are selected. As described above, the selection pattern of the explanation content is determined in advance for each user state, and the explanation content selection unit 222 displays the explanation content according to the user state supplied from the user state acquisition unit 234 according to the selection pattern. Make a selection. The selected content is explained by the operation explanation unit 134.

  Of course, a selection pattern other than the example of the table 282 may be applied to each user state.

  FIG. 11 is a diagram for explaining an example of selection of explanation content based on the external environment. The contents of the external environment acquired by the external environment acquisition unit 231 include, for example, “weather”, “brightness”, “temperature”, and the like as shown in the table 291 of FIG. 11A. The external environment acquisition unit 231 acquires information about these external environments by controlling various sensors of the input unit 121 and supplies the information to the explanation content selection unit 222, for example.

  Of course, contents other than the items in the table 291 may be included, or some of the items in the table 291 may be omitted.

  The description content is selected based on each content of the external environment. For example, depending on whether "weather" is "sunny", "rain and snow", or "mist", whether "brightness" is "bright" or "dark", ”Is“ high ”,“ comfortable ”, or“ low ”, or whether the“ road surface ”is“ dry ”,“ wet ”, or“ snow ” The contents are selected as shown in Table 292 of FIG. 11B. In Table 292, white circles indicate contents selected according to the external environment. For example, when “weather” is “rain or snow”, “wiper switch handling method” or the like is selected. For example, when “weather” is “mist”, “fog lamp switch” The “handling method” is selected. In this way, the selection pattern of the explanation content is determined in advance for each external environment, and the explanation content selection unit 222 displays the explanation content according to the external environment supplied from the external environment acquisition unit 231 according to the selection pattern. Make a selection. The selected content is explained by the operation explanation unit 134.

  Of course, a selection pattern other than the example in Table 292 may be applied to each user state.

  An example of the flow of operation support processing when the above description is confirmed and the content is selected will be described with reference to the flowchart of FIG.

  When the operation support process is started, the processes in steps S201 to S204 are executed in the same manner as in steps S101 to S104 in FIG.

  That is, in step S201, the imaging control unit 131 acquires user information, in step S202, the user authentication unit 132 authenticates the user, and in step S203, the explanation necessity determination unit 133 performs explanation necessity determination according to the user. . Note that the details of the necessity determination process are the same as those described with reference to FIG.

  In step S <b> 204, the operation explanation unit 134 determines whether it is necessary to explain the operation method based on the result of the necessity explanation process.

  If it is determined that there is a need for explanation, the process proceeds to step S205. In step S <b> 205, the explanation confirmation unit 221 performs explanation confirmation processing for confirming with the user whether or not to explain the operation. In step S206, the explanation confirmation unit 221 determines whether to explain the operation method based on the processing result (confirmation result) in step S205. If it is determined to be described, the process proceeds to step S207.

  In step S207, the explanation content selection unit 222 and the external environment acquisition unit 231 to the user state acquisition unit 234 perform explanation content selection processing for selecting the explanation content according to the situation based on various information.

  In step S <b> 208, the operation explanation unit 134 explains the operation for the set contents to be explained by the above processing. When the process of step S208 ends, the process proceeds to step S209.

  If it is determined in step S204 that the description of the operation method is not necessary, or if it is determined in step S206 that the operation method is not described, the process proceeds to step S209.

  In step S209, the registration information updating unit 135 updates the contents of the registration information 141 as appropriate, as in step S106 of FIG. After the process of step S209 ends, the operation support process ends.

  Next, an example of the flow of explanation confirmation processing executed in step S205 of FIG. 12 will be described with reference to the flowchart of FIG.

  When the explanation confirmation process is started, the explanation confirmation unit 221 presents, for example, a message prompting the user to confirm the contents of the operation explanation or the availability of the operation explanation by controlling the monitor, the speaker, or the like of the output unit 122 in step S221. Then, the user confirms whether or not the operation can be explained by controlling a button, a touch panel, or the like of the input unit 121 to accept a user instruction.

  In step S222, the explanation confirmation unit 221 determines whether or not an instruction to execute an operation explanation has been given based on a user instruction or the like received in the input unit 121 by the process in step S221. If it is determined that the user has agreed with the operation explanation and has instructed execution of the explanation, the process proceeds to step S223. In step S223, the explanation confirming unit 221 sets the operation explanation. When the process of step S223 ends, the process returns to step S205 of FIG. 12, and the processes after step S206 are executed.

  If it is determined in step S222 that the user has opposed the operation description and has not instructed execution of the operation description, the process proceeds to step S224. In step S224, the explanation confirmation unit 221 sets that no explanation of operation is performed. When the process of step S224 ends, the process returns to step S205 of FIG. 12, and the processes after step S206 are executed.

  Next, an example of the flow of explanation content selection processing executed in step S207 of FIG. 12 will be described with reference to the flowchart of FIG.

  When the explanation content selection process is started, the vehicle state acquisition unit 232 acquires information on the vehicle state in step S241. In step S242, the explanation content selection unit 222 selects the explanation content based on the vehicle state acquired in step S241.

  In step S243, the vehicle position acquisition unit 233 acquires information regarding the vehicle position or the movement route. In step S244, the explanation content selection unit 222 selects the explanation content based on the vehicle position or the movement route acquired in step S243.

  In step S245, the user status acquisition unit 234 acquires information related to the user status. In step S246, the explanation content selection unit 222 selects the explanation content based on the user state acquired in step S245.

  In step S247, the external environment acquisition unit 231 acquires information regarding the external environment. In step S248, the explanation content selection unit 222 selects the explanation content based on the external environment acquired in step S247.

  When the process of step S248 ends, the process returns to step S207 of FIG. 12, and the processes after step S208 are executed.

  As described above, the vehicle operation support device 200 determines the necessity of the operation explanation based on the user information (registration information 141) registered in advance as in the vehicle operation support device 100 of FIG. By doing so, it is possible to perform operation support more suitable for the user, and it is possible to further suppress operations unfamiliar to the user. Thereby, the user who receives operation assistance can operate the vehicle more easily.

  Moreover, since the vehicle operation assistance apparatus 200 can confirm whether the operation explanation is possible or not, it can explain the operation based on the user's judgment. For example, the user may be familiar with the vehicle operation method even if the user has not been in the vehicle for a long time. If the description of the content grasped by the user is performed again, the description is not necessary for the user, and the user may be uncomfortable. As described above, the vehicle operation support apparatus 200 confirms whether or not the description is possible, and can omit the description of the content that the user has determined to be unnecessary, so that the user may feel uncomfortable. Occurrence can be suppressed.

  Furthermore, the vehicle operation support apparatus 200 can select the description content based on various information such as the external environment, the vehicle state, the vehicle position and the movement route, or the user state. Thereby, the vehicle operation assistance apparatus 200 can perform more appropriate explanation according to the situation. In other words, the vehicle operation support apparatus 200 can omit an unnecessary description in the current situation by selecting the content. That is, the vehicle operation support apparatus 200 can suppress the occurrence of a situation in which the user is uncomfortable by giving unnecessary explanation.

  In the above description, the device for assisting the vehicle operation by explaining the operation of the vehicle to the user who operates the vehicle has been described. It may be a support device for supporting. That is, the present invention can be applied to a support device that assists a user who operates an arbitrary device to operate the device.

  The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, it may be configured as a personal computer as shown in FIG.

  In FIG. 15, the CPU 401 of the personal computer 400 executes various processes according to a program stored in the ROM 402 or a program loaded from the storage unit 413 to the RAM 403. The RAM 403 also appropriately stores data necessary for the CPU 401 to execute various processes.

  The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output interface 410 is also connected to the bus 404.

  The input / output interface 410 includes an input unit 411 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display), an output unit 412 including a speaker, and a hard disk. A communication unit 414 including a storage unit 413 and a modem is connected. The communication unit 414 performs communication processing via a network including the Internet.

  A drive 415 is connected to the input / output interface 410 as necessary, and a removable medium 421 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is It is installed in the storage unit 413 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 15, the recording medium is distributed to distribute the program to the user separately from the apparatus main body, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk ( It is only composed of removable media 421 consisting of CD-ROM (compact disc-read only memory), DVD (including digital versatile disc), magneto-optical disc (including MD (mini disc)), or semiconductor memory. Rather, it is composed of a ROM 402 on which a program is recorded and a hard disk included in the storage unit 413, which is distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of devices (apparatuses).

  In addition, in the above description, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit). Of course, a configuration other than that described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100 Vehicle operation assistance apparatus 111 Control part 131 Imaging control part 132 User authentication part 133 Explanation necessity determination part 134 Operation explanation part 135 Registration information update part 141 Registration information 221 Explanation confirmation part 222 Explanation content selection part 231 External environment acquisition part 232 Vehicle Status acquisition unit 233 Vehicle position acquisition unit 234 User status acquisition unit

Claims (14)

Imaging means for imaging a user of a predetermined device;
Storage means for storing information for each user;
Authentication means for authenticating the user based on the image of the user obtained by imaging by the imaging means and the information for each user stored in the storage means;
Determination means for determining whether or not it is necessary to explain the operation of the device to the user based on an authentication result by the authentication means;
An operation support apparatus comprising: an explanation unit that performs the explanation when the judgment unit judges that the explanation is necessary. The operation support according to claim 1, wherein the determination unit determines that it is necessary to explain the operation of the device to the user when the user captured by the imaging unit uses the device for the first time. apparatus. The determination unit determines that it is necessary to explain the operation of the device to the user when the user imaged by the imaging unit has not used the device for a predetermined period or longer. The operation support apparatus according to 1. The operation according to claim 1, wherein the determination unit determines that it is necessary to explain the operation of the apparatus to the user when there is an operation that has not been performed by the user imaged by the imaging unit. Support device. The operation support according to claim 1, wherein the determination unit determines that it is necessary to explain the operation of the device to the user when the user imaged by the imaging unit is not the owner of the device. apparatus. The determination unit determines that it is necessary to explain the operation of the device to the user when the operation method of the device owned by the user captured by the imaging unit is different from the operation method of the device. The operation support apparatus according to claim 1. The operation support apparatus according to claim 1, further comprising a selection unit that selects a content to be explained according to a situation from a plurality of contents included in the explanation determined to be explained by the judging unit. . Further comprising device status acquisition means for acquiring information relating to the status of the device,
The operation support apparatus according to claim 7, wherein the selection unit selects the content to be explained based on information on the state of the device acquired by the device state acquisition unit. It further comprises position acquisition means for acquiring information related to the position or movement path of the device,
The operation support apparatus according to claim 7, wherein the selection unit selects the content to be explained based on information on the position or movement path of the apparatus acquired by the position acquisition unit. Further comprising user status acquisition means for acquiring information relating to the user status;
The operation support apparatus according to claim 7, wherein the selection unit selects the content to be explained based on information on the user state acquired by the user state acquisition unit. Further comprising external environment acquisition means for acquiring information relating to the external environment of the device;
The operation support apparatus according to claim 7, wherein the selection unit selects the content to be described based on information about the external environment of the apparatus acquired by the external environment acquisition unit. The operation support apparatus according to claim 1, further comprising confirmation means for confirming whether or not to perform the explanation determined to be necessary by the determination means. An operation support method for an operation support apparatus,
The imaging means of the operation support device images a user of a predetermined device,
The authentication unit of the operation support device performs authentication of the user based on the image of the user obtained by imaging and information for each user stored in the storage unit,
The determination unit of the operation support device determines whether or not it is necessary to explain the operation of the device to the user based on an authentication result,
The operation support method is an operation support method for performing the description when it is determined that the description is necessary. Computer
Imaging means for imaging a user of a predetermined device;
Storage means for storing information for each user;
Authentication means for authenticating the user based on the image of the user obtained by imaging by the imaging means and the information for each user stored in the storage means;
Determination means for determining whether or not it is necessary to explain the operation of the device to the user based on an authentication result by the authentication means;
A program for causing a function to serve as explanation means for performing the explanation when the judgment means judges that the explanation is necessary.

Images