JP2007255971A - On-vehicle electronic device, and operation control method of same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform automatic setting to a suitable operation state corresponding to an individual user. <P>SOLUTION: An imaging part capable of imaging a user on board is provided on a position near a display part or an operation panel or the like of a navigation device or an audio reproduction device, and face feature data are detected from an image imaged by the imaging part (F102). The face feature data are used as information for discriminating the individual user. When the individual user is discriminated, each control corresponding to the individual user is performed in navigation operation or in audio reproduction operation (F104, F105, F106). For example, navigation output according to user's liking or necessity is performed, as a traveling guide by map image display or a voice in the case of the navigation device. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an in-vehicle electronic device such as a navigation device or an audio playback device, and an operation control method thereof.

JP 2003-32776 A JP-A-7-329639 Japanese Patent Laid-Open No. 9-90963

For example, navigation devices and audio playback devices are widely used as in-vehicle electronic devices.
And, as these in-vehicle devices, in particular, an imaging unit (camera) is provided, and for example, the number of passengers and the boarding position of a car are discriminated from the taken images, and the acoustic characteristics are adjusted according to those situations. Is described in the above patent document.

By the way, in-vehicle electronic devices have different preference for setting and necessity of operation for each individual user. For example, when a single automobile is used by a family, there are a plurality of users of in-vehicle electronic devices, but each user has different preferences and needs regarding the operation of the in-vehicle electronic devices.
For example, in the case of a navigation device, the desired content of the display or voice guide varies from user to user, resulting in an excessive guide for each individual user, or conversely, the desired guide content is not output.
In the case of an audio playback device, preferences such as volume and sound settings vary from user to user.
For this reason, each time a user who gets on the vehicle changes, each person needs to re-operate to his / her favorite state, which is troublesome. Or, if the operation is troublesome, an unfavorable operation state must be accepted.

  Therefore, an object of the present invention is to provide an in-vehicle electronic device that can perform an operation suitable for a user who gets on the vehicle without performing troublesome operations each time even if the user who gets on the vehicle changes.

  An in-vehicle electronic device according to the present invention includes an imaging unit arranged so as to capture an image of a person in a vehicle, and image analysis that analyzes image data captured by the imaging unit and generates facial feature data of the captured person A memory unit for storing user information corresponding to the facial feature data, and reading out the user information corresponding to the facial feature data generated by the image analysis unit from the memory unit, and controlling the operation based on the read user information The control part which performs is provided.

Moreover, the said vehicle-mounted electronic device is a vehicle-mounted navigation apparatus, Comprising: A display part and the navigation information acquisition part which acquires navigation information are further provided. And when the said control part performs the map display by the map information as the navigation information acquired in the said navigation information acquisition part in the said display part, it corresponds to the said face feature data produced | generated by the said image analysis part. The display contents on the map display are controlled based on the user information read from the memory unit.
The on-vehicle electronic device is an on-vehicle navigation device, and further includes an audio output unit, a navigation information acquisition unit that acquires navigation information, and a position detection unit that detects current position information. The control unit generates, from the audio output unit, the image analysis unit with respect to an operation for executing audio output based on the navigation information acquired by the navigation information acquisition unit or the current position information detected by the position detection unit. The voice output content is controlled based on the user information read from the memory unit corresponding to the face feature data.

The navigation information here is map data, image data to be displayed on the map, POI (Point of Interest) data, lane / route information, toll road toll information, driving attention and guidance information, road This is information that is the source of information to be output as navigation operations, such as information and traffic jam information.
The navigation information acquisition unit is a part that acquires information by reproducing navigation information from a recording medium such as an optical disk or HDD (Hard Disc Drive), or by receiving navigation information by communicating with the outside. .

The in-vehicle electronic device is an in-vehicle audio playback device, and an audio signal processing unit that performs signal processing on an audio signal supplied from an audio source, and an audio signal that has been signal-processed by the audio signal processing unit. Or an audio output processing unit for supplying to a plurality of speakers. The control unit controls signal processing in the audio signal processing unit based on user information read from the memory unit corresponding to the facial feature data generated by the image analysis unit.
The signal processing in the audio signal processing unit includes, for example, volume adjustment processing, acoustic processing such as equalizing and reverb, and output balance setting processing of a multi-channel speaker.
The in-vehicle electronic device is an in-vehicle audio reproduction device, and selects a selection unit that selects audio signals from a plurality of audio sources, and supplies the audio signal selected by the selection unit to one or a plurality of speakers. And an audio output processing unit. The control unit controls an audio signal selection operation in the selection unit based on user information read from the memory unit corresponding to the facial feature data generated by the image analysis unit.

Also, the face feature data generated by the image analysis unit is relative position information of face components.
The controller further includes an input unit for inputting the user information, and the control unit stores the user information input by the input unit in the memory unit in association with the facial feature data generated by the image analysis unit. Let
The control unit further includes an operation input unit, and the control unit generates user information based on an operation content using the operation input unit or a history of the operation content, and corresponds to the facial feature data generated by the image analysis unit. And stored in the memory unit.

  An operation control method according to the present invention includes an imaging unit arranged to capture an image of a person in a vehicle, and an image analysis unit that analyzes image data captured by the imaging unit and generates facial feature data of the captured person And a memory unit for storing user information corresponding to the facial feature data, a method for controlling the operation of the in-vehicle electronic device, the step of generating facial feature data in the image analysis unit, and the memory unit, A step of reading user information corresponding to the facial feature data, and a step of executing operation control based on the read user information.

  In the case of the present invention described above, for example, an imaging unit capable of imaging a user who is on board is provided at a position such as a display unit or an operation panel of a navigation device or an audio playback device, and facial feature data is obtained from an image captured by the imaging unit. To detect. The face feature data is used as information for identifying the individual user. When the individual user is identified, control according to the individual user is performed in the navigation operation and the audio reproduction operation.

According to the present invention, the user can be identified based on the facial feature data of the person captured by the imaging unit, and the motion control based on the user information corresponding to the user is performed, so that the motion suitable for each user can be performed. Done. For example, navigation output according to the user's preference and necessity can be performed as a map image display in the case of a navigation device or a travel guide by voice. Similarly, in the case of an audio playback device, the volume, sound quality (acoustic effect), balance, audio source selection, and the like according to the user's preferences are performed as the audio signal to be played back and output.
That is, as an in-vehicle electronic device for a vehicle on which a plurality of users get on, an individual user can perform a navigation operation and an audio reproduction operation that are comfortable for the user who is on board without performing an adjustment operation.

Hereinafter, examples of a navigation device and an audio playback device will be described in the following order as embodiments of the in-vehicle electronic device of the present invention.
[1. First Embodiment: Navigation Device]
[2. Second Embodiment: Audio Playback Device]
[3. Effects and modifications of the embodiment]

[1. First Embodiment: Navigation Device]

FIG. 1 is a block diagram of an in-vehicle navigation device 10 according to an embodiment.
1 includes an arithmetic processing unit 1, an operation input unit 2, a video signal processing unit 3, a position detection unit 4, a sensor unit 5, a disk drive unit 6, a voice synthesis / playback processing unit 7, and a display unit 8. A speaker unit 9, an imaging unit 16, an image analysis unit 17, and a communication unit 18. Each of these units may be provided integrally, or may be a device unit of a separate casing, and may be connected to be communicable by wire or wirelessly.

The operation input unit 2 is a device unit that receives a user operation and transmits the operation information to the arithmetic processing unit 1. For example, it is realized as an operator such as various keys, a jog dial, or a stick provided on the panel of the main body casing of the navigation device 10, or as a remote controller provided separately from the main body casing. In the case of a remote controller, operation information of operating elements such as keys and dials provided on the remote controller is transmitted to the arithmetic processing unit 1 by infrared rays or radio waves, or a casing having the arithmetic processing unit 1 is wired. Connect and send operation information.
The operation input unit 2 may be a touch panel on a display screen. In that case, it is provided integrally with the display unit 8. Furthermore, a mode in which a menu or icon display displayed on the display unit 8 is clicked with a mouse or a predetermined pointing device is also considered as one mode of the operation input unit 2.

The display unit 8 and the speaker unit 9 serve as an image or audio output device unit for the user.
The display unit 8 displays various navigation images such as a map image, a route guidance image, an operation menu, an icon, POI (Point of Interest) information, a search list, etc. generated by the processing for the navigation operation in the arithmetic processing unit 1. To do.
The video signal processing unit 3 performs necessary processing on the navigation image data generated by the arithmetic processing unit 1 and other image data, and causes the display unit 8 to execute display.
The speaker unit 9 outputs sound based on processing for navigation operation in the arithmetic processing unit 1.
The voice synthesis / playback processing unit 7 executes voice synthesis processing for outputting navigation voice, for example, guide voice by route guidance processing, and outputs the voice from the speaker unit 9.

The position detection unit 4 is a device unit that detects the position of the navigation device 10, that is, the position of an automobile on which the navigation device 10 is mounted, and is a GPS positioning device that uses, for example, GPS (Global Positioning System). As already known, the GPS positioning device calculates the position from the geostationary satellite by receiving radio waves transmitted from a plurality of GPS satellites, and obtains the latitude and longitude as the position on the map of the own vehicle.
The position detection unit 4 supplies latitude and longitude information as the measured vehicle position to the arithmetic processing unit 1.

The sensor unit 5 is a device unit that detects a traveling state or behavior of a car. For example, a vehicle speed sensor that detects the number of rotations of a wheel, a vibration gyro sensor that detects the rotational angular velocity of a vehicle body, an acceleration sensor that detects vehicle acceleration, and a vibration sensor that detects vehicle vibration. In addition, a device that detects the state of the vehicle travel operation device, for example, a parking brake switch, a brake lamp switch, a steering angle sensor, a throttle opening sensor, and the like is also assumed.
Information detected by the sensor unit 5 is supplied to the arithmetic processing unit 1. In the arithmetic processing unit 1, by using the detection information from the sensor unit 5 and the position information from the position detection unit 4 together, it is possible to perform more accurate vehicle position detection.

The disc drive unit 6 is a playback device that plays back a disc 90 as an external recording medium such as a CD (Compact Disc) system, a DVD (Digital Versatile Disc) system, or a Blu-ray Disc system.
By loading a disc 90 on which navigation information including map data and the like is recorded in the disc drive device 6, the navigation device 10 can read the navigation information and perform navigation processing based on the map data and the like. Become.
In this specification, “navigation information” is used as a general term for data used for navigation processing, and the navigation information includes map data, a map image for generating a map image, an icon on the map image, and the like. This includes map drawing data, route calculation data, route guidance data, POI (Point of Interest) data, search data, management data, and the like. The POI data indicates various point information such as restaurants, various stores, sports facilities, and entertainment facilities. For example, the POI data is information that enables these various points to be systematically searched by genre or the like.
In general, in an in-vehicle navigation device, a suitable route is calculated according to a user setting a destination, and a navigation operation for displaying the route on a map image is performed on a display unit. For this navigation operation, it is first necessary for the user to input a destination, but there are various methods for inputting the destination. For example, a method for inputting an address or a place name of the destination. And a method for specifying a position on a map, and a method for searching and selecting a destination. A POI input method is known as a search method. For example, in response to a user operation, a search is performed on a database in which point information as POI data is classified by category or the like, and search results are displayed in a list. The user can register as a destination by designating one of the listed entries. The user input for the search is, for example, specifying a category in a menu format, etc., and searching using the specified category as a key, or inputting a character etc. as a key by the user and searching based on this There are various forms.

The disk drive device unit 6 may be provided integrally with the main body housing having the arithmetic processing unit 1, or may be provided separately and communicated with the arithmetic processing unit 1 by wire or wirelessly.
In the description, a CD-type or DVD-type disc 90 is given as an example of an external recording medium to be reproduced. However, other types of portable recording media such as an optical disc, a magneto-optical disc, or a memory card are used. It may be used as an external recording medium. Of course, in that case, a playback device corresponding to the external recording medium is provided. Further, a disk drive unit 6 as an HDD may be provided.

The imaging unit 16 performs sampling, gain adjustment, video signal processing (luminance processing, color processing, correction) on an imaging optical lens system, a lens driving system, an imaging device unit using a CCD sensor or a CMOS sensor, and an image signal captured by the imaging device unit. The imaging signal processing circuit for performing processing etc. is mounted, and image data is output by the imaging operation. The imaging unit 16 captures the face of the user in the car.
FIG. 3 shows a navigation device 10 arranged on a front console in an automobile. On the front panel, an image is displayed together with a screen as a display unit 8 and various operators as an operation input unit 2. An imaging lens 16a in the unit 16 is provided. The imaging direction of the imaging lens 16a is set so that, for example, the position of the face of the user at the driver's seat is captured as the subject direction.
The imaging direction of the imaging unit 16 may be fixed in the direction of the driver's face, but may be changed to the passenger seat direction or the rear seat direction. Alternatively, the interior of the vehicle may be imaged at a wide angle so that all faces of users in the vehicle can be captured as subjects.

The image analysis unit 17 performs analysis processing on the captured image data obtained by the imaging unit 16.
The image analysis unit 17 performs a process of extracting a human face image from the captured image data as an object to be processed, and a process of generating face feature data FDT from the extracted face image.
The face feature data FDT is, for example, the relative position information of the constituent elements of the face. Here, the face feature data FDT is an example including two pieces of ratio information Fa and Fb.
The ratio information Fa is a ratio of the distance EN between the center of the eye and the nose and the distance Ed between the eyes shown in FIG. For example, Fa = Ed / EN.
The ratio information Fb is the ratio of the distance EM between the center of the eye and the mouth and the distance Ed between the eyes. For example, Fb = Ed / EM.
The relative position information of such facial components is information that is unique to each individual and is not affected by changes in appearance due to an attachment such as a hairstyle or glasses. Furthermore, it is known that it does not change with aging. Therefore, the face feature data FDT including the ratio information Fa and Fb is information that can identify the individual face of the user with high accuracy.
The image analysis unit 17 generates such face feature data FDT from the captured image data and supplies it to the arithmetic processing unit 1.

  The communication unit 18 is a communication unit that communicates with the information server via a mobile phone network or another wireless network. The arithmetic processing unit 1 can obtain, for example, road conditions, traffic jam information, and the like by communication with a predetermined information server by the communication unit 18.

The arithmetic processing unit 1 is a device unit that performs necessary arithmetic processing as a navigation operation. For example, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and a nonvolatile memory 14. The interface unit 15 is provided.
The ROM 12 stores operation programs, constants used for various processes, and fixed information. The operation program may be stored in the disk 90 loaded in the disk drive unit 6 and loaded into the arithmetic processing unit 1 during operation.
The RAM 13 is used as a work area or a program load area, or is used for temporarily storing information. Further, it is also used as a temporary storage area for buffering or data processing for data read by the disk drive unit 6.

The non-volatile memory 14 is a rewritable memory that can hold stored data even when the system power supply is interrupted, and is a flash memory, for example. An SRAM (Static Random Access Memory) backed up by a battery may be used. The nonvolatile memory 14 stores coefficients used for various processes, information registered by the user, designated route information, past travel history information, and the like.
Further, in this example, the nonvolatile memory 14 is also used for storing a user database, which will be described later.

  The CPU 11 performs information processing and control processing necessary as a navigation device based on a program stored in the ROM 12. For these information processing and control processing, the RAM 13 is used as a work area, or information stored in the ROM 12 or the nonvolatile memory 14 is referred to.

  The interface 15 includes an arithmetic processing unit 1 and other units, that is, an operation input unit 2, a video signal processing unit 3, a position detection unit 4, a sensor unit 5, a disk drive device unit 6, a voice synthesis / reproduction processing unit 7, and an image analysis. Input / output is performed between the unit 17 and the communication unit 18. The interface unit 15 inputs operation information and various input information from the operation input unit 2, position information from the position detection unit 4, and detection information by the sensor unit 5, and transmits them to the CPU 11. The interface unit 15 supplies control information, image data, and audio data output from the CPU 11 to the video signal processing unit 3 and the audio synthesis / reproduction processing unit 7 to execute necessary display output and audio output as a navigation system. . Further, the interface unit 15 transmits an access request from the CPU 11 to the disk drive device unit 6 and causes the disk drive device unit 6 to execute required data reproduction. Further, the interface unit 15 takes in the navigation information transferred by the reproduction operation in the disk drive device unit 6 into the arithmetic processing unit 1. The interface unit 15 transmits the facial feature data 17 generated by the image analysis unit 17 to the CPU 11. The interface unit 15 exchanges data for data communication performed by the communication unit 18.

FIG. 2 shows functions that the CPU 11 has, that is, functions that are realized by arithmetic processing based on a program stored in the ROM 12 or the like.
As functions of the CPU 11, a navigation processing control function 21, a facial feature data correspondence learning / setting function 22, and a facial feature data correspondence control function 23 are provided. Each of these functions may be realized by individual program modules, and a program for executing a plurality of functions may be mixed in one program.

The navigation processing control function 21 is a function for controlling processing operations required as a navigation device, and performs processing instructions and timing control for other units to execute systematic navigation operations.
For example, the navigation processing control function 21 performs processing such as registration of the user's destination and relay point, route search, and route guidance accompanying the progress of the host vehicle. Also, a normal navigation mode and a simple navigation mode, which will be described later, are set, and processing instructions corresponding to the mode are given.
The navigation processing control function 21 controls the disk drive device unit 6 to read navigation information from the disk 90. For example, map information, map drawing data, POI data, guide information, etc. in a required range are read into the RAM 13 according to the vehicle position, the position designated by the user, or the route information designated by the user.
Further, the navigation processing control function 21 determines the own vehicle position and the behavior of the own vehicle based on the position information from the position detection unit 4 and the detection information from the sensor unit 5.
The navigation processing control function 21 generates image data to be displayed on the display unit 8, for example, image data for a map image, a menu image, an icon image, various image images, and various guide images.
The navigation processing control function 21 performs voice output control such as route guidance voice.

The face feature data correspondence learning / setting function 22 is a function for registering user information about each user identified by the face feature data FDT in the user database by automatic learning processing or setting processing according to user operation. .
The user database is a database as shown in FIG. 5, for example, and this user database is stored in a storage area in the nonvolatile memory 14, for example.
As shown in FIG. 5, the user database is a database for registering user information for each face feature data FDT (FDT1, FDT2,...), That is, for each individual user. In this case, destination history information, preference type information, and voice guide selection information are cited as examples of user information.

For example, it is assumed that the face feature data FDT1 is “user A” and the face feature data FDT2 is “user B”.
The destination history information is obtained by registering destination information set by each user.
For example, when the user A selects a destination using the navigation device 10, destination information PP11, PP12,... Indicating the destination is registered as destination history information corresponding to the face feature data FDT1.
When the user B selects a destination using the navigation device 10, destination information PP21, PP22,... Indicating the destination is registered as destination history information corresponding to the face feature data FDT2.

The preference type information is information indicating a field in which the user is interested and a field (preference type) in which a guide as a navigation device is desired. This includes, for example, “sports”, “fashion”, “shopping”, “music”, “gourmet”, “leisure”, and fields that are further reclassified (for example, “baseball”, “soccer”, “ Information indicating “tennis”, etc.) is the preference type information. This type may be in accordance with the classification in the POI data.
For example, if the field in which the user A is interested is “sports” and “music”, the value LJ1 indicating “sports” or the value LJ2 indicating “music” as preference type information corresponding to the face feature data FDT1 Is registered.
If the field in which user B is interested is “fashion”, “shopping”, and “gourmet”, the value LJ3 indicating “fashion” as “preference type information” corresponding to the face feature data FDT2, “shopping” is set. A value LJ4 indicating, a value LJ5 indicating “gourmet”, and the like are registered.

The voice guide selection information is information indicating the content selected by the user for outputting voice navigation.
As the navigation operation by the navigation processing control function 21, various guide outputs by voice are performed. The contents of the voice include, for example, “route guidance voice” instructing right turn, left turn, road width, number of lanes, curve, etc. , “Road information voice” to guide up and down, “Congestion information voice”, “Toll road toll guide voice”, “Recommended driving guidance voice” to guide the appropriate course and lane, etc. There are various types such as “attention voice” to be urged and “operation guide voice” of the operation method of the navigation device 10.
Among these, a voice guide that each user thinks is necessary is registered as voice guide selection information.
For example, when the user A selects that only the minimum necessary voice guidance is required, for example, a value AG1 indicating “route guidance voice” is registered corresponding to the face feature data FDT1.
Also, when the user B selects “route guidance voice”, “traffic congestion information voice”, and “operation guide voice” as necessary, values AG1, AG2, and AG3 indicating these are registered corresponding to the face feature data FDT2. Is done.

  Depending on the facial feature data correspondence learning / setting function 22, a data group corresponding to the facial feature data FDT is generated as such a user database.

The face feature data correspondence control function 23 is a function that performs control based on the contents registered in the user database in map image display and voice output as navigation operations. That is, the face feature data correspondence control function 23 discriminates a user (for example, a driver) who is in the vehicle from the face feature data FDT, and if the user is a user of the face feature data FDT1, the face feature data FDT1 is stored in the user database. The output control of the image and the sound is performed so as to follow the contents registered corresponding to.
For example, when the user A is a driver, icons and names of facilities and stores corresponding to the preference type information LJ1 and LJ2 registered corresponding to the face feature data FDT1 are displayed on the map image. Information such as facilities and stores corresponding to the preference type information LJ can be obtained by searching the POI data. Further, each destination stored as destination history information corresponding to the face feature data FDT1 is displayed. The voice guide is controlled so that only the voice of the content selected as the voice guide selection information of the face feature data FDT1 is output.
On the other hand, when the user B is a driver, icons and names of facilities and stores corresponding to the preference type information LJ3, LJ4, and LJ5 registered corresponding to the face feature data FDT2 are displayed on the map image. Further, each destination stored as the destination history information corresponding to the face feature data FDT2 is displayed. The voice guide is controlled so as to output only the voice of the content selected as the voice guide selection information of the face feature data FDT2.
That is, the face feature data correspondence control function 23 controls to switch the contents to be displayed and output in the navigation process executed by the navigation process control function 21 for each user.

  The processing operation of the navigation device 10 having the above configuration will be described. In the following operation example, the imaging unit 16 sets the range around the position of the driver's face when sitting in the driver's seat as the subject direction, and thus the image analysis unit 17 uses the driver's face feature data FDT. It is assumed that it is generated.

FIG. 6 shows processing of the CPU 11 when the navigation device 10 is turned on. For example, the navigation device 10 is turned on when the user sets the engine key of the automobile to the accessory position or starts the engine.
When the power is turned on, the CPU 11 performs activation processing as step F101, and loads the operation program and controls activation of each unit.

When the power-on startup process is completed, the CPU 11 starts the imaging process of the imaging unit 16 and the generation process of the facial feature data FDT of the image analysis unit 17 in step F102, and the facial feature data FDT from the image analysis unit 17 is started. Start importing.
The generation of the facial feature data FDT in the image analysis unit 17 and the capture of the facial feature data FDT by the CPU 11 may be performed continuously at a time interval required for the generation of the facial feature data FDT, or You may make it perform regularly with the time interval of a grade. Also, considering that the driver does not frequently change when the car engine is activated, after the facial feature data FDT is captured during driving, the driver may stop the vehicle, stop the engine, etc. Timing that may be replaced may be detected from the information of the sensor unit 5, and the face feature data FDT may be newly fetched accordingly.

After fetching the face feature data FDT, the CPU 11 checks in step F103 whether the face feature data FDT corresponds to a user registered in the user database. That is, the user database is searched to check whether the same (or similar within a certain allowable range) face feature data FDT exists.
Registration of the user database is performed by the processing described later with reference to FIGS. 8 and 9, but if the CPU 11 has performed registration processing or learning registration processing according to user operations in the past, User information having contents as shown in FIG. 5 is stored for one or a plurality of face feature data FDT.

If the facial feature data FDT captured this time is facial feature data FDT that is not registered in the user database, the CPU 11 advances the process to step F106, sets a normal navigation operation, and performs the navigation process of step F107. To move on. The normal navigation operation setting is, for example, a setting determined by default in the apparatus as the content to be displayed and output in the navigation process and the content of the voice guide to be output.
In the navigation processing at step F107, based on the control of the navigation processing control function 21 described above, the navigation information is read from the disk drive device section 6, the current position information is acquired from the position detection section 4, and the sensor section 5 is detected. Acquiring information, communicating via the communication unit 18, and detecting an operation from the operation input unit 2, and based on these, display a map image or the like on the display unit 8 to the user, or from the speaker unit 9 The guide voice is output.

On the other hand, when it is determined in step F103 that the current facial feature data FDT (that is, the driver) is the facial feature data FDT registered in the user database, the CPU 11 performs processing in steps F104 and F105 as facial feature data. Processing by the correspondence control function 23 is performed. First, in step F104, user information is read from the user database. For example, if the captured face feature data FDT is “FDT1”, destination history information, preference type information, and voice guide selection information registered for the face feature data “FDT1” in the user database of FIG. Will be read.
In step F105, the CPU 11 sets a navigation operation based on the read user information. This is a process in which the facial feature data correspondence control function 23 instructs that the display process or the voice guide output process based on the user information be performed in the navigation operation process executed by the navigation process control function 21. In this process, the operation setting related to the display output and the voice guide output is set not to the normal setting but to the setting corresponding to the individual user who is the current driver.
Subsequently, as the navigation processing in step F107, the navigation operation is performed based on the control of the navigation processing control function 21 as described above. In this case, the display output and the setting state corresponding to the individual user are performed. Voice guidance output is performed.

In addition, after the start of the capture of the facial feature data FDT in step F102, the CPU 11 sequentially detects the facial feature even during the navigation operation in step F107, for example, when there is a possibility that the driver may change regularly. Data FDT is captured, and it is confirmed whether or not the captured facial feature data FDT has changed (that is, whether or not the driver has changed).
If no change in the face feature data FDT is detected, the navigation processing in step F107 is continued in the set state up to that point. However, if a change in the face feature data FDT is detected, the process returns from step F108 to F103, and is described above. Perform the process.
Therefore, if the driver is replaced, the navigation operation setting is performed again in step F105 or F106, and the subsequent navigation operation setting in step F107 is changed.

As can be seen from the process of FIG. 9, in the navigation process of step F107, the operation setting, that is, the display content and the voice guide output content are switched according to the driver. An output example based on such an operation setting corresponding to an individual user is shown.
For example, FIGS. 7A and 7B are examples of map images displayed on the display unit 8. For example, a map image 30 centered on the vehicle position 31 is displayed, and a destination 32 (or waypoint) is displayed on the map image 30. Although not shown in the figure, the route to the destination 32 is displayed in a different color or the like.
Here, FIG. 7A shows a display example when the driver is the user A (face feature data FDT1), and FIG. 7B shows a display example when the driver is the user B (face feature data FDT2). Yes.

7 (a) and 7 (b), the basic map image 30 is the same, but in the case of FIG. 7 (a), preference type information registered in the user database corresponding to the face feature data FDT1. The destination history information is reflected.
That is, if the preference type information indicates that the user A likes “sports” or “music”, an image 33 corresponding to the preference type information is displayed as shown in FIG. For example, the CPU 11 refers to the POI data, detects stores and facilities related to “sports” and “music” within the currently displayed map range, and displays them as an image 33 for presenting the stores and the like. By doing so, a sport shop, a music shop, etc. are clearly shown as the image 33.
Further, if a point registered in the destination history information as a destination in the past by the user A exists in the currently displayed map range, an image 34 presenting the past destination is displayed.
By performing such a display, the user A can recognize that there are shops and facilities that he / she is interested in near the current position, a place visited in the past, and the like, and a useful map display for the user A It becomes.

On the other hand, when the user B is a driver, the display is as shown in FIG.
That is, if the preference type information indicates that the user B likes “fashion”, “shopping”, etc., an image 33 corresponding to the preference type information is displayed as shown in FIG. 7B. . For example, the CPU 11 refers to the POI data, detects stores and facilities related to “fashion” and “shopping” within the currently displayed map range, and displays them as an image 33 for presenting the stores and the like. By doing so, a fashion-related store or the like is clearly shown as the image 33.
Further, if a point where the user B has been registered in the destination history information as a destination in the past is within the currently displayed map range, an image 34 presenting the past destination is displayed.
By performing such display, the user B can recognize that there is a store or facility in which he / she is interested in near the current position, a place visited in the past, and the like, and a map display useful for the user B It becomes.

  For example, in the navigation process in step F107 as described above, the CPU 11 displays a map corresponding to the user's preference and history determined by the face feature data FDT, that is, a store corresponding to the preference type information registered in the user database, By displaying facilities and the like and past destinations by destination history information, it is possible to perform effective navigation display according to the individual user.

Further, the content of the voice guide output from the speaker unit 9 is also controlled by the CPU 11 according to the registered content of the user database.
In step F107, the CPU 11 requires various types of voice such as route guidance voice, road information voice, traffic jam information voice, toll road toll guidance voice, recommended driving guidance voice, caution voice, and operation guide voice as voice guide information. Output at the timing.
That is, the CPU 11 outputs the route guidance voice for notifying the right turn / left turn / straight ahead from the speaker unit 9 based on the current position information, the route search result to the destination, the map data, for example, when approaching the intersection, etc. Let
Further, the CPU 11 outputs road information voice, toll guidance voice, recommended travel guidance voice, caution voice, etc. at a required timing according to the information added to the map data and the current position.
Further, the CPU 11 outputs the sound of the traffic jam information received via the communication unit 18 at a required timing.
In addition, the CPU 11 outputs an operation guide voice for the user to perform an operation with the operation input unit 2 at a required timing according to the necessity of the operation, the content being displayed, and the like.

For example, in the normal setting in step F106, it is assumed that the setting is made so as to output all the guide voices of these contents.
On the other hand, in the case where the setting according to the individual user is made in step F105, only the voice registered as the voice guide selection information in the user database is output.
That is, for the user A, when only the route guidance voice is registered as the voice guide selection information corresponding to the facial feature data FDT1, when the user A is the driver, only the route guidance voice is output in the setting of step F105. During the navigation process of step F107, only route guidance voice is output as voice guide information.
For user B, when route guidance voice, traffic jam information voice, and operation guide voice are registered as voice guide selection information corresponding to face feature data FDT2, if user B is a driver, the setting of step F105 is performed. The route guidance voice, the traffic jam information voice, and the operation guide voice are set to be output in step F107. During the navigation process in step F107, the voice guide information includes the route guidance voice, the traffic jam information voice, and the operation guide voice. Is output only.

  For example, during the navigation processing in step F107 as described above, the CPU 11 performs guide voice output according to the voice guide selection information for the user determined by the face feature data FDT, which is suitable for the individual user. Voice guidance can be performed.

By the way, the above explanation is based on the assumption that user information corresponding to individual users is registered in the user database. Hereinafter, a process for registering user information in the user database will be described.
Registration of user information in the user database is performed based on a user registration operation or automatically by a learning operation.

  FIG. 8 shows processing of the CPU 11 when the user performs a registration operation. The process in FIG. 8 may be started when the user performs a registration mode operation using the operation input unit 2 during the navigation process in step F107 in FIG. The processing of FIG. 8 is performed as processing by the CPU 11 and the facial feature data correspondence learning / setting function 22.

When the CPU 11 detects that the user has performed the registration mode operation using the operation input unit 2, the process proceeds from step F201 to F202 in FIG.
In step F202, the CPU 11 checks whether or not the facial feature data FDT captured at that time is the facial feature data FDT registered in the user database. If the facial feature data FDT has not been taken in for a certain period of time, the facial feature data FDT is newly fetched at this point, and the facial feature data FDT is registered in the user database. It may be determined whether or not there is.

If the current face feature data FDT (that is, the current driver) is face feature data FDT not registered in the user database, the process proceeds from step F203 to F204, and the current face feature data FDT is set as new face feature data. After newly registering in the user database, the process proceeds to step F205.
If the current face feature data FDT is already registered face feature data, the process proceeds from step F203 to F205 as it is.

In step F205, the CPU 11 displays a registration screen on the display unit 8.
The registration screen is, for example, a menu format that allows the user to select the user's hobbies / preferences and the content of the voice guide. The user can select and operate the hobbies / preferences and desired audio while viewing the screen display. Make it possible to enter the contents of the guide.
For example, as a selection screen for preference types, items in each field such as “sports”, “fashion”, “shopping”, “music”, “gourmet”, and “leisure” are displayed in a menu format so that the user can select from these items. . Of course, the menu screen may be hierarchically advanced so that these lower classifications can be selected.
In addition, as a selection screen for the content of the guide voice, “route guidance voice”, “road information voice”, “congestion information voice”, “toll road voice guidance voice”, “recommended driving guidance voice”, “caution voice”, “operation guide” Items such as “speech” are displayed, and the user can output and select the sound content.

The CPU 11 displays such a registration screen, and waits for a user's selection decision operation while switching the menu screen according to the user operation. When the user performs an operation of selecting and determining an item, the process proceeds from step F206 to F207 to store the selected item. This is repeated until the registration end operation is performed.
Each selection result is memorize | stored in CPU11 because a user performs operation which selects a preference classification and a guide audio | voice according to the display of a menu screen or operation guidance, for example.
When the user completes the selection operation for registration and performs the registration end operation, the CPU 11 proceeds from step F208 to F209, and registers the selected content in the user database in correspondence with the face feature data FDT. That is, according to the user's selection operation, the preference type information and the voice guide selection information are registered in the user database in a state corresponding to the user's face feature data FDT.

  In this process, a new user is registered unconditionally (steps F203 → F204). However, when registering the new user's face feature data FDT, for example, a password set in advance by the car owner or the like May be requested so that the person who gets on the vehicle is not registered unnecessarily.

FIG. 9 shows a processing example in which the CPU 11 registers the destination history information in the user database and registers the preference type information by the learning operation.
The process in FIG. 9 is also performed during the navigation process in step F107 in FIG. 6. In this case, the process is performed when the user performs a destination designation operation using the operation input unit 2.
In general, in an in-vehicle navigation device, a suitable route is calculated according to a user setting a destination, and a navigation operation for displaying the route on a map image is performed on a display unit.
For this navigation operation, it is first necessary for the user to input a destination, but there are various methods for inputting the destination. For example, a method for inputting an address or a place name of the destination. And a method for specifying a position on a map, and a method for searching and selecting a destination.
When the user intends to specify a destination, the CPU 11 first causes the display unit 8 to execute a display requesting the user to input information necessary for route search such as a departure point, a destination, and a waypoint. The user inputs a destination or the like using the operation input unit 2 in accordance with menu display, guide display, map display, or the like on the display unit 8. The CPU 11 sets a departure place, a waypoint, a destination, and the like according to a user input.
If such a destination designation operation is performed, the CPU 11 advances the process to step F302, and performs route search calculation for the destination (route point). That is, the optimum route from the starting point to the destination, and in some cases, the optimum route via the waypoint is searched. Then, the search result is determined as route information to the destination.
The route information determined here is reflected in the display and voice guidance in the subsequent navigation processing.

In the CPU 11, up to step F302 is performed as normal navigation processing control by the navigation processing control function 21, but in the case of this example, step F303 and subsequent steps are performed as processing by the facial feature data correspondence learning / setting function 22 of the CPU 11.
In step F303, the CPU 11 checks whether or not the facial feature data FDT captured at that time is the facial feature data FDT registered in the user database. If the facial feature data FDT has not been taken in for a certain period of time, the facial feature data FDT is newly fetched at this point, and the facial feature data FDT is registered in the user database. It may be determined whether or not there is.

If the current face feature data FDT (that is, the current driver) is face feature data FDT not registered in the user database, the process proceeds from step F304 to F305, and the current face feature data FDT is set as new face feature data. After newly registering in the user database, the process proceeds to step F306.
If the current face feature data FDT is already registered face feature data, the process proceeds from step F304 to F306 as it is.

In step F306, the destination information set this time is registered in the user database. That is, destination information is additionally registered as destination history information in the user database in association with the face feature data FDT acquired at this time.
With this process, every time a user designates a destination in the navigation apparatus 10, the current destination is registered in the user database as destination history information corresponding to the user.

In step F307, a preference type determination process is performed from the registered destination history information. That is, a learning process is performed in which the destination added to the destination history information this time is included, the type of each destination registered in the destination history information at this time is determined, and the user's preference is estimated.
For example, for each destination registered in the history, the destination where classification can be determined is extracted with reference to the POI data. Then, the extracted classifications of each destination, for example, classifications such as sports relations and shopping relations, are totaled, and the classification that the user has many opportunities as the destinations is determined. For example, the user's preference type is estimated by performing a process such as determining a classification that is a destination more than a predetermined number of times.

For example, if a certain preference type cannot be determined in Step F307 because there are a small number of destinations registered in the destination history information or the destination is too dispersed in various fields, it is determined that the determination cannot be made in Step F308. Finish the process.
On the other hand, if the user's preference type can be estimated from the classification tendency of the destination registered in the destination history information, the process proceeds to step F309. In step F309, it is determined whether or not the preference type is already registered in the user database as the preference type information corresponding to the user (face feature data FDT). If not registered, the process proceeds to step F310, and the CPU 11 registers the currently determined preference type in the preference type information of the user database as a new preference type corresponding to the user (face feature data FDT), and ends the process.

For example, according to the processing of FIG. 9, destination history information is additionally registered in the user database in accordance with the user's destination designation, and the user preference type estimated from the destination history information is registered as preference type information. become.
Then, by registering the user information in the user database through the processes shown in FIGS. 8 and 9, as described in the process of FIG. 6 above, the user's personal destination history, preference type information, voice guide selection information The navigation process corresponding to is executed.

[2. Second Embodiment: Audio Playback Device]

Next, a second embodiment will be described. The second embodiment is an example of an on-vehicle audio playback device.
FIG. 10 shows the configuration of the audio playback device 100. The audio playback apparatus 100 is an in-vehicle audio playback apparatus attached to a front console of an automobile as shown in FIG. 12, for example, a playback function or a radio tuner function of an audio disc such as a CD (Compact Disc) or a minidisc. Etc., and a device for outputting reproduced sound such as music from a speaker disposed in the automobile.

In the audio reproduction device 100 of FIG. 10, the CPU 110 controls each unit for audio reproduction in accordance with an operation program and operation input.
The memory unit 111 is used for storing program codes executed by the CPU 110 and temporarily storing work data being executed. In the case of this figure, the memory unit 111 is shown as including both a volatile memory and a nonvolatile memory. For example, it includes a non-volatile memory such as a ROM for storing a program, a RAM for a calculation work area and various temporary storages, and an EEP-ROM (Electrically Erasable and Programmable Read Only Memory).
The nonvolatile memory as the memory unit 111 stores a user database for registering user information corresponding to individual users.

The operation unit 112 includes various operation keys and operators such as a dial disposed on the front panel of the audio playback device 100. The user can perform channel selection operations such as an audio mode (source switching) operation, volume (volume) operation, sound quality operation, disc playback operation, fast forward / fast reverse operation, cueing operation, radio broadcasting, and the like by the operation unit 112. The
The display unit 113 is, for example, a display unit using a liquid crystal panel, and is provided on the front panel of the audio playback device 100 as shown in FIG. In this display unit 113, based on the control of the CPU 110, an audio mode display, a display indicating operations such as playback and fast-forwarding, a spectrum analyzer display, a volume level display, a track number of a song being played back, and a received broadcast The station is displayed.

As a source of audio signals to be reproduced and output, a disk reproducing unit 115, a tuner unit 116, and an external input unit 117 are shown. The selector 118 is a circuit that selects the source of these audio signals.
In addition, as a source of the audio signal, an HDD (Hard Disc Drive) in which music or the like is stored, a memory card reader for reproducing the audio signal from the memory card, or the like may be provided.

The disc playback unit 115 is a playback unit for an optical disc such as a CD (Compact Disc) / DVD (Digital Versatile Disc), or a playback unit for an optical disc such as a mini disc / magneto-optical disc.
When the user performs an operation related to disc playback from the operation unit 112, the CPU 110 causes the disc playback unit 115 to start a playback operation or to execute an operation (such as a cue search) according to the operation. When playback is performed by the disk playback unit 115, the CPU 110 controls the selector 118 to select the disk playback unit 115. The audio signal reproduced from the disk by the disk reproducing unit 115 is supplied to a DSP (Digital Signal Processor) 114 via the selector 118 as digital data.

  The tuner unit 116 is a broadcast receiver such as a radio broadcast and receives / demodulates broadcast waves. When the tuner unit 116 performs broadcast reception, the CPU 110 controls the selector 118 to select the tuner unit 116. The audio signal demodulated by the tuner unit 116 is converted into digital data and supplied to the DSP 114 via the selector 118.

The external input unit 117 is a circuit unit that inputs an audio signal from an external device that is also mounted in the automobile, such as a changer system or a navigation device.
When the external input unit 117 is designated as a source by a user operation, the CPU 110 controls the selector 118 to select the external input unit 117. The audio signal (digital data) input from the external device by the external input unit 117 is supplied to the DSP 114 via the selector 118.

The DSP 114 performs various signal processing on the output audio signal. For example, acoustic effect processing such as equalization / reverb, volume adjustment processing, time alignment processing, channel divide processing, speaker balance adjustment processing, and the like are performed.
Volume adjustment processing, sound effect processing, speaker balance adjustment processing, and the like are performed based on a user operation from the operation unit 112.

The DSP 114 performs various types of audio signal processing as described above, and outputs, for example, a 4-channel audio signal output from a 4-channel speaker.
The audio signals of the respective channels are converted into analog signals by the D / A converters 119a, 119b, 119c, and 119d, and then amplified by the amplifier units 120a, 120b, 120c, and 120d, and the speakers 121a, 121b, 121c, and 121d, respectively. Is output as audio. The speakers 121a, 121b, 121c, and 121d are arranged in the vehicle as, for example, a front right speaker, a front left speaker, a rear right speaker, and a rear left speaker.
Note that the speaker balance adjustment processing performed by the DSP 114 is variable setting processing such as volume balance between the front speaker and the rear speaker and volume balance between the left speaker and the right speaker.
The user can arbitrarily set the volume balance between the front side speaker and the rear side speaker by operation.

The imaging unit 123 performs sampling, gain adjustment, video signal processing (luminance processing, color processing, correction) on an imaging optical lens system, a lens driving system, an imaging device unit using a CCD sensor or a CMOS sensor, and an image signal captured by the imaging device unit. The imaging signal processing circuit for performing processing etc. is mounted, and image data is output by the imaging operation. The imaging unit 123 images the face of the user in the car.
The imaging lens 123a in the imaging unit 123 is provided on the front panel of the audio reproduction device 100 as shown in FIG. 12, and this imaging lens 123a captures the position of the face of the user at the driver's seat as the subject direction, for example. The imaging direction is set.
Of course, the imaging direction of the imaging unit 123 may be fixed in the direction of the driver's face, but may be changed to the passenger seat direction or the rear seat direction. Alternatively, the interior of the vehicle may be imaged at a wide angle so that all faces of users in the vehicle can be captured as subjects.

The image analysis unit 124 performs analysis processing on the captured image data obtained by the imaging unit 123.
The image analysis unit 124 performs a process of extracting a human face image from the captured image data as a processing target object, and a process of generating face feature data FDT from the extracted face image. As in the case of the first embodiment, the face feature data FDT is, for example, the relative position information of the constituent elements of the face, and includes two pieces of data of the ratio information Fa and Fb described with reference to FIG.
The image analysis unit 124 supplies the face feature data FDT generated from the captured image data to the CPU 110.

FIG. 11 shows functions that the CPU 100 has, that is, functions that are realized by arithmetic processing based on a program stored in a ROM or the like of the memory unit 111.
As functions of the CPU 100, an audio playback device control function 131, a face feature data correspondence learning / setting function 132, and a face feature data correspondence control function 133 are provided. Each of these functions may be realized by individual program modules, and a program for executing a plurality of functions may be mixed in one program.

The audio playback control function 131 is a function for controlling processing operations required as an audio playback device, and performs processing instructions and timing control for other units to execute necessary audio playback operations.
For example, operation control of the disc playback unit 115 and the tuner unit 116 which are audio sources, switching control of the hard-to-come end portion 118, control of audio signal processing of the DSP 114, imaging control of the imaging unit 123, display control of the display unit 113 Processing such as detection of an operation input of the operation unit 112 is performed.

The face feature data correspondence learning / setting function 132 is a function for registering user information about each user identified by the face feature data FDT in the user database by automatic learning processing or setting processing according to user operation. .
The user database is a database as shown in FIG. 13, for example, and this user database is stored in a nonvolatile memory in the memory unit 111, for example.
As shown in FIG. 13, the user database is a database for registering user information for each face feature data FDT (FDT1, FDT2,...), That is, for each individual user. In this case, acoustic setting information and source setting information are given as examples of user information. The sound setting information includes sound effect value EF, volume value VOL, speaker balance adjustment value SPB, etc. for equalizing and reverb. The source setting information AS is a value indicating one of the three audio sources, for example, the disc playback unit 115, the tuner unit 116, and the external input unit 117 as shown in FIG.

For example, it is assumed that the face feature data FDT1 is “user A” and the face feature data FDT2 is “user B”.
The sound setting information is information in which sound setting values set by each user are registered. For example, the acoustic effect value EF, the volume value VOL, and the speaker balance adjustment value SPB registered by the user A for using the audio playback device 100 are registered corresponding to the face feature data FDT1.
Also, the acoustic effect value EF, volume value VOL, and speaker balance adjustment value SPB registered by the user B for use of the audio playback device 100 are registered corresponding to the face feature data FDT2.

The source setting information AS is information indicating the audio source selected by the user. For example, when the user A selects the disc playback unit 115 as an audio source for normal use, a value indicating the disc playback unit 115 is registered as source setting information AS corresponding to the face feature data FDT1. When the user B selects the tuner unit 116 as an audio source for normal use, a value indicating the tuner unit 116 is registered as source setting information AS corresponding to the face feature data FDT2.
Depending on the facial feature data correspondence learning / setting function 132, a data group corresponding to the facial feature data FDT is generated as such a user database.

The face feature data correspondence control function 133 performs control based on the contents registered in the user database regarding the source selection in the selector unit 118 and the audio signal processing in the DSP 114 during the audio reproduction operation. It is a function to do. That is, the face feature data correspondence control function 133 discriminates a user (for example, a driver) who is in the vehicle from the face feature data FDT, and if the user is a user of the face feature data FDT1, the face feature data FDT1 is stored in the user database. Control is performed so that source selection and audio signal processing are executed in accordance with the contents registered corresponding to the above.
For example, when the user A is a driver, the source indicated in the source setting information AS registered corresponding to the face feature data FDT1 is selected as the first audio source and indicated by the acoustic setting information. The sound effect, volume, and speaker balance adjustment are performed as the values to be set.
On the other hand, when the user B is a driver, the source indicated in the source setting information AS registered corresponding to the face feature data FDT2 is selected as the first audio source, and the acoustic setting information As the values shown, the acoustic effect, volume, and speaker balance are adjusted.
That is, the face feature data correspondence control function 133 controls to switch the source selection and audio signal processing contents during audio playback for each user.

  The processing operation of the audio playback apparatus 100 having the above configuration will be described. Note that the imaging unit 123 assumes that the range centered on the position of the driver's face when sitting in the driver's seat is the subject direction, and therefore the image analysis unit 124 generates the driver's face feature data FDT. .

FIG. 14 shows processing of the CPU 110 when the audio playback device 100 is turned on. For example, the audio playback device 100 is turned on when the user sets the engine key of the car to the accessory position or starts the engine.
When the power is turned on, the CPU 110 performs activation processing as step F401, and loads an operation program and controls activation of each unit.

When the power-on startup process is completed, the CPU 110 starts the imaging process of the imaging unit 123 and the generation process of the facial feature data FDT of the image analysis unit 124 in step F402, and the facial feature data FDT from the image analysis unit 124 is started. Start importing.
As in the case of the first embodiment, the generation of the facial feature data FDT by the image analysis unit 124 and the capture of the facial feature data FDT by the CPU 110 are always performed at time intervals required for the generation of the facial feature data FDT. It may be performed continuously or periodically at a certain time interval. Further, it is possible to detect when the vehicle is stopped, when the engine is stopped, etc., and to newly capture the face feature data FDT at a timing at which the driver may change.

When the facial feature data FDT is captured, the CPU 110 confirms in step F403 whether the facial feature data FDT corresponds to a user registered in the user database. That is, the user database is searched to check whether the same (or similar within a certain allowable range) face feature data FDT exists.
If the facial feature data FDT fetched this time is facial feature data FDT that is not registered in the user database, the CPU 110 advances the processing to step F406, sets a normal audio reproduction operation, and performs the audio of step F407. Change to playback process. The normal audio reproduction setting is a setting determined by default in the apparatus, for example, as the audio source to be selected first, the sound effect value, the volume value, and the speaker balance adjustment value. Alternatively, these values at the time of previous power-off may be stored (so-called last memory) and set according to the stored values.

  In the audio playback process in step F407, playback audio output from the selected audio source is performed based on the user's operation or the control of the audio playback control function 131 described above.

On the other hand, if it is determined in step F403 that the current facial feature data FDT (that is, the driver) is the facial feature data FDT registered in the user database, the CPU 110 performs processing in steps F404 and F405 to perform facial feature data. Processing by the correspondence control function 133 is performed. First, in step F404, user information is read from the user database. For example, if the captured face feature data FDT is “FDT1”, the acoustic setting information EF, VOL, SPB, and source setting information AS registered for the face feature data “FDT1” in the user database of FIG. read out.
In step F405, the CPU 110 sets audio playback operation, that is, audio source and audio signal processing, based on the read user information.
Subsequently, an audio playback operation is performed as the audio playback processing in step F407. In this case, the operation is performed in a setting state corresponding to the individual user. That is, in this case, the CPU 110 controls the DSP 114 so that the volume adjustment, the sound effect processing, and the speaker balance adjustment processing are performed with the values indicated in the sound setting information EF, VOL, and SPB, and the selector 110 is selected based on the source setting information AS. The switching control of the unit 118 and the operation control of the part designated as the source are performed.

It should be noted that after capturing the facial feature data FDT in step F402, the CPU 110 sequentially detects the face even during the audio playback operation in step F407, for example, periodically or when there is a possibility of change of the driver. Feature data FDT is captured, and it is confirmed whether or not the captured facial feature data FDT has changed (that is, whether or not the driver has changed).
If no change in the face feature data FDT is detected, the audio reproduction process in step F407 is continued in the setting state until then. If a change in the face feature data FDT is detected, the process returns from step F408 to F403, The above-described processing is performed.
Therefore, if the driver changes, the audio playback operation setting is performed again in step F405 or F406, and the subsequent navigation operation setting in step F407 is changed.

As can be seen from the processing in FIG. 14, in the audio reproduction processing in step F407, the operation setting, that is, the value of source selection or audio signal processing is switched according to the driver.
For example, the tendency of selecting an audio source differs for each individual user, for example, a person who always listens to disc playback music such as a CD or a person who mainly listens to radio broadcasts.
There are also differences between those who like loud sounds and those who don't like loud sounds. There are also preferences for setting sound effects and speaker balance.
In the case of this example, for each user, an audio source, sound volume, sound effect, speaker balance adjustment, and the like suitable for the user are automatically performed. This makes it possible to enjoy comfortable audio playback for each user, without having to switch the source selection or re-adjust the volume, sound effects, and speaker balance each time the user changes. become.

Registration of user information in the user database is performed based on a user registration operation or automatically by a learning operation.
The processing of the CPU 110 when the user performs a registration operation may be almost the same as that in FIG.
That is, after performing the processing of steps F201 to F204 in FIG. 8, in step F205, a screen for selecting an audio source and a screen for setting the volume, sound effect, and speaker balance are displayed. In step F209, acoustic setting information (EF, VOL, SPB) and source setting information AS may be registered in the user database as user information corresponding to the facial feature data FDT.
In order to register the sound setting information (EF, VOL, SPB), for example, the user can determine the sound volume, sound effect state, and speaker balance adjustment state by changing the user's operation in a state in which reproduced sound is output. These values when operated may be registered as sound effect value EF, volume value VOL, and speaker balance adjustment value SPB as sound setting information.

In addition to registration based on such user operations, registration by learning processing can also be performed.
For example, the user database has contents as shown in FIG. That is, the acoustic setting history information and the source history information are registered in the user database as well as the acoustic setting information (EF, VOL, SPB) and the source setting information AS.

For example, during the audio reproduction process in step F407 in FIG. 14, the process in FIG. 16 is performed according to the user's operation.
FIG. 16 shows processing when the user performs volume change operation, sound effect change operation, speaker balance change operation, and source change operation.

When the user performs a volume change operation, the CPU 110 advances the process from step F501 to F505, and controls the DSP 114 for volume change according to the operation. As a result, the output volume of the music being played is changed. Next, in step F506, the volume value after this change is registered in the user database. That is, it is additionally registered as the volume value VOLxx of the sound setting history information in association with the face feature data FDT acquired at this time.
In step F505, the volume value VOLxx additionally registered in the sound setting history information is added, and the volume value registered in the history at that time (for example, in the case of the user A, VOL11, VOL12... VOLxx in FIG. 15). Is used to calculate the volume value preferred by the user. For example, the average value of the volume values VOL11, VOL12,... Registered in the history is calculated, and this is set as the volume value preferred by the user. When taking the average value, it is preferable to obtain the distribution and standard deviation of each volume value registered as a history, and exclude the volume value extremely different from the representative volume from the average value calculation.
For example, when the volume value is calculated as an average value or the like, it is updated as the volume value VOL in the new acoustic setting information.

In addition, when the user performs an operation of changing the equalizing, reverb level, or the like as the sound effect, the CPU 110 advances the process from step F502 to F508, and controls the DSP 114 to change the sound effect value according to the operation. As a result, the sound effect of the output volume of the music being played is changed. Next, in step F509, the sound effect value after this change is registered in the user database. That is, it is additionally registered as the acoustic effect value EFxx of the acoustic setting history information in association with the face feature data FDT acquired at this time.
In step F510, the acoustic setting value EFxx additionally registered in the acoustic setting history information is added, and the acoustic effect values registered in the history at that time (in the case of the user A, EF11, EF12... EFxx) are used. The sound volume value that this user likes is calculated. For example, the average value of the acoustic effect values EF11, EF12,... Registered in the history is calculated, and this is set as the acoustic effect value preferred by the user. In this case as well, sound effect values that are extremely different from typical sound effect values in the sound effect values registered in the history may be excluded from the average value calculation.
For example, when the sound effect value is calculated as an average value or the like, it is updated as the sound effect value EF in the new sound setting information.

In addition, when the user performs a speaker balance change operation, the CPU 110 advances the processing from step F503 to F511, and controls the DSP 114 to change the speaker balance according to the operation. As a result, the speaker balance of the output of music or the like being reproduced is changed. In step F512, the speaker balance adjustment value after the change is registered in the user database. That is, it is additionally registered as the speaker balance adjustment value SPBxx of the sound setting history information in association with the face feature data FDT acquired at this time.
In step F513, the speaker balance adjustment value SPBxx additionally registered in the acoustic setting history information is added, and the speaker balance adjustment value (SPB11, SPB12... SPBxx in the case of the user A) registered at that time is added. Use this to calculate the speaker balance adjustment value preferred by this user. An average value may be calculated for this.
For example, when the speaker balance adjustment value is calculated as an average value or the like, it is updated as the speaker balance adjustment value SPB in the new sound setting information.

Further, when the user performs an audio source switching operation, the CPU 110 advances the processing from step F504 to F514, and performs switching control of the selector unit 118 and operation control of the newly selected audio source (for example, the tuner unit 116 is selected). In this case, the operation of the tuner unit 116 is controlled). As a result, the audio source to be reproduced and output is switched. Next, in step F515, a value indicating the audio source after the change is registered in the user database. That is, a value indicating the current audio source is additionally registered in the source history information in association with the face feature data FDT acquired at this time.
In step F516, the audio source preferred by the user is discriminated using the audio source that is additionally registered in the source history information and the audio source that is registered in history at that time. For example, the disc playback unit 115 (AS 1), the tuner unit 116 (AS 2), and the external input unit 117 (AS 3) are aggregated as the audio sources, and the number of registered history is counted, and the user prefers the maximum number of audio sources. Judge as an audio source. Then, the value of the source setting information AS is updated with the determined audio source.

For example, by performing such processing, in the user database as shown in FIG. 15, the sound setting information (EF, VOL, SPB) and the source setting information AS are favored over time according to the operation at the time of use by the user. The tendency is learned, and as a result, the sound setting information (EF, VOL, SPB) and the source setting information AS indicate the user's favorite setting state. Then, when the audio playback operation is set in step F405 of FIG. 14, the acoustic setting information (EF, VOL, SPB) and the source setting information AS as such learning results are reflected, so that the user's preference In this state, audio playback is performed.

[3. Effects and modifications of the embodiment]

Although the embodiments have been described above, according to these embodiments, the user is identified based on the captured human face feature data FDT, and the user database is based on the user information corresponding to the user. By executing the operation control, an operation suitable for each user is performed.
For example, navigation output according to the user's preference and necessity can be performed as a map image display in the case of the navigation device 10 or a travel guide by voice.
Similarly, in the case of the audio playback device 100, the volume, sound quality (sound effect), speaker balance, audio source selection, and the like according to the user's preference are performed as the audio signal to be played back and output.
In other words, as an in-vehicle electronic device for a vehicle on which a plurality of users ride, individual users can perform navigation operations and audio playback operations that are comfortable for the rider (driver) without having to perform adjustment operations. it can.

Note that the present invention is not limited to the example of the embodiment, and various modifications can be considered.
The configurations of the navigation processing 10 and the audio playback device 100 are not limited to the above example, and various configurations are conceivable. A combined device of a navigation device and an audio playback device is also envisaged.
In addition to these, video equipment, television tuner equipment, and the like are also assumed as in-vehicle electronic equipment.

The user information registered in the user database is not limited to the above example. For example, in navigation devices, video playback devices, television devices, etc., the values that each user prefers are registered in the user database as adjustment values for the brightness level, contrast, sharpness, etc. of the display unit. It may be set.
The user can also select channel selection preferences for audio playback devices and television tuner devices.

In the embodiment, the user who picks up the image and detects the facial feature data FDT is the driver. However, the present invention is not limited to this, and a person in the front passenger seat or a person in the rear seat may be set as the imaging target. The person (seat position) may be variable.
Further, the entire vehicle interior can be imaged, and when a plurality of persons get on, registration corresponding to the person may be performed. That is, a user database is configured by a combination of detected face feature data FDT.
Assuming that the car navigation device 10 and the audio playback device 100 are used by a family of three users A, B, and C, the subject direction is set so that the imaging units 16 and 123 capture the entire interior of the vehicle interior. In this case, the detected face feature data FDT may be as follows.
・ User A only ・ User B only ・ User C only ・ Users A and B
・ Users A and C
・ Users B and C
・ Users A, B, and C
These seven cases are separately registered in the user database. In each case, user information is registered according to the operation status and the registration operation.
In this way, appropriate navigation processing and audio reproduction processing can be realized according to the combination of the person on board.

By providing the imaging units 16 and 123 in the navigation device 10 and the audio playback device 100, the following processing is also possible.
For example, the brightness of the vehicle interior is detected from the captured image signal, and the brightness of the display units 8 and 113 is set in accordance with the brightness of the vehicle interior. This makes it possible to reduce the brightness of the display screen when the room gets dark, and to adjust the brightness of the display screen for car audio and car navigation, which is very difficult to see during sunny days. .

  Also, registering the facial feature data FDT in the user database can be used for car security. That is, when a person with facial feature data FDT that is not registered in the user database is sitting in the driver's seat, processing such as not starting the engine is possible.

  Moreover, it is also possible to use as a photo diary linked to the position information by periodically imaging the passenger compartment and recording the captured image data for a certain period together with the current position information obtained by the navigation device, for example.

1 is a block diagram of a navigation device according to a first embodiment of the present invention. It is explanatory drawing of a function structure of the navigation apparatus of 1st Embodiment. It is explanatory drawing of the installation state of the navigation apparatus of 1st Embodiment. It is explanatory drawing of the face feature data of embodiment. It is explanatory drawing of the user database of 1st Embodiment. It is a flowchart of the process of 1st Embodiment. It is explanatory drawing of the example of a map display of 1st Embodiment. It is a flowchart of the process at the time of registration operation of 1st Embodiment. It is a flowchart of the registration process by learning of 1st Embodiment. It is a block diagram of the audio reproduction apparatus of the 2nd Embodiment of this invention. It is explanatory drawing of a function structure of the audio reproduction apparatus of 2nd Embodiment. It is explanatory drawing of the installation state of the audio reproduction apparatus of 2nd Embodiment. It is explanatory drawing of the user database of 2nd Embodiment. It is a flowchart of the process of 2nd Embodiment. It is explanatory drawing of the modification of the user database of 2nd Embodiment. It is a flowchart of the learning process of 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Computation processing part, 2 Operation input part, 3 Video signal processing part, 4 Position detection part, 5 Sensor part, 6 Disk drive apparatus part, 7 Voice synthesis / reproduction processing part, 8 Display part, 9 Speaker part, 11 CPU, 12 ROM, 13 RAM, 14 Non-volatile memory, 15 Interface unit, 16 Imaging unit, 17 Image analysis unit, 21 Navigation processing control function, 22 Facial feature data correspondence learning / setting function, 23 Facial feature data correspondence control function, 100 Audio Playback device, 110 CPU 110, 111 Memory unit, 112 Operation unit, 113 Display unit, 114 DSP, 123 Imaging unit, 124 Image analysis unit, 131 Audio playback control function, 132 Face feature data correspondence learning / setting function, 133 Face feature data Corresponding control function

Claims (9)

An imaging unit arranged to image a person in the vehicle;
An image analysis unit that analyzes image data captured by the imaging unit and generates facial feature data of the captured person;
A memory unit for storing user information corresponding to the facial feature data;
A control unit that reads out user information corresponding to the facial feature data generated by the image analysis unit from the memory unit, and executes operation control based on the read-out user information;
An in-vehicle electronic device characterized by comprising: The in-vehicle electronic device is an in-vehicle navigation device,
A display unit;
A navigation information acquisition unit for acquiring navigation information;
Further comprising
When the control unit causes the display unit to execute map display using map information as navigation information acquired by the navigation information acquisition unit, the control unit corresponds to the face feature data generated by the image analysis unit. The vehicle-mounted electronic device according to claim 1, wherein display contents on a map display are controlled based on user information read from the unit. The in-vehicle electronic device is an in-vehicle navigation device,
An audio output unit;
A navigation information acquisition unit for acquiring navigation information;
A position detector for detecting current position information;
Further comprising
The control unit is generated by the image analysis unit with respect to an operation for executing audio output based on the navigation information acquired by the navigation information acquisition unit or the current position information detected by the position detection unit from the audio output unit. The in-vehicle electronic device according to claim 1, wherein the audio output content is controlled based on user information read from the memory unit corresponding to the face feature data. The in-vehicle electronic device is an in-vehicle audio playback device,
An audio signal processing unit that performs signal processing on an audio signal supplied from an audio source;
An audio output processing unit for supplying the audio signal processed by the audio signal processing unit to one or a plurality of speakers;
Further comprising
The control unit controls signal processing in the audio signal processing unit based on user information read from the memory unit corresponding to face feature data generated by the image analysis unit. Item 2. The vehicle-mounted electronic device according to Item 1. The in-vehicle electronic device is an in-vehicle audio playback device,
A selector for selecting audio signals from multiple audio sources;
An audio output processing unit for supplying the audio signal selected by the selection unit to one or a plurality of speakers;
Further comprising
The control unit controls an audio signal selection operation in the selection unit based on user information read from the memory unit corresponding to the facial feature data generated in the image analysis unit. The in-vehicle electronic device according to claim 1.   The in-vehicle electronic device according to claim 1, wherein the face feature data generated by the image analysis unit is relative position information of face components. An input unit for inputting the user information;
The in-vehicle unit according to claim 1, wherein the control unit stores the user information input by the input unit in the memory unit in association with the facial feature data generated by the image analysis unit. Electronic equipment. An operation input unit is further provided,
The control unit generates user information based on an operation content using the operation input unit or a history of the operation content, and stores the user information in the memory unit in association with the facial feature data generated by the image analysis unit. The vehicle-mounted electronic device according to claim 1. An imaging unit arranged to image a person in the vehicle;
An image analysis unit that analyzes the image data captured by the imaging unit and generates facial feature data of the captured person;
A memory unit for storing user information corresponding to the facial feature data;
As an operation control method for an in-vehicle electronic device equipped with
Generating facial feature data in the image analysis unit;
Reading user information corresponding to the facial feature data from the memory unit;
Executing operation control based on the read user information;
An operation control method for an on-vehicle electronic device, comprising:

Images