JP4032492B2 - Agent device - Google Patents

Agent device Download PDF

Info

Publication number
JP4032492B2
JP4032492B2 JP09538698A JP9538698A JP4032492B2 JP 4032492 B2 JP4032492 B2 JP 4032492B2 JP 09538698 A JP09538698 A JP 09538698A JP 9538698 A JP9538698 A JP 9538698A JP 4032492 B2 JP4032492 B2 JP 4032492B2
Authority
JP
Japan
Prior art keywords
agent
background
data
voice
response
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP09538698A
Other languages
Japanese (ja)
Other versions
JPH11272639A (en
Inventor
康二 向井
孝二 堀
松田  学
智氣 窪田
和英 足立
Original Assignee
株式会社エクォス・リサーチ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社エクォス・リサーチ filed Critical 株式会社エクォス・リサーチ
Priority to JP09538698A priority Critical patent/JP4032492B2/en
Priority claimed from US09/119,238 external-priority patent/US6249720B1/en
Publication of JPH11272639A publication Critical patent/JPH11272639A/en
Application granted granted Critical
Publication of JP4032492B2 publication Critical patent/JP4032492B2/en
Anticipated expiration legal-status Critical
Application status is Expired - Fee Related legal-status Critical

Links

Images

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an agent device, and more particularly, to an agent device having a communication function that enables conversation within a vehicle with an anthropomorphic agent.
[0002]
[Prior art]
Conventionally, a radio and a cassette tape player are installed in a vehicle as a means for improving a driving environment by a driver.
In addition, there is a vehicle in which a traveling environment is improved by enjoying a conversation with an acquaintance outside the vehicle using a wireless communication device such as an amateur radio or a mobile phone mounted on the vehicle.
[0003]
[Problems to be solved by the invention]
The conventional radio or the like in the vehicle as described above only provides one-way information to the driver, and cannot perform a two-way conversation or the like.
On the other hand, in the case of using a mobile phone or the like, it is possible to talk, but it has been necessary to search for the other party by waiting for a call or dialing. Even if the other party was found, it did not give an appropriate conversation tailored to the driver's one-sided circumstances such as vehicle conditions.
In this way, conventional vehicles do not have anthropomorphized agents according to the history of the vehicle, such as the past state of the vehicle, or the driver's state, so the vehicle is a tool that is simply a vehicle that is not attached to it. In some cases, it only had a role.
[0004]
Japanese Patent Laid-Open No. 9-102098 discloses a technique for transmitting information to the driver by human facial expressions and actions.
However, the technology described in this publication does not change the display based on past driver response history, gender, user information such as age, and is always the same when the same situation occurs. Is displayed. That is, the same display is always performed for a limited sensor output, and it should fall within the category of conventional instruments with improved visibility. In addition, in a vehicle, there is a wide variety of information that should be recognized by the driver, including inside and outside the vehicle, and it is necessary to grasp the information at a glance from the viewpoint of safety and the like. However, there is a limit to the amount of information in order to transmit various types of information with good visibility based on human facial expressions and actions.
[0005]
The present invention can transmit various kinds of information and communicate with the driver according to the action by the anthropomorphic agent and the background of the agent. Agent device To provide Eyes Target.
[0006]
[Means for Solving the Problems]
According to the first aspect of the present invention, an anthropomorphic agent, an image display device that displays the background of the agent, an action determination unit that determines an action of the agent displayed on the image display device, and the image display device Background determining means for determining the background displayed on the screen, an agent for performing the action determined by the action determining means, and an image display means for displaying the background determined by the background determining means on the image display device, Voice recognition means for recognizing a voice response of a user to an agent's action displayed on the image display device, a positive word group consisting of a plurality of words representing affirmation by the voice recognition means, and a plurality representing negation A recognition rate calculating means for calculating a speech recognition rate for each word of a negative word group consisting of The background determination means comprises When waiting for voice input of a word indicating affirmation or a word indicating denial as a response by voice from the user, voice input is made to the user among the words of the positive word group and the words of the negative word group that can be recognized by voice. Recommendation A placard displaying the word with the highest recognition rate in the positive word group and the word with the highest recognition rate in the negative word group The agent device is characterized in that the placard displaying the message is determined as the background.
Claim 2 In the described invention, With audio output device, Speech recognition by the speech recognition means is According to the audio output device Agent's Audio output Voice recognition state determination means for determining whether or not voice recognition is not possible due to determination of the voice recognition result or the voice recognition result, and the image display device includes a personified agent and a display screen of the agent A background including a frame displayed at the edge of the background is displayed, and the background determination means determines whether the background frame is in a case where it is determined that the speech recognition is not possible and a state where the speech recognition is possible. The color of the image is determined to be a different color. 1 The described agent device is provided.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a preferred embodiment of the agent device of the present invention will be described in detail with reference to FIGS.
(1) Outline of the embodiment
In the agent device of the present embodiment, the anthropomorphized agent and the background thereof are displayed on a display device in the vehicle by an image (a planar image, a stereoscopic image such as holography). Judgment of the situation of the vehicle including the vehicle itself, the driver, the passenger, the oncoming vehicle, etc. (including the response and reaction of the driver), and based on the vehicle situation at each time point, Respond with various variations (action = action and voice). Further, the background is determined and displayed based on the vehicle situation, similar to the agent action. As a result, the driver can interact (communicate) with his own agent in the vehicle, and the environment in the vehicle can be made comfortable. In addition, both the agent and the background can acquire a lot of information in a clearly distinguished manner, making the environment in the vehicle more comfortable.
Here, the anthropomorphic agent in the present embodiment has the same identity as a specific human being, a creature, a cartoon character, etc., and the creature with that identity maintains the identity and continuity. Outputs the trend (response by action and voice). In addition, identity and continuity are expressed as a personality with a unique personality, and can be regarded as a kind of pseudo-life form in electronic equipment. The agent of the present embodiment that appears in the vehicle is a subject that is pseudo-personalized (virtual personalized) that is determined in the same manner as a human.
Further, in the present embodiment, the agent determines the situation of the vehicle including the vehicle itself and the driver, performs various operations such as route guidance and device operation on behalf of the driver, and further assists the driver. The vehicle status, driver response, and the like are learned, and various actions are performed according to the judgment including the learning result. Therefore, even in the same vehicle situation, the content of communication differs depending on the past learning content. In some cases, there are misjudgments within a range that is not relevant to the vehicle, and an unnecessary (due) response may be caused by this misjudgment. Based on the driver's response, it is determined whether or not there is a determination error, and learning is performed.
Furthermore, in the present embodiment, the display of the background is determined based on the judgment including the learning result such as the vehicle state and the driver's response, and the effect is further enhanced by displaying the background according to the driver. Communication and good driving environment can be provided.
[0011]
(2) Details of the embodiment
FIG. 1 is a block diagram showing the configuration of the agent device in this embodiment.
In the present embodiment, an overall processing unit 1 that controls the entire communication function is provided. The overall processing unit 1 searches for a route to a set destination and provides guidance by voice or image display, a navigation processing unit 10, an agent processing unit 11, an I / F unit 12 for the navigation processing unit 10 and the agent processing unit 11. , An image processing unit 13 for processing an image output and an input image such as an agent image and a map image, an audio control unit 14 for controlling an audio output and an input voice such as an agent voice and a route guidance voice, and a vehicle and a driver It has a situation information processing unit 15 that processes detection data of various situations.
The agent processing unit 11 determines the action and background of the agent to appear in the vehicle from the vehicle status, learns the vehicle status and the past response by the driver, etc., and gives the driver appropriate conversation and control. It is designed to respond accordingly.
[0012]
The navigation processing unit 10 and the agent processing unit 11 are a CPU (central processing unit) that controls data processing and operation of each unit, and a ROM, RAM, and timer connected to the CPU via a bus line such as a data bus or a control bus. Etc. Both the processing units 10 and 11 are connected to a network, and can acquire processing data of each other.
The ROM is a read-only memory in which various data and programs for controlling by the CPU are stored in advance, and the RAM is a random access memory used by the CPU as a working memory.
[0013]
In the navigation processing unit 10 and the agent processing unit 11 of the present embodiment, the CPU reads various programs stored in the ROM and executes various processes. The CPU reads a computer program from an external recording medium set in the recording medium driving device 23, and stores (installs) the computer program in an agent storage device 29, a navigation data storage device, or another storage device such as a hard disk (not shown). Alternatively, a necessary program or the like may be read from the storage device into the RAM and executed. Further, a necessary program or the like may be directly read from the recording medium driving device 23 into the RAM and executed.
[0014]
The navigation processing unit 10 is connected to a current position detection device 21 and a navigation data storage device 30, the agent processing unit 11 is connected to an agent data storage device 29, and the I / F unit 12 is connected to an input device 22 and a storage medium. A drive device 23 and a communication control device 24 are connected, a display device 27 and an imaging device 28 are connected to the image processing unit 13, a sound processing device 25 and a microphone 26 are connected to the sound control unit 14, and a situation information processing unit A status sensor unit 40 is connected to 15.
[0015]
The current position detection device 21 is for detecting the absolute position (by latitude and longitude) of the vehicle, and includes a GPS (Global Positioning System) reception device 211 that measures the position of the vehicle using an artificial satellite, A sensor 212, a steering angle sensor 213, a distance sensor 214, a beacon receiving device 215 that receives position information from beacons arranged on the road, and the like are used.
The GPS receiver 211 and the beacon receiver 215 can measure the position independently, but in a place where the GPS receiver 211 and the beacon receiver 215 cannot receive, both the direction sensor 212 and the distance sensor 214 are used. The current position is detected by dead reckoning navigation.
The direction sensor 212 is, for example, a geomagnetic sensor that detects the azimuth of the vehicle by detecting geomagnetism, a gyroscope such as a gas rate gyroscope or an optical fiber gyroscope that detects the rotational angular velocity of the vehicle and integrates the angular velocity to obtain the azimuth of the vehicle, A wheel sensor is used that calculates the amount of displacement in the azimuth by detecting the turning of the vehicle based on the output pulse difference (movement distance difference).
The steering angle sensor 213 detects the steering angle α using an optical rotation sensor, a rotation resistance volume, or the like attached to the rotating portion of the steering.
For the distance sensor 214, for example, various methods are used such as detecting and counting the number of rotations of the wheel, or detecting acceleration and integrating twice.
[0016]
The input device 22 is means for inputting information about the user (such as age, sex, hobbies, personality, etc.) as a vehicle situation, or for a driver to respond to an inquiry from an agent. Note that the information related to the user is not limited to the case where the user inputs from the input device 22. For example, the agent makes various inquiries regarding whether or not he / she likes professional baseball, the name of a favorite team, etc. to the user. You may make it acquire.
In addition, the input device 22 is a predetermined travel environment (transmission condition) of a vehicle to which a current location (departure point), a destination (arrival point) at the start of travel in the navigation process, a traffic request for information such as traffic information is transmitted to the information provider ), For inputting the type (model) of the mobile phone used in the vehicle.
As the input device 22, various devices such as a touch panel (functioning as a switch), a keyboard, a mouse, a light pen, a joystick, an infrared remote controller, and a voice recognition device can be used. Moreover, you may provide the receiving part which receives the remote control using infrared rays etc., and the various signals transmitted from a remote control. In addition to the joystick for moving the cursor displayed on the screen, various keys such as menu designation keys (buttons) and numeric keys are arranged on the remote control.
[0017]
The recording medium driving device 23 is a driving device used to read a computer program for the navigation processing unit 10 and the agent processing unit 11 to perform various processes from an external recording medium. The computer program recorded on the recording medium includes various programs and data.
Here, the recording medium refers to a recording medium on which a computer program is recorded. Specifically, a magnetic recording medium such as a floppy disk, a hard disk, or a magnetic tape, a semiconductor recording medium such as a memory chip or an IC card, a CD- Recording media such as ROM, MO, PD (phase change rewritable optical disc), etc., paper such as paper cards, paper tapes, printed materials for reading programs using character recognition devices (and paper) Recording medium using a medium having a corresponding function) and other recording media on which a computer program is recorded by various methods.
[0018]
In addition to reading the computer program from these various recording media, the recording medium driving device 23, when the recording medium is a writable recording medium such as a floppy disk or an IC card, performs navigation processing unit 10 and agent processing. The RAM of the unit 11 and the data of the storage devices 29 and 30 can be written to the recording medium.
For example, the learning contents (learning item data, response data) regarding the agent function, information regarding the user, etc. are stored in the IC card, and the IC card storing these data is used even when driving other vehicles. Thus, it becomes possible to communicate with the learned agent according to his / her preference (according to the past situation of reception). This makes it possible for an agent specific to the driver to appear in the vehicle instead of the agent for each vehicle.
[0019]
The communication control device 24 is connected to a mobile phone including various wireless communication devices. The communication control unit 24 communicates with an information providing station that provides data related to traffic information such as road congestion and traffic regulations, as well as telephone line calls, and karaoke data used for communication karaoke in a car. It is possible to communicate with an information providing station that provides information.
In addition, learning data related to the agent function and information related to the user can be transmitted and received via the communication control device 24.
[0020]
The voice output device 25 is composed of a plurality of speakers arranged in the vehicle, and is controlled by the voice control unit 14, for example, guidance voice when performing route guidance by voice, voice according to the action of the agent, Sound is output. The audio output device 25 may also be used as an audio speaker. The voice control device 14 can control the timbre, accent, and the like of the voice output from the voice output device 25 in accordance with the driver's input of the tuning instruction.
The microphone 26 serves as voice input means for inputting / outputting voice to be recognized by the voice control unit 14, for example, input voice such as a destination in navigation processing, a driver's conversation with the agent (response, etc.), and the like. Function. The microphone 26 may be used also as a microphone for performing karaoke such as online karaoke, and a dedicated microphone having directivity is used in order to accurately collect the driver's voice. Also good.
A hands-free unit may be formed by the audio output device 25 and the microphone 26 so that a telephone call can be made without using a mobile phone.
[0021]
On the display device 27, a road map for route guidance and various image information by the processing of the navigation processing unit 10 are displayed, and various actions (moving images) of the agent by the agent processing unit 11 are displayed. . When an agent is displayed, the agent background is displayed at the same time. Further, images inside and outside the vehicle imaged by the imaging device 28 are also displayed after being processed by the image processing unit 13.
As the display device 27, various display devices such as a liquid crystal display device and a CRT are used.
The display device 27 may have a function as the input device 22 such as a touch panel.
[0022]
The imaging device 28 is composed of a camera equipped with a CCD (charge coupled device) for capturing an image, and images the front, rear, right side, and left side of the vehicle in addition to the in-vehicle camera that images the driver. Each outside camera is arranged. An image captured by each camera of the imaging device 28 is supplied to the image processing unit 13, and processing such as image recognition is performed, and each recognition result is also used for determining a program number by the agent processing unit 11. ing.
[0023]
The agent data storage device 29 is a storage device that stores various data (including programs) necessary for realizing the agent function according to the present embodiment. As the agent data storage device 29, for example, various recording media such as a floppy disk, a hard disk, a CD-ROM, an optical disk, a magnetic tape, an IC card, an optical card, and a driving device thereof are used.
In this case, for example, the learning item data 292 and the response data 293 are composed of an IC card or a floppy disk that is easy to carry, and the other data is composed of a hard disk. However, these driving devices may be used as the driving device.
[0024]
The agent data storage device 29 includes an agent program 290, a program selection table 291, learning item data 292, response data 293, image data 294 for displaying the appearance, behavior, and background of the agent illustrated in FIG. A table 296, response recognition data 298, and various data necessary for processing for other agents are stored.
[0025]
The agent program 290 displays an agent processing program for realizing the agent function, and detailed actions when the agent and the driver communicate with each other on the display device 27 together with a background, and a conversation corresponding to the action. Communication programs for output from the audio output device 25 are stored in the order of program numbers.
The agent program 290 stores review type audio data for the audio of each program number, and the driver can select the audio from the input device 22 or the like together with the selection of the appearance of the agent. It is like that. Agent voices include male voices, female voices, child voices, mechanical voices, animal voices, voices of specific voice actors and actors, voices of specific characters, etc. The driver chooses. Note that the agent's voice selection can be changed as needed.
[0026]
The program selection table 291 is a table for selecting a communication program stored in the agent program 290.
FIG. 2 shows the program selection table 291. FIG. 3 shows the contents of the actions (actions and utterances) of the agent corresponding to each program number selected in the program selection table 291.
The program numbers shown in FIGS. 2 and 3 coincide with the numbers of the communication programs stored in the agent program 290.
[0027]
FIG. 4 shows several screens about the “slow bow” action of the agent displayed on the display device 27 by the program numbers 00001 to 00002 in FIGS.
As shown in FIG. 4, the agent E is expressed as a bow bowed by bowing while tightening the mouth and placing the hand on the knee. The words (speech) spoken by Agent E along with this action can be changed depending on the vehicle situation, the learning situation, the personality of the agent, and the like.
[0028]
When the coolant temperature of the engine is low, the action “Let me sleep ...” is selected according to the condition of the engine. As a sleepy expression, use a facial expression with a drop of wrinkles, yawning and stretching, performing prescribed actions (bowing, etc.), rubbing eyes first, and making movements and vocalization slower than usual Can be expressed as These sleepy expressions are not always the same, but are appropriately changed by learning the number of actions.
For example, rubbing eyes once every three times (A action), yawning once every ten times (B action), and other than that, a facial expression with a heel down (C action). These changes are realized by combining the additional program of action B or action C with the basic program of action A. As to which action is to be combined, the number of program executions of the basic action A is counted as a learning item, and the additional program is combined according to the number of times.
In addition, when expressing the action “energeticly”, it is expressed by increasing the inflection of the voice or by making the agent E appear on the screen while running.
[0029]
Each item displayed in FIG. 2 represents a selection condition for selecting each program number, and is determined from various conditions of the vehicle and the driver detected by the state sensor 40 (time, start location) , Cooling water temperature, shift position position, accelerator opening, etc.) and items determined from learning contents stored in learning item data 292 and response data 293 (number of times of today's IG ON, elapsed time since last end, And the total number of startups).
In the program selection table 291, a program that satisfies all these items is always determined uniquely. In the table, “◯” indicates an item that must be satisfied in order to select the program number, and “−” and “No” indicate items that are not considered in selecting the program. .
[0030]
2 and 3 describe actions and selection conditions related to communication (greetings) when the ignition is turned on, but a program for selecting a program that prescribes various other actions (actions and utterances) Various numbers and selection conditions are also defined.
For example, on the condition that a sudden brake is stepped on, a program is also defined in which an agent acts “sitoshimochi” or “tatara” or makes a surprise voice. The selection of each action by the agent is changed by learning for sudden braking. For example, “Sirimochi” is applied from the first sudden braking to the third, and “Tatara” is performed from the fourth to the tenth. From the second time onward, take the action of “holding one foot one step forward” and let the agent get used to sudden braking step by step. Then, when there is an interval of one week from the last sudden braking, the vehicle is moved backward by one step.
[0031]
The learning item data 292 and the response data 293 are data obtained as a result of the agent learning by the driver's driving operation and response. Therefore, the learning item data 292 and the response data 293 are stored / updated (learned) for each driver.
[0032]
Both the learning item data 292 and the response data 293 are data that is stored and updated by learning of the agent, and the contents thereof are conceptually shown in FIGS. 5 and 6, respectively.
As shown in FIG. 5, the learning item data 292 includes the total number of activations, the previous end date and time, and the number of times the ignition is turned on today, which are selection condition items for selecting a professional communication program in the program selection table 291 (FIG. 2). The remaining amount at the time of refueling five times, the audio operating conditions and the operating equipment at that time, etc. are stored. Further, the number of times of rest / date and time, a default value, and other data for determining whether to start the program selected according to the selection condition (whether to take a rest) are stored.
[0033]
The total number of activations stores the total number of times the ignition has been activated, and is counted up each time the ignition is turned on.
The previous end date and time is stored every time the ignition is turned off.
The number of times the ignition is turned on today stores the number of times the ignition is turned on that day and the end time of the day. The count is incremented every time the ignition is turned on, but the data is initialized to “0” when the day ends. The end time of the day is stored as 24:00 as a default value. This time can be changed according to the life pattern of the user (driver). When the time is changed, the changed time is stored.
[0034]
In the previous five refueling remaining amounts, the remaining amount of fuel detected immediately before refueling (gasoline) is stored, and each time data is newly refueled, each data is shifted to the left (the oldest leftmost (The data is deleted.) The remaining amount immediately before refueling is stored on the rightmost side.
This data indicates that when a detection value G1 of a fuel detection sensor 415, which will be described later, becomes equal to or less than the average value G2 of the remaining amount of fuel for all five times (G1 ≦ G2), the agent E appears on the display device 27 and supplies fuel. The prompting action is displayed on the display device 27, and a sound such as “I am hungry! I want gasoline!” Is output from the sound output device 25.
The audio operating conditions are the time zone and place when the audio switches such as radio, CD, MD, cassette tape player, and TV are turned on. In the case of radio and TV, the selected station is further selected. Applicable. The operating equipment is audio such as radio, CD, MD, cassette tape player and the like. The audio operation condition and the operation device are stored for the past five times when the audio is switched on.
[0035]
The number of times / date of rest stores, for each program number, the number of times of rest without executing even if the corresponding communication program is selected. The number of times / date of rest is stored for an agent action in which the rest item is set as a learning item, such as an agent act (program number 00123) that proposes to stop the air conditioner described later.
When the driver's response to the agent's proposal or conversation is rejection (rejection) or disregard (or no response), “rest” is selectively set according to the communication program.
[0036]
In the default value, initial setting values for each item such as time, number of times, temperature, vehicle speed, date and time are stored, and the value changed in the learning item, such as the end time of the day described above, is the initial value. Used to return to
[0037]
Other data stored in the learning item data 292 includes, for example, the birthday of the driver and related persons (this is a user input item), the national holiday, the event, such as Christmas, Valentine's Day, White Day, etc. Stores the date. There is also a special menu communication program for each event day. For example, an agent disguised as Santa Claus appears on Christmas Eve.
[0038]
In the response data 293 of FIG. 6, a history of user responses to agent actions is stored for each communication program number whose user response is a learning item. As for the user response data, the latest response date and time and the content of the response are stored for a predetermined number of times (program number 00123 is two times) as in communication program numbers 00123 and 00125 in FIG. Only the latest response content is stored once (thus updating every time there is a response), only the latest response content is stored a predetermined time, and the latest date and time and response content are stored once. In other cases, only the latest date and time are stored once or a predetermined number of times.
Symbols A, B, and C displayed in FIG. 6 (A) represent response contents. As shown in FIG. 6 (B), when symbol A is ignored, when symbol B is rejected, This represents the case where the symbol C is received. The response content of the driver is determined from the result of voice recognition with respect to the driver's voice input from the microphone 26 and the input result from the input device 22.
In this embodiment, the driver's response is ignored, categorized into three patterns of rejection and acceptance, but “strongly rejected”, “angry”, and “happy” were newly added. Also good. In this case, the learning item data 292 (for example, the number of days off) and the response data 293 are additionally changed according to the newly added response.
[0039]
The image data 294 of the agent data storage device 29 shown in FIG. 1 stores images in which the appearances of a plurality of types of agents and the respective backgrounds are combined with respect to the behavior of the program number of the communication program of the agent program 290. ing. The appearance of the agent can be selected from the input device 22 or the like according to the driver's preference, and the background selection table based on the situation of the selected agent obtained by various sensors or the like. An image is displayed together with the background determined by H.296. The selection of the appearance of the agent can be changed in a timely manner as in the case of voice.
The appearance of the agent stored in the image data 294 does not have to be a human (male, female) appearance. For example, the appearance of an animal itself such as a chick, a dog, a cat, a frog, a mouse, or a human design It may be an animal appearance (illustrated), a robot appearance, a specific character appearance, or the like. In addition, the age of the agent does not need to be constant. The learning function of the agent is that it is initially a child's appearance and grows over time, changing its appearance (changing to the appearance of an adult, It may change to the appearance of
[0040]
Examples of background images stored in the image data 294 include landscapes representing time zones such as sunrise and starry sky, landscapes representing seasons such as the sea, snowy mountains, and autumn leaves, landscapes representing destinations such as golf courses and the sea, "Yes" and "No" placards when agents are waiting for a "Yes" or "No" response, patterns of notes when listening to music on radio or CD, Examples include a frame of each color that represents a response recognition state for each color indicating whether or not a voice response can be recognized.
[0041]
FIG. 7 shows a background selection table 296 for selecting an agent background displayed on the display device 27. As shown on the left side of the table, the background includes images of landscapes and patterns displayed on the back of the agent, images of belongings possessed by the agent, and frames displayed along the inner frame of the display screen of the display device 27. There is an image. As shown in FIG. 7, these backgrounds are determined based on various items such as time zone, season, running state, operating equipment, agent state, and destination set in car navigation. ing.
Each of these items is determined in association with items determined from various situations such as the vehicle running state detected by the state sensor 40 and learning contents stored in the learning item data 292 and the response data 293. There are items (“Yes” and “No” placards, “YES” and “NO” placards, background selection when operating audio, etc.).
Then, one or more backgrounds are selected based on the selection condition. When a plurality of backgrounds are selected, the background, belongings, and frame are not selected redundantly. In the table, “◯” indicates an item that must be satisfied in order to select the background, and “No mark” indicates an item that must not be satisfied in order to select the background.
[0042]
FIGS. 8A, 8B, 8C, and 8D show an example in which the background selected by the above-described background selection table 296 is displayed on the display device together with the agent in the present embodiment.
FIG. 8A shows a background in which the driver's voice cannot be recognized because the agent is speaking or judging the voice recognition result, and a red frame R indicating that voice recognition is impossible is displayed as the background. ing. In this display (a), the agent is seated and displayed. This represents that the vehicle is stopped by the posture of the agent.
FIG. 8B shows a state of waiting for an affirmative or negative word as a response from the driver, and a yellow frame Y indicating that only an affirmative and negative word can be recognized as a background is displayed. Has been. In addition, placards of “yes” and “no” are displayed on the agent's belongings (background) as recommended words of responses that can be recognized by voice. In this display (b) to (d), the agent is displayed standing, and this represents that the vehicle is traveling by the posture of the agent.
FIG. 8C is a screen in a state of waiting for voice input of the destination prefecture in the navigation system, and a green frame G representing a voice recognition enabled state is displayed as a background. In addition, a placard of “prefecture” is displayed on the agent's belongings (background) as a recommended word of responses that can be recognized by voice.
In FIG. 8D, a green frame G representing a state in which a voice of a normal communication such as “pick up CD” or “open a window” can be recognized without particular limitation.
[0043]
Similar to the learning item data 292 and response data 293 described above, the response recognition data 298 is data obtained by the agent as a result of a response from the driver, and the data is stored / updated (learned) for each driver. Is done.
[0044]
FIG. 9 shows the response recognition data.
As shown in FIG. 9, the response recognition data 298 includes a response recognition result, a response recognition rate obtained from the response recognition result, and a maximum response in each of a positive word group and a negative word group. A recognition rate word is stored for each driver.
Response recognition results include positive and negative words such as “Yes” and “No”, “YES” and “NO”, “Yes” and “NO”, “Ye” and “Yadayo” as background. This is data indicating whether or not the word response from the driver was correctly recognized when displayed on the card. Whether or not the response was correctly recognized is determined based on the driver's reaction when the agent performs control etc. based on the result of recognizing the acquired response. , “No”, “YES”, “NO”,... Are stored 10 times each.
[0045]
As the response recognition rate, the response recognition rate obtained from each of the above-mentioned response recognition results is obtained by the following Equation 1 and stored.
[0046]
[Expression 1]
Response recognition rate = (number of times a response has been correctly recognized / number of responses acquired) × 100
[0047]
The highest recognition rate word stores a word having the highest recognition rate in each of a group of positive words such as “Yes” and “YES” and a group of negative words such as “No” and “NO”. When displaying a background having two placards, affirmative and negative, these highest recognition rate words are selected in the background selection table.
[0048]
FIG. 10 shows the contents of a data file stored in the navigation data storage device 30 (FIG. 1).
As shown in FIG. 10, the navigation data storage device 30 includes various data files used for route guidance, such as a communication area data file 301, a drawing map data file 302, an intersection data file 303, a node data file 304, a road A data file 305, a search data file 306, and a photo data file 307 are stored.
As the navigation data storage device 4, for example, various recording media such as a floppy disk, a hard disk, a CD-ROM, an optical disk, a magnetic tape, an IC card, an optical card, and a driving device thereof are used.
The navigation data storage device 4 may be composed of a plurality of different types of recording media and driving devices. For example, the search data file 46 is a readable / writable recording medium (for example, a flash memory or the like), and other files are formed of a CD-ROM, and these drive devices are used as drive devices.
[0049]
In the communication area data file 301, a mobile phone that is connected to the communication control device 24 or used in the vehicle without connection is displayed on the display device 5, or a route search is performed for the communication area. The communication area data for use at the time is stored for each type of mobile phone. Each communication area data for each type of mobile phone is numbered and managed so that it can be easily searched, and the communicable area can be expressed by the inside surrounded by a closed curve. And is specified by the position data of the bending point. Note that the communication area data may be generated by dividing the communicable area into large and small square areas and using the coordinate data of two points in a diagonal relationship.
The content stored in the communication area data file 301 is preferably updateable as the area in which the mobile phone can be used is expanded or reduced. For this purpose, by using the mobile phone and the communication control device 24, The communication area data file 301 can be updated with the latest data by communicating with the information providing station. The communication area data file 301 may be composed of a floppy disk, an IC card, etc., and rewritten with the latest data.
The drawing map data file 302 stores drawing map data to be drawn on the display device 27. The drawing map data stores a hierarchical map, for example, map data for each hierarchy such as Japan, Kanto region, Tokyo, Kanda from the highest layer. Map data is attached to each level of map data.
[0050]
The intersection data file 303 includes intersection numbers, intersection names, intersection coordinates (latitude and longitude), road numbers starting and ending at the intersection, and presence / absence of signals as intersection data. Stored.
The node data file 304 stores node data consisting of information such as latitude and longitude that specifies the coordinates of each point on each road. In other words, this node data is data relating to a single point on the road. When a node connecting nodes is called an arc, the road is expressed by connecting each of a plurality of node rows with an arc. .
The road data file 305 includes a road number that identifies each road, an intersection number that is a start point and an end point, a road number that has the same start point and end point, road thickness, prohibition information such as entry prohibition, Photo number etc. are stored.
Road network data composed of intersection data, node data, and road data stored in the intersection data file 303, node data file 304, and road data file 305, respectively, is used for route search.
[0051]
The search data file 306 stores intersection sequence data, node sequence data, and the like constituting the route generated by the route search. The intersection string data includes information such as an intersection name, an intersection number, a photo number showing a characteristic landscape of the intersection, a turning angle, and a distance. The node string data includes information such as east longitude and north latitude indicating the position of the node.
In the photo data file 307, photographs taken of characteristic scenery or the like that can be seen at each intersection or straight ahead are stored in a digital, analog, or negative film format corresponding to the photograph number.
[0052]
FIG. 11 shows various sensors constituting the situation sensor unit 40.
As shown in FIG. 11, the situation sensor unit 40 includes an ignition sensor 401, a vehicle speed sensor 402, an accelerator sensor 403, a brake sensor 404, a side brake detection sensor 405, a shift position detection sensor 406, a winker detection sensor 407, a wiper detection sensor 408, Light detection sensor 409, seat belt detection sensor 410, door opening / closing detection sensor 411, passenger detection sensor 412, indoor temperature detection sensor 413, outdoor temperature detection sensor 414, fuel detection sensor 415, water temperature detection sensor 416, ABS detection sensor 417, Air conditioner sensor 418, weight sensor 419, front inter-vehicle distance sensor 420, rear inter-vehicle distance sensor 421, body temperature sensor 422, heart rate sensor 423, sweat sensor 424, electroencephalogram sensor 425, eye tracer 426, infrared sensor 42 , Vehicles such as 428 such as tire pressure drop detection sensor, belt looseness detection sensor, window open / close state sensor, horn sensor, indoor humidity sensor, outdoor humidity sensor, oil temperature detection sensor, hydraulic pressure detection sensor, etc. Various sensors are provided for detecting the situation, driver situation, in-vehicle situation, and the like.
These various sensors are arranged at predetermined positions according to the respective sensing purposes.
In addition, when each of these sensors does not exist as an independent sensor, the case where it senses indirectly from another sensor detection signal is included. For example, a tire air pressure decrease detection sensor indirectly detects a decrease in air pressure by a change in a signal from a wheel speed sensor.
[0053]
The ignition sensor 401 detects ON and OFF of the ignition.
As the vehicle speed sensor 402, a conventionally known vehicle speed sensor such as one that calculates the vehicle speed by detecting the rotational angular speed or the number of rotations of the speedometer cable can be used without particular limitation.
The accelerator sensor 403 detects the amount of depression of the accelerator pedal.
The brake sensor 404 detects the amount of depression of the brake, and detects whether or not a sudden brake is applied based on the depression force, the depression speed, and the like.
The side brake detection sensor 405 detects whether or not the side brake is applied.
The shift position detection sensor 406 detects the shift lever position.
The blinker detection sensor 407 detects the blinking direction of the blinker.
The wiper detection sensor 408 detects the driving state (speed, etc.) of the wiper.
The light detection sensor 409 detects the lighting state of each lamp such as a head lamp, tail lamp, fog lamp, and room lamp.
The seat belt detection sensor 410 detects whether the driver and the passenger (auxiliary seat, rear seat) are wearing the seat belt. If it is not worn, the agent appears as appropriate (to the extent that it is not disliked), and performs warnings, cautions, comments, etc. (the degree is changed by learning).
[0054]
The door open / close detection sensor 411 detects the open / closed state of the door, and in the case of a so-called half-door, the agent notifies the fact. The door opening / closing detection sensor 411 can detect opening / closing of each door according to the vehicle type, such as a driver's seat door, a passenger seat door, a rear driver seat side door, and a rear passenger seat side door.
The passenger detection sensor 412 is a sensor that detects whether or not a passenger is on the passenger seat or the rear seat. The passenger detection sensor 412 is detected from an in-vehicle image captured by the imaging device 28, or is disposed in an auxiliary seat or the like. Detect with pressure sensor or scale.
The indoor temperature detection sensor 413 detects the indoor air temperature, and the outdoor temperature detection sensor 414 detects the air temperature outside the vehicle.
The fuel detection sensor 415 detects the remaining amount of fuel such as gasoline and light oil. The detected values for the past five times immediately before refueling are stored in the learning item data 292, and when the average value is reached, the agent informs that it is the refueling time.
[0055]
The water temperature detection sensor 416 detects the temperature of the cooling water. Immediately after the ignition is turned on, if the detected temperature is low, the agent often acts to sleep. On the other hand, if the water temperature is too high, before overheating, the agent informs that effect along with the “dull” behavior.
The ABS detection sensor 417 detects whether or not the ABS is activated to prevent the tire from being locked due to a sudden brake and to ensure the maneuverability and vehicle stability.
The air conditioner sensor 418 detects the operation state of the air conditioner. For example, ON / OFF of the air conditioner, set temperature, air volume, etc. are detected.
The weight sensor 419 is a sensor that detects the weight of the driver. A driver is identified from this weight or from the weight and the image of the imaging device 28, and an agent learned in relation to the driver is caused to appear. That is, by using the learning item data 292 and the response data 293 learned by the agent for the specified driver, an agent dedicated to the driver appears.
The front inter-vehicle distance sensor 420 detects the distance to other vehicles and obstacles in front of the vehicle, and the rear inter-vehicle distance sensor 421 detects the distance to other vehicles and obstacles behind.
[0056]
The body temperature sensor 422 is a heart rate sensor 423, and the sweat sensor 424 is a sensor that detects the body temperature, heart rate, and sweat state of the driver. For example, each sensor is arranged on the handle surface and detected from the state of the driver's hand. To do. Alternatively, as the body temperature sensor 422, the temperature distribution of each part of the driver may be detected by thermography using an infrared detection element.
The electroencephalogram sensor 425 is a sensor that detects a driver's brain wave, and detects, for example, an α wave, a β wave, or the like to check the driver's arousal state.
The eye tracer 426 detects the movement of the user's line of sight and determines whether the user is looking for an object outside the vehicle, searching for the object inside the vehicle, or a wakeful state during normal driving.
The infrared sensor 427 detects the movement of the user's hand and the movement of the face.
[0057]
Next, the operation of the present embodiment configured as described above will be described.
FIG. 12 is a flowchart showing the main operation of processing by the agent of this embodiment.
When the ignition sensor 401 detects that the ignition is turned on, the agent processing unit 11 first performs initial setting (step 11). As initial settings, processing such as clearing the RAM, setting the work area for each process in the RAM, loading the program selection table 291 (FIG. 2) into the RAM, and setting the flag to 0 is performed. In the agent processing of this embodiment, the start of the processing is set to ignition ON. However, for example, the processing may be started when any door opening / closing is detected by the door opening / closing detection sensor 411.
[0058]
Next, the agent processing unit 11 identifies the driver (Step 12). In other words, the agent processing unit 11 analyzes the voice when the driver gives a greeting first, identifies the driver, identifies the driver by analyzing the captured image, or the weight sensor 419. The driver is identified from the weight detected in step 1, or the driver is identified from the set seat position and the angle of the rearview mirror. For the identified driver, a special communication program for inquiring “Is Mr. XX?” Is activated separately from the agent processing described later, and the driver is confirmed.
[0059]
When the driver is specified, the agent processing unit 11 next grasps the current situation (step 13).
That is, the agent processing unit 11 detects the detection value supplied from each sensor of the situation sensor unit 40 to the situation information processing unit 15, the processing result of the image captured by the imaging device 28, and the vehicle detected by the current position detection device 21. Is acquired and stored in a predetermined area of the RAM, and the current state such as the state of the vehicle is grasped from the stored data. For example, when the temperature of the cooling water detected by the water temperature detection sensor 416 is t1, the agent processing unit 11 stores the temperature t1 in the RAM, and if the t1 is equal to or less than a predetermined threshold value t2, It is grasped that the cooling water temperature (see FIG. 2) is in a low state.
As the current situation, the driver's request that is recognized by voice based on the input from the microphone 26, for example, “Please call XXX” or “Show a restaurant in this area. ”Or“ Please play the CD. ”Is also recognized as the current situation. In this case, the words “CD” and “Kake” included in the recognized voice are the selection conditions (horizontal axis items) of the program selection table 291 (FIG. 2).
Further, the agent processing unit 11 checks the learning item data 292 and the response data 293 in the agent data storage device 29 as a grasp of the current situation, thereby grasping the state (learning data) that the agent has learned so far.
[0060]
When the agent processing unit 11 grasps the current situation, as will be described in detail later with reference to FIG. 11, the agent processing unit 11 performs processing of the agent according to the grasped situation (step 14).
The processing of the agent here includes various processing such as judgment, action (behavior + utterance), control, learning, inspection, etc. by the agent, but also includes a case where no operation is performed depending on the grasped current situation.
[0061]
Next, the agent processing unit 11 determines whether or not to end the process of the main operation (step 15). If not ended (step 15; N), the process returns to step 13 and repeats the process.
In the case of ending one, that is, after the ignition sensor 401 detects that the ignition is turned off (step 13) and completes the termination process (step 14) such as turning off the indoor lamp (step 15; Y), the main The processing operation is terminated.
[0062]
FIG. 13 is a flowchart showing the processing operation of the agent according to the grasped situation.
Based on the program selection table 291 shown in FIG. 2, the agent processing unit 11 can start a communication program (number of communication programs) that can be started in the current state based on the grasped current situation (number of activations, current weather, time, etc.). ) (Step 21). If there is no corresponding program (step 21; N), the process returns to the main routine.
On the other hand, if there is a communication program that can be activated (step 21; Y), the program number is determined. Then, the driver's response history with respect to the determined program number is confirmed from the response data 293, and it is confirmed whether or not the activation of the communication program of the program number is in a state to be rested (step 22).
[0063]
When not in a rest state (step 22; N), the agent processing unit 11 confirms whether the communication program to be started is a control program for controlling the vehicle and various devices mounted on the vehicle (step 23). In the case of a control program (step 23; Y), this communication program is started as it is, and control according to the program is performed (step 29).
[0064]
If the communication program is not a control program (step 23; N), the current situation (current time, season, operating device, voice recognition status of the agent, destination in the car navigation system, etc.) that has been grasped and the agent From the response recognition data 298 of the data storage device 29, the background of the agent is determined according to the background selection table 296 (step 24).
Subsequently, by starting the communication program for the appearance of the selected agent and the determined background image, the agent action (behavior and voice) as shown in FIGS. The conforming image is displayed on the display device 27, and the sound is output from the sound output device 25 (step 25).
[0065]
When this communication program is not a response acquisition program for acquiring a response from the driver (step 26; N), the process returns to the main routine. When this communication program is a response acquisition program (step 26; Y), the agent processing unit 11 uses the voice recognition result based on the input from the microphone 26 to the driver's response to the agent action by the activation of the communication program. Or it acquires from the input result from the input device 22 (step 27). Then, the agent processing unit 11 accumulates data related to the current communication program to cause the agent to learn (step 28), and returns to the main routine.
As the accumulation of data, for example, when the communication program is not activated (step 22; Y), the number column of the corresponding program number in the learning item data 292 is counted up. However, when the number of times stored in the number of times / date of learning item data 292 is Ka times and the number of times of rest determined from the history of the response data 293 for the program number is Kb times, Ka = If it is Kb-1, this is the prescribed number of days off. Therefore, the data (stored in the corresponding position) in the program number column of the learning item data 292 and the response data 293 is cleared.
[0066]
In other cases (after step 27 and after step 29), if there is a learning item in the grasped current situation (step 13), the value of the learning item data 292 is updated and the response content is stored as a history. If the program number is to be stored, the response content acquired in step 17 is stored in response data 293 (FIG. 6). In the case of the response history, when the predetermined number of data defined for each program number has already been stored, the oldest data is discarded and new data is stored. If the response acquired in step 25 is related to the answerback for the positive or negative word acquired before that, the correctness of the recognition result for the positive or negative word is acquired from this response. Data on the recognition result is stored in the response recognition data 298, and the response recognition rate and, if necessary, the maximum recognition rate word are rewritten.
[0067]
Next, a case where the radio is operated will be described as a specific action by the agent processing described above.
FIG. 14 conceptually shows the contents of specific agent processing after the ignition is turned on.
As shown in FIG. 14A, the agent processing unit 11 determines that the current position obtained from the current position (latitude, longitude) detected by the current position detection device 21 at the current time is 17:00 as the current situation. It is assumed that the status of “Tokyo” or the like has been grasped in step 13. The learning data checked for the learning item data 292 and the response data 293 is that the radio is operated in Tokyo from 17:00 to 18:00 in the past five audio operating conditions as the audio operating conditions. In the case of listening to the radio, it is assumed that the J-wave has been selected most frequently.
[0068]
Based on the above grasping state, the agent processing unit 11 selects a corresponding communication program from the program selection table 291. In other words, it is assumed that the communication program (program for making a proposal for applying radio) with program number 00501 is selected (step 21; Y). Then, it is confirmed that this communication program is not a subject of rest (step 22; N).
Since this communication program is not a control program (step 23; N), the agent processing unit 11 subsequently determines a background from the background selection table 296 (step 24). In this case, since the time zone is from 15 o'clock to 18 o'clock, the communication program to be activated is accompanied by speech and the voice recognition state is impossible, etc., the background is sunset and the frame is red. It is determined.
Then, the communication program of that number is started (step 25), and the selected agent and the determined background (sunset back BK and red frame R) are displayed on the display device 27 as shown in FIG. Display an image and make an inquiry by saying "Is it on radio?" Since this communication program is not a response acquisition program (step 26; N), the process directly returns to the main routine.
[0069]
Subsequently, this time, as a current situation, it is newly grasped that an inquiry has been made by the program number 00501, and in the agent process, a communication program that acquires a positive or negative response of the program number 00xxx. (Step 21; Y). Since it is not a day off (step 22; N) and is not a control program (step 23; N), the agent processing unit 11 determines the background from the background recognition table 296 and the highest recognition word of the response recognition data 298 ( Step 24).
In this case, the time zone is from 15:00 to 18:00, the communication program to be activated can recognize only positive and negative words, the highest recognition words are "Yes" and "Yadayo", etc. Therefore, the background whose sunset is the sunset, whose placards are “Yes” and “Yadayo” and whose frame is yellow is determined.
[0070]
Then, the agent processing unit 11 activates this communication program and, as shown in FIG. 14C, displays the background determined as the selected agent (sunset back BK, placard PC, and yellow frame Y). An image is displayed on the display device 27 (step 25), and a response is acquired as a response acquisition program (step 26; Y) (step 27). Here, it is assumed that a response “Yes” has been acquired.
Then, as learning of the agent, the response data of program number 00501 is updated (step 28), and the process returns to the main routine.
[0071]
This time, as a current situation, it is newly grasped that a positive response for radio ON has been acquired by the program number 00xxx, and in the agent processing, the communication program for applying the radio of program number 00 △△△ is It is judged that it exists (step 21; Y). And since it is not a rest (step 22; N) and is a control program (step 23; Y), a program is started and control which turns on a radio is performed. At this time, the selected station most frequently selected in Tokyo, which is the current location, is acquired from the learning item 292 data, and is automatically tuned to this station.
Then, the audio operation condition and the operation audio device of the learning item data 292 are rewritten. In addition, since the driver does not respond to stopping the radio such as “No” or “Stop”, it is understood that the response recognition of “Yes” was correct, and “Yes” in the response recognition data 298 The response recognition rate of “Yes” is acquired and rewritten based on the recognized result, the rewritten data, and the maximum recognition rate word is rewritten as necessary (step 28), and the process returns to the main routine.
[0072]
This time, it is newly determined that the radio is operating as the current status, and it is determined that there is a communication program for displaying the vehicle status of program number 00 ▽ ▽ ▽ as the agent processing (step 21; Y). And since it is not a rest (step 22; N) and it is not a control program, a background is selected based on a background selection table.
In this case, because the time zone is from 15:00 to 18:00, the radio is operating, and any utterances can be recognized, the back is sunset, the belongings are radio cassettes, The background in which is green is determined.
[0073]
Subsequently, the agent processing unit 11 activates this communication program, and displays an image of the selected agent and the background determined (sunset back BK, belongings radio cassette and green frame) on the display device 27 (step 25). . Since this program is not a response acquisition program (step 26; N), the process directly returns to the main routine.
[0074]
As described above, according to the present embodiment, in addition to an anthropomorphic agent display and sound, the agent background is displayed, so that a large amount of information can be included without reducing the visibility of the information. It is possible to display a rich screen, transmit a lot of information to the driver, and establish a rich communication according to the driver and the situation.
According to the present embodiment, by separating information transmitted by the agent from information transmitted by the background, information can be transmitted with better visibility.
According to the present embodiment, the agent's back and the agent's belongings are displayed as the background, and the types of information to be transmitted are classified, so that information can be transmitted with better visibility. it can.
[0075]
According to this embodiment, an agent that is anthropomorphic from the learning result based on the past history etc. as well as the current situation of the vehicle / driver, acts to suit the situation, and communicates with the driver And can make the driving environment in the car comfortable.
According to the present embodiment, the background is determined from the learning result based on the past history and the like as well as the current vehicle / driver situation, and information is provided, so that the driving environment in the vehicle is made comfortable. Can do.
[0076]
According to this embodiment, a positive word and a negative word with a high recognition rate are acquired by learning, and a word with a high recognition rate is presented by a placard as a background, so that the driver makes a response according to this Thus, the agent can accurately respond, and efficient and good communication between the driver and the agent becomes possible.
According to the present embodiment, since the voice recognition state of the agent is represented by a frame as a background, the agent can recognize the response from the driver when the driver responds at a timing according to this, Efficient and good communication between the driver and the agent is possible.
[0077]
The agent device of the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.
For example, in each of the embodiments described above, combined image data of each of the selectable agent appearances and each background is stored as the image data 294, but the agent appearance image data and the background image are stored. The data may be stored separately, and the selected appearance and the determined background data may be combined and displayed on the display device.
The background is not limited to the above, and for example, a musical note pattern or a picture of an operating device may be displayed as an agent's back when operating the audio, in addition to the landscape.
For example, when a tree is displayed as a background and the tree flutters to the left according to the vehicle speed, or a road is displayed under the agent's feet sideways, and the vehicle is located on an uphill, the agent May be displayed as if the agent is climbing uphill, and when the vehicle is located on the downhill, the front side of the agent may be lowered and displayed as if the agent is going downhill. . Such road conditions and the like are easily overlooked by the instrument and are important from the viewpoint of safety, and displaying with good visibility as a background together with the agent in the vehicle is a great advantage.
[0078]
In this embodiment, whether or not the response recognition result is correct is determined from the driver's reaction to the control based on the response recognition result. However, the present invention is not limited to this, and the voice response to the recognition result is determined. Judgment is made by answering from the driver (input operation) by backing up, or by answering from the driver for answerbacks based on images such as raising the same placard as the recognized group. May be.
Provided with background preference learning means to acquire the driver's preference for the displayed background from the agent's inquiry response or situation sensor, prepared multiple backgrounds that can be selected under the same other conditions, to the driver Accordingly, a background with a high degree of preference may be selectively displayed.
[0079]
【The invention's effect】
Of the present invention According to the agent device, since a lot of information is transmitted to the driver with high visibility by the anthropomorphized agent and the background of the agent, it is possible to establish a rich communication according to the situation of the vehicle.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration for realizing a communication function in an embodiment of the present invention.
FIG. 2 is an explanatory diagram conceptually showing the contents of a program selection table in the embodiment.
FIG. 3 is an explanatory diagram showing an agent's action (action and voice) corresponding to each program number in the embodiment;
FIG. 4 is an explanatory diagram showing a number of screens about the “slow bow” action of the agent displayed on the display device by starting program numbers 00001 to 00002 in the embodiment.
FIG. 5 is an explanatory diagram conceptually showing the contents of learning item data in the embodiment.
FIG. 6 is an explanatory diagram conceptually showing the contents of response data in the embodiment.
FIG. 7 is an explanatory diagram conceptually showing the contents of a background selection table in the embodiment.
FIG. 8 is a diagram showing an example of an agent and a background displayed on the display device in the embodiment, where (a) is a display in a voice recognition disabled state, and (b) is only an affirmative word and a negative word. A display in a state where speech can be recognized, (c) shows a display requesting input of a prefecture, and (d) shows a display in a state where any speech can be recognized.
FIG. 9 is an explanatory diagram conceptually showing the contents of response recognition data in the embodiment.
FIG. 10 is an explanatory diagram conceptually showing the contents of a data file stored in the navigation data storage device in the embodiment.
FIG. 11 is an explanatory diagram showing various sensors constituting the situation sensor unit in the embodiment.
FIG. 12 is a flowchart showing a main operation by an agent in the embodiment;
FIG. 13 is a flowchart showing the operation of an agent process according to the embodiment.
FIG. 14 is an explanatory diagram conceptually showing the contents of specific agent processing after the ignition is turned on in the embodiment.
[Explanation of symbols]
1 Overall processing section
10 Navigation processing part
11 Agent processing part
12 I / F section
13 Image processing unit
14 Voice control unit
15 Situation information processing department
21 Current position detection device
22 Input device
23 Storage medium drive
24 Communication control device
25 Audio output device
26 microphone
27 Display device
28 Imaging device
29 Agent Data Storage Device
30 Navigation data storage device
40 Situation sensor section

Claims (2)

  1. An image display device for displaying an anthropomorphized agent and a background of the agent;
    Action determining means for determining an action of an agent displayed on the image display device;
    Background determining means for determining the background displayed on the image display device;
    Image display means for causing the image display device to display an agent that performs the action determined by the action determination means and the background determined by the background determination means;
    Voice recognition means for recognizing a voice response of the user to the agent's action displayed on the image display device;
    A recognition rate calculating means for calculating a speech recognition rate for each word of a negative word group consisting of a plurality of words representing affirmation and a negative word group consisting of a plurality of words representing a negation by the voice recognition means ;
    When the background determination unit waits for speech input of a word indicating affirmation or a word indicating denial as a voice response from the user , the background determination unit includes the words of the positive word group and the words of the negative word group that can be recognized by voice. A placard displaying the word with the highest recognition rate in the positive word group and a placard displaying the word with the highest recognition rate in the negative word group as words to recommend voice input to the user Decide as background,
    An agent device characterized by that.
  2. With audio output device,
    Voice recognition state determination means for determining whether or not the voice recognition by the voice recognition means is in a state where voice recognition cannot be performed due to the voice output of the agent by the voice output device or the determination of the voice recognition result;
    The image display device displays an anthropomorphic agent and a background including a frame displayed at the edge of the display screen of the agent,
    The background determination means determines the color of the background frame to be different between the case where it is determined that the voice recognition is impossible and the case where the voice recognition is possible.
    The agent device according to claim 1 .
JP09538698A 1998-03-23 1998-03-23 Agent device Expired - Fee Related JP4032492B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP09538698A JP4032492B2 (en) 1998-03-23 1998-03-23 Agent device

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
JP09538698A JP4032492B2 (en) 1998-03-23 1998-03-23 Agent device
US09/119,238 US6249720B1 (en) 1997-07-22 1998-07-21 Device mounted in vehicle
EP19980113679 EP0893308B1 (en) 1997-07-22 1998-07-22 Device mounted in vehicle
DE1998636846 DE69836846T2 (en) 1997-07-22 1998-07-22 In the motor vehicle built-in device
JP2005098227A JP2005313886A (en) 1997-07-22 2005-03-30 Agent apparatus
JP2005098224A JP3918850B2 (en) 1997-07-22 2005-03-30 Agent device
JP2005098225A JP3891202B2 (en) 1997-07-22 2005-03-30 Agent device
JP2005098223A JP2005313885A (en) 1997-07-22 2005-03-30 Agent apparatus
JP2005098226A JP2005293587A (en) 1997-07-22 2005-03-30 Agent device
JP2008156559A JP4353310B2 (en) 1997-07-22 2008-06-16 In-vehicle device

Publications (2)

Publication Number Publication Date
JPH11272639A JPH11272639A (en) 1999-10-08
JP4032492B2 true JP4032492B2 (en) 2008-01-16

Family

ID=14136221

Family Applications (1)

Application Number Title Priority Date Filing Date
JP09538698A Expired - Fee Related JP4032492B2 (en) 1998-03-23 1998-03-23 Agent device

Country Status (1)

Country Link
JP (1) JP4032492B2 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6411898B2 (en) 2000-04-24 2002-06-25 Matsushita Electric Industrial Co., Ltd. Navigation device
JP2003091299A (en) * 2001-07-13 2003-03-28 Honda Motor Co Ltd On-vehicle voice recognition device
US20040225416A1 (en) * 2001-11-13 2004-11-11 Tomoki Kubota Data creation apparatus
EP2140341B1 (en) 2007-04-26 2012-04-25 Ford Global Technologies, LLC Emotive advisory system and method
JP6312053B2 (en) * 2013-09-06 2018-04-18 株式会社ユピテル System and program
JP6240863B2 (en) * 2016-04-27 2017-12-06 株式会社ユピテル Driving support system and driving support program
JP2018205211A (en) * 2017-06-07 2018-12-27 本田技研工業株式会社 Information provision device and information provision method

Also Published As

Publication number Publication date
JPH11272639A (en) 1999-10-08

Similar Documents

Publication Publication Date Title
US7369938B2 (en) Navigation system having means for determining a route with optimized consumption
CN1690654B (en) Route searching apparatus
US7805223B2 (en) Dialogue system
US7783422B2 (en) Navigation device and method of navigating vehicle
EP2090989B1 (en) Image display system, display device and display method
US6388582B2 (en) Lane guidance display method, and navigation device and recording medium for realizing the method
US9630631B2 (en) System and method for dynamic in-vehicle virtual reality
US6285924B1 (en) On-vehicle input and output apparatus
JP3448134B2 (en) Vehicle navigation device
CN101194143B (en) Navigation device with camera information
US8170795B2 (en) Navigation system with animated intersection view
JP2008058409A (en) Speech recognizing method and speech recognizing device
US20050256635A1 (en) System and method for assigning a level of urgency to navigation cues
US6941224B2 (en) Method and apparatus for recording voice and location information
JP4886597B2 (en) Lane determination device, lane determination method, and navigation device using the same
JPWO2005038745A1 (en) Driving information storage method and driving information storage system for motorcycles
US7584051B2 (en) Navigation system and program for the same
US7813872B2 (en) Image processing system, method and apparatus for correlating position data with image data
JPWO2006035755A1 (en) Mobile navigation information display method and mobile navigation information display device
JP3985230B2 (en) Vehicle navigation apparatus and program
JP3876463B2 (en) Map information providing apparatus and method
EP1956600B1 (en) Image display system, display apparatus, and display method
JP2013178266A (en) Method and system for providing user with landmark information in driving guidance
US9014979B2 (en) Generating photogenic routes from starting to destination locations
US20080147308A1 (en) Integrating Navigation Systems

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20050217

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20070301

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20070427

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20070703

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20070903

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20071002

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20071015

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101102

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111102

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121102

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121102

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131102

Year of fee payment: 6

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

LAPS Cancellation because of no payment of annual fees