JP4934455B2 - Drive information knowledge system, method and program - Google Patents

Description

  The present invention relates to a technique for acquiring information related to a scene outside a vehicle during driving.

  People not only enjoy the drive time itself by going out by vehicle, but also confirm their knowledge by actually watching with their own eyes or listening with their ears. They also exchange knowledge with each other during the drive by talking to their passengers. Furthermore, based on the topics and keywords in the conversation during driving, the user accesses map information and information on the Internet to acquire general knowledge of society and knowledge of others. In this way, you can be aware of changes in the situation and the existence of new facts by verifying your knowledge, correcting mistakes, and checking against past memories.

  In other words, it can be said that the drive is an opportunity to update and acquire its own knowledge. Nevertheless, there is no system that supports the provision of information on conversations during driving and surrounding situations (scenes outside the vehicle). In addition, recalling past experiences from driving experiences, collating them, providing information according to personal preferences and knowledge from driving experiences, and assisting in storing and distributing knowledge acquired from driving experiences There is no system to perform.

2. Description of the Related Art Conventionally, an apparatus that displays driving-related records of a drive on a screen is known (see, for example, Patent Document 1). This device accumulates driving-related information and improves amusement by changing the display of driving support information in the next drive. The travel-related information in the present invention is data relating to travel distance, toll road fee, and visited place.
Japanese Patent Laid-Open No. 2002-071379 (page 2-5, FIG. 1)

  However, although the information displayed on the screen with the conventional apparatus has a value as a fact, it is insufficient as information from the viewpoint of supporting acquisition and sharing of individual knowledge. In particular, it lacks essential information that represents the personal experience and knowledge of how people felt during driving, how they felt, what they thought about, and how they reacted. ing. In other words, it is not considered to make the information obtained during the driving into knowledge. The meaning of “knowledge” will be described later.

  By the way, conventionally, there is a drive recorder for recording a drive scene. The drive recorder records only the video (accident video, etc.) at the time when a large acceleration is generated in the vehicle as drive data, and the normal drive data is discarded. In other words, the drive recorder does not consider playing recorded drive data to recreate and enjoy the drive experience or extracting knowledge from it.

  Conventionally, a system such as a drive simulator has also been developed. The drive simulator generates an imaginary drive scene using computer graphics, and generates motion according to the generated topography, road conditions, and mechanical characteristics of the vehicle body. The purpose of this drive simulator is to generate a sense that the driver is driving a car with a sense of presence, and to analyze the driving behavior of the driver according to the generated situation. Therefore, the drive simulator does not take into account the actual driving experience.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a drive information knowledge system capable of knowledgeizing information during driving (particularly information relating to a scene outside the vehicle).

  The knowledge system for driving information according to the present invention includes a conversation data recording means for recording a conversation outside a vehicle conducted by a user with respect to a scene outside the vehicle as conversation data, and an object included in the scene outside the vehicle is pointing. And a pointing position detecting means for detecting a pointing position when the outside scene conversation is performed and user information when the outside scene conversation is performed, and based on a conversation excitement degree indicating a degree of excitement of the outside scene conversation. Quantization means for extracting, as conversation quantum data, a portion of the data where the outside scene conversation is excited, and an object related to the object pointed to when the outside scene conversation is raised in the conversation quantum data Data is used as search data for the conversation quantum data. And it includes a data adding means for adding a data storage means for storing the conversation quantum data the retrieval data is added, the.

  Thereby, based on the degree of conversation excitement obtained from the user information during the conversation outside the vehicle scene, the part where the conversation outside the vehicle scene is extracted is extracted as the conversation quantum data. In addition, the object related data related to the object pointed when the conversation is excited is added to the conversation quantum data as search data. Accordingly, it is possible to easily find the conversation quantum data when the user's conversation is raised with respect to the scene outside the vehicle while using the object-related data regarding the object pointed when the conversation is raised as a clue. .

  Here, “knowledge” refers to extracting a useful part from certain information (for example, conversation data) and adding search data. In addition to the useful parts that the user needs, the information includes useless parts that the user does not need much. Therefore, first, a process of extracting (also referred to as quantization) a useful part from information (for example, conversation data) is performed. Moreover, even if useful information cannot be extracted immediately when needed, it cannot be said that the information is knowledge of the user. Therefore, next, processing for adding search data to the useful information (for example, conversation quantized data) (also referred to as search enablement) is performed. Information in this way (= quantization + search enablement) makes it possible to easily find information useful for the user using the search data as a clue, and use that useful information as the user's knowledge Will be able to.

  The knowledge system for driving information according to the present invention includes a driving experience data recording means for recording a video including a scene outside the vehicle being driven as driving experience data, and using the driving experience data. Simulation means for performing a drive simulation for reproducing an outside scene, wherein the conversation data recording means records an outside scene conversation performed by the user when reproducing the outside scene during driving as conversation data, The pointing position detection means detects a pointing position when an object included in the outside scene being driven is pointed when the outside scene being driven is reproduced, and the quantization means The conversation of the outside scene when playing the outside scene of Based on the degree of conversation excitement indicating the degree of ascent, the portion of the conversation data where the outside scene conversation has risen is extracted as conversation quantum data, and the data adding means reproduces the outside scene during driving The object related data related to the object pointed when the outside scene conversation of the vehicle is excited may be added as search data for the conversation quantum data.

  As a result, when a drive simulation is performed, a video of a scene outside the vehicle recorded as drive experience data is reproduced. Then, based on the degree of conversation excitement, when the outside scene during driving is reproduced, a portion where the outside scene conversation is excited is extracted as conversation quantum data. In addition, the object related data related to the object pointed when the conversation is excited is added to the conversation quantum data as search data. Therefore, with the conversation quantum data when the user's conversation is raised about the scene outside the vehicle being reproduced by the drive simulation as a clue, the object related data regarding the object pointed when the conversation is raised, Can be easily found.

  Further, the drive information knowledge system according to the present invention comprises simulation experience data recording means for recording, as simulation experience data, a video including a scene outside the vehicle being reproduced that was reproduced in the drive simulation, and the simulation means The simulation experience data is used to reproduce the drive simulation scene, to recursively reproduce the vehicle exterior scene reproduced in the drive simulation, and the conversation data recording unit is configured to reproduce the vehicle exterior scene during the drive. The outside scene conversation performed by the user when recursively played back is recorded as conversation data, and the pointing position detecting means recursively plays back the outside scene during driving when the outside scene during driving is replayed. The objects included in the scene A pointing position at the time of inching is detected, and the quantization means is based on a conversation excitement degree indicating a degree of excitement of the outside scene conversation when the outside scene during driving is recursively reproduced. A part of the conversation data where the outside scene conversation is excited is extracted as conversation quantum data, and the data adding means is when the outside scene conversation when the outside scene during driving is recursively reproduced. You may add the object relevant data regarding the said object currently pointed as search data of the said conversation quantum data.

  Thereby, when the scene of the drive simulation is reproduced, the video of the scene outside the vehicle reproduced by the drive simulation is recursively reproduced. Here, “recursively reproduced” means that another image (for example, an image of a scene outside the vehicle being driven) is reproduced in the reproduced image (for example, an image of a scene of a drive simulation). That means. Then, based on the degree of conversation excitement, when the outside scene during driving is recursively reproduced, a portion where the outside scene conversation is excited is extracted as conversation quantum data. In addition, the object related data related to the object pointed when the conversation is excited is added to the conversation quantum data as search data. Therefore, with the conversation quantum data when the user's conversation is raised about the scene outside the vehicle being recursively played back as a clue, the object related data regarding the object pointed at when the conversation is raised, Can be easily found.

  In the drive information knowledge system according to the present invention, the quantization means obtains the degree of conversation excitement from user voice information during conversation outside the vehicle included in the conversation data, and the conversation excitement of the conversation data. A portion having a degree equal to or higher than a predetermined threshold may be extracted as the conversation quantum data.

  Thereby, based on the conversation excitement degree calculated | required from the user audio | voice information in a scene outside a vehicle conversation, the part whose conversation excitement degree is more than a predetermined threshold value is extracted as conversation quantum data. The user voice information during the outside scene conversation (for example, the volume of the outside scene conversation, the number of utterances, the interval between utterances, etc.) corresponds to the degree of excitement of the user's conversation. Therefore, it is possible to accurately select the part of the conversation outside the vehicle that is excited from the conversation data.

  The drive information knowledge method according to the present invention is based on the conversation excitement level indicating the degree of excitement of the outside scene conversation among the conversation data in which the outside scene conversation performed by the user regarding the outside scene during driving is recorded. A portion where the outside scene conversation is excited is extracted as conversation quantum data, and the conversation quantum data is an object included in the outside scene when the outside scene conversation is raised, and is pointed during the outside scene conversation. Object related data relating to the target object is added as search data for the conversation quantum data.

  Also in this method, as in the above-described system, the conversation quantum data when the user's conversation rises with respect to the outside scene while driving, the object related data regarding the object pointed when the conversation rises. It is possible to find it easily as a clue.

  The knowledge program for driving information according to the present invention is based on the conversation excitement level indicating the degree of excitement of the outside scene conversation among conversation data in which the outside scene conversation performed by the user regarding the outside scene during driving is recorded in a computer. A process for extracting the portion where the outside scene conversation is excited as conversation quantum data, and an object included in the outside scene when the outside scene conversation is raised in the conversation quantum data, A process of adding object related data related to an object pointed during a scene conversation as search data for the conversation quantum data is executed.

  In this program, as in the above system, the conversation quantum data when the user's conversation is excited about the outside scene while driving, the object related data regarding the object pointed at when the conversation is excited, It is possible to find it easily as a clue.

  According to the present invention, based on the degree of conversation excitement, the portion of the conversation outside the vehicle that is excited is extracted as conversation quantum data, and the object related to the object pointed to when the conversation is excited is included in the conversation quantum data. Data is added as search data. As described above, by knowledge of the information on the conversation outside the vehicle during driving, it is possible to easily find the quantum data of conversation when the conversation of the user is raised using the object-related data as a clue.

  A drive information knowledge system according to an embodiment of the present invention will be described below with reference to the drawings. This knowledge system for driving information records conversations made by the user regarding the scene outside the car as conversation data, and converts the recorded conversation data into knowledge (= quantization + searchability) to convert the conversation quantum data. Create and enable referral, processing, and distribution of the conversation quantum data.

  The configuration of the drive information knowledge system according to the present embodiment will be described with reference to FIGS. FIG. 1 is a diagram showing a schematic configuration of a drive information knowledge system 1. As shown in FIG. 1, an in-vehicle system 2 mounted on a vehicle driven by a user A and a home system 3 installed at the home of the user A are provided. The in-vehicle system 2 and the home system 3 are connected to each other via a network 4 so that data communication is possible. A shared database 5 that can share information among a plurality of users is connected to the network 4. The network 4 is also connected to an in-vehicle system 2 mounted on the vehicles of other users and a home system 3 installed at the homes of other users.

  In the example of FIG. 1, for convenience of explanation, only the in-vehicle system 2 of the user A, the home system 3 of the user A, the shared database 5, and the home system 3 of the user X (one of a plurality of other users) are illustrated. Yes. In FIG. 1, user A is a driver and user B is a passenger. That is, it can be said that the user A and the user B are driving experience persons (passengers). User C is a drive non-experience person who has not experienced the drive of user A and user B.

(In-vehicle system 2)
FIG. 2 is a diagram illustrating a schematic configuration of the in-vehicle system 2. As shown in FIGS. 1 and 2, a user's vehicle is provided with a plurality of in-vehicle cameras 6, a plurality of in-vehicle microphones 7, and a plurality of in-vehicle sensors 8. The in-vehicle camera 6 includes a camera for a scene outside the vehicle that captures images outside the vehicle and a camera for a scene inside the vehicle that captures images inside the vehicle. The in-vehicle microphone 7 records the conversation of the user in the car while driving. The in-vehicle sensor 8 includes a GPS sensor that measures the current position of the vehicle, a vehicle speed sensor that measures the vehicle speed, a vehicle orientation sensor that measures the vehicle orientation, a date sensor that measures the date and time, and the like. The in-vehicle system 2 includes a sensor manager 9 that controls operations of the in-vehicle camera 6, the in-vehicle microphone 7, and the in-vehicle sensor 8.

  The in-vehicle system 2 includes an in-vehicle database 10 in which data such as drive experience data is recorded, and a recording server 11 that performs data registration in the in-vehicle database 10. Via the sensor manager 9, the recording server 11 sends a driving image taken by the in-vehicle camera 6, vehicle information obtained by the in-vehicle sensor 8, a driving conversation recorded by the in-vehicle microphone 7, and the like. It is done.

  As shown in FIG. 2, the recording server 11 includes an image processing unit 12 that processes video during driving taken by the in-vehicle camera 6, a vehicle information receiving unit 13 that receives vehicle information during driving from the in-vehicle sensor 8, A voice processing unit 14 for processing a conversation during driving recorded by the in-vehicle microphone 7 is provided. The in-vehicle database 10 includes a drive experience database 15 in which drive experience data is recorded, an authentication database 16 in which authentication data is recorded, and a profile database 17 in which profile data is recorded.

  The drive experience data is data in which various information in the drive is recorded in synchronization. For example, the driving experience data includes images of the outside and inside the vehicle taken by the in-vehicle camera 6, conversations during the driving recorded by the in-vehicle microphone 7, and various vehicle information obtained from the in-vehicle sensor 8. . Therefore, the conversation (external scene conversation) performed by the user regarding the outside scene during driving is also included in the driving experience data.

  Driving experience data is based on passenger ID, date, destination, geographic coordinates, speed, direction of travel, voice input, character input, etc. in addition to video around the vehicle, video inside the vehicle, and voice inside the vehicle during driving. All the comments made by the passengers are synchronized and converted into data. In the drive experience data, metadata indicating a time point when an operation with high followability between users (a conversation excitement) occurs is added in a form that can be searched later.

  The authentication data is data (for example, a user authentication password) for performing user authentication of the passenger. The profile data is data (for example, a user authentication profile) for performing profile authentication of the passenger. In addition, the vehicle is provided with a passenger monitor 18 that acquires a passenger profile (for example, a face profile of the user A or the user B) in order to perform profile authentication.

  As shown in FIGS. 1 and 2, the in-vehicle system 2 is used to operate in-vehicle devices such as the in-vehicle camera 6, the in-vehicle microphone 7, and the in-vehicle sensor 8, and to input data to the in-vehicle database 10. A user interface (user I / F) 19 is provided. Further, the recording server 11 includes a control unit 20 that receives various operations and inputs from the user I / F 19 and performs various controls, and a recording processing unit 21 that performs processing such as data registration for the in-vehicle database 10. It has been.

(Home system 3)
FIG. 3 is a diagram showing a schematic configuration of the home system 3. As shown in FIG. 1 and FIG. 3, a home database 22 in which data such as drive experience data is recorded and a simulation server 23 that performs drive simulation described later using the drive experience data are provided at the user's home. Yes. The drive simulation means that a video such as a scene outside the vehicle being driven is reproduced along with the conversation. Here, the simulation server 23 corresponds to the simulation means of the present invention.

  Further, the home system 3 includes a display 24 on which an image during driving reproduced by the drive simulation is displayed, and a user I / F 25 that displays a pointer on the display 24 to indicate the user's pointing position P. . It can be said that this user I / F 25 has a function of a pointing device. In the present embodiment, as a pointing method, for example, a method based on gaze detection, a method based on detection of a pointing direction and a pattern recognition of pointing motion, a method based on word spotting, a method of pointing on a display device, or the like is used. Here, the method based on gaze detection refers to a method based on gaze detection in which a portion viewed when the conversation is excited is detected as a pointing position. In addition, the pattern recognition method of pointing motion is to indicate the depth and direction by moving the fingertip forward (depth direction) while pointing, and then moving the fingertip sideways (for example, rightward). Say the method. The word spotting method refers to a method in which an object is indicated by words by saying the name of a building. Also, the method of instructing on the display device is to display an image of a scene outside the vehicle on the touch screen and instruct the object on the touch screen, or overlay the touch panel on the car window, etc. This is a method of pointing an object by touching with.

  A map display window 26 is displayed on the display 24 so as to be superimposed on the image being driven. In the map display window 26, a map indicating the current position of the vehicle being driven and related information related to the pointed object O are superimposed and displayed.

  As shown in FIG. 3, the simulation server 23 receives an operation or input from the user I / F 25 and performs various controls, and a pointing that detects a pointing position P instructed by the user I / F 25. A position detection processing unit 28 is provided. The simulation server 23 includes a video playback control unit 29 that plays back video and a map display control unit 30 that creates a map display window 26. Here, the pointing position detection processing unit 28 corresponds to the pointing position detection means of the present invention.

  The simulation server 23 includes a drive simulator 31 that performs a drive simulation, a drive experience data server 32 that performs a search for drive experience data used in the drive simulation, and a related information data server that performs a search for related information used in the drive simulation. 33 is provided.

  As shown in FIGS. 1 and 3, a plurality of home cameras 34 and a plurality of home microphones 35 are provided at the user's home. The home camera 34 is an indoor camera installed indoors and captures a scene of a drive simulation. The home microphone 35 is an indoor microphone installed indoors, and records the user's conversation during the drive simulation. The home system 3 includes a sensor manager 36 that controls operations of the home camera 34 and the home microphone 35. The simulation server 23 receives, via the sensor manager 36, an image during drive simulation shot by the home camera 34, a conversation during drive simulation recorded by the home microphone 35, and the like.

  As shown in FIG. 3, the simulation server 23 includes an image processing unit 37 that processes indoor video captured by the home camera 34 and an audio processing unit 38 that processes indoor conversation recorded by the home microphone 35. ing. The simulation server 23 includes a recording processing unit 39 that performs data registration in the home database 22. The recording processor 39 records the video and conversation during the drive simulation in the home database 22 as conversation data. Here, the home database 22 corresponds to the conversation data recording means of the present invention.

  Moreover, the simulation server 23 includes a knowledge processing unit 40 that performs conversation data knowledge processing described later. The knowledge processing unit 40 includes a quantization unit 41 that extracts, as conversation quantum data, a portion of conversation data in which the conversation outside the vehicle scene is excited based on the conversation excitement level. Also, the knowledge processing unit 40 includes a data adding unit 42 that adds related information (object related data) related to the object O that is pointed when the outside-the-scene scene conversation is excited to the conversation quantum data as search data. I have. Here, the quantization unit 41 corresponds to the quantization unit of the present invention, and the data addition unit 42 corresponds to the data addition unit of the present invention.

  The degree of conversation excitement can be obtained from user information during drive simulation. The user information includes user voice information during conversation (volume of the user's conversation, number of utterances, interval between utterances, etc.), body movement of the user during conversation, blood pressure, pulse, body temperature, pupil size, etc. included.

  As shown in FIG. 3, the home database 22 includes a drive experience database 43 in which drive experience data is recorded, and a related information database in which related information (for example, names and origins) related to buildings, stores, mountains, lakes, and the like is recorded. 44, a map database 45 in which map data is recorded, and a conversation quantum database 46 in which conversation quantum data to which search data is added are recorded. The drive experience data 43 acquired from the drive experience database 15 of the vehicle via the network 4 is recorded in the drive experience database 43 at home. Here, the conversation quantum database 46 corresponds to the data storage means of the present invention. The drive experience database 43 corresponds to the drive experience data storage means of the present invention.

(Shared database)
In the shared database 5, drive experience data and conversation quantum data are recorded so as to be accessible via a network 4 from a plurality of users. That is, the drive experience data and the conversation quantum data are generally shared so that they can be accessed from the network 4.

(Drive simulation)
Here, the drive simulation will be described with reference to FIGS. FIG. 4 is an explanatory diagram of drive simulation. FIG. 4 illustrates a state in which a user A (driver) and a user B (passenger) return to their home after driving and reproduce a scene in the driving. In this example, the user C who has not experienced the drive also watches the video of the drive simulation together with the user A and the user B.

  In this drive simulation, a scene during driving as shown in FIG. Here, along with the scenes of the user A and the user B in the car (in-car scene), the road and building scenes (out-car scene) outside the car are reproduced. On the display 24, a pointer (indicated by a circle in FIG. 4) indicating the pointing position P designated by the user (for example, the user A) is displayed.

  In the drive simulation, various conversations are performed between the users A to C. For example, for a building in a scene outside the vehicle being driven, the user A matches the pointing position P (pointer position) with the building position on the screen and asks "Do you know the name of that building?" . Then, the user B answers “I don't know”, or the user C answers “The name of the building is XX”.

  The images and conversations of the users A to C during such a drive simulation are captured and recorded by the home camera 34 and the home microphone 35 and recorded in the home database 22 as conversation data. The conversation data includes a conversation (external scene conversation) performed by the users A to C when the external scene during driving is reproduced.

  In addition, the conversation data includes an image of a scene outside the vehicle being driven. Therefore, it can be said that this conversation data corresponds to the drive experience data of the present invention. Further, the conversation data includes an image of a scene outside the vehicle being driven, which is reproduced by the drive simulation. Therefore, it can be said that this conversation data corresponds to the simulation experience data of the present invention. Moreover, it can be said that the home database 22 corresponds to the simulation experience data recording means of the present invention.

  FIG. 5 is an explanatory diagram showing another example of drive simulation. FIG. 5 illustrates a state in which the user X and the user Y are reproducing the user A and the user B drive simulation scenes. User X and user Y are other users who have not experienced the drive of user A or user B.

  In this drive simulation, as shown in FIG. 5, a drive simulation scene of another user (users A to C) is displayed on the display 24. In this case, in addition to the scenes of the users A to C watching the drive simulation at the home of the user A, the scenes of the user A and the user B in the car (in-car scene), the roads and buildings outside the car (the out-of-car scene) ) Is recursively played back. On the display 24, a pointer (indicated by a circle in FIG. 5) indicating the pointing position P designated by the user (for example, the user X) is displayed. In FIG. 5, for convenience of explanation, the pointer indicating the pointing position P designated by another user (for example, user A) in the drive simulation is omitted.

  Also in this drive simulation, various conversations are performed between the user X and the user Y. For example, for a building in a vehicle exterior scene that is recursively played, the user X matches the pointing position P (pointer position) with the position of the building on the screen and says “I heard the origin of that building name. When the question “is there?”, The user Y answers that “I think that the name of the building originated from XX”.

  The images and conversations of the users X and Y during reproduction of such a drive simulation are photographed and recorded by the home camera 34 and home microphone 35 and recorded in the home database 22 as conversation data. This conversation data includes a conversation (ex-vehicle scene conversation) performed by the users X and Y when the outside-vehicle scene being driven is recursively reproduced.

(Knowledge conversation data)
Here, the conversation data knowledge process will be described with reference to FIG. The conversation data is recorded video and conversation when a drive simulation is performed for a certain drive. As shown in FIG. 6, the conversation data is provided with metadata MD indicating when and where the person is related to the drive scene.

  The portion of the conversation data where the scene conversation outside the vehicle is excited is extracted as the conversation quantum data. For example, in the conversation data, a portion for 5 minutes before and after a portion where the conversation excitement level is larger than the threshold is extracted as the conversation quantum data. As described above, the quantization unit 41 of the knowledge processing unit 40 performs a process of cutting out (quantizing) a portion that can be quantized from the conversation data.

  The conversation quantum data is given metadata MD indicating the conversation data from which the conversation quantum data is cut out. That is, the metadata MD includes information indicating which part of which conversation data is cut out. The metadata MD is used as data (association data) for associating conversation quantum data and conversation quantum data, or conversation quantum data and conversation data. In this case, the conversation quantum data may be associated with the original conversation data obtained by cutting out the conversation quantum data. Further, the conversation quantum data may be associated with conversation data different from the part of the original conversation data from which the conversation quantum data is cut out.

  Further, related information (object related data) related to the object O that is pointed when the outside scene conversation is excited is given to the conversation quantum data as annotation data AD. For example, a keyword spoken by the user when a scene conversation outside the vehicle is excited is given as annotation data AD. Further, the user's comment on the object O that is pointed when the outside-scene conversation is excited is given as annotation data AD. That is, the annotation data AD includes information on what user has made what kind of remarks and comments. This annotation data AD is used as data (search data) for searching conversation quantum data.

  In this case, the in-vehicle sensor 8 can know the current position of the vehicle and the traveling direction (direction). In addition, the pointing position detection processing unit 28 can detect the pointing position P on the display 24. Therefore, the object O of the scene outside the vehicle pointed to by the user can be specified from the map data, and related information related to the object O can be searched from the related information database 44.

  The operation of the drive information knowledge system 1 configured as described above will be described. Here, the overall flow of the operation of recording a video, conversation, and in-vehicle sensor information at the time of driving and performing a drive simulation in an environment outside the vehicle such as at home using the recorded driving experience data will be described first.

  First, the vehicle-mounted camera 6 captures images inside and outside the vehicle, and the vehicle-mounted microphone 7 records ambient sounds outside the vehicle and conversations inside the vehicle. Further, information such as date / time, geographical coordinates, vehicle speed, vehicle direction, and the like is acquired by the in-vehicle sensor 8. Then, information such as video and conversation is recorded synchronously as drive experience data. Drive experience data is recorded in two modes: an automatic recording mode and a mode in which a user designates a recording section. The drive experience data is recorded only when the user permits it. This permission is performed by collating user authentication data and user profile information registered in advance. Only when not registered, ask the user for permission on the spot.

  Next, the drive experience data recorded in the in-vehicle database 10 is transmitted to the simulation server 23 at home through a medium such as a wireless network or a flash memory. At this time, a URL corresponding to the data may be generated and accessed from the network 4.

  Then, the drive experience data is reproduced in the drive simulation environment at the user's home. Also during the reproduction of the drive experience data, the home camera 34 and the home microphone 35 record the drive scene being reproduced and the conversation of the user on the spot (conversation during simulation) as conversation data. On the display 24, related information retrieved using the position information as a key is superimposed and displayed. When the user indicates a position of interest, a pointer is displayed at the pointing position P on the screen, and information on the pointing position P is also recorded as conversation data.

  In this way, based on the conversation data recorded in the drive simulation, a scene where conversation is excited is determined and cut out as another video data (conversation quantum data) (conversation quantization). The conversation quantum data and the conversation data can hold the link information as metadata and refer to each other. That is, conversation quantum data generated from the same conversation data can be referred to each other through the conversation data. Annotation data is added to the conversation quantum data. The annotation data includes a pointer indicating the position in the video and comment data such as text. The comment data may be speech or other conversational quanta other than characters. Such annotation is performed by using, for example, RDF.

  The conversation quantum data is registered in the shared database 5. This enables conversational quantum data to become social knowledge and to be distributed in general. In addition, a general person (third party) can newly generate conversation quantum data, and can perform editing such as adding annotations to existing conversation quantum data.

  The conversation quantum data recorded in the home database 22 or the shared database 5 may be referred to during driving through the wireless network 4. This makes it possible to access conversational quantum data extracted from past driving experience data even during driving.

  Next, with reference to FIG. 7 and FIG. 8, the operation of drive simulation and conversation data knowledge will be described in detail. FIG. 7 is a flowchart for explaining an operation when performing a drive simulation using the drive information knowledge system 1 according to the present embodiment. FIG. 8 is a sequence diagram for explaining an operation when performing a drive simulation using the drive information knowledge system 1 according to the present embodiment to make the conversation data known.

  As shown in FIGS. 7 and 8, when a drive simulation is performed, first, the drive simulator 31 is activated (S1). Then, a search condition is input to the drive simulator 31 to search drive experience data for performing a drive simulation (S2).

  When a search result is received from the drive simulator 31, desired drive experience data is selected from the search result (S3). When the drive simulator 31 receives a selection request from the user, the drive simulator 31 sends an instruction to start the reproduction of the drive experience data to the drive experience data server 32. The drive experience data server 32 accesses the drive experience database 43 and displays a video of the drive scene. In this way, the drive experience data is reproduced (S4).

  In the drive simulator 31, it is determined whether or not there is a map display request from the user (S5). When a map display request is made, the drive simulator 31 activates the map display window 26. Then, the map screen is displayed by accessing the map database 45 (S6).

  The user instructs the user's interest location (object O) in the outside scene with the user I / F 25 while having a conversation about the scene being driven. The pointing position detection processing unit 28 detects the pointing position P on the display 24 (S7). In the drive simulator 31, it is determined whether or not the detected pointing position P is in the outside scene (S8).

  If the pointed pointing position P is in the scene outside the vehicle, the drive simulator 31 calculates the coordinates (geographic coordinates) of the object O on the map and specifies the object O. The related information data server 33 accesses the related information database 44 and searches for related information (object related data) related to the object O (S9).

  The drive simulator 31 receives the search result of the related information from the related information data server 33, and determines whether the related information of the object O has been found (S10). When related information (object related data) relating to the object O is found, an optimal display position of the related information on the display 24 is calculated, and the related information is superimposed and displayed (S11).

  If a drive simulation end request is made (S12), the drive simulation ends.

  In this way, the stored drive experience data is retrieved, the selected data is reproduced on the display 24, and the drive experience is reproduced. Then, based on the geographical coordinates and orientation added to the drive experience data, the correspondence between each point on the display 24 and the area of the geographical coordinates is taken, and the related information using the geographical coordinates in the corresponding area as a link key. Is displayed and superimposed on the display 24.

  When the drive simulation is performed, the video and audio during the drive simulation are recorded as conversation data. From the conversation data, sections that can be made known are cut out (conversation quantization), and the conversation quantum data is generated by adding (annotated) the knowledge content. The conversation quantum data and the conversation data are related by metadata so that they can be referred to each other.

  The conversation quantum data acquired in this way is recorded in the shared database 5 so that it can be distributed socially. Thereby, the chance that the conversation quantum data is reused can be increased, and more knowledge can be collected. Specifically, identifiers for identifying resources on the network are assigned to the drive experience data and the conversation quantum, and these are stored in the shared database 5 accessible from the network 4.

  In the present embodiment, through the drive simulation, a knowledgeable part is cut out as a conversation quantum, and related information and comments are added and recorded. At this time, the object related data regarding the object O pointed during the drive simulation is presented on the display 24. Note that once recorded conversational quantum data can be presented as related information during the next drive simulation.

  According to the drive information knowledge system 1 according to the present embodiment, the swelled part of the outside scene conversation is extracted as the conversation quantum data based on the conversation excitement degree, and the conversation is excited in the conversation quantum data. The object related data related to the object O pointed at is added as search data. As described above, by knowledge of the information on the conversation outside the vehicle during driving, it becomes possible to easily find the conversation quantum data when the conversation of the user is raised using the object-related data as a clue.

  In this case, the conversation quantum database 46 is constructed from the video / conversation / vehicle information recorded by the drive, and knowledge for complementing the conversation quantum database 46 is added by sharing. Therefore, knowledge with high reusability can be extracted from the driving experience and can be distributed socially. In other words, it is possible not only to record the driving experience but also to extract social knowledge with high reusability.

  In this embodiment, based on the degree of conversation excitement obtained from the user information during the outside scene conversation, a portion where the outside scene conversation is excited is extracted as conversation quantum data. In addition, the object related data related to the object O pointed to when the conversation is excited is added to the conversation quantum data as search data. Therefore, it is possible to easily find the conversation quantum data when the user's conversation rises with respect to the scene outside the vehicle while using the object-related data related to the object O pointed to when the conversation is raised. it can.

  In order to record a drive scene, it is conceivable to use a device such as a digital camera. However, if the driver himself takes a picture with the digital camera during driving, the driver's safe driving is hindered. Even when a passenger other than the driver takes a picture, if the user concentrates on taking the picture, the photographer will be prevented from enjoying the drive itself. For example, too much focus on shooting a drive scene interrupts the conversation during the drive, preventing the enjoyment of the drive itself. Then it is the end of this fall. In this embodiment, by preparing a knowledge process after driving, the driver and passengers during driving can enjoy the driving experience itself.

  Just by recording the drive history like a conventional device, it is impossible to convey what the person felt or thought on the spot. It is also possible to use conversation while driving. However, people don't always share their thoughts on the spot, so they extract valuable knowledge only after going through the process of recreating the scene during the drive and talking to each other what they came up with after driving. be able to.

  In the present embodiment, when the drive simulation is performed, the video of the scene outside the vehicle recorded as the drive experience data is reproduced. Then, based on the degree of conversation excitement, when the outside scene during driving is reproduced, a portion where the outside scene conversation is excited is extracted as conversation quantum data. In addition, the object related data related to the object O pointed to when the conversation is excited is added to the conversation quantum data as search data. Therefore, the conversation quantum data when the user's conversation rises with respect to the scene outside the vehicle that is reproduced in the drive simulation is used as a clue to the object related data related to the object O that is pointed when the conversation is raised. Can be easily found.

  Further, in the present embodiment, when the scene of the drive simulation is reproduced, the video of the outside scene reproduced by the drive simulation is recursively reproduced. Then, based on the degree of conversation excitement, when the outside scene during driving is recursively reproduced, a portion where the outside scene conversation is excited is extracted as conversation quantum data. In addition, the object related data related to the object O pointed to when the conversation is excited is added to the conversation quantum data as search data. Therefore, the conversation quantum data when the user's conversation rises with respect to the scene outside the vehicle being recursively reproduced is used as a clue to the object-related data related to the object O pointed to when the conversation is raised. Can be easily found.

  In addition, of the information obtained from the driving experience, the part that you want to know is very limited. Therefore, in order to efficiently perform the knowledge process, a mechanism that can accurately search for points that are candidates for knowledge is important.

  In the present embodiment, a portion where the conversation excitement level is equal to or greater than a predetermined threshold is extracted as the conversation quantum data based on the conversation excitement level obtained from the user voice information during the conversation outside the vehicle. The user voice information during the outside scene conversation (for example, the volume of the outside scene conversation, the number of utterances, the interval between utterances, etc.) corresponds to the degree of excitement of the user's conversation. Therefore, it is possible to accurately select a part of the conversation outside the vehicle (a point that is a candidate for knowledge) from the conversation data.

  Even if the driver finds an object that he / she finds interesting during driving, the object can be confirmed in an environment outside the vehicle such as at home by recording the scene outside the vehicle. Therefore, knowledge acquisition and distribution can be realized while ensuring convenience and safety in driving without lowering the driver's awareness of safe driving.

  In addition, if there is something related to the current drive scene in the conversation quantum accumulated in the past, you can know what other people are thinking in the same situation , You can enjoy the drive scene effectively.

  In addition, you can analyze your driving method by reviewing the driving scene after driving. Therefore, knowledge related to drive technology can be extracted.

  By referring to the driving experience data of other people, it becomes possible to perform a pseudo driving trip for a place that has never been actually visited. In this way, it is possible to acquire various knowledge while enjoying the driving experience.

  The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified according to the purpose within the scope of the claims. is there.

  For example, in the above description, the example in which the conversation data is recorded during the drive simulation (or during the reproduction of the drive simulation) has been described. However, the scope of the present invention is not limited to this, and the conversation data is recorded. May be performed during the drive.

  As described above, the present invention has an effect that it is possible to easily find conversation quantum data when a user's conversation is excited, and is useful as a knowledge system for drive information.

It is explanatory drawing which shows the outline of the knowledge-izing system of the drive information of this Embodiment. It is explanatory drawing which shows the outline of the vehicle-mounted system of this Embodiment. It is explanatory drawing which shows the outline of the home system of this Embodiment. It is explanatory drawing which shows the mode of reproduction | regeneration (drive simulation) of the scene outside a vehicle in the drive of this Embodiment. It is explanatory drawing which shows the mode of the reproduction | regeneration of the scene of the drive simulation of this Embodiment (recursive reproduction of the scene outside a vehicle in drive). It is explanatory drawing of the knowledge-ization process of the conversation data of this Embodiment. It is a flowchart explaining operation | movement of the reproduction | regeneration (drive simulation) of the scene outside a vehicle in the drive in this Embodiment. It is a sequence diagram explaining operation | movement at the time of performing drive simulation in this Embodiment and making knowledge of conversation data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Knowledge system 2 In-vehicle system 3 Home system 4 Network 5 Shared database 22 Home database 23 Simulation server 40 Knowledge processing part 41 Quantization part 42 Data addition part 43 Drive experience database 46 Conversation quantum database

Claims (5)

Conversation data recording means for recording, as conversation data, an out- of- vehicle scene conversation as a conversation performed by a user with respect to an out-of-vehicle scene that is a scene around the vehicle being driven;
Pointing for detecting a position indicated by the pointing device as a pointing position when an object included in the image of the outside scene is indicated using a pointing device when the image of the outside scene during driving is displayed Position detecting means;
A conversation excitement degree indicating a degree of excitement of the outside scene conversation is obtained from user voice information during the outside scene conversation included in the conversation data, and a portion of the conversation data in which the conversation excitement degree is a predetermined threshold or more, Quantization means to extract as conversational quantum data;
Data adding means for adding object-related data related to the object indicated by the pointing device in the video of the scene outside the vehicle to the conversation quantum data as search data for the conversation quantum data;
Data storage means for storing the conversation quantum data to which the search data is added;
A drive information knowledge system comprising: Drive experience data recording means for recording video including a scene outside the vehicle being driven as drive experience data;
Using the drive experience data, a simulation means for performing a drive simulation for reproducing the vehicle exterior scene during the driving;
With
The conversation data recording means records, as conversation data, an out-of-vehicle scene conversation performed by the user when reproducing the out-of-vehicle scene by the drive simulation ,
The pointing position detecting means detects a pointing position when an object included in the driving outside scene is instructed by using the pointing device when the driving outside scene is reproduced by the driving simulation. And
It said quantizing means on the basis of the prior Machinery story upsurge degree when playing outside scene in the drive by the driving simulation, a portion the conversation upsurge degree is not smaller than a predetermined threshold value among the conversation data, conversation Extract as quantum data,
The data adding means adds object-related data relating to the object indicated by using the pointing device in the image of the scene outside the vehicle reproduced by the drive simulation as search data for the conversation quantum data. The drive information knowledge system according to claim 1. A simulation experience data recording means for recording a video including a scene outside the vehicle being driven reproduced by the drive simulation as simulation experience data,
The simulation means replays the scene of the drive simulation using the simulation experience data, recursively reproduces the outside scene reproduced in the drive simulation,
The conversation data recording means records, as conversation data, an out-of-vehicle scene conversation performed by the user when the out-of-vehicle scene being recursively reproduced by the drive simulation .
The pointing position detecting means is a pointing when an object included in the outside scene during driving is instructed by using the pointing device when the outside scene during driving is recursively reproduced by the driving simulation. Detect the position,
The quantization means, on the basis of the driving simulation outside scene in the drive by the prior Machinery story upsurge degree when playing recursively, part the conversation upsurge degree is not smaller than a predetermined threshold value among the speech data Are extracted as conversational quantum data,
The data adding means includes object related data relating to the object instructed using the pointing device in the video of the outside scene recursively reproduced by the drive simulation, and data for searching the conversation quantum data. The drive information knowledge system according to claim 2, wherein: A method used in a drive information knowledge system,
Recording an out-of-vehicle scene conversation as a conversation a user has made about an out-of-vehicle scene that is a scene around the vehicle being driven;
When an object included in the image of the vehicle exterior scene is instructed by using a pointing device when the image of the vehicle exterior scene being driven is displayed, a position instructed by the pointing device is detected as a pointing position. When,
A conversation excitement degree indicating a degree of excitement of the outside scene conversation is obtained from user voice information during the outside scene conversation included in the conversation data, and a portion of the conversation data in which the conversation excitement degree is a predetermined threshold or more, Extracting it as conversational quantum data,
Adding to the conversation quantum data object related data related to the object indicated by using the pointing device in the video of the scene outside the vehicle as search data for the conversation quantum data;
Storing the conversation quantum data to which the search data is added;
A drive information knowledge method comprising: On the computer,
A function for recording a conversation outside the vehicle as a conversation performed by the user regarding a scene outside the vehicle during driving, as conversation data;
A function of detecting a position indicated by the pointing device as a pointing position when an object included in the image of the outside scene is indicated using a pointing device when the image of the outside scene during driving is displayed When,
A conversation excitement degree indicating a degree of excitement of the outside scene conversation is obtained from user voice information during the outside scene conversation included in the conversation data, and a portion of the conversation data in which the conversation excitement degree is a predetermined threshold or more, A function to extract as conversational quantum data,
A function of adding object-related data related to the object indicated by the pointing device in the video of the scene outside the vehicle to the conversation quantum data as search data for the conversation quantum data;
A function of storing the conversation quantum data to which the search data is added;
Drive information knowledge program to realize

Images