JP2004064409A - Information recording device, information recording method, and information recording program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device capable of using the line of sight of the passenger of a vehicle, for a use different from a conventional use such as the driving of a vehicle. <P>SOLUTION: This information recording device is provided with a line of sight detecting means 38 for detecting the line of sight of the passenger of a vehicle, visualization object specifying means 18 and 40 for specifying a visualization object visualized by the passenger of a vehicle based on the detected line of sight, and an image data storage means 40 for making a camera photograph the image of the visualization object, and for storing the image data. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an information recording device, an information recording method, and an information recording program, and more particularly, to an information recording device, an information recording method, and an information recording program that record information based on an occupant's line of sight.
[0002]
[Prior art]
A driver's line of sight is detected using a camera (eye camera) mounted in front of the driver's seat of the vehicle, and switches of an air conditioner are operated using the detected line of sight (Japanese Patent Application Laid-Open No. 6-261863). The illumination direction of the headlight and the brightness of the meter display are adjusted in accordance with the driver's visual recognition behavior (Japanese Patent Laid-Open No. 9-76815), and the degree of concentration on the driver's driving operation is estimated based on the driver's line of sight. (Japanese Unexamined Patent Publication No. 2000-211153) has been proposed.
[0003]
[Problems to be solved by the invention]
However, the conventional device only uses the detected line of sight of the occupant such as the driver for driving the vehicle or operating the on-vehicle equipment. The present invention has been made in such a situation, and it is an object of the present invention to provide an apparatus and the like that can use the line of sight of a vehicle occupant in an application different from a conventional application such as driving a vehicle. I do.
[0004]
[Means for Solving the Problems]
According to the present invention, a line-of-sight detection unit that detects a line of sight of an occupant of the vehicle, and a visual object specifying unit that specifies a visual target object that the vehicle occupant is visually recognizing based on the detected line of sight, An information recording device comprising: an image data storage unit that causes a camera to capture an image of a viewing target object and stores the image data.
[0005]
According to the present invention having such a configuration, a scenery or the like that the occupant wants to see and record while the vehicle is running can be saved as image data without performing a particularly troublesome operation.
[0006]
In a preferred aspect of the present invention, there is provided a visual object recording switch operated by the occupant to activate the visual object specifying means and the image data storage means. According to such a configuration, recording of image data that accurately reflects the occupant's will is performed.
[0007]
In another preferred aspect of the present invention, when the occupant visually recognizes the target object for a predetermined time or more, the visual object specifying device and the image data storage device are operated. In another preferred aspect of the present invention, the image data storage unit confirms with the occupant whether the image data needs to be stored.
[0008]
According to another preferred aspect of the present invention, a focal length of a lens of the camera is changed according to a distance to the visual recognition target. According to such a configuration, it is possible to record image data in which the size of the visual recognition target object is appropriate.
[0009]
Another image data storage means of the present invention stores audio information together with the image data. According to such a configuration, image data with a sense of reality can be obtained.
[0010]
According to another preferred aspect of the present invention, the image data storage means stores time data at the time of shooting together with the image data, so that the image data can be taken out in order of time. According to such a configuration, image data can be easily arranged.
[0011]
According to another preferred aspect of the present invention, the image data further includes information search means for searching for information on a visual recognition target included in an image, wherein the image data storage means is searched together with the image data. Information about the visual recognition target included in the image is stored.
[0012]
According to another aspect of the present invention, a step of detecting a line of sight of an occupant of the vehicle, a step of specifying a visual recognition target visually recognized by the occupant of the vehicle based on the detected line of sight, Causing the camera to capture image data of the target object and storing the image data.
[0013]
According to another aspect of the present invention, a line of sight of an occupant of the vehicle is detected, and based on the detected line of sight, an object to be visually recognized by an occupant of the vehicle is specified, and an image of the object to be visually recognized is provided. There is provided an information recording program characterized by causing a camera to shoot data and operating a computer to store the image data.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a basic configuration diagram showing a route guidance device which is an embodiment of the information recording device of the present invention.
[0015]
In FIG. 1, reference numeral 1 indicates a vehicle route guidance device (navigation device). An information center 2 is provided outside the vehicle. The information center 2 includes a host computer 4, a database 6 for map information and the like for storing map information and the like, and a various content database 8 for storing various contents. Have. The map information includes, in addition to road data on roads, building data on building names, accommodation facilities, and the like.
[0016]
The route guidance device 1 is connectable to the Internet 12 via a communication station 10 installed in each region, and is configured to be able to receive various information including map information from the information center 2.
[0017]
In addition, since the route guidance device 1 includes a DVD-ROM storing map information for route guidance, even when the map information is not received from the information center 2, it is possible to provide route guidance and information on buildings. it can. When the map information is received from the information center 2, more accurate route guidance and the like can be performed based on the latest map information.
[0018]
The route guidance apparatus 1 communicates various information such as map information and the like from the information center 2 to the Internet 12 via the Internet 12 and the CPU 14 for performing route guidance (route guidance) of a vehicle to a destination based on input data and occupant operation. A transmission / reception device 16 for receiving via the station 10 and transmitting information from the vehicle side to the information center 2 is provided. The transmission / reception device 16 is, for example, a car phone or a mobile phone.
[0019]
The route guidance device 1 further includes an HDD storage device 18 for storing map information and the like received from the information center 2, a display 20 for displaying various information such as map information, and a map information from the information center 2. A DVD-ROM 22 for storing map information prepared independently on the vehicle side, an operation switch 24 for the occupant to set a destination and request map information and the like to the information center 2, and a case where information cannot be received from the information center 2. An alarm 26 is provided to warn the user.
[0020]
The route guidance device 1 includes a GPS receiver 28 that detects the current position of the vehicle, a vehicle speed sensor 30, and a gyro sensor 32. The GPS receiver 28 receives a radio wave from a satellite to detect the current position of the vehicle, the vehicle speed sensor 30 detects the speed of the vehicle to obtain the moving distance, and the gyro sensor 32 detects the moving direction of the vehicle. The route guidance device 1 is configured to always accurately calculate the current position and orientation of the vehicle from the detection results of these sensors 28, 30, and 32.
[0021]
In addition, the route guidance device 1 includes a VICS receiver 34 for obtaining congestion information and the like from the information center 2 or another traffic information center (not shown).
[0022]
The route guidance device 1 of the present embodiment is a navigation device that performs route guidance while detecting the driver's line of sight and performing guidance as needed to guide the line of sight in a predetermined direction. Further, the route guidance device 1 specifies (estimates) objects such as buildings and landmarks to which the driver's gaze is directed according to a request from the driver, and uses the vehicle-mounted camera to identify these visible objects. It has a function of shooting and storing the image data.
[0023]
To execute such a function, the route guidance device 1 includes a CCD camera 36, an eye camera 38, and a visual image recording device 40.
[0024]
The CCD camera 36 is mounted on the upper part of the vehicle, and is configured to be able to photograph scenes in front and side of the vehicle. The CCD camera 36 has a zoom function, and can photograph a predetermined object at a wide angle or at an enlarged scale. It should be noted that the CCD camera may be capable of photographing 360 degrees around the vehicle.
[0025]
The eye camera 38 is attached to an upper position in the cabin, and is configured to detect a driver's line of sight and a distance to a gazing target by photographing a driver's pupil and the like. ing. Here, the eye camera includes, for example, an EOG method, a photoelectric element type EOG (P-EOG) method, a corneal reflection method, first and fourth Purkinje image detection methods, a contact lens method, a search coil method, and an infrared fundus camera method. Various techniques such as are applied.
[0026]
The visual recognition image recording device 40 includes a computer, detects the driver's line of sight based on the driver's line of sight detected by the eye camera 38 (the direction of the line of sight and / or the distance to the gazing target), and detects the current line of sight of the vehicle. The position and orientation of the driver and the driver's line of sight are collated with the map data to identify (estimate) what the driver is visually recognizing, and to cause the CCD camera 36 to take an image of the visual recognition target object, The image data of the target object is stored in the route guidance device 1 or in an external storage device connected via a network.
[0027]
FIG. 2 is a drawing showing the periphery of the driver's seat of a vehicle equipped with the route guidance device 1 of the present embodiment.
[0028]
A speaker 42 is attached to the A pillar near the driver's seat, and the speaker 42 provides voice guidance and notification to the driver. Further, on the windshield 44, the same content as the display content of the display 20 is displayed as the auxiliary display 20a by the OHD device. Further, a viewing target recording switch 48 is attached to the steering wheel 46. The viewing target recording switch 48 is a switch that is pressed by the driver when the driver recognizes the viewing target by the viewing image recording device 40 and records an image of the viewing target. Note that the route guidance device 1 of the present embodiment also performs the same processing as when the viewing target recording switch 48 is pressed, for example, by a voice input of “record an image of this building” to a microphone (not shown). It is configured to be performed.
[0029]
Next, the contents of the processing executed when the visual recognition image recording apparatus 40 records an image of a visual recognition target will be described with reference to the flowchart of FIG. This image recording is performed in parallel with normal route guidance.
[0030]
First, in step S1, it is determined whether or not the driver has instructed to save an image of the object visually recognized. This determination is made based on whether the viewing target recording switch 48 is pressed or whether there is a voice recording instruction such as “Record an image of this building.” A configuration in which the determination is made based on only one of these may be adopted.
[0031]
Further, when the same object is gazing for a predetermined time or more, the same processing as when the viewing target recording switch 48 is pressed may be started. In this case, it is preferable to determine whether or not the user is gazing for a predetermined time or more based on the detection result of the eye camera or the like. In such a configuration, it is preferable that the driver confirms whether or not to execute the image data recording of the visual recognition target before executing the image recording process.
[0032]
When YES is determined in the step S1, the process proceeds to a step S2 to estimate what the driver is visually recognizing. In other words, the driver's gaze (the direction and the distance to the object to be watched), which is detected from the photographing result of the eye camera 38, and the current position and direction of the vehicle are compared with the map information, and the driver visually recognizes them. Is estimated (specified).
[0033]
Next, in step S3, the accuracy of the estimation is verified. That is, the estimation accuracy of the visual recognition object performed in step S2 is classified according to the state of the target object in the line of sight. Specifically, for example, when the driver's visible objects are not clearly understood, when the focus is out of focus, when there are many buildings ahead of the line of sight and it is not possible to specify which building is being watched at all For example, it is assumed that the estimation level is “small”, and that, for example, a plurality of buildings exist adjacent to the detected line of sight, and the driver is looking at any of these buildings. , The estimated level is “medium”.
[0034]
Next, the process proceeds to step S4, where it is determined whether the estimated level is "small". If YES in step S4, it is impossible to record a valid image, so the process proceeds to step S5, and an instruction to prompt re-viewing is issued. This instruction is performed on the display 20 or by voice.
[0035]
When NO is determined in the step S4, the process proceeds to a step S6, and it is determined whether or not the estimated level is “medium”. When YES is determined in the step S6, the process proceeds to a step S7, in which the lens of the CCD camera 36 is zoomed to the wide-angle side to capture a wide-angle image in the viewing direction.
[0036]
When NO is determined in the step S6, it is determined that the estimated level is “large”, and the process proceeds to a step S8, in which the lens of the CCD camera 36 is zoomed to the telephoto side to capture an enlarged image of the visual recognition target. Further, at the time of shooting in step S7 or step S8, voice data such as a driver's voice may be recorded.
[0037]
When the processing in step S7 or step S8 is completed, the flow advances to step S9 to record the data of the captured image together with or in association with other data such as the date and time of shooting, the shooting location (address), and the name of the visual recognition target. . The photographing time is preferably obtained from a clock or the like in the computer. In addition, it is preferable that data such as a photographing place and a name of a visual recognition target be obtained by searching map data when specifying a visual recognition target. In a configuration for recording audio or the like at the time of shooting, this audio data is also recorded together with the image data. The recording destination of the image data is, for example, the HDD storage device 18 in the route guidance device 1 or the server of the information center 2.
[0038]
If NO in step S1, or if the process in step S5 ends, the process returns. In addition to the photographing date and time, a gaze time during which the user is gazing at the visual target may be recorded together with or in association with the image data.
[0039]
Since the image data recorded in this manner is recorded together with or associated with the shooting date and time, it can be taken out in chronological order or displayed side by side.
[0040]
FIG. 4 is an example in which captured images are displayed in chronological order. As shown in FIG. 4, each photographed image is displayed with a photographing date and time, a photographing place, and a photographing target name. Since the estimated level of the first recording target in this example, OO hospital, was “large”, the OO hospital was enlarged and photographed by the zoom function. In addition, the driver was gazing at the vicinity of the □ ● building, which is the third recording target, in the vicinity of the ☆ mark, but since there were other buildings around, the estimation level was set to “medium”, and the visibility position was changed. It is recorded as a wide-angle image centered. Further, in a configuration in which audio data at the time of shooting is recorded, it is preferable that audio data be reproduced for each image. Note that display items, display formats, and the like are not limited to this example.
[0041]
In a configuration in which the gaze time can be recorded together with the image data, the gaze time may be displayed in addition to the image or the like as in the example shown in FIG.
[0042]
In a configuration in which image data is recorded in a server such as the information center 2, it is also possible to total the objects of the recorded image data on the server side, create a ranking or the like of a facility of interest, and publish the ranking. .
[0043]
The present invention is not limited to the above embodiments, and various changes or modifications can be made within the scope of technical matters described in the claims.
[0044]
Furthermore, in the above-described embodiment, only the driver's line of sight is targeted, but the line of sight of other occupants may be detected and image recording may be performed based on these.
[0045]
Furthermore, in the above embodiment, the visual image recording device of the present invention was incorporated into the route guidance device, but the present invention is not limited to this.
[0046]
【The invention's effect】
As described above, according to the present invention, there is provided a device or the like that can use the line of sight of the occupant of a vehicle for a purpose different from a conventional purpose such as driving the vehicle.
[Brief description of the drawings]
FIG. 1 is a basic configuration diagram showing one embodiment of a route guidance device of the present invention.
FIG. 2 is a view showing the vicinity of a driver's seat of a vehicle equipped with the route guidance device according to one embodiment of the present invention.
FIG. 3 is a flowchart showing the contents of processing executed when an image of a visual recognition target is recorded by the route guidance device according to the embodiment of the present invention.
FIG. 4 is an example of display of an image recorded by the visual image recording device according to the embodiment of the present invention.
FIG. 5 is another example of the display of an image recorded by the visual image recording device according to the embodiment of the present invention.
[Explanation of symbols]
1: Route guidance device (navigation device)
18: HDD storage device 22: DVD-ROM
38: eye camera 40: visual image recording device 48: visual target recording switch

Claims (10)

Gaze detection means for detecting the gaze of a vehicle occupant;
Based on the detected line of sight, a visual object specifying unit that specifies a visual target that is being visually recognized by an occupant of the vehicle,
Image data storage means for causing a camera to shoot an image of the viewing target object and storing the image data.
An information recording device, characterized in that: A visual object record switch operated by the occupant to activate the visual object specifying means and the image data storage means,
The information recording device according to claim 1. When the occupant visually recognizes the target object for a predetermined time or more, the visual object specifying device and the image data storage device are operated.
The information recording device according to claim 1. The image data storage unit confirms with the occupant whether or not the image data needs to be stored;
The information recording device according to claim 3. Changing the focal length of the camera lens according to the distance to the viewing target,
The information recording device according to claim 1. The image data storage means stores audio information together with the image data.
The information recording device according to claim 1. The image data storage means, together with the image data, stores time data at the time of shooting, and allows the image data to be taken out in chronological order,
The information recording device according to claim 1. The image data further includes an information search unit that searches for information on a visual recognition target included in the image,
The image data storage means, together with the image data, to store information on the visual recognition target included in the searched image,
The information recording device according to claim 1. Detecting a line of sight of a vehicle occupant;
Based on the detected line of sight, a step of identifying a visual recognition target that the occupant of the vehicle is visually recognizing,
Causing the camera to capture image data of the visual recognition target, and storing the image data.
An information recording method, characterized in that: Detects the line of sight of the occupants of the vehicle,
On the basis of the detected line of sight, a visual recognition target that the occupant of the vehicle is visually recognizing is specified,
Causing a camera to shoot image data of the visual recognition target object, and operating a computer to store the image data;
An information recording program, characterized in that:

Images