JP2007172378A - Apparatus for specifying object to be gazed at - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for specifying an object to be gazed at that is capable of precisely specifying any outdoor object to be gazed at, which an occupant on a moving body has gazed at. <P>SOLUTION: A camera 1 images an outdoor field including an occupant's viewing field to create images Pc. When a line-of-sight sensor detects the direction of the occupant's line of sight, a line-of-sight behavior determining part determines if the occupant's line-of-sight behavior is in the state of gazing according to changes with time in the occupant's viewing field images Pg created every predetermined time. If the line-of-sight behavior is in the state of gazing, a character recognizing part recognizes character information T on the viewing field images Pg included in the outdoor object O to be gazed at which the occupant has gazed at. Based on the character information detected by the character recognizing part and information on the current position of the moving body detected by a GPS sensor, an object determining part refers to attribute information on outdoor objects including character information and position information previously stored in an object information storage device, and determines the outdoor object O to be gazed at which the occupant gazed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a gaze target object identification device that identifies an outdoor gaze target object gazed by a passenger of a moving body.

2. Description of the Related Art Conventionally, an apparatus that estimates a target object that is of interest to a passenger of a moving body such as an automobile is generally known. As an apparatus of this type, for example, in the interest level estimation apparatus described in Patent Document 1, reference is made to prestored map information and outdoor object position information from the current position of the moving body obtained by the self-position detecting means. Then, based on the spatial arrangement relationship between the position information and the sight line direction of the occupant obtained by the sight line detection means and its duration, the degree of interest of the occupant for the outdoor object existing around the current position is calculated. ing.
JP 2005-6897 A

  By the way, in the interest level estimation device described in Patent Document 1, in order to compare the spatial overlap between the line of sight of the passenger whose movement range is smaller than that of the moving object and the outdoor object in the line of sight, It is necessary to detect the position of the moving body and the position of the outdoor object with high accuracy. However, since a moving body such as an automobile normally moves at a high speed, position information with sufficient accuracy cannot be obtained by GPS (Global Positioning System) which is a self-position detecting means.

  That is, in the prior art as described in Patent Document 1, since the consistency of the position information of the moving body with respect to the map information stored in advance and the position information of the outdoor object is insufficient, There is a problem that it is difficult to accurately specify an outdoor gaze target object.

  Accordingly, an object of the present invention is to provide a gaze target object identification device that can accurately identify an outdoor gaze target object that a passenger of a mobile object gazes at.

  The gaze target object specifying device according to the present invention is an apparatus for specifying an outdoor gaze target object that is watched by a passenger of a moving body, and is an outdoor photographed image including a passenger's field of view imaged by an outdoor photographing means. The visual line of the passenger's line-of-sight direction detected by the line-of-sight detection means is sequentially generated every predetermined time, and the line of sight for determining the gaze state of the passenger's line-of-sight behavior from the temporal change of the visual field image A behavior determination unit, a character recognition unit that detects and recognizes character information on a visual field image included in an outdoor gaze target object that the passenger gazes at, and character information and position detection means detected by the character recognition unit By referring to the attribute information of the outdoor object including character information and position information stored in advance in the target information storage device based on the information on the current position of the moving object detected by the Characterized in that it includes a visual outdoor gaze target object determining object determination unit.

  In the gaze target object identifying device according to the present invention, when the outdoor photographing unit photographs the outdoor in front of the moving body including the occupant's visual field area, and the sight line detecting unit detects the sight line direction of the occupant, Sequentially generates a field-of-view image in the sight line direction of the passenger at predetermined time intervals, and determines a gaze state of the sight line behavior of the passenger from the temporal change of the field image. Here, if the sight line behavior of the occupant is in the gaze state, the character recognition unit detects and recognizes character information on the visual field image included in the outdoor gazing target object gazed by the occupant. Then, the target determination unit obtains character information and position information stored in advance in the target information storage device based on the character information detected by the character recognition unit and the current position information of the moving object detected by the position detection unit. By referring to the attribute information of the included outdoor object, the outdoor gaze target object watched by the passenger is determined. As a result, the outdoor gaze target object that the occupant of the mobile object gazes at is accurately identified.

  In the gaze target object specifying device of the present invention, when interest level determination means for determining the degree of interest of the occupant with respect to the gaze target object is provided, the degree of interest of the occupant with respect to the outdoor gaze target object gazed by the passenger is determined. This is preferable because it can be determined.

  Here, the degree of interest determination means determines the degree of interest of the occupant based on the gazing duration of the occupant with respect to the gaze target object determined by the target determination unit and the speed of the moving body detected by the speed sensor. It can be constituted as follows. Also, the interest level determination means can be configured to determine the interest level of the passenger based on the frequency of the reference request operation for the attribute information of the outdoor object stored in the target information storage device.

  Also, in the gaze target object specifying device of the present invention provided with the interest level determination means, the interest level for each gaze target object received from the interest level determination means via the communication device is stored in the target evaluation information storage device. It is possible to provide an advertisement effect determination unit that collates with evaluation information, evaluates the collation result as an advertisement effect, and stores the result in the target evaluation information storage device. In this case, it is possible to evaluate the advertising effect of the outdoor gaze target object that the passenger gazes at.

  According to the gaze target object specifying device according to the present invention, when the outdoor photographing unit photographs the outdoor in front of the moving object including the passenger's visual field area, and the visual line detecting unit detects the passenger's visual line direction, The determination unit sequentially generates a passenger's visual field image at predetermined time intervals, and determines a gaze state of the passenger's gaze behavior from the temporal change of the visual field image. Here, if the sight line behavior of the occupant is in the gaze state, the character recognition unit detects and recognizes character information on the visual field image included in the outdoor gazing target object gazed by the occupant. Then, the target determination unit includes character information and position information stored in advance in the target information storage device based on the character information detected by the character recognition unit and the current position information of the moving object detected by the position detection unit. By referring to the attribute information of the outdoor object, an outdoor gaze target object that is watched by the passenger is determined. Therefore, it is possible to accurately specify the outdoor target object that the passenger of the moving object has observed.

  Hereinafter, an embodiment of a gaze target object specifying device according to the present invention will be described with reference to the drawings. In this description, the same or similar components are denoted by the same reference numerals, and redundant description may be omitted. Here, in the drawings to be referred to, FIG. 1 is a block diagram showing the configuration of the gaze target object specifying device according to the first embodiment, and FIG. 2 is a field of view of the occupant's field of view taken outdoors by the camera shown in FIG. It is a schematic diagram explaining the process which carries out projective conversion to an image.

  The outdoor object specifying device according to the first embodiment is a device for specifying an object to be watched such as an outdoor advertisement watched by a passenger of an automobile as the own vehicle, for example, as shown in FIG. A gaze sensor 2 as a gaze detection means, a gaze behavior determination section 3, a character recognition section 4, a target determination section 5, a target information storage device 6, and a GPS (Global Positioning System) sensor 7 as a position detection means. I have.

  Here, the line-of-sight sensor 2, the line-of-sight behavior determining unit 3, the character recognizing unit 4, the target determining unit 5, and the target information storage device 6 are microcomputer hardware and software such as an in-vehicle ECU (Electric Control Unit) not shown. This ECU is composed of an input / output interface I / O, an A / D converter, a ROM (Read Only Memory) that stores programs and data, and a RAM (Random Access Memory) that temporarily stores input data and the like. A CPU (Central Processing Unit) for executing the program is provided.

  The camera 1 is composed of an image photographing device composed of one or more CCD cameras and infrared cameras, and is installed near the upper part of the front window of the host vehicle (not shown) so as to include the passenger's field of view. Take an outdoor image in front of the vehicle. In this camera 1, the three-dimensional positions (see FIG. 2) in the X, Y, and Z directions with respect to the reference coordinate system are calibrated in advance, and the three-dimensional positions are fixed so as not to change.

  Such a camera 1 includes all of the captured image data and camera parameters (camera focal length, lens distortion coefficient, image center, rotation matrix for determining the position and direction of the camera in space, translation vector, etc.). Is output to the gaze behavior determination unit 3 together with the measurement time information.

  The line-of-sight sensor 2 includes an image capturing device including one or more CCD cameras and infrared cameras, and one arithmetic device that processes image data captured by the image capturing device. The image capturing device that constitutes the line-of-sight sensor 2 is installed at an appropriate position in the passenger compartment of the host vehicle so that the vicinity of the head including the face of the passenger can be captured. In this image capturing apparatus, the three-dimensional positions in the X, Y, and Z directions (see FIG. 2) with respect to the reference coordinate system are calibrated in advance, and the three-dimensional positions are fixed so as not to change.

  The arithmetic unit that constitutes the line-of-sight sensor 2 inputs image data of the vicinity of the passenger's head captured by the image capturing device and performs image processing, whereby the center position of the passenger's head and the center positions of the left and right eyeballs are processed. And a direction vector from the center position of the head to the center position of the left and right eyeballs is calculated as the sight line direction of the passenger. Then, all the calculated line-of-sight data is repeatedly output to the line-of-sight behavior determination unit 3 together with the measurement time information at predetermined time intervals.

  The line-of-sight behavior determination unit 3 processes a field-of-view image of the passenger's line-of-sight detected by the line-of-sight sensor 2 every predetermined time by processing an outdoor captured image including the field of view of the passenger captured by the camera 1. It produces | generates sequentially and determines the gaze state of a passenger | crew's gaze behavior from the change of the visual field image. Hereinafter, the procedure of the gaze behavior determination process by the gaze behavior determination unit 3 will be described with reference to the flowchart shown in FIG.

  First, in step S10, visual field image data is generated from line-of-sight data and imaging data. In step S10, the line-of-sight axis Ag, which is the line-of-sight direction of the occupant shown in FIG. 2, is calculated from the line-of-sight data based on the line-of-sight data and the photographing data determined to synchronize the measurement time information, and from the photographing data, FIG. The camera axis Ac, which is the shooting direction of the camera 1 shown in FIG. Based on the calculated information about the camera axis Ac and the line-of-sight axis Ag, a matrix for projectively transforming the camera axis Ac to the line-of-sight axis Ag is obtained, and the data of the photographed image Pc photographed by the camera 1 is projectively transformed. The visual field images Pg in the direction of the visual axis Ag are sequentially generated every predetermined time. On the visual field image Pg, the shape of the outdoor object to be watched O on the photographed image Pc is converted into the shape seen by the passenger's line of sight.

  Here, the synchronization determination of the measurement time information is based on the measurement time of the line-of-sight data, and the imaging data at the measurement time closest to the measurement time of the line-of-sight data is selected. When the camera 1 is composed of an image photographing device including two or more CCD cameras or infrared cameras, the photographing data of the image photographing device having the camera axis Ac with the smallest inner product with the visual axis Ag is selected. .

  In the next step S11, a difference dI between the coordinate value of the feature point m on the visual field image data at the measurement time t and the feature point m on the visual field image data at the comparative measurement time tcmp is calculated. In step S11, first, a feature point m whose three-dimensional position in the reference coordinate system is known and unchanged is set in advance for the data of the captured image Pc obtained by the camera 1. Then, the coordinate value of the feature point m on the captured image data is (u, v), and the coordinate value (u ′, v ′) of the feature point m on the field-of-view image data is calculated by projective transformation. Subsequently, the coordinate value of the feature point m on the visual field image data I (tcmp) at the measurement time tcmp for comparison and the coordinate value of the feature point m on the visual field image data I (t) at the measurement time t are obtained. Is calculated as an image difference dI.

  In the subsequent step S12, the driver's eye movement speed (line-of-sight movement speed) v is calculated based on the time difference dt between the measurement time t and the comparison measurement time tcmp and the calculated image difference dI.

  In the next step S13, the calculated eye movement speed v is compared with a preset threshold value thdv of the eye movement speed to determine whether or not the eye movement speed v is within the threshold thdv. If the determination result is YES and the eye movement speed v is within the threshold thdv, the time difference dt is added to the time flag tflag in the subsequent step S14, and then the process proceeds to step S15.

  On the other hand, if the determination result in step S13 is NO and the eye movement speed v exceeds the threshold thdv, the time flag tflag is reset to tflag = 0 in step S16, and then the process proceeds to step S15.

  In step S15, the time flag tflag is compared with a preset time flag threshold thdtflag to determine whether or not the time flag tflag is greater than or equal to the threshold thdtflag. If the determination result is YES and the time flag tflag is greater than or equal to the threshold thdtflag, the gaze behavior at the measurement time t is regarded as a gaze state, and Gact (t) indicating the gaze behavior in the next step S17 is changed to Gact (t ) = 1, and Tdur (t) indicating the duration of the gaze state of the gaze behavior until the measurement time t is set to Tdur (t) = tflag, and then the process proceeds to step S18.

  On the other hand, if the determination result in step S15 is NO and the time flag tflag is less than the threshold thdtflag, the gaze behavior at the measurement time t is regarded as a moving state, and Gact (t) indicating the gaze behavior in step S19 is Gact. (T) = 0, and Tdur (t) indicating the duration of the gaze state of the gaze behavior is set to Tdur (t) = 0, and then the process proceeds to step S18.

  In step S18, the values of measurement time t, line-of-sight behavior Gact (t), and duration Tdur (t) are output to the character recognition unit 4 shown in FIG. 1, and the line-of-sight behavior determination process by the line-of-sight behavior determination unit 3 is terminated. .

  The character recognition unit 4 shown in FIG. 1 is a gaze target object O such as an outdoor advertisement watched by the passenger if the gaze behavior of the passenger determined by the gaze behavior determination unit 3 is a gaze state (see FIG. 2). 2 detects and recognizes character information T such as a telephone number and URL contained in the field of view image Pg in FIG. Hereinafter, the procedure of the character recognition process by the character recognition part 4 is demonstrated along the flowchart shown in FIG.

  First, in step S20, whether Gact (t) indicating the line-of-sight behavior is Gact (t) = 1, and whether Tdur (t) indicating the duration of the gaze state of the line-of-sight behavior is Tdur (t)> 0. Determine whether or not. If this determination result is NO, the determination in step S20 is repeated. However, if the determination result is YES and the gaze behavior of the passenger's line-of-sight behavior is maintained, the visual field image at the measurement time t in the subsequent step S21. A central portion on the data I (t) is determined as a gaze area for the passenger.

  In the next step S22, image processing such as edge processing is performed on the central portion on the field-of-view image data I (t), which is the gazing area of the passenger, and the edge determined as character information T such as a telephone number or URL An area including a character is extracted as a character area. Thereafter, in step S23, it is determined whether or not there is an edge determined to be a character part such as a telephone number or URL in the character area.

  If the determination result in step S23 is YES and an edge that is determined to be a character part such as a telephone number or URL exists in the character area, template matching is performed on the extracted character area in the subsequent step S24. By performing this pattern identification processing, a suitable character string such as a telephone number or URL is determined as a character string Char (t) at the measurement time t.

  In the next step S25, it is determined whether or not there is a matching character string Char (t) such as a telephone number or a URL. If the determination result is YES, in the following step S26, the matching character string Char (t ) Is output to the object determination unit 5 shown in FIG. 1, and then the character recognition processing by the character recognition unit 4 is terminated.

  On the other hand, if the determination result in step S23 is NO or the determination result in step S25 is NO, it is assumed that there is no matching character string Char (t), and the character string Char (t) is converted to Char (t). After setting = NULL, the character recognition process by the character recognition unit 4 is terminated.

  The target determination unit 5 shown in FIG. 1 includes outdoor object attribute information (ID, name, installation position, telephone number, URL, etc.) stored in advance in the target information storage device 6 and the own vehicle. And the information on the current position of the host vehicle measured by the GPS sensor 7 installed at the appropriate position.

  The target determination unit 5 is based on information on a character string Char (t) such as a telephone number or URL input from the character recognition unit 4 and information on the current position of the host vehicle input from the GPS sensor 7. By referring to the attribute information (ID, name, installation position, telephone number, URL, etc.) of the outdoor object input from the information storage device 6, the object to be watched O such as an outdoor advertisement watched by the passenger is specified. Hereinafter, the procedure of the object determination process by the object determination unit 5 will be described with reference to the flowchart shown in FIG.

  First, in step S30, the installation position of the outdoor object input from the target information storage device 6 is compared with the current position of the host vehicle input from the GPS sensor 7, and the threshold between which the distance is set in advance is compared. In the following case, a predetermined outdoor object existing in the vicinity of the current position of the host vehicle is extracted as a specific outdoor object.

  In the next step S31, it is determined whether or not a specific outdoor object exists in the vicinity of the current position of the host vehicle. If the determination result is YES and the specific outdoor object is extracted, the following step is performed. In S32, the telephone number and URL associated with the extracted specific outdoor object are compared with the telephone number and URL character string Char (t) input from the character recognition unit 4.

  In the next step S33, it is determined whether or not there is a specific outdoor object having a matching telephone number or URL. If the determination result is YES, in the subsequent step S34, a specific outdoor object with a matching telephone number or URL is specified as the gaze target object Oatn (t) at the measurement time t, and then the target by the target determination unit 5 The determination process ends.

  On the other hand, when the determination result of step S31 is NO or when the determination result of step S33 is NO, it is considered that the gaze target object Oatn (t) does not exist at the measurement time t, and Oatn (t) = at step S35. After setting to NULL, the target determination process by the target determination unit 5 is terminated.

  As described above, in the gaze target object specifying device according to the first embodiment of the present invention, the camera 1 captures the outdoors in front of the host vehicle including the field of view of the passenger and the captured image Pc (see FIG. 2). And the visual line sensor 2 calculates the visual axis Ag that is the direction of the occupant's visual line, the visual line behavior determination unit 3 performs projective conversion on the data of the captured image Pc, so that the visual field in the direction of the visual axis Ag of the occupant Images Pg (see FIG. 2) are sequentially generated every predetermined time. Then, the gaze state of the sight line behavior of the passenger is determined based on the temporal change of the sequentially generated visual field images Pg.

  Here, if the sight line behavior of the occupant is in the gazing state, the character recognition unit 4 uses the character information T such as the telephone number or URL on the field-of-view image Pg included in the outdoor gazing target object O gazed by the occupant. Is recognized as a character string Char (t). Then, the target determination unit 5 stores in advance in the target information storage device 6 based on the character information T such as the telephone number and URL detected by the character recognition unit 4 and the current position information of the host vehicle detected by the GPS sensor 7. By referring to the stored attribute information (ID, name, installation position, telephone number, URL, etc.) of the outdoor object, the outdoor target object O gazeed by the passenger at the measurement time t t).

  Therefore, according to the gaze target object specifying device according to the first embodiment, it is possible to accurately specify the specific outdoor object watched by the passenger of the host vehicle as the gaze target object Oatn at the measurement time t. As a result, it becomes possible to investigate which outdoor advertisement the passenger of the own vehicle paid attention to various outdoor advertisements such as signs installed outdoors.

  Next, a gaze target object specifying device according to the second embodiment of the present invention will be described with reference to FIG. The gaze target object specifying device of the second embodiment is obtained by adding a vehicle speed sensor 8 and an interest level determination unit 9 as an interest level determination means to the configuration of the gaze target object specification device of the first embodiment shown in FIG. Since the other components other than these are the same as those of the gaze target object identifying device of the first embodiment, description thereof is omitted by attaching the same reference numerals.

  Here, the interest level determination unit 9 receives information on a specific gaze target object Oatn (t) that the passenger gazes at the measurement time t from the target determination unit 5 and also the vehicle of the host vehicle at the measurement time t. Information on the speed vel (t) is input from the vehicle speed sensor 8.

  The interest level determination unit 9 uses the gazing duration of the passenger with respect to the gaze target object Oatn (t) input from the target determination unit 5 and the vehicle speed vel (t) of the host vehicle input from the vehicle speed sensor 8. Based on this, the degree of interest of the passenger with respect to the gaze target object Oatn (t) is determined. Hereinafter, the procedure of the interest level determination process by the interest level determination unit 9 will be described with reference to the flowchart shown in FIG.

  First, in step S40, it is determined whether or not the gaze target object Oatn (t) at the measurement time t is Oatn (t) ≠ NULL. If this determination result is YES and there is a gaze target object Oatn (t), it is considered that the passenger is gaze at the gaze target object O at the measurement time t, and the process proceeds to the next step S41.

  In step S41, the gaze target object Oatn (t) at the measurement time t is compared with the gaze target object Oatn (tprv) at the time tprv immediately before the measurement time t, and whether or not Oatn (t) = Oatn (tprv). Determine whether. If this determination result is YES, it is considered that the passenger is gazing at the outdoor target object O, and the process proceeds to the next step S42.

  On the other hand, when the determination result of step S40 is NO or the determination result of step S41 is NO, it is considered that the passenger is not gazing at the outdoor target object O at the measurement time t. After resetting the gazing time Tatn (t) of the passenger with respect to the gaze target object O to Tatn (t) NULL, the interest level determination process by the interest level determination unit 9 ends.

  In step S42, the passenger's gaze time Tatn (t) is updated as Tatn (t) = Tatn (tprv) + (t−tprv). In subsequent step S44, it is determined whether or not the gaze time Tatn (t) is Tatn (t)> 0.

  Here, if the determination result of step S44 is NO, the interest level determination process by the interest level determination unit 9 is terminated, but if the determination result is YES, the passenger continues to watch the outdoor object of interest O. The process proceeds to the next step S45.

  In step S45, based on the vehicle speed vel (t) of the host vehicle input from the vehicle speed sensor 8, a coefficient α (t) representing the passing time of the host vehicle passing near the outdoor object of interest O is calculated. . Here, the coefficient α (t) is calculated based on a preset classification criterion. For example, when the vehicle speed is lower than a predetermined speed, the outdoor gaze target object O is the passenger's Since the time for entering the field of view is considered to be long, the coefficient α (t) is set in the range of α (t) <1. On the contrary, when the vehicle speed is higher than the predetermined speed, it is considered that the time during which the outdoor gazing target object O enters the field of view of the passenger is shortened. Set. When the vehicle speed is in the normal speed range, the coefficient α (t) is set to α (t) = 1.

  In the next step S46, an interest time TInt (Oatn (t)), which is a time during which the passenger is interested in the gaze target object Oatn (t) at the measurement time t, is calculated. The interest time TInt (Oatn (t)) is calculated by using the gaze time Tatn (t) and the coefficient α (t), “TInt (Oatn (t)) = Tatn (tprv) + α (t) * (t− tprv) ".

  In the subsequent step S47, the time immediately before the gazing state of the passenger with respect to the gaze target object Oatn (t), that is, the gaze time Tatn (tnxt) at the measurement time tnxt following the measurement time t is Tatn (tnxt) = 0. It is determined whether or not.

  If this determination result is NO, the process returns to step S40, but if the determination result is YES, the interest time TInt (Oatn (t)) is maximized, so that value is boarded on the gaze target object Oatn (t). It is determined as the degree of interest of the person. Then, the gaze target object Oatn (t) is recorded as the interest target object Oref (t), and the interest time TInt (Oatn (t)) is recorded as the interest level Int (Oref). Thereafter, the interest level determination unit 9 The interest level determination process is terminated.

  As described above, according to the gaze target object specifying device according to the second embodiment of the present invention, the gazing duration of the occupant with respect to the gaze target object Oatn (t) input from the target determination unit 5 and the vehicle speed sensor 8, the interest level determination process of the interest level determination unit 9 based on the vehicle speed vel (t) of the host vehicle input from 8 is used to convert the gaze target object Oatn (t) that the passenger is gazing into the interest target object Oref (t ) As the interest level Int (Oref), and the interest time TInt (Oatn (t)) when the passenger is interested in the gaze target object Oatn (t). 9 can be recorded.

  The information on the object of interest Oref (t) and the interest level Int (Oref) recorded in the interest level determination unit 9 is used as an index for measuring the advertising effect of various outdoor advertisements such as signs installed outdoors. The information can be used as information when determining, for example, the installation usage amount of an outdoor advertisement.

  In addition, when the degree of interest Int (Oref) for the object of interest Oref (t) recorded in the degree-of-interest determination unit 9 is extremely high, content that induces the driver to look aside is included. It can also be determined that the advertisement is an outdoor advertisement, and can be used as an appropriate determination index of the advertisement content.

  Next, a gaze target object specifying device according to the third embodiment of the present invention will be described with reference to FIG. The gaze target object specifying device according to the third embodiment includes a reference operation detection unit 10, an interest level determination unit 11 as an interest level determination unit, and a display in the configuration of the gaze target object specification device according to the first embodiment shown in FIG. 12 are added, and the other components other than these are the same as those of the gaze target object identifying device of the first embodiment, and thus the same reference numerals are given and description thereof is omitted.

  The interest determination unit 11 is configured in substantially the same manner as the interest level determination unit 9 illustrated in FIG. 6, and is an interest target object Oref that is an outdoor gaze target object O in which the passenger of the own vehicle is interested, and its interest. Int (Oref) information is recorded.

  The reference operation detection unit 10 is configured to include any one of a touch panel, a remote controller, a microphone, and the like, and items displayed on the display 12, that is, attribute information of outdoor objects stored in the target information storage device 6 ( A passenger's reference request operation for ID, name, installation position, telephone number, URL, etc.) is detected, and the operation information is output to the interest determination unit 11.

  The display 12 displays the information of the object of interest Oref recorded in the interest determination unit 11 and reflects the passenger's operation information detected by the reference operation detection means 10 and then the interest requested by the passenger. Detailed information for the target object Oref is displayed.

  Here, the degree-of-interest determination unit 11 determines the degree of interest of the passenger with respect to the gaze target object Oatn (t) based on the frequency of the reference request operation input from the reference operation detection unit 10. Hereinafter, the procedure of the interest level determination process by the interest level determination unit 11 will be described with reference to the flowchart shown in FIG.

  First, in step S50, information related to the gaze target object Oatn (t) input from the target determination unit 5 is accumulated. In next step S51, information related to the gaze target object Oatn (t) is displayed on the display 12.

  In a succeeding step S52, it is determined whether or not a reference request operation from the reference operation detection unit 10 for the gaze target object Oatn (t) displayed on the display 12 is detected. If this determination result is NO, the determination in step S52 is repeated, but if the determination result is YES and a reference request operation from the reference operation detection means 10 is detected, the process proceeds to the subsequent step S53.

  In step S53, the outdoor object attribute information (ID, name, installation position, telephone number, URL, etc.) stored in the target information storage device 6 is referred to. In the next step SS54, detailed information regarding the referenced object of interest (reference object) Oref is acquired.

  In a succeeding step S55, detailed information on the object of interest (reference object) Oref acquired in step SS54 is displayed on the display 12. Then, in the next step S56, the interest level Int (Oref) is incremented and updated on the assumption that the passenger is interested in the object of interest (reference object) Oref. The interest determination process ends.

  As described above, according to the gaze target object specifying device according to the third embodiment of the present invention, the attributes of the outdoor object stored in the target information storage device 6 through a series of interest determination processes by the interest determination unit 11. The reference operation detection means 10 can detect a passenger's reference request operation for information (ID, name, installation position, telephone number, URL, etc.), and is input from the reference operation detection means 10 to the interest determination unit 11. Based on the frequency of the reference request operation, the degree of interest of the passenger with respect to the gaze target object Oatn (t) can be determined.

  Next, a gaze target object specifying device according to the fourth embodiment of the present invention will be described with reference to FIG. The gaze target object specifying device of the fourth embodiment includes communication devices 13 and 14, a target evaluation information storage device 15, and an advertisement effect determining unit 16 in the configuration of the gaze target object specifying device of the second embodiment shown in FIG. 6. The other components other than those added are the same as those of the gaze target object identifying device according to the second embodiment, and thus the same reference numerals are given and description thereof is omitted.

  In the gaze target object identifying device of the fourth embodiment, the interest levels of a plurality of passengers with respect to a specific outdoor object that is a gaze target for the passenger are totalized, and the interest levels totaled within a certain period are accumulated so far. The evaluation information for the specific outdoor object is collated, and the evaluation information taking into account the degree of interest collected within a certain period is updated according to the collation result. This evaluation information is evaluated as an advertising effect, and can be used to determine a price to be paid to a provider of a specific outdoor object installation location.

  Here, the communication device 13 inputs information regarding the interest level Int (Oref) to the object of interest Oref recorded in the interest level determination unit 9 as the interest level of a plurality of passengers with respect to a specific outdoor object, and performs communication. Transmit to device 14. The communication device 14 outputs information related to the degree of interest Int (Oref) received from the communication device 13 to the advertising effectiveness determination unit 16.

  The advertisement effect determination unit 16 refers to the evaluation information stored in the target evaluation information storage device 15 that has been accumulated so far as evaluation information for a specific outdoor object, and then includes evaluation information that takes into account the degree of interest that is aggregated within a certain period of time. Update. Hereinafter, the procedure of the advertisement effect determination process by the advertisement effect determination unit 16 will be described with reference to the flowchart shown in FIG.

  First, in step S60, the degree of interest Int (Oref) related to the object of interest Oref that the passenger has shown an interest in is input from the communication device 14. In the next step S61, the interest level Int (Oref) input from the communication device 14 is added to the sum Sum (Oref) of the interest level Int (Oref) input and recorded so far, and the interest level Int ( The sum Sum (Oref) of Oref) is updated.

  In a succeeding step SS62, it is determined whether or not a predetermined setting period for counting the interest level Int (Oref) has ended. If this determination result is NO, the processing from step S60 to step SS62 is repeated, but if the determination result is YES and the set period has ended, the processing proceeds to the subsequent step S63.

  In step S63, the sum Sum (Oref) of the updated degree of interest Int (Oref) is determined as the evaluation information Vcur within a fixed setting period, and the fixed setting period is determined as the evaluation period Pcur.

  In the next step S64, the previous evaluation information Vave stored in the target evaluation information storage device 15 and the evaluation period Psum are referred to, and the current evaluation information Vcur is compared with the stored evaluation information Vave. The current evaluation period Pcur is compared with the accumulated evaluation period Psum.

  In subsequent step S65, the evaluation information is updated to Vupd, and the evaluation period is updated to Pupd. Here, the evaluation information Vupd is based on the evaluation information Vcur within the set period, the accumulated evaluation information Vave, the accumulated evaluation period Psum, and the current evaluation period Pcur, “Vupd = (Vcur + Vave * Psum) / (Pcur + Psum) ”. The evaluation period Pupd is calculated by the equation “Pupd = Pcur + Psum”.

  In the next step S66, it is determined whether or not the evaluation information has been updated for all interest object (reference object) Oref for each object of interest Oref. If this determination result is NO, the processes from step S63 to step SS66 are repeated, but if the determination result is YES, the process proceeds to the subsequent step S67.

  In step S67, the updated evaluation information is regarded as the advertising effect of the object of interest Oref in which the plurality of passengers have shown interest, and is compared with the price conversion table stored in advance in the object evaluation information storage device 15, A consideration for the object Oref, that is, a payment consideration for the installation location provider of the object of interest Oref is determined.

  Then, in the next step S68, all the update results are recorded in the target evaluation information storage device 15, and thereafter the advertising effect determination process by the advertising effect determination unit 16 is ended.

    As described above, according to the gaze target object specifying device according to the fourth embodiment of the present invention, the series of the advertisement effect determination process by the advertisement effect determination unit 16 determines the interest object Oref that the passenger has shown interest in. Advertising effects can be evaluated as consideration.

  The gaze target object specifying device according to the present invention is not limited to the above-described embodiments, and may be, for example, a gaze target object specifying device having the configuration shown in FIG. This gaze target object specifying device includes communication devices 13 and 14 in the gaze target object specifying device of the fourth embodiment shown in FIG. 10, the target, in the configuration of the gaze target object specifying device of the third embodiment shown in FIG. 8. An evaluation information storage device 15 and an advertisement effect determination unit 16 are added. Also in the gaze target object specifying device having the configuration shown in FIG. 12, the same effects as those of the gaze target object specifying device of the fourth embodiment shown in FIG. 10 can be obtained.

It is a block diagram which shows the structure of the gaze target object specific device which concerns on 1st Embodiment of this invention. It is a schematic diagram explaining the process which carries out projective conversion of the outdoor picked-up image image | photographed with the camera shown in FIG. 1 to a passenger | crew's visual field image. It is a flowchart which shows the procedure of the gaze behavior determination process by the gaze behavior determination part shown in FIG. It is a flowchart which shows the procedure of the character recognition process by the character recognition part shown in FIG. It is a flowchart which shows the procedure of the object determination process by the object determination part shown in FIG. It is a block diagram which shows the structure of the gaze target object specific device which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the interest degree determination process by the interest degree determination part shown in FIG. It is a block diagram which shows the structure of the gaze target object specific device which concerns on 3rd Embodiment of this invention. It is a flowchart which shows the procedure of the interest degree determination process by the interest degree determination part shown in FIG. It is a block diagram which shows the structure of the gaze target object specific device which concerns on 4th Embodiment of this invention. It is a flowchart which shows the procedure of the advertisement effect determination process by the advertisement effect determination part shown in FIG. It is a block diagram which shows the structure of the gaze target object identification device which concerns on other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera, 2 ... Line-of-sight sensor, 3 ... Line-of-sight behavior determination part, 4 ... Character recognition part, 5 ... Object determination part, 6 ... Object information storage device, 7 ... GPS sensor, 8 ... Vehicle speed sensor, 9 ... Interest determination part DESCRIPTION OF SYMBOLS 10 ... Reference operation detection means, 11 ... Interest determination part, 12 ... Display, 13 ... Communication apparatus, 14 ... Communication apparatus, 15 ... Target evaluation information storage device, 16 ... Advertising effect determination part, O ... Outdoor gaze target object , Pg: Field image, Pc: Captured image, T: Character information, Ac: Camera axis, Ag: Line of sight.

Claims (4)

A device that identifies an outdoor object to be watched by a passenger of a moving object,
By processing an outdoor photographed image including the passenger's visual field imaged by the outdoor photographing means, a visual field image of the passenger's visual line direction detected by the visual line detecting means is sequentially generated every predetermined time, and the visual field A line-of-sight behavior determination unit that determines the gaze state of the passenger's line-of-sight behavior from temporal changes in the image;
A character recognition unit that detects and recognizes character information on the field-of-view image included in an outdoor gaze target object watched by a passenger;
Outdoor object attribute information including character information and position information stored in advance in the target information storage device based on the character information detected by the character recognition unit and the current position information of the moving object detected by the position detection means A target object identification device comprising: a target determination unit that determines an outdoor target object to be watched by a passenger by referring to.   Interest level determination for determining the degree of interest of the occupant with respect to the gaze target object based on the gaze duration of the occupant with respect to the gaze target object determined by the target determination unit and the speed of the moving body detected by the speed sensor The gaze target object specifying device according to claim 1, further comprising: means.   An interest level determination means is provided for determining an interest level of the passenger with respect to the gaze target object based on a frequency of a reference request operation for attribute information of the outdoor object stored in the target information storage device. The gaze target object specifying device according to claim 1. The degree of interest for each gaze target object received from the degree-of-interest determination means via the communication device is collated with the evaluation information stored in the object evaluation information storage device, and the comparison result is evaluated as an advertising effect to evaluate the object evaluation information. The gaze target object specifying device according to claim 2, further comprising an advertisement effect determination unit that accumulates in a storage device.

Images