JP2016081465A - Display controller, display control method, display control program and projector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display controller capable of allowing an occupant to comprehend the type and state of an object.SOLUTION: A display controller 103 includes: a determination section 105 that determines, when a recognition section 101 recognizes an object, a first position which is separated away a predetermined distance from a position determined by a determination section 102 and a second position which is a generally symmetrical position with respect to the first position determined by the first determination section, and which generally positioned on a straight line which connects the first position and the position determined by the determination section 102; and a control unit 106 that controls a display unit 104 to generate a predetermined image representing a presentation image at a first position when displayed on a display medium 130 for a period of predetermined time from a first timing, and controls the display unit 104 to generate a predetermined image which represents a presentation image at a second position when displayed on the display medium 130 for a period of predetermined time from a second timing before a predetermined time has elapsed from the first timing.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control device, a display control method, a display control program, and a projection device that control display of information provided to a passenger such as a vehicle.

  2. Description of the Related Art In recent years, a head-up display (HUD) device has been actively developed that projects an image onto a windshield of a vehicle so that a driver can visually recognize the image. In this apparatus, the visibility of an object can be improved by projecting an image so as to be superimposed on an object in the real world (for example, another vehicle, a pedestrian, a building, or the like).

  In such a head-up display device, a so-called double image problem occurs in the driver's field of view. Since the left and right eyes of a human are approximately 6 to 6.5 cm apart, when the driver gazes at a certain object, the images appearing on the retinas of the left and right eyes are displaced, that is, binocular parallax occurs. Therefore, an object that is being watched by a human being is visually recognized as a single image, but an object that is located farther or closer than the object that is being watched is two images, that is, a double image. Become visible. As a method for solving the problem of the double image, for example, there is a method disclosed in Patent Document 1 (hereinafter referred to as a conventional technique).

  In the prior art, for the purpose of suppressing the influence of the double image, it is superimposed on the position of the object detected by the camera or the like, and an image that does not give a sense of incongruity even when viewed as a double image is generated. To display. Specifically, by generating an image having a dot pattern or I mark shape and displaying the image in association with the object, the driver can detect the presence of the object and its presence even when a double image is generated. The position can be grasped.

Japanese Patent No. 4970379

  However, the above-described conventional technology supports the presence / absence of an object and the position of the object, and cannot support the understanding of the type and state of the object. Therefore, the conventional technology is insufficient as an image display for improving the recognition of the object.

  An object of the present invention is to provide a display control device, a display control method, a display control program, and a projection device that allow an occupant to grasp the type and state of an object.

  A display control apparatus according to an aspect of the present invention acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image; Determining a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light; 1 A display control device in a display system including a determination unit and a display unit that generates a predetermined image and displays the predetermined image on the display medium, wherein the predetermined object is recognized by the recognition unit In this case, the first position that is a predetermined distance away from the position determined by the first determination unit, and a position that substantially exists on a straight line connecting the position determined by the first determination unit and the first position. And determined by the first determining unit. A second determination unit that determines a second position that is substantially symmetrical to the first position with respect to the first position, and the second determination unit is displayed on the display medium for a predetermined time from the first timing. Sometimes the display unit is controlled to generate a predetermined image representing a predetermined presentation image at the first position, and the predetermined time from the second timing before the predetermined time elapses from the first timing. A control unit that controls the display unit so as to generate the predetermined image representing the predetermined presentation image at the second position when displayed on the display medium for a period of time.

  A display control method according to an aspect of the present invention acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image; Determination to determine a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light And a display control method including a display unit that generates a predetermined image and displays the predetermined image on the display medium, and when the predetermined object is recognized by the recognition unit, A first position that is a predetermined distance away from the position determined by the determination unit; and a line that connects the position determined by the determination unit and the first position; and 1 The position determined by the determination unit And determining a second position that is substantially symmetrical to the first position, and the predetermined position is set to the first position when displayed on the display medium for a predetermined time from the first timing. The display unit is controlled so as to generate a predetermined image representing a presentation image, and displayed on the display medium for a predetermined time from a second timing before the predetermined time elapses from the first timing. When this is done, the display unit is controlled so as to generate the predetermined image representing the predetermined presentation image at the second position.

  A display control program according to an aspect of the present invention acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image; Determination to determine a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light And a display control method for causing a computer in a display system to generate a predetermined image and displaying the predetermined image on the display medium. When the object is recognized, the object is substantially on a straight line connecting the first position separated from the position determined by the determination unit by a predetermined distance, and the position determined by the determination unit and the first position. A process of determining a second position that exists and is substantially symmetrical to the first position with respect to the position determined by the first determination unit; and a predetermined time from the first timing A process for controlling the display unit to generate a predetermined image representing a predetermined presentation image at the first position when displayed on the display medium, and the predetermined time from the first timing. The display unit is configured to generate the predetermined image representing the predetermined presentation image at the second position when displayed on the display medium for a predetermined time from a second timing before the passage. The process to control was executed.

  A projection apparatus according to an aspect of the present invention acquires a foreground image that is an image of a foreground of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image, A first position for determining a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through the display medium that transmits light. A display unit including a determination unit, and a display unit that generates a predetermined image and displays the predetermined image on the display medium; and when the predetermined object is recognized by the recognition unit, A first position that is a predetermined distance away from the position determined by the first determination unit, and substantially exists on a straight line connecting the position determined by the first determination unit and the first position; And the position determined by the first determination unit A second determination unit that determines a second position that is substantially symmetrical to the first position, and when displayed on the display medium for a predetermined time from a first timing, The display unit is controlled so as to generate a predetermined image representing a predetermined presentation image at a first position, and during the predetermined time from a second timing before the predetermined time elapses from the first timing. A control unit that controls the display unit so as to generate the predetermined image representing the predetermined presentation image at the second position when displayed on the display medium.

  According to the present invention, the occupant can grasp the type and state of the object.

The block diagram which shows the structural example of the display system which concerns on embodiment of this invention The figure which shows an example of the foreground which concerns on embodiment of this invention The figure which shows an example of the positional relationship of the target object which concerns on embodiment of this invention, a display medium, and a passenger | crew The figure which shows an example when displaying an image on a display medium The figure which shows an example when the image shown to FIG. 4A appears as a double image The figure which shows an example when an image is displayed on the display medium at the 1st timing which concerns on embodiment of this invention. The figure which shows an example when the image shown to FIG. 5A looks as a double image The figure which shows an example when an image is displayed on the display medium at the 2nd timing which concerns on embodiment of this invention. The figure which shows an example when the image shown to FIG. 6A looks as a double image The flowchart which shows the operation example of the display system which concerns on embodiment of this invention. The figure which shows an example when the image displayed at the 1st timing is seen as a double image, when correction | amendment processing is not performed when a passenger | crew's head inclines. The figure which shows an example when the image displayed at the 2nd timing is seen as a double image, when correction | amendment processing is not performed when a passenger | crew's head inclines. The figure which shows an example when the image displayed at the 1st timing looks like a double image when correction | amendment processing is performed when a passenger | crew's head inclines The figure which shows an example when the correction | amendment process is performed when the passenger | crew's head inclines and the image displayed at the 2nd timing looks as a double image The figure which shows an example of the change of the brightness | luminance of the image displayed at the 1st timing. The figure which shows an example of the change of the brightness | luminance of the image displayed at the 2nd timing. The figure which shows an example of the change of the brightness | luminance of the image displayed at the 1st timing. The figure which shows an example of the change of the brightness | luminance of the image displayed at the 2nd timing. The block diagram which shows the hardware structural example of the display system which concerns on embodiment of this invention, and a display control apparatus

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a configuration example of the display system 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration example of a display system 100 according to the present embodiment.

  The display system 100 is a system that is used in a moving body such as a vehicle and supports driving of the vehicle. The display system 100 may be an in-vehicle device or a device brought into a vehicle. Although the display system 100 is described as being applied to a vehicle, the moving body is not limited to the vehicle, and may be a ship, an aircraft, or the like. In this embodiment, an example in which the user is a vehicle occupant, particularly a vehicle driver, is described, but the present invention is not limited thereto. Furthermore, the display system 100 may be used in a wearable computer (for example, an HMD described later) that can be worn on the user's body.

  As shown in FIG. 1, a camera 110, a camera 120, and a display medium 130 are connected to the display system 100.

  The camera 110 is mounted inside or outside the vehicle and photographs a field of view in front of the driver (hereinafter referred to as foreground). An image photographed by the camera 110 (hereinafter referred to as a foreground image) is output to the recognition unit 101 described later. In the present embodiment, the camera 110 is used as means for sensing the foreground. However, the present invention is not limited to this. For example, a radar or the like may be used.

  The camera 120 is a camera called an inner camera or a driver monitoring camera, and is mounted inside the vehicle to photograph the driver's face. An image photographed by the camera 120 (hereinafter referred to as a face image) is output to the determination unit 102 described later.

  The display medium 130 is a medium that is used in a vehicle and displays an image (details will be described later) generated by the display unit 104 described later. In the present embodiment, the display medium 130 is a transmissive display medium that transmits light.

  Examples of the display medium 130 include a head up display (HUD), an LCD (Liquid Crystal Display), an HMD (Head-Mounted Display or Helmet-Mounted Display), an eyeglass-type display (Smart Glasses), and other dedicated displays. The display etc. are applied. Further, the HUD may be, for example, a windshield of a vehicle, a glass surface provided separately, a plastic surface (for example, a combiner), or the like. The windshield may be, for example, a windshield, a vehicle side glass, or a rear glass.

  Further, the HUD may be a transmissive display provided on or inside the windshield. Here, the transmissive display is, for example, a transmissive organic EL display or a transparent display using glass that emits light when irradiated with light of a specific wavelength. The driver can view the display on the transmissive display at the same time as viewing the background. Thus, the display medium 130 is a display medium that transmits light. In either case, the image is displayed on the display medium 130.

  In the example of FIG. 1, the display system 100 is configured not to include the display medium 130, but may be configured to include the display medium 130.

  In FIG. 1, the display system 100 includes a recognition unit 101, a determination unit 102, a display control device 103, and a display unit 104. In addition, the display control apparatus 103 includes a determination unit 105 and a control unit 106. Hereinafter, each part will be described.

  The recognition unit 101 extracts a feature amount included in the foreground image acquired from the camera 110, and recognizes an object based on the extracted feature amount. Further, when the recognition unit 101 recognizes an object, the recognition unit 101 calculates the position of the object. The target object is, for example, a white line on a road, a sign, a road marking, a curb, a guardrail, a traffic light, a utility pole, a vehicle, a person, a building, and the like. Note that the object recognition process and the object position calculation process are well known, and thus detailed description thereof is omitted.

  Further, when the recognition unit 101 recognizes a predetermined object, the recognition unit 101 detects the contour of the object. For example, the recognition unit 101 detects the contour of the target object by performing edge extraction using a specific differential filter on the foreground image. In addition, since the detection process of the edge of a target object is well-known, detailed description is abbreviate | omitted.

  Then, the recognition unit 101 outputs information indicating the position of the object (hereinafter referred to as object position information) to the determination unit 102, and displays information indicating the outline of the object (hereinafter referred to as contour information) as the display control device 103. To the control unit 106.

  Here, an example of the driver's foreground is shown in FIG. As shown in FIG. 2, when there is a pedestrian 200 trying to cross the road on which the vehicle is traveling in front of the vehicle, the driver can visually recognize the pedestrian 200 via the display medium 130. In this case, the recognition unit 101 recognizes the pedestrian 200 and calculates the position of the pedestrian 200. The recognition unit 101 detects the contour of the pedestrian 200. Then, the recognizing unit 101 outputs object position information indicating the position of the pedestrian 200 to the determining unit 102, and outputs contour information indicating the outline of the pedestrian 200 to the display control device 103.

  The determination unit 102 detects a driver's eye area (hereinafter referred to as an eye area) based on the face image acquired from the camera 120. The eye area is a predetermined area including both eyes, for example. In addition, since the process which detects an eye area | region by performing image processing with respect to a face image is well-known, detailed description is abbreviate | omitted. In the present embodiment, the determination unit 102 detects the eye area based on the face image. However, the eye area may be a predetermined area, or a line of sight that detects the driver's line of sight, for example. It may be an area detected based on the line of sight detected by a detection unit (not shown).

  Then, the determination unit 102 determines the position of a predetermined point (for example, the left or right eye or the center between the left eye and the right eye) in the detected eye region, and the object position information acquired from the recognition unit 101. The position on the display medium 130 where the image generated by the display unit 104 is displayed (hereinafter referred to as an image display position) is determined from the relationship between the position of the object indicated by In other words, this image display position is the position of the object on the display medium 130 when the object is observed through the display medium 130 from a predetermined point in the eye area. Since the image display position determination process is publicly known, detailed description thereof is omitted.

  An example of the image display position determination process will be described with reference to FIG. FIG. 3 is a diagram schematically showing the positional relationship between the pedestrian 200, the display medium 130, and the driver's eyes when the driver gazes at the pedestrian 200 via the display medium 130. In FIG. 3, the broken line indicates the line of sight of the driver who is gazing at the pedestrian 200.

  In FIG. 3, the imaginary line L1 is a horizontal line passing through the right eye position P1 and the left eye position P2 of the driver. The virtual line L2 is a line that passes through the position P3 of a predetermined point in the driver's eye area and the position P4 of the pedestrian 200 and is perpendicular to the virtual line L1. The position P3 is, for example, a central position between the right eye position P1 and the left eye position P2. Note that the virtual lines L1 and L2 are shown for explanation, and are not displayed on the display medium 130.

  In FIG. 3, for example, the determination unit 102 determines the position P5 where the virtual line L2 passing through the position P3 and the position P4 intersects the display medium 130 as the image display position. Conventionally, an image is displayed at the image display position determined in this manner. In the present embodiment, the first position and the second position are determined by shifting the image display position determined here by a predetermined distance. The determination unit 105 (to be described later) performs a process for determining the position. Details of the determination processing of the first position and the second position will be described later.

  Then, the determination unit 102 outputs information indicating the image display position (hereinafter referred to as image display position information) to the determination unit 105 of the display control device 103. In addition to the image display position information, the determination unit 102 also includes information on the object recognized by the recognition unit 101 (for example, the object position information and distance information indicating the distance from the camera 110 to the object). Obtained from the recognition unit 101 and output to the determination unit 105.

  The determination unit 105 determines a first position and a second position that are separated from the image display position indicated by the image display position information by a predetermined distance.

  Here, an example of the determination process of the first position and the second position will be described with reference to FIG. In FIG. 3, it is assumed that both the display medium 130 and the virtual line L1 are horizontal.

  First, the determination unit 105 determines the distance from the position P4 of the pedestrian 200 to the image display position P5 based on the distance from the pedestrian 200 to the camera 110 and the predetermined distance from the camera 110 to the display medium 130. D2 is calculated.

  Next, the determination unit 105 determines the distance D4 from the position P4 to the position P3 based on the calculated distance D2 and the distance from the image display position P5 to the position P3 of a predetermined point in the driver's eye area. Is calculated.

  Next, the determination unit 105 calculates a distance D3 from the position P3 to the position P2 of the driver's left eye.

Next, the determination unit 105 calculates a distance D1 for shifting the image display position P5 by using a similarity ratio between the triangle formed by the positions P4, P6, and P5 and the triangle formed by the positions P4, P2, and P3. That is, the determination unit 105 calculates the distance D1 by the following formula.
D1 = D2 × D3 / D4

  Next, the determination unit 105 determines a position P6 obtained by shifting the image display position P5 by the distance D1 leftward as the first position.

  Next, the determination unit 105 determines a position P7 obtained by shifting the image display position P5 to the right by the distance D1 as the second position. The second position P7 substantially exists on a straight line connecting the image display position P5 and the position P6, and is substantially symmetric with respect to the position P6 with respect to the image display position P5.

  Then, the determination unit 105 outputs position information indicating the first position and the second position to the control unit 106.

  The control unit 106 controls the display unit 104 to generate and display a predetermined image based on the contour information and the position information. For example, the control unit 106 controls the display unit 104 by generating control information indicating the content of control for generation and display of a predetermined image and outputting the control information to the display unit 104. Details of the control by the control unit 106 will be described later.

  The predetermined image is an image generated by the display unit 104 and is an image indicating a presentation image (for example, an image 300 described later) presented to the vehicle occupant when displayed on the display medium 130. That is, the contents of the presented image and the predetermined image are substantially the same.

  The display unit 104 generates a predetermined image based on the control information acquired from the control unit 106 and displays the predetermined image on the display medium 130. For example, when the display medium 130 is a HUD, the display unit 104 projects (projects) a predetermined image on the display medium 130 using a projector function. Thereby, the predetermined image is visually recognized as a virtual image by the driver. In addition, since the principle by which the image projected on the display medium is visually recognized by the driver as a virtual image is well known, detailed description is omitted.

  In the present embodiment, the following description will be given on the assumption that the predetermined image is projected on the display medium 130 and the occupant recognizes it as a virtual image and that the predetermined image is displayed on the display medium 130. That is, in the following description, an event in which a predetermined image is projected on the HUD and visually recognized as a virtual image by the occupant is expressed as display.

  Note that the display unit 104 may display an image on the HUD using, for example, the principle of a hologram instead of using the projector function. In the case of using a hologram, a system using a light guide plate that guides a parallel light beam group that satisfies the total internal reflection condition of the light guide plate by total internal reflection may be used. In the method using the light guide plate, an image is not directly projected as in the projector, but for the sake of convenience of explanation, the definition of projection or display will be described as in the projector method.

  In the present embodiment, the display unit 104 generates a predetermined image. However, the predetermined image may be generated by the display control device 103 or another component not shown.

  Further, the display unit 104 may be included in the display control device 103 to configure as a projection device.

  Next, a specific example of control processing by the control unit 106 will be described below. Here, as a comparative example with the present embodiment, first, an image representing a point cloud operated as biological motion (details will be described later) is the position of the pedestrian 200 on the display medium 130 (the above-described image display position P5). ) Will be described with reference to FIG. 4, and then the control processing of the present embodiment will be described with reference to FIGS. 5 and 6.

  First, the control process in the comparative example will be described. FIG. 4A shows an example when the display unit 104 generates an image based on the control information of the control unit 106 and displays the image on the display medium 130. In this case, the control unit 106 recognizes that the object is a person based on the contour information indicating the contour of the pedestrian 200. Then, the control unit 106 generates a predetermined image 300 as an image representing the shape of the person, and uses the image 300 as the position of the pedestrian 200 on the display medium 130 (the above-described image display position P5). The display unit 104 is controlled so as to be displayed.

  Here, the image 300 will be described. The image 300 is an image representing a point group representing the skeleton of the pedestrian 200 (for example, a human head, shoulders, elbows, wrists, knees, ankles, etc.). This point cloud operates as a biological motion. Therefore, the arrangement of the point group changes in accordance with the movement of the pedestrian 200, but the distance between the predetermined points and the predetermined points (hereinafter referred to as point-to-points) corresponds to the distance between the regions. There is no difference in the arrangement of points outside the constraints. That is, the distance between points does not exceed a predetermined distance (actual distance between parts). In addition, the said point group does not necessarily need to show the position of each site | part of a target object correctly, and may show the approximate position of each site | part.

  Such an image 300 allows the driver to grasp the type of the object and the state of the object without directing his / her line of sight to the object existing in the peripheral visual field. For example, since the driver can intuitively recognize human-specific movements by changing the arrangement of the point cloud, even if various objects (for example, buildings, vehicles, etc.) exist in the field of view, the pedestrian 200 Can be recognized instantly.

  In this embodiment, the target to be operated as biological motion has been described as a point. However, if it can be recognized that the person is a pedestrian 200, a shape such as a perfect circle, an ellipse, or a rectangle having an area can be used. These markers may be used.

  FIG. 4B shows an example of how the image 300 shown in FIG. 4A looks. When the driver views the pedestrian 200 via the display medium 130, the image 300L is seen on the left side of the pedestrian 200 and the image 300R is seen on the right side of the pedestrian 200, as shown in FIG. 4B. That is, for the driver, the image 300 shown in FIG. 4A appears as a double image. Therefore, it is difficult for the driver to accurately grasp the number of objects and the position where the objects are present.

  Therefore, in the present embodiment, the control unit 106 performs the following control process so that the driver can easily grasp the type and state of the object. Next, control processing according to the present embodiment will be described.

  First, the control unit 106 recognizes that the object is a person based on contour information indicating the contour of the pedestrian 200. The control unit 106 generates a predetermined image 300 as an image representing the shape of the person for a predetermined time from the first timing, and the image 300 is displayed on the left side from the position of the pedestrian 200 on the display medium 130. The display unit 104 is controlled to display at the first position P6 shifted by the distance D1. An example at this time is shown in FIGS. 5A and 5B.

  FIG. 5A shows an example in which the image 300 is displayed at the first position P6 on the display medium 130. FIG. As shown in FIG. 5A, the image 300 is displayed shifted to the left from the position of the pedestrian 200 (the image display position P5).

  FIG. 5B shows an example of how the image 300 shown in FIG. 5A looks. When the driver views the pedestrian 200 through the display medium 130, the image 300L is seen on the left side of the pedestrian 200 and the image 300R is seen at the position of the pedestrian 200, as shown in FIG. 5B. That is, for the driver, the image 300 shown in FIG. 5A appears as a double image, but the image 300R appears to overlap the position of the pedestrian 200.

  Next, the control unit 106 generates the image 300 for a predetermined time from the second timing, and shifts the image 300 from the position of the pedestrian 200 on the display medium 130 to the right by the distance D1. The display unit 104 is controlled to display at the second position P7. An example at this time is shown in FIGS. 6A and 6B. The second timing is a timing before a predetermined time elapses from the first timing described above.

  FIG. 6A shows an example in which the image 300 is displayed at the second position P7 on the display medium 130. FIG. As shown in FIG. 6A, the image 300 is displayed shifted to the right from the position of the pedestrian 200 (the image display position P5).

  FIG. 6B shows an example of how the image 300 shown in FIG. 6A looks. When the driver views the pedestrian 200 via the display medium 130, the image 300R is seen on the right side of the pedestrian 200 and the image 300L is seen at the position of the pedestrian 200, as shown in FIG. 6B. That is, for the driver, the image 300 shown in FIG. 6A appears as a double image, but the image 300L appears to overlap the position of the pedestrian 200.

  The control unit 106 repeatedly performs the control process at the first timing and the control process at the second timing. As a result, for the driver, since either the image 300R or the image 300L always appears to overlap the position of the pedestrian 200, the driver determines whether the object is a pedestrian (type of object). And whether the object is about to cross the roadway (state of the object).

  The configuration example of the display system 100 according to the present embodiment has been described above.

  Next, an operation example of the display system 100 according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart showing an operation example of the display system 100 according to the present embodiment.

  First, the recognition unit 101 extracts a feature amount included in the foreground image acquired from the camera 110, and recognizes an object based on the extracted feature amount (step S101). Further, when recognizing the target object, the recognition unit 101 calculates the position of the target object and detects the contour of the target object.

  Next, the determination unit 102 determines the image display position based on the relationship between the position of a predetermined point in the driver's eye area, the position of the object calculated by the recognition unit 101, and the position of the display medium 130. (Step S102). As described above, the image display position is the position of the object on the display medium 130 when the object is observed through the display medium 130 from a predetermined point in the driver's eye area.

  Next, the determination unit 105 determines a first position that is a predetermined distance away from the image display position in the left direction and a second position that is a predetermined distance away from the image display position in the right direction (step S103).

  Next, the control unit 106 controls the display unit 104 to generate and display a predetermined image for a predetermined time from the first timing (step S104).

  The predetermined image is an image generated on the display unit 104 and is an image indicating a presentation image presented to the driver when displayed at the first position of the display medium 130. The presentation image is an image showing the shape of the object recognized by the recognition unit 101, and is, for example, the image 300 including the point group operating as biological motion shown in FIG. 5A.

  By the control processing in step S104 described above, the display unit 104 generates a predetermined image for a predetermined time from the first timing, and displays the predetermined image on the display medium 130 (see, for example, FIG. 5A). At this time, when the driver visually recognizes the object via the display medium 130, the presented image appears as a double image (see, for example, FIG. 5B).

  Next, the control unit 106 controls the display unit 104 to generate and display a predetermined image for a predetermined time from the second timing (step S105). As described above, the second timing is a timing before a predetermined time elapses from the first timing.

  The predetermined image is an image generated on the display unit 104 and is an image showing a presentation image presented to the driver when displayed at the second position of the display medium 130. The presentation image is an image showing the shape of the object recognized by the recognition unit 101, and is, for example, the image 300 including the point group operating as biological motion shown in FIG. 6A.

  By the control processing in step S105 described above, the display unit 104 generates a predetermined image for a predetermined time from the second timing, and displays the predetermined image on the display medium 130 (see, for example, FIG. 6A). At this time, when the driver visually recognizes the object via the display medium 130, the second presented image appears as a double image (see, for example, FIG. 6B).

  The operation example of the display system 100 according to the present embodiment has been described above.

  In the above description, the first position and the second position are determined so that all the point groups of the image 300 appear to overlap the pedestrian 200 as shown in FIGS. 5B and 6B. A part of the group may not overlap with the pedestrian 200.

  In the above description, the first position is the position P6 shifted from the image display position by a predetermined distance to the left, and the second position is the position P7 shifted by a predetermined distance from the image display position to the right. The first position may be the position P7, and the second position may be the position P6.

  Thus, in the present embodiment, when the driver gazes at the object through the display medium, one of the double images of the image displayed on the display medium always appears to overlap the object. Of the multiple images, the one that appears to overlap the object functions as biological motion, so it can recognize the object, and on the other hand it functions as biological motion because the display is missing in time. Therefore, the object cannot be recognized. That is, the driver can recognize the object only from the image displayed at the position overlapping the object, and can accurately grasp the position and number of the object.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made. Hereinafter, modified examples will be described.

(Modification 1)
In the above-described embodiment, the case where the line connecting the positions of both eyes of the driver (the virtual line L1 shown in FIG. 3) is horizontal has been described. It may not be horizontal. In this case, in the present modification, correction processing is performed based on the distance D1 and the head inclination. The inclination of the head here is an inclination in the rotation direction with the front-rear direction of the head as an axis. The inclination of the head is detected by a detection unit (not shown) that detects the inclination of the head based on the driver's face image captured by the camera 120, for example. Since the process of detecting the tilt of the head is known, a detailed description thereof will be omitted.

  First, a case where correction processing is not performed when the head is tilted will be described with reference to FIGS. 8A and 8B. FIG. 8A shows how the double image is seen during a predetermined time from the first timing when the horizontal imaginary line L1 is inclined by the angle θ1 to become the imaginary line L1 ′ due to the tilt of the head. ing. Further, FIG. 8B shows how the double image is seen during a predetermined time from the second timing when the horizontal imaginary line L1 is inclined by the angle θ1 to become the imaginary line L1 ′ because the head is tilted. Is shown. Note that the virtual lines L1 and L 'are illustrated for explanation, and are not displayed on the display medium 130 or the like.

  As shown in FIG. 8A, the image 300 </ b> R appears to be shifted above the pedestrian 200. Further, as shown in FIG. 8B, the image 300L appears to be shifted downward from the pedestrian 200. Therefore, since the double image which overlaps with the pedestrian 200 seems to shift | deviate up and down, a driver | operator is hard to visually recognize.

  Therefore, in the present modification, the control unit 106 performs the following correction process based on the distance D1 and the angle θ1. Here, as the coordinate system on the surface of the display medium 130, the right horizontal direction is the positive direction of the X axis, and the lower vertical direction is the positive direction of the Y axis.

First, for example, the control unit 106 calculates the correction amount x in the X-axis direction and the correction amount y in the Y-axis direction by the following formulas.
x = D1 (1-cos θ1)
y = D1sin θ1

  Next, the control unit 106 corrects the first position based on the calculated correction amounts x and y. For example, the control unit 106 shifts the first position by the correction amount x in the positive direction of the X axis and shifts the first position by the correction amount y in the positive direction of the Y axis. Thereby, the corrected first position is determined.

  Then, the control unit 106 controls the display unit 104 to display the image 300 at the corrected first position for a predetermined time from the first timing. The appearance of the double image at this time is shown in FIG. 8C. As shown in FIG. 8C, the entire point group of the image 300R appears to overlap the pedestrian 200.

  Next, the control unit 106 corrects the second position based on the calculated correction amounts x and y. For example, the control unit 106 shifts the second position by the correction amount x in the negative direction of the X axis and shifts the second position by the correction amount y in the negative direction of the Y axis. Thereby, the corrected second position is determined.

  Then, the control unit 106 controls the display unit 104 to display the image 300 at the corrected second position for a predetermined time from the second timing. FIG. 8D shows how the double image looks at this time. As shown in FIG. 8D, the entire point group of the image 300 </ b> L appears to overlap the pedestrian 200.

  Thus, according to this modification, when the driver tilts the head, the line connecting the first position and the second position is based on the tilt angle θ1 and the distance D1 of the head. The first position and the second position are corrected so as to be parallel to a line (imaginary line L ′) connecting the positions of both eyes of the driver. As a result, even when the driver tilts his / her head, one of the double images does not deviate from the object and appears to overlap the object, thereby improving visibility.

(Modification 2)
In the above embodiment, the control process by the control unit 106 may be executed when the driver visually recognizes the object.

  For example, the control unit 106 obtains line-of-sight information indicating the line of sight detected by the line-of-sight detection unit (not shown) from the line-of-sight detection unit, and also indicates an object position indicating the position of the object recognized by the recognition unit 101. Information is acquired from the recognition unit 101. And the control part 106 determines whether the driver | operator visually recognized the target object based on eyes | visual_axis information and target object position information. Hereinafter, an example of the object visually recognized by the driver will be described as the pedestrian 200 described above.

  As a result of the determination, when it is determined that the driver has visually recognized the pedestrian 200, as described above, the control unit 106 displays the display unit 104 based on the contour information indicating the contour of only the pedestrian 200 visually recognized by the driver. A process for determining the image 300 to be generated is performed. Then, the control unit 106 controls the display unit 104 to generate and display the determined image 300.

  Thereby, since the image 300 is displayed only with respect to the pedestrian 200 visually recognized by the driver, the driver can visually recognize the images 300R and 300L at a necessary timing. At this time, if the driver gazes at the pedestrian 200 via the display medium 130, the image 300R in FIG. 5B and the image 300L in FIG. A person's line of sight can be guided to the pedestrian 200.

  Thereafter, when a certain time has elapsed since the display of the image 300 described above, the control unit 106 may control the display unit 104 to stop generating and displaying the image 300.

  Thereby, after attracting the driver's consciousness to the pedestrian 200 by displaying the image 300, it is possible to prevent the driver from visually recognizing the images 300R and 300L for a longer time than necessary, thereby improving safety. Can be expected.

  As described above, according to this modification, since the display and non-display of the image are performed depending on whether or not the driver is viewing the object, the driver can view the image at an appropriate timing, The viewing of the image can be stopped.

(Modification 3)
You may control the brightness | luminance of the image demonstrated in the said embodiment as follows. Hereinafter, two luminance control processes will be described.

  First, the first luminance control process performed by the control unit 106 will be described with reference to FIGS. 9A and 9B. FIG. 9A shows a change in luminance of the image 300 shown in FIG. 5A, and FIG. 9B shows a change in luminance of the image 300 shown in FIG. 6A. 9A and 9B, the horizontal axis represents elapsed time, and the vertical axis represents luminance.

  As shown in FIGS. 9A and 9B, in the first brightness control process, the brightness of the image 300 in FIG. 6A is turned on (upper limit value) before the brightness of the image 300 in FIG. 5A is turned off (lower limit value). In addition, the luminance of the image 300 in FIG. 5A is controlled to be ON (upper limit) before the luminance of the image 300 in FIG. 6A is OFF (lower limit value, which may be zero). The By this control, the image 300 of FIG. 6A appears before the image 300 of FIG. 5A disappears, and the image 300 of FIG. 5A appears before the image 300 of FIG. 6A disappears.

  In this first luminance control process, as shown in FIGS. 9A and 9B, the gradient of the rising and falling of each luminance is steep (for example, right angle), so that the driver can view the image 300L in FIG. 5B. It moves to the image 300L of 6B, and it appears that the image 300R of FIG. 6B moves to the image 300R of FIG. 5B.

  Next, the second luminance control process performed by the control unit 106 will be described with reference to FIGS. 9C and 9D. 9C shows a change in luminance of the image 300 shown in FIG. AB, and FIG. 9D shows a change in luminance of the image 300 shown in FIG. 6A. 9A and 9B, the horizontal axis represents elapsed time, and the vertical axis represents luminance.

  As shown in FIGS. 9C and 9D, in the second brightness control process, the image 300 in FIG. 6A is turned off before the brightness of the image 300 in FIG. 5A becomes OFF (lower limit value), as in the first brightness control process. The brightness of the image 300 in FIG. 6A is turned on (upper limit value) before the brightness of the image 300 in FIG. 6A is turned off (lower limit value). Controlled. By this control, the image 300 of FIG. 6A appears before the image 300 of FIG. 5A disappears, and the image 300 of FIG. 5A appears before the image 300 of FIG. 6A disappears.

  In the second luminance control process, as shown in FIGS. 9C and 9D, the gradient of the rising and falling of each luminance is gentle (for example, a predetermined obtuse angle). And each of the images 300R of FIG. 6B appear to blink.

  In this way, by making the rising and falling slopes (in other words, the speed) of the image brightness steep or gentle, the image of the double image that does not overlap the object can be seen. You can choose which way.

  In the above, the modification of embodiment of this invention was demonstrated. Note that the above-described modifications may be arbitrarily combined.

  The functions of the units of the display system 100 and the display control apparatus 103 described above can be realized by a computer program.

  FIG. 9 is a diagram illustrating a hardware configuration of a computer that realizes the functions of the respective units by a program. The computer 1000 includes an input device 1001 such as an input button and a touch pad, an output device 1002 such as a display and a speaker, a CPU (Central Processing Unit) 1003, a ROM (Read Only Memory) 1004, and a RAM (Random Access Memory) 1005. . The computer 1000 is a reading device that reads information from a recording medium such as a hard disk device, a storage device 1006 such as an SSD (Solid State Drive), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. 1007 includes a transmission / reception device 1008 that performs communication via a network. Each of the above parts is connected by a bus 1009.

  Then, the reading device 1007 reads the program from a recording medium on which a program for realizing the functions of the above-described units is recorded, and stores the program in the storage device 1006. Alternatively, the transmission / reception device 1008 communicates with a server device connected to the network, and causes the storage device 1006 to store a program for realizing the function of each unit downloaded from the server device.

  Then, the CPU 1003 copies the program stored in the storage device 1006 to the RAM 1005, and sequentially reads and executes the instructions included in the program from the RAM 1005, thereby realizing the functions of the above-described units. Further, when executing the program, the RAM 1005 or the storage device 1006 stores information obtained by the various processes described in the first embodiment and is used as appropriate.

  The present invention relates to a display control device, a display control method, a display control program, and a projection device that control the display of information provided to a user (for example, a passenger such as a vehicle or a user wearing a display device). Useful for.

DESCRIPTION OF SYMBOLS 100 Display system 101 Recognition part 102 Determination part 103 Display control apparatus 104 Display part 105 Determination part 106 Control part 110 Camera 120 Camera 130 Display medium 1000 Computer 1001 Input apparatus 1002 Output apparatus 1003 CPU
1004 ROM
1005 RAM
1006 Storage device 1007 Reading device 1008 Transmission / reception device 1009 Bus

Claims (8)

A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image;
Determining a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light; 1 decision part;
A display unit that generates a predetermined image and displays the predetermined image on the display medium;
A display control apparatus in a display system including:
When the predetermined object is recognized by the recognition unit, a first position that is a predetermined distance away from a position determined by the first determination unit, a position determined by the first determination unit, and the first A second position that is substantially on a straight line connecting the first position and a position that is substantially symmetrical to the first position with respect to the position determined by the first determination unit. A second determining unit for determining;
The display unit is controlled to generate a predetermined image representing a predetermined presentation image at the first position when displayed on the display medium for a predetermined time from the first timing, and the first timing The predetermined image representing the predetermined presentation image is generated at the second position when displayed on the display medium for the predetermined time from the second timing before the predetermined time elapses from A control unit for controlling the display unit,
Display control device. The second determination unit includes
When the occupant visually recognizes the predetermined object via the display medium, the predetermined presentation image becomes a double image and is visually recognized by the occupant during the predetermined time from the first timing. One of the predetermined presentation images visually recognized as the double image and one of the predetermined presentation images visually recognized as the double image from the second timing to the predetermined time. Determining the first position and the second position such that the first position and the predetermined object on the display medium are visually recognized.
The display control apparatus according to claim 1. The controller is
When a tilt of the head of the occupant is detected, a line connecting the first position and the second position is based on the tilt angle of the head and the predetermined distance. Correcting the first position and the second position so as to be parallel to a line connecting the positions of both eyes;
The display control apparatus according to claim 1 or 2. The controller is
Based on the line of sight of the occupant and the position of the predetermined object, it is determined whether the occupant has visually recognized the predetermined object,
When the occupant determines that the predetermined object has been visually recognized, the display unit is controlled to generate the predetermined image.
The display control apparatus of any one of Claim 1 to 3. The controller is
Controlling the rate of rise and fall of the brightness of the predetermined presentation image;
The display control apparatus of any one of Claim 1 to 4. A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image;
Determination to determine a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light And
A display unit that generates a predetermined image and displays the predetermined image on the display medium;
A display control method in a display system including:
When the predetermined object is recognized by the recognizing unit, a first position that is a predetermined distance away from a position determined by the determining unit, a position determined by the determining unit, and the first position A second position which is substantially on a connected straight line and which is substantially symmetrical to the first position with respect to the position determined by the first determination unit;
Controlling the display unit to generate a predetermined image representing a predetermined presentation image at the first position when displayed on the display medium for a predetermined time from a first timing;
The predetermined image representing the predetermined presentation image at the second position when displayed on the display medium for a predetermined time from a second timing before the predetermined time elapses from the first timing. Controlling the display unit to generate an image;
Display control method. A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image;
Determination to determine a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light And
A display unit that generates a predetermined image and displays the predetermined image on the display medium;
A display control method to be executed by a computer in a display system including:
In the computer,
When the predetermined object is recognized by the recognizing unit, a first position that is a predetermined distance away from a position determined by the determining unit, a position determined by the determining unit, and the first position A process of determining a second position that substantially exists on the connected straight line and that is substantially symmetrical to the first position with respect to the position determined by the first determination unit; ,
A process for controlling the display unit to generate a predetermined image representing a predetermined presentation image at the first position when displayed on the display medium for a predetermined time from a first timing;
The predetermined image representing the predetermined presentation image at the second position when displayed on the display medium for a predetermined time from a second timing before the predetermined time elapses from the first timing. Controlling the display unit to generate an image, and
Display control program. A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object, and recognizes a predetermined object existing in front of the occupant from the foreground image;
Determining a position of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the eye area of the occupant through a display medium that transmits light; 1 decision part;
A projection apparatus in a display system comprising:
A display unit that generates a predetermined image and displays the predetermined image on the display medium;
When the predetermined object is recognized by the recognition unit, a first position that is a predetermined distance away from a position determined by the first determination unit, a position determined by the first determination unit, and the first A second position that is substantially on a straight line connecting the first position and a position that is substantially symmetrical to the first position with respect to the position determined by the first determination unit. A second determining unit for determining;
The display unit is controlled to generate a predetermined image representing a predetermined presentation image at the first position when displayed on the display medium for a predetermined time from the first timing, and the first timing The predetermined image representing the predetermined presentation image is generated at the second position when displayed on the display medium for the predetermined time from the second timing before the predetermined time elapses from A control unit for controlling the display unit,
Projection device.

Images