JP2016081464A - Display control program and display controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a display control program and a display controller allowing an occupant to precisely comprehend an existence of an object.SOLUTION: A display controller 103 has a control unit 106 that controls a display unit 104 to display, at a first timing, a first predetermined image which represents a virtual image 302 at the right side position 216 and a virtual image 301 at the left side position 214 when displayed on a display medium 130; and to display a second predetermined image, at a second timing different from the first timing, which represents a virtual image 402 at the right side position 216 and a virtual image 401 at the left side position 214 when displayed on the display medium 130.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a display control program and a display control apparatus for controlling display of information provided to a passenger such as a vehicle.

  2. Description of the Related Art In recent years, a head-up display (HUD) device has been actively developed that allows an occupant such as a driver to visually recognize an image by projecting the image onto a windshield of a vehicle. In this apparatus, the visibility of an object can be improved by projecting an image so as to be superimposed on an object in the real world (for example, another vehicle, a pedestrian, a building, or the like).

  In such a head-up display device, a so-called double image problem occurs in the occupant's field of view. In other words, when a display that emphasizes the object on the windshield is superimposed on the position of the object in the real world, it is displayed on the windshield from the distance to the object being watched by the occupant and the viewpoint of the occupant. The distance to the image is different. In this case, as shown in FIG. 1, the image 30 displayed on the windshield 20 looks double, side by side, with the object 10 (pedestrian) watched by the occupant sandwiched between them. As a result, there is a problem that the occupant cannot correctly recognize the displayed information.

  As a method for solving the problem of the double image, for example, there is a method disclosed in Patent Document 1 (hereinafter referred to as a conventional technique).

  In the prior art, in order to suppress the influence of the double image, it is superimposed on the position of the object detected by a camera or the like, and an image that does not give a sense of incongruity even when viewed as a double image is generated and displayed. To do. Specifically, as shown in FIG. 2, when a double image is generated by generating an image 40 having a dot pattern shape and displaying the image 40 on the windshield 20 in association with the object 10. However, it is said that the occupant can grasp the existence and position of the object.

Japanese Patent No. 4970379

  However, in the above-described prior art, it is possible to prevent the visibility of the object from being lowered by separating the display on the windshield from left and right, but an image superimposed on the position of the object due to the influence of the double image. Since the display area becomes large, it is difficult for the passenger to accurately grasp the object.

  The objective of this invention is providing the display control program and display control apparatus which can make a passenger | crew know the presence of a target object correctly.

  A display control program according to an aspect of the present invention acquires a foreground image that is a foreground image of an occupant of a moving object, recognizes a predetermined object included in the foreground image, and the occupant's eye area Specifying a predetermined position of an area of the predetermined object on the display medium when the predetermined object is observed from a predetermined point of the image through a display medium that transmits light A display control program to be executed by a computer in a display system, including: a display unit that generates a predetermined image based on a recognition result of the recognition unit, and displays the predetermined image on the display medium, In addition, when the predetermined object is recognized by the recognition unit, the predetermined position specified by the specifying unit on the display medium based on the recognition result. A right side position that is a predetermined distance away from the left side position, and a left side position that is substantially symmetrical to the right side position with respect to the predetermined position, and when displayed on the display medium, A first predetermined image indicating the first right image and the first left image at the left position is displayed at a first timing and displayed on the display medium when the first right image is displayed at the right position. A process for controlling the display unit to display the second predetermined image indicating the second right image and the second predetermined image indicating the second left image at the left position at a second timing different from the first timing. The first and second predetermined images are images each composed of a plurality of elements, and the first right image is a first element that is a part of the plurality of elements. The above-mentioned elements The left image is composed of second elements other than the first element among the plurality of elements, the second right image is a part of the plurality of elements, and The second left image is composed of a fourth element other than the third element among the plurality of elements. The third left image is composed of a third element different from the first element.

  A display control device according to an aspect of the present invention acquires a foreground image that is a foreground image of a moving body occupant, recognizes a predetermined object included in the foreground image, and the occupant's eye area Specifying a predetermined position of an area of the predetermined object on the display medium when the predetermined object is observed from a predetermined point of the image through a display medium that transmits light And a display control unit that generates a predetermined image based on the recognition result of the recognition unit and displays the predetermined image on the display medium. When the target object is recognized, on the display medium, on the display medium, a right position that is a predetermined distance away from the predetermined position specified by the specifying unit, and the predetermined position A determination unit that determines a left side position that is substantially symmetric with respect to the right side position, a first right image is displayed at the right side position when displayed on the display medium, and a first right image is displayed at the left side position. A first predetermined image indicating one left image is displayed at a first timing, and when displayed on the display medium, a second right image is displayed at the right position and a second image is displayed at the left position. A control unit that controls the display unit so as to display a second predetermined image indicating a left image at a second timing different from the first timing, and the first and second predetermined images Are images composed of a plurality of elements, the first right image is composed of a part of the first elements, and the first left image is the plurality of elements. Second of the elements other than the first element The second right image is composed of a part of the plurality of elements and a third element different from the first element, and the second left image is composed of the plurality of elements. It is comprised from 4th elements other than the said 3rd element among elements.

  According to the present invention, it is possible to cause an occupant to accurately grasp the presence of an object.

It is a figure explaining the problem of a prior art. It is a figure explaining the problem of a prior art (technology described in patent document 1). It is a block diagram which shows the structural example of the display system which concerns on this Embodiment. It is a figure which shows the distance of the camera and a pedestrian in the horizontal direction and front direction of a vehicle. It is a flowchart which shows the operation example of the display system which concerns on this Embodiment. It is a figure explaining the process which determines the right side position and left side position with respect to a target object display position. It is a figure explaining the process which determines the right side position and left side position with respect to a target object display position. It is a figure explaining the display process of a virtual image. It is a figure explaining the display process of a virtual image. The relationship between two virtual images displayed on the display medium and how the virtual image is viewed from the driver is shown. It is a flowchart which shows the modification of operation | movement of the display system which concerns on this Embodiment. The relationship between two virtual images displayed on the display medium and how the virtual image is viewed from the driver is shown. The relationship between two virtual images displayed on the display medium and how the virtual image is viewed from the driver is shown. It is a block diagram which shows the hardware structural example of the display system which concerns on embodiment of this invention, and a display control apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  First, a configuration example of the display system 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating a configuration example of the display system 100 according to the present embodiment.

  The display system 100 is a system that is used in a moving body such as a vehicle and supports driving of the vehicle. Specifically, the display system 100 is a system that prevents traffic accidents by recognizing information on pedestrians existing around a moving body and presenting information indicating the presence of pedestrians to passengers. It is. The display system 100 may be a system that detects a vehicle ahead of a moving body and displays information related to the vehicle ahead, thereby preventing a rear-end collision with the vehicle ahead. Further, the display system 100 may be a system that prevents a collision with a bicycle by detecting a bicycle that is traveling in front of a moving body and displaying information related to the bicycle.

  Furthermore, the display system 100 may be an in-vehicle device or a device brought into a vehicle. Although the display system 100 is described as being applied to a vehicle, the moving body is not limited to a vehicle, and may be a ship, an aircraft, or the like. In the present embodiment, an example in which the user is a vehicle occupant, particularly a vehicle driver, is described, but the present invention is not limited thereto. Furthermore, the display system 100 may be used in a wearable computer (for example, an HMD described later) that can be worn by the user.

  As shown in FIG. 1, a camera 110, a camera 120, and a display medium 130 are connected to the display system 100.

  The camera 110 is mounted inside or outside the vehicle and photographs a field of view in front of the driver (hereinafter referred to as “foreground”). An image photographed by the camera 110 (hereinafter referred to as “foreground image”) is output to the recognition unit 101. In this embodiment, the camera 110 is used as a means for sensing the foreground. However, the present invention is not limited to this. For example, a radar or the like may be used.

  The camera 120 is a camera called an inner camera or a driver monitoring camera. The camera 120 is mounted inside the vehicle, photographs the driver's face, and detects the driver's viewpoint direction based on the photographed face image. Information indicating the driver's viewpoint direction detected by the camera 120 (hereinafter referred to as “viewpoint direction information”) is output to the recognition unit 101. Further, the face image photographed by the camera 120 is output to the specifying unit 102.

  The display medium 130 is a medium that is used in a vehicle and displays a virtual image (details will be described later) generated by the display unit 104 described later. In the present embodiment, the display medium 130 is a transmissive display medium that transmits light.

  Examples of the display medium 130 include a head up display (HUD), an LCD (Liquid Crystal Display), an HMD (Head-Mounted Display or Helmet-Mounted Display), a glasses-type display (Smart Glasses), and a navigation display. , Meter displays, other dedicated displays, etc. are applied. The HUD may be, for example, a vehicle windshield, or may be a separately provided glass surface, plastic surface (for example, a combiner), or the like. The windshield may be, for example, a windshield, a vehicle side glass or a back glass.

  Further, the HUD may be a transmissive display provided on the surface or inside of the windshield. Here, the transmissive display is, for example, a transmissive organic EL display or a transparent display using glass that emits light when irradiated with light of a specific wavelength. The driver can view the display on the transmissive display at the same time as viewing the background. Thus, the display medium 130 is a display medium that transmits light. In either case, the virtual image is displayed on the display medium 130.

  In the example of FIG. 1, the display system 100 is configured not to include the display medium 130, but may be configured to include the display medium 130.

  In FIG. 1, the display system 100 includes a recognition unit 101, a specification unit 102, a display control device 103, and a display unit 104. In addition, the display control apparatus 103 includes a determination unit 105 and a control unit 106. Hereinafter, each part will be described.

  The recognition unit 101 extracts a feature amount included in the foreground image acquired from the camera 110, and recognizes an object by performing pattern matching processing or the like on the extracted feature amount. When a radar is used instead of the camera 110 as means for sensing the foreground, the recognition unit 101 extracts and recognizes an object by clustering or machine learning.

  Further, when the recognition unit 101 recognizes an object, the recognition unit 101 calculates the position of the object. The target object is, for example, a moving body (for example, a vehicle, a person, a bicycle, a two-wheeled vehicle, etc.), a white line on a road, a sign, a road marking, a curb, a guardrail, a traffic light, a utility pole, a vehicle, a person, a building, and the like. Note that the object recognition process and the object position calculation process are well known, and thus detailed description thereof is omitted.

  Further, when the recognition unit 101 recognizes a predetermined object, the recognition unit 101 detects the contour of the object. For example, the recognition unit 101 detects the contour of the target object by performing edge extraction using a specific differential filter on the foreground image. Then, the recognition unit 101 specifies the type of the target object based on the detected contour of the target object. In addition, since the detection process of the edge of a target object is well-known, detailed description is abbreviate | omitted. In the present embodiment, an example is described in which when a predetermined object is recognized, the outline of the object is detected, but the process of detecting the outline of the object is not necessarily essential. For example, when a human-shaped marker (virtual image) is precisely superimposed on the contour of a pedestrian, it is necessary to detect the contour of the object, but a template for a standard human-shaped marker (such as a flip book) This is because it is not necessary to detect the contour of the target object when displaying the image in accordance with the position of the pedestrian.

  When the recognition unit 101 specifies the type of the object as a pedestrian, as shown in FIG. 4, the position of the pedestrian (X, Y), the relative angle of the pedestrian with respect to the traveling direction of the vehicle, the pedestrian Recognizes the relative speed of the vehicle. The position of the pedestrian (X, Y) in the camera coordinate system with the position where the camera 110 is installed as a reference point (point O in the figure) and the lateral and forward directions of the vehicle as the X and Y axes, respectively Coordinates. That is, the position X of the pedestrian indicates the distance between the camera 110 and the pedestrian in the lateral direction of the vehicle. The position Y of the pedestrian indicates the distance between the camera 110 and the pedestrian in the traveling direction of the vehicle.

  The recognition unit 101 outputs information including the position and type of the target object (hereinafter referred to as “target position information”) to the specifying unit 102. When the recognition unit 101 recognizes a plurality of objects, the recognition unit 101 acquires the viewpoint direction information output from the camera 120, and the object located in the viewpoint direction indicated by the acquired viewpoint direction information, that is, the direction in which the driver is gazing. Is identified. Then, the recognition unit 101 outputs object position information including the position and type of the specified object to the specifying unit 102. In addition, the recognition unit 101 outputs information indicating the contour of the object (hereinafter referred to as “contour information”) to the control unit 106 of the display control device 103. In addition to information indicating the shape of the contour, the contour information includes information indicating the position of the contour and the type of the object.

  The identification unit 102 detects a driver's eye area (hereinafter referred to as “eye area”) based on the face image output from the camera 120. The eye area is a predetermined area including both eyes, for example. In addition, since the process which detects an eye area | region by performing image processing with respect to a face image is well-known, detailed description is abbreviate | omitted. In the present embodiment, the specifying unit 102 detects the eye area based on the face image. However, the eye area may be a predetermined area, or a line of sight that detects the driver's line of sight. An area detected based on the line of sight detected by a detection unit (not shown) may be used.

  Then, the specifying unit 102 detects the position of a predetermined point (for example, the left or right eye or the center between the left eye and the right eye) in the detected eye region, and the object position output from the recognition unit 101. From the relationship between the position of the object indicated in the information and the position of the display medium 130, the position on the display medium 130 where the object is displayed (hereinafter referred to as “object display position”) is specified. In other words, the target object display position is a position of the target object on the display medium 130 when the target object is observed through the display medium 130 from a predetermined point in the eye area. Since the object display position specifying process is well known, detailed description thereof is omitted.

  Then, the specifying unit 102 outputs information indicating the object display position (hereinafter referred to as “object display position information”) to the determination unit 105 of the display control device 103.

  Based on the recognition result of the object by the recognition unit 101, the determination unit 105 is separated from the object display position indicated by the object display position information output from the specifying unit 102 on the display medium 130 by a predetermined distance. A right side position and a left side position that is substantially symmetrical to the right side position with respect to the object display position are determined. A specific method for determining the right side position and the left side position will be described later. The determining unit 105 outputs information indicating the determined left position and right position (hereinafter referred to as “left / right position information”) to the control unit 106.

  The control unit 106 generates a virtual image based on the contour information output from the recognition unit 101, and displays the virtual image on the display medium 130 based on the left-right position information output from the determination unit 105. The unit 104 is controlled. For example, the control unit 106 generates control information indicating the content of control for generation and display of the virtual image based on the contour information and the left and right position information, and outputs the control information to the display unit 104 to display the control unit 106. 104 is controlled. Details of the control by the control unit 106 will be described later.

  The display unit 104 generates a virtual image based on the control information output from the control unit 106 and displays the virtual image on the display medium 130. For example, when the display medium 130 is a HUD, the display unit 104 projects (projects) a virtual image on the display medium 130 using a projector function. Thereby, a virtual image is visually recognized as a virtual image by the driver. In addition, since the principle by which a driver | operator visually recognizes the virtual image projected on the display medium as a virtual image is well-known, detailed description is abbreviate | omitted.

  In the present embodiment, the following description will be made assuming that the virtual image is projected onto the display medium 130 to cause the occupant to recognize it as a virtual image and the virtual image is displayed on the display medium 130. That is, in the following description, a phenomenon in which a virtual image is projected onto the HUD and visually recognized as a virtual image by the passenger is described as display.

  Instead of using the projector function, the display unit 104 may display a virtual image on the HUD using, for example, a hologram principle. In the case of using a hologram, a method using a light guide plate that guides a parallel light beam group that satisfies the internal total reflection condition of the light guide plate by total internal reflection may be used. In the method using the light guide plate, a virtual image is not directly projected as in the projector, but for the sake of convenience of explanation, the definition of projection or display will be described as in the projector method.

  In the present embodiment, the display unit 104 generates a virtual image. However, the virtual image may be generated by the display control device 103 or another component not shown.

  Further, the display unit 104 may be included in the display control device 103 to configure as a projection device.

  Next, an operation example of the display system 100 according to the present embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing an operation example of the display system 100 according to the present embodiment. The process in step S100 is started when the display system 100 is activated in conjunction with, for example, the vehicle engine changing from an off state to an on state.

  First, the recognition unit 101 extracts a feature amount included in a foreground image acquired from the camera 110, and recognizes an object by performing a pattern matching process or the like on the extracted feature amount (step S100). Then, the recognizing unit 101 outputs object position information including the position and type of the object to the specifying unit 102. Further, the recognition unit 101 outputs contour information indicating the contour of the object to the control unit 106 of the display control device 103.

  Next, the specifying unit 102 determines whether or not the type of the object included in the object position information output from the recognition unit 101 is a pedestrian (step S120). As a result of this determination, when the type of the object is not a pedestrian (step S120, NO), the process transitions to step S220. On the other hand, when the type of the object is a pedestrian (step S120, YES), the specifying unit 102 uses the position of a predetermined point in the driver's eye area and the object position information output from the recognition unit 101. The object display position that is the position on the display medium 130 where the pedestrian is displayed is specified from the relationship between the position of the displayed pedestrian (object) and the position of the display medium 130 (step S140). Then, the specifying unit 102 outputs object display information indicating the object display position to the determination unit 105 of the display control device 103.

  Next, the determination unit 105 determines a predetermined distance from the object display position indicated by the object display position information output from the specifying unit 102 on the display medium 130 based on the recognition result of the object by the recognition unit 101. A right position that is far away from the target position and a left position that is substantially symmetrical to the right position with respect to the object display position are determined (step S160). Then, the determination unit 105 outputs left and right position information indicating the determined left side position and right side position to the control unit 106.

Here, the process of step S160 will be described with reference to FIG. In FIG. 6, 200 is a pedestrian located in front of the vehicle, 220 is a left eye of the driver, and 230 is a right eye of the driver. In the example of FIG. 6, the pedestrian 200 is located in front of the driver, that is, in front of the center 225 between the left eye 220 and the right eye 230. Reference numeral 212 denotes a position (object display position) on the display medium 130 on which the pedestrian 200 is displayed, which is specified by the specifying unit 102. L1 is the distance from the driver's viewpoint (left eye 220, right eye 230) to the display medium 130. The determination unit 105 can calculate the distance L <b> 1 based on a predetermined positional relationship between the camera 120 and the display medium 130 and a driver's face image captured by the camera 120. L2 is the distance from the display medium 130 to the pedestrian 200. The determination unit 105 can calculate the distance L2 based on the distance from the pedestrian 200 to the camera 110 and the predetermined distance from the camera 110 to the display medium 130. L3 is the distance between the left eye 220 and the right eye 230 of the driver (6 to 7 [cm] on average). The determination unit 105 can calculate the distance L3 based on the driver's face image photographed by the camera 120. In this case, the determination unit 105 determines a right position 216 and a left position 214 that are separated from the object display position 212 by a predetermined distance d. The distance d can be calculated by the following equation (1).
d = 1/2 * L2 * L3 / (L1 + L2) (1)
In calculating the distance L3, it is not always necessary to calculate the distance L3 based on the driver's face image captured by the camera 120. That is, it is not always essential to provide the camera 120. For example, a predetermined value such as 6.5 [cm] may be set as the distance L3.

  In FIG. 6, the case where the pedestrian 200 is located in front of the driver has been described. However, the right position 216 and the left position 214 are similarly set even when the position of the pedestrian 200 deviates from the front of the driver to the left and right. Can be determined. FIG. 7 shows a case where the position of the pedestrian 200 is shifted leftward from the front of the driver. Also in this case, the distance d between the object display position 212 and the right position 216 and the left position 214 can be calculated using the above equation (1). 6 and 7, the right position 216 and the left position 214 can be determined in the same manner even when the position of the driver's head is shifted left and right from the position of the pedestrian 200. In addition, when the driver's head position rotates in the vehicle roll direction (in other words, the direction in which the driver rotates his / her neck on a plane perpendicular to the traveling direction of the vehicle), the determination unit 105 is determined as a position rotated about the right position 216 and the left position 214 by the same angle as the rotation angle.

  Based on the contour information output from the recognition unit 101 and the left and right side position information output from the determination unit 105, the control unit 106 generates control information indicating the content of control for generation and display of the virtual image, The control information is output to the display unit 104.

  Here, the virtual image display processing will be described with reference to FIG. In FIG. 8, reference numerals 301 and 302 denote virtual images displayed at the left position 214 and the right position 216 on the display medium 130, respectively. Reference numeral 303 denotes a virtual image that is visible in the left direction of the pedestrian 200 when the driver views the virtual image 301 (corresponding to the “first left image” of the present invention) displayed on the display medium 130 with the right eye 230. . A virtual image 304 is visible to the right of the pedestrian 200 when the driver views a virtual image 302 (corresponding to the “first right image” of the present invention) displayed on the display medium 130 with the left eye 220. . The virtual images 301 and 302 are preferably displayed with the same color and brightness. When the driver gazes at the pedestrian 200, a virtual image other than the virtual image 303 among the double images of the virtual image 301 and a virtual image other than the virtual image 304 among the double images of the virtual image 302 are displayed on the pedestrian 200. It is a virtual image of a person's shape that appears to be combined in position.

  Here, the virtual image 301 and the virtual image 302 are a first predetermined image composed of a plurality of elements, and the virtual image 302 is a part of a plurality of elements constituting the first predetermined image. The virtual image 301 is composed of a first element, and the virtual image 301 is composed of a second element other than the first element among the plurality of elements.

  Thus, by making a virtual image showing the shape of a person visible at the position of the pedestrian 200, the driver can be made aware of the presence of the pedestrian 200.

  Next, the display unit 104 generates a virtual image based on the control information output from the control unit 106 (step S180). Subsequently, the display unit 104 displays the generated virtual image on the display medium 130 (step S200).

  Thereafter, the display system 100 determines whether or not it is necessary to end the process (step S220). For example, the display system 100 determines that the process needs to be terminated when the display system 100 is stopped in conjunction with the vehicle engine being switched from an on state to an off state, while the vehicle engine is in an on state. If it is not in the off state, it is determined that the process need not be terminated.

  As a result of the determination, when it is necessary to end the process (step S220, YES), the display system 100 ends the process in FIG. On the other hand, when it is not necessary to end the process (step S220, NO), the process after step S100 is executed again. Thereafter, when a virtual image is displayed in step S200, a virtual image different from the virtual image displayed in the previous step S200 is displayed. Specifically, the left and right virtual images obtained by dividing the plurality of elements in a different way from the way of dividing the plurality of elements constituting the first predetermined image into the virtual image 301 and the virtual image 302 are shown. Is displayed. This point will be described in detail with reference to FIG.

  In FIG. 9, 401 and 402 are virtual images displayed at the left position 214 and the right position 216 on the display medium 130, respectively. Reference numeral 403 denotes a virtual image that is visible to the left of the pedestrian 200 when the driver sees the virtual image 401 (corresponding to the “second left image” of the present invention) displayed on the display medium 130 with the right eye 230. . Reference numeral 404 denotes a virtual image that is visible in the right direction of the pedestrian 200 when the driver views the virtual image 402 (corresponding to the “second right image” of the present invention) displayed on the display medium 130 with the left eye 220. . The virtual images 401 and 402 are preferably displayed with the same color and brightness. When the driver gazes at the pedestrian 200, a virtual image other than the virtual image 403 among the double images of the virtual image 401 and a virtual image other than the virtual image 404 among the double images of the virtual image 402 are displayed. It is a virtual image of a person's shape that appears to be combined in position.

  Here, the virtual image 401 and the virtual image 402 are a second predetermined image composed of a plurality of elements, and the virtual image 402 is a part of the plurality of elements constituting the second predetermined image. And the virtual image 401 is comprised from 4th elements other than a 3rd element among the said several elements from the 3rd element different from a 1st element. That is, the virtual images 401 and 402 are different images composed of elements different from the virtual images 301 and 302, respectively.

  As described above, in the present embodiment, virtual images 301 and 401 are displayed at the left position 214 on the display medium 130 and virtual images 302 and 402 are displayed at the right position 216 on the assumption that double images are generated. doing. Further, in the present embodiment, the virtual images displayed at the left position 214 and the right position 216 on the display medium 130 are changed as time passes. As a result, only the virtual images 305 and 405 superimposed on the position of the pedestrian 200 among the virtual images 303, 304, 305, 403, 404, and 405 are clearly displayed to the driver. The presence of a pedestrian can be accurately grasped. This point will be described in more detail below.

  FIG. 10 shows two virtual images displayed at the left position 214 and the right position 216 of the display medium 130 every moment, a virtual image seen in the left direction of the pedestrian 200, a virtual image seen superimposed on the position of the pedestrian 200, And the relationship with the virtual image seen to the right direction of the pedestrian 200 is shown. As shown in FIG. 10, the position and number of elements constituting the virtual image that appears to be superimposed on the position of the pedestrian 200 does not change significantly with the passage of time, and the virtual image is clearly visually recognized by the driver. On the other hand, with the passage of time, the position and the number of elements constituting the virtual image that can be seen in the left and right directions of the pedestrian 200 change greatly at a high speed, so that the virtual image is visually recognized by the driver. That is, since only the virtual image displayed superimposed on the position of the pedestrian 200 is clearly seen by the driver, the driver can accurately grasp the presence of the pedestrian.

  Here, it is generally said that a critical fusion frequency (CFF: Critical Flicker Frequency) that is a limit frequency of light and dark that can be perceived by humans is 30 to 40 [Hz]. Accordingly, the time interval for displaying the two virtual images at the left position 214 and the right position 216 of the display medium 130 is the reciprocal of the frequency equal to or higher than the critical fusion frequency, so that the driver can move in the left and right directions of the pedestrian 200. Can prevent the virtual image from being recognized.

  In the above embodiment, an example in which two virtual images are displayed once on the display medium 130 each time a pedestrian is recognized as an object has been described. However, a pedestrian is recognized once as an object. Each time, two virtual images may be switched and displayed on the display medium 130 N times (N is a natural number greater than 1). The operation of the display system 100 in this case will be described with reference to FIG. FIG. 11 is a flowchart showing a modified example of the operation of display system 100 according to the present embodiment. The process in step S300 is started when the display system 100 is activated in conjunction with, for example, the vehicle engine changing from an off state to an on state.

  First, the recognition unit 101 extracts a feature amount included in a foreground image acquired from the camera 110, and recognizes an object by performing a pattern matching process or the like on the extracted feature amount (step S300). Then, the recognizing unit 101 outputs object position information including the position and type of the object to the specifying unit 102. Further, the recognition unit 101 outputs contour information indicating the contour of the object to the control unit 106 of the display control device 103.

  Next, the specifying unit 102 determines whether or not the type of the object included in the object position information output from the recognition unit 101 is a pedestrian (step S320). As a result of this determination, when the type of the object is not a pedestrian (step S320, NO), the process transitions to step S500. On the other hand, when the type of the object is a pedestrian (YES in step S320), the specifying unit 102 uses the position of a predetermined point in the driver's eye area and the object position information output from the recognition unit 101. From the relationship between the position of the pedestrian (object) to be displayed and the position of the display medium 130, the object display position that is the position on the display medium 130 where the pedestrian is displayed is specified (step S340). Then, the specifying unit 102 outputs object display position information indicating the object display position to the determining unit 105 of the display control device 103.

  Next, the determination unit 105 determines a predetermined distance from the object display position indicated by the object display position information output from the specifying unit 102 on the display medium 130 based on the recognition result of the object by the recognition unit 101. A right position that is far away from the target position and a left position that is substantially symmetrical to the right position with respect to the object display position are determined (step S360). Then, the determination unit 105 outputs left and right position information indicating the determined left side position and right side position to the control unit 106.

  Based on the contour information output from the recognition unit 101 and the left and right side position information output from the determination unit 105, the control unit 106 generates control information indicating the content of control for generation and display of the virtual image, The control information is output to the display unit 104.

  Next, the display unit 104 generates two virtual images as first virtual images based on the control information output from the control unit 106 (step S380). Next, the display unit 104 displays the generated first virtual image on the display medium 130 (step S400).

  Next, the display unit 104 generates two virtual images as second virtual images based on the control information output from the control unit 106 (step S420). Next, the display unit 104 displays the generated second virtual image on the display medium 130 (step S440). Thereafter, the display unit 104 repeats the process of generating two virtual images and displaying them on the display medium 130 based on the control information output from the control unit 106. Then, the display unit 104 generates two virtual images as the Nth virtual image based on the control information output from the control unit 106 (step S460). Next, the display unit 104 displays the generated Nth virtual image on the display medium 130 (step S480). In the processes of steps S400 to S480, the virtual images displayed at the left position 214 and the right position 216 on the display medium 130 are changed every moment.

  Finally, the display system 100 determines whether it is necessary to end the process (step S500). As a result of the determination, when it is necessary to end the process (step S500, YES), the display system 100 ends the process in FIG. On the other hand, when it is not necessary to end the process (step S500, NO), the process returns to before step S300.

  In the above-described embodiment, the virtual image 301 including elements that generate the virtual image at the left side position 214 and the right side position 216 of the display medium 130 so that the virtual image of the human shape can be seen at the position of the pedestrian 200. , 302, 401, 402 has been described, but the present invention is not limited to this.

  For example, as shown in FIG. 12, a predetermined image composed of a plurality of fan-shaped or semi-circular elements may be displayed so that a circular virtual image can be seen at the position of the pedestrian 200. The predetermined image includes a virtual image displayed at the left position 214 of the display medium 130 and a virtual image displayed at the right position 216. The virtual image displayed at the right position 216 includes a part of a plurality of elements constituting the predetermined image, and the virtual image displayed at the right position 216 includes the above-described plurality of elements. Consists of other elements other than some elements.

  Further, as shown in FIG. 13, a predetermined image composed of a plurality of elements that are a quadrangle or a quadrangle whose corners are rounded so that a rectangular frame-shaped virtual image can be seen at the position of the pedestrian 200 is displayed. You may make it make it. This predetermined image also includes a virtual image displayed at the left position 214 of the display medium 130 and a virtual image displayed at the right position 216. The virtual image displayed at the right position 216 includes a part of a plurality of elements constituting the predetermined image, and the virtual image displayed at the right position 216 includes the above-described plurality of elements. Consists of other elements other than some elements.

  The functions of the units of the display system 100 and the display control apparatus 103 described above can be realized by a computer program.

  FIG. 14 is a diagram illustrating a hardware configuration of a computer that realizes the functions of the respective units by a program. The computer 1000 includes an input device 1001 such as an input button and a touch pad, an output device 1002 such as a display and a speaker, a CPU (Central Processing Unit) 1003, a ROM (Read Only Memory) 1004, and a RAM (Random Access Memory) 1005. . The computer 1000 is a reading device that reads information from a recording medium such as a hard disk device, a storage device 1006 such as an SSD (Solid State Drive), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. 1007 includes a transmission / reception device 1008 that performs communication via a network. Each of the above parts is connected by a bus 1009.

  Then, the reading device 1007 reads the program from a recording medium on which a program for realizing the functions of the above-described units is recorded, and stores the program in the storage device 1006. Alternatively, the transmission / reception device 1008 communicates with a server device connected to the network, and causes the storage device 1006 to store a program for realizing the function of each unit downloaded from the server device.

  Then, the CPU 1003 copies the program stored in the storage device 1006 to the RAM 1005, and sequentially reads and executes the instructions included in the program from the RAM 1005, thereby realizing the functions of the above-described units. Further, when executing the program, the RAM 1005 or the storage device 1006 stores information obtained by the various processes described in the above embodiment, and is used as appropriate.

  INDUSTRIAL APPLICABILITY The present invention is useful for a display control program and a display control apparatus that can cause an occupant to accurately grasp the presence of an object.

DESCRIPTION OF SYMBOLS 100 Display system 101 Recognition part 102 Identification part 103 Display control apparatus 104 Display part 105 Determination part 106 Control part 110,120 Camera 130 Display medium 200 Pedestrian 212 Object display position 214 Left position 216 Right position 220 Left eye 225 Center 230 Right eye 301 , 302, 401, 402 Virtual image 303, 304, 305, 403, 404, 405 Virtual image 1000 Computer 1001 Input device 1002 Output device 1003 CPU
1004 ROM
1005 RAM

Claims (4)

A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object and recognizes a predetermined object included in the foreground image;
A predetermined area of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the occupant's eye area through a display medium that transmits light. A specific part for specifying the position;
A display unit that generates a predetermined image based on the recognition result of the recognition unit and displays the predetermined image on the display medium;
A display control program to be executed by a computer in a display system including:
In the computer,
When the predetermined object is recognized by the recognition unit, based on the recognition result, on the display medium, a right side position separated from the predetermined position specified by the specifying unit by a predetermined distance; and Determining a left side position that is substantially symmetrical to the right side position with respect to the predetermined position;
When displayed on the display medium, the first predetermined image indicating the first right image at the right position and the first left image at the left position is displayed at a first timing,
When displayed on the display medium, a second predetermined image showing a second right image at the right position and a second left image at the left position is different from the first timing. The process of controlling the display unit to execute display at the timing of 2,
Each of the first and second predetermined images is an image composed of a plurality of elements,
The first right image is composed of a first element that is a part of the plurality of elements,
The first left image is composed of a second element other than the first element among the plurality of elements,
The second right image is a part of the plurality of elements, and includes a third element different from the first element.
The second left image is composed of a fourth element other than the third element among the plurality of elements.
Display control program. A recognition unit that acquires a foreground image that is a foreground image of an occupant of a moving object and recognizes a predetermined object included in the foreground image;
A predetermined area of the predetermined object on the display medium when the predetermined object is observed from a predetermined point in the occupant's eye area through a display medium that transmits light. A specific part for specifying the position;
A display unit that generates a predetermined image based on the recognition result of the recognition unit and displays the predetermined image on the display medium;
A display control apparatus in a display system including:
When the predetermined object is recognized by the recognition unit, based on the recognition result, on the display medium, a right side position separated from the predetermined position specified by the specifying unit by a predetermined distance; and Determining a left side position that is substantially symmetrical to the right side position with respect to the predetermined position;
When displayed on the display medium, the first predetermined image indicating the first right image at the right position and the first left image at the left position is displayed at a first timing, and the display A second timing different from the first timing is displayed when the second predetermined right image is displayed at the right position and the second left image is displayed at the left position when displayed on the medium. A control unit for controlling the display unit to display at
With
Each of the first and second predetermined images is an image composed of a plurality of elements,
The first right image is composed of a part of first elements among the plurality of elements,
The first left image is composed of a second element other than the first element among the plurality of elements,
The second right image is composed of a part of the plurality of elements and a third element different from the first element,
The second left image is composed of a fourth element other than the third element among the plurality of elements.
Display control device. The time interval between the first timing and the second timing is the reciprocal of a frequency equal to or higher than a critical fusion frequency.
The display control apparatus according to claim 2. The predetermined object is an object being watched by the occupant,
The recognizing unit acquires the occupant's viewpoint direction information, and recognizes the predetermined object based on the acquired viewpoint direction information;
The display control apparatus according to claim 2 or 3.