JP2017126009A - Display control device, display control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique, in displaying information on an object present in a real space, to suppress a reduction in visibility.SOLUTION: Display control means controls display means which displays an additional image and enables a user to simultaneously visually recognize a real space and the additional image. Setting means sets part of an area that can be displayed by the display area as a restriction area to restrict the display of the additional image. The display control means restricts, in the restriction area, the display of the additional image related to an object in a real space corresponding to the restriction area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control technique for controlling the display of a display device that allows a user to visually recognize a real space around the user and information that is not included in the real space around the user.

  A display device having a function of displaying information at a position overlapping with a part of the real space of the user's visual field range is becoming widespread. For example, a display device that uses a windshield of an automobile as a projection surface and displays information for a driver such as speed display on the windshield as a projection surface is known. The driver can view the real space that is visible through the windshield as the projection plane and the information displayed on the projection plane at the same time.

  A wearable display device that can be worn and used by a user is also known. For example, a display device of glasses type or helmet type. This wearable display device superimposes and displays information on a background that is a real space on the wearer's field of view. The user can view the information simultaneously with the background that is the real space.

  In this way, a display device in which the user visually recognizes both the background and the information at the same time by superimposing information on the real space on the field of view is referred to as a head-up display (HUD). Of the head-up displays, a display device that is used by being mounted on the head is particularly called a head-mounted display (HMD).

  A display device that superimposes information on the real space in the field of view and allows the user to view both the object and the information on the real space simultaneously. There is a problem that the visibility of an object is lowered.

  Conventionally, in such a display device, a technique for improving visibility and reducing fatigue has been proposed by appropriately displaying information. Patent Document 1 discloses a technique for correcting a position where information is displayed based on the behavior of a vehicle. Patent Document 2 discloses a technique for determining an object whose information is to be displayed on a background image that can be seen through a projection plane based on a user's line-of-sight direction and vehicle speed.

  However, the technique disclosed in Patent Document 1 determines the display position of information based on the behavior of the vehicle, and does not use the behavior of the user, for example, the information on the place where the user is gazing. Therefore, there is a possibility of displaying information at a position different from the user's line-of-sight direction. Information displayed at a position different from the direction of the user's line of sight has a problem in that the visibility is lowered and the movement of the line of sight is also required, which increases the fatigue of the user.

  In addition, the technique disclosed in Patent Document 2 does not designate a position for displaying information. For this reason, there is a problem in that information is displayed at a position on an object that is likely to be noticed by the user in the vicinity of the line-of-sight direction, causing a reduction in visibility.

Japanese Patent Application Laid-Open No. 2015-077786 Japanese Patent Laying-Open No. 2015-080988

  The present invention has been made to solve the above problems. In a display device in which information is superimposed on a real space and a user visually recognizes both the real space and the information at the same time, an object is to reduce a decrease in visibility.

  As a means for solving the above problems, the display control apparatus of the present invention has the following configuration. That is, a display control unit that controls a display unit capable of displaying an additional image and allowing a user to view the real space and the additional image at the same time, and a partial area of the displayable region in the display unit, Setting means for setting as a restricted area for restricting the display of the additional image, and the display control means restricts the display of the additional image relating to the object in the real space corresponding to the restricted area in the restricted area.

  ADVANTAGE OF THE INVENTION According to this invention, the fall of visibility can be reduced in the display apparatus which superimposes information on real space and a user visually recognizes both real space and information simultaneously.

It is a functional block diagram which shows 1st Embodiment of this invention. It is a figure explaining the coordinate system which each embodiment of the present invention uses. It is a figure explaining the gaze vector in each embodiment of the present invention. It is a flowchart explaining the process which extracts the object in each embodiment of this invention. It is a figure explaining the object in each embodiment of the present invention. It is a figure explaining the example of a display in each embodiment of the present invention. It is a functional block diagram which shows 2nd Embodiment of this invention. It is a figure explaining the setting method of the area | region in 2nd Embodiment of this invention. It is a figure explaining the area | region set in 2nd Embodiment of this invention. It is a functional block diagram which shows 3rd Embodiment of this invention. It is a figure which shows operation | movement of the display apparatus corresponding to the command of the setting process of a restriction | limiting area | region according to 3rd Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the display apparatus 5 and the display control apparatus 10 in each embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the embodiments and details can be changed without departing from the spirit and scope of the present invention. Therefore, embodiments of the present invention are not limited to the description. In addition, the embodiments described below may be implemented on hardware or software unless otherwise specified. In addition, the same code | symbol is attached | subjected in principle to the member which has the same function, and the repeated description is abbreviate | omitted.

  In addition, a display device that superimposes information in the real space on the field of view according to each embodiment and allows the user to visually recognize both the background and the information at the same time is related to the present invention unless there is a possibility of confusion in the following description. It is described as a display device. Further, the user of the display device according to each embodiment is described as a user unless there is a possibility of confusion in the following description.

  Prior to describing the embodiment of the present invention, a coordinate system used in the following description will be described with reference to FIG. A visual plane 21 shown in FIG. 2 is a virtual display surface defined at a position where information displayed by the display device according to each embodiment of the present invention is imaged. The viewing plane 21 is defined within the visual field range of the user. The coordinate system used in the following description is a right-handed system in which the center of the viewing plane 21 is the origin (0, 0, 0) and the direction perpendicular to the viewing plane 21 is the z-axis. And as for the direction of az axis, let the direction away from a user's eyeball be a positive direction.

  In the following description, the display device according to each embodiment of the present invention may allow the user to directly view the real space in order to make the user visually recognize the background, or the user can view an image of the real space. It may be visually recognized. That is, in each embodiment, the “real space” may be the real space itself, or an image obtained by capturing the real space around the user is displayed by the display device according to each embodiment of the present invention. It may be displayed at a position corresponding to.

“Information” is a display object displayed by the display device according to each embodiment of the present invention, and is, for example, text, graphics, a still image, or a moving image. The “information” may be a combination thereof. Further, “information” displayed by the display unit 5 described below is an example of “additional image”.
<First Embodiment>
The operation of the display device 5 in the first embodiment and the display control device 10 that controls the display of the display device 5 will be described with reference to FIG. Here, the display control apparatus 10 includes a setting unit 1, a target extraction unit 2, an information acquisition unit 3, and a display control unit 4.

  The setting unit 1 defines a restriction area 32 that restricts display of information that is an additional image on the viewing plane 21. The target extraction unit 2 extracts a target (object) existing in the restricted area 32. Here, the target (object) is, for example, a building or a product. The information acquisition unit 3 acquires information related to a target existing in the restricted area 32 extracted by the target extraction unit 2. The display control unit 4 sets the display position of information that is an additional image related to the target existing in the restricted area 32 acquired by the information acquisition unit 3 to a position that satisfies the following four conditions A to D. . The conditions B and C do not necessarily have to be satisfied, but if the conditions B and C are satisfied, the visibility of information can be further improved.

Condition A Outside the restricted area 32 Condition B Adjacent to the restricted area 32 Condition C Display of each information is not superimposed Condition D Displayable area of the display unit 5 The display unit 5 is located at the display position set by the display control unit 4 at the information acquisition unit. 3 displays information on the target existing in the restricted area 32 acquired.

  Next, the operation of the setting unit 1 will be described in detail. In the present embodiment, the operation of the setting unit 1 will be further described by taking as an example a method for setting the restriction region 32 based on the user's line-of-sight direction. Here, the user's line-of-sight direction is represented by a vector, and is defined as a line-of-sight vector 101.

  FIG. 3 shows the relationship between the restricted area 32 and the line-of-sight vector 101. The direction of the line-of-sight vector 101 is a direction from the eyeball 23 toward the viewing plane 21. A point where the line-of-sight vector 101 and the viewing plane 21 intersect is defined as an intersection 31. In the present embodiment, the shape of the restriction region 32 is a circle having a radius r with the intersection 31 as the center. Here, when the coordinates of the intersection point 31 are set as (xa, yb, 0), the vector equation representing the circle with the radius r centering on the intersection point 31 is expressed by Equation 1. Therefore, the restricted area 32 is an area represented by the definition formula shown in Equation 2.

The setting unit 1 of the present embodiment starts the setting process of the restricted area 32 based on a user instruction. Therefore, the display control apparatus 10 applies the restriction area 32 until the user newly instructs the setting start of the restriction area 32.

  The present embodiment does not depend on the shape of the restriction region 32. Accordingly, the shape of the restriction region 32 is not limited to a circle, and may be an ellipse, a rectangle, or an arbitrary shape, and may be appropriately changed according to the embodiment of the display control device 10.

  Since the restriction region 32 is set based on the line-of-sight vector 101, the setting unit 1 includes the line-of-sight vector detection unit 8 that detects and outputs the line-of-sight vector 101. Since this embodiment does not depend on the embodiment of the line-of-sight vector detection unit 8, the line-of-sight vector detection unit 8 can be implemented by applying a known technique. The same gaze vector detection technology as the scleral reflection method that detects the difference in reflectance between white and black eyes, the corneal reflection method that utilizes the difference between the center of curvature of the cornea and the center of rotation of the eyeball, and the same as the corneal reflection method There is a pupillary corneal reflection method in which the center of the pupil is extracted based on the above principle. Of these, the scleral reflection method is suitable for a head-mounted display mounted on the head because the detector is lightweight. On the other hand, the corneal reflection method is suitable for a head-up display having a high degree of freedom in the mounting position of the detector because a detector capable of capturing a moving image is required to detect the rotation of the eyeball.

  Next, the procedure by which the target extraction unit 2 extracts a target existing in the restricted area 32 will be described in detail using the flowchart shown in FIG.

  In step S101, an image of the real space that is visually recognized by the user is acquired, affine transformed, and projected onto the viewing plane 21. In the present embodiment, processing is performed to detect an edge from the real space image projected in the subsequent step S102 and divide it into regions, so that at least the real space image has a characteristic capable of detecting an edge. Good. Therefore, the real space video acquired in step S101 only needs to have at least luminance information. Of course, if the color signal is used for area division in step S102, it is necessary to include the color signal in the real space image to be acquired. The present embodiment does not depend on the configuration for acquiring the above-described real space video. Therefore, it is possible to implement a configuration in which a public space image is acquired by applying a known technique. For example, it is possible to realize a configuration in which luminance information in real space is acquired by a camera.

  In step S102, the image projected on the viewing plane 21 is divided into regions. In the present embodiment, the region is divided depending on the location where the luminance changes rapidly, that is, the edge. Accordingly, the luminance component of the projected image is subjected to a differential operation to detect an edge, and the detected edge is applied to perform region division. As a result of the area division, the number of areas obtained is held in a variable k. Further, at least the following two attributes are recorded for each area obtained by division. Of course, the attribute to be recorded is not limited to the following two items, and may be arbitrarily added.

a) Coordinates of vertices constituting regions obtained by division b) Unique region identifier added to each region obtained by division Loop 1 is constituted between step S103 and step S106. Variable i is a loop variable, and the value of variable i is updated one by one from 1 to the value held by variable k, and loop 1 is executed. In loop 1, the intersection with the restricted area 32 is determined for each area obtained by dividing in step S102.

  In step S104 of the loop 1, the intersection between the area i and the restricted area 32 is determined. If there is an intersection, the process branches to step S105. On the other hand, if there is no intersection between the region i and the restricted region 32, the value of the loop variable i is updated, and the loop 1 is executed again. In step S104, the intersection is determined by the following procedure.

S104-1) represents a line segment connecting a vertex in area i and a vertex adjacent to it
Calculate equation S
Let the coordinates of a vertex in region i be (xin, yin),
If the coordinates of adjacent vertices are placed as (xin + 1, yn + 1),
The equation S representing the line segment is Equation 3. S104-2) A coordinate point common to the calculated line segment and the restricted area 32 is obtained.
Calculated from equation S and definition formula 101 of restricted region 32
That is, the solution is obtained by combining Equation 3 and Equation S104-3) If there is a common coordinate point, it is determined that there is an intersection. S104-4) If there is no common coordinate point, it is determined that there is no intersection.

In step S105 of the loop 1, the area identifier of the area that intersects with the restricted area 32 is recorded.

  After the end of loop 1, the target extraction unit 2 outputs the area identifier recorded in step S105.

  Next, the relationship between the restriction region 32 and the object extracted by the object extraction unit 2 will be described with reference to FIG. FIG. 5A is a diagram showing the relationship between the real space visually recognized by the user and the restricted area 32 on the viewing plane 21. A restricted area 32 having a radius r is set around the intersection 31 of the line-of-sight vector 101 indicating the line of sight of the user and the viewing plane 21. FIG. 5B shows the result of extracting the target according to the flowchart of FIG. 4 with respect to the real space and the restricted area 32 visually recognized by the user shown in FIG. In FIG. 5B, the hatched object is the extracted object. As illustrated in FIG. 5B, the target extraction unit 2 extracts a target that intersects with the restricted area 32 as a target that exists in the restricted area 32.

  Next, the operation of the information acquisition unit 3 will be described in detail. In the present embodiment, the information acquisition unit 3 searches for information on the target extracted by the target extraction unit 2 via the network. Therefore, the information acquisition unit 3 has a function of connecting to a network. A network connection function can be implemented by applying a known technique. For example, network connection using a mobile phone network such as G3 communication may be applied, or network connection using a public wireless LAN may be applied.

  In the present embodiment, information is obtained by image search using the target image data extracted by the target extraction unit 2 as an index, keyword search using a search keyword extracted from the image data by applying a character recognition technique, and the like. You may search. Moreover, there is no restriction | limiting in application of the search engine used for a search. Anyone may apply a search website where anyone can freely search for information, and if the user has authority, this does not preclude the application of the member-only website.

  Next, the operation of the display control unit 4 will be described in detail. The display control unit 4 sets the display position of the information related to the target existing in the restricted area 32 searched by the information acquisition unit 3. The display position is outside the restricted area 32 and adjacent to the restricted area 32, and does not overlap the display of each information, and is a position within the displayable area of the display unit 5.

  Here, the number of objects existing in the restricted area 32 is m. For an object existing in the nth restricted area 32, an equation representing an area for displaying information on the object on the viewing plane 21 is defined as Pn. The display control unit 4 arranges Pn so that all Pn satisfy the following conditions 1 and 2.

Condition 1 {Pn | 1 <= n <= m} has a unique solution with Equation 1 Condition 2 {Pn | 1 <= n <= m} does not have a common solution with each other. This will be described in detail. The embodiment of the display device 5 differs depending on the method in which the user visually recognizes the real space of the background.

  First, if the user directly recognizes the real space in the background, the video display device 73 can implement the display device 5 by applying a transmissive liquid crystal panel, an organic EL panel, a Maxwell optical system, or a half mirror. .

  In an embodiment in which a transmissive liquid crystal panel or an organic EL panel is applied to the video display device 73, a video in which information displayed by the video display device 73 is superimposed on a real space of the background transmitted through the video display device 73 is displayed on the user. Is visually recognized. Therefore, the video display device 73 needs to have a sufficient transmittance to visually recognize the real space of the background.

  In the display device 5 to which the Maxwell optical system is applied, the real space of the background is visually recognized by the user without passing through the video display device 73 or the half mirror. The information displayed by the video display device 73 directly reaches the user's retina. Therefore, in this embodiment, the type of the video display device 73 is not limited as long as an optical system capable of Maxwell's view can be configured.

  In the case of the display device 5 to which the half mirror is applied, the user visually recognizes a video in which the real space of the background transmitted through the half mirror and the information displayed by the video display device 73 reflected by the half mirror are superimposed. Therefore, in this embodiment, an image display device having sufficient luminance to be visually recognized by the user even after being reflected by the half mirror is applied. For example, a liquid crystal panel, an organic EL panel, a laser projector, or the like can be applied.

  On the other hand, if the method allows the user to visually recognize an image captured in real space, the image display device does not need to be a reflection type and does not need to be able to perform Maxwell's view. Therefore, it can be implemented by applying a liquid crystal panel, an organic EL panel, a laser projector, or the like.

  Next, an embodiment to which the display device according to this embodiment is applied will be described with reference to FIG. FIG. 6A is a diagram illustrating an embodiment of an image visually recognized by the user when the display control unit 4 does not set a position for displaying information. As is clear from FIG. 6A, the display of information is superimposed, and the visibility is greatly deteriorated. FIG. 6B is a diagram illustrating an embodiment of an image visually recognized by the user when the display control unit 4 sets a position for displaying information. As shown in FIG. 6 (b), the display control unit 4 displays the information 103 related to the object in the restricted area 32 outside the restricted area 32 and adjacent to the restricted area 32, without overlapping each information. It is displayed at a position in the displayable area of the display unit 5.

  Therefore, according to the display device in the first embodiment, it is possible to display the information 103 related to the objects in all the restricted areas 32 at a position where the user can easily recognize.

Second Embodiment
Next, a second embodiment will be described with reference to FIG. Note that description of the same functions as those of the other embodiments is omitted.

  In the second embodiment, the locus of the pointing tool is used to define the shape and position of the restriction region 32. Here, the pointing tool is a user's body such as a finger or an arm, or a pen-shaped or bar-shaped instrument. Hereinafter, unless there is a possibility of confusion, the movement locus of the pointing tool is described as a locus.

  The setting unit 1 in the second embodiment acquires a trajectory, and sets the shape and position of the restriction region 32 based on the acquired trajectory. The format of the information representing the locus may be arbitrary. For example, the vector data format uses coordinate values on the viewing plane 21. Since this vector data format is defined on the viewing plane 21, the value of the z coordinate is always 0. Further, the setting unit 1 outputs information representing the acquired trajectory to the display control unit 4. The display control unit 4 displays a locus 52 on the viewing plane 21 based on the information.

  Next, the operation of the setting unit 1 in the second embodiment will be described in detail using a user's finger as an indicator and also using FIG.

  FIG. 8 shows a real space visually recognized by the user. As shown in FIG. 8A, the user is pushing the fingertip 51 from the right direction to the front surface of the display control device 10. Here, when the user instructs to start acquiring a locus by pressing a button (not shown) or by a predetermined operation by the fingertip 51, the setting unit 1 starts acquiring the locus.

  The setting unit 1 recognizes the fingertip 51 from the video captured by the real space imaging unit 9 by a method such as pattern matching, and acquires the locus thereof. Then, the trajectory is projectively converted to the viewing plane 21 and recorded in the vector data format. Since the z component is always 0 by projective transformation to the viewing plane 21, only the x coordinate value and the y coordinate value after the projective transformation need be recorded. Further, a procedure for the setting unit 1 to recognize that the fingertip 51 is the pointing tool may be performed before starting the acquisition of the trajectory. For example, the user instructs the setting unit 1 to start the recognition procedure of the pointing tool depending on pressing of a button (not shown) or a predetermined operation by the fingertip 51.

  The setting unit 1 sends the acquired trajectory information to the display control unit 4 and displays the trajectory 52 on the viewing plane 21. By this display, the user can check the locus 52, and the shape and position of the restricted area 32 can be easily set.

  When the setting unit 1 detects that the region where the locus is closed is configured, the acquisition of the locus is terminated. Note that the acquisition of the trajectory may be ended even if the start point and the end point are not separated to form a closed region. If an example is given, you may complete | finish when it becomes a clearance below a predetermined threshold value.

  After completing the acquisition of the locus, the setting unit 1 defines the shape and position of the restriction region 32 based on the closed region formed by the locus, and sets the restriction region 32. Here, after completing the acquisition of the trajectory, a procedure for confirming with the user whether or not to apply the closed region formed by the trajectory to the setting of the restricted region 32 may be performed. For example, a menu including a predetermined message is displayed, and the user selects applicability. If the user selects application, the setting unit 1 sets the restricted area 32 by applying the acquired trajectory. On the other hand, if the user selects non-application, the setting unit 1 discards the acquired trajectory and waits for an acquisition start instruction for the trajectory.

  FIG. 9 is a diagram illustrating a closed region 53 formed by a trajectory and a restricted region 32 set by the second embodiment. The setting unit 1 according to the second embodiment sets a rectangular area circumscribing the closed area 53 formed by the locus as the restriction area 32. A rectangular area circumscribing the closed area 53 formed by the locus is calculated from the acquired vector data of the locus from the maximum value of the x coordinate, the minimum value of the x coordinate, the maximum value of the y coordinate, and the minimum value of the y coordinate. To do. In the present embodiment, the shape of the restriction region 32 is a rectangle, but the shape of the restriction region 32 is not limited to a rectangle, and may be a circle, an ellipse, or a polygon. Alternatively, the closed area 53 may be used as the restricted area 32 as it is.

  The setting unit 1 may acquire the locus of the pointing tool using a method other than pattern matching. For example, if a pen-shaped or bar-shaped instrument is used as the pointing device, a magnet or an infrared LED is incorporated into the instrument, and the locus of the instrument is obtained by detecting a change in the magnetic field or a light trace. It may be.

  Therefore, according to the second embodiment, it is possible to set the restricted region 32 having a shape desired by the user at a position desired by the user.

<Third Embodiment>
Next, a third embodiment will be described. In addition, description is abbreviate | omitted about the part similar to other embodiment.

  In the third embodiment, the limited region 32 is continuously and periodically performed based on the line-of-sight vector 101 without depending on a user instruction. When the setting unit 1 of the third embodiment detects a change in the line-of-sight vector 101, the setting unit 1 sets the restriction region 32 again. Therefore, the position of the restriction region 32 changes with the change of the line-of-sight vector 101, the target existing in the restriction region 32 extracted by the target extraction unit 2 also changes, and the restriction unit 32 acquired by the acquisition unit 3 also changes. Information on existing objects also changes. Therefore, in the third embodiment, the content of information displayed on the display device 5 and the display position of the information change following the change in the line-of-sight vector 101.

  However, the setting unit 1 according to the third embodiment may change the line-of-sight vector 101 if any one of the following three conditions A to C is satisfied, or if a plurality of conditions are satisfied at the same time. The setting of the restricted area 32 is not updated without watching.

Condition A The change of the line-of-sight vector 101 occurs at a time interval less than a predefined time. Condition B The movement range of the line-of-sight vector 101 is within the restricted area 32 and the display range of the information 103. Condition C The line-of-sight vector 101 is within the restricted area 32 and After moving out of the information 103 area,
Returning to the range of the restriction area 32 and the information 103 within a predefined time period Hereinafter, embodiments of the above conditions will be described in detail.

  The time defined by condition A is t seconds. The setting unit 1 of the third embodiment detects a change in the line-of-sight vector 101 at intervals of t seconds. Since the detection interval is t seconds, the setting unit 1 of the third embodiment does not detect a change in the line-of-sight vector 101 that occurs at an interval of less than t seconds. Therefore, the update interval of the restriction area 32 is t seconds or more, and the content of information displayed on the display device 5 and the update interval of the display position are also t seconds or more.

  The condition B is a condition for suppressing the update of the restriction area 32 if the user is paying attention to the object in the restriction area 32 or if the user is referring to the information 103.

  Condition C also adds to Condition B a condition that suppresses updating of the restriction area 32 if the line-of-sight vector 101 moves outside the restriction area 32 and the information 103 and returns within a predefined time period. Depending on this additional condition, the condition C suppresses the update of the restricted area 32 more strongly than the condition B.

  Note that the conditions for suppressing the update of the setting of the restriction area 32 are not limited to the above conditions A to C. The conditions may be arbitrarily defined according to the embodiment of the display control apparatus 10.

  As described above, according to the third embodiment, it is possible to set the restriction region 32 adapted to changes in the user's line of sight. Further, if a specific condition is satisfied, it is possible to suppress unnecessary setting update of the restricted area 32.

<Fourth embodiment>
Next, a fourth embodiment will be described with reference to FIG. In addition, description is abbreviate | omitted about the part similar to other embodiment.

  A display device 5 shown in FIG. 10 is a wearable display device, and includes a GPS (Global Positioning System) receiver 11, an acceleration sensor 12, a gyro sensor 13, and a geomagnetic sensor 14.

  The setting unit 1 of the fourth embodiment controls the setting process of the restricted area 32 by the operation of the display device 5. A command for setting the restricted area 32 and the operation of the display device corresponding to the command are defined below in advance. An example of the command and the operation of the display device is shown in FIG.

a) Start of setting of the limited area 32. The display device 5 is moved up and down once in a predetermined speed range. b) The setting of the limited area 32 is released. The display device 5 is rotated counterclockwise in the predetermined speed range. c) Maintaining setting Rotate display device 5 clockwise in a predetermined speed range d) Non-display of information 103 Display device 5 reciprocates left and right in a predetermined speed range The operation of display device 5 includes acceleration sensor 12 and gyroscope The sensor 13 is used for detection. The setting unit 1 of the fourth embodiment determines whether or not the detected operation of the display device 5 is an operation of the display device corresponding to a command for setting the restriction area 32 defined in advance. A setting process of the restricted area 32 corresponding to the command is performed.

  Next, operation | movement of the object extraction part 2 of 4th Embodiment is demonstrated. The target extraction unit 2 of the fourth embodiment extracts a target by the following four procedures from S201 to S204.

  In step S201, the global coordinate system coordinate value of the position where the display device 5 exists and the display device 5 are directed from the output of the GPS receiver 11, the acceleration sensor 12, the geomagnetic sensor 14, and the gyro sensor 13 as necessary. Direction is calculated.

  In step S202, a vector equation representing the viewing plane 21 defined on the global coordinate system is calculated from the global coordinate system coordinate value of the display device 5 calculated in step S201 and the direction in which the display device 5 is directed.

  In step S <b> 203, the range of the global coordinate system that the user can project to the viewing plane 21 is calculated from the global coordinate system coordinate value of the display device 5 and the vector equation of the viewing plane 21.

  Step S204 searches the map information for objects that exist in the range of the global coordinate system that can be projected onto the viewing plane 21 calculated in step S204. The map information to be searched may be held in the display control device 10 of the fourth embodiment, or may be searched from the network via the information acquisition unit 3 of the fourth embodiment. When searching for an object, an object that cannot be visually recognized is not extracted from the position where the display device 5 is present, for example, due to the shape and size of the object and being in the shadow of another object.

  As described above, according to the fourth embodiment, the user can more easily control the setting of the restricted area 32 and the display of the information 103 depending on the sensors included in the display device 5. Further, according to the fourth embodiment, it is possible to extract the target with higher accuracy by referring to the map information.

<Fifth Embodiment>
Next, the hardware configuration of the display control device 10 and the display device 5 will be described with reference to FIG. FIG. 12 is an embodiment in which the display control device 10 and the display device 5 are separate housings and communicate with each other.

  First, the hardware configuration of the display device 5 will be described. For example, the communication interface 74 is compliant with a standard such as a wireless LAN and functions as a communication unit.

  A RAM (Random Access Memory) 72 is a memory that stores a computer program executed by a CPU (Central Processing Unit) 71 and data acquired from the outside via a communication interface 74. The RAM 72 also functions as a work area and buffer memory used by the CPU 71 when executing the program, and also as a frame memory for storing video to be displayed.

  The CPU 71 controls the display device 5 by executing a computer program stored in the RAM 72. In addition to the CPU, a processor such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) may be used.

  The video display device 73 is a display element such as a liquid crystal panel, an organic EL panel, or a laser projector.

  A GPS (Global Positioning System) receiver 11 receives signals from a plurality of GPS satellites and outputs position information on global coordinates based on the signals. This position information includes information indicating altitude.

  The acceleration sensor 12 detects acceleration in the x-axis, y-axis, and z-axis three directions. In the present embodiment, the x-axis is the left-right direction of the wearable display device 5, the y-axis is the up-down direction of the wearable display device 5, and the z-axis is the front-back direction of the wearable display device 5. The acceleration sensor 12 can detect and output the tilt together by detecting gravitational acceleration on each of the x-axis, y-axis, and z-axis.

  A bus 75 is a bus that connects hardware such as the CPU 71.

  Next, the hardware configuration of the display control apparatus 10 will be described.

  For example, the communication interface 84 conforms to a standard such as a wireless LAN and functions as a communication unit.

  The RAM 82 is a memory that stores a computer program executed by the CPU 81 and data acquired from the outside via the communication interface 74. The RAM 82 also functions as a work area and a buffer memory that the CPU 81 uses when executing a program.

  The CPU 81 executes the functions of the setting unit 1, the target extraction unit 2, the information acquisition unit 3, and the display control unit 4 by executing a computer program stored in the RAM 82. In addition to the CPU, each function unit including the setting unit 1 may be implemented using a processor such as a DSP or an ASIC.

  An HDD (Hard Disk Drive) 83 stores an operating system program and data. The HDD 83 stores a computer program for causing the CPU 81 to realize the functions of the respective units. Instead of the HDD 83, an SSD (Solid State Drive) may be used.

  Computer programs and data stored in the HDD 83 are appropriately loaded into the RAM 82 and executed by the CPU 81 under the control of the CPU 81.

  A bus 85 is a bus connecting the hardware.

  The camera 86 and the camera 87 correspond to the eyeball imaging unit 8 and the real space imaging unit 9. The camera 86 and the camera 87 have an image sensor, a lens, and a signal processing circuit. The image sensor receives light imaged through the lens and converts the received light into electric charges to generate an image signal. For example, a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) can be used as the imaging element. In addition, a CCD image sensor (Charge Coupled Device Image Sensor) may be used as the imaging device. The signal processing circuit digitizes the imaging signal converted by the imaging element to generate captured image data. The signal processing circuit may compression-encode image data and generate compression-encoded captured image data.

  In the present invention, a program that realizes one or more functions of each embodiment is supplied to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read the program. It can also be realized by processing to be executed. It can also be realized by a circuit that realizes one or more functions (for example, an ASIC). In addition, the information is displayed outside the restricted area 32, but may be as follows. That is, when information is displayed in the restricted area 32, the information may be displayed in a translucent display form.

DESCRIPTION OF SYMBOLS 1 Setting part 2 Object extraction part 3 Information acquisition part 4 Display control part 5 Display apparatus 10 Display control apparatus 32 Restriction area 101 Line-of-sight vector

Claims (13)

Display control means for controlling the display means capable of displaying the additional image and allowing the user to view the real space and the additional image at the same time;
Setting means for setting a partial area of the displayable area in the display means as a restriction area for restricting the display of the additional image;
The display control device, wherein the display control unit limits display of an additional image related to an object in a real space corresponding to the restricted area in the restricted area. The display control apparatus according to claim 1, wherein the display control unit displays an additional image related to an object corresponding to the restricted area in an area outside the restricted area. The setting means sets the restricted area based on at least one of the user's line of sight, the user's body trajectory, and the movement trajectory of the pointing tool by the user. The display control apparatus according to claim 1 or 2. The display control apparatus according to claim 1, wherein the setting unit sets the restricted area based on an instruction from the user. The display control apparatus according to claim 1, wherein the setting unit sets the restricted area based on a change in the line of sight of the user. The said setting means does not implement the setting of the said restriction | limiting area | region based on the change of the said user's eyes | visual_axis when the change of the said user's eyes | visual_axis generate | occur | produces in the time interval less than predetermined time. 5. The display control device according to 5. The said setting means does not implement the setting of the said restriction | limiting area | region based on the change of the said user's eyes | visual_axis, when the change of the said user's eyes | visual_axis is the change of the eyes | visual_axis in the said restriction | limiting area | region. Or the display control apparatus of 6. When the change in the user's line of sight is a change in the line of sight to the area corresponding to the additional image displayed by the display unit, the setting unit sets the restriction area based on the change in the line of sight of the user. The display control device according to claim 5, wherein the display control device is not implemented. The setting unit is configured to change the line of sight of the user to an area outside the area corresponding to the restricted area and the additional image, and correspond to the restricted area and the additional image within a predetermined time. The display control device according to any one of claims 5 to 8, wherein when the line of sight changes to the area to be performed, the restriction area is not set based on a change in the line of sight of the user. Comprising obtaining means for obtaining information about the object;
The display control apparatus according to claim 1, wherein the additional image is an image generated from information acquired by the acquisition unit. The display control apparatus according to claim 10, wherein the acquisition unit searches the information from an external storage device via a network. A display control step for controlling a display means capable of displaying an additional image and allowing a user to view the real space and the additional image at the same time;
A setting step of setting a partial area of the displayable area in the display means as a restriction area for restricting display of the additional image, and
In the display control step, display of an additional image related to an object in real space corresponding to the restriction area is restricted in the restriction area.   A program that causes a computer to function as each unit of the display control device according to any one of claims 1 to 11.