JP5517170B2 - Display device and control method - Google Patents

Description

  The present invention relates to a technique for displaying information on an object based on a photographed image.

  Conventionally, the object is identified by searching the database from the position where the captured image is observed in real time by the display means (current location) and the position of the object obtained from the direction and angle from that position. However, there is a technique for displaying information about the specified object (hereinafter also referred to as “related information”) on a display unit. For example, Patent Document 1 discloses a technique for displaying a captured image on a display unit, recognizing an object on a screen, and displaying related information in the vicinity of the object. Furthermore, Patent Document 1 discloses a technique for simplifying the display mode of related information when objects of the same type are densely packed. In addition, technologies related to the present invention are disclosed in Patent Literature 2 and Patent Literature 3.

JP 2010-009192 A JP 2008-216309 A JP 2010-066659 A

  When different types of objects are densely present, displaying related information corresponding to all objects in the vicinity makes it difficult for the user to know which related information corresponds to which object. In addition, when it is necessary to display related information at a position distant from the object due to the display space or the like, it is difficult to understand these correspondences.

  The present invention has been made to solve the above-described problems, and a main object of the present invention is to provide a display device that can easily recognize the correspondence between an object and related information.

The invention described in claim 1 is a captured image acquisition unit that acquires a captured image, an object specifying unit that specifies an object in the captured image, and a related information acquisition unit that acquires related information related to the object. the a recognizing characteristic information recognizing means the shape of the object based on the captured image, and display control means for displaying the relevant information about the object on the display means in association with the object, wherein the display The control means determines a font for displaying the related information based on the shape of the object .

The invention according to claim 7 is a control method executed by a display device including an imaging unit, a control unit, and a display unit, wherein the imaging unit acquires a captured image, and An object specifying step in which a control unit specifies an object in the captured image; a related information acquiring step in which the control unit acquires related information about the object; and the control unit based on the captured image A feature information recognition step for recognizing the shape of an object, and a display control step for causing the control unit to display related information related to the target object on the display unit in association with the target object . A font for displaying the related information is determined based on the shape of the object .

It is an example of the block block diagram of the display apparatus which concerns on a present Example. It is a figure which shows the outline | summary of the information registered into the database. (A) shows an example of a normal time image. (B) shows an example of a control time image. (A) shows a control-time video when character information is registered for A and B. (B) shows a control-time image when Mr. A and Mr. B are close to each other within a predetermined distance and the display colors are the same color. It is an example of the flowchart which shows the process sequence of a present Example.

  According to a preferred embodiment of the present invention, the display device acquires captured image acquisition means for acquiring a captured image, target object specifying means for specifying a target object in the captured image, and related information related to the target object. Related information acquisition means, feature information extraction means for extracting feature information of the object based on the photographed image, a display unit for displaying the related information, and display of the related information based on the feature information Display control means for determining an aspect.

  The display device includes a captured image acquisition unit, an object specifying unit, a related information acquisition unit, a feature information extraction unit, a display unit, and a display control unit. The captured image acquisition means acquires a captured image generated by a camera or the like. The object specifying unit specifies an object in the captured image. Here, the “object” is a specific type of object or person such as a person, a facility, or a car, and is a display target of related information. The related information acquisition unit acquires related information related to the identified object. The feature information extraction unit extracts feature information of the object based on the captured image. The display control means determines the display mode of the related information based on the extracted feature information. As described above, the display device determines the display mode of the related information based on the feature information of the target object in the captured image, thereby clarifying the correspondence between the target object and the related information, and displays the related information on the display unit. Can be displayed.

  In one aspect of the display device, the feature information is information related to a color of the object, and the display control unit determines a display color of the related information based on the feature information. By doing so, the display device can clarify the correspondence between the object and the related information.

  In another aspect of the display device, the feature information is information related to a shape of the object, and the display control unit determines the shape of the related information based on the feature information. Here, the “shape” includes the concept of size. Also according to this aspect, the display device can clarify the correspondence between the object and the related information.

  In another aspect of the display device, the feature information extraction unit extracts a plurality of characteristic colors characteristic of the object based on the captured image, and the display control unit extracts the related information, The related information is displayed on the display unit with a characteristic color different from the background color of the display position. By doing in this way, the display apparatus can suppress that related information becomes difficult to see due to the background around the object.

  In another aspect of the display device, when the feature information extraction unit cannot extract a feature color different from a background color of the display position of the related information, the display control unit The display color of the related information and the background color are changed to different positions. Also according to this aspect, the display device can suppress the related information from becoming difficult to see due to the background.

  In another aspect of the display device, the display control unit may display the related information when the objects are close to each other and the display modes of the related information of the objects are the same or approximate. Is determined based on the position where the other object is present. Here, “the display modes are the same or similar” means that, for example, when related information is classified into one of predetermined types based on display color and / or shape, it belongs to the same type. According to this aspect, the display device can suppress the occurrence of misidentification in the correspondence between the object and the related information.

  In another aspect of the display device, the feature information extraction unit extracts a plurality of characteristic colors characteristic of the target object based on the captured image, and the display control unit determines that the target object is close to each other. When the display color of the related information of the target object is the same or similar, the display color of at least one of the related information is changed to another characteristic color. Also according to this aspect, the display device can suppress the occurrence of misidentification in the correspondence between the object and the related information.

  In another aspect of the display device, the display unit is a head-up display, a head-mounted display, or a display that displays the captured image.

  According to another embodiment of the present invention, a control method executed by a display device, a captured image acquisition step of acquiring a captured image, an object specifying step of specifying an object in the captured image, A related information acquisition step of acquiring related information related to the object; a feature information extraction step of extracting feature information of the object based on the captured image; and a display mode of the related information based on the feature information. A display control step for determining. By executing this control method, the display device can display the related information by clarifying the correspondence between the object and the related information.

  According to another embodiment of the present invention, a program executed by a display device, which is a captured image acquisition unit that acquires a captured image, an object specifying unit that specifies an object in the captured image, and the object Related information acquiring means for acquiring related information related to the image, feature information extracting means for extracting feature information of the object based on the photographed image, and display control means for determining a display mode of the related information based on the feature information The display device is caused to function. By installing and executing this program, the display device can clarify the correspondence between the object and the related information and display the related information. Preferably, the above program is stored in a storage medium.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[Schematic configuration diagram]
FIG. 1 shows an example of a schematic configuration diagram of a display device 1 which is an example of a display device according to the present invention. The display device 1 is a wearable display device (head mounted display) for displaying information to a vehicle occupant. The display device 1 can communicate with a server device 8 that manages position information and the like transmitted from each terminal. The display device 1 includes a current position information acquisition unit 2, an orientation information acquisition unit 3, an imaging unit 4, a control unit 5, and a display unit 6. As will be described later, the display device 1 generates related information such as a name for a person, facility, and the like photographed in the photographed image and displays the related information.

  The current position information acquisition unit 2 is, for example, a GPS receiver, and detects the absolute position (current position) of the display device 1 from latitude and longitude information received from a plurality of GPS satellites. The current position information acquisition unit 2 transmits the acquired current position information (also referred to as “current position information Ip”) to the control unit 5 at a constant or indefinite period.

  The direction information acquisition unit 3 is, for example, an electronic compass or a six-axis sensor, and detects the direction in which the imaging unit 4 performs shooting. The direction information acquisition unit 3 transmits the detected direction information (also referred to as “direction information Id”) to the control unit 5 at a constant or indefinite period.

  The imaging unit 4 generates an image captured based on light incident on the image sensor (also simply referred to as “captured image If”). When the user wears the display device 1, the imaging unit 4 has an imaging range that substantially matches the user's field of view. The imaging unit 4 transmits the generated captured image If to the control unit 5.

  The control unit 5 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and controls the entire display device 1. The control unit 5 includes an object specifying unit 51 and a display determining unit 52 which will be described later. Then, as will be described later, the control unit 5 functions as “photographed image acquisition means”, “target object specifying means”, “related information acquisition means”, “feature information extraction means”, and “display control means” in the present invention. To do.

  The display unit 6 displays related information. The display unit 6 includes, for example, a projector and a combiner. The projector includes, for example, a light source and a liquid crystal panel, drives the light source and the liquid crystal panel based on image data corresponding to the related information input from the display determination unit 52, and combines light rays corresponding to the image data. Irradiate. The combiner is constituted by, for example, a half mirror or a holographic optical element (HOE), and transmits a part of the external light and reflects a light beam irradiated by the projector. As a result, a part of the external light transmitted through the combiner and the reflected light of the light beam irradiated by the projector are combined in the combiner and visually recognized by the user.

  Next, the process which the target object specific part 51 and the display determination part 52 with which the control part 5 is provided is demonstrated concretely.

  The object specifying unit 51 specifies an object (also referred to as “display object”) whose related information is to be displayed on the display unit 6 from the captured image If. Here, it is assumed that the display objects are people, facilities, and cars. Specifically, the object specifying unit 51 determines a display object from the photographed image If by comparing, for example, the photographed image If with a template image of a person, facility, car, or the like that is the display object. An area (also referred to as “display object area T”) is specified. Then, the object specifying unit 51 estimates the actual position of each display object based on the position of the display object region T in the captured image If in addition to the current position information Ip and the orientation information Id. Hereinafter, the position information of each display object estimated by the object specifying unit 51 is also referred to as “estimated position information Ie”.

  The display determination unit 52 refers to the database 10 included in the server device 8 based on the estimated position information Ie and the captured image If, and determines related information, its display mode, and its display position. As described above, the display determination unit 52 includes the display content determination unit 521, the display mode determination unit 522, and the display position determination unit 523.

  First, processing executed by the display content determination unit 521 will be described. Schematically, the display content determination unit 521 determines related information to be attached to each display object by referring to the database 10 included in the server device 8 based on the estimated position information Ie.

  Here, the specific contents of the database 10 will be described with reference to FIG. FIG. 2 is a diagram showing an outline of information held in the database 10. As illustrated in FIG. 2, the database 10 stores “user information” that is information about a user who uses the display device 1 and “display target information” that is information about a display target.

  The user information includes current position information Ip and direction information Id. These pieces of information are uploaded from the display device 1 and updated at a constant or indefinite period.

  The display object information is divided for each person, facility, and vehicle, which are types of display objects. Here, when the type of the display object is a person, a name, current position information Ip, and character information are stored as shown in FIG. Here, the “character information” is information that specifies related information, a display mode, and the like. The name and character information are registered in advance, for example, and the current position information Ip is uploaded and updated at a constant or indefinite period from a mobile terminal with a GPS function used by the target person.

  Next, when the type of the display object is a facility, the name of the facility, position information, information on the shape of the facility, and character information are mainly stored in the database 10. Such information is registered in the database 10 in advance, for example. When the type of the display object is a car, information on the car type, information on the driver, current position information Ip, information on the shape, and character information are mainly stored in the database 10. Among these pieces of information, the current position information Ip is uploaded and updated from the vehicle at a constant or indefinite period, and other information is registered in the database 10 in advance, for example.

  Here, with reference to FIG. 1 again, the process executed by the display content determination unit 521 will be specifically described. The display content determination unit 521 searches the database 10 for a display object having current position information Ip or position information that is within an error range that can be considered to match or match the estimated position information Ie. The above range is determined in advance based on, for example, experiments. When the display content determining unit 521 specifies on the database 10 a display object having the current position information Ip or position information that matches or substantially matches the estimated position information Ie, the name or character information of the display object The relevant information is determined based on Specifically, when the display object is a person, the display content determination unit 521 determines the related information as the name registered in the database 10 or the position, role, nickname, etc. specified by the character information. In addition, when the display object is a facility, the display content determination unit 521 determines the related information as a name registered in the database 10 or an abbreviation designated by character information, and when the display object is a car. Defines the related information in the name specified by the vehicle type, the driver, or the character information.

  Next, processing executed by the display mode determination unit 522 will be described. The display mode determination unit 522 determines a color (also simply referred to as “display color”) for displaying the related information determined by the display content determination unit 521 based on the color information of the display target region T of each display target. . Here, the color information described above is an example of “feature information” in the present invention.

  Specifically, the display mode determination unit 522 extracts the most characteristic color (also referred to as “first characteristic color”) in the display object region T corresponding to each display object. For example, the display mode determination unit 522 occupies a predetermined ratio or more in all the pixels of the display target area T when each pixel of the display target area T is classified into one of predetermined colors, and The color occupying the largest ratio is determined as the feature color. The predetermined ratio is determined in advance as a ratio that is considered to be characteristic based on, for example, experiments. Then, the display mode determination unit 522 determines the first feature color as the display color when the first feature color exists, and sets the predetermined basic color as the display color when the first feature color does not exist. Stipulated in The basic color is, for example, black or fluorescent green.

  In addition, the display mode determination unit 522 determines whether the first characteristic color is the same as or close to the background color (unit, also referred to as “background color”) of the portion displaying the related information based on the captured image If. If it is determined that it is the same as or close to the background color, the display object region T displays a characteristic color next to the first characteristic color (also referred to as “second characteristic color”). Set to color. For example, the display mode determination unit 522 specifies the display position of the related information from the position of the display object region T, and the color indicated by the pixel of the captured image If overlapping the display position matches or approximates the first characteristic color. If it is determined, the display color of the related information is determined as the second characteristic color. Here, the “color indicated by the pixel of the captured image If overlapping the display position” may be, for example, a color indicated by an average of pixel values of the corresponding pixel, and one representative pixel among the corresponding pixels. The color indicated by the value may be used. Further, for example, the display mode determination unit 522 digitizes the first characteristic color and the background color according to a predetermined method, and determines that these colors match or approximate when these values are within a predetermined difference. Hereinafter, when the first characteristic color and the second characteristic color are not particularly distinguished, these are simply collectively referred to as “characteristic colors”.

  Next, processing executed by the display position determination unit 523 will be described. When the display position determination unit 523 determines that the two display objects are close to each other and the display modes of the related information attached thereto are the same or similar, the display position of each related information is determined. And based on the position of the other display object. Specifically, in this case, the display position determination unit 523 sets the display position of each related information as the position on the opposite side of the other display object with respect to the position of one display object (hereinafter referred to as “diagonal”). It is also called “position”.) For example, the display position determination unit 523, based on the captured image If, the display object region T of the display object having the same category such as person, facility, etc. is close within a predetermined distance, and these display colors match or In the case of approximation, these pieces of related information are arranged at diagonal positions. As a result, the display position determination unit 523 misidentifies the user even when the two display objects are close to each other and the display modes of the related information attached thereto are the same or similar. It is possible to display the related information of each display object without making it.

[Display example]
Next, a specific display example will be described with reference to FIGS. 3 and 4, the hatching of the mesh indicates orange, the hatched hatch indicates green, the horizontal hatch indicates pink, and the vertical hatch indicates blue.

  FIG. 3A shows an image (also referred to as “normal-time image”) viewed by the user via the display unit 6 corresponding to a case where related information is not displayed. That is, the normal time image shown in FIG. 3A substantially matches the image visually recognized when the user does not wear the display device 1 and also substantially matches the captured image If. In FIG. 3A, the symbol “9” indicates the user's viewpoint. The normal video shown in FIG. 3A is a person “Mr. A” wearing pink clothes, a person “Mr. B” wearing blue clothes, an orange shop “Shop C”, Including a green shop "Store D".

  FIG. 3B shows an image (also referred to as “control-time image”) visually recognized by the user when related information is displayed on the display unit 6. As shown in FIG. 3B, the letter “Mr. A” is displayed in pink near the lower side of the person wearing pink clothes. Similarly, the letter “Mr. B” is displayed in blue near the lower side of the person wearing blue clothes. In addition, the word “shop C” is displayed in orange near the lower side of the orange shop. Furthermore, the letter “Shop D” is displayed in green near the lower side of the green shop.

  In this case, processing executed by the control unit 5 will be described. The object specifying unit 51 specifies four display object regions T from the captured image If and calculates estimated position information Ie of each display object. Then, the display content determination unit 521 specifies the name or name of each display object based on the position information registered in the database 10 based on the estimated position information Ie, and uses the name or the like as related information. Then, the display mode determination unit 522 specifies the first feature color from each display object region T, and causes the display unit 6 to display each related information by each specified first feature color. In this case, the display position is set to a predetermined standard position (here, near the lower side of each display object region T). In this way, the control unit 5 can display the related information by clearly associating each display target with the related information by displaying the related information in the first characteristic color of each display target.

  FIG. 4A shows a control-time video when character information is registered for A and B on the database 10. Here, it is assumed that the character information of Mr. A is registered as “enemy” and the character information of Mr. B is registered as “friend” on the database 10. In this case, as shown in FIG. 4A, the display content determination unit 521 determines related information using character information in preference to the name. Further, the display mode determination unit 522 highlights the display using the character information by surrounding the related information with a square frame. By doing in this way, the display determination part 52 gives the user relevant information such as the position and role of the target person in a game divided into demons, enemies, and other events, for example, when playing demons. Can be actively presented.

  In addition to FIG. 4A, when the display mode determination unit 522 specifies the display mode such as color, size, and font in addition to the related information in the character information, the display mode determination unit 522 Generate information.

  FIG. 4B shows a control-time image when Mr. A and Mr. B are close to each other within a predetermined distance and the display colors of these related information are the same color (pink). As shown in FIG. 4B, in this case, the display position determining unit 523 arranges each related information at a diagonal position. By doing in this way, the display apparatus 1 can prevent the correspondence with a display target object and related information becoming unclear, and can make a user discriminate | determine A and B appropriately.

[Processing flow]
Next, a processing procedure executed by the control unit 5 in this embodiment will be described with reference to FIG. FIG. 5 is an example of a flowchart showing a processing procedure executed by the control unit 5 in this embodiment. The control unit 5 repeatedly executes the process of the flowchart shown in FIG. 5 according to a predetermined cycle.

  First, the control unit 5 acquires current position information Ip (step S101). Specifically, the control unit 5 receives the current position information Ip from the current position information acquisition unit 2. Next, the control unit 5 acquires the orientation information Id (step S102). Specifically, the control unit 5 receives the direction information Id from the direction information acquisition unit 3. Further, the control unit 5 acquires the captured image If (step S103). Specifically, the control unit 5 receives the captured image If from the imaging unit 4.

  Next, the control part 5 specifies a display target object from the picked-up image If (step S104). Specifically, the control unit 5 specifies the display object region T from the captured image If. Then, the control unit 5 calculates the estimated position information Ie of each display object based on the position of the display object region T in the captured image If in addition to the current position information Ip and the orientation information Id.

  And the control part 5 determines whether a display target object exists in the picked-up image If (step S105). And the control part 5 searches the information of a display target object from the database 10, when a display target object exists (step S105; Yes) (step S106). Specifically, the control unit 5 communicates with the server device 8 and searches the database 10 for relevant display target information based on the estimated position information Ie. On the other hand, when the control unit 5 determines that there is no display object (step S105; No), the control unit 5 determines that it is not necessary to display related information, and ends the process of the flowchart.

  Next, in step S107, the control unit 5 determines related information (step S107). Specifically, the control unit 5 determines the name, name, character information, and the like of the display object retrieved from the database 10 as related information to be displayed.

  Next, the control unit 5 determines whether or not the first characteristic color exists (step S108). Specifically, the control unit 5 extracts the first characteristic color based on the pixel value of each display object region T.

  When determining that the first characteristic color is present (step S108; Yes), the control unit 5 determines whether the first characteristic color is close to the background color (step S109). If the control unit 5 determines that the feature color is close to the background color (step S109; Yes), the control unit 5 determines the display color as the second feature color (step S110). In this way, the control unit 5 makes it difficult for the user to visually recognize the related information because the display color of the related information approximates the background color around the display object such as a road or a wall. Can be suppressed. On the other hand, when determining that the first feature color is not close to the background color (step S109; No), the control unit 5 determines the display color as the first feature color (step S111). In this way, the control unit 5 displays the related information using the most characteristic color of the display object, thereby allowing the user to easily grasp the correspondence between the display object and the related information. it can.

  On the other hand, when the control unit 5 determines in step S108 that the first characteristic color does not exist (step S108; No), the display color of the display object is determined as a predetermined basic color such as black or fluorescent green. (Step S112).

  Next, the control unit 5 determines whether or not each display object is close to a display object having similar characteristics (step S113). For example, the control unit 5 determines whether display objects having the same category such as people and facilities are close to each other within a predetermined distance, and whether or not these display colors match or approximate. And when the control part 5 judges that it is close to the display target object which has a similar characteristic (step S113; Yes), the display position of related information is defined on the opposite side to the said display target object (step S114). . That is, the control unit 5 determines the display position of the related information of one display object at a position away from the other display object. By doing in this way, the control part 5 can suppress that misidentification arises in the correspondence of a display target object and related information.

  On the other hand, when it is determined that the control unit 5 is not close to a display object having similar characteristics (step S113; No), the control unit 5 determines the display position of the related information at the normal display position. Then, the control unit 5 displays the related information on the display unit 6 at the determined display position in the determined display mode (step S116).

[Modification]
Hereafter, each modification of a present Example is demonstrated. Each of these modified examples can be applied to the above-described embodiments in any combination.

(Modification 1)
In the above description, the display mode determination unit 522 extracts the first characteristic color and the second characteristic color from the display object region T of the captured image If, and determines these as display colors for the related information. However, the method to which the present invention is applicable is not limited to this.

  Instead of or in addition to this, the display mode determination unit 522 may determine the display mode of the related information based on the shape of the display object. For example, when the display object is a person, the display mode determination unit 522 identifies the person's height, width, and the like from the display object area T, and changes the font of the related information accordingly. More specifically, if the display mode determination unit 522 determines that the person is a good person based on the height, the width, and the like, the font of the related information is bolded compared to other persons.

  In another example, the display mode determination unit 522 may determine the aspect ratio of the display area of the related information of the person who is the display object according to the height and the width. Specifically, the display mode determination unit 522 increases the vertical width of the display area as the height increases, and increases the horizontal width of the display area as the horizontal width increases.

  Also by this, the display device 1 can display the related information by associating the display object with the related information so that the user can easily understand it intuitively. Thus, the shape is an example of “feature information” in the present invention.

(Modification 2)
In FIG. 1, the display determination unit 52 includes a display position determination unit 523 so that when two display objects having similar characteristics are close to each other, the display positions of these related information are set to diagonal positions. Were determined. However, the configuration to which the present invention is applicable is not limited to this. Instead of or in addition to this, in this case, the display determination unit 52 may change the display color of the related information of at least one of these display objects to the second characteristic color. Also by this, the display determination unit 52 preferably suppresses the user from misidentifying the correspondence between the display object and the related information due to the proximity of two display objects having similar characteristics. be able to.

(Modification 3)
In steps S109 and S110 of FIG. 4, the display determination unit 52 sets the display color to the second characteristic color when the display color of the related information and the background color of the display object are the same or approximate. However, the method to which the present invention is applicable is not limited to this.

  Instead of this, or in addition to this, the display determination unit 52 may change the display position of the related information from the standard position to a position around the display object that is the same as the display color and not approximate. Good. Also by this, the display determination part 52 can suppress that it becomes difficult to visually recognize related information.

  In another example, in this case, when the second feature color can be extracted, the display determination unit 52 determines the display color of the related information as the second feature color, and when the second feature color cannot be extracted, The information display position may be changed as described above.

(Modification 4)
In the above-described embodiment, the display device 1 is a head mounted display. However, the display device 1 to which the present invention is applicable is not limited to this.

  Instead of this, the display device 1 may be a head-up display installed in a vehicle. Even in this case, the display device 1 includes the current position information acquisition unit 2, the orientation information acquisition unit 3, the imaging unit 4, the control unit 5, and the display unit 6. And the imaging part 4 is installed toward the direction substantially corresponded with the driver | operator's eyes | visual_axis at the time of a driving | operation. In addition, a light source unit that irradiates light corresponding to the display unit 6 and an optical element such as a combiner that reflects the irradiated light toward the driver are installed in the vehicle. Even in this case, the display device 1 preferably allows the user to visually recognize the relevant information of the display object included in the scenery on the scenery outside the vehicle visually recognized by the driver via the optical element. Can do.

  In another example, the display device 1 may be a navigation device that performs navigation by attaching a guidance display to the captured image If acquired from the imaging unit 4. Even in this case, the display device 1 can extract the display object from the photographed image If based on the above-described embodiment, and can superimpose related information on the photographed image If to be displayed.

(Modification 5)
In the description of the embodiment, the control unit 5 causes the display unit 6 to display related information on the assumption that the orientation of the imaging unit 4 matches the line-of-sight direction of the user. Instead, the control unit 5 detects the user's line-of-sight direction with a camera or the like, calculates the difference between the detected line-of-sight direction and the imaging direction of the imaging unit 4, and displays the related information display position according to the difference. May be shifted. Thereby, the control unit 5 corrects the position of the related information according to the user's line of sight even when the user's line of sight and the shooting direction of the imaging unit 4 do not match, and the display object and its related Information can be appropriately associated with the user and recognized.

(Modification 6)
In the description of FIG. 1, the database 10 is held by the server device 8. However, the configuration to which the present invention is applicable is not limited to this. Instead of this, the database 10 may be held by the display device 1.

  The present invention can be suitably applied to a head-mounted display, a head-up display, a navigation device, and other display devices.

DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Current position information acquisition part 4 Imaging part 5 Control part 6 Display part

Claims (7)

A captured image acquisition means for acquiring a captured image;
An object specifying means for specifying an object in the captured image;
Related information acquisition means for acquiring related information on the object;
Feature information recognition means for recognizing the shape of the object based on the captured image;
Display control means for displaying related information related to the object on the display means in association with the object ;
The display control unit determines a font for displaying the related information based on a shape of the object . The display device according to claim 1, wherein the display control unit determines a thickness of a character when displaying the related information based on a shape of the object. The feature information recognizing means recognizes the vertical and horizontal sizes of the object;
3. The display device according to claim 1, wherein the display control unit determines a vertical size of a region in which the related information is displayed based on a vertical size of the object. . The feature information recognizing means recognizes the vertical and horizontal sizes of the object;
The said display control means determines the magnitude | size of the horizontal direction of the area | region which displays the said relevant information based on the magnitude | size of the horizontal direction of the said target object, The Claim 1 thru | or 3 characterized by the above-mentioned. The display device described in 1. The feature information recognizing means recognizes the vertical and horizontal sizes of the object;
The display device according to any one of claims 1 to 3, wherein the display control unit determines a font for displaying the related information based on a size of the object in a horizontal direction. The display device according to any one of claims 1 to 5 , wherein the display means is a head-up display or a head-mounted display. A control method executed by a display device including an imaging unit, a control unit, and a display unit,
A captured image acquisition step in which the imaging unit acquires a captured image;
An object specifying step in which the control unit specifies an object in the captured image;
A related information acquisition step in which the control unit acquires related information about the object; and
A feature information recognition step in which the control unit recognizes the shape of the object based on the captured image;
A display control step in which the control unit displays related information related to the object on the display unit in association with the object ;
The display control step determines a font for displaying the related information based on the shape of the object .

Images