JP2014153095A - Information display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information display device capable of making a correspondence relation between each facility and information thereon understandable when additionally displaying information on each facility included in an actual image.SOLUTION: A navigation device comprises: a camera 8 which acquires an actual image including a surrounding facility by photographing; a facility extraction section 32 which extracts a facility image included in the actual image; a facility information acquisition section 34 which acquires facility information corresponding to the facility extracted by the facility extraction section 32; and an image composition section 36 which displays the facility image and the facility information, which correspond to each other, by associating with each other on the actual image.

Description

  The present invention relates to an information display device that displays information corresponding to a facility included in an image captured using a camera.

  2. Description of the Related Art Conventionally, an information providing apparatus that provides predetermined information added to a real image by a camera is known (for example, see Patent Document 1). In this information providing apparatus, service information such as advertising billboards is pasted on a building using 3D computer graphics technology on an actually photographed urban landscape, or a road guide on a road An arrow is added and provided to the user.

JP 2003-216882 A

  By the way, in the information providing apparatus of Patent Document 1 described above, service information or the like is affixed to each of a plurality of buildings (facility) included in an actual image or in the vicinity of the building, and the number of buildings increases. When they are dense, there is a problem that it becomes difficult to understand the correspondence between each building and information. For example, when there are multiple adjacent buildings, it is difficult to understand the correspondence if the position of the information to be added does not protrude from the corresponding building, but if the area of the building in the actual image becomes small The location of information will always protrude from the building.

  The present invention was created in view of such points, and its purpose is to make it easy to understand the correspondence between each facility and information when information is added to and displayed on each facility included in the actual image. It is an object of the present invention to provide an information display device that can do this.

  In order to solve the above-described problem, an information display device according to the present invention includes an imaging unit that acquires a real image including surrounding facilities by imaging, a facility extraction unit that extracts a facility image included in the real image, and a facility extraction A facility information acquisition unit that acquires facility information corresponding to the facility corresponding to the facility image extracted by the unit, and an image composition unit that draws the facility image and facility information corresponding to each other in the actual image. ing. When the facility information is added to each facility image included in the actual image and displayed, the corresponding facility image and the facility information are displayed in association with each other so that the correspondence between each facility and the facility information can be easily understood. Can do.

  Moreover, it is desirable that the facility extraction means described above extracts a facility image for each building. By extracting the facility image included in the actual image in units of buildings, it is possible to easily grasp the range of facilities to which facility information is added.

  In addition, it is desirable that the facility extraction unit described above performs an image recognition process on the actual image to extract the facility image. The boundary of each building can be detected by the image recognition process, and each facility image included in the actual image can be reliably extracted.

  Further, the apparatus further includes position detection means for detecting an imaging position by the imaging means described above, and the facility information acquisition means includes the imaging position detected by the position detection means and the actual image of each facility image extracted by the facility extraction means. It is desirable to identify the location of each facility based on the location at and to obtain facility information corresponding to this identified location. Each facility can be identified based on the imaging position and the position of each facility image in the actual image, and facility information corresponding to each facility can be easily obtained.

  Further, the information processing apparatus further includes facility information storage means for storing the facility information described above together with the facility position, and the facility information acquisition means stores the facility information corresponding to the identified facility position from the facility information stored in the facility information storage means. It is desirable to obtain. Thereby, facility information can be acquired reliably based on a facility position.

  In addition, it is desirable that the above-described image composition means add an auxiliary image for extracting facility information from the facility image between the facility image and facility information associated with each other. Alternatively, it is desirable that the image composition means described above adds the same coloring to the facility image and facility information associated with each other. As a result, the facility image to which the facility information is added can be easily understood, and the correspondence between the facility information and the facility image can be easily understood.

  Further, it is desirable that the above-described image composition unit adds a frame image that emphasizes the outermost portion of the facility image extracted by the facility extraction unit. As a result, the range of the facility to which the facility information is added is visually clarified, and the range of the facility image corresponding to the associated facility information can be easily understood.

It is a figure which shows the structure of the navigation apparatus of one Embodiment. It is a flowchart which shows the operation | movement procedure which associates and displays facility information on each facility image contained in a real image. It is a figure which shows an example of a real image. It is a figure which shows the some facility image extracted within the real image. It is a figure which shows the actual image to which coloring and facility information were added for every facility image. It is a figure which shows the actual image with which detailed facility information was added to the selected facility. It is a figure which shows the structure of a modification. It is a figure which shows the structure of a modification.

  Hereinafter, a navigation device according to an embodiment to which an information display device of the present invention is applied will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a navigation device according to an embodiment. The navigation device shown in FIG. 1 includes a navigation controller 1, a map data storage device 3, an operation unit 4, a vehicle position detection unit 5, a display device 6, and a camera 8. This navigation device is mounted on a vehicle.

  The navigation controller 1 executes a predetermined operation program using a CPU, a ROM, a RAM, etc., thereby displaying a map image display operation, facility search operation, route search / guidance operation around the vehicle position, and the information display device of the present invention. Various functions such as operation are realized. A detailed configuration and specific examples of the operation of the navigation controller 1 will be described later.

  The map data storage device 3 is a storage medium that stores map data necessary for map display, facility search, route search, and the like and a reading device thereof. The map data storage device 3 stores map data in units of rectangular figure leaves divided into appropriate sizes by longitude and latitude. The map data of each leaf can be specified and read by designating the leaf number. The map data storage device 3 is realized by a hard disk device or a semiconductor memory, or by a DVD and its reading device. Further, the map data storage device 3 may be replaced with a communication device to acquire map data from an external map distribution server (not shown).

  The operation unit 4 is for receiving user instructions (operations), and includes various operation buttons and operation knobs. In addition, the operation unit 4 includes a touch panel attached to the screen of the display device 6, and the user can directly perform an operation instruction by pointing a part on the screen with a finger or the like. Yes. The vehicle position detection unit 5 includes, for example, a GPS receiver, an orientation sensor, a distance sensor, and the like, detects the vehicle position (longitude, latitude) at a predetermined timing, and outputs a detection result.

  The display device 6 is configured by, for example, an LCD (liquid crystal display device), and is captured by a map image or intersection guidance image around the vehicle position or by the camera 8 based on a video signal output from the navigation controller 1. An actual image or an image obtained by adding facility information to the actual image is displayed. The camera 8 is attached to a predetermined position (for example, a central position) on the front side of the vehicle, and images a predetermined range in front of the vehicle with a predetermined resolution. In addition, you may make it use this camera 8 provided for the other use (for example, for drive recorders). Alternatively, instead of using an image acquired using the in-vehicle camera 8, a user (driver or passenger) may capture and use an image obtained by photographing the front of the vehicle with a digital camera.

  Next, a detailed configuration of the navigation controller 1 will be described. A navigation controller 1 shown in FIG. 1 includes a map buffer 10, a map readout control unit 12, a map drawing unit 14, a vehicle position calculation unit 20, an image storage unit 30, a facility extraction unit 32, a facility information acquisition unit 34, and an image composition unit 36. The input processing unit 50 and the display processing unit 60 are included.

  The map buffer 10 temporarily stores map data read from the map data storage device 3. This map data includes facility information necessary for operation as the information display device of the present invention. In the facility information, in addition to the overall information of each facility, in the case of a building of a plurality of floors, detailed facility information such as a tenant for each floor is included together with the facility position. The map reading control unit 12 sends a request for reading a predetermined range of map data in accordance with the vehicle position calculated by the vehicle position calculating unit 20 and the position designated by the user by operating the operation unit 4. Output to. Based on the map data stored in the map buffer 10, the map drawing unit 14 performs drawing processing necessary for displaying a map image on the display device 6 to create map image drawing data.

  The vehicle position calculation unit 20 calculates the vehicle position based on the detection data output from the vehicle position detection unit 5, and if the calculated vehicle position is not on the road of the map data, the vehicle position is calculated. Perform map matching processing to correct.

  The image storage unit 30 stores an actual image obtained by imaging with the camera 8. The facility extraction unit 32 performs an image recognition process on the real image stored in the image storage unit 30 to extract a facility image. In this extraction process, facility images are recognized in units of buildings, and even adjacent buildings are classified as different buildings.

  The facility information acquisition unit 34 acquires facility information corresponding to the facility corresponding to the facility image extracted by the facility extraction unit 32. The map buffer 10 stores facility information of each facility along with the facility position. The facility information acquisition unit 34 specifies the position of each facility image extracted by the facility extraction unit 32 in the actual image, and also specifies the imaging position (vehicle position calculated by the vehicle position calculation unit 20) and each of the specified locations. The position (absolute position) of each facility is specified based on the position of the facility image, and the facility information corresponding to the specified absolute position is read from the map buffer 10.

  The image composition unit 36 adds coloring to each facility image so that the facility images extracted by the facility extracting unit 32 can be distinguished from each other, and the facility corresponding to the facility of each facility image in the vicinity of each facility image. Draw information overlaid. In addition, in order to clarify which facility image corresponds to each facility information, each facility information is given the same color as the corresponding facility image, and the facility information from the corresponding facility image A supplementary image for pulling out and linking is added. A specific example of display by the image composition unit 36 will be described later.

  The input processing unit 50 detects the operation content (input content) of the operation unit 4 and outputs an instruction corresponding to the input content toward the configuration waiting for input. The display processing unit 60 selectively displays a map image drawn by the map drawing unit 14 or an actual image drawn by superimposing facility information by the image composition unit 36. Instead of displaying selectively, it may be divided and displayed within one screen.

  The above-described camera 8 is an imaging unit, a facility extraction unit 32 is a facility extraction unit, a facility information acquisition unit 34 is a facility information acquisition unit, an image composition unit 36 is an image composition unit, a vehicle position detection unit 5, a vehicle position calculation. The unit 20 corresponds to the position detection means, and the map buffer 10 corresponds to the facility information storage means.

  The navigation device of the present embodiment has such a configuration. Next, an operation for adding and displaying facility information to an actual image obtained by imaging by the camera 8 will be described.

  FIG. 2 is a flowchart showing an operation procedure for displaying facility information in association with each facility image included in the actual image. Imaging by the camera 8 is performed in response to an operation instruction from the user (step 100). The obtained image data of the real image is stored in the image storage unit 30. Further, the facility extraction unit 32 performs image recognition processing on the actual image based on the image data stored in the image storage unit 30 and extracts one or more facility images included in the actual image in units of buildings. (Step 102). FIG. 3 is a diagram illustrating an example of a real image. FIG. 4 is a diagram showing a plurality of facility images extracted from the actual image. In FIG. 4, partial images to which reference signs A to I are attached are facility images, and a portion that is visible in the actual image is extracted in units of buildings (in this example, buildings).

  Next, the facility information acquisition unit 34 pays attention to the one facility image extracted by the facility extraction unit 32, and reads and acquires the facility information corresponding to this facility image from the map buffer 10 (step 104). Thereafter, the facility information acquisition unit 34 determines whether there is an unprocessed facility image (step 106). If there is, the facility information acquisition unit 34 makes an affirmative determination and returns to step 104, paying attention to the next facility image. Get information. Further, when the acquisition of the facility information is completed for all the facility images (in the example illustrated in FIG. 4, the facility images to which the signs A to I are added), a negative determination is made in the determination of step 106.

  Next, the image composition unit 36 performs a drawing process for coloring the actual image in units of facility images and adding the corresponding facility information in the vicinity of each facility image in a linked state (step 108). Further, the display processing unit 60 displays the actual image drawn with the coloring and facility information drawn by the image composition unit 36 on the display device 6 (step 110).

  FIG. 5 is a diagram illustrating an actual image in which coloring and facility information are added for each facility image. In FIG. 5, it is difficult to understand the state of coloring, but the facility images to which the signs A to I shown in FIG. 4 are attached are colored with colors that can be distinguished from each other. In the example shown in FIG. 5, a balloon (the whole balloon or a linked portion where the balloon is drawn from the facility image corresponds to the auxiliary image) is output from each facility image, and facility information (for details of the facility information is included therein). (It is omitted in this figure). Such facility information includes a concept called “air tag”. Note that the addition of facility information is omitted for facilities for which facility information does not exist. Moreover, although it is preferable from the viewpoint of ensuring visibility that the facility information is the entire information of one or two facilities in units of facilities, there may be three or more facilities information.

  Next, the facility extraction unit 32 determines whether any facility (facility image) has been selected by the user operating the operation unit 4 (step 112). If it is not selected, a negative determination is made, and the process returns to step 110 to continue the display operation. If any facility is selected by the user, an affirmative determination is made in the determination of step 112.

  Next, the facility information acquisition unit 34 acquires detailed facility information of the selected facility (step 114). In addition, the image composition unit 36 adds color to the actual image in units of facility images (or leaves the color attached in step 108), and details corresponding to the vicinity of the facility image of the selected facility. A drawing process for adding facility information in a linked state is performed (step 116). Further, the display processing unit 60 displays the actual image drawn with the coloring and facility information drawn by the image composition unit 36 on the display device 6 (step 118).

  FIG. 6 is a diagram illustrating a real image in which detailed facility information is added to the selected facility. The number of detailed facility information may be three or more. However, if there are too many pieces of information, the number of detailed facility information that is displayed on the actual image at one time is limited, and each time an operation such as pointing to the facility is performed again, The detailed facility information may be updated.

  As described above, in the navigation device of the present embodiment, when the facility information is added to each facility image included in the actual image and displayed, the corresponding facility image and the facility information are displayed in association with each other. It is possible to make the correspondence between the facility information and the facility information easier to understand. Further, by extracting the facility image included in the actual image in units of buildings, it is possible to easily grasp the range of the facility to which the facility information is added. Furthermore, by performing an image recognition process on the real image and extracting the facility image, the boundary for each building can be detected, and each facility image included in the real image can be reliably extracted.

  In addition, a facility image to which facility information is added by adding an auxiliary image for extracting facility information from the facility image between facility images and facility information associated with each other or by adding the same coloring to the facility image and facility information associated with each other Images can be easily grasped, and the correspondence between facility information and facility images can be easily understood.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention. For example, in the above-described embodiment, a different color is assigned to each of the extracted facility images, but instead of coloring, a frame image that emphasizes the outermost part of each facility image may be added. . As a result, the range of the facility to which the facility information is added is visually clarified, and the range of the facility image corresponding to the associated facility information can be easily understood.

  In the embodiment described above, the configuration of the information display device of the present invention is incorporated in the navigation device. However, as shown in FIG. 7, the information display device 200 is configured separately from the navigation device 100, and necessary information is properly navigated. You may make it input / output between apparatuses. The information display device 200 includes at least the image storage unit 30, the facility extraction unit 32, the facility information acquisition unit 34, and the image composition unit 36 shown in FIG.

  Also, as shown in FIG. 8, instead of acquiring facility information from the navigation device, the facility information is acquired from the server 400 via the network 300, or the camera 8A with a GPS function (or a mobile phone with a camera with a GPS function). Etc.) may be used to acquire a real image with an imaging position. Alternatively, an actual image with position information may be acquired via the network 300. In particular, when acquiring facility information from the server 400 or the like, since the latest facility information can be acquired each time, new facility information is displayed preferentially, campaign information within the validity period, etc. The facility information can be displayed with priority.

  As described above, according to the present invention, when facility information is added to each facility image included in the actual image and displayed, the corresponding facility image and facility information are displayed in association with each other, The correspondence with facility information can be easily understood.

DESCRIPTION OF SYMBOLS 1 Navigation controller 3 Map data storage device 4 Operation part 5 Vehicle position detection part 6 Display apparatus 8 Camera 10 Map buffer 12 Map reading control part 14 Map drawing part 20 Vehicle position calculation part 30 Image storage part 32 Facility extraction part 34 Facility information acquisition Unit 36 Image composition unit 50 Input processing unit 60 Display processing unit

Claims (8)

Imaging means for acquiring a real image including surrounding facilities by imaging;
Facility extraction means for extracting a facility image included in the real image;
Facility information acquisition means for acquiring facility information corresponding to a facility corresponding to the facility image extracted by the facility extraction means;
Image combining means for drawing the facility image and the facility information corresponding to each other in the real image,
An information display device comprising: In claim 1,
The facility extraction unit extracts the facility image for each building. In claim 2,
The facility extraction means extracts the facility image by performing image recognition processing on a real image. In any one of Claims 1-3,
It further comprises position detection means for detecting an imaging position by the imaging means,
The facility information acquisition means specifies the position of each facility based on the imaging position detected by the position detection means and the position in the actual image of each facility image extracted by the facility extraction means. An information display device characterized by acquiring facility information corresponding to a specified position. In claim 4,
A facility information storage means for storing facility information together with the facility location;
The facility information acquisition unit acquires the facility information corresponding to the specified facility position from the facility information stored in the facility information storage unit. In any one of Claims 1-5,
The information display device, wherein the image composition unit adds an auxiliary image for extracting the facility information from the facility image between the facility image and the facility information associated with each other. In any one of Claims 1-6,
The information display device, wherein the image composition unit adds the same coloring to the facility image and the facility information associated with each other. In any one of Claims 1-7,
The information display device, wherein the image composition unit adds a frame image that emphasizes the outermost portion of the facility image extracted by the facility extraction unit.

Images