JP3053169B2 - Communication device and system using landscape label - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention superimposes and displays, on an image display device, geographical information about each partial area in an image taken by a user using a landscape image input device such as a camera. The present invention relates to a communication device that teaches a user by voice guidance or the like.

[0002]

2. Description of the Related Art Conventionally, there have been various navigation systems as systems for instructing users on geographical information about the area where the users are located.

FIG. 13 is a block diagram of a navigation device disclosed in Japanese Patent Application Laid-Open No. 8-273000. This device receives a vehicle position data and a motion data, and receives a position update unit 71 that updates the vehicle position with reference to the road map data, and displays display road data and display background data based on the map data and the like. Display data generator 7 to be generated
2, a three-dimensional moving image data creating unit 73 for creating three-dimensional moving image data based on these display data, and a storage unit 74.
When a travel route including a waypoint is set in advance, the device has a function of setting a route while viewing a real moving image display screen along a road that actually exists instead of a map screen.

According to this device, when a user travels along a route that actually exists, the user can view a moving image display (for example, FIG. 14) along the route.

[0005]

However, in the case of using the same device, the human finally associates the real scene with the geographical information in the computer world with the naked eye, so that the person in the real scene can be used. Aware of what is. In other words, what is the actual building, road or mountain in front of the user is displayed on a moving image, and based on the symbols etc. in the map, humans rely on the naked eye to work the human brain unconsciously. It must be understood by performing the work of association. At street corners, etc., comparing the map on the computer with the actual scenery, grasping the direction and finding landmarks, gazing at the direction, understanding the characteristics of the building in that direction, and then looking at the map again Understand what the building is.

[0006] For this reason, there is a problem that the trouble of comparing the map on the computer with the actual scenery many times and associating with the human cannot be omitted. Especially in the dark or at night, it is difficult to see the actual scenery and to take measures.

SUMMARY OF THE INVENTION An object of the present invention is to provide a landscape label-based communication device that teaches a user by associating geographic information on a computer with each part in an image of a real landscape (hereinafter referred to as a landscape image). Is to provide a system.

[0008]

According to the present invention, map data on a computer is created in advance as three-dimensional data, and a position at which an image (hereinafter referred to as a landscape image to distinguish it from a CG image) is input. , Camera angle, focal length, and image size at the time of shooting, and computer graphics (hereinafter referred to as CG) when viewed from the position at the time of shooting the actual scenery, the camera angle, and the focal length in the three-dimensional map space on the computer. This is to achieve the association by acquiring geographical information in the image and superimposing and displaying the geographical information on a landscape image which is a real scene. The geographic information is the name of a structure or the like or its attribute information in an image, and the attribute information means information on all attributes (for example, contour, color, etc.) of the structure. In this specification, the term structure is used to mean all data having a certain geographical structure in the map DB, including natural topography such as mountains, rivers, and the sea, in addition to artificial structures. In obtaining the geographic information, the position of the landscape image (hereinafter, referred to as an assigned position) is obtained based on the camera position, the camera angle, the focal length, and the image size, and the name or attribute information of the structure is superimposed and displayed.

Further, in the present invention, the communication address (telephone number, FAX number, IP address, e-mail address, WWW page address, etc.) of the communication device related to the structure is provided.
, A screen position specifying means for instructing a user on the screen position of the visual device, and a communication processing means for setting a communication path with the communication address when the communication address is passed. .

When the user designates a position on the screen of the visual device, the screen position designation means finds a structure in the label information corresponding to the designated screen position, and determines a structure of the communication device associated with the found structure. The communication address is obtained from the communication address storage means and passed to the communication processing means. The communication processing unit sets a communication path with the communication address based on the received communication address information.

[0011] Thus, the user can call the resident of the structure of the label information, send the information by facsimile or e-mail only by touching the place where the screen is located.

When the user designates a position on the screen of the visual device, the communication processing means temporarily displays the received communication address information on the screen of the visual device, and responds to the user's request to start communication by using the communication address. It can be configured to set a communication path with the server.

Further, a target condition designating means for designating a target condition to the user is provided, and the label information creating means outputs the target condition identification information indicating that the target condition designated by the user is the target condition. When the destination identification information is added to the label information, the label information output means can display that the structure matches the destination condition.

Further, when the user designates a position on the screen of the visual device, the screen position designation means finds a structure in the label information corresponding to the designated screen position, and a communication related to the found structure. The communication address of the device is obtained from the communication address storage means, and the communication address is passed to the label information output means. The label information output means can display the received communication address information on the screen.

In order to further increase the accuracy of associating the structure in the real scenery image with the structure in the CG image, the previously acquired structure is associated with each partial region of the landscape image by pattern matching. . A CG image is created based on the acquired structure, a partial region in the CG image is associated with the partial region of the landscape image by pattern matching, and a structure based on the associated partial region is created. Ask for things.

Here, an example of a method for creating a CG image will be described. The three-dimensional map DB is accessed based on the previously acquired position, camera angle, focal length, and image size to determine the visual field space in the three-dimensional map space. A structure in the visual field space is obtained, and three-dimensional data of each structure is three-dimensionally projected and converted onto the projection plane using the camera screen as a projection plane. Further, of the line data constituting the projected figure of each structure, line data hidden by other structures and invisible is erased using a normal vector method or the like. The CG image is divided into regions based on the line data remaining after the elimination of the hidden lines. Since the three-dimensional map DB is used, the name of the structure that is the basis of the area can be associated with each area.

Then, the structure name of the partial area of the CG image associated with each partial area of the landscape image is extracted by pattern matching. The position coordinates of the actual scenery image on which the extracted structure name is to be superimposed are obtained by three-dimensionally projecting the position coordinates of the structure in the three-dimensional map space onto the projection plane. Label information is created from the position coordinates of the actual scenery image on which the extracted structure name is to be superimposed. Based on the label information, the structure name is superimposed on a landscape image as a real scene and displayed on a visual device.

[0018] The landscape label utilizing type communication apparatus of the present invention comprises: an image acquisition unit for acquiring an image; a position information acquisition unit for acquiring a camera position at the time of image acquisition; a camera angle, a focal length, and an image size at the time of image acquisition. Camera attribute information acquiring means for acquiring the image information, image processing means for dividing the acquired image into a plurality of partial areas, managing the map information, and obtaining the map information based on the acquired camera position, camera angle, focal length, and image size. Finding the visual space in the space, the map information management means for acquiring the structure existing in the visual space, and associating the acquired structure with the partial area of the image, Label information creating means for creating label information including the name or attribute information of a structure and an assigned position, and a communication address storage means for storing a communication address of a communication device associated with the structure Communication processing means for setting a communication path with the communication address based on the received communication address, and a name of a structure or an attribute thereof at a position in an image corresponding to a given position in the acquired label information Label information output means for superimposing information and displaying the superimposed image on a visual device, and allowing a user to indicate a position on the screen of the visual device, wherein the user indicates a position on the screen of the visual device. When instructed, the structure in the label information corresponding to the designated screen position is obtained, the communication address of the communication device associated with the obtained structure is obtained from the communication address storage means, and the communication address is obtained by the communication. There are a screen position designating means to be passed to the processing means, and a control means for controlling each of the above means.

According to another aspect of the present invention, there is provided a landscape label-based communication device, comprising: an image acquisition unit for acquiring an image; a position information acquisition unit for acquiring a camera position at the time of image acquisition; A camera attribute information acquiring unit for acquiring an image size, an image processing unit for dividing an acquired image into a plurality of partial areas, and managing map information, based on the acquired camera position, camera angle, focal length, and image size. In the map information space, a view space is obtained, and map information management means for acquiring a structure existing in the view space is associated with the acquired structure with respect to the partial region of the image. Label information creating means for creating label information including the name or attribute information of the structure and the assigned position, and a communication address record for storing a communication address of a communication device associated with the structure. Means, superimposing the name of the structure or its attribute information at the position in the image corresponding to the assigned position in the acquired label information, displaying the superimposed image on a visual device, and receiving the communication address, Label information output means for displaying this on the visual device, and allowing the user to indicate the position on the screen of the visual device, and when the user indicates the position on the screen of the visual device, the designated screen A screen position for obtaining the structure in the label information corresponding to the position, obtaining a communication address of a communication device associated with the obtained structure from the communication address storage means, and passing the communication address to the label information output means It has a designation means and a control means for controlling each of the means.

A landscape label-based communication system according to the present invention comprises a communication terminal and a communication center, the communication terminal comprising: an image obtaining means for obtaining an image; and a position information obtaining means for obtaining a camera position at the time of obtaining the image. Camera attribute information acquiring means for acquiring a camera angle, a focal length, and an image size at the time of image acquisition; image processing means for dividing the acquired image into a plurality of partial areas; information on area division of the image and the camera position Communication control means for transmitting information of the camera angle, the focal length, the image size, and the structure in the label information to the communication terminal via a communication network, and receiving the label information and the communication address from the communication center; Communication processing means for setting a path to the communication address based on the received communication address; and name or attribute information of a structure in the label information. Is superimposed on a position corresponding to the application position in the image, a label information output unit that displays the superimposed image on a visual device, and a user can instruct a position on the screen of the visual device, and the user can When a position on the screen of the visual device is designated, the structure in the label information corresponding to the designated screen position is obtained, and information on the structure is output to the communication control unit. Screen position designation means for passing the communication address of the communication device associated with the structure to the communication processing means, and terminal control means for controlling each of the means, wherein the communication center communicates with the communication network. Information on the area division of the image, the camera position, the camera angle, the focal length, the image size, and information on the structure in the label information are received from the communication terminal via the communication terminal, and the communication terminal receives the information. Communication control means for transmitting the bell information, and managing the map information, obtaining a visual field space in the map information space based on the received camera position, camera angle, focal length, and image size, and presenting in the visual field space Map information management means for acquiring a structure, and associating the acquired structure with the partial area of the image, and label information including a name or attribute information and an assigned position of the associated structure. Label information creating means for creating, communication address storage means for storing the communication address of the communication device associated with the structure, and outputting the communication address of the structure indicated by the information of the structure in the label information; It has a center control means for controlling the means.

Another landscape label-based communication system according to the present invention comprises a communication terminal and a communication center, wherein the communication terminal includes an image acquisition unit for acquiring an image, and a position information acquisition unit for acquiring a camera position at the time of image acquisition. Means, camera attribute information acquisition means for acquiring a camera angle, a focal length, and an image size at the time of image acquisition; image processing means for dividing the acquired image into a plurality of partial areas; information on area division of the image; Communication control means for transmitting information on a camera position, the camera angle, the focal length, the image size, and a structure in label information to the terminal via a communication network, and receiving label information and a communication address from the communication center. And superimposing the name or attribute information of the structure in the label information at a position corresponding to the application position in the image, displaying the superimposed image on a visual device, and Upon receiving the communication address, label information output means for displaying the communication address on the visual device, and a user may be instructed on a position on the screen of the visual device, and the user indicates a position on the screen of the visual device. Then, the structure in the label information corresponding to the designated screen position is obtained, the information of the structure is output to the communication control means, and the information related to the structure received by the communication control means is obtained. Screen position specifying means for passing a communication address of a communication device to the label information output means, and terminal control means for controlling each of the means, wherein the communication center transmits the image of the image from the communication terminal via the communication network. Information on the area division, the camera position, the camera angle, the focal length, the image size, and information on the structure in the label information are received, and the label information and the communication are transmitted to the communication terminal. A communication control means for transmitting an address, a map information is managed, a view space is obtained in a map information space based on a received camera position, a camera angle, a focal length, and an image size, and a structure existing in the view space. Map information management means for acquiring an object, associating the acquired structure with the partial area of the image, and creating the label information including the name or attribute information of the associated structure and the assigned position Label information creating means for storing, a communication address of a communication device associated with the structure, a communication address storage means for outputting a communication address of a structure indicated by information of the structure in the label information, and each of the means Has a center control means for controlling the

According to the embodiment of the present invention, when the user designates a position on the screen of the visual device, the communication processing means displays the received communication address once on the screen of the visual device, and And a communication path with a communication address is set in response to the communication start request.

According to an embodiment of the present invention, there is provided a target condition designating means for designating a target condition to a user, and the label information creating means is a target condition for the target condition designated by the user. Is added to the label information, and when the destination identification information is added to the label information, the label information output means indicates that the structure matches the target condition. It is configured as follows.

According to the aspect of the present invention, the label information creating means creates a CG image based on the acquired structure, and associates the partial area in the CG image with the partial area of the image by pattern matching. The association and the structure of the associated partial area are obtained, and label information including the name or attribute information of the structure and the assigned position is created.

According to an embodiment of the present invention, the label information creating means performs a three-dimensional projection conversion of the acquired structure on a camera screen, erases a structure that cannot be seen from the viewpoint, creates a CG image, and creates a CG image. The CG image is divided into partial regions by the outline of the middle partial region, and the partial region of the image is associated with the partial region of the CG image by pattern matching. The structure on which the partial area is based is determined, and label information including the name or attribute information of the structure and the assigned position is created.

[0026]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of a landscape label-based communication device according to an embodiment of the present invention, and FIG. 2 is a flowchart of processing of the landscape label-based communication device of FIG.

The landscape label-based communication device according to the present embodiment is, for example, an image acquisition unit 1 which is a digital camera for acquiring a landscape image, and a GPS receiver for acquiring a camera position when acquiring an image. Location information acquisition unit 2
A camera attribute information acquiring unit 3 which is a three-dimensional electronic compass attached to a digital camera, for example, acquiring a camera angle, a focal length, and an image size when acquiring an image; An image processing unit 4 that manages geographic information, obtains a visual field space in a map information space based on the acquired camera position, camera angle, focal length, and image size, and includes a structure existing in the visual field space. The map information management unit 5 for acquiring an object and the user can indicate the position on the screen of the visual device, and when the user specifies the position on the screen of the visual device, the label information corresponding to the specified screen position The structure inside is obtained, and the communication address of the communication device related to the obtained structure is obtained from the communication address storage unit 9,
Screen position specifying unit 6 that passes a communication address to communication processing unit 10
And an information creating unit 7 for associating the acquired structure with the partial region of the image and creating label information including the name or attribute information of the associated structure and the assigned position.
And a label information output unit 8 for superimposing the name of the structure or its attribute information at a position in the image corresponding to the assigned position in the acquired label information, and displaying the superimposed image on a visual device.
A communication address storage unit 9 for storing a communication address of a communication device related to a structure, a communication processing unit 10 for setting a communication path with the communication address when the communication address is passed, A target condition specifying unit 12 that allows a user to specify a condition to be performed, and a control unit 11 that controls these units 1 to 10 and 12.

Next, the operation of this embodiment will be described in detail with reference to FIG.

When the communication device is started, first, the control unit 11
Sends a processing start command to the position information acquisition unit 2, the camera attribute information acquisition unit 3, and the image acquisition unit 1 in order to acquire information on the landscape image. The position information acquisition unit 2 receives a command from the control unit 11, collects position information with a GPS receiver or the like every second, and passes the collected position information to the control unit 11 (step 21). Here, the time interval is not limited to the unit of seconds, but may be any value. The image acquisition unit 1 receives a command from the control unit 11, acquires a landscape image every second, and passes it to the control unit 11 (step 2).
2). The camera attribute information acquisition unit 3 receives a command from the control unit 11, acquires a camera angle of a landscape image recording device such as a camera at the time of image capturing as a set of a horizontal angle and an elevation angle (step 23), and simultaneously has a landscape having a zoom function. If it is an image device, the focal length is obtained (step 24). Since the image size is fixed for each landscape image device, the control unit 11 holds the image size information. The control unit 11 holds the collected information as a landscape image file.

FIG. 3 shows the file format of the data structure of the landscape image file. The landscape image file has header information and image data. The header information includes position information, camera angle information, focal length, time information, image size, type, and size of an image file. As location information,
East longitude, north latitude, and elevation data (for example, 137 degrees east 5 degrees
5 minutes 10 seconds, latitude 34 degrees 34 minutes 30 seconds north, altitude 101m3
3 cm). The camera angle includes horizontal angle data and elevation angle data (for example, horizontal angle 254 degrees clockwise, elevation angle 15 degrees, etc.). The focal length data is the focal length (for example, 28 mm) of the camera lens at the time of capturing an image. The time information includes the time at the time of shooting (for example, 15:06:17, January 31, 1997, Japan time). As the image size of the image file, the vertical and horizontal pixel sizes (for example, 64
0 × 480). File type (TIF
E format, 8-bit color, etc.). It also has the number of bytes of the file (307.2 KB, etc.). The image data itself has, for example, a binary format.

After storing the landscape image file, the control unit 11 instructs the image processing unit 4 to extract the outline from the landscape image and divide the landscape image into a plurality of regions.
The image processing unit 4 performs a differentiation process on the basis of the density difference in the landscape image to extract a contour line (step 2).
5) The region is divided by performing labeling with the outline as a boundary (step 26). Note that the technical term of labeling used here is a technical term used in image area division, and is different from landscape labeling. As a procedure, first, the image is converted into a monochrome grayscale image. Since the contour is a portion where the brightness changes suddenly, the contour is extracted by performing a differentiation process to find a portion where the differential value is larger than a threshold value. At this time, the line width of the outline is one pixel, and the outlines are connected. For this purpose, a thinning process is performed to obtain a connected line having a line width of one pixel. Here, it is sufficient to use a conventional method for the differentiation processing and the thinning processing.

The obtained contour is regarded as the contour of the area, and an operation of numbering the area constituted by the contour is performed.
The largest number among the numbers is the number of areas, and the number of pixels in the area indicates the area of the area. FIG. 8 shows an example in which a landscape image is divided into a plurality of partial regions. Note that a measure of similarity (closeness) between regions may be introduced, and a clustering process may be performed to combine a plurality of regions having similar properties into one region. Any existing clustering method may be used.

When the control unit 11 completes the region division processing of the landscape image, it passes the header information of the landscape image file to the map information management unit 5, accesses the map DB, and performs the processing of calculating the visual field space. Make a request (Step 2
7). An example of the map information management unit 5 is a map database program. The map information management unit 5 manages three-dimensional map data. Although two-dimensional map data may be used, in this case, since there is no height information, the accuracy of the position at which the labeling is applied to the actual scenery is inferior. When the two-dimensional map data is used as the basis, the processing is performed by supplementing the height information. For example, in the case of two-dimensional data of a house, if there is floor information indicating the number of floors of the house, the height of the house is estimated by multiplying the number of floors by a certain number, and estimated as two-dimensional data. 3D data is created based on the height information. Even if there is no floor information,
Height information can be estimated by, for example, allocating a certain number of heights according to the area of the house figure, and similarly, three-dimensional data is created based on the estimated height information. In this way, three-dimensional data is created and the process proceeds.

FIG. 4 shows an example of three-dimensional map data. FIG.
(1) shows a map information space expressed in two dimensions, and FIG.
(2) shows a map information space expressed in three dimensions. This 3
With respect to the three-dimensional map information space, the map information management unit 5 receives a command from the control unit 11 and calculates a visual field space based on the header information of the landscape image file (step 28). FIG. 5 shows a calculation example of the visual field space. First, the XY axes stretch in the horizontal direction,
The Z axis extends in the vertical direction. The position of the viewpoint E is set in the three-dimensional map information space from the position information in the header information of the landscape image file. For example, 137 degrees east longitude 55
Minute 19 seconds, latitude 34 degrees 34 minutes 30 seconds north, altitude 101m33
If it is cm, the corresponding coordinates in the map mesh number corresponding to the cm are set. Similarly, the camera angle direction is set based on the horizontal angle and the elevation angle in the camera angle information in the header information.
The focal point F is set at a point advanced from the viewpoint E by the focal length on a straight line representing the camera angle direction. The line-of-sight direction vector is a unit vector having a length of 1 from the viewpoint E on the straight line. In the image size of the landscape image file, the width x on the X-axis of the camera screen is set from the horizontal size, and the width y on the Y-axis is set from the vertical size. The horizontal x vertical y plane is set so as to be perpendicular to the camera angle direction with respect to the line of sight vector and to include the focal point F. Straight lines connecting the four corner points of the camera screen are obtained from the coordinates of the viewpoint E, and a three-dimensional space formed by four half lines extending from the viewpoint E is defined as a visual field space. FIG. 6 shows an example of the visual field space in the three-dimensional map space. XZ 3D map space
It is viewed from a plane. FIG. 7 is a cross-sectional view along the XY plane of a space that is surrounded by oblique lines in FIG. 6 and that belongs to the viewing space. In the example of FIG. 6, buildings and mountains in the visual field space are included.

Further, the map information management unit 5 determines a structure existing in the determined visual space. For each structure, it is calculated whether or not each vertex constituting the solid representing the structure exists in the internal region of the viewing space. Usually, a two-dimensional map space is divided by a two-dimensional mesh of a fixed size. As a method of cutting a mesh in the three-dimensional map space, a mesh is cut at regular intervals in the height direction in addition to the two-dimensional mesh in the vertical and horizontal directions. The space is divided by a rectangular parallelepiped unit space. First, the presence or absence of an overlapping portion between the rectangular parallelepiped unit space and the visual field space is checked, and the number of the three-dimensional unit map space having the overlapping portion is obtained. The number of the three-dimensional unit map space here is the same as a so-called mesh number. For structures in the overlapping 3D unit map space,
Investigate whether there is any overlap with the visual field space. A straight line connecting the coordinates of the vertices constituting the structure and the coordinates of the viewpoint is obtained. If the straight line has an intersection with the camera screen of FIG. 7, it is within the visual field space. If at least one vertex of the plurality of vertices constituting the structure satisfies this condition, the structure has an overlapping portion with the viewing space.

If a structure is included in the view space or a part of the structure, the process enters a process of three-dimensionally projecting each structure onto the projection surface using the camera screen as a projection surface (step 29). ). Here, as shown in FIG. 7, after the point P is re-expressed in the coordinate system based on the viewpoint E based on the following equation (1), the point P is projected on the camera screen to obtain the intersection Q. .

[0038]

Embedded image Here, point P = (x, y, z): coordinates of the vertices constituting the structure point E = (ex, ey, ez): coordinates of the viewpoint vector L = (lx, ly, lz): branch line direction vector (Unit vector) Point P ′ = (x ′, y ′, z ′): coordinates r = (lx ² + ly ² ) ^1/2 when the point P is expressed based on the viewpoint E of the point P = 4 points Q = ( x, y): The projection point t of the point P on the camera screen is the focal length. In the three-dimensional projection conversion, first, for each structure, the surface on which the vertex extends is obtained. For example, in the case of a structure represented by a rectangular parallelepiped, six surfaces are obtained. When each plane is projected onto the camera screen, the distance between the viewpoint and the corresponding point on the plane is calculated for each pixel on the camera screen included in the projection area, and stored as a depth value (Z value) in the memory. Store. For each surface of each structure, the depth value (Z
Value) and store it in memory. Note that z ′ in (Equation 1) represents a depth value (Z value) from the viewpoint.

Among the structures three-dimensionally projected and transformed on the camera screen, there are structures visible from the viewpoint and structures not visible. Among them, it is necessary to find only the structure that can be seen from the viewpoint, and to find the surface on the opposite side from the viewpoint or the surface that is blocked by other structures. Therefore, hidden surface processing is performed (step 3
0). There are various methods for processing the hidden surface.
Use the buffer method. Other scan line methods and ray tracing methods may be used.

A pixel on the camera screen is arbitrarily determined, and a plane having the smallest depth value for the pixel is determined. As described above, when processing is sequentially performed on each surface of each structure, the surface closest to the viewpoint is left for each pixel on the camera screen. The plane closest to the viewpoint is determined for each pixel on the camera screen, and pixels on the camera screen that share the plane closest to the viewpoint generally form an area.
There can be a plurality of regions consisting of pixels with the common surface being the closest surface. The region obtained in this way is a region obtained by performing a three-dimensional projection conversion of the partial region of the structure seen from the viewpoint. Surfaces on the other side of the view and those obstructed by other structures have been erased.

The area thus formed forms a CG image area (step 31).

With respect to the vertex coordinates of the two-dimensional figure constituting the CG image area, three-dimensional coordinates before projection transformation are obtained, and the correspondence between the two is stored in the memory as link information. It is used to determine which structure the two-dimensional area is a projection of based on the link information.

Based on the line data remaining after erasing hidden lines,
The CG image is divided into regions. Since the three-dimensional map DB is used, the name of the structure serving as the basis of each area can be associated with each area. The divided areas of the CG image are numbered sequentially. FIG. 9 shows an example in which a CG image is divided into a plurality of partial areas.

When the CG image region division processing is completed, the control unit 11 instructs the label information creation unit 7 to associate the CG image division region with the landscape image division region. The label information creating unit 7 associates the divided regions of the CG image with the divided regions of the landscape image by template matching (step 32, see FIG. 11).

Among the divided regions of the landscape image, the regions are sequentially associated with the divided regions of the CG image in ascending order of the number (for example, No. 1). For matching, any of the conventional matching methods may be used, but here, a simple template matching method is used. That is, the two regions to be compared are superimposed, and when the ratio of the overlapping portion is equal to or more than a certain ratio determined as the threshold value, the regions are associated as the region relating to the same structure (step 33). For example, the coordinate value of each pixel in the first divided region R1 of the landscape image is set to (A, B). The value of the pixel at coordinates (A, B) is 1 because it is inside the area. In the first divided area S1 of the CG image, if the coordinates (A, B) are within the area S1, the pixel value is 1 and overlaps. Thus, the overlap coefficient K at the coordinates (A, B)
(A, B) is determined to be 1 when overlapping and 0 when not overlapping. The coordinates (A, B) are moved in the region R1, and the overlap coefficient K (A, B) is obtained. Then, for the number N1 of coordinates (A, B) moved in the region R1, the overlap coefficient K
The number N2 of coordinates where (A, B) was 1 is obtained, and N1 /
When N2 is equal to or larger than the threshold value, it is determined that the divided region R1 of the landscape image and the divided region S1 of the CG image correspond to each other. This association is performed from the first to the last divided area of the landscape image. In addition, as the matching method, an evaluation function that has the same value even if there is a slight displacement in the XY directions may be used.

After associating the partial area of the CG image with the partial area of the landscape image, the label information creating unit 7 further obtains information to be superimposed on each partial area of the landscape image, The process for creating information (step 34) is entered. First, a partial area of the CG image corresponding to the partial area of the landscape image is extracted. The partial region of the extracted CG image is originally obtained by three-dimensionally projecting and converting a certain surface of the three-dimensional structure in the three-dimensional map space onto the camera screen. Therefore, the surface of the three-dimensional structure on which the three-dimensional projection conversion is based is obtained using the depth value (Z value) of the partial region of the CG image as a key. The link information created at the time of the three-dimensional projection conversion may be used as a key. Based on the surface of the original structure, the three-dimensional map DB is accessed to acquire the name or attribute information of the structure. Here, the attribute information means information accompanying the structure, and may be any information as long as the information is related to the structure. Then, the position coordinates where the name or attribute information is to be superimposed are determined for the partial area of the landscape image. How to decide may be decided any way. For example, the center of gravity of the figure extending the partial area may be used. Label information is created from the name or attribute information of the structure and the assigned position coordinates (step 34). At this time, the destination identification flag is set to “1” to indicate that the structure is the target condition specified by the user in the target condition specifying unit 12 to indicate the target condition. Table 1
Shows an example of label information.

[0047]

[Table 1] When the label information creation unit 7 finishes creating the label information, it passes the label information to the control unit 11.

Upon receiving the label information, the control unit 11
It instructs the label information output unit 8 to display the label information on the visual device. Here, the visual device includes a video display device such as a display or a head-mounted display. The label information output unit 8 superimposes the name or attribute information of the structure in the label information at a position in the landscape image (step 35), and displays the superimposed landscape image on the video display device (step 36). Here, for a structure whose destination identification flag is “1”, it is displayed that the structure matches the destination condition.

Here, when the user designates the position on the screen of the visual device by the screen position designation unit 6, the control unit 11
The structure in the label information corresponding to the designated screen position is obtained, the communication address of the communication device associated with the obtained structure is obtained from the communication address storage unit 9, and the communication address is passed to the communication processing unit 10 (step 37). ). Communication processing unit 10
Displays the received communication address information on the screen of the visual device, and sets a communication path with the communication address in response to the user's communication start request (step 38).

The label information output unit 8 may display the communication address on the screen of the visual device without passing the communication address to the communication processing unit 10.

When the scenery labeling process is performed continuously, the above-described series of processing procedures is executed again.

FIG. 12 is a configuration diagram of a landscape label-based communication system in which the communication device of FIG. 1 is applied to a communication system.

The landscape label-based communication system includes a communication terminal 40, a communication center 50, and a communication network 60.

The communication terminal 40 includes an image acquisition unit 41 for acquiring an image, a position information acquisition unit 42 for acquiring a camera position at the time of image acquisition, and a camera for acquiring a camera angle, a focal length, and an image size at the time of image acquisition. An attribute information acquisition unit 43, an image processing unit 45 that divides the acquired image into a plurality of partial areas,
Via the communication network 60, information relating to the area division of the image, the camera position, the camera angle, the focal length, the image size, the destination identification information indicating the target place or the building, and the information of the structure in the label information are transmitted. Send to the communication center 50,
A communication control unit 46 for receiving label information and a communication address from the communication center 50; and superimposing the name or attribute information of the structure in the label information at a position corresponding to the assigned position in the image, and visualizing the superimposed image. If the destination identification flag in the label information is “1”, a label information output unit 47 that indicates that the structure is a target place or building and a communication address are passed. A communication processing unit 48 for setting a path to the communication address;
The user can indicate the position on the screen of the visual device, and when the user specifies the position on the screen of the visual device, a structure in the label information corresponding to the specified screen position is obtained. A screen position specifying unit 44 that outputs information to the communication control unit 46 and passes the communication address of the communication device associated with the structure received by the communication control unit 46 to the communication processing unit 10, and uses a target condition. Condition specification part 4 to be specified by the user
4 'and a terminal control unit 49.

The communication center 50 receives information from the communication terminal 40 via the communication network 60 on the area division of the image, the camera position, the camera angle, the focal length, the image size, the destination identification information, and the information on the structure in the label information. And a communication control unit 53 that transmits label information and a communication address to the communication terminal 40, manages the map information, and stores the map information space in the map information space based on the received camera position, camera angle, focal length, and image size. A map information management unit 51 for obtaining a view space and acquiring structures existing in the view space, and a communication address of a communication device associated with each structure are stored, and a structure indicated by information of the structure in the label information A communication address storage unit 55 that outputs a communication address of an object, and the acquired structure is associated with the partial region of an image by pattern matching, and the associated structure is Create label information including the name or attribute information of the object and the assigned position, and if the structure is the target place or structure specified by the user (if the destination identification information is received), the purpose It is composed of a label information creation unit 52 that sets the location identification flag to “1”, and a center control unit 54 that controls the above units.

[0056]

As described above, according to the present invention, the geographical information on the computer and each part in the scenery image of the actual scene can be presented to the user in association with each other.
It is not necessary for a human to compare the map on the computer with the actual scenery and associate it with the human, and it is possible to know the telephone number of the resident etc. living in the building at that place simply by touching the place on the screen It is possible to make a telephone call or the like as it is, and it can be a new telephone directory convenient for the user.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a landscape label-based communication device according to an embodiment of the present invention.

FIG. 2 is a flowchart of a process of the communication device using a landscape label of FIG. 1;

FIG. 3 is a diagram showing a data structure of a landscape image file.

FIG. 4 is a diagram showing an example of a two-dimensional map (FIG. 1 (1)) and its three-dimensional map (FIG. 2 (2)).

FIG. 5 is a diagram illustrating a calculation method of a visual field space.

FIG. 6 is a diagram illustrating an example of a visual field space in a three-dimensional map space.

FIG. 7 is a diagram showing an example of a projection view.

FIG. 8 is a diagram showing an example of area division of a landscape image.

FIG. 9 is a diagram illustrating an example of area division of a CG image.

FIG. 10 is an explanatory diagram of pattern matching between a partial region of a landscape image and a partial region of a CG image.

FIG. 11 is a diagram illustrating an example of superimposition of label information on a landscape image.

FIG. 12 is a configuration diagram of a landscape communication system of the present invention.

FIG. 13 is a configuration diagram of a navigation device disclosed in Japanese Patent Application Laid-Open No. 8-273000.

FIG. 14 is a diagram illustrating a display example of a moving image.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image acquisition part 2 Position information acquisition part 3 Camera attribute information acquisition part 4 Image processing part 5 Map information management part 6 Screen position specification part 7 Label information creation part 8 Label information output part 9 Communication address storage part 10 Communication processing part 11 Control Unit 12 target condition specifying unit 21 to 38 step 40 communication terminal 41 image obtaining unit 42 position information obtaining unit 43 camera attribute information obtaining unit 44 screen position specifying unit 44 'target condition specifying unit 45 image processing unit 46 communication control unit 47 label information Output unit 48 Communication processing unit 49 Terminal control unit 50 Communication center 51 Map information management unit 52 Label information creation unit 53 Communication control unit 54 Center control unit 55 Communication address storage unit 60 Communication network

Continued on the front page (72) Inventor Takeshi Ikeda 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Akira Suzuki 3-192-1, Nishi-Shinjuku, Shinjuku-ku, Tokyo Sun (56) References JP-A-8-255239 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00-17/50 G09B 23/00 -29/14 G08G 1/00-9/02

Claims (12)

(57) [Claims] An image acquisition unit for acquiring an image; a position information acquisition unit for acquiring a camera position at the time of image acquisition; a camera attribute information acquisition unit for acquiring a camera angle, a focal length, and an image size at the time of image acquisition. , Image processing means for dividing the acquired image into a plurality of partial areas, managing map information, and calculating a visual field space in the map information space based on the acquired camera position, camera angle, focal length, and image size, Map information management means for acquiring a structure existing in the visual field space, and associating the acquired structure with the partial region of the image, and assigning the name or attribute information of the associated structure and assignment Label information creating means for creating label information including a position; communication address storage means for storing a communication address of a communication device associated with the structure; Communication processing means for setting a communication path with the communication address based on the information, and the name of the structure or its attribute information is superimposed on the position in the image corresponding to the assigned position in the acquired label information. Label information output means for displaying the image on a visual device, and allowing a user to indicate a position on the screen of the visual device, wherein the user indicates a position on the screen of the visual device,
The structure in the label information corresponding to the designated screen position is obtained, the communication address of the communication device associated with the obtained structure is obtained from the communication address storage unit, and the communication address is sent to the communication processing unit. A landscape label-based communication device, comprising: a screen position designation unit to be passed; and a control unit that controls each of the units. 2. The communication processing means, when the user designates a position on the screen of the visual device, displays the received communication address information on the screen of the visual device, and requests the user to start communication. It is configured to set a communication path with the communication address according to
The device according to claim 1. 3. An image acquisition unit for acquiring an image, a position information acquisition unit for acquiring a camera position at the time of image acquisition, a camera attribute information acquisition unit for acquiring a camera angle, a focal length, and an image size at the time of image acquisition. , Image processing means for dividing the acquired image into a plurality of partial areas, managing map information, and calculating a visual field space in the map information space based on the acquired camera position, camera angle, focal length, and image size, Map information management means for acquiring a structure existing in the visual field space, and associating the acquired structure with the partial region of the image, and assigning the name or attribute information of the associated structure and assignment Label information creating means for creating label information including a position; communication address storage means for storing a communication address of a communication device associated with the structure; Label information for superimposing the name of the structure or its attribute information at a position in the image corresponding to the position, displaying the superimposed image on a visual device, and receiving the communication address on the visual device. Output means, and may allow a user to indicate the position on the screen of the visual device, and when the user indicates the position on the screen of the visual device,
The structure in the label information corresponding to the designated screen position is obtained, the communication address of a communication device associated with the obtained structure is obtained from the communication address storage means, and the communication address is output to the label information output means. A landscape position using type communication device, comprising: a screen position designating means for passing to a user; and a control means for controlling each of the means. 4. The apparatus according to claim 1, further comprising a target condition designating unit for designating a target condition by the user, wherein the label information creating unit identifies the target condition designated by the user as a target condition. Information is added to the label information, and the label information output means displays, when the destination identification information is added to the label information, that the structure matches the destination condition. Apparatus according to any one of the preceding claims, wherein the apparatus is configured. 5. The label information creating means creates a CG image, which is a computer graphics image, based on the acquired structure, and a part of the CG image by pattern matching with respect to the partial area of the image. The apparatus according to any one of claims 1 to 4, wherein the regions are associated with each other, a structure of the associated partial region is obtained, and label information including the name or attribute information of the structure and an assigned position is created. 6. The label information creating means performs a three-dimensional projection conversion of the acquired structure on a camera screen, creates a CG image by erasing a structure that cannot be seen from a viewpoint, and generates a contour of a partial area in the CG image. Dividing the CG image into partial areas by lines,
The partial area of the image and the partial area of the CG image are associated with each other by pattern matching, and the structure that is the basis of the partial area of the CG image is determined for the partial area of the image. The apparatus according to claim 1, wherein label information including name or attribute information and an assigned position is created. 7. A communication terminal, comprising: a communication terminal; a communication center; wherein the communication terminal is an image acquisition unit for acquiring an image, a position information acquisition unit for acquiring a camera position at the time of image acquisition, a camera angle and a focus at the time of image acquisition. Camera attribute information acquiring means for acquiring distance and image size, image processing means for dividing the acquired image into a plurality of partial areas, information on area division of the image, the camera position, the camera angle, the focal length, Communication control means for transmitting the information on the structure in the image size and label information to the communication terminal via a communication network, and receiving label information and a communication address from the communication center, based on the received communication address Communication processing means for setting a path with the communication address, and overlapping the name or attribute information of the structure in the label information with a position corresponding to the assigned position in the image. Then, label information output means for displaying the superimposed image on a visual device, and can cause a user to indicate the position on the screen of the visual device, when the user indicates the position on the screen of the visual device, A communication device associated with the structure, which obtains the structure in the label information corresponding to the instructed screen position, outputs information of the structure to the communication control means, and is received by the communication control means; Screen position specifying means for passing a communication address of the image processing apparatus to the communication processing means, and terminal control means for controlling each of the means. The communication center relates to area division of the image from the communication terminal via the communication network. Communication for receiving information, the camera position, the camera angle, the focal length, the image size, and information on a structure in the label information, and transmitting the label information and a communication address to the communication terminal; A map that manages control means and map information, obtains a visual space in the map information space based on the received camera position, camera angle, focal length, and image size, and acquires structures existing in the visual space. Information management means for associating the acquired structure with the partial area of the image, and label information creating means for creating the label information including the name or attribute information and the assigned position of the associated structure Communication address storage means for storing a communication address of a communication device associated with the structure, and outputting a communication address of the structure indicated by the information of the structure in the label information, and a center control for controlling the respective means Landscape label using communication system having means. 8. When the user designates a position on the screen of the visual device, the communication processing means displays the received communication address information on the screen of the visual device, and requests the user to start communication. The system according to claim 7, wherein the system is configured to set a communication path with the communication address in response to the request. 9. A communication terminal, comprising: a communication center; an image acquisition unit for acquiring an image; a position information acquisition unit for acquiring a camera position at the time of image acquisition; a camera angle and a focus at the time of image acquisition. Camera attribute information acquiring means for acquiring distance and image size, image processing means for dividing the acquired image into a plurality of partial areas, information on area division of the image, the camera position, the camera angle, the focal length, Communication control means for transmitting the information on the structure in the image size and the label information to the terminal via a communication network, and receiving the label information and the communication address from the communication center; and the name of the structure in the label information Alternatively, the attribute information is superimposed on a position corresponding to the application position in the image, the superimposed image is displayed on a visual device, and when the communication address is received, the attribute information is displayed on the visual device. Label information output means to be displayed on the screen and the user to indicate a position on the screen of the visual device, and when the user indicates a position on the screen of the visual device, the user corresponds to the specified screen position. Obtaining the structure in the label information, outputting the information of the structure to the communication control means, and receiving the communication address of the communication device associated with the structure, received by the communication control means, in the label information output means And a terminal control unit for controlling each of the units, wherein the communication center is configured to provide information on area division of the image, the camera position, and the camera from the communication terminal via the communication network. Communication control means for receiving information on a corner, the focal length, the image size, and a structure in the label information, and transmitting the label information and the communication address to the communication terminal; Map information management means for obtaining a view space in the map information space based on the received camera position, camera angle, focal length, and image size, and acquiring a structure existing in the view space, Label information creating means for associating the acquired structure with the partial region of the image, and creating the label information including the name or attribute information and the assigned position of the associated structure, Use of a landscape label having communication address storage means for storing a communication address of an associated communication device and outputting a communication address of a structure indicated by the information of the structure in the label information, and a center control means for controlling the respective means Communications system. 10. The communication terminal includes a target condition designating unit that allows a user to designate a target condition, and the label information creating unit determines that the target condition designated by the user is a target condition. The label information output means is configured to add the destination condition identification information to the label information, and that the destination object identification information is added to the label information, and that the structure matches the destination condition. 10. The system according to any one of claims 7 to 9, configured to display. 11. The label information creating means creates a CG image, which is a computer graphics image, based on the acquired structure, and a part in the CG image by pattern matching with respect to the partial area of the image. The system according to any one of claims 7 to 10, wherein regions are associated with each other, a structure of the associated partial region is obtained, and label information including a name or attribute information of the structure and an assigned position is created. 12. The label information creating means performs a three-dimensional projection conversion of the acquired structure on a camera screen, creates a CG image by erasing a structure that cannot be seen from a viewpoint, and generates a contour of a partial area in the CG image. The CG image is divided into partial regions by lines, and the partial regions of the image and the partial regions of the CG image are associated by pattern matching, and the base of the partial region of the CG image is associated with the partial regions of the image. The system according to any one of claims 7 to 10, wherein the system obtains a structure that has become unusual, and creates label information including the name or attribute information of the structure and the assigned position.