JP3156645B2 - Information transmission type landscape labeling device and system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention superimposes and displays, on an image display device, geographical information about each partial area in an image taken by a user using a landscape image input device such as a camera. The present invention relates to a device that teaches a user by voice guidance or the like.

[0002]

2. Description of the Related Art Conventionally, there have been various navigation systems as systems for instructing users on geographical information about the area where the users are located.

FIG. 14 is a block diagram of a navigation device disclosed in Japanese Patent Application Laid-Open No. 8-273000. This device receives a vehicle position data and a motion data, and receives a position update unit 71 that updates the vehicle position with reference to the road map data, and displays display road data and display background data based on the map data and the like. Display data generator 7 to be generated
2, a three-dimensional moving image data creating unit 73 for creating three-dimensional moving image data based on these display data, and a storage unit 74.
When a travel route including a waypoint is set in advance, the device has a function of setting a route while viewing a real moving image display screen along a road that actually exists instead of a map screen.

According to this device, when a user travels along an actual route, the user can see a moving image display (for example, FIG. 15) along the route.

[0005]

However, in the case of using the same device, the human finally associates the real scene with the geographical information in the computer world with the naked eye, so that the person in the real scene can be used. You have to recognize what is. In other words, what is the actual building, road or mountain in front of the user is displayed on a moving image, and based on the symbols etc. in the map, humans rely on the naked eye to work the human brain unconsciously. It must be understood by performing the work of association. At street corners, etc., comparing the map on the computer with the actual scenery, grasping the direction and finding landmarks, gazing at the direction, understanding the characteristics of the building in that direction, and then looking at the map again Understand what the building is.

[0006] For this reason, there is a problem that the trouble of comparing the map on the computer with the actual scenery many times and associating with the human cannot be omitted. Especially in the dark or at night, it is difficult to see the actual scenery and to take measures.

An object of the present invention is to provide a landscape labeling apparatus and system for associating a user with geographical information on a computer and each part in an image of a real landscape (hereinafter referred to as a landscape image). It is to be.

[0008]

According to the present invention, map data on a computer is created in advance as three-dimensional data, and a position at which an image (hereinafter referred to as a landscape image to distinguish it from a CG image) is input. (Camera position), camera angle, focal length, and image size are acquired at the time of shooting, and computer graphics when viewed from the position, camera angle, and focal length at the time of shooting a real scene in a three-dimensional map space on a computer (Hereinafter, this is referred to as CG.) This is to achieve the association by acquiring geographical information in an image and superimposing and displaying the geographical information on a landscape image which is a real scene. The geographic information is the name of a structure or the like or its attribute information in an image, and the attribute information means information on all attributes (for example, contour, color, etc.) of the structure. In this specification, the term structure is used to mean all data having a certain geographical structure in the map DB, including natural topography such as mountains, rivers, and the sea, in addition to artificial structures. When acquiring geographic information, a landscape image is obtained based on a camera position, a camera angle, a focal length, and an image size, and a structure of a plurality of images is obtained. The position of the landscape image where the structure is to be taken (hereinafter referred to as the assigned position) is obtained, and the name or attribute information of the structure is superimposed and displayed.

Further, in order to further improve the accuracy of associating the structure in the landscape image with the structure in the CG image,
The previously acquired structure is associated with each partial region of the landscape image by pattern matching. A CG image is created on the basis of the acquired structure, a partial region in the CG image is associated with the partial region of the landscape image by pattern matching, and the structure as a source of the associated partial region Ask for.

Here, an example of a method for creating a CG image will be described. By accessing the three-dimensional map DB based on the camera position, camera angle, focal length, and image size obtained earlier,
Obtain the visual field space in the three-dimensional map space. A structure in the visual field space is obtained, and three-dimensional data of each structure is three-dimensionally projected and converted onto the projection plane using the camera screen as a projection plane. Further, of the line data constituting the projected figure of each structure, line data hidden by other structures and invisible is erased using a normal vector method or the like. The CG image is divided into regions based on the line data remaining after the elimination of the hidden lines. 3D map DB
Is used, the name of the structure that is the basis of the area can be associated with each area.

Then, the structure name of the partial area of the CG image associated with each partial area of the landscape image is extracted by pattern matching. The position coordinates of the actual scenery image on which the extracted structure name is to be superimposed are obtained by three-dimensionally projecting the position coordinates of the structure in the three-dimensional map space onto the projection plane. Label information is created from the position coordinates of the actual scenery image on which the extracted structure name is to be superimposed. Based on the label information, the structure name is superimposed on a landscape image as a real scene and displayed on a visual device.

The present invention further comprises transmission information management means for receiving, registering, and retrieving advertisement information and guidance information from a user who wants to transmit information such as a shopping street. As a result, the information of the user on the transmitting side functions like an advertisement and is transmitted to the user on the receiving side.

The transmission information management means manages information to be transmitted (called transmission information) in advance using the number of the structure as a key. After the label information creation means creates the label information in the above procedure and passes it to the label information output means via the control means, it also passes the label information to the transmission information management means. Then, the transmission information management means searches for the structure in the label information as to whether or not there is transmission information by using the structure as a key. As a result of the search, if there is information to be transmitted, the transmission information is transmitted through the control means. Pass to label information output means. The label information output means displays the transmission information as attribute information of the structure.

An information transmitting type landscape labeling apparatus according to the present invention comprises: an image obtaining means for obtaining an image; a position information obtaining means for obtaining a camera position at the time of obtaining the image; a camera angle and a focal length when the image is obtained; A camera attribute information acquisition unit that acquires an image size, and manages map information, obtains a visual field space in a map information space based on the acquired camera position, camera angle, focal length, and image size. Map information management means for acquiring an existing structure, label information creation means for creating label information including the name of the structure or its attribute information, and an assigned position, and transmission information from a user who desires information transmission. Transmission information management means for managing corresponding to the structure, checking presence / absence of transmission information regarding the structure in the label information, and outputting the transmission information if the transmission information exists; The name of the structure or its attribute information is superimposed on the position in the image corresponding to the position information in the created label information, the superimposed image is displayed on the visual device, and the transmission information is output. Then, there are label information output means for displaying the transmission information as attribute information of the structure, and control means for controlling each of the above means.

According to another aspect of the present invention, there is provided an information transmitting type landscape labeling apparatus, comprising: an image obtaining unit for obtaining an image; a position information obtaining unit for obtaining a camera position at the time of obtaining the image; a camera angle and a focal length at the time of obtaining the image; A camera attribute information acquiring unit for acquiring an image size, an image processing unit for dividing an acquired image into a plurality of partial areas, and managing map information, based on the acquired camera position, camera angle, focal length, and image size. In the map information space, a view space is obtained, and map information management means for acquiring a structure existing in the view space is associated with the acquired structure with respect to the partial region of the image. Label information creating means for creating label information including the name of the structure or its attribute information and the assigned position, and transmission information from a user who desires information transmission corresponding to each structure. The presence or absence of transmission information on the structure in the label information, and if there is transmission information, transmission information management means for outputting the transmission information; and an image corresponding to the position information of the created label information. The name of the structure or its attribute information is superimposed at the middle position, the superimposed image is displayed on the visual device, and when the transmission information is output, the transmission information is displayed as the attribute information of the structure. It has a label information output unit and a control unit for controlling each of the above units. According to an embodiment of the present invention, the label information creating means creates a CG image based on the acquired structure,
A partial area in the CG image is associated with the partial area of the image by pattern matching, a structure of the associated partial area is obtained, and label information including the name or attribute information of the structure and the assigned position Create

According to an embodiment of the present invention, the label information creating means performs a three-dimensional projection conversion of the acquired structure on a camera screen, erases a structure that cannot be seen from the viewpoint, creates a CG image, and creates a CG image. The CG image is divided into partial regions by the contour line of the middle partial region, and the partial region of the image is
The partial area of the G image is associated with the partial area of the image by pattern matching, the partial area of the image is associated, the structure on which the partial area of the CG image is based is obtained, and the name or attribute information of the structure and the assigned position are determined. Create label information to include.

An information transmission type landscape labeling system according to the present invention comprises a landscape labeling terminal and a landscape labeling center, wherein the landscape labeling terminal includes image acquisition means for acquiring an image, and position information for acquiring a camera position at the time of image acquisition. Acquiring means, camera attribute information acquiring means for acquiring a camera angle, a focal length, and an image size at the time of acquiring an image, image processing means for dividing the acquired image into a plurality of partial areas, and information regarding area division of the image. Communication control means for transmitting the camera position, the camera angle, the focal length, and the image size to the landscape labeling center via a communication network, and receiving label information and transmission information from the landscape labeling center; and The name of the structure in the information or its attribute information is superimposed on the corresponding position in the image, and the superimposed image is viewed. A display unit for displaying the transmitted information as attribute information of the structure when the transmitted information is displayed on a device, and a terminal control unit for controlling the respective units, and The labeling center receives information related to area division of the image, the camera position, the camera angle, the focal length, and the image size from the landscape labeling terminal via the communication network,
Communication control means for transmitting the label information and the transmission information to the scenery labeling terminal; managing the map information; and a view space within the map information space based on the received camera position, camera angle, focal length, and image size. And map information management means for acquiring a structure existing in the visual field space, and associating the acquired structure with the partial region of the image, and the name or attribute of the associated structure A label information creating unit for creating the label information including information and an assigned position, and information transmitted from a user who desires information transmission is managed in correspondence with each structure, and transmitted with respect to the structure in the label information. It has transmission information management means for checking the presence or absence of information and outputting the transmission information if it exists, and a center control means for controlling each of the above means.

[0018]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a configuration diagram of an information transmission type landscape labeling apparatus according to a first embodiment of the present invention.

The landscape labeling apparatus according to the present embodiment acquires an image, for example, an image acquisition unit 1 which is a digital camera, and acquires a camera position when acquiring an image, for example, a position information acquisition unit which is a GPS receiver. 2. When acquiring an image, acquire a camera angle, a focal length, and an image size;
For example, a camera attribute information acquisition unit 3, which is a three-dimensional electronic compass attached to a digital camera, manages map information, and provides a visual field in a map information space based on the acquired camera position, camera angle, focal length, and image size. Obtain a space and obtain structures existing in the visual space, for example, map D
A map information management unit 5, which is a B management program; a label information creation unit 6A, which creates label information including a name, an outline, and a position of a structure; and transmission information from a user who wants to transmit information corresponding to each structure. The presence / absence of transmission information on the structure in the label information is checked, and if there is transmission information, the transmission information management unit 7 that outputs the transmission information; The name or outline of the map information is superimposed on the position in the image corresponding to the information,
A label information output unit 8A for displaying the superimposed image on a visual device (not shown) and displaying the transmission information as attribute information of the structure when the transmission information is output; It is composed of a control unit 9A for controlling.

FIG. 2 is a configuration diagram of an information transmission type landscape labeling apparatus according to a second embodiment of the present invention, and FIG. 3 is a flowchart of processing of the landscape labeling apparatus of FIG.

The landscape labeling apparatus of the present embodiment acquires a landscape image, for example, an image acquisition unit 1 which is a digital camera, and acquires a camera position at the time of acquiring an image.
For example, a position information acquisition unit 2 that is a GPS receiver, and a camera angle, a focal length, and an image size that are also acquired when acquiring an image.
Camera attribute information acquisition unit 3 which is a three-dimensional electronic compass, and image processing unit 4 which divides the acquired image into a plurality of partial areas
Map information management that manages geographic information, obtains a visual space in the map information space based on the acquired camera position, camera angle, focal length, and image size, and acquires structures existing in the visual space. A label information creating unit 6B that associates the unit 5 with the acquired structure with respect to the partial region of the image by pattern matching, and creates label information including the name or attribute information and the assigned position of the associated structure
And information transmitted from a user who desires information transmission is managed in association with each structure, and the presence or absence of transmission information is checked with respect to the structure in the label information. The output information manager 7 superimposes the name of the structure or its attribute information on the position in the image corresponding to the information on the assigned position in the created label information, and displays the superimposed image on the visual device. Further, when the transmission information is output, a label information output unit 8B for displaying the transmission information as attribute information of the structure, and a control unit 9B for controlling each of the units 1 to 8B.

Next, the operation of this embodiment will be described in detail.

When the landscape labeling device is activated, first, the control unit 9B obtains information on the landscape image by:
A processing start command is sent to the position information acquisition unit 2, the camera attribute information acquisition unit 3, and the image acquisition unit 1. The position information acquisition unit 2 receives a command from the control unit 9B, collects position information with a GPS receiver or the like every second, and passes it to the control unit 9B (step 21). Here, the time interval is not limited to the unit of seconds, but may be any value. The camera attribute information acquisition unit 3 includes a control unit 9B
(Step 2), the camera angle of the landscape image recording device such as a camera at the time of image capturing is acquired as a set of the horizontal angle and the elevation angle.
2) At the same time, if it is a landscape image device having a zoom function, the focal length is obtained (step 23). Image acquisition unit 1
Receives a command from the control unit 8B, acquires a landscape image every second, and passes it to the control unit 9B (step 22). Since the image size is fixed for each landscape image device, the control unit 9B holds the image size information. The control unit 9B holds the collected information as a landscape image file.

FIG. 4 shows the file format of the data structure of the landscape image file. The landscape image file has header information and image data. The header information includes position information, camera angle information, focal length, time information, image size, type, and size of an image file. As location information,
East longitude, north latitude, and elevation data (for example, 137 degrees east 5 degrees
5 minutes 10 seconds, latitude 34 degrees 34 minutes 30 seconds north, altitude 101m3
3 cm). The camera angle includes horizontal angle data and elevation angle data (for example, horizontal angle 254 degrees clockwise, elevation angle 15 degrees, etc.). The focal length data is the focal length (for example, 28 mm) of the camera lens at the time of capturing an image. The time information includes the time at the time of shooting (for example, 15:06:17, January 31, 1997, Japan time). As the image size of the image file, the vertical and horizontal pixel sizes (for example, 64
0 × 480). File type (TIF
E format, 8-bit color, etc.). It also has the number of bytes of the file (307.2 KB, etc.). The image data itself has, for example, a binary format.

After storing the landscape image file, the control unit 9B instructs the image processing unit 5 to extract the outline from the landscape image and divide the landscape image into a plurality of regions.
The image processing unit 4 performs a differentiation process on the basis of the density difference in the landscape image to extract a contour line (step 2).
5) The region is divided by performing labeling with the outline as a boundary (step 26). Note that the technical term of labeling used here is a technical term used in image area division, and is different from landscape labeling, which is the name of the present invention. As a procedure, first, the image is converted into a monochrome grayscale image. Since the contour is a portion where the brightness changes suddenly, the contour is extracted by performing a differentiation process to find a portion where the differential value is larger than a threshold value. At this time, the line width of the outline is one pixel, and the outlines are connected. For this purpose, a thinning process is performed to obtain a connected line having a line width of one pixel. Where the differentiation process,
It is sufficient to use a conventional method for the thinning processing.

The obtained contour is regarded as the contour of the area, and an operation of numbering the area constituted by the contour is performed.
The largest number among the numbers is the number of areas, and the number of pixels in the area indicates the area of the area. FIG. 9 shows an example in which a landscape image is divided into a plurality of partial areas. Note that a measure of similarity (closeness) between regions may be introduced, and a clustering process may be performed to combine a plurality of regions having similar properties into one region. Any existing clustering method may be used.

When the control unit 9B completes the landscape image area division processing, the control unit 9B accesses the map DB to the map information management unit 5 and passes the header information of the landscape image file to perform the processing for calculating the view space. Is issued (step 27).
An example of the map information management unit 5 is a map database program. The map information management unit 5 manages three-dimensional map data. Although two-dimensional map data may be used, in this case, since there is no height information, the accuracy of the position at which the labeling is applied to the actual scenery is inferior. When the two-dimensional map data is used as the basis, the processing is performed by supplementing the height information. For example, in the case of two-dimensional data of a house, if there is floor information indicating the number of floors of the house, the number of floors is multiplied by a certain number to estimate the height of the house, and the two-dimensional data is obtained. 3 based on the height information
Create dimensional data. Even when there is no floor information, the height information can be estimated by assigning a certain number of heights according to the area of the house figure, and similarly, three-dimensional data is created based on the estimated height information. . In this way, three-dimensional data is created and the process proceeds.

FIG. 5 shows an example of three-dimensional map data. FIG.
(1) shows a map information space expressed in two dimensions, and FIG.
(2) shows a map information space expressed in three dimensions. This 3
With respect to the three-dimensional map information space, the map information management unit 5 receives a command from the control unit 9B and calculates the visual field space based on the header information of the landscape image file (step 28). FIG. 6 shows a calculation example of the visual field space. First, the XY axes stretch in the horizontal direction,
The Z axis extends in the vertical direction. The position of the viewpoint E is set in the three-dimensional map information space from the position information in the header information of the landscape image file. For example, 137 degrees east longitude 55
Minute 19 seconds, latitude 34 degrees 34 minutes 30 seconds north, altitude 101m33
If it is cm, the corresponding coordinates in the map mesh number corresponding to the cm are set. Similarly, the camera angle direction is set based on the horizontal angle and the elevation angle in the camera angle information in the header information. The focal point F is set at a point advanced from the viewpoint E by the focal length on a straight line representing the camera angle direction. The line-of-sight direction vector is a unit vector having a length of 1 from the viewpoint E on the straight line. In the image size of the landscape image file, the width x on the X-axis of the camera screen is set from the horizontal size, and the width y on the Y-axis is set from the vertical size. The horizontal x vertical y plane is set so as to be perpendicular to the camera angle direction with respect to the line of sight vector and to include the focal point F. Straight lines connecting the four corner points of the camera screen are obtained from the coordinates of the viewpoint E, and a three-dimensional space formed by four half lines extending from the viewpoint E is defined as a visual field space. FIG. 7 shows an example of the visual field space in the three-dimensional map space. The three-dimensional map space is viewed from the XZ plane. FIG. 8 is a cross-sectional view of the space belonging to the viewing space in the XY plane in FIG. In the example of FIG. 7, buildings and mountains in the viewing space are included.

Further, the map information management unit 5 determines a structure existing in the determined visual field space. For each structure, it is calculated whether or not each vertex constituting the solid representing the structure exists in the internal region of the viewing space. Usually, a two-dimensional map space is divided by a two-dimensional mesh of a fixed size. As a method of cutting a mesh in the three-dimensional map space, a mesh is cut at regular intervals in the height direction in addition to the two-dimensional mesh in the vertical and horizontal directions. The space is divided by a rectangular parallelepiped unit space. First, the presence or absence of an overlapping portion between the rectangular parallelepiped unit space and the visual field space is checked, and the number of the three-dimensional unit map space having the overlapping portion is obtained. The number of the three-dimensional unit map space here is the same as a so-called mesh number. For structures in the overlapping 3D unit map space,
Investigate whether there is any overlap with the visual field space. A straight line connecting the coordinates of the vertices constituting the structure and the coordinates of the viewpoint is obtained. If the straight line has an intersection with the camera screen of FIG. 8, it is within the visual field space. If at least one vertex of the plurality of vertices constituting the structure satisfies this condition, the structure has an overlapping portion with the viewing space.

When a structure is included in the view space or a part of the structure, the process enters a process of three-dimensionally projecting each structure onto the projection surface using the camera screen as a projection surface (step 29). ). Here, as shown in FIG. 8, after the point P is re-expressed in the coordinate system based on the viewpoint E based on the following equation (1), the point P is projected on the camera screen to obtain the intersection Q. .

[0032]

(Equation 1) Here, point P = (x, y, z): coordinates of the vertices constituting the structure point E = (ex, ey, ez): coordinates of the viewpoint vector L = (lx, ly, lz): gaze direction vector (Unit vector) Point P ′ = (x ′, y ′, z ′): coordinates in a coordinate system based on the viewpoint E of the point P r = (lx ² + ly ² ) ^1/2 intersection Q = (X, Y): The projection point t of the point P on the camera screen is the focal length. In the three-dimensional projection conversion, first, for each structure, the surface on which the vertex extends is obtained. For example, in the case of a structure represented by a rectangular parallelepiped, six surfaces are obtained. When each plane is projected onto the camera screen, the distance between the viewpoint and the corresponding point on the plane is calculated for each pixel on the camera screen included in the projection area, and stored as a depth value (Z value) in the memory. Store. For each surface of each structure, the depth value (Z
Value) and store it in memory. Note that z ′ in (Equation 1) represents a depth value (Z value) from the viewpoint.

The structures three-dimensionally transformed on the camera screen include structures that can be seen from the viewpoint and structures that cannot be seen. Among them, it is necessary to find only the structure that can be seen from the viewpoint, and to find the surface on the opposite side from the viewpoint or the surface that is blocked by other structures. Therefore, hidden surface processing is performed (step 3
0). There are various methods for processing the hidden surface.
Use the buffer method. Other scan line methods and ray tracing methods may be used.

A pixel on the camera screen is arbitrarily determined, and a plane having the smallest depth value for the pixel is determined. As described above, when processing is sequentially performed on each surface of each structure, the surface closest to the viewpoint is left for each pixel on the camera screen. The plane closest to the viewpoint is determined for each pixel on the camera screen, and pixels on the camera screen that share the plane closest to the viewpoint generally form an area.
There can be a plurality of regions consisting of pixels with the common surface being the closest surface. The region obtained in this way is a region obtained by performing a three-dimensional projection conversion of the partial region of the structure seen from the viewpoint. Surfaces on the other side of the view and those obstructed by other structures have been erased.

The area thus formed forms a CG image area (step 31).

With respect to the coordinates of the vertices of the two-dimensional figure constituting the CG image area, three-dimensional coordinates before projection transformation are obtained, and the correspondence between the two is stored in the memory as link information. It is used to determine which structure the two-dimensional area is a projection of based on the link information.

Based on the line data remaining after erasing hidden lines,
The CG image is divided into regions. Since the three-dimensional map DB is used, the name of the structure serving as the basis of each area can be associated with each area. The divided areas of the CG image are numbered sequentially. FIG. 10 shows an example in which a CG image is divided into a plurality of partial regions.

When the CG image region division processing is completed, the control unit 9B instructs the label information creation unit 6B to associate the CG image division region with the landscape image division region. The label information creation unit 6B associates the divided regions of the CG image with the divided regions of the landscape image by template matching (step 32, see FIG. 11).

Of the divided areas of the landscape image, the areas are associated with the divided areas of the CG image in ascending order of the number (for example, No. 1). For matching, any of the conventional matching methods may be used, but here, a simple template matching method is used. That is, two regions to be compared are superimposed, and when the ratio of overlapping portions is equal to or greater than a certain ratio determined as a threshold, the regions are associated as regions relating to the same structure. For example, the coordinate value of each pixel in the first divided region R1 of the landscape image is set to (A, B). The value of the pixel at coordinates (A, B) is 1 because it is inside the area. In the first divided area S1 of the CG image,
If the coordinates (A, B) are within the area S1, the pixel value is 1 and overlaps, but if the coordinates (A, B) are outside the area S1, the pixel value is 0 and does not overlap. In this way, the overlap coefficient K (A, B) at the coordinates (A, B) is determined to be 1 when they overlap and 0 when they do not overlap. The coordinates (A, B) are moved within the region R1, and the overlap coefficient K
(A, B) is obtained. Then, with respect to the number N1 of coordinates (A, B) moved in the region R1, the overlap coefficient K (A,
The number N2 of coordinates where B) was 1 is obtained, and when N1 / N2 is equal to or larger than the threshold value, it is determined that the divided region R1 of the landscape image and the divided region S1 of the CG image correspond to each other. This association is performed from the first to the last divided area of the landscape image. In addition, as the matching method, an evaluation function that has the same value even if there is a slight displacement in the XY directions may be used.

After associating the partial area of the landscape image with the partial area of the CG image, the label information creating section 6B extracts the structure of the associated partial area (step 3).
3) Further, processing to obtain information to be superimposed for each partial region of the landscape image and create label information together with the position to be superimposed (step 34) is entered. First, a partial area of the CG image corresponding to the partial area of the landscape image is extracted. The partial region of the extracted CG image is originally obtained by three-dimensionally projecting and converting a certain surface of the three-dimensional structure in the three-dimensional map space onto the camera screen. So 3
The surface of the three-dimensional structure that is the basis of the three-dimensional projection transformation is determined using the depth value (Z value) of the partial region of the CG image as a key. The link information created at the time of the three-dimensional projection conversion may be used as a key. Based on the surface of the original structure, the three-dimensional map DB is accessed to acquire the name or attribute information of the structure. Here, the attribute information means information accompanying the structure, and may be any information as long as the information is related to the structure. Then, the position coordinates where the name or attribute information is to be superimposed are determined for the partial area of the landscape image. How to decide may be decided any way. For example,
The center of gravity of the figure extending the partial area may be used. Label information is created from the name or attribute information of the structure and the assigned position coordinates. Table 1 shows an example of label information.

[0041]

[Table 1] After completing the creation of the label information, the label information creating unit 6B passes the label information to the control unit 9B.

Upon receiving the label information, the control section 9B
It instructs the label information output unit 8B to display the label information on the visual device, and passes the label information to the transmission information management unit 7. Here, the visual device includes a video display device such as a display or a head-mounted display.

The transmission information management unit 7 receives the generated label information via the control unit 9B, searches for the presence or absence of transmission information regarding the structure in the label information using the structure as a key, and if there is transmission information, The transmission information is transferred to the label information output unit 8B via the control unit 9B (step 35).

The label information output section 8B superimposes the name or attribute information of the structure in the label information at a position corresponding to the assigned position in the landscape image (step 36), and transmits the transmission information to the name or attribute information of the structure. (Step 3
7) Display the superimposed landscape image on the video display device (step 38). FIG. 12 shows an example of a landscape image on which label information is superimposed.

When the label information output section 8B outputs the label information, it notifies the control section 9B of the completion of the output. Control unit 9B
Upon receiving the output completion notification, if the landscape labeling process is to be performed continuously, the series of processing procedures described above are executed again.

FIG. 13 is a configuration diagram of an information transmission type landscape labeling system in which the landscape labeling device of FIG. 2 is applied to a communication system.

The landscape labeling system includes a landscape labeling terminal 40, a landscape labeling center 50, and a communication network 60.

The landscape labeling terminal 40 acquires an image acquiring section 41 for acquiring an image, a position information acquiring section 42 for acquiring a camera position when acquiring an image, and acquires a camera angle, a focal length, and an image size when acquiring an image. Camera attribute information acquisition unit 43
And an image processing unit 44 that divides the acquired image into a plurality of partial areas, and information about the area division of the image, the camera position, the camera angle, the focal length, and the image size, which are transmitted to the landscape labeling center 50 via the communication network 60. A communication control unit 45 for transmitting and receiving the label information and the transmission information from the landscape labeling center 50; and superimposing the name or attribute information of the structure in the label information at a corresponding position in the image, and visually recognizing the superimposed image. Displayed on the device and when outgoing information is received,
It comprises a label information output unit 47 for displaying the transmission information as attribute information of the structure, and a terminal control unit 46 for controlling the above-mentioned units.

The landscape labeling center 50 has a communication network 60.
A communication control unit 53 that receives information related to area division of the image, a camera position, a camera angle, a focal length, and an image size from the landscape labeling terminal 40 via the landscape labeling terminal 40, and transmits label information and transmission information to the landscape labeling terminal 40; A map information management unit 51 that manages map information, obtains a view space in a map information space based on the received camera position, camera angle, focal length, and image size, and acquires a structure existing in the view space. A label information creating unit 52 that associates the acquired structure with the partial region of the image by pattern matching, and creates label information including the name or attribute information of the associated structure and the assigned position. In addition, information transmitted from the user who wants to transmit information is managed for each structure, and the presence or absence of the transmitted information is checked for the structure in the label information. , If there calling information, the transmission information management unit 54 for outputting the transmission information, and a central control unit 55 for controlling the respective units.

The operation of this system is the same as the operation of the apparatus shown in FIG.

[0051]

As described above, according to the present invention, the geographical information on the computer and each part in the scenery image of the actual scene can be presented to the user in association with each other.
It is not necessary for a human to compare the map on the computer with the actual scenery and associate it with the human, and advertisements and guidance information can be sent to each terminal.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an information transmission type landscape labeling device according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram of an information transmission type landscape labeling device according to a second embodiment of the present invention.

FIG. 3 is a flowchart of processing of a landscape labeling device according to a second embodiment.

FIG. 4 is a diagram showing a data structure of a landscape image file.

FIG. 5 is a diagram showing an example of a two-dimensional map (FIG. 1 (1)) and its three-dimensional map (FIG. 2 (2)).

FIG. 6 is a diagram illustrating a calculation method of a visual field space.

FIG. 7 is a diagram showing an example of a visual field space in a three-dimensional map space.

FIG. 8 is a diagram showing an example of a projection view.

FIG. 9 is a diagram showing an example of area division of a landscape image.

FIG. 10 is a diagram showing an example of area division of a CG image.

FIG. 11 is an explanatory diagram of pattern etching of a partial region of a landscape image and a partial region of a CG image.

FIG. 12 is a diagram illustrating an example of superimposition of label information on a landscape image.

FIG. 13 is a configuration diagram of an information transmission type landscape labeling system of the present invention.

FIG. 14 is a configuration diagram of a navigation device disclosed in Japanese Patent Application Laid-Open No. 8-273000.

FIG. 15 is a diagram illustrating a display example of a moving image.

[Explanation of symbols]

 Reference Signs List 1 image acquisition unit 2 position information acquisition unit 3 camera attribute information acquisition unit 4 image processing unit 5 map information management unit 6A, 6B label information creation unit 7 transmission information management unit 8A, 8B label information output unit 9A, 9B control unit 21 to 21 38 Step 40 Scenery labeling terminal 41 Image acquisition unit 42 Position information acquisition unit 43 Camera attribute information acquisition unit 44 Image processing unit 45 Communication control unit 46 Terminal control unit 47 Label information output unit 50 Scenery labeling center 51 Map information management unit 52 Label information Creation unit 53 Communication control unit 54 Transmission information management unit 55 Center control unit 60 Communication network

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI G09G 5/36 510 G09G 5/36 510C (72) Inventor Akira Suzuki Nihon Shinjuku 3-9-1-2 Nishishinjuku, Shinjuku-ku, Tokyo Inside Telephone Co., Ltd. (72) Inventor Takeshi Ikeda 3-19-2 Nishi Shinjuku, Shinjuku-ku, Tokyo Japan Telegraph and Telephone Co., Ltd. (56) References JP-A-8-255239 (JP, A) JP-A-9 -33271 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00-17/50 G09B 23/00-29/14 G08G 1/00-9/02 G06F 17 / 30 G09G 1/00-5/42 JICST file (JOIS)

Claims (7)

(57) [Claims] An image acquisition unit for acquiring an image; a position information acquisition unit for acquiring a camera position at the time of image acquisition; and a camera attribute information acquisition for acquiring a camera angle, a focal length, and an image size when the image is acquired. Means, managing map information, obtaining a view space in a map information space based on the obtained camera position, camera angle, focal length, and image size, and obtaining map information that acquires structures existing in the view space. Management means; label information creating means for creating label information including the name of the structure or its attribute information and the assigned position; and managing transmission information from a user who desires information transmission corresponding to each structure. A transmission information management means for checking the presence or absence of transmission information for the structure in the label information, and outputting the transmission information if the transmission information exists, and an assignment position in the created label information. The name of the structure or its attribute information is superimposed at a position in the image corresponding to the information of the information, the superimposed image is displayed on a visual device, and when the transmission information is output, the transmission information is displayed in the structure. An information transmission type landscape labeling device, comprising: label information output means for displaying as attribute information of an object; and control means for controlling each of the above means. 2. An image acquisition unit for acquiring an image, a position information acquisition unit for acquiring a camera position at the time of image acquisition, a camera attribute information acquisition unit for acquiring a camera angle, a focal length, and an image size at the time of image acquisition. , Image processing means for dividing the acquired image into a plurality of partial areas, managing map information, and calculating a visual field space in the map information space based on the acquired camera position, camera angle, focal length, and image size, Map information management means for acquiring a structure existing in the visual space, and associating the acquired structure with the partial region of the image, and the name of the associated structure or its attribute information and Label information creating means for creating label information including an assigned position, and information transmitted from a user who desires information transmission is managed in correspondence with each structure, and the structure in the label information is Transmission information management means for examining the presence or absence of transmission information in relation to the transmission information, and outputting the transmission information if the transmission information exists; and the name of the structure or its attribute at the position in the image corresponding to the position information of the created label information. A label information output unit that superimposes information, displays the superimposed image on a visual device, and, when the transmission information is output, displays the transmission information as attribute information of the structure; An information transmission type landscape labeling device having a control means for performing the operation. 3. The label information creating means creates a CG image, which is a computer graphics image, based on the acquired structure, and a part in the CG image is subjected to pattern matching with respect to the partial area of the image. 3. The apparatus according to claim 2, wherein the regions are associated, a structure of the associated partial region is obtained, and label information including the name or attribute information of the structure and the assigned position is created. 4. The label information creating means performs a three-dimensional projection conversion of the acquired structure onto a camera screen, creates a CG image by erasing a structure that cannot be seen from a viewpoint, and creates a contour of a partial area in the CG image. Dividing the CG image into partial areas by lines,
The partial area of the image and the partial area of the CG image are associated with each other by pattern matching, and the structure that is the basis of the partial area of the CG image is determined for the partial area of the image. 3. The apparatus according to claim 2, wherein label information including name or attribute information and an assigned position is created. 5. A landscape labeling terminal and a landscape labeling center, wherein the landscape labeling terminal is an image acquisition unit for acquiring an image, a position information acquisition unit for acquiring a camera position at the time of image acquisition, and a camera at the time of image acquisition. Camera attribute information acquiring means for acquiring an angle, a focal length, and an image size, image processing means for dividing the acquired image into a plurality of partial areas, information on area division of the image, the camera position and the camera angle, Communication control means for transmitting the focal length and the image size to the landscape labeling center via a communication network, and receiving label information and transmission information from the landscape labeling center, and a name of a structure in the label information or The attribute information is superimposed at a corresponding position in the image, the superimposed image is displayed on a visual device, and the transmission information is output. And a label information output means for displaying the transmission information as attribute information of the structure, and a terminal control means for controlling each of the means, wherein the landscape labeling center performs the landscape labeling via the communication network. Communication control means for receiving information on the area division of the image, the camera position, the camera angle, the focal length, and the image size from a terminal, and transmitting the label information and the transmission information to the landscape labeling terminal, Map information management means for managing map information, obtaining a view space in the map information space based on the received camera position, camera angle, focal length, and image size, and acquiring a structure existing in the view space; Before associating the acquired structure with the partial region of the image and including the name or attribute information and the assigned position of the associated structure Label information creation means for creating label information, and information transmitted from a user who desires information transmission is managed in correspondence with each structure, and the presence or absence of transmission information on the structure in the label information is checked and transmitted. An information transmission type landscape labeling system having transmission information management means for outputting the transmission information if information exists, and a center control means for controlling each of the above means. 6. The label information creating means creates a CG image, which is a computer graphics image, based on the acquired structure, and a part in the CG image by pattern matching with respect to the partial area of the image. 6. The system according to claim 5, wherein the areas are associated, a structure of the associated partial area is determined, and label information including the name or attribute information of the structure and the assigned position is created. 7. The label information creating means performs a three-dimensional projection conversion of the acquired structure on a camera screen, creates a CG image by erasing a structure that cannot be seen from a viewpoint, and creates a contour of a partial region in the CG image. Dividing the CG image into partial areas by lines,
The partial area of the image and the partial area of the CG image are associated with each other by pattern matching, and the structure that is the basis of the partial area of the CG image is determined for the partial area of the image. The system according to claim 5, wherein label information including name or attribute information and an assigned position is created.