WO2008072429A1

WO2008072429A1 - System for displaying image data associated with map information

Info

Publication number: WO2008072429A1
Application number: PCT/JP2007/071583
Authority: WO
Inventors: Kazuo Oda; Satoshi Nakayama; Hideyuki Kanehara; Nobuyuki Mizutani; Tomoyasu Ota; Hideto Satoh; Jun Yagome
Original assignee: Locationview Co.
Priority date: 2006-12-12
Filing date: 2007-11-06
Publication date: 2008-06-19

Abstract

A system for displaying image data associated with map information comprises a storage for storing map data and clipped image data, a map displaying means for displaying a map generated from the map data, a photographing location displaying means for displaying a photographing location on the map by using the clipped image data, a photographed image displaying means for displaying an image of the clipped image data, a route search processing means for searching for a route from a specified starting point to a specified destination point on the basis of the map data, and an image search means for searching for a specified image on the basis of the clipped image data.

Description

Specification

Image data display system with geographic information

Technical field

[0001] The present invention collects image data for grasping the geographical environment by, for example, shooting by road driving or flying over the air, and depending on the required geographical range, image resolution, and amount of image data. The present invention relates to an image data display system associated with geographic information that is extracted, processed, managed spatially and temporally in association with geographic coordinates, and displayed in conjunction with a map. Background art

[0002] With the advancement of information technology and the development of large-capacity communication networks, large-capacity databases such as photographs and moving images can be easily handled on display media such as the Internet, car navigation systems, and mobile phones. .

[0003] With this, there is a need to accurately convey the actual landscape to the users without deformation by using images as geographic attributes in geographic information systems (GIS) and map search sites on the Internet. Many have come.

[0004] For example, as disclosed in Japanese Patent No. 3099103, a technique has been developed for taking a cityscape image taken from a specific road and using it for road and river maintenance management.

[0005] With this technology, images obtained from features on road contours are managed on a one-dimensional feature such as a road center line in association with the distance from the start point specified on the feature. Is done.

[0006] In addition, as disclosed in Japanese Patent Publication No. 06-83937, artificially created CG images that are not actually captured images are partially related to roads with a graph structure. There is also a technology to manage it. Recently, as disclosed in Japanese Patent Publication 2005-3752, a technique for displaying an actual image instead of a CG image has been developed.

[0007] However, according to the conventional technology, only one type of image can be managed in association with one piece of map information. Therefore, it is impossible to select and display from a plurality of images having different seasons and time zones. [0008] That is, it is impossible to simultaneously manage images taken in different seasons or time zones, or to select and display images in the desired season or time zone.

[0009] Therefore, it is not possible to display the optimal image that satisfies the condition to be viewed together with the map by specifying the condition and searching using data configured by associating a plurality of images with one map information. There is a problem!

[0010] With respect to image search, a high-speed search method is required because a central search of a large number of image files stored in a large-scale database is performed. In order to be able to use it in combination with existing geographic information, for example, when an arbitrary point (for example, a building) is selected, a peripheral image having coordinates near the coordinates of the point as an attribute is searched, and the image It is necessary to be able to calculate the azimuth angle based on the direction of travel and to display the target building on the front.

[0011] Furthermore, as an advanced image search method, it is possible to perform a semantic search of an image by simply searching for a place, that is, a search based on the contents of an object being captured.

[0012] For example, it is necessary to search for an image corresponding to geographical contents such as an image near a park and an image near a large store.

[0013] The present invention has been made to solve the above problems, and associates image data obtained by photographing the surrounding environment of a route network having a graph structure with geographic information including a surrounding map. The purpose is to display the structured data.

[0014] Specifically, the collected image and the road map are displayed in an interlocked manner so that the target road can be easily selected on the intersection collected image and the direction of the collected image can be displayed in a linked manner on the road map. Get the way.

Disclosure of the invention

[0015] The image data display system with geographic information according to the present invention uses an omnidirectional camera that shoots every fixed time mounted on a mobile body, and a hybrid type position sensor having GPS, and includes a plurality of route networks. Replays all-around image with time information obtained by moving the link and displays the image of the rectangular area on the screen, while displaying the path selection polygon for selecting one of the links An image data display system associated with geographic information, which stores, for each link in a route network, attribute information including position information and time information from a position sensor and position information for each surrounding image in each link and each surrounding image. Memory of The first link includes the shooting plan information including the link ID when the link was shot and the first actual shooting information including the first actual shooting date when shooting was actually performed based on the shooting plan information. As the related information, the second ID that is different from the first actual shooting date when the actual shooting was performed based on the same link ID, shooting plan information, and shooting plan information as the link ID included in the first link related information. Link information and node including second storage means storing second actual shooting information having shooting date as second link related information, link ID, link vector information, and node IDs at both ends of the link A route network with node information including a link ID that connects to the third storage means that stores the background map, and a map composed of the route network and the background map is displayed in a different area from the rectangular image, and the link is made. Choice The attribute information having the link ID of the selected link and the link ID of the selected link is read, and a plurality of all surrounding images in the selected link related to the attribute information are sequentially read from the first storage means in ascending or reverse order. When the all-around image is read from the all-around image reading means to be read and the first storage means, the attribute information related to the all-around image is read, and the position information of the read attribute information is stored in the node of the route network. When it is located, the reading of the all-around image is stopped, and the means for obtaining all the directions of the link to the node from the route network, the read all-around image, and the obtained node are obtained. The route selection polygon that indicates the direction of all links is displayed when one of the route selection polygon and the means for interlocking display of the route selection polygon on the rectangular area of the screen is selected. The link ID corresponding to the position of the route selection polygon is read from the route network, and the read link ID has the read link ID, actual shooting information, and shooting plan information indicating whether or not all surrounding images are associated with the link ID! Judgment is made based on whether the first link related information or the second link related information exists. If it exists, the suspension is canceled and the read link ID is set as the selected link ID to the all-around image reading means. And a means for outputting.

The present invention relates to map information such as roads and images, for example, property information such as condominiums and stores for real estate transactions, route environment indications from town sights and stations to target real estate properties and stores, Visualize the real situation in the local area by using images as geographical attributes to represent places, such as advertising activities in shopping streets, location and surrounding situations in emergency calls such as firefighting and police, and local situation in road river management. Can tell This is an image data display system associated with geographic information.

[0017] In the present invention, the surrounding environment of a road network having a graph structure (as shown in Fig. 17, the node is an intersection and the network information is a continuous line segment indicating a road center line between the intersections). Using the data configured by associating multiple image data obtained by shooting and geographic information including the surrounding map, it is possible to display the image corresponding to the corresponding point while interlocking with the map. .

[0018] In addition, if a target town is obtained in a wide range and data is provided to provide a service, the user can freely select an arbitrary place to be examined. Also, for example, if you prepare images taken by seasons or at different times, it is an image data display system with geographic information that can provide images of town changes and environmental changes.

[0019] As a means for providing information to an unspecified number of users in response to these needs, web distribution can be used for many purposes.

[0020] In order to display and search the video database in association with the map, the point where the video was shot (shooting point), the direction taken, and the time taken (season) that were not managed by conventional 2D geographic coordinates By managing multidimensional spatio-temporal information as metadata, it becomes possible to search and provide necessary information to users.

[0021] If such video information is collected in advance and a large amount of data can be provided quickly according to the range and position specified by the user, a business model can be built! . In addition, even if a new image is taken, if it is managed in the same way as already collected data, it can be easily reused as required.

[0022] Furthermore, if it is possible to use images taken at a plurality of times in which only one shooting is performed on the same part, an image in a state desired by the user, for example, an image designating a season or time, etc. Provision becomes possible. For example, if images with different shooting years are available, it will be possible to investigate changes over time in a specific area. In addition, “I want to see the image of the cherry blossom season” and “I can see the image at the time of autumn leaves”, it is possible to see the state, the state of the cityscape and the tourist road as seen by the user.

[0023] In the case of wide roads, the power of the buildings on the side close to the driving route is often reflected. The buildings on the opposite lane may not be reflected well. In other words, round trip to better represent the image of the railway Need to shoot. Even in the case of reciprocal shooting, it may be managed in the same manner as in the case of multi-period shooting as well as the traveling direction.

[0024] In addition, it is possible to search an image of a specific feature from a large amount of image data and display its position using attached information (such as shooting direction) attached to the image. This is a system.

[0025] Further, according to the present invention, an icon for selecting a road is displayed at an intersection, a corresponding link is allocated by selecting this icon, and an image in this link is displayed and displayed on a map. The position of the captured image and the shooting range and direction are known. For this reason, the operator can easily determine where the image is. In addition, since the direction of travel can be selected from the image, the operability is excellent.

Brief Description of Drawings

[0026] FIG. 1 is a configuration diagram of an image data management system with map information association and a display system.

FIG. 2 is a configuration diagram of an image capturing subsystem.

FIG. 3 is a block diagram of an image data editing subsystem.

[Fig. 4] A map displayed by the map data display means.

FIG. 5 is an image displayed on the road surface of image data display means.

FIG. 6 is a diagram showing a method of image thinning by the image thinning means.

FIG. 7 is a configuration diagram of an image data management subsystem.

FIG. 8 is a block diagram of the photographing plan means.

FIG. 9 is a configuration diagram of photographing information capturing means.

FIG. 10 is a configuration diagram of management means.

FIG. 11 is a block diagram of data extraction means.

FIG. 12 is a structural diagram of a graph structure database.

FIG. 13 is a structural diagram of a camera information database.

FIG. 14 is a structural diagram of the plan 'actual imaging information database'.

FIG. 15 is a structural diagram of a link related information database.

FIG. 16 is a structural diagram of a physical image database.

FIG. 17 is a diagram showing a feature having a graph structure. FIG. 18 is a flowchart illustrating a procedure in the image data management system with a geographic information association according to the embodiment.

FIG. 19] is a flowchart for explaining a procedure in the image data management system with a geographic information association according to the embodiment.

FIG. 20 is a flowchart for explaining a procedure in the image data management system with geographic information according to the embodiment.

FIG. 21] is a flowchart illustrating a procedure in the image data management system with a geographic information association according to the embodiment.

22] It is a configuration diagram of a map information association image data display system.

[23] This is a display screen example of the map information association image data display system.

[FIG. 24] An example of annotation overlay display on the display screen of the map information association image data display system.

[25] This is a display screen example of the map information association image data display system.

[Fig.26] Flowchart of note data search of map information related image data display system.

[27] FIG. 27 is a detailed configuration diagram of the image data display system 100 with geographic information.

FIG. 28 is an explanatory diagram for simultaneously displaying a photographing position and a corresponding image.

FIG. 29] It is an explanatory diagram showing a more specific configuration of cut-out image data.

FIG. 30 is an explanatory diagram for explaining a more specific configuration of a physical image database.

FIG. 31] This is an explanatory diagram of thinning out of the present embodiment, display of shooting positions, and registration of cut-out data.

FIG. 32 is an explanatory diagram for explaining a traveling direction mark display means.

33] It is an explanatory diagram explaining the display of the note data on the image.

FIG. 34 is a schematic configuration diagram of a geographic information-related image data management system according to the second embodiment.

FIG. 35] is an explanatory diagram of image grouping for links.

FIG. 36 is a block diagram for displaying a traveling direction selection polygon.

[Gakuen 37] A comparison of images corresponding to the same link taken at different times. FIG. 38 is a comparison diagram of images corresponding to the same link photographed in different directions.

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic.

As shown in FIGS. 1 and 22, the image data display system 100 with a geographic information association according to Embodiment 1 of the present invention includes a geographic display means 101, a photographing position display means 102, a photographed image display means 103, and a route search interlocking means. 104, an image search means 105, and a storage device 106 that stores map data 108 and cut-out image data f4.

[0029] The geographic display means 101 reads the map data 108 stored in the storage device 106, and displays the map while scrolling, switching scales, switching display items, switching display colors, and the like.

The imaging position display means 102 stores a storage device on the map displayed by the geographic display means 101.

Cut-out image data stored in 106 f4 (image file (collected image data), image file additional information (image acquisition coordinates, image acquisition orientation, image acquisition link ID, order in acquisition link, image acquisition camera information (Shooting height etc.)) From the graph structure of the link (including the list of IDs of links connected to each node), the shooting position (image acquisition coordinates on the map) is drawn as shown in Fig. 23. The user designates this drawing position with a mouse click or the like, and displays a photographed image at a desired photographing point.

The captured image display means 103 includes an image display unit and an image continuous display unit.

[0032] As shown in FIG. 23, when one of the shooting positions of the shooting position display means 102 displayed on the geographic display means 101 is designated by a mouse operation or the like, the image display unit displays the image closest to the shooting position. Search using the image acquisition coordinates of the image file attachment information of the cut-out image data, and read and display the closest image file.

[0033] In addition to the simple display that displays the frame image at the specified shooting position as it is, as shown in Fig. 25, the stored image is an all-round image (360 ° circumference is taken and equirectangular projection ( If the image was taken with a vertical projection (a projection in which the horizontal projection is linear to the horizontal orientation) An image taken from the shadow center with an arbitrary camera parameter (camera direction 'focal length (angle of view)) is synthesized and displayed by means of CG processing or the like. This kind of display method is used to specify camera parameters.

Specifically, the image is converted into a rectangular area and displayed using texture mapping and recording means generally used in CG (computer graphics).

[0035] Further, when the map data 108 includes annotation data (data consisting of a combination of the feature name on the map 'feature type and position coordinates on the map), as shown in FIG. The orientation of the object to which the annotation data is added as seen from the image acquisition coordinates is calculated from the image acquisition coordinates and image acquisition orientation included in the cut-out image data f4 and the annotation display position included in the map data 108) and displayed. Draw annotation data at the corresponding orientation in the image. It is preferable that note data, an image, and image acquisition coordinates are stored in advance in association with each other.

[0036] The continuous image display unit displays images having the same link ID sequentially in ascending or reverse order according to the shooting order in the acquisition link, so that the power is continuously moved. And display it as a video. The continuous display can be controlled with the movement direction designation button and the pause button. The movement direction designation button is displayed as a symbol corresponding to the traveling direction of the image data associated with the same route. The moving speed can be specified in the moving speed specification box.

[0037] When the first or last order in the acquisition link is reached in the continuous image display, the continuous display is discontinued and the route is selected. In route selection, the node ID of the last displayed side is checked from the attached information (also called attribute information) of the image file. Next, the graph structure included in the cut-out image data is examined, and a link connected to the node is searched. Furthermore, by checking the image acquisition link ID included in the cut-out image data, if there is image data (collected image data) that captures the link, it is displayed on the image and displayed on each path. Draw a click button, etc., and specify which route image to display next.

[0038] The route search interlocking means 104 uses the route network included in the map data 108 to search for a route from the specified start point coordinates to the end point coordinates, and the geographical display means 101 The route is displayed on the displayed map, and for the portion where the cutout image data f4 corresponding to the route exists, the shooting position of the image acquisition coordinates of the image is also drawn on the map displayed by the geographic display means 101.

It should be noted that the route search interlocking means 104 draws only the shooting positions of all image acquisition coordinates on the route during the route search, and does not draw the rest. At this time, if a shooting position on the route is selected, the other shooting positions are drawn in a different color.

[0040] The above-described route network includes, for example, a travel locus pattern, a road shape, a width, and the like.

[0041] A network having a graph structure includes a road, a railroad, a seaway, a pipeline, an airway, a tunnel, and the like, and may be collectively referred to as a route network.

[0042] Further, images from the start point to the end point are sequentially drawn.

The image search means 105 searches for an image of the object being photographed using the note data including the facility name and the like included in the map data 108. The note data includes the note string to be drawn on the map, the note display position, and the note type! /.

The image search is performed according to the procedure shown in the flowchart of FIG.

[0045] Step S200: Verification of note data and image

Each note data is searched for an image that has been photographed and judged to be the best! Typically, an image file having a shooting position coordinate closest to the note display position of the note data is assigned, and the image file attached information of the cut-out image data having this image file is searched and selected. . However, if there is no image that meets the defined criteria (eg distance threshold, value), there is no image corresponding to the note data.

[0046] Step S201: Keyword input

Enter a keyword including the name and type of the object you want to search. The types of features include public facilities, parks, and “intersection names”. In addition, the keyword specifies all or part of the feature name (Shinjuku Station is examined! /, In this case, “Shinjuku”, etc.).

[0047] Step S202: Note data search

Search for note data corresponding to the above keywords. At this time, in step S200 Only notes with corresponding images selected are considered. Matching with keywords is performed on note strings or note types.

[0048] Step S203: Note candidate selection

If multiple notes match keywords, they are displayed on the screen and selected by the user.

[0049] Step S204: Image display

An image corresponding to the note selected by the user is cut out from the image data and displayed on the image display unit of the computer screen.

[0050] Next, the map data 108 and the cut-out image data f4 stored in the storage device 106 will be described.

[0051] The map data 108 includes network data fO such as roads, a background map, note data given to objects existing on the map, and map symbol data.

[0052] The map data 108 may be general data created for use in a navigation system or the like, rather than data created exclusively.

On the other hand, the cut-out image data f4 includes an image file, image file attached information, and graph structure information. The image file attachment information includes the image acquisition coordinates, image acquisition orientation, image acquisition link ID, shooting start node ID, shooting end node ID, and shooting order counted from the shooting start side for each image file. And graph structure information associated with the acquired image and image acquisition camera information (such as shooting height).

[0054] The graph structure of the link indicates the connection relation of the link from which the image is acquired. For each node included in the data, a list ID of the node ID and the link ID of the link connected to the node is displayed.

Further, the image acquisition direction is the direction of the central axis of the camera at the time of shooting.

[0056] When the image is an omnidirectional image, the center of the omnidirectional image viewed from the shooting point is the corresponding orientation.

[0057] The image acquisition link ID corresponds to the ID of the center of the road where the image was taken when the route network is used as a graph structure.

[0058] Further, the node is a coordinate point for specifying, for example, a road intersection or a dead end. The link is a vector connecting each node, and includes data such as the link start point coordinates, end point coordinates, link direction, and link road width.

The cut-out image data f4 is data created by an image data management system with geographic information association 10 described later.

[0060] The image data display system 100 associated with geographic information may be a program operating alone on a computer. In addition, the cut-out image data f4 is a data server, map data 1

08 may be realized as a system on the Internet with a client or data server.

[0061] (Image data management system with geographic coordinates 10)

Next, a description will be given of the image data management system 10 with geographic coordinates that creates the cut-out image data f4 used in the image data display system 100 with association with geographic information.

Yes

[0062] As shown in FIG. 1, the image data management system 10 with geographic coordinates is divided into an image capturing subsystem 20, an image data editing subsystem 30, an image data management subsystem 40, and an image data management subsystem 40. Consists of connected storage devices 50.

[0063] First, the image capturing subsystem 20 is connected to the image sensor 22, and an image data collection unit 24 that collects an image of the surrounding environment along the link as collected image data dl together with the photographing time, and a hybrid type A position sensor 23 is connected, and a moving means equipped with a position data collecting means 25 for collecting the photographing position coordinates of the image together with the coordinate measurement time as the collected image attribute information d2 is included. The collected image data dl from the above-described image capturing subsystem is a group of images captured at regular intervals within the link.

[0064] Also included are a hybrid position sensor 22, a speed sensor 'acceleration sensor', an angular acceleration sensor, and the like.

Here, for example, a link refers to a road between two points (from one intersection to the next intersection) having the graph structure shown in FIG. For convenience, one point is marked as the starting point, and the other point is marked as the ending point, but it does not mean the starting point force, the end point and! /, Or the direction of the link.

[0066] Further, the moving means refers to an aircraft for flying along a link and shooting from above, or a vehicle 21 traveling on a link road. [0067] Subsequently, the image data editing subsystem 30 includes a map data display means 31 for displaying links, an image data display means 32 for displaying image data collected by the image data collection means 24, a photographing time and Link start and end point passage time registration means 33 that records the start time and end point passage time of the link from the coordinate measurement time (GPS time), and image thinning that thins out data from the multiple collected image data dl in the link at regular intervals. Means 34 and actual photographing management information registration means 35 are included.

[0068] Further, the image data management subsystem 40 includes an imaging plan means 41 for managing an imaging plan by the image imaging subsystem 20, the collected image data dl edited by the image data editing subsystem 30, and the collected image attribute information d2. Shooting information capturing means 42 for capturing information, data extracting means 44 for extracting information that meets the specified conditions from information recorded in the storage device 50, registration of information in the storage device 50, and storage device 50 Management means 43 for deleting information from the storage device 50 and printing information in the storage device 50.

[0069] The storage device 50 includes a graph structure database 51 that records link information, and an image sensor.

The camera information database 52 that records 22 information, the plan / actual shooting information database 53 that records the shooting plan information, and the collected image data dl and the collected image attribute information d2 captured by the capturing information capturing means 42, respectively. It includes a collected image database 55 and a collected image attribute information database 56 to be recorded, and a link related information database 54 for recording management information for associating information of each database for each link as shown in FIG.

[0070] Data exchange between the image capturing subsystem 20, the image data editing subsystem 30, and the image data management subsystem 40 is performed via a portable storage medium such as a disk. Each subsystem can be provided with data communication means, and can be performed by wired or wireless data communication.

[0071] (Image capture subsystem 20)

Next, each unit of the image capturing subsystem 20 will be described in detail. It should be noted that here, the vehicle 21 is taken as an example of the moving means and the force S and the vehicle 21 are not limited.

As shown in FIG. 12, the image capturing subsystem 20 is a subsystem intended to capture a peripheral image along a link used in the embodiment of the present invention. [0073] First, the image data collection means 24 captures an image while traveling on a road as a link using one or a plurality of synchronized cameras attached to the vehicle 21 as the image sensor 22. At this time, in order to synchronize with the position information described below, the time transmitted from the global radio positioning system (GPS) detected by the hybrid position sensor 23 is displayed in each time series. It is added to the frame and recorded in the temporary memory 26 as the collected image data dl.

Note that the time IJ is given to each image frame in units of 1/1000 seconds using the detected GPS time and the clock of the computer on which the image capturing subsystem 20 is mounted.

[0075] Next, the position data collecting means 25 uses a hybrid position sensor 23 combined with GPS and other sensors (gyro, vehicle speed sensors) attached to the same vehicle 21 to obtain the position obtained from the GPS. Information (coordinates), vehicle speed information obtained from the vehicle speed sensor, acceleration information and direction information obtained from the gyro are collected. Further, the GPS time is assigned to these pieces of information and recorded in the temporary memory 26 as the collected image attribute information d2.

[0076] When the vehicle speed obtained from the hybrid position sensor 23 is 0, the position data collecting unit 25 is provided with a process for instructing the image data collecting unit 24 not to collect image data. You can also stop shooting while stopped. As a result, when the vehicle is stopped due to traffic light or traffic congestion, it is possible to temporarily stop recording image data and save the data collection capacity.

[0077] The collected image data dl and the collected image attribute information d2 are assigned time information using a common GPS time as shown in FIG. The force S is used to associate the position with the image using as a medium.

In addition, the camera direction can be obtained from the relative direction of the camera relative to the vehicle 21 and the direction of the vehicle 21. The relative orientation is measured in advance. Normally, the image data collection cycle is shorter, so the position, speed, and direction in each image frame can be obtained by subtracting the position, speed, and direction at the previous and subsequent times.

[0079] The collected image data dl and the collected image attribute information d2 recorded in the temporary memory 26 are output to a portable storage medium such as a disc as a primary imaging information import file fl, and the next image data editing subsystem Input to 30. Note that the position data collection unit 25 is created by the shooting plan unit 41 of the image data management subsystem 40 described later, and is output to the portable storage medium as a shooting information export file f3 described later. The link shape and information contained in dO are read, and the position information obtained from the hybrid position sensor 23 is superimposed on the computer screen. As a result, the operator what force ^s I force was data collection of the link of the collection target, Ru.

[0081] (Image data editing subsystem 30)

Next, each unit of the image data editing subsystem 30 will be described in detail.

[0082] As shown in FIG. 3, the image data editing subsystem 30 uses the collected image data d2 collected by the image capturing subsystem 20 and output as the primary image capturing information import file fl as the implementation of the present invention. Edited for use in the form of the image data, created by the shooting plan means 41 of the image data management subsystem 40 described later, and using the shape information of the link information output as the shooting information export file f3, a graph such as a route network It is a subsystem designed to correlate and optimize map information with a structure.

First, as shown in FIG. 2, the map data display means 31 has, as input information, a primary imaging information import file fl output from the imaging system 20 having the same structure as a physical image database to be described later. Have. As shown in FIG. 4, the link shape is displayed from the time-series position information included in the collected image attribute information d2 constituting the primary shooting information import file fl and the map information included in the shooting information export file f3. The time series position information is displayed as a point sequence, and the link shape is displayed as a continuous line segment.

[0084] Next, the image data display means 32 displays the collected image data dl of the primary imaging information import file fl as shown in FIG. When the image is displayed, the image collection position is displayed as a symbol in the map displayed by the map data display means 31. In addition, the display of image data can be sequentially switched back and forth in time series.

[0085] Next, the link start / end point passage time registration means 33 displays the link selected by the mouse on the map shown in FIG. 4 displayed by the map data display means 31, or the link being selected that has been sequentially selected. Specify and register the passage time corresponding to the start and end points. Passing time is one Collection of the next shooting information import file fl Using the time series position information included in the image attribute information d2 to specify from the time of the time series position corresponding to the start point or end point, and the collection of the primary shooting information import file fl There is a method to specify from the image shooting time given to the image data dl. The transit time of the link start point and end point is the same as that for the shooting plan export file, along with information such as shooting date and time, camera information key ID used for shooting, and physical storage location (such as hard disk volume name) of the image to be stored. Saved in the actual shooting result information d3 in the format.

[0086] There are two methods for designating from the time series position: a manual designation method for designating the time series position corresponding to the start point or end point of the link with a mouse or the like, and an automatic designation method for automatically designating.

[0087] The automatic designation method is achieved by searching for a time-series position closest to the start point or end point of the link. You may use the map matching technology used in the navigation system of automobiles, that is, the technology that automatically corresponds to the route you are traveling in consideration of not only the position but also the direction of travel and speed. In this case, the closest point is not necessarily selected.

In the method of designating from the image capturing time, the collected image data dl corresponding to the start time of the link and the end time is displayed on the image display screen. By looking at the roads and intersections captured in the image, and if the operator determines that the position is shifted at the intersection, etc., the previous or next image is displayed in time series by operating the time series screen. The image capture time of the image at the intersection determined by the operator can be corrected and registered as the transit time of the link start point or end point.

[0089] If the start point passage time of the link registered by the link start / end point passage time registration means 33 is earlier than the end point passage time, the image is taken while traveling from the start point to the end point on the corresponding link. It is judged that it was done. Conversely, if the start point passage time is later than the end point passage time, it is determined that the image is taken while traveling from the end point toward the start point on the corresponding link.

[0090] The link start and end passage times registered by the link start / end point passage time registration means 33 are output to the secondary imaging information import file f2. [0091] Finally, the image thinning means 34 thins the collected image data dl of the primary shooting information import file fl in accordance with the moving distance or the like for the purpose of reducing the data capacity. As an example of the thinning method, as shown in FIG. 6, there is the following method. Starting from one frame, the distance from the starting frame to the i-th frame is calculated for each frame using the formula [Equation 1].

[0092] [Equation 1]

L [0] = 0. 0;

L [i] = L [i- l] + (v [i— l] + v [i]) / 2 * (t [i] — t [i— 1]);

Where v [i]: speed of i-th frame (meters / second)

t [i]: Time of i-th frame IJ (seconds)

Let the decimation interval be d, and let L [jn] be the distance closest to n times n (n is an integer). Frames other than jn are thinned out. If time information is also attached to the image frame after thinning, the image can be thinned without affecting other parts of the database.

[0093] For example, when shooting at 15 frames per second from a car traveling at 10 km / h, 54,000 images are acquired in a 10 km section. If this is thinned out every lm, it will be 10,000 sheets, and the data volume can be reduced to less than one-fifth.

[0094] In this way, the amount of data can be reduced by thinning out the number of image frames to such an extent that there is no human visual problem, and the amount of memory required for storage can be reduced. Can be made inexpensively. Furthermore, if the amount of data to be subjected to image display processing is small, the display processing speed can be improved and the burden on a display device such as a computer can be reduced.

[0095] If it is not necessary to reduce the amount of data due to the low cost of the memory or the improvement of the processing capacity of the computer, the image thinning means 34 is not particularly required.

[0096] The actual shooting management information registration means 35 inputs the attribute information (shooting area information, operator, etc.) at the time of actual shooting to be registered in the plan / actual shooting information database, which will be described later, and enters the actual shooting management information d4. input.

[0097] (Image Data Management Subsystem 40)

Next, each unit of the image data management subsystem 40 will be described in detail.

[0098] The image data management subsystem 40, as shown in FIG. Planning of equipment to be used-Necessary from management of progress to management of collected image data dl actually captured and edited, and to extract and provide collected image data dl that meets user requirements It is a subsystem for the purpose of batch management of various data groups.

First, as shown in FIG. 8, the shooting plan means 41 includes a screen display unit 411, a link selection unit 412, a camera information input unit 414, a shooting plan information input unit 413, and a shooting information export file creation unit 415. Composed.

The screen display unit 411 displays the node and link of the graph structure database 51 recorded in the storage device 50 on the computer screen.

[0101] The link selection unit 412 selects and determines a link for which shooting is planned from among the links displayed by the screen display unit 411. Link selection methods include rectangular area specification by mouse operation on the screen, selection by position such as polygon area specification, selection by attribute information included in the graph structure information such as road width and type, and position and attribute information. There is a composite selection

[0102] For example, when extracting a prefectural road of a certain municipality, the purpose is achieved if only the prefectural road is extracted in advance by selection based on attribute information, and then the municipality boundary is designated by polygon designation.

The camera information input unit 414 records the input camera information used for shooting in the storage device 50 as the camera information database 52. At this time, the camera information key ID specified automatically or arbitrarily is assigned so that individual camera information can be identified.

[0104] The shooting plan information input unit 413 displays the input shooting plan such as the shooting area (link group selected by the link selection unit 412), the scheduled shooting start date, the shooting operator, and the camera information key ID of the camera to be used. Is recorded in the storage device 50 as a plan / actual shooting information database 53. At this time, an automatic or arbitrarily designated shooting plan key ID is assigned so that individual shooting plans can be identified.

[0105] When the shooting plan ID is automatically generated, a combination of the creation date and the camera information key ID is used so that the ID does not overlap.

[0106] The shooting information export file creation unit 415 combines the link selected by the link selection unit 412 and the information input by the shooting plan information input unit 413, and displays them in the format shown in FIG. As link related information, it is recorded in the link related information database 54 in the storage device 50. At this stage, the link-related information and the link shape recorded in the graph structure database are output to the portable storage medium as a photographing information export file f3.

Next, as shown in FIG. 9, the shooting information capturing means 42 includes an actual shooting management information recording unit 421 and an image data recording unit 422.

[0108] The actual shooting management information recording unit 421 stores the actual shooting management information d3 such as the photographer at the time of shooting, which is input as the secondary shooting information import file f2 from the image data editing subsystem 30, as the actual shooting key. Assign ID and plan · Record in actual shooting information database 53.

[0109] The actual shooting ID should be a combination of the shooting date and the camera information key ID used, etc., so that IDs do not overlap.

[0110] Furthermore, the shooting information related to the shot link, that is, the shooting date and time, the camera information key ID used for shooting, the passage time of the link start point and end point, and the physical storage location of the image to be stored (such as the volume name of the hard disk) ) Etc., the actual shooting result information d 3 is stored in the link related information database 54 created and saved for each link key ID by the shooting information export file creation unit 415. Update to.

[0111] In addition, the image data recording unit 422 collects the collected image data dl and the collected image attribute information d2 input from the image data editing subsystem 30 as the secondary shooting information import file f2, and the collected image database 55 and the collected image attribute information d2, respectively. Record in the image attribute information database 56.

As shown in FIG. 10, the management unit 43 includes a network registration unit 431, a printing unit 432, and a maintenance unit 433.

[0113] The network registration unit 431 reads route network data created and distributed externally and registers it in the graph structure database 51.

The printing unit 432 prints the information of each database recorded in the storage device 50 using a printer.

[0115] The maintenance unit 433 performs periodic knock-up of each database recorded in the storage device 50, and stores unnecessary data (such as shooting plan information that is no longer shot). Delete it. In addition, by searching for the work progress code in the link related information database 54 reflecting the shooting plan information dO and the actual shooting management information d3, the power of shooting progress is totaled.

Finally, as shown in FIG. 11, the data extraction means 44 includes an extraction condition input unit 441, a graph structure information extraction unit 442, a link related information extraction unit 443, a collected image data extraction unit 444, and a collected image. An attribute information extraction unit 554 is included.

[0117] The extraction condition input unit 441 inputs conditions for extraction from the database. Conditions include positional conditions, link attribute conditions, and shooting attribute conditions. The positional conditions include a rectangular area and a polygon area obtained by operating the mouse on the screen or reading a file.

[0118] The link attribute condition includes designation of attribute information included in the graph structure information such as road width and type. The shooting attribute condition includes designation of a shooting date and a shooting time zone.

[0119] The graph structure information extraction unit 442 receives an extraction condition input unit from the graph structure database 51.

A link that matches the positional condition and the link attribute condition input in 441 is extracted.

Further, the extracted link IDs and the node IDs at both ends thereof are checked, the link IDs connected to each node are listed, and stored as the graph structure information of the link of the cut-out image data.

[0121] The link related information extraction unit 443 extracts the link corresponding to the ID of the link extracted above from the link related information database 54. Further, link information that matches the photographing attribute condition input by the extraction condition input unit 441 is extracted from the information.

The collected image data extraction unit 444 extracts information that meets the conditions input by the extraction condition input unit 441, from the collected image database 55.

In other words, the collected image data extraction unit 444 searches for an image photographed with the camera information key ID stored in each link related information extracted above from the collected image database. To do. Further, an image photographed between the viewpoint passage time and the end point passage time is extracted from the image, cut out as an image file, and stored as an image file in the image data.

[0124] Also, the image acquisition link ID, the node ID on the shooting start side, the node IDs on both sides of the shooting collection, and the shooting order are cut out and stored as image file attribute information in the image data. The collected image attribute information extraction unit 445 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image attribute information database 56.

That is, information such as image acquisition coordinates and image acquisition orientation corresponding to the image file extracted above is extracted and stored as image file attribute information in the cut-out image data.

[0127] Information cut out as described above is output as cut-out image data f4. This includes image files, image file attachment information, link related information, link graph structure information, and the like.

[0128] Note that the link can be cut out by the following method.

[0129] · Specifying the positional condition (cutting out by geographic mesh or administrative area): The ability to select the link to cut out by specifying the mouse on the computer screen interface or specifying the area with the mouse. In addition, the inside of the outer polygon such as the administrative area may be registered in advance, and the link in the outer polygon corresponding to the area designated by the user may be extracted.

[0130] For example, if the city, town, and village boundaries of Tokyo are created separately as polygons, saved as external files on the magnetic disk, and the polygon is read by specifying the file when necessary, the mouse Can be used as an alternative to polygon input.

[0131] 'Specify link attribute condition (road type' road width, etc.): Specify the type of link required for shooting and refer to the information stored in the graph structure database 51. Can be extracted.

Specification of shooting attribute condition: By referring to shooting date / time information included in the link related information database 54, it is possible to specify and extract the shooting time of the image.

On the other hand, the image corresponding to each link is extracted based on the information described in the link related information database 54. That is, from the collected image database 55 recorded in the storage location of the storage device 50 described in the record corresponding to the link key ID of the link to be extracted, the image is captured by the camera described in the camera information key ID, In addition, it is only necessary to extract the time that the image frame is between the start point passage time and the end point passage time.

[0134] The cut out image data can include the following. [0135] · Image file

• Image file attribute information

• Image acquisition coordinates (latitude and longitude of each frame)

• Image acquisition direction (image center direction)

• Image acquisition link ID (ID of the road centerline that captured the image)

• Node ID on the shooting start side

• Node ID at the end of shooting

• Shooting order

• Image acquisition camera information (shooting height etc.)

'Link graph structure (also called network information)

'For each node ID, a list of link IDs that connect to that node

(Each database in storage device 50)

Next, each database recorded in the storage device 50 will be described.

[0136] As shown in Fig. 1, the database used in the embodiment of the present invention is roughly divided into an image information database 50a for managing the relationship between feature information having a graph structure and captured images. The physical image data base 50b that stores the actually captured images and their associated information can be divided into two groups.

[0137] The image information database 50a includes a graph structure database 51 that stores the graph structure, a camera information database 52 that stores information attached to the shooting, such as the camera used for shooting, and the shooting plan and actual shooting times. There are four types: a plan-actual imaging information database 53 corresponding to each link storing information, and a link-related information database 54 that associates the above information and captured images for each link.

The graph structure database 51 stores feature information having a graph structure as shown in FIG. When a route network is used as geographic information, a road center line connecting the intersection as a node of the graph structure and the intersection as a link corresponds.

[0139] In the graph structure database 51, as shown in Fig. 12, each node is defined by geographic coordinates and identified by a node key ID. Each link is defined by the geographical coordinates of both ends or the node key ID) and the link shape, and is identified by the link key ID. In addition, link (road Both ends of the (road center line) are connected to nodes (intersections), and node key IDs indicating the intersections of the start and end points of links are given to each link. This reveals the connection relationship between nodes and links, that is, the network structure.

In the graph structure database 51, attached information can be attached to nodes and links. For example, in the case of a route network, it is possible to give information such as the name of an intersection if it is an intersection, and information such as road width and regulatory information (such as one-way traffic) if it is a link.

Next, as shown in FIG. 13, device information such as a camera to be photographed, a car to be mounted, and a camera height is stored in the camera information database 52, and is identified by a camera information key ID.

[0142] The plan / actual shooting information database 53 records shooting plan information dO and actual shooting management information corresponding to each link, as shown in FIG. That is, the operator, the camera information key ID of the camera used for the work, and the image shooting date. Each shooting plan and actual shooting management information are identified by shooting plan key ID and actual shooting key ID, respectively.

[0143] As shown in Fig. 15, the link related information database 54 includes a link key ID, a shooting plan key ID, an actual shooting key ID, and a work progress code indicating the shooting status (planned, shot, edited, A table containing a series of information such as shooting date, camera information key ID used for shooting, transit time of link start and end points, and physical storage location of the image to be stored (volume name of the node disk, etc.) With structure.

[0144] After the shooting plan, a record in which only items not related to actual shooting such as link key ID, shooting plan key ID, and camera information key ID to be used are set for the link scheduled to be shot Is recorded. At this time, the work progress code is not photographed.

[0145] After shooting, the actual shooting key ID, shooting date and time, and camera information key ID used for shooting are stored, and the work progress code is updated.

[0146] The link related information database 54 is a table that can have a plurality of records having the same link key ID, as indicated by the hatched portion in FIG. For this reason, it is possible to store multiple shooting results for one link key ID.

As shown in FIG. 16, the collected image database 55 and the collected image attribute information database 56 belong to the physical image database.

[0148] The collected image database 55 includes a series of images actually taken in time series, The shooting camera information (camera information key ID) used to collect each image, the shooting date, and the shooting time are also recorded.

[0149] The collected image attribute information database 56 includes a position at the time of shooting, a speed at the time of shooting, and a shooting direction.

Information such as the shooting date, shooting time (GPS time), and shooting camera is stored.

[0150] Since the information in the collected image database 55 and the information in the collected image attribute information database 56 have time information, they are associated with each other by the time information.

In the embodiment of the present invention, only one storage device 50 for storing the database is shown, but the storage device 50 is not limited to one. In order to store a large amount of image data, the physical image database 50b may be stored in a storage medium physically different from the image information database 50a. Actually, it is divided and stored on a large-capacity hard disk of 100 GB or more.

[0152] In the database structure according to the embodiment of the present invention, images of a plurality of periods can be simultaneously stored for one link.

Next, the procedure in the image data management system 10 with a geographic information association according to the embodiment of the present invention will be described with reference to FIGS.

[0154] (Data registration)

Step S100: The link information from the external route network data f0 is registered in the graph structure database 51 by the management means 43 of the image data management subsystem 40.

[0155] (Shooting plan)

Step S101: Screen display section of the photographing plan means 41 of the image data management subsystem 40

By 411, the graph structure of the link is displayed from the link information of the graph structure database 51.

Step S102: Link selection unit of photographing plan means 41 of image data management subsystem 40

In 412, the link to be photographed is selected from the displayed link graph structure.

Step S103: The camera information input unit 414 of the shooting plan means 41 of the image data management subsystem 40 registers information about the camera used for shooting in the camera information database 52.

Step S104: Shooting Plan Information of Image Data Management Subsystem 40 Shooting Plan Information of 41 The information input unit 413 prepares a shooting plan for the extracted shooting target link,

• Enter the actual shooting information database 53.

Step S105: The shooting information status port file creation unit 415 of the shooting plan means 41 of the image data management subsystem 40 outputs the shooting plan information dO to the shooting information export file f3.

[0160] (shooting)

Step S106: The link shape in the information of the shooting information export file f3 is read into the image shooting subsystem 20 and the link to be shot is displayed.

[0161] Step S107: A captured image obtained by taking an image while actually traveling on the road of the link by the moving means (vehicle 21), and giving the photographing time by the image data collecting means 24 of the image taking subsystem 20. Collect data dl.

[0162] Step S108: Also, by the position data collection means 25 of the image capturing subsystem 20

Collect collected image attribute information d2 with GPS time.

Step S109: The image capturing subsystem 20 outputs the collected image data dl and the collected image attribute information d2 together with the actual shooting result information to the primary shooting information import file fl.

[0164] (Image data editing)

Step S110: The image data editing subsystem 30 and the shooting information export file f

Enter 3 and the primary shooting information import file fl.

Step S111: Register actual photographing management information.

Step S112: The map data display means 31 of the image data editing subsystem 30 displays the map data.

Step S113: The image data display means 32 of the image data editing subsystem 30 displays the images collected for the link designated on the map data.

[0168] Step S114: Time information is obtained from the displayed collected image data dl and the collected image attribute information d2, and the link start / end time registration means 33 of the image data editing subsystem 30 determines the link information. Record the passage time of the start and end points.

Step S115: Furthermore, the image thinning means 34 of the image data editing subsystem 30 thins out the image from the collected image data dl as necessary to reduce the data amount. [0170] Step SI 16: Output the edited information to the secondary shooting information import file f2.

Yes

[0171] (Information record 'Updated')

Step S117: The information of the secondary shooting information import file f2 is input to the image data management subsystem 40.

Step S118: The actual photographing information data is recorded in the plan / actual photographing information database 53 by the photographing information capturing means 42 of the image data management subsystem 40.

Step S119: The photographing information capturing means 42 of the image data management subsystem 40 updates the information in the link related information database 54.

[0174] (Data extraction)

Step S120: The data extraction means 44 of the image data management subsystem 40 extracts data from the collected image database 55 based on the information in the link related information database 54 according to the user's requirements, and provides link information (network information). Etc. and output as cut-out image data f4.

[0175] (Display)

Step S121: The geographic data-related image data display system 100 of the present invention displays a map and a surrounding image of a designated link (road) by using the cut-out image data f4 and commercially available map data.

[0176] The following effects can be obtained by the image data management system with geographic information described above, and the user's request can be obtained by reading the created cut-out image data with the image data display system with geographic information 100. The local image corresponding to can be displayed.

[0177] In other words, by cooperating with the image capturing subsystem 20, it is possible to manage the power of which areas have not yet been imaged, how much shooting has progressed! it can.

[0178] In addition, since a shooting plan in which necessary portions are cut out from graph structure data such as roads can be created, shooting operations can be performed in parallel using a plurality of vehicles 21 based on the shooting plan. Thus, a large amount of wide-area images can be collected.

[0179] Camera information used for shooting as the camera information database 52 (shooting height and power of the camera The identification number of the camera itself and the camera calibration information for each identification number) are centrally managed, so measurements can be made later using the captured images.

[0180] Using the data structure of the link-related information database 54, the actually captured images are associated with map information having a graph structure such as a complex route network with three or more intersections or intersections of five or more Y-junctions. Can be managed.

[0181] Furthermore, since multiple types of images can be managed in association with the same link, images taken in different seasons and time zones can be provided in accordance with user requests.

[0182] Furthermore, when actually providing image data to a user, necessary data can be extracted from the entire image data captured over a wide area by specifying the following conditions.

[0183] · Geospatial segmentation (Geographic mesh or administrative region segmentation)

• Cut out by link type (road type, road width, etc.)

• Cut out the image according to the shooting time

The means of each subsystem and the arrangement of retained data shown in the drawings for explaining the embodiments of the present invention are merely examples, and are not limited to realizing the functions of the present invention. Therefore, it goes without saying that various combinations of arrangements of processing means are conceivable as long as the functions of the present invention can be realized.

[0184] The following explains in detail the image data display system 100 with relation to geographic information. As the first stage, supplementary explanations are given for the primary shooting information import file fl, secondary shooting information import fino, cropped image data f4, etc. To do.

[0185] The primary shooting information import file fl in Figs. 1, 2, 3, and 7 includes image collection data dl with time and collection image attribute information d2. The collected image attribute information d2 includes position information (coordinates) obtained from GPS, vehicle speed information obtained from a vehicle speed sensor, gyro force, obtained acceleration information and direction information, GPS time, and the like.

[0186] The shooting position display means 102 reads the collected image attribute information d2 of the cutout data f4.

This time-series information (image acquisition coordinates) can be displayed as a symbol (for example, a circle: the shooting position hereinafter) as shown in FIG.

[0187] This will be described with reference to FIG. Figure 31 (a) shows the shooting position when the car was shot at regular intervals while moving faster or slower. [0188] Fig. 31 (b) shows that the collected image data dl (with shooting time) captured at regular intervals stored in the primary shooting information import file fl at this time is arranged on the distance axis (coordinates). Is shown.

[0189] The map data display means 31 in Fig. 1 reads the image acquisition coordinates of the collected image attribute information d2 having the time of the collected image data dl, for example, and the image acquisition coordinates are stored in the distance axis (X — Allocate to memory defined in the Y coordinate system.

[0190] At this time, the image thinning means 34 is involved.

[0191] As shown in FIG. 31 (c), the image thinning means 34 sequentially sets the thinning interval d (distance) to the extent that there is no human visual problem on the distance axis from the shooting start point. When a perpendicular is drawn to the distance axis in Fig. 31 (b) at every interval d, it is determined whether the image acquisition coordinates (imaging position) of the collected image data dl exist. As shown in Fig. 31 (d), the image acquisition coordinates (shooting position) are assigned to the distance axis as representative shooting points (simply referred to as shooting positions) and related to the assigned image acquisition coordinates (shooting positions). Only the collected image data dl and the collected image attribute information d2 to be extracted are extracted and stored in the secondary imaging information import file f2.

In this extraction, for example, time is extracted as a key. That is, only the collected image data dl and the collected image attribute information d2 at the time when the location of the bold line (representative shooting point) in FIG.

[0193] On the other hand, when the representative shooting points are defined on the distance axis, the link start / end point passage time registration means 33 performs the actual shooting result having the time of the collected image attribute data d2 including the position information of these representative shooting points. Information d3 is extracted from the shooting information export file f3 and stored in the secondary shooting information import file f2.

[0194] Further, when the representative photographing points are defined, the actual photographing management information registering means 35 performs the actual photographing management having the time of the collected image attribute data d2 having the image acquisition coordinates (photographing positions) of these representative photographing points. Information d4 (shooting area information, operator information, etc.) is extracted from the shooting information tab file f3 and stored in the secondary shooting information import file f2.

That is, the secondary shooting information import file f2 stores the collected image data dl, the collected image attribute information d2, the actual shooting result information d3, the actual shooting management information d4, etc. of each representative shooting point. It is.

[0196] Then, the actual shooting management information recording unit 421 of the shooting information capturing means 42 of the image data management subsystem 40 assigns an actual shooting key ID to the actual shooting management information d3 of the secondary shooting information import file f2. Plan · Record in the actual shooting information database 53.

Furthermore, the image data recording unit 422 records the collected image data dl and the collected image attribute information d2 of the secondary imaging information import file f 2 in the collected image database 55 and the collected image attribute information database 56, respectively. The actual shooting management information d4 is stored in the plan / actual shooting information database 53.

Further, as shown in FIG. 11, the data extraction means 44 includes an extraction condition input unit 441, a graph structure information extraction unit 442, a link related information extraction unit 443, a collected image data extraction unit 444, and a collected image attribute. It consists of an information extraction unit 554,

The extraction condition input unit 441 inputs conditions for extraction from the database. Conditions include positional conditions, link attribute conditions, and shooting attribute conditions. Positional conditions, also called spatial clipping conditions, include rectangular areas and polygonal areas that can be obtained from mouse operations on the screen and reading files. Various information (image files, attribute information, etc.) in these areas can be clipped. Call it image data!

[0199] The link attribute condition includes designation of attribute information included in the graph structure information such as road width and type. The shooting attribute condition includes designation of a shooting date and a shooting time zone.

[0200] The graph structure information extraction unit 442 receives the extraction condition input unit from the graph structure database 51.

[0201] Further, the extracted link IDs and node IDs at both ends thereof are checked, the link IDs connected to each node are listed, and stored as the graph structure information of the link of the cut-out image data.

[0202] The link related information extraction unit 443 extracts the link corresponding to the link ID extracted above from the link related information database 54. Further, link information that matches the photographing attribute condition input by the extraction condition input unit 441 is extracted from the information.

[0203] The collected image data extraction unit 444 extracts information that meets the conditions input by the extraction condition input unit 441, from the collected image database 55.

[0204] The collected image data extraction unit 444 executes each of the resources extracted from the collected image database. Search for images taken with the camera information key ID on the shooting date stored in the link related information. Furthermore, an image taken between the viewpoint passage time and the end point passage time is extracted from the image, cut out as an image file, and stored as an image file in the image data.

[0205] Further, the image acquisition link ID, the node ID on the shooting start side, the node IDs on both sides of the shooting collection, and the shooting order are cut out and stored as image file attribute information in the image data.

The collected image attribute information extraction unit 445 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image attribute information database 56.

[0208] That is, the data extraction means 44 uses all the cut-out image data f4 (graph structure information, link-related information, image file) corresponding to the representative shooting point shown in Fig. 31 (d) of the cut-out condition (area: area). , Image file attribute information... Is stored in the storage device 106 (see FIG. 31 (f)).

[0209] As described below, the cut-out image data f4 is used to display a map and an image file by the image data display system 100 with geographic information association.

[0210] FIG. 27 is a more specific configuration diagram of the image data display system 100 with relation to geographic information.

[0211] As shown in FIG. 27, the image data display system 100 with a geographic information association is a storage device.

106, geographical display means 101, photographing position display means 102, photographed image display means 103, traveling direction mark display means 107, on-map photographing direction display means 110, note data / registration display means 109, etc. .

[0212] In addition to the map data 108 and the cut-out image data f4, the storage device 106 stores note data related information f6 and the like.

[0213] This note data related information f6 is preferably registered in advance by the note data 'registration display means 109.

[0214] The aforementioned map data 108 preferably has a hierarchical structure. Eg background map This is a hierarchical structure such as the world of network, network information, note data, and map symbol data. The annotation data is the name of the feature, and this name is associated with the coordinates of the annotation data field.

[0215] Furthermore, the storage device 106 stores a plurality of pieces of cut-out image data f4 in the cut-out condition area. These cut-out image data f4 are preferably given an ID code. More specifically, the data structure is as shown in FIG.

Furthermore, the physical image database has a data structure shown in FIG. 30 more specifically.

[0217] When the region (display range) of the extraction condition is input, the geographic display means 101 displays the background map, network information, note data, and map symbol in this region in an overlapping manner. At this time, only the note data in the vicinity of the road of the network data may be extracted and displayed.

[0218] As shown in Fig. 24, the note data 'registration display means 109 has a function of displaying note data on an image on a computer screen.

[0219] In order to realize the display of the note data, the note data 'registration display means 109 is an image that exists within a certain distance from the coordinates of the note data of the map data (for example, the coordinates of the character at the center of the note data). The acquisition coordinates are searched, and among these image acquisition coordinates, the image acquisition coordinates closest to the note data are searched. Then, the ID of the cut-out image data having the image acquisition coordinates is registered in advance in the storage device 106 as the annotation data related information f6 together with the annotation data and the coordinates of the annotation data.

[0220] For example, as shown in Fig. 33 (a), when the network data fO is a road, the image acquisition coordinates exist as Ga, Gb- ···, and note data exist as Ca, Cb '··· In this case, the cut-out image data having the image acquisition coordinates at the distance closest to the coordinates of the annotation data Ca, Gb ′. Then, the direction with respect to the coordinates of the annotation data is obtained from the image acquisition coordinates of the retrieved cut-out image data. This azimuth is indicated by how many times (this angle is Θ k) from the camera center.

[0221] Then, for each piece of note data, Gb '..., the ID of the cut-out image data having the closest image acquisition coordinate is associated, and the orientation Θ k and the image acquisition coordinate are associated (Fig. 33 (b )

) o [0222] The image of the image file linked to this ID is a 360-degree panoramic image, and the center position of the image corresponding to the camera center is the camera center (0 degrees). The coordinates of the annotation data on the image are calculated at the position of the eye level in the direction of Θ k from the camera center. In other words, it has a memory that defines a field for displaying note data (X-Y coordinate system: corresponding to the coordinate system of the panoramic image), and the coordinates of the note data on this memory (note data display on the image) Definition).

[0223] That is, the note data related information f6 includes the ID of the adjacent clipped image data, the note data display position and direction (Θ k), the characters of the note data, and the like.

[0224] The geographic information means 101 reads out map data corresponding to the input segmentation condition area from the storage device 106 and displays it on the computer screen. This map data consists of background maps, note data, map symbol data, and the like.

[0225] The shooting position display means 102 extracts the image acquisition coordinates of all the cut-out image data f4 stored in the storage device 106, and positions on the map on the screen corresponding to these coordinates (for example, on the road) A symbol (photographing position symbol and label) indicating the image acquisition coordinates (photographing position) is displayed at the photographing point (see Figs. 23 and 24).

[0226] That is, since all the shooting position symbols existing in the area of the extraction condition are displayed, the operator can determine at a glance where the image is on the map.

[0227] Then, when the operator designates any of these photographing position symbols with a mouse or the like, the photographing position display means 102 displays the designated photographing position symbol with another symbol (for example, a cross or a double circle). , Different colors).

[0228] When the shooting position symbol on the map on the screen is designated, the map traveling direction mark display means 107 indicates that the shooting order of images having coordinates corresponding to the shooting position symbol is the start number in the same link or It is determined whether it is the end number.

[0229] If it is the start number or end number, it is determined as an intersection. If it is judged as an intersection, the connection link ID associated with the intersection ID of the link is read.

[0230] Then, the coordinates of the image near the intersection on the connection link ID retrieved from the intersection coordinates are read, and the azimuth Hi (3 or 4 differences, ···) of all links connected to the intersection The arrow is displayed on the link according to these directions (see Fig. 28). On the other hand, when a shooting position symbol on the computer screen is selected, the shot image display means 103 extracts an image file of cut-out image data having image acquisition coordinates of the shooting position symbol, and the operator extracts the image file from this image file. The image of the input display range is displayed on the screen.

[0232] The image file of the cut-out image data described above is a 360-degree panoramic image. For this reason, the operator designates a display range of about 50 degrees to the left and right, for example, centering on the shooting direction.

[0233] The image of the image file in this display range is displayed on the computer screen (see Fig. 28), and the display range and shooting direction are notified to the map shooting direction display means 110.

[0234] When the shooting direction and the display range are notified, the on-map image display direction display means 110 displays the display range (from the shooting direction) based on the shooting direction from the image acquisition coordinates (shooting position) on the map on the computer screen. A polygon is generated in the angle direction of the current display range (in the above case, 50 degrees left and right), and this polygon is overlaid on the map on the computer screen (refer to the fan of the map display range in Fig. 28).

[0235] Further, the on-image traveling direction mark display means 107 obtains the direction of the connection link (road) included in the shooting range in the shooting direction from the intersection.

[0236] The direction of the connection link is defined by an angle. For example, as shown in Fig. 32 (a), when the shooting direction is the camera center (0 degree), the direction of the center line of the road ka from the camera center is Θ p, and the direction of the center line of the road kb is It can be defined by Θ i.

[0237] The orientation of Θ p and the orientation of Θ i are determined from the center of the display range (0 degrees), and a triangular polygon (through-through) is placed three-dimensionally at the position on the screen corresponding to the position on the road. (Figure 3

2 (c)).

[0238] When this polygon is selected, the captured image display means 103 sequentially extracts the image file of the shooting position of the road in the selected direction and displays it according to the display range.

[0239] On the other hand, the note data registration and display means 109 reads the specified note (00 apartment, 00 hospital, 00 office), and the ID of the image being displayed and the information related to the note data f 6 When the ID of the image data registered in is matched, the close note data is extracted from the map data. Next, the extracted note data is displayed on an image on a computer screen.

[0241] This annotation data is displayed by assigning the annotation data related information f 6 having the image acquisition coordinates (imaging position) when the imaging position symbol on the road on the computer screen has moved to the imaging position. Then, the character string of the note data is displayed on the computer screen according to the note data display position and direction of the assigned note data related information f6 (see Figure 24).

[0242] Therefore, just looking at the computer screen, you can quickly find what you want.

[0243] In the above embodiment, when the cut-out image data fi is generated, the cut-out image data fi is displayed. However, images related to privacy such as a face and a license plate also exist. is doing. Such privacy-related images may be subjected to a process of blurring with a Gaussian finorator or the like, and stored in the storage device 106 as cut-out image data fi. As a result, privacy-related images are displayed in a blurred manner on the display unit.

In the second embodiment of the present invention, the road network and the background map of the shooting plan area included in the shooting plan information dO are displayed on the screen, and these links are shot, and the acquired collected image data dl (all Ambient images: Both image files! /, And collected image attribute data d2 are managed according to the shooting time, and images of a predetermined rectangular area of these all-around images (also referred to as camera parameter designation display images) are displayed on the screen. In the case of an intersection, a path selection polygon for selecting a path is displayed. Then, a number of camera parameter designation display images in the course (link) corresponding to the selected course selection polygon are reproduced. At this time, a map including a link is displayed on the screen, and the configuration and operation for displaying the position of the camera parameter designation display image with a symbol are supplemented.

FIG. 34 is a schematic configuration diagram of the image data management system with geographic information association according to the second embodiment. In FIG. 34, description of components having the same reference numerals as those in FIG. 1 is omitted.

[0246] (Image capture system) The image capturing subsystem 20 of the image data management system with geographical information association according to the second embodiment includes an image sensor 22, a hybrid type position sensor 23, an image data collecting unit 24, and an actual image capturing management information input unit 72. It has.

[0247] The actual shooting management information input means 72 displays a screen for inputting actual shooting management information before the start of shooting, and the operator name, actual shooting camera information key ID, and actual shooting date (time) are displayed on this input screen. Is input and stored in a memory (not shown).

[0248] Then, after shooting, the actual shooting management information input means 72 opens the above-described input screen and inputs a shooting result (shooted, shot not possible, etc.).

[0249] Further, the actual photographing management information input means 72 stores the actual worker name, actual photographing date, photographing result, and the like input on the input screen in the temporary memory 26 as actual photographing input information d3a. At this time, an actual shooting key ID combining the actual shooting date and the actual shooting camera information key ID is generated and included in the actual shooting input information d3a. Of course, the actual shooting plan key ID may be entered manually.

[0250] That is, every time the inside of the link is shot, the collected image data dl, the collected image attribute data d2, and the actual shooting input information d3a are stored as a set in the temporary memory 26, and the primary shooting information is imported. Output to file fl.

[0251] (Image data editing subsystem)

The image data editing subsystem 30 has the same configuration as that shown in FIG. 1, and includes a large number of collected image data dl, a large number of collected image attribute data d2, and actual shooting for each link in the secondary shooting information import file f2. Result information d3b and actual photographing management information d4 are stored. This actual shooting result information d3b is the same as the actual shooting result information d3 shown in Fig. 1.

[0252] (Image management subsystem)

In addition, the image data management subsystem 40 includes a photographing plan means 41, a photographing information capturing means 42 (see FIG. 34), and a management means 43, which are the same as those in FIG. Means 73 are provided.

[0253] The above-described data extraction means 73 inputs the extraction area (link ID or area) and the shooting time (year, month, day, time, normal, reverse direction) and has a link function having the link ID of the input extraction condition. Link information is allocated from the link information related information database 54, and this link ID, shooting date / time, collected image data dl, and collected image attribute data d2 are output to the import file f4.

[0254] Then, the data extraction means 73 reads the link ID, and if a link ID with the same number exists in the link related information database 54, the shooting plan key ID and the actual shooting key are recorded in the records of these link IDs. If the ID is written and the actual shooting information (actual shooting result information) is associated, the collected image data of all the link related information having the same link ID dl The collected image attribute information data d2, link-related information, map data, etc. are extracted as cut-out data and output (stored) in the import file f4. Figure 34 is divided into multiple sections to emphasize that it is cut-out data that includes information with different shooting times.

That is, as shown in FIGS. 37 and 38, this cut-out image data is extracted from the screen shot at different times on the same link.

[0256] Figure 37 shows images extracted by shooting on the same link at different dates. 37—A1 shows an image taken in the morning (part of the entire surrounding image). — 37— A2 below A1 shows a map with shooting locations. 37—B1 shows an image taken at night (a part of the all-around image), and 37—B2 below 37—B1 shows a map with the shooting position.

[0257] Fig. 38 shows images extracted by shooting on the same link in different movement directions.

38-A1 shows an image (part of the all-around image) taken in the north lane (front), and 38-A2 below 38-A1 shows a map with the shooting position. 38—B1 shows an image (part of the all-around image) taken when driving in the south lane (backward direction), and 38—B2 below 38—B1 shows a map with the shooting position.

Here, an outline of the operation of the configuration of FIG. 34 will be described.

[0259] The shooting plan means 41 of the image data management subsystem 40 displays a road network including a link to be shot on the screen (with a background map), and selects a link. At this time, an input screen (not shown) for shooting plan is displayed. The link ID of the selected link is automatically written in the input screen for this shooting plan. Also, the camera information of the camera information database 52 is displayed on the screen, and the camera key ID (camera to be shot) of the selected camera information is taken. Write to the shadow plan input screen.

[0260] Further, the shooting plan means 41 displays a shooting date (including time) on the shooting plan input screen.

When the scheduled work person inputs, a shooting plan key ID is generated by combining the scheduled shooting date and the camera key ID scheduled to be shot.

[0261] The shooting plan means 41 uses the shooting plan information dO (shooting plan key ID, link ID, planned work person, scheduled camera information key ID, scheduled shooting date, etc.) Save to 53. At this time, it is preferable that the hard disk volume name and work progress code (not photographed) are input and included in the photographing plan information dO.

[0262] Furthermore, the shooting plan means 41 reads the plan 'actual shooting information database dO, and records the link ID, shooting plan key ID, scheduled shooting date, scheduled shooting camera information key ID and the link related information database 54 record. Write. At this time, write the hard disk volume name and work progress code.

[0263] As shown in Fig. 15, the record in the link related information database 54 described above is linked ID, shooting plan key ID, actual shooting key ID, work progress code, shooting result, shooting date, start point passing time, and end point passing. The record has an area where the time and storage location are written.

Yes

[0264] In addition, if you want to capture the morning and evening on the same link, select the same link and enter the planned shooting date (including time), scheduled work person, etc. on the input screen for the shooting plan. Let me input it. At this time, an imaging plan key ID is generated by combining the scheduled imaging date and the scheduled camera key ID. Then, the work plan information dO having the same link ID is stored in the plan 'actual photographing information database 53.

The imaging plan means 41 reads the work plan information dO having the same link ID and stores it in the link related information database 54. That is, link related information having the same link shown in FIG. 15 is generated in advance.

[0266] The imaging plan means 41 records such link related information in the link related information database 54 as fe.

[0267] The shooting plan means 41 sequentially reads these link-related information records, and for each read, the link ID, shooting plan key ID, shooting preliminary date, scheduled work person, etc. of the record are recorded. Shooting plan information dO is output to shooting information export file f3.

[0268] At this time, it is preferable to output the shooting plan information dO including the road network and the background map.

On the other hand, the position data collection means 25 of the image capturing subsystem 20 displays the road network and background map of the shooting plan information dO in the shooting information etsport file f3 to confirm the link to be shot.

[0270] Then, when the image is taken while actually traveling on the road of the link by the moving means (vehicle 21), the collected image data to which the photographing time is given by the image data collecting means 24 of the image photographing subsystem 20 dl And the collected image attribute data d2 are collected by the position data collecting means 25.

[0271] A plurality of collected image data dl and collected image attribute data d3 in these links are output to the primary imaging import file fl.

[0272] Then, after shooting, the actual shooting management information input means 72 opens the above-mentioned input screen, and the shooting result (shooting completed, shooting disabled, etc.), actual worker name, actual shooting date and time, actual shooting camera information key I

Input D etc. At this time, an actual photographing key ID is generated.

In other words, the actual shooting management information input means 72 has the shooting plan key ID, the actual shooting key ID, the shooting result (taken), the work progress code (taken), the actual worker name, the actual shooting date, and storage. The location etc. (actual shooting input information d3a) is output to the primary shooting information import file fl.

[0274] On the other hand, the image data editing subsystem 30 reads the actual shooting input information d3a stored in the primary shooting information import file fl! /, And the actual shooting key I of this actual shooting input information d3a.

D, shooting result (photographed), work progress code (photographed), actual worker name, actual shooting date, storage location, etc. are linked and the link ID of shooting plan information dO is set as actual shooting management information d4 Save to the next shooting information import file f2.

[0275] Further, the map data display means 31 of the image data editing subsystem 30 displays map data, and the image data display means 32 displays the link specified on the map data.

V, Display the collected image (collected image).

[0276] At this time, in the method of designating from the photographing time, for example, the operator displays the collected image data dl corresponding to the time of the end point of the selected link on the screen. [0277] The operator looks at the roads and intersections photographed in the image, and if it is determined that the position is shifted at the intersection, etc., the operator operates the screen in chronological order. Display the previous or next image. Then, the photographing time of the image at the intersection determined by the operator is corrected and registered as the passage time of the start point or end point of the link.

[0278] This correction registration is because the image sensor 22 (camera) takes a picture at a certain time, and the vehicle moves at a predetermined speed (5km / h, •• Okm '' 60km '' 80km). For this reason, each link has multiple collection images and attribute information. In addition, since the vehicle stops at the intersection and the speed is slow, there are a large number of collected images and attribute information at this intersection. That is, there are a large number of collected images and attribute information in the vicinity of the node. Therefore, it is necessary to determine the collected images and attribute information at the start point (node) and end point (node) of one link.

[0279] Therefore, when the operator looks at the collected images of roads and intersections displayed on the screen and determines that the position is misaligned, the operator operates the screen in time-series manner. The image before or after is displayed, and the shooting time of the image at the intersection determined by the operator is the passage time of the start point or end point of the link.

[0280] Thus, even if there are a large number of collected images at an intersection, etc., the one at the specified passing time is the collected image at the intersection, and the one between this start point and the end point is within the link. Specifying the passage time of the start point and the end point means that the most appropriate one of the collected image data dl and attribute information data d3 is assigned to the link. Become. Also called image grouping. (See Fig. 35)

That is, as shown in FIG. 35, when the shooting time (a) of the collected image and the shooting time (b) of another collected image in the link for which the link start / end point passage time registration means 33 is selected are designated, The shooting time (a) is assigned to the start point (Ea) of the selected link, and the shooting time (b) is assigned to the end point (Eb). Then, the passage time of the start point and the end point is added to the actual shooting input information d3a having this link ID, and stored as the actual shooting result information d3b in the secondary shooting information import file f2.

[0281] Further, the image thinning means 34 of the image editing data system 30 moves the collected image data dl of the primary shooting information import file fl for the purpose of reducing the data capacity. Thin out according to the distance. As an example of the thinning method, there are the following methods as shown in FIG. Using one frame (collected image data: also called image frame) as the starting point, the distance from the starting frame to the i-th frame is calculated by Equation 1 for each frame.

That is, for each link ID of the selected link, the image thinning means 34 is associated with the link ID! /, The speed information and the time information of the time-series information for each shooting time (fixed time). For each), find each distance Li (Ll, L2--': each distance to the end point) from the specified start point (eg start point). For each distance d that is an integral multiple of a predetermined fixed distance, the distance di is the closest to this integral multiple distance di! /, And the distance is obtained from the respective distance Li, and these obtained distances Lni (Lnl, Lnj2 ,...), Each attribute information (d2) having position information corresponding to the attribute information (d2) and a collected image (dl) of the attribute information are extracted as extracted data from a plurality of collected images and attribute information for each collected image. Yes. That is, in the secondary import file f2, a large number of collected images dl and attribute information d2 after being thinned within the link are stored for each link.

[0283] Therefore, the secondary shooting import file f2 contains the actual shooting result information d3b including the link ID, shooting plan key ID, actual shooting key ID, and actual shooting date for each link. The actual image management information d4, a large number of collected image data dl thinned out in the link, and these collected image attribute data d2 are stored as a set.

On the other hand, the shooting information capturing means 42 of the image data management subsystem 40 stores the actual shooting management information d4 for each link of the secondary shooting import file f2 in the plan 'actual shooting information database 53 of the storage device 50 and Read actual shooting result information d3b for each link

Yes

[0285] Next, in the record having the link ID and shooting plan key ID of this actual shooting result information d3b in the link related information database 54, the actual shooting key ID of the actual shooting result information d3b, the actual shooting date, and the camera actually shooting Write the information key ID, the start time and end time, and the work progress code (already edited). In other words, the contents at the time of shooting planning of the record are updated (scheduled shooting date, camera information key ID, work progress code, shooting result: V where nothing else has been written, location is newly written).

[0286] In addition, the collected image data dl of the link of the secondary shooting import file f2 is collected. In addition to storing in the image database 55, a large number of collected image attribute data d 2 is stored in the collected image attribute information database 56.

[0287] Then, the shooting information capturing means 42 re-reads the secondary shooting import file f2, and collects image data dl, collected image attribute data d2, and actual shooting result information d3b for each link (may be the same link). If actual shooting management information d4 exists, it is newly read. As described above, the actual shooting result information d3b actual shooting key ID, actual shooting date ID and actual shooting are recorded in the record having the link ID and shooting plan key ID of this shooting result information d3b in the link related information database 54. Write the camera information key ID, start time, end time, and work progress code (edited). In other words, the contents at the time of shooting planning of the record are updated (Scheduled shooting date, camera information key ID, work progress code, shooting result: everything else is written, nare, location is newly written)

Further, the collected image data dl of the link of the secondary shooting import file f2 is stored in the collected image database 55, and the collected image attribute data d2 is stored in the collected image attribute information database 56.

[0288] Therefore, if the actual shooting management information d4, actual shooting result information d3b, collected image data dl, and collected image attribute data d2 of the same link exist in the secondary shooting import file f2, the link related information As shown in FIG. 15, link related information having the same link ID is stored in the database 54.

[0289] Then, the data extraction means 73 inputs the extraction area (link ID or area) and shooting time (year / month / day, time, forward / reverse direction) and inputs the input extraction condition link ID and shooting time. The link related information having etc. is allocated from the link information related information database 54. Next, a large number of collected image data dl and collected image attribute data d2 between the start point passage time and the end point passage time of the link related information related to this link ID are also read out and output to the import file f4.

At this time, link related information having a link ID corresponding to the extraction condition is read from the link related information database 52 and output.

[0291] Also, the data extraction means 73 reads the link IDs, and if a link ID having the same number exists in the link related information database 54, the data extraction means 73 records the photometer in the records of these link IDs. If the image key ID and actual shooting key ID are written! /, And the actual shooting information (actual shooting result information) is associated, all the links having the same link ID A large number of collected image data dl and collected image attribute information data d2 are extracted and output (stored) in an import file f4.

[0292] Further, the data extracting means 73 reads out the link related information having the link ID of the same number from the link related information database 52 and outputs it.

[0293] 40-C1 in Fig. 37 shows an example of link-related information. This 40-C1 link-related information is a more specific example of Figure 15.

[0294] Also, reference numeral 41-D1 in FIG. 38 shows an example of link-related information. This 41-C1 link related information is a more specific example of Figure 15.

Note that the data extraction means 73 outputs network information corresponding to the link type or area, as well as note data, map symbol data, and the like to the storage device 106.

[0296] Here, at the intersection shown in Fig. 28 or Fig. 32, in order to read a triangular polygon (also referred to as a traveling direction selection polygon Pi) for selecting a traveling road and display the collected image data of that link. This configuration will be described with reference to FIG.

As shown in FIG. 36, the image data display system 100 with relation to geographic information includes a traveling direction arrow link determination unit 107 in addition to the photographed image display unit 103.

[0298] The captured image display means 103 includes a camera parameter designation image display unit 103a and an all-around image extraction unit 103b. Further, the traveling direction arrow link determination means 107 includes a selection arrow reading unit 107a, a connection point determination unit 107b, and a traveling direction arrow creation unit 107d, and the collected image data dl of the extracted image data (also referred to as extracted data) is included in the link. When the first or last of the order is reached, the continuous display of the camera parameter designation display image is discontinued, and the route is selected. In route selection, the node ID of the last displayed side is first checked from the attached information (also called attribute information) of the image file.

[0299] Next, the graph structure included in the cut-out image data is examined, and a link connected to the node is searched. Furthermore, by examining the image acquisition link ID included in the cut-out image data, if there is image data (collected image data) that captured the link, an arrow or click button is displayed on each path displayed on the image. Etc. (Drawing direction selection polygon Pi) Next, the user designates which route image to display.

[0300] Then, a link having the coordinates of the selected course direction selection polygon Pk is retrieved, and camera parameter designation display images of the collected image data dl related to the retrieved link are displayed in ascending or reverse order.

[0301] The camera parameter designation image display unit 103a of the captured image display means 103 uses the image file (collected image data dl: all-around image) from the all-around image extraction unit 103b for the production of CG (composite graphic). It is converted into a rectangular area using a method called texture mapping and displayed (camera parameter designation display image).

[0302] Further, the traveling direction selection polygon Pi (Pl, ··· Ρ4) from the traveling direction arrow creation unit 107c of the traveling direction arrow link determination 107 is displayed so as to overlap the collected image data d2.

[0303] For example, when the shooting time (year, month, day, time, direction of travel) is input, the all-around image retrieval unit 103b reads the link related information corresponding to the shooting time from the storage device 10b, and starts the link related information. A lot of collected image data dl having time information (shooting date and time) of passage time and end point passage time is allocated, read out in ascending or reverse order, and output to the camera parameter designated image display unit 103a.

[0304] Further, when a position is specified on the road (link) in the road map, the all-around image extraction unit 103b assigns attribute information d2 having this position, and collects image data related to this attribute information. dl is output to the camera parameter designation image display unit 103a.

[0305] That is, if the link-related information having the same link number exists in the selected link, the omnidirectional image extraction unit 103b uses the shooting time as the extraction condition and takes a picture of this extraction condition. There is a search method by the same link time to read link related information with time. Then, when the link-related information includes an actual shooting key ID, the all-around image extraction unit 103b has multiple pieces of time information within the passage time of the start point and the end point of the link-related information. The collected image data dl are extracted from the storage device 10b in ascending or reverse order and output to the camera parameter designated image display unit 103a.

[0306] The connection point determination unit 107 of the travel direction arrow link determination unit 107 determines whether the position specified on the road map is a connection point (for example, an intersection) or not in the storage device 10b. [0307] Further, the connection point determination unit 107 of the traveling direction arrow link determination unit 107 determines that the connection point is reached when the all-around image extraction unit 103b extracts the start point or the end point collection image.

[0308] When the connection point is determined, a stop instruction is output to the all-around image extraction unit 103b to stop reading the collected image data dl (image file). That is, the continuous display of the collected image data dl (image file) is stopped.

[0309] When the traveling direction arrow creation unit 107c is determined to be a connection point, it reads the connection point from the network information, reads the direction of each link connected from the node information of the link at this connection point, The direction information corresponding to these link directions, the preset display position, and the traveling direction selection polygon Pi are output to the camera parameter designation image display unit 103a.

[0310] Thus, for example, if the intersection is a four-way intersection at the connection point, the travel direction selection polygons Pl, P2, P3, and P4 are the four-link images of the camera parameter designation image display unit 103a. The camera parameter designation display image (rectangular area) of this all-around image is displayed. Since this camera parameter designation display image is a predetermined field of view range (rectangular region), it is an image in the traveling direction, so two traveling direction selection polygons are displayed.

[0311] The selection arrow reading unit 107a reads the selection position and notifies the selection link determination unit 107d when any of the screen direction selection polygons Pl and P2 is selected.

[0312] The selected link determination unit 107 obtains the selected position from the all-around image (image file: collected image data dl) existing in the camera parameter designation image display unit 103a, and stores the link having the obtained position coordinates. Obtained from the network information fo of device 10b. Then, the link ID of the obtained link is output to the all-around image extraction unit 103b.

[0313] Therefore, the omnidirectional image extraction unit 103b displays the image data (collected image data: omnidirectional image) obtained by capturing the link related to the link ID included in the cut-out image data in ascending or reverse order. .

In this ascending order, the start time of the link is early! /, And the time of the end point is late! /. In this case, a young number is assigned to the start point and an old number is assigned to the end point, and read in ascending order. Conversely, if the start point is late! /, And the end point is early! / The point is the young one, the number is the old one! /, The number is assigned, the young! / Is read out in order!

[0315] 40—C1 in Figure 37 is a specific example of link-related information, and the top record of 40—C1 is

This is link related information at the time of lunch. Also, the next record of 40—C1 shows link related information when the same link is taken at night.

[0316] Furthermore, the top record of 41—D2 in FIG. 38 is link related information when taken in the morning, and the next record is link related information when taken in the afternoon. This link related information indicates that the picture was taken from the end point to the start point.

[0317] In the above embodiment, the road is described as a road, but a pipeline, a park, a mountainous area, a seaway

It may be an air route.

[0318] Means of each subsystem shown in each figure for explaining the embodiment of the present invention and the arrangement of data to be held are merely examples, and are limited to realizing the function of the present invention. Not. Therefore, it goes without saying that various combinations of arrangements of processing means are conceivable as long as the functions of the present invention can be realized.

Claims

The scope of the claims

An all-around image with time information obtained by moving multiple links in a route network using an all-around camera mounted on a moving body at regular intervals and a hybrid position sensor with GPS. A geographic information-related image data display system that displays an image of a rectangular area on the screen while reproducing and displaying a path selection polygon for selecting one of the plurality of links.

First storage means for storing, for each link of the route network, attribute information including position information and time information from the position sensor for the entire surrounding image and the entire surrounding image in the link;

Shooting plan information including the link ID when the link is shot and first actual shooting information including a first actual shooting date when shooting is actually performed based on the shooting plan information. As related information, a link ID that is the same as the link ID, the shooting plan information, and a second actual shooting date that is different from the first actual shooting date when the actual shooting was performed based on the shooting plan information. Second storage means for storing the second actual photographing information as second link related information;

A route network having link information including the link ID, vector information of the link, and node IDs at both ends of the link, and node information including the link ID connected to the node, and a background map storing a background diagram. 3 storage means;

Geographic display means for selecting a link by displaying a map comprising the route network and a background map in an area different from the image of the rectangular area;

Read the attribute information having the link ID of the selected link, and sequentially read the plurality of all-around images in the selected link related to the attribute information from the first storage means in ascending or reverse order. Image reading means;

When the omnidirectional image is read from the first storage means, the attribute information related to the omnidirectional image is read, and when the position information of the read attribute information is located at a node of the route network Means for stopping reading of the all-around image and obtaining all directions of links connected to the node from the route network;

All the surrounding images that have been read out and all the links that lead to the obtained node Means for displaying the traveling path selection polygon indicating a direction in an interlocking manner by overlapping the rectangular area of the screen;

When one of the route selection polygons is selected, the link ID corresponding to the position of the selected route selection polygon is read from the route network, and the all-around image is associated with the read link ID. Whether the first link-related information or the second link-related information having the read link ID, the actual shooting information, and the shooting plan information exists. Means for releasing the stop and outputting the read link ID as the selected link ID to the all-around image reading means;

An image data display system associated with geographic information, characterized by comprising:

[2] The first link related information is

At the start points of the both ends, the time information of the all-around image that has been optimized is used as the passage time of the start point, and the time information of the all-around image that has been optimized at the end point is the passage time of the end point,

The all-around image reading means includes:

2. The image data display associated with geographic information according to claim 1, wherein the all-around image is sequentially read out in the ascending order or the reverse order by comparing the passage time at the start point and the passage time at the end point of the selected link. system.

[3] The image display device further includes a photographing position display means for displaying an identification symbol on the screen for notifying the display position of the currently displayed all-around image every time the position is designated by the link. The image data display system with geographic information relation according to claim 1 or 2.

[4] The all-around image, the attribute information, the first link-related information, and the second link-related information are data extracted from the plurality of all-around images and attribute information power according to the shooting time,

Means for inputting the shooting time when the link is selected;

The first link related information having the link ID of the selected link is allocated, and the link ID of the allocated first link related information is output to the all-around image reading means as the selected link ID. Means to do The image data display system with associated geographic information according to claim 1, further comprising:

5. The display of image data associated with geographic information according to claim 1, further comprising means for displaying the range displayed by the image of the rectangular area on the map by a fan-shaped polygon. system.

6. The image data display system with associated geographic information according to claim 1 or 5, wherein the travel path selection polygon and the fan-shaped polygon are subjected to force processing.

[7] The determination as to whether or not an all-around image is associated with the read link is based on the link ID, the shooting plan key ID indicating that there is a shooting plan for this link, and the fact that this link has been shot. When the shooting key ID is present in the first link related information or the second link related information having the selected link ID, it is determined that the all-around image is present. The image data display system with geographic information according to any one of claims 1 to 6.

[8] The map includes feature and note data, and the note data is associated with the feature by coordinates,

Means for retrieving the attribute information having the position information closest to the coordinates of the annotation data for each of the annotation data, and storing the attribute information in association with the annotation data;

Means for superimposing and displaying the annotation data of the attribute information on the all-around image of the attribute information when the attribute information is assigned;

The image data display system with associated geographic information according to claim 1, further comprising:

[9] The annotation data is obtained by obtaining an orientation from the shooting position to the annotation data of the attribute information related to the displayed all-around image, and is set in the orientation and in advance. 9. The image data display system with associated geographic information according to claim 8, wherein the image is displayed in an overlapping manner.

[10] The omnidirectional image is displayed by simply displaying the omnidirectional image related to the attribute information as it is, and by displaying the omnidirectional image in an arbitrary direction from the photographing center and in a predetermined field of view. An image of the rectangular area displayed as a computer graphic as an image taken at a corner 10. The geographic information-related image data display system according to claim 1, further comprising an image display.

An all-around image with time information obtained by moving multiple links in a route network using an all-around camera mounted on a moving body at regular intervals and a hybrid position sensor with GPS. An image data display program associated with geographic information that displays an image of a rectangular area on the screen while reproducing and displaying a traveling path selection polygon for selecting one of the links.

A route network having link information including the link ID, vector information of the link, and node IDs at both ends of the link, and node information including the link ID connected to the node, and a background map storing a background diagram. With 3 storage means

With

On the computer,

A geographical display means for selecting a link by displaying a map comprising the route network and a background map in a region different from the image of the rectangular region;

Read the attribute information having the link ID of the selected link, and sequentially read the plurality of all-around images in the selected link related to the attribute information from the first storage means in ascending or reverse order. Image reading means,

When the omnidirectional image is read from the first storage means, the attribute information related to the omnidirectional image is read, and the position information of the read attribute information is the route network. Means for stopping the reading of the all-around image when it is located at a work node, and obtaining all directions of links linked to the node from the route network;

Means for displaying the all-around image read out and the route selection polygon indicating the directions of all links connected to the obtained node in an interlocked manner on the rectangular area of the screen;

When one of the route selection polygons is selected, the link ID corresponding to the position of the selected route selection polygon is read from the route network, and the all-around image is associated with the read link ID. Whether the first link-related information or the second link-related information having the read link ID, the actual shooting information, and the shooting plan information exists. Means for releasing the stop and outputting the read link ID to the all-around image reading means as the selected link ID

Image data display program with geographic information to execute the function as

[12] The first link related information is:

In the computer,

The all-around image reading means includes:

12. The geographic information-related function according to claim 11, wherein a function for sequentially reading out the entire surrounding image in the ascending order or the reverse order is performed by comparing the passage time at the start point and the passage time at the end point of the selected link. Image data display program.

[13] In the computer,

12. The function for performing a function as a photographing position display means for displaying an identification symbol for notifying a display position of an all-around image currently displayed on the screen every time a position is designated by the link. Image data display program with geographic information related to 12.

[14] The all-around image, the attribute information, the first link-related information, and the second link The related information is data extracted from the plurality of surrounding images and attribute information power according to the shooting time,

In the computer,

Means for inputting the shooting time when the link is selected;

The first link related information having the link ID of the selected link is allocated, and the link ID of the allocated first link related information is output to the all-around image reading means as the selected link ID. Means to do

14. The program for displaying image data with associated geographic information according to claim 11 for executing the function as the above.

[15] In the computer,

15. The program for displaying image data associated with geographic information according to claim 11 for executing a function as means for displaying a range where the image of the rectangular area is displayed on the map by a fan-shaped polygon.

[16] In the computer,

16. The program for displaying image data associated with geographic information according to claim 11 or 15, wherein the travel path selection polygon and the fan-shaped polygon perform a function as means for performing a blurring process.

[17] In the computer,

Whether or not an all-around image is associated with the read link is determined by determining the link ID, the shooting plan key ID indicating that there is a shooting plan for this link, and the actual shooting key ID indicating that this link has been shot. Are present in the first link related information or the second link related information having the selected link ID, and function as means for determining that the all-around image is present. 17. The image data display program associated with geographic information according to claim 11 for execution.

[18] The map includes feature and note data, and the note data is associated with the feature by coordinates,

In the computer,

For each note data, the position information is closest to the coordinates of the note data. Means for retrieving and storing the attribute information in association with the note data; and means for causing the note data of the attribute information to be superimposed and displayed on the all-around image of the attribute information when the attribute information is allocated.

18. The image data display program with associated geographic information according to claim 11, wherein said image data display program is for executing a function as the image information.

[19] In the computer,

The annotation data obtains an orientation from the shooting position to the annotation data of the attribute information related to the displayed all-around image, and is superposed and displayed at the azimuth and preset height. Means to do

19. The program for displaying image data with associated geographic information according to claim 18, for executing the function as the above.

[20] In the computer,

For the display of the omnidirectional image, a simple display in which the omnidirectional image related to the attribute information is displayed as it is, and the omnidirectional image is photographed in an arbitrary direction from a photographing center and at a predetermined viewing angle. For displaying a computer graphic as a processed image

20. The geographical information association image data display program according to claim 11 for executing a function as a geographic information.