JP2008243185A - System for managing image data associated with geographical coordinates - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for consistent photographing planning, and image collection and management so as to make a wide range photographing plan, collect a great deal of images and manage image data associated with geographical information. <P>SOLUTION: A system for managing image data associated with geographic coordinates comprises an image photographing subsystem 20 comprising a transportation means connected with an image sensor 22, and mounted with an image data collection means 24 for collecting as image data photographed images of surrounding environment along a map link, together with the photographing time and a position data collection means 25; an image data editing subsystem 30; an image data managing subsystem 40; and a storage device 50 comprising a collected image database 51 and a collected image attribute information database 56 respectively for recording captured image data 55 and collected image attribute information; and a link associated information database 54 for recording management information for associating each database information for each map link. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention captures and collects image data for grasping a geographical environment, for example, by shooting on a road or flying over the sky, and extracting and processing according to a required geographical range, image resolution, and image data amount, The present invention relates to an image data management system associated with geographic coordinates that is managed spatially and temporally in association with geographic coordinates and obtains data to be displayed in conjunction with a map.

With the advancement of information technology and the development of large-capacity communication networks, large-capacity databases such as photographs and moving images can now be handled easily on display media such as the Internet, car navigation, and mobile phones.

Along with this, there is a growing need to accurately convey the actual landscape to the user without deformation by using an image as a geographic attribute in a geographic information system (GIS) or a map search site on the Internet.

  A technique has been developed for taking a cityscape image taken from a specific road and using it for maintenance of roads and rivers (see, for example, Patent Document 1).

  In this technique, an image obtained from a feature on a road contour line is managed on a one-dimensional feature such as a road center line in association with a distance from a start point designated on the feature.

In addition, there is a technique for managing an artificially created CG image in association with a road having a graph structure in part, not an actually captured image (see, for example, Patent Document 2). Recently, a technique for displaying an actual image has been developed (see, for example, Patent Document 3).
Japanese Patent No. 3099103 Japanese Patent Laid-Open No. 6-83937 JP 2005-3752 A

(Managing data with consistency, reducing the amount of data)
In the present invention, it is possible to provide a user with image data with a geographic coordinate relationship, which basically captures a range designated by the user and reduces the file size, and also manages a system from which image types are centrally managed. The main purpose is to obtain.

  For example, the townscape (trees, roads, buildings, etc.) seen from the road changes depending on the year.

  Further, the image may require an image taken in the traveling direction or an image taken from the opposite direction in one path.

  Furthermore, in order to finish shooting in a short time, there are cases where shooting is performed with different cameras in different vehicles.

  In addition, the shooting position of the image shot by the camera may differ depending on the camera, GPS, INS performance and the map coordinate system.

  However, since the above cited references are managed by associating an image with a link ID, the image is taken at a different time in one route with the link ID alone or from the opposite direction. The computer cannot determine whether the image is correctly associated with the shooting position. For this reason, there is a problem (problem) that an operator needs to input various parameters in order to display a desired image.

  On the other hand, in the above-mentioned Patent Document 3, an image of a shooting point actually shot is registered in a database and displayed in conjunction with a shooting point on a map, but an image is registered corresponding to the shooting point. Therefore, it consumes a lot of database capacity. For this reason, it is not suitable for a general personal computer having a low-capacity database.

  Therefore, even in one route, it is possible to centrally manage aged images (taken with a camera), images taken with different cameras, or bidirectional images related to the correct shooting position. At the same time, a system capable of registering and displaying these images on a general personal computer has been desired.

  Further, when there are a plurality of routes and photographing without planning, it takes time and cost, and the capacity of the database becomes large.

  It is not necessary to photograph the same route when a plurality of cameras have taken different routes, there are routes that do not require images, or when there is not much time. In other words, making a shooting plan efficiently reduces time, cost, and database capacity.

  In other words, when an image corresponding to the shooting position of the map is requested promptly, if the shooting plan is not firm, the image cannot be provided promptly until the shooting is completed after the request is received.

  Furthermore, the coordinates of the shooting location may differ from the coordinates of the map depending on the performance of the camera, GPS, INS, and the like.

  Even if there is a slight difference in the performance of the camera, GPS, INS, etc., the present invention accurately defines the shooting position on the map, and even if the images are acquired over time, bi-directional, different cameras, It is an object of the present invention to obtain a system that can reduce the capacity of a database and that can be easily provided by centrally managing these images in a management form that can be easily understood.

That means
(Issues for collecting image data)
Since the amount of image data to be captured and collected by road driving or flying over the air becomes large, it is necessary to link with the imaging system. In particular, it is important to be able to obtain information such as which districts have not been photographed and how much photography has progressed. In addition, when a large amount of wide-area images are collected, it is necessary to perform photographing work in parallel using a plurality of vehicles. At this time, it is necessary to create a shooting plan in which necessary portions are cut out from the graph structure data such as roads and collect images according to the shooting plan.

  The main object of the present invention is to basically capture a range designated by the user and provide it to the user as geographic coordinate-related image data. Therefore, when quickness is required, there is a problem that data cannot be provided until the photographing is completed after the request is received.

  Therefore, in order to plan a wide range of shooting plans in advance, collect a large amount of images, and manage image data in association with geographic information, a system for consistent shooting plans and image collection / management is required. .

(Issues for collecting image data)
Since the amount of image data to be captured and collected by road driving or flying over the air becomes large, it is necessary to link with the imaging system. In particular, it is important to be able to obtain information such as which districts have not been photographed and how much photography has progressed. In addition, when a large amount of wide-area images are collected, it is necessary to perform photographing work in parallel using a plurality of vehicles. At this time, it is necessary to create a shooting plan in which necessary portions are cut out from the graph structure data such as roads and collect images according to the shooting plan.

  In addition, if you want to perform measurement later using the captured image, for example, when performing stereo image measurement or simple measurement using a single photograph, the actual captured camera information, that is, the shooting height of the camera or the identification of the camera itself It is necessary that the calibration information of the camera with the number and each identification number is managed together with the image and can be referred to. When capturing a large amount of images over a wide area, not only a single camera system but also a plurality of camera systems may be used. Therefore, it is necessary to centrally manage camera information used for capturing.

(Issues for image data management)
In the technique disclosed in the above-mentioned patent document, an actually captured image is complicated by existing map data, particularly, as shown in FIG. It is managed in association with map information having a graph structure such as road network data.

  Further, since there is only one type of image that can be managed in association with map information at one location, a plurality of types of images are managed in association with locations on the same road, and a database is constructed for each type.

  For example, depending on the object to be shown, the season and time zone when the image was taken may be important. For example, images taken in the spring, summer, autumn, and winter seasons are stored in the sightseeing route.

  There is a spatio-temporal geographic information system as a system for managing images for each multi-period and season, which displays a version in a certain time section. Therefore, for example, it is possible to view only images of a specific season (for example, autumn) from images taken over a plurality of years.

  In addition, for a wide road, it may be necessary to travel back and forth in order to photograph the line, and it is possible to manage the traveling direction with respect to the road for images obtained by traveling back and forth.

  When actually providing image data to a user, it is necessary to cut out necessary data from the entire image data photographed over a wide area. As a cutting condition, it is necessary to be able to select the following.

・ Geospatial segmentation (geographic mesh or administrative region segmentation)
・ Cut out by link type (road type, road width, etc.)
-Cutting out images according to shooting time Further, when managing a large number of images, it is necessary to thin out the images halfway or reduce the image size in order to reduce the image capacity. Even when image data is physically converted, consistency is maintained as an image database.

(Issues of image data display)
In searching for images, a large-scale database is searched from a large number of image files, so a high-speed search method is required. In order to be able to use it in combination with existing geographic information, for example, when an arbitrary point (for example, a building) is selected, the surrounding area is searched, the azimuth is calculated based on the traveling direction of the image, and the target It is necessary to be able to display the building.

  Furthermore, as an advanced image search method, it is necessary to perform not only search by location but also semantic search of an image, that is, search based on the contents of an object being captured.

The image data management system with geographic coordinates of the present invention fixes an image sensor to a moving means, and collects image data taken by the image sensor at regular intervals along with the movement of the moving means based on a route network of a map. In a map information association image data management system for managing this in association with the route network,
A graph structure database that stores the route network having a link structure, a camera information database that records information of the image sensor, information on the shooting plan of the route network by the moving means, and an actual shooting that actually takes the route network A plan / actual shooting information database for recording information, a collection image / attribute for recording the peripheral image data of the fixed time every day taken by the image sensor and the shooting position coordinates of the image together with the coordinate measurement time as the collected image attribute information A storage device including an information database; a link start / end point passage time registration unit that records a passage time of a start point and an end point of the map link from the photographing time and the coordinate measurement time; and the predetermined time with reference to a fixed travel distance The predetermined travel distance from a plurality of surrounding image data collected every time An image data editing subsystem including an image thinning means for extracting only images for each image and thinning out other image data to reduce the data capacity, and each information in each database for each route network in the shooting time and each information When the management means provided with the management information associated with each assigned ID code, the route network ID and the shooting time are input, the management information including the ID and the shooting time is allocated and allocated. The gist of the present invention is that it includes an image data management subsystem including data extraction means for outputting each information of each database associated with management information from the storage device.

  According to the present invention, a consistent shooting plan and image collection / management can be performed in order to plan a wide range of shooting plans, collect a large amount of images, and manage image data in association with geographic information. it can.

  In addition, by associating map information such as roads with images and using images as geographical attributes that represent places, it is possible to visually convey the actual local situation to the user.

  Furthermore, since a plurality of images having different seasons and time zones can be managed in association with one piece of map information, it is possible to provide an image closest to the time and time conditions required by the user.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic.

  As shown in FIG. 1, an image data management system 10 associated with geographic coordinates according to an embodiment of the present invention includes an image capturing subsystem 20, an image data editing subsystem 30, an image data management subsystem 40, and an image data management subsystem. The storage device 50 is connected to the system 40.

  First, the image capturing subsystem 20 is connected to the image sensor 22, and an image data collecting unit 24 that collects an image of the surrounding environment along the map link as the collected image data d1 together with the photographing time, and the hybrid position sensor 23. , And a moving means equipped with a position data collecting means 25 for collecting the photographing position coordinates of the image as the collected image attribute information d2 together with the coordinate measurement time.

  Here, the map link means a road between two points (from one intersection to the next intersection) having the graph structure shown in FIG. For convenience, one point is referred to as a start point and the other point is referred to as an end point, but this does not mean the directionality of the link from the start point to the end point.

  The aforementioned graph structure is also referred to as a road network, and is a network structure in which an intersection is a node and a continuous line segment indicating a road center line between intersections as shown in FIG. Including width.

  In FIG. 17, in addition to the T-shape or cross, the intersection includes a Y-shaped path and a 5-way difference.

  The moving means refers to an aircraft for flying along a map link and shooting from above, or a vehicle 21 traveling on a road of the map link.

  Subsequently, the image data editing subsystem 30 includes a map data display unit 31 that displays a map link, an image data display unit 32 that displays image data collected by the image data collection unit 24, a photographing time and a coordinate measurement time ( Link start and end point passage time registration means 33 for recording the start time and end point of the map link from GPS time), image thinning means 34 for thinning out the data from the image data at regular intervals, and actual photographing management information registration means 35; including.

  Further, the image data management subsystem 40 is a photographing plan means 41 for managing a photographing plan by the image photographing subsystem 20, and a photographing that captures the collected image data d1 and the collected image attribute information d2 edited by the image data editing subsystem 30. Information capturing means 42, data extracting means 44 for extracting information that meets the specified conditions from information recorded in the storage device 50, registration of information in the storage device 50, and deletion of information from the storage device 50 And management means 43 for printing information in the storage device 50.

  The storage device 50 includes a graph structure database 51 that records map link information, a camera information database 52 that records image sensor 22 information, a plan / actual shooting information database 53 that records shooting plan information, and shooting information. A collection image database 55 and a collected image attribute information database 56 for recording the collected image data d1 and collected image attribute information d2 captured by the capturing unit 42, and a link for recording management information for associating information of each database for each map link Related information database 54.

  Data exchange between the image capturing subsystem 20, the image data editing subsystem 30, and the image data management subsystem 40 is performed via a portable storage medium such as a disk. The data communication means can be provided in the wireless communication using wired or wireless data communication.

(Image capture subsystem 20)
Next, each unit of the image capturing subsystem 20 will be described in detail. Here, the vehicle 21 will be described as an example of the moving means, but is not limited to the vehicle 21.

  As shown in FIG. 12, the image capturing subsystem 20 is a subsystem intended to capture an image along a map link used in the embodiment of the present invention.

  First, the image data collection means 24 uses one or a plurality of synchronized cameras attached to the vehicle 21 as the image sensor 22 to capture an image while traveling on a road that is a map link. At this time, in order to synchronize with the position information described below, the time transmitted from the global radio positioning system (GPS) detected by the hybrid position sensor 23 is assigned to each image frame obtained in time series. Then, it is recorded in the temporary memory 26 as the collected image data d1.

  The time is assigned to each image frame in units of 1/1000 second using the detected GPS time and the clock of the computer on which the image capturing subsystem 20 is mounted.

  Next, the position data collection means 25 uses the hybrid position sensor 23 combined with GPS and other sensors (gyro / vehicle speed sensor) attached to the same vehicle 21 to obtain position information (coordinates) obtained from the GPS, Time, vehicle speed information obtained from a vehicle speed sensor, acceleration information and direction information obtained from a gyro are collected. Further, the GPS time is given to these pieces of information and recorded in the temporary memory 26 as the collected image attribute information d2.

  When the vehicle speed obtained from the hybrid type position sensor 23 is 0, the position data collecting unit 25 is provided with a process for instructing the image data collecting unit 24 not to collect image data to perform shooting during stoppage. You can also quit. As a result, when a stop state occurs due to a signal or traffic jam, the recording of image data can be temporarily stopped to save the data collection capacity.

  As shown in FIG. 16, the collected image data d1 and the collected image attribute information d2 that are collected are given time information using a common GPS time. Can be associated.

  In addition, if you want to perform measurement later using the captured image, for example, when performing stereo image measurement or simple measurement using a single photograph, the actual captured camera information, that is, the shooting height of the camera or the identification of the camera itself It is necessary that the calibration information of the camera with the number and each identification number is managed together with the image and can be referred to. When capturing a large amount of images over a wide area, not only a single camera system but also a plurality of camera systems may be used. Therefore, it is necessary to centrally manage camera information used for capturing.

  For this reason, the image data collection means 24 stores simultaneously information on which camera was used for shooting, which is included in the shooting plan information d0.

  The camera orientation can be obtained from the relative orientation of the camera relative to the vehicle 21 and the orientation of the vehicle 21. The relative orientation is measured in advance. Since the image data collection cycle is usually shorter, the position / velocity / orientation, etc. in each image frame can be obtained by interpolating the position / velocity / orientation at the previous and subsequent times. That is, d1 includes the camera orientation.

  The collected image data d1 and the collected image attribute information d2 recorded in the temporary memory 26 are output to a portable storage medium such as a disk as a primary shooting information import file f1 and input to the next image data editing subsystem 30. The

  The position data collection unit 25 includes a link shape included in the shooting plan information d0 output to the portable storage medium as a shooting information export file f3 described later by the shooting planning unit 41 of the image data management subsystem 40 described later. The information is read, and the position information obtained from the hybrid position sensor 23 is superimposed on the computer screen. Thereby, it can be understood which map link of the collection target the operator has collected.

  That is, the position data collection means 25 captures the shooting plan information d0 (road network: link ID, link coordinates, node coordinates..., Shooting plan link ID, road width,... Stored in the shooting information export file f3.・), And based on this information, the shooting links are displayed in different colors on the computer screen. In other words, you can know which link to shoot. If the shooting plan information d0 includes a shot (actual shooting key ID), the link can be displayed in a different color so that the shooting is not performed.

(Image data editing subsystem 30)
Next, each unit of the image data editing subsystem 30 will be described in detail.

  As shown in FIG. 3, the image data editing subsystem 30 uses the collected image data d2 collected by the image photographing subsystem 20 and output as the primary photographing information import file f1 in the embodiment of the present invention. The map information such as a road network is used for map information having a graph structure using the shape information of the link information that is edited and created by the shooting plan means 41 of the image data management subsystem 40 described later and output as the shooting information export file f3. It is a subsystem that aims to correlate and optimize.

  First, as shown in FIG. 4, the map data display means 31 has the same structure as a physical image database, which will be described later, as a primary image information import file f1 output from the image capturing subsystem 20 as input information. The link shape is displayed from the time-series position information included in the collected image attribute information d2 constituting the primary information import file f1 and the map information included in the imaging information export file f3. The time series position information is displayed as a point sequence, and the link shape is displayed as a continuous line segment.

  Next, the image data display means 32 displays the collected image data d1 of the primary imaging information import file f1 as shown in FIG. When the image is displayed, the image collection position is displayed as a symbol (such as ▼ symbol) in the map displayed by the map data display means 31. Further, the display of image data can be sequentially switched back and forth in time series.

  Next, the link start / end point passage time registration unit 33 sets the start point and end point of the link selected by the mouse on the map shown in FIG. Specify and register the corresponding transit time. The passing time is specified from the time of the time series position corresponding to the start point or the end point using the time series position information included in the collected image attribute information d2 of the primary shooting information import file f1, and the primary shooting information import file. There is a method of designating from the image capturing time given to the collected image data d1 of f1. The start time and end time of the link are in the same format as the shooting plan export file, together with information such as the shooting date and time, the camera information key ID used for shooting, and the physical storage location (such as the hard disk volume name) of the image to be stored. Is saved in the actual photographing result information d3.

  As a method of designating from the time series position, there are a manual designation method for designating a time series position corresponding to the start point or end point of the link with a mouse or the like, and an automatic designation method for automatically designating.

  The automatic designation method is achieved by searching for the time series position closest to the start point or end point of the link. A map matching technique used in an automobile navigation system, that is, a technique of automatically associating with a route along which the user is traveling in consideration of not only the position but also the traveling direction and speed may be used. In this case, the closest point is not necessarily selected.

  In the method of designating from the image shooting time, the collected image data d1 corresponding to the start time of the link is displayed on the image display screen. By looking at the roads and intersections captured in the image, if it is determined that the operator is misaligned at the intersection, etc., the previous or next image is displayed in time series by operating the time series screen. The photographing time of the image at the intersection position determined by the operator can be corrected and registered as the link start point or end point passage time.

  If the start point passage time of the link registered by the link start / end point passage time registration means 33 is earlier than the end point passage time, it is determined that the image was taken while traveling from the start point to the end point on the corresponding link. . Conversely, if the start point passage time is later than the end point passage time, it is determined that the image is taken while traveling from the end point toward the start point on the corresponding link.

  The link start and end passage times registered by the link start / end point passage time registration means 33 are output to the secondary imaging information import file f2.

  Finally, the image thinning unit 34 thins the collected image data d1 of the primary photographing information import file f1 according to the movement distance or the like for the purpose of reducing the data capacity. As an example of the thinning method, there is the following method as shown in FIG. Starting from one frame, the distance from the starting frame to the i-th frame is calculated for each frame by the following formula.

[Equation 1]
L [0] = 0.0;
L [i] = L [i-1] + (v [i-1] + v [i]) / 2 * (t [i] -t [i-1]);
Where v [i]: speed of i-th frame (meter / second)
t [i]: Time of the i-th frame (second)
Let d be the thinning interval, and let L [jn] be the distance closest to n times d (n is an integer). Frames other than jn are thinned out. If the time information is also attached to the image frame after thinning, the image can be thinned without affecting other parts of the database.

  For example, when shooting at 15 frames per second from a car traveling at 10 km / h, 54,000 images are acquired in a 10 km section. If this is thinned out every 1 m, it becomes 10,000 sheets, and the data amount can be pushed to 1/5 or less.

  In this way, the amount of data can be reduced by thinning out the number of image frames to such an extent that there is no human visual problem, and the amount of memory necessary for storage can be reduced, so that the system can be made inexpensive. Furthermore, if the amount of data to be subjected to image display processing is small, the display processing speed can be improved and the burden on a display device such as a computer can be reduced.

  In the case where it is not necessary to reduce the amount of data due to the low cost of the memory or the improvement of the processing capacity of the computer, the image thinning means 34 is not particularly required.

  The actual shooting management information registration means 35 inputs attribute information (shooting area information / worker, etc.) at the time of actual shooting registered in a plan / real shooting information database, which will be described later, and stores it in the actual shooting management information d4.

(Image data management subsystem 40)
Next, each unit of the image data management subsystem 40 will be described in detail.

  As shown in FIG. 7, the image data management subsystem 40 manages the collected image data d1 actually shot and edited from the plan / progress management of the place to be shot and the equipment used for shooting, and the like. This subsystem is intended to collectively manage a data group necessary for extracting and providing the collected image data d1 that meets the user's requirements.

  First, as shown in FIG. 8, the shooting plan means 41 includes a screen display unit 411, a link selection unit 412, a camera information input unit 414, a shooting plan information input unit 413, and a shooting information export file creation unit 415. .

  The screen display unit 411 displays the nodes and links of the graph structure database 51 recorded in the storage device 50 on the computer screen.

  The link selection unit 412 selects and determines a link for which shooting is planned from among the links displayed by the screen display unit 411. Link selection methods include rectangular area specification by mouse operation on the screen, selection by position such as polygon area specification, selection by attribute information included in graph structure information such as road width and type, and composite selection by position and attribute information is there.

  For example, when a prefectural road of a certain municipality is extracted, only the prefectural road is extracted in advance by selection based on attribute information, and then the municipality boundary is designated by polygon designation, thereby achieving the purpose.

  The camera information input unit 414 records the input camera information used for shooting in the storage device 50 as the camera information database 52. At this time, a camera information key ID that is automatically or arbitrarily designated is assigned so that individual camera information can be identified.

  The shooting plan information input unit 413 receives input shooting plan information such as a shooting area (a group of links selected by the link selection unit 412), a shooting start date, a shooting operator, and a camera information key ID of a camera to be used. The plan / actual shooting information database 53 is recorded in the storage device 50. At this time, an imaging plan key ID designated automatically or arbitrarily is assigned so that each imaging plan can be identified.

  When the shooting plan is automatically generated, a combination of the creation date and camera information key ID is used so that IDs do not overlap.

  The shooting information export file creation unit 415 combines the link selected by the link selection unit 412 and the information input by the shooting plan information input unit 413, and links in the storage device 50 as link related information in the format shown in FIG. Record in the related information database 54. At this stage, the link related information and the link shape recorded in the graph structure database are further output to the portable storage medium as a photographing information export file f3.

  Next, as shown in FIG. 9, the photographing information capturing unit 42 includes an actual photographing management information recording unit 421 and an image data recording unit 422.

  The actual photographing management information recording unit 421 gives the actual photographing key ID to the actual photographing management information d4 such as a photographer at the time of photographing, which is input from the image data editing subsystem 30 as the secondary photographing information import file f2. Record in the plan / actual imaging information database 53.

  The actual shooting ID is a combination of the shooting date / time and the camera information key ID used so that the IDs do not overlap.

  Further, the shooting information related to the shot link, that is, the shooting date and time, the camera information key ID used for shooting, the passage time of the start and end points of the link, and the physical storage location (such as the volume name of the hard disk) of the stored image, etc. The actual photographing result information d3 storing the information is updated to the corresponding link key ID portion of the link related information database 54 created and stored for each link key ID by the photographing information export file creation unit 415.

  In addition, the image data recording unit 422 receives the collected image data d1 and the collected image attribute information d2 input from the image data editing subsystem 30 as the secondary shooting information import file f2, and the collected image database 55 and the collected image attribute information database, respectively. 56.

  As shown in FIG. 10, the management unit 43 includes a network registration unit 431, a printing unit 432, and a maintenance unit 433.

  The network registration unit 431 reads road network data created and distributed externally and registers it in the graph structure database 51.

  The printing unit 432 prints information of each database recorded in the storage device 50 with a printer.

  The maintenance unit 433 periodically backs up each database recorded in the storage device 50 and deletes unnecessary data (such as shooting plan information that is no longer shot). In addition, by searching for the work progress code in the link related information database 54 in which the shooting plan information d0 and the actual shooting management information d3 are reflected, the degree of shooting progress is totaled.

  Finally, as shown in FIG. 11, the data extraction unit 44 includes an extraction condition input unit 441, a graph structure information extraction unit 442, a link related information extraction unit 443, a collected image data extraction unit 444, and a collected image attribute information extraction unit. 554.

  The extraction condition input unit 441 inputs conditions for extraction from the database. The conditions include a positional condition, a link attribute condition, and a shooting attribute condition. The positional condition includes a rectangular area, a polygon area, and the like obtained by operating the mouse on the screen or reading a file.

  The link attribute condition includes designation of attribute information included in the graph structure information such as road width and type. The shooting attribute condition includes designation of a shooting date and a shooting time zone.

  The graph structure information extraction unit 442 extracts a link corresponding to the link ID extracted above from the graph structure database 51. Further, link information that matches the photographing attribute condition input by the extraction condition input unit 441 is extracted.

  The link related information extraction unit 443 extracts information that meets the conditions input by the extraction condition input unit 441 from the link related information database 54.

  The collected image data extraction unit 444 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image database 55.

  That is, the collected image data extraction unit 444 searches for an image photographed with the camera information key ID on the photographing date stored in each link related information extracted from the collected image database.

  Further, an image photographed between the start point passage time and the end point passage time is extracted from the image, cut out as an image file, and stored as an image file in the image data.

  Further, the image link ID, the node ID on the shooting start side, the node ID on the shooting vehicle side, and the shooting order are cut out and stored as image file attribute information in the image data.

  The collected image attribute information extraction unit 445 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image attribute information database 56.

That is, information such as image acquisition coordinates and image acquisition orientation corresponding to the image file extracted above is extracted and stored as image file attribute information in the cut-out image data.

  The information cut out as described above is output as cut-out image data f4. This includes the graph structure of the image file, image file attached information, and links.

  Note that the link can be cut out by the following method.

 -Geospatial designation (extraction by geographic mesh or administrative area): The link to be extracted can be selected by mouse designation on the computer screen interface or area designation with the mouse. Also, the inside of the outer polygon such as the administrative area may be registered in advance, and the link in the outer polygon corresponding to the area designated by the user may be extracted.

 For example, the Tokyo Metropolitan City / Municipal border is created separately as a polygon, saved as an external file on the magnetic disk, and when the polygon is read when necessary, the polygon is input with the mouse. It can be used as an alternative.

  -Specifying link attribute conditions (road type, road width, etc.): If you specify the type of link required for shooting and refer to the information stored in the graph structure database 51, it corresponds to the type of link Can be extracted.

  Specification of shooting attribute condition: By referring to shooting date / time information included in the link related information database 54, it is possible to specify and extract the shooting time of the image.

  On the other hand, an image corresponding to each link is extracted based on information described in the link related information database 54. That is, from the collected image database 55 recorded in the storage location of the storage device 50 described in the record corresponding to the link key ID of the link to be extracted, the camera described in the camera information key ID is photographed, and What is necessary is just to extract the time which is attached to the image frame between the start point passage time and the end point passage time.

  The cut out image data can include the following.

-Image file-Image file attribute information-Image acquisition coordinates (latitude and longitude of each frame)
-Image acquisition direction (image center direction)
・ Image acquisition link ID (ID link of the road centerline that captured the image)
・ Node ID on the shooting start side
・ Node ID at the end of shooting
・ Shooting order ・ Image acquisition camera information (shooting height etc.)
Link graph structure List of link IDs connected to each node ID (each database in the storage device 50)
Next, each database recorded in the storage device 50 will be described.

  As shown in FIG. 1, the database used in the embodiment of the present invention is roughly classified into an image information database 50a for managing the relationship between feature information having a graph structure and a photographed image, and an actual photographing. Divided into two groups, a physical image database 50b for storing the captured images and their associated information.

  The image information database 50a stores a graph structure database 51 for storing a graph structure, a camera information database 52 for storing information attached to shooting such as a camera used for shooting, a shooting plan, and information at the time of actual shooting. The plan / actual shooting information database 53 corresponding to each link and the link related information database 54 for associating the above information and the shot image for each link belong.

  The graph structure database 51 stores feature information having a graph structure as shown in FIG. When a road network is used as geographic information, a road center line connecting the intersection as a node of the graph structure and the intersection as a link corresponds.

In the graph structure database 51, as shown in FIG. 12, each node is defined by geographic coordinates and identified by a node key ID. Each link is defined by the geographical coordinates (or node key ID) at both ends and the link shape, and is identified by the link key ID. Note that both ends of the link (road center line) are connected to nodes (intersections), and a node key ID indicating the intersection of the start point and end point of the link is assigned to each link. As a result, the connection relationship between the node and the link, that is, the network structure becomes clear.
That is, the graph structure database 51 stores a graph structure such as complex road network data including Y-shaped roads, intersections of five or more roads, and three-dimensional intersections in association with map information. In FIG. 12, link information and node information are associated, and when there are a plurality of nodes, a plurality of node information is also associated.

  In the graph structure database 51, attached information can be attached to nodes and links. For example, in the case of a road network, it is possible to give information such as an intersection name if it is an intersection, and information on road width and regulation information (such as one-way traffic) if it is a link.

  A network having a graph structure includes a road, a track, a waterway, a channel, a pipeline, and the like, and is also referred to as a route network.

  Next, as shown in FIG. 13, the camera information database 52 stores device information such as a camera to be photographed, a mounted vehicle, and a camera height, and is identified by a camera information key ID.

  When capturing a large amount of images over a wide area, not only a single camera system but also a plurality of camera systems may be used. Therefore, it is necessary to centrally manage camera information used for capturing.

  For this reason, in the camera information database 52, the camera information actually shot, that is, the shooting height of the camera, the identification number of the camera itself, and the calibration information of the camera of each identification number can be managed and referenced together with the image. (See FIG. 13).

  In the plan / actual shooting information database 53, shooting plan information d0 and actual shooting management information corresponding to each link are recorded as shown in FIG. That is, the operator, the camera information key ID of the camera used for the work, and the date of image capture. Each shooting plan and actual shooting management information are identified by a shooting plan key ID and an actual shooting key ID, respectively.

  Since the amount of image data that is captured and collected by road driving or the like becomes large, it is necessary to cooperate with a photographing system. In particular, it is important to be able to obtain information such as which districts have not been photographed and how much photography has progressed. In addition, when a large amount of wide-area images are collected, it is necessary to perform photographing work in parallel using a plurality of vehicles. At this time, it is necessary to create a shooting plan in which necessary portions are cut out from the graph structure data such as roads and collect images according to the shooting plan.

  For this reason, in order to plan a wide range of shooting plans in advance, collect a large amount of images, and manage image data in association with geographic information, a system for consistent shooting plans and image collection and management is required. is there.

  In other words, if video information is collected in a wide range and a large amount of data in advance, it is possible to construct a business model that can be quickly provided according to the range and position designated by the user. Also, when a new image is taken, if it is managed in the same way as already collected data, it can be easily reused upon request.

  If this plan requires quickness, it was not possible to provide data from the time the request was received until the end of shooting, but it is only necessary to shoot only the necessary parts, so the user's request in a short time I can answer.

  As shown in FIG. 15, the link related information database 54 includes a link key ID, a shooting plan key ID, an actual shooting key ID, a work progress code indicating a shooting status (planned, shot, edited, not shot, etc.), It has a table structure having a series of information such as a shooting date, a camera information key ID used for shooting, a passage start time of a link start point and an end point, and a physical storage location (such as a volume name of a hard disk) of an image to be stored.

  If there is only one type of image that can be managed in association with map information at one location, it is necessary to construct a database for each type, as it is possible to manage multiple types of images in association with the same road location. There is. For example, depending on the object to be shown, the season and time zone when the image was taken may be important. For example, an image taken in each season of spring, summer, autumn, and winter may be stored in a sightseeing route.

  Even in such a case, as shown in FIG. 15, since a plurality of actual photographing key IDs are associated with one link key ID, the above-described case can be solved.

  In other words, there is a spatio-temporal geographic information system as a system for managing images for each multi-period and season, which displays a version in a certain time section. For this reason, for example, it is possible to view only images of a specific season (for example, autumn) from images taken over a plurality of years.

  In addition, for a wide road, it may be necessary to travel back and forth in order to photograph the line, and it becomes possible to manage the traveling direction with respect to the road for an image obtained by traveling back and forth.

  After the shooting plan, a record in which only items not related to actual shooting, such as a link key ID, a shooting plan key ID, and a camera information key ID scheduled to be used, are set is recorded for a link scheduled to be shot. The At this time, the work progress code is not photographed.

  After shooting, the actual shooting key ID, shooting date and time, and camera information key ID used for shooting are stored, and the work progress code is updated.

  That is, the link related information database 54 is a table that can have a plurality of records having the same link key ID, as indicated by the hatched portion in FIG. For this reason, it is possible to store a plurality of photographing results for one link key ID.

  As shown in FIG. 16, the collected image database 55 and the collected image attribute information database 56 belong to the physical image database.

  The collected image database 55 records a series of images actually taken in time series, as well as shooting camera information (camera information key ID) used for collecting each image, shooting date, and shooting time. .

  The collected image attribute information database 56 stores information such as a shooting position, a shooting speed, a shooting direction, a shooting time (GPS time), and a shooting camera.

  Since the information in the collected image database 55 and the information in the collected image attribute information database 56 each have time information, they are associated with each other according to the time information.

  In the embodiment of the present invention, only one storage device 50 for storing the database is illustrated, but the storage device 50 is not limited to one. The physical image database 50b may be stored in a storage medium physically different from the image information database 50a in order to store a large amount of image data. Actually, it is divided and stored in a large-capacity hard disk of 100 gigabytes or more.

  In the database structure according to the embodiment of the present invention, images of a plurality of periods can be simultaneously stored for one link.

  Next, the procedure in the image data management system with a geographic information association according to the embodiment of the present invention will be described with reference to the flowcharts of FIGS.

(Data registration)
Step S100: The link information from the external road network data f0 is registered in the graph structure database 51 by the management means 43 of the image data management subsystem 40.

(Shooting plan)
Step S101: The graph structure of the link is displayed from the link information of the graph structure database 51 by the screen display unit 411 of the photographing plan means 41 of the image data management subsystem 40.

  Step S102: The link selection unit 412 of the photographing plan means 41 of the image data management subsystem 40 selects a link to be photographed from the displayed link graph structure.

  Step S103: Information relating to the camera used for shooting is registered in the camera information database 52 by the camera information input unit 414 of the shooting plan means 41 of the image data management subsystem 40.

  Step S104: The shooting plan information input unit 413 of the shooting plan means 41 of the image data management subsystem 40 draws a shooting plan for the extracted shooting target link and inputs it to the plan / actual shooting information database 53.

  Step S105: The shooting plan information d0 is output to the shooting information export file f3 by the shooting information export file creation unit 415 of the shooting plan means 41 of the image data management subsystem 40.

(Shooting)
Step S106: The link shape in the information of the shooting information export file f3 is read into the image shooting subsystem 20, and the link to be shot is displayed.

  Step S107: The image is taken while actually traveling on the road of the link by the moving means (vehicle 21), and the collected image data d1 to which the photographing time is given is collected by the image data collecting means 24 of the image photographing subsystem 20. .

  Step S108: Collected image attribute information d2 to which the GPS time is added is collected by the position data collecting means 25 of the image capturing subsystem 20.

  Step S109: The image capturing subsystem 20 outputs the collected image data d1 and the collected image attribute information d2 together with the actual shooting result information to the primary shooting information import file f1.

(Image data editing)
Step S110: The photographing information export file f3 and the primary photographing information import file f1 are input to the image data editing subsystem 30.

  Step S111: Register actual photographing management information.

  Step S112: Map data is displayed by the map data display means 31 of the image data editing subsystem 30.

  Step S113: The image data display means 32 of the image data editing subsystem 30 displays the images collected for the link specified on the map data.

  Step S114: Time information is obtained from the displayed collected image data d1 and the collected image attribute information d2, and the link start and end point passage time registration means 33 of the image data editing subsystem 30 passes the start and end points of the corresponding link. Record the time.

  Step S115: Further, the image thinning means 34 of the image data editing subsystem 30 thins out the image from the collected image data d1 as necessary to reduce the data amount.

  Step S116: The edited information is output to the secondary shooting information import file f2.

(Information recording / update)
Step S117: The information of the secondary shooting information import file f2 is input to the image data management subsystem 40.

  Step S118: The actual photographing information data is recorded in the plan / actual photographing information database 53 by the photographing information capturing means 42 of the image data management subsystem 40.

  Step S119: The information in the link related information database 54 is updated by the photographing information capturing means 42 of the image data management subsystem 40.

(Data extraction)
Step S120: The data extraction means 44 of the image data management subsystem 40 extracts data from the collected image database 55 based on the information in the link related information database 54 in accordance with the user's request conditions. Output as f4.

(display)
Step S121: The image data display system 100 with relation to geographic information displays a peripheral image of a link (road) designated as a map by using the cut-out image data f4 and commercially available map data.

  In order to display the cutout image data f4, the image data display system with geographic information association 100, as shown in FIG. 1, includes a map display means 101, a photographing position display means 102, a photographed image display means 103, a route search interlocking means. 104, the image search means 105, and the memory | storage device 106 which stores the map data 108 and the cut-out image data f4 are preferable.

  The above-described problem is solved by the above-described image data management system with an associated geographic information according to the embodiment of the present invention.

  In other words, by cooperating with the imaging subsystem, it is possible to manage information such as which district has not been imaged and how much imaging has progressed.

  In addition, since it is possible to create a shooting plan that cuts out the necessary parts from the graph structure data such as roads, it is possible to perform shooting operations in parallel using a plurality of vehicles based on the shooting plan. Large quantities can be collected.

  Camera information (camera shooting height, camera identification number, camera calibration information for each identification number) used for shooting as a camera information database is centrally managed. Measurement can be performed.

  The data structure of the link related information database manages the actually captured images in association with map information having a graph structure such as complex road network data such as Y-junctions, intersections of five or more differences, and three-dimensional intersections. be able to.

  In addition, since a plurality of types of images can be managed in association with the same link, images taken in different seasons and time zones can be provided according to the user's request.

  Furthermore, when actually providing image data to a user, necessary data can be extracted from the entire image data photographed over a wide area by specifying the following conditions.

・ Geospatial segmentation (segmentation by geographic mesh or administrative area)
・ Cut out by link type (road type, road width, etc.)
-Extraction of image according to photographing time The means of each subsystem and the arrangement of data held in the drawings for explaining the embodiment of the present invention are merely examples, and are for realizing the functions of the present invention. It is not limited to. Therefore, it goes without saying that various combinations of arrangements of processing means are conceivable within a range in which the functions of the present invention can be realized.

  The description of this embodiment will be supplemented below.

  The description is supplemented for the image data management subsystem 40 that manages data in a consistent manner and reduces the amount of data and manages a panoramic image at a shooting position even in a general personal computer.

  In this description, the description is supplemented with respect to the primary shooting information import file f1, the secondary shooting information import file f2, the cut-out image data f4, and the like obtained by the image shooting subsystem 20 and the image data editing subsystem 30.

  When actually providing image data to a user, it is necessary to cut out necessary data from the entire image data photographed over a wide area. As a cutting condition, it is necessary to be able to select the following.

・ Geospatial segmentation (geographic mesh or administrative region segmentation)
・ Cut out by link type (road type, road width, etc.)
-Cutting out images according to shooting time Further, when managing a large number of images, it is necessary to thin out the images halfway or reduce the image size in order to reduce the image capacity. Even when image data is physically converted, it is necessary to maintain consistency as an image database.

  As shown in FIG. 1, the shooting plan generated by the shooting plan means 41 is stored in the primary shooting information import file f1.

  If video information is collected in a wide range and a large amount of data in advance, it is possible to construct a business model that can be quickly provided according to the range and position designated by the user. Also, when a new image is taken, if it is managed in the same way as already collected data, it can be easily reused upon request.

  Furthermore, if it is possible to use not only a single shooting of the same part but also a plurality of times, it is possible to provide an image in a state desired by the user, for example, an image specifying a season or time. Become. For example, if images with different shooting years can be used, it is possible to investigate secular changes in a specific district. In addition, it is possible to view the cityscape, sightseeing road, etc. in a state that the user wants to see, such as “I want to see the image at the time of cherry blossoms” or “I want to see the image at the time of autumn leaves”.

  In the case of wide roads, the buildings on the side close to the driving route appear well, but the buildings on the opposite lane may not appear well. In other words, it is necessary to perform reciprocal shooting in order to better express the image along the railway. Even in the case of reciprocal shooting, it may be managed in the same manner as in the case of the multi-period shooting as well as the traveling direction.

  However, when images are taken at fixed time intervals while traveling, a large amount of images are acquired, and as a result, a large-capacity database is required.

  In addition, when there is a request to quickly obtain an image viewed from a road on a map and display it in conjunction with the map, it is necessary to quickly acquire the image even in a wide area. In such a case, shooting may be performed with a large number of vehicles. Therefore, a shooting plan is made so as not to shoot the same place.

(Issues for collecting image data)
In other words, since the amount of image data that is taken and collected by road driving or flying over the sky without planning is large, it is necessary to cooperate with the photographing system. In particular, it is important to be able to obtain information such as which districts have not been photographed and how much photography has progressed. In addition, when a large amount of wide-area images are collected, it is necessary to perform photographing work in parallel using a plurality of vehicles. For this reason, an imaging plan in which a necessary part is cut out from graph structure data such as a road is created, and images are collected according to the plan.

  In addition, when you want to perform measurement later using the captured image, for example, when performing stereo image measurement or simple measurement using a single photograph, the actual captured camera information, that is, the camera's shooting height or the camera's own identification It is necessary that the calibration information of the camera with the number and each identification number is managed together with the image and can be referred to.

  When shooting a large amount of images over a wide area, not only a single camera system but also a plurality of camera systems may be used. Therefore, it is necessary to centrally manage camera information used for shooting.

For this reason, the image data management subsystem 40 includes a management unit 43 and a shooting plan unit 41.

  As shown in FIG. 8, the shooting plan means 41 includes a screen display unit 411, a link selection unit 412, a shooting plan information input unit 413, a camera information input unit 414, a shooting information export file f3, and the like.

  The screen display unit 411 in FIG. 8 displays all the nodes and links of the graph structure (road) of the designated area recorded in the storage device 50 on the computer screen.

  The link selection unit 412 selects and determines a link for which shooting is planned from among the links displayed by the screen display unit 411. Link selection methods include rectangular area specification by mouse operation on the screen, selection by position such as polygon area specification, selection by attribute information included in graph structure information such as road width and type, and composite selection by position and attribute information is there.

  For example, when a prefectural road of a certain municipality is extracted, only the prefectural road is extracted in advance by selection based on attribute information, and then the municipality boundary is designated by polygon designation, thereby achieving the purpose.

  At this time, the camera information input unit 414 records the input camera information used for shooting in the storage device 50 as the camera information database 52. At this time, a camera information key ID that is automatically or arbitrarily designated is assigned so that individual camera information can be identified.

  The shooting plan information input unit 413 receives input shooting plan information such as a shooting area (a group of links selected by the link selection unit 412), a shooting start date, a shooting operator, and a camera information key ID of a camera to be used. The plan / actual shooting information database 53 is recorded in the storage device 50. At this time, an imaging plan key ID designated automatically or arbitrarily is assigned so that each imaging plan can be identified.

  When the shooting plan is automatically generated, a combination of the creation date and camera information key ID is used so that IDs do not overlap.

  On the other hand, links and nodes that have already been photographed are displayed in different colors on the computer screen.

  Therefore, it can be seen from the computer screen which link is not photographed or the photographing result.

  The shooting information export file creation unit 415 combines the link selected by the link selection unit 412 and the information input by the shooting plan information input unit 413 and records them in the link related information database 54 in the storage device 50 as link related information. To do. At this stage, the link related information and the link shape recorded in the graph structure database are further output to the portable storage medium as a photographing information export file f3.

  That is, the shooting plan information d0 (shooting plan key ID, shooting area information (target link key ID), planned work person, scheduled shooting camera information key ID, scheduled shooting date, etc.) shown in FIG. 14 is carried as the shooting information export file f3. Output to type storage medium. When shooting is finished, an actual shooting key ID and a shooting result are added. This photographing result indicates whether or not the photographing has been performed normally.

  On the other hand, as shown in FIG. 9, the photographing information capturing unit 42 includes an actual photographing management information recording unit 421 and an image data recording unit 422.

  The actual photographing management information recording unit 421 receives actual photographing management information d4 (actual photographing key ID, photographing district information (photographing area information () at the time of photographing) input from the image data editing subsystem 30 as the secondary photographing information import file f2. The target link key ID), the work implement, the shooting execution camera information key ID, the shooting execution date, the shooting result, and the like are assigned to the actual shooting key ID and recorded in the plan / real shooting information database 53.

  The aforementioned secondary import file f2 includes collected image data d1, collected image attribute information d2, actual shooting result information d3, actual shooting management information d4, and the like of each representative shooting point.

  Information stored in the secondary photographing information import file f2 is obtained by the image data editing subsystem 30.

  These files are used by the image display system. An outline of the image display system will be described.

  As a previous stage, the description is supplemented for the primary shooting information import file f1, the secondary shooting information import file f2, the cutout image data f4, and the like.

  The primary shooting information import file f1 includes image collection data d1 with time and collected image attribute information d2. The collected image attribute information d2 includes position information (coordinates) obtained from GPS, vehicle speed information obtained from a vehicle speed sensor, acceleration information and direction information obtained from a gyro, GPS time, and the like.

  The shooting position display means 102 reads the collected image attribute information d2 of the cut-out data f4, and this time-series information (image acquisition coordinates) is displayed on the network data as a symbol (for example, a circle: hereinafter shooting position) as shown in FIG. (Road) can be displayed.

  The road at this time uses the network data of the map data included in the photographing information export file f3.

  This will be described with reference to FIG. FIG. 23A shows the shooting position when the vehicle is shot at regular time intervals while being accelerated or slowed down.

FIG. 23 (b) shows that the collected image data d1 (with shooting time) captured at regular intervals stored in the primary shooting information import file f1 at this time are arranged on the distance axis (coordinates). .

  The map data display means 31 in FIG. 1 reads the image acquisition coordinates of the collected image attribute information d2 having the time of the collected image data d1, for example, and the image acquisition coordinates are defined in the distance axis (XY coordinate system). Allocated memory).

  At this time, the image thinning means 34 is involved.

  As shown in FIG. 23 (c), the image thinning means 34 sets a thinning interval d (distance: 1 m, for example) that does not cause human visual problems to the distance axis sequentially from the photographing start point, and this thinning interval. For each d, when a perpendicular is drawn with respect to the distance axis of FIG. 23B, it is determined whether or not the image acquisition coordinates (shooting position) of the collected image data d1 exist. As shown in FIG. 23 (d), coordinates (shooting positions) are assigned to the distance axis as representative shooting points (simply referred to as shooting positions), and collected images related to the assigned image acquisition coordinates (shooting positions). Only the data d1 and the collected image attribute information d2 are extracted and stored in the secondary shooting information import file f2.

  In this extraction, for example, time is extracted as a key. That is, only the collected image data d1 and the collected image attribute information d2 at the location of the thick line (representative photographing point) in FIG. 23B are extracted.

  On the other hand, when the representative photographing points are defined on the distance axis, the link start / end point passage time registration unit 33 obtains the actual photographing result information d3 having the time of the collected image attribute data d2 including the position information of these representative photographing points. It is obtained by extracting from the export file f3 and storing it in the secondary shooting information import file f2.

  As shown in FIG. 1, the image data management subsystem 40 includes shooting information capturing means 42. The shooting information capturing means 42 includes an actual shooting management information storage unit 421, an image data storage unit 422, and the like. The management information storage unit 421 allows the actual shooting management information d4 (shooting area information, worker information, etc.) having the time of the collected image attribute data d2 having the image acquisition coordinates (shooting position) of the representative shooting point, and each representative shooting point. Collected image data d1, collected image attribute information d2, actual photographing result information d3, and the like are captured and stored in the image information database.

  At this time, an actual shooting key ID is assigned to the actual shooting management information d3 and recorded in the plan / actual shooting information database 53.

  Further, the image data recording unit 422 records the collected image data d1 and the collected image attribute information d2 of the secondary shooting information import file f2 in the collected image database 55 and the collected image attribute information database 56, respectively. The actual photographing management information d4 is stored in the plan / actual photographing information database 53.

Further, as shown in FIG. 11, the data extraction means 44 shown in FIG. 1 includes an extraction condition input unit 441, a graph structure information extraction unit 442, a link related information extraction unit 443, a collected image data extraction unit 444, and a collected image attribute. An information extraction unit 554,
The extraction condition input unit 441 inputs conditions for extraction from the database. The conditions include a positional condition, a link attribute condition, and a shooting attribute condition. Positional conditions are also referred to as spatial clipping conditions. There are rectangular areas and polygonal areas obtained from mouse operations on the screen and file reading, and various information (image files, attribute information, etc.) in these areas are clipped images. It is called data.

  The link attribute condition includes designation of attribute information included in the graph structure information such as road width and type. The shooting attribute condition includes designation of a shooting date and a shooting time zone.

  The graph structure information extraction unit 442 extracts, from the graph structure database 51, a link that matches the positional condition and the link attribute condition input by the extraction condition input unit 441.

  Further, the extracted link ID and the node IDs at both ends thereof are checked, the link IDs connected to each node are listed, and stored as the graph structure information of the link of the cut-out image data.

  The link related information extraction unit 443 extracts the link corresponding to the link ID extracted above from the link related information database 54. Further, link information that matches the photographing attribute condition input by the extraction condition input unit 441 is extracted from the information.

  The collected image data extraction unit 444 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image database 55.

  The collected image data extraction unit 444 searches for an image taken with the camera information key ID on the shooting date stored in each link related information extracted from the collected image database. Further, an image photographed between the viewpoint passage time and the end point passage time is extracted from the image, cut out as an image file, and stored as an image file in the image data.

  Further, the image acquisition link ID, the node ID on the shooting start side, the node IDs on both sides of the shooting collection, and the shooting order are cut out and stored as image file attribute information in the image data.

  The collected image attribute information extraction unit 445 extracts information that meets the conditions input by the extraction condition input unit 441 from the collected image attribute information database 56.

  That is, information such as image acquisition coordinates and image acquisition orientation corresponding to the image file extracted above is extracted and stored as image file attribute information image file attributes in the cut-out image data.

  That is, the data extraction unit 44 extracts all the cutout image data f4 (graph structure information, link related information, image file, image file attribute information) corresponding to the representative shooting spot shown in FIG. 23D of the cutout condition (area: area). ..) Is stored in the storage device 106 (see FIG. 23F).

  As shown in FIG. 15, the link related information includes a link key ID, a shooting plan ID, an actual shooting key ID, a work progress code, a shooting date, a camera information key ID, a start point passing time, an end point passing time, a storage location, and the like. Consists of.

  For example, in FIG. 15 (you may display and select any record), the link key ID is “1” “2” “3” “3” “4” “5” “6”. This indicates that there are two types of images in one link (two “3”).

  Some records with the link key ID “3” have the actual photographing key “4”. This indicates that the shooting date and time are different.

  In addition, there is a record whose link key ID is “5” in which the actual photographing key ID column is blank. This indicates unphotographed.

  That is, by reading the table of the link related information database, it can be determined whether there is a link that has not been shot or there are a large number of images in the link (for example, according to the four seasons).

That is, in this embodiment, even if there are a large number of images with different shooting dates on the link, there is only one link key ID.

  As described below, the cut-out image data f4 is used to display a map and an image file by the image data display system 100 with a geographic information association.

  FIG. 24 is a block diagram showing a more specific configuration of the image data display system 100 with a geographic information association.

  As shown in FIG. 24, the image data display system 100 with a geographic information association includes a storage direction display means 107, a map, in addition to the storage device 106, the geographical display means 101, the photographing position display means 102, and the photographed image display means 103. An upper photographing direction display unit 110, an annotation data / registration display unit 109, and the like are provided.

  The storage device 106 stores note data related information f6 and the like in addition to the map data 108 and the cutout image data f4.

  The note data related information f6 is preferably registered in advance by the note data / registration display means 109.

  The aforementioned map data 108 preferably has a hierarchical structure. For example, a hierarchical structure including a background map field, network information field, annotation data field, map symbol data field, and the like. The annotation data is the name of the feature, and this name is associated with the coordinates of the annotation data field.

  Further, the storage device 106 stores a plurality of pieces of cut-out image data f4 within the cut-out condition area. It is preferable that an ID code is given to these cut-out image data f4. More specifically, the data structure shown in FIG.

  Furthermore, the physical image database has a data configuration shown in FIG. 26 more specifically.

  When the region for the clipping condition is input, the geographic display unit 101 displays the background map, network information, note data, and map symbol in this region in an overlapping manner. At this time, only the note data in the vicinity of the road of the network data may be extracted and displayed.

  The annotation data / registration display means 109 has a function of displaying the annotation data on an image on a computer screen as shown in FIG.

  In order to realize the display of the annotation data, the annotation data / registration display means 109 searches for the image acquisition coordinates existing within a certain distance from the coordinates of the annotation data of the map data (for example, the coordinates of the center character of the annotation data). Among these image acquisition coordinates, an image acquisition coordinate closest to the note data is searched. Then, the ID of the cut-out image data having the image acquisition coordinates is registered in advance in the storage device 106 as the annotation data related information f6 together with the annotation data, the coordinates of the annotation data, and the like.

  For example, as shown in FIG. 28A, when the network data f0 is a road, the image acquisition coordinates exist as Ga, Gb... And the note data exist as Ca, Cb. The cut-out image data having the image acquisition coordinates at the closest distance to the coordinates of the annotation data Ca, Gb. Then, the direction with respect to the coordinates of the note data is obtained from the image acquisition coordinates of the retrieved cut-out image data. This orientation is indicated by how many times θk is from the camera center.

  For each piece of note data, Gb..., The ID of the cut-out image data having the closest image acquisition coordinates is made to correspond, and the azimuth θk and the image acquisition coordinates are made to correspond (FIG. 28B).

  The image of the image file linked to this ID is a 360-degree panoramic image, and the center position of the image corresponding to the camera center is the camera center (0 degree). The coordinates of the annotation data on the image are obtained at the position corresponding to the height of the line of sight in the direction of θk from the camera center. That is, a memory defining a field (XY coordinate system: corresponding to the coordinate system of the panoramic image) for displaying the note data is provided, and the coordinates of the note data on this memory (referred to as the note data display position on the image). ) Is defined.

  That is, the note data related information f6 includes the ID of the adjacent clipped image data, the note data display position, the direction (θk), the characters of the note data, and the like.

  The geographic information unit 101 reads out map data corresponding to the input region of the extraction condition from the storage device 106 and displays it on the computer screen. This map data includes a background map, network information (why necessary for map data), annotation data, map symbol data, and the like.

  The imaging position display means 102 extracts the image acquisition coordinates of all the cutout image data f4 stored in the storage device 106, and sets the image acquisition coordinates (imaging) at positions on the map (for example, roads) corresponding to the coordinates. (Position) is displayed (refer to FIG. 27).

  That is, since all the shooting position symbols existing in the region for the cutout condition are displayed, the operator can determine at a glance where the image is on the map.

  When the operator designates any of these photographing position symbols with a mouse or the like, the photographing position display means 102 displays the designated photographing position symbol with another symbol (for example, cross, double circle, different color). ).

  When the shooting position symbol on the map on the screen is designated, the on-map shooting direction display means 110 stores link information (link, intersection ID, coordinates, connection link ID, etc.) having coordinates corresponding to the shooting position symbol. It searches from the apparatus 106 and determines whether intersection ID is contained in the searched link information.

  When the intersection ID is included, it is determined as an intersection. When it is determined that the vehicle is an intersection, the connection link ID associated with the intersection ID is read, and the image acquisition orientation (azimuth in the central axis direction of the camera) associated with the image acquisition coordinates corresponding to the shooting position symbol is read.

  Then, the end point is read from the start point of the link of the connection link ID searched from the coordinates of the intersection, and the azimuth Hi (3 differences or 4 differences,...) Of all the links connected to the intersection is obtained. An arrow is displayed on the link (see FIG. 22).

  On the other hand, when a shooting position symbol on the computer screen is selected, the shot image display means 103 extracts an image file of cut-out image data having image acquisition coordinates of the shooting position symbol, and displays the operator input from this image file. Display the range image on the screen.

  The image file of the cutout image data described above is a 360-degree panoramic image. For this reason, for example, the operator designates a display range of about 50 degrees on the left and right with the photographing direction as the center.

  The image of the image file in the display range is displayed on the computer screen, and the display range and shooting direction are notified to the map shooting direction display means 110.

  When the shooting direction and the display range are notified, the on-map shooting direction display means 110 is an angle of the display range (50 degrees left and right from the shooting direction) centered on the shooting direction from the image acquisition coordinates (shooting position) on the map of the computer screen. Polygons are generated in the direction, and these polygons are overlaid on the computer screen image (see FIG. 22).

  Further, the traveling direction mark display means 107 obtains the direction of the connection link (road) included in the shooting range in the shooting direction from the intersection.

  The direction of this connecting link is defined by an angle. For example, as shown in FIG. 29A, when the shooting direction is the camera center (0 degree), the direction of the center line of the road ka from the camera center is θp, and the direction of the center line of the road kb is θi. Can be defined.

  The direction of θp and the direction of θi are determined from the center (0 degree) of the display range, and a triangular polygon (through) is displayed three-dimensionally at a position on the screen that corresponds to the position of the line of sight (FIG. 29 ( c)).

  When this polygon is selected, the captured image display means 103 sequentially extracts the image file of the shooting position of the road in the selected direction and displays it according to the display range.

  On the other hand, the note data registration / display unit 109 reads a specified note (XX apartment, XX hospital, XX city hall,...), And when it comes to the image acquisition coordinates (shooting position), a certain distance from this shooting position. The nearest note data is extracted from the map data. When the image acquisition coordinates are reached, the image of the image file is counted frame by frame (for example, 1 m per count) while the image scroll button or the playback button is pressed. The image file related to the acquired coordinates corresponding to this count value is assigned and displayed.

  Next, the extracted note data is displayed on an image on a computer screen.

  The display of the note data reserves the note data related information f6 having the image acquisition coordinates (shooting position) when the shooting position symbol on the road on the computer screen has moved to the shooting position. Then, the character string of the note data is displayed on the computer screen according to the note data display position and direction of the assigned note data related information f6 (see FIG. 27).

  Therefore, the target object can be found immediately by simply looking at the computer screen.

<Other embodiments>
In this embodiment, when a cityscape is photographed while traveling on a road, an image relating to privacy such as a face and a license plate is also captured.

  In the present embodiment, privacy-related images are provided as cut-out images f4 that cannot be discriminated.

  FIG. 30 is a schematic configuration diagram of another embodiment. In FIG. 30, the description of the same part as in FIG. 1 is omitted, and an image relating to privacy is described as a face.

  The present embodiment includes an image processing subsystem 60, which includes a face position rectangular range specifying unit 130, a face position rectangular range processing unit 131, and the like.

  That is, for the purpose of taking privacy into consideration, an image processing subsystem is provided that identifies a portion of the image where the face is reflected and processes the image so that the face cannot be identified.

  FIG. 31 is a detailed configuration diagram of the image processing subsystem 60. As shown in FIG. 31, the face position rectangular range specifying unit 130 includes an image reading unit 130a and a rectangular range extracting unit 130b.

  The face position rectangular range processing unit 131 includes an image reading unit 131a, a local filter process 131b, and an image storage unit 131c.

  The image reading means 130a of the face position rectangular range specifying means 131 reads each image file included in the cut-out image data f4 and specifies the position where the face is located. As a specific specifying method, an image file is displayed on a computer monitor, and an operator uses a manual processing method in which a position including a face is specified by a mouse click or the like within a rectangular range on the image, or an image is detected by face recognition software. Either or both of the automatic processing methods are used to process the file and automatically extract the rectangular area containing the face.

  The rectangular range is expressed by a combination of the upper left coordinate of the rectangular range and the rectangular size. When there are a plurality of face images, a plurality of rectangular ranges are also extracted by the rectangular range extracting unit 130b.

  Next, the face position rectangular range processing means 131 is a means for locally processing an image for the rectangular range 132 specified above. Specifically, each of the image files included in the cut-out image data f4 is read by the image reading unit 131a, a local filter process 131b such as a Gaussian filter is processed on the rectangular range 132, and the image storage unit 131c is processed. The processed result is stored in an image file included in the cut-out image data f4.

  The image data management subsystem may include an image processing subsystem that processes images for the purpose of considering privacy related to portrait rights such as human faces when distributing image data or distributing via the Internet. .

It is a block diagram of a map information correlation image data management system and a display system. It is a block diagram of an image photographing subsystem. It is a block diagram of an image data editing subsystem. It is a map displayed by a map data display means. The image data display means is an image displayed on the road surface. It is a figure which shows the method of the image thinning-out by an image thinning means. It is a block diagram of an image data management subsystem. It is a block diagram of a photography plan means. It is a block diagram of an imaging | photography information acquisition means. It is a block diagram of a management means. It is a block diagram of a data extraction means. It is a structure figure of a graph structure database. It is a structural diagram of a camera information database. It is a structure figure of a plan and real imaging | photography information database. It is a structural diagram of a link related information database. It is a structure figure of a physical image database. It is a figure which shows the feature with a graph structure. It is a flowchart explaining the procedure in the image data management system with a geographic information association by embodiment. It is a flowchart explaining the procedure in the image data management system with a geographic information association by embodiment. It is a flowchart explaining the procedure in the image data management system with a geographic information association by embodiment. It is a flowchart explaining the procedure in the image data management system with a geographic information association by embodiment. It is explanatory drawing which displays a photographing position and a corresponding image simultaneously. It is explanatory drawing of the thinning-out of this Embodiment, a display of an imaging | photography position, and registration of cut-out data. It is the block diagram which made more specific the image display system 100 with a geographic information relation. It is explanatory drawing explaining the more specific structure of clipped image data. It is explanatory drawing explaining the more specific structure of a physical database. It is explanatory drawing of the annotation overlay display of a map information correlation image display system. It is explanatory drawing explaining the display to the image of annotation data. It is explanatory drawing explaining a advancing direction display means. It is a schematic block diagram of the image data management system with a geographic information association of other embodiment. It is a detailed block diagram of an image processing subsystem.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Geographic coordinate related image data management system 20 Image capturing subsystem 21 Vehicle (moving means)
22 Image sensor 23 Hybrid type position sensor 24 Image data collection means 25 Position data collection means 30 Image data editing subsystem 31 Map data display means 32 Image data display means 33 Link start / end point passage time registration means 34 Image thinning means 40 Image data management Subsystem 41 Imaging plan means 42 Imaging information fetch means 43 Management means 44 Data extraction means 50 Storage device 50a Image information database 50b Physical image database 51 Graph structure database 52 Camera information database 53 Plan / actual imaging information database 54 Link related information database 55 Collected image database 56 Collected image attribute information database 100 Image data display system 101 with geographic information association Map display means 102 Shooting position display means 103 Photographed image display hand 104 route search interlocking means 105 image search means 106 storage device 108 map data d0 shooting plan information d1 collected image data d2 collected image attribute information d3 actual shooting result information f1 primary shooting information import file f2 secondary shooting information import file f3 shooting information export File f4 Cutout image data

Claims (6)

A map in which an image sensor is fixed to a moving means, and image data captured by the image sensor is collected at regular intervals along with the movement of the moving means based on a route network of the map, and this is associated with the route network and managed. In the information association image data management system,
A graph structure database that stores the route network having a link structure, a camera information database that records information of the image sensor, information on the shooting plan of the route network by the moving means, and an actual shooting that actually takes the route network A plan / actual shooting information database for recording information, a collection image / attribute for recording the peripheral image data of the fixed time every day taken by the image sensor and the shooting position coordinates of the image together with the coordinate measurement time as the collected image attribute information A storage device including an information database;
A link start / end point passage time registration means for recording the passage time of the start point and end point of the map link from the photographing time and the coordinate measurement time, and a plurality of surroundings collected at regular intervals based on a certain travel distance An image data editing subsystem including an image thinning means for extracting only the images for each predetermined travel distance from the image data and thinning out other image data to reduce the data capacity;
When a management means including management information in which each information of each database is associated with each shooting network by the shooting time and each ID code assigned to each information, and the ID of the route network and the shooting time are input An image data management subsystem including data extraction means for allocating the management information including the ID and photographing time and outputting each information of each database associated with the allocated management information from the storage device; A map information association image data management system characterized by comprising:   2. The image data editing subsystem according to claim 1, wherein, when there is no image data that matches the travel distance of the constant interval, the image data editing subsystem extracts image data that is closest to the travel distance of the constant interval. Map information association image data management system.   3. The map information association image data management according to claim 1, wherein when the map area is input, only the information in the area is cut out and output as image data. 4. system.   The map information association image data management system according to claim 3, wherein the image data management subsystem further includes a management unit including a network registration unit, a printing unit, a maintenance unit, and a work management unit.   5. The map information association image according to claim 3, wherein the shooting plan means includes a screen display processing unit, a link selection processing unit, a camera information input unit, a shooting plan information input unit, and a data registration unit. Data management system.   The map information association image data according to any one of claims 1 to 5, further comprising an image processing subsystem that identifies a specific image in the cut-out image data and performs blurring processing on the specific image. Management system.