JP4673799B2 - Image processing system - Google Patents

Description

  The present invention relates to an image processing system that handles large-capacity images using satellite images and aerial photographs. More specifically, the present invention relates to a large-capacity image database that is frequently used in a large-scale image search / viewing system and is frequently added and updated. The present invention relates to a system for efficiently adding and updating images.

  In recent years, the use of images taken from high altitudes such as satellite images and aerial photos has become widespread, and satellite images taken from various places are gathered in a database, and services that allow users to easily browse images of the area they want Is provided.

  Such an image providing service can be used in various ways such as searching for an address, understanding the surrounding land use status, and confirming a transportation service in cooperation with map information. In order to provide accurate information in these services, it is desirable to create a database of new images as much as possible, and frequent updating of the image database is required.

  In such image providing services, captured image data is rarely provided to users without processing, and image processing that can be superimposed on a map by removing distortion at the time of shooting is performed, or the original image is used Image data corrected to a brightness and color tone that is preferred by the user is used. In particular, when a satellite image is used, the original image is often bluish due to the influence of the atmospheric layer because the shooting location is altitude. In order to make this look natural (similar to the color that a building or ground can see from a normal viewpoint), it is necessary to greatly correct the color tone.

  The correction of brightness and color tone in the case of a general image is performed by examining the luminance and color distribution in the image and redistributing it into an ideal form while judging by an expert ( Non-patent documents 1 to 3).

Supervised by Hideo Kusaka Color Image Engineering Dai Nippon Printing 1997 Yasuyuki Hayakawa, PhotoShopCS Professional Course Color Correction, Peach Press, 2004 Yukiichi Sakai, Digital Image Processing Introduction, Corona, 1997

  However, in the case of a satellite image or the like obtained by photographing a wide range, it is unrealistic to perform the above general correction on all the images in the database because of high cost.

  Therefore, the following correction method is taken in the image database as described above. First, a part of a wide range of image databases is extracted as a representative area at a certain ratio, and ideal image correction is performed using only the representative area (creation of representative data). Thereafter, other portions are corrected in accordance with the same conditions as those used when creating the representative data, for example, the correspondence table for each pixel of the original image data and the representative data.

  In addition to such color tone correction, correction for mitigating the unnaturalness of joints that occurs when images of adjacent areas taken at different times are individually corrected is also required.

  In addition, regarding the large-scale image content as described above, it is not sufficient to collect the images in the service area once, and the information to be provided is updated, so that it is frequently updated. In particular, the update accompanied with the correction of the image requires a part of human labor and is troublesome, which causes an increase in the cost of the service itself.

  In addition, the image correction by the above-described conventional method results in a wide range of image correction results greatly depending on how to select a shooting area as representative data and the judgment of an expert when correcting the representative data.

  The feature is that the location selected for the representative data is corrected centering on that portion, so the location of the representative data and the location similar to the representative data are corrected most ideally. When the location selected as the representative data is biased, the correction result at a location that differs greatly from the representative data often becomes less than ideal. If an image with a non-ideal correction is provided to the user, the user may feel uncomfortable with the image, or the attractiveness of the shooting area may be impaired, leading to a decrease in service quality. is there.

  For this reason, the location of the representative data is preferably a location that includes the characteristics of the entire correction range with no bias in what is shown as much as possible. Therefore, the location of the representative data is usually selected randomly or by the operator. However, when the image database becomes large, there is a problem that it is difficult to maintain the quality of the corrected image because a large amount of representative data is selected and the check cost increases. There is.

  It is an object of the present invention to improve the satisfaction of a user who has viewed an image by performing image correction in a form desirable for the user while suppressing update costs.

  In order to solve the problem that the correction result of the image varies greatly depending on how to select the representative data shown in the above problem, and the non-ideal correction is applied, the image browsing service of the image providing service is analyzed with respect to the image providing service. The image is corrected by the following method.

  When correcting an image, the most frequently viewed part of the shooting area of the image is selected as representative data, and the entire image is corrected according to the correction. This makes it possible to correct the area most frequently seen by the user most ideally, and then to correct the area based on the area in which the user is interested (= there are many opportunities to see). The average quality of the corrected image can be improved.

  In addition, it is possible to update an object having higher information value with higher priority. It is also possible to add attribute information related to an image such as data such as a location where the image provided when the image is provided to the user and a usage method of the user to the viewing log to determine the representative region. In this way, by using the statistical information of the number of times images are viewed, priority can be given to the update of the area where the user's interest is gathered, and it is possible to suppress the update of the part with low browsing frequency. It is possible to realize image data that is updated more frequently as the object has a higher information value for the intended user.

  According to the image processing system of the present invention, it is possible to improve the average satisfaction level of the corrected image viewed by the user by correcting the image in consideration of an area where many users are interested. Further, it is possible to preferentially update an object having a high information value for the user while suppressing an update cost for making the image provided to the user fresher.

  Hereinafter, a satellite image providing service including an image processing system according to an embodiment of the present invention will be described with reference to the drawings. In the following, a correction relating to the color tone having the strongest impression (a color shade, brightness / darkness, strength, etc.) of the image correction will be described as an example. However, the type of correction is not limited to this. FIG. 1 is a diagram showing an example of the configuration of a system that includes the image processing system according to the present embodiment and that provides the latest satellite images and maps to member users via the Internet. The system shown in FIG. 1 includes an image providing center side information processing system and a user side Internet utilization system, which are communicably connected via a communication line network NT. The communication line network NT indicates, for example, a telephone line including ISDN, a dedicated line, a cable TV line, and the like.

  The user-side Internet utilization system shown in FIG. 1 includes a client PC (personal computer) 101 and a communication device 103 that connects the client PC 101 to a communication line network NT. In the client PC 101, a client program 102 is installed on a recording medium such as a hard disk. By executing the client program 102, an image provided from the image providing center can be viewed through the communication line network NT. In the present embodiment, the client-side processing device is described by taking a personal computer as an example. However, any other terminal that can use the Internet, for example, a mobile phone, a PHS, a PDA, a wearable computer, or the like Even equipment is applicable.

  The system on the image providing center side includes a data management server 104 and a service management server 109, a storage device 106 such as a disk array connected thereto, a communication device 111 for connecting the data management server 104 to the communication line network NT, It has. The data management server 104 stores the image update program 105 in a built-in recording medium (for example, ROM), and creates the image data 107 stored in the storage device 106 such as a hard disk by executing the image update program 105. ,Update.

  The service management server 109 includes an image providing service program 110 in an internal recording medium (ROM or the like), and by causing the computer to execute the program, the image in the image data 107 is displayed on the user side as necessary. Can be provided. Further, the contents browsed by the user can be recorded as browsing log data 108. The data management server 104 and the service management server 109 may be integrated or may be configured as separate devices.

  The system shown in FIG. 1 is operated by a user sending a browsing request to the service management server 109 through the browsing client program 102, and the service management server 109 distributes an image at a position that matches the browsing request. The image update program 105 prepares the latest image by updating the image data 107 as needed. The present embodiment is characterized in that a log of distribution data is stored at the time of image service operation, and an image preferred by the user is created more efficiently by using the log when updating the image data. Details will be described with reference to FIG. FIG. 2 is a diagram showing a flow of a satellite image providing service business using the satellite image processing apparatus. This satellite image providing service concludes a membership contract with the desired user (including individuals, corporations, organizations, etc.) at the image providing center, and the user who becomes a member designates the position of the image he / she wants to see through the Internet. The image providing center side distributes the satellite image at the corresponding position. At this time, the operations performed at the image providing center include an image distribution operation 2A for distributing an image in response to a member's image browsing request 201 and an image for maintaining the freshness of each image to be distributed by the image distribution operation 2A. It can be classified into the image update business 2B to be updated. The image distribution business 2A is a business that is performed on a daily basis and directly exchanges data with the user. When there is a browsing request from the registered user to the service management server 109 (201), the service management server 109 searches the storage device 106 for the image data 107 showing the requested position (202), and the data at that time The browsing log data 108 is transferred to the management server 109 and stored (203). When the searched image is found, the service management server 109 loads the corresponding image data 107 (204) and distributes the data to the user (205).

  The image update operation 2B is an operation for updating the image data 107 in the storage device 106 which is periodically performed in order to provide new information by the user, and uses browsing log data stored by the data management server. The satellite image capturing source orders an imaging region to be updated and a satellite capturing schedule (206), and the satellite image capturing source captures and sells the latest image according to the request (207). After that, the data management server image update program 105 performs image processing so that the image and the map can be superimposed on the satellite image, or corrects the color tone to be easier to see, and then replaces the old image at the same position (208). . In particular, the present embodiment can improve the efficiency of operations indicated by reference numerals 206 and 208 in the image update operation by using the browsing log. Hereinafter, a case where the browsing log is not used and a case where the browsing log is used will be described. For the correction of the color tone, for example, the density conversion method described in Non-Patent Document 2 can be used. The density conversion formula used for the representative area described below may be used as the overall density conversion formula, for example.

  FIG. 3 is a flowchart showing a procedure for updating an image according to the conventional technique when the browsing log is not used unlike the present embodiment. When the browsing log is not used, the designation of the shooting area and schedule for the image seller shown in 206 of FIG. 2 is performed uniformly in the area handled by the service, and the updated images of all areas are purchased (301). Thereafter, image processing indicated by 302 to 310 is performed on each purchased image. This corresponds to the image processing / update processing indicated by reference numeral 208 in FIG.

  The updated image is read (302). Next, a representative position (representative region) is determined from the read image (303). Satellite images can be captured over a wide area at a time, and the number of pixels often exceeds 100 million pixels (vertical and horizontal 10,000 pixels). It is a multi-pixel image. As the representative region, for example, a part of about 1 million pixels (vertical and horizontal 1000 pixels) is determined within a range that can be displayed at a time on a normal monitor. At this time, the condition for selecting the representative region is either a random position in the image or a portion determined as an appropriate representative region representing the feature of the updated image determined by the user.

  Next, the image of the representative region portion selected in (303) is cut out (304), and the image is corrected only in that portion (305). This image correction includes, for example, luminance correction and color tone correction (see Non-Patent Documents 1 and 2 above). About these correction methods, since the existing technique is used, detailed description is abbreviate | omitted.

  Next, the change of the corresponding pixel in the image before and after the correction processing performed in (305) is examined, and a correspondence table is created so that the corrected pixel value can be obtained from the pixel value before correction. Since this correspondence table may not be able to correspond to all pixel values, a function for correcting from the pixel values before correction may be created. (306) Hereinafter, this correspondence table is referred to as a LUT (LookUpTable).

  Next, in accordance with the LUT created in (306), the image other than the representative portion is corrected (307). Next, the position of the updated image is confirmed and the surrounding image is read (308), and the connection processing is performed on the updated image and the surrounding image (309-). The connection process is called mosaic, and is to unify the coordinates of a plurality of images and connect them like a single image. At this time, if the colors of adjacent images are greatly different, an unnatural image is formed. Therefore, processing such as histogram matching is performed to smooth the boundary. (Refer nonpatent literature 3). Although connection processing takes a long processing time, it only affects when viewing an image including the relevant part, so it is not necessarily emphasized, but by doing this, an unnatural image is presented to the user. Opportunities can be reduced. Thereafter, the image is updated (309), and repeated image processing is performed for every shooting scene for all the updated images. When there are no more shooting scenes, the processing ends.

  FIG. 4 is a diagram illustrating an example of the browsing log indicated by reference numeral 108 in FIG. As shown in FIG. 2 (203), the browsing log is created in response to the browsing request of the user during the image distribution operation, and is stored in the storage device 106 of the data management server. By using the browsing log 108, it is possible to check which area of an image is being browsed well and which part of the image of a certain area is being browsed frequently. Further, by storing more detailed information as the browsing log 108, it is possible to grasp changes in browsing information and detailed user preferences.

  As shown in FIG. 4 (A), the browsing log 108 includes a browsing time and a browsing position (assuming that the browsing area is a rectangle, the coordinates of the upper left corner) (The coordinates of the lower right corner) are stored as minimum information, and further detailed information, for example, information including the purpose of use and the age of the user is recorded.

  When the browsing log 108 is actually used, it is used not as individual data but as statistical data. FIG. 4B is a diagram showing the result of statistics of the browsing log in a two-dimensional and visual manner. When distributing satellite images in units of blocks cut out at a fixed position, the number of times viewed for each distributed block is statistically calculated. Generally, when distributing images, the position (range) of images that can be viewed by the user can be freely set. Therefore, the entire image is divided into several blocks, and some of them are viewed (referenced) by the user. ), The browsing frequency is calculated using a method of adding a score corresponding to the size of the browsed portion included in the block.

  FIG. 4C shows an example of the block size set at this time. Browsing frequency statistics are performed in two stages: a block from a macro viewpoint (left figure) and a block from a micro viewpoint (right figure).

  The statistics from a macro viewpoint are, for example, the range in which the entire service is provided, and the number of times of browsing is totaled by taking one block as a satellite image photographing unit (usually about 10 km per side). From a micro viewpoint, one block from a macro viewpoint is further divided and the number of times of browsing is counted as about one screen (usually about 500 m per side) of an image to be distributed. The method of using these statistical data will be described in detail with reference to FIG. FIG. 5 is a flowchart showing the flow of the image update method using the browsing log according to this embodiment. FIG. 5A is a diagram showing the entire update process, and FIG. 5B is a diagram showing the details of the process of step 502 in FIG. 5A in particular.

  First, before specifying the shooting area and schedule for the image vendor indicated by reference numeral 206, the image viewing log of the macro viewpoint shown in FIG. 4 is read (step 501), and updating is required with reference to the viewing log. A shooting position is selected (step 502). For selection of an image that needs to be updated, for example, it is checked whether or not the image has been browsed more than a certain number (FIG. 5B: Step 5021). It is considered that there is an update (step 5023), and if not (No), it is not updated (step 5024). Alternatively, it is evaluated whether or not a certain period has elapsed from the photographing time of the currently provided image (step 5022), and if it exceeds a certain value (Yes), it is considered that updating is necessary (step 5023). (No) update is not performed (step 5024). The processing from steps 5021 to 5024 is performed in the entire range provided to the user, and it is checked whether or not each image needs to be updated. Then, the image selected in step 502 is purchased (step 503). Through the processing from step 501 to step 503, it is possible to update the image efficiently by preferentially updating an image that is actively browsed or an image at a position that has become too old.

  Thereafter, image processing including steps shown in steps 504 to 514 is performed on each purchased image. These processes correspond to the image processing / updating process indicated by reference numeral 208.

  First, an updated image is read (step 504), then the position of the updated image is confirmed (step 505), and a viewing log from a micro viewpoint of the image at that position is read (step 506). With reference to the read log read here, for example, a place with high frequency is determined as a representative area (step 507). A place with the highest frequency may be set as the representative area, or a place including all of the numbers 1 to 3 in the order of high frequency may be set as the representative area. The algorithm for determining the representative area is not limited. It is also possible to perform correction in order, such as the first representative area having the highest frequency and the second second representative area. In that case, it is preferable to give a high coefficient so that the correction based on the representative area with higher frequency becomes stronger.

  In the subsequent processing, similarly to the processing shown in FIG. 3, the image of the representative region selected in step 507 is cut out (step 508), and the image is corrected only in that portion (step 509). Next, for the images before and after correction performed in step 509, the corresponding pixel change is examined, and for example, an LUT is created (step 510). Next, an image other than the representative portion is corrected according to the LUT created in step 510 (step 511). Next, the position of the updated image is confirmed and the surrounding image is read (step 512), and the connection process is performed on the updated image and the surrounding image (step 513). Thereafter, the image is updated (step 514), the processing from step 504 to step 514 is repeated, and after repeating the image processing for all the updated images, this processing is terminated.

  FIG. 6 is a diagram illustrating a comparative example that represents a difference in color tone depending on the degree of correction. The image shown in FIG. 6A is a diagram showing an original image supplied from a satellite image capturing source before correction. FIG. 6B shows an example of a result of applying the image correction processing shown in FIG. 4 or 5 to the original image shown in FIG. 6A and a part of the image used as representative data (representative). It is a figure which shows the area | region 1). In this image, the representative area is occupied by an urban area with many buildings, which is suitable when the number of times of browsing in the urban area is large. FIG. 6C shows an example in which image correction is applied to the original image shown in FIG. 6A by selecting a representative area different from that in FIG. 6B, and used as representative data at that time. A part of the image (representative region 2) is shown. In this image, the representative area is mostly occupied by green space, which is suitable when the number of times of viewing the portion including the green space is large. It is difficult to determine which of the correction results shown in FIGS. 6B and 6C is suitable by simply looking at the image, and changes depending on the user's preference for viewing. There is.

  Even in such a case, by using the technique according to the present embodiment, there is an advantage that overall user satisfaction can be increased only by correcting the representative area according to the user's browsing frequency. .

  Needless to say, if the number of browsing times is statistically processed for each user, a correction preferable for the user can be performed. Further, in the above example, the image of the region with high browsing frequency is described as the representative image, but the image with the region with low browsing frequency is set as the non-representative image, and the correction method of the region different from the correction method in the non-representative region is used. For example, it is possible to correct the entire image by using it. Next, an image processing system according to the second embodiment of the present invention will be described. In the first embodiment, a system has been described in which only the number of times a user browses an image position is used for image correction. FIG. 7 is a diagram illustrating an example of a table in which a browsing log and detailed information are associated with each other according to the present embodiment. In the present embodiment, attribute information about an image such as data such as a location where an image provided when an image is provided to a user and a user's usage method are added to the browsing log. For example, as shown in FIG. 7, when 1 to 3 are extracted as representative areas, the user whose user ID is A001 is that the image utilization method is a travel plan and the hobby is a trip to a resort area. In this case, in addition to the number of log viewing times, the representative area is determined centering on the resort area even for the same image. On the other hand, for a user whose user ID is B001, if the main purpose is to use an image for the study of a building, an area with many buildings is selected from representative areas 1 to 3 with a large number of browsing related to log browsing. It is preferable to determine the representative area. In this case, it is possible to determine and correct a more preferable representative area from among a plurality of representative areas by taking into account user information unique to the user, independently of the browsing log.

  In this way, by adding more detailed information to the browsing log, it is possible to further analyze the viewer's preference and provide a desirable image. This includes, for example, how to use the image, what kind of application is used to view the image, and a browsing log that includes the user's age group and gender, etc. By providing and applying image correction according to it, a service with higher user satisfaction can be performed.

  Next, an image processing system according to the third embodiment of the present invention will be described. The first and second embodiments are characterized in that image correction of other areas is also performed using a correction method of an image of a representative area having a large number of browsing times obtained from the browsing log. That is, it is characterized in that the image is frequently updated in a region having a high browsing frequency obtained from the browsing log. For example, in step S503 in FIG. 5A, when purchasing an updated image, it can be estimated that an image having a high browsing frequency is viewed with a large number of purchases (update priority is high) when viewed from the purchaser. Therefore, it is preferable that the image processing system also prepares for image changes (changes due to terrain and seasons) by frequently performing re-photographing and the like for images with a high viewing frequency.

  As described above, according to the image processing system according to each embodiment of the present invention, the average satisfaction degree of the corrected image seen by the user can be corrected by considering an area in which many users are interested. Can be improved. In addition, there is an advantage that an object having a high information value can be updated with higher priority while suppressing an update cost for making the image provided to the user fresher.

  The present invention can be used as an image processing system for providing satellite images.

1 is an overall configuration diagram showing a system configuration example according to an embodiment of the present invention. It is a figure showing the flow of the operation | work on the side which performs the image provision service which has an image processing system by the 1st Embodiment of this invention. It is the flowchart figure explaining the general image update process. It is a figure showing an example of the structure of the browsing log data handled in this Embodiment, and statistical data. It is the flowchart explaining the detail in the case of performing the image update process by this Embodiment. It is a flowchart for selecting the position which needs an update among FIG. 5A. FIGS. 6A to 6C are image diagrams showing examples of image correction according to the present embodiment. It is a figure which shows an example of the table which linked | related the browsing log and detailed information used for the image processing by the 2nd Embodiment of this invention.

Explanation of symbols

101 ... Client PC
102 ... Browsing client program 103 ... Communication device 104 ... Data management server 105 ... Image update program 106 ... Storage device 107 ... Image data 108 ... Browsing log data 109 ... Service management server 110 ... Image providing service required program 111 ... Communication device

Claims (4)

An image processing system for providing an image taken from a high sky,
Based on the browsing frequency information obtained from the browsing log related to browsing of images, a representative area that is a partial area with a high browsing frequency is obtained from the entire image, and an image correction method for the representative area is set to an area other than the representative area. An image processing system characterized by being applied to image correction of an image including   Image updating means for updating an image accompanied by color tone correction processing is added, and attribute information related to user image browsing is added to the browsing log, and based on the browsing frequency information and attribute information related to user image browsing. The image processing system according to claim 1, wherein the representative area is determined. An image processing system for providing an image taken from a high sky,
Based on the browsing frequency information obtained from the browsing log related to browsing of images, a representative area that is a partial area having a high browsing frequency is obtained from the entire image, and the priority for updating the image of the representative area is increased. A featured image processing system. An image processing method for providing an image taken from a high sky,
Reading a macro-viewpoint image browsing log, referring to the image browsing log and selecting a shooting position that needs to be updated;
For each selected image, reading an update image, confirming the position of the update image, and reading a viewing log from a micro viewpoint of the image at that position;
Determining a representative area with reference to the read browsing log, setting a place with high frequency as a representative area, and applying a color tone correction method of the representative area to a color tone correction of an image including an area other than the representative area; An image processing method comprising:

Images