JP2011138263A - Management system and printer utilizing the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform processing conscious to an output on a server while accumulating an image on the server. <P>SOLUTION: This image management system includes: an image management server 30 including a storage part 34 storing a plurality of images; and a PC 10 accessible to the image management server 30 via a network NT. The PC 10 can input a retrieval condition to the image management server 30 through a UI screen displayed on a display part 11 by the image management server 30, and the image management server 30 includes a data practical use part M4 retrieving an image according with the retrieval condition based on a characteristic amount of each image or a retrieval history stored in the storage part 34 associated to the image from the plurality of images, or executing image processing based on the characteristic amount of the image or the retrieval history, or information wherein an output device and the characteristic amount of the image are associated. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image management system and a printing apparatus using the image management system.

  Conventionally, a technique for storing various data on a server, accessing the server via a network, and extracting necessary information by searching is known (for example, Patent Document 1).

JP 2007-72961 A

  However, the servers described in Patent Document 1 described above are generally managed individually by individuals, groups, and traders. This situation is the same for systems that store photos as electronic data in an external storage and can input and output as necessary. Most of the photos and electronic data are individually managed by individuals, organizations, and vendors. Yes. For this reason, it is necessary to take care of photo management / maintenance and infrastructure for various input / output and service provision. Also, the flow from input to output tends to be complicated. In addition, data is easily buried.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image management system capable of performing processing in consideration of up to output on a server while accumulating images on the server, and a printing apparatus using the image management system.

  In order to solve the above problems, an image management system according to the present invention includes an image management server including a storage medium that stores a plurality of images, and a client that can access the image management server via a network. In the image management system, the client includes an input unit for inputting a search condition to the image management server, and the image management server stores the image in association with the image from the plurality of images. The image processing apparatus includes a search unit that searches for an image that matches the search condition based on the output condition and the feature amount of the image. The number of clients may be one or two or more.

  In the above configuration, the input unit of the client can access the management server via the network and input search conditions. When a search condition is input from the input unit, the search unit of the management server searches for an image that matches the search condition from a plurality of images stored in the storage medium based on the search condition. The storage medium stores an output condition and an image feature amount in association with the image, and the search unit performs a search based on the output condition and the image feature amount. The output condition is a history of searches and image processing performed in the management server, information provided by the user for an image associated with the output condition, and the like. The history can use not only the history associated with the image associated with the output condition but also the history of all searches and image processing performed on the management server. The feature amount of the image is information obtained by analyzing metadata attached to the image associated with the output condition or the image data itself. As described above, by performing a search using the output condition and the feature amount of the image, the client can easily search for a desired image from a large number of images stored in the image management system. Become.

  As another aspect of the image management system according to the present invention, an image management server comprising an image management server comprising a storage medium for storing a plurality of images, and a client accessible to the image management server via a network. In the system, the client includes a first selection unit that selects an image on the storage medium, and the image management server stores the image selected by the first selection unit in the storage medium in association with the image. The image processing unit may be configured to perform image processing based on the stored output condition, the feature amount of the image, and information that associates the output device with the feature amount of the image. The number of clients may be one or two or more.

  In this configuration, the first selection unit of the client can access the management server via the network and select an image from the plurality of images. One or more arbitrary numbers of images can be selected from the first selection unit. When an image is selected by the first selection unit, the image processing unit of the management server performs image processing on the selected image. The image processing executed here is performed on the basis of information relating the output condition and the image feature amount stored in the storage medium in association with the image, and the output device and the image feature amount. The output condition and the feature amount of the image are the same as those described above. The information that associates the output device with the image feature amount is, for example, a color conversion table that defines the color correspondence between the color system of the image data and the color system of the output device, such as an ICC profile. In this way, by performing image processing using information that associates output conditions, image feature amounts, and output devices with image feature amounts, even a user unfamiliar with image processing can obtain an image appropriate for the image. Processing can be performed.

Further, as a selective aspect of the present invention, the image management server includes an output condition, an image feature amount, and the search condition stored in the storage medium in association with the image from the plurality of images. A search unit that searches for an image based on the key, and a reception unit that receives key information. The search unit has a predetermined distance in the image feature amount space between the image feature amount created based on the key information. The images within the range may be searched from a plurality of images stored in the storage medium.
In this configuration, the search unit searches the image database for an image whose distance from the image feature amount created based on the key information is within a predetermined distance in the image feature amount space. As this predetermined distance, a user's designation may be accepted, or a preset value may be used. The image feature amount space is a virtual space configured with various image feature amounts characterizing an image as dimensions (variables). The key information can be specified as a keyword or can be specified as an image. In the case of designating with a keyword, it is possible to convert into an image feature amount based on this correspondence relationship by creating a correspondence relationship between the image feature amount recalled by the keyword and the keyword in advance. On the other hand, when the key information is designated by an image, the image feature amount can be calculated from the image by various known methods.
The image searched by the search unit is similar to the key information, but is similar to the search target image, and there is a possibility that the search result includes the search target image. high. Further, even if the search result does not include the search target image itself, there is a high possibility that the search result includes an image similar to the search target image.
Therefore, it is highly likely that the user can find an image close to the user-desired search target image from the search results of the search unit. When the user finds an image with a high degree of similarity to the search target image in the search result, the user can perform a search using this image as new key information. Of course, the search may be performed by accepting an image other than the search result as key information. With this configuration, it is possible to provide a client user with a means for easily searching for a target image from a large number of images.

  Further, as a selective aspect of the present invention, the image management server further includes a counting unit that counts the number of searches of the search unit, and the search unit is used for the search as the number of searches increases. Increase the amount of image feature information. The image feature amount is information that characterizes the image. The greater the information amount of the image feature amount, the more accurately the specific image can be specified. When the information amount is small, the image can be specified only roughly. When a search that links the characteristics of the image feature amount related to the information amount with the process of clarifying the search target image in the user's image is realized, usability is improved. That is, as the number of searches increases and the image of the user gradually becomes clear, the image feature quantity creation unit creates an image feature quantity with an information quantity that can accurately identify the key information. The search result of the copy also transitions so that many images with high similarity to the key information are included. Therefore, even if the searcher is not familiar with the feature amount for specifying the image or is not skilled in the search, it is possible to search the image database for the image of the searcher.

  As a selective aspect of the present invention, the client includes a second selection unit that selects an image from the search result of the search unit, and the image management server includes the key information and the second selection unit. A search history storage unit that stores a correlation with the selected image as a search history may be provided, and the search unit may search for an image having a high degree of correlation in the search history. The image selected by the second selection unit is an image selected by the user from the search results, and it is considered that there is a strong correlation with the key information in the user image. That is, it is presumed that the correlation between the key information such as a keyword and an image and the image selected by the second selection unit was useful information for the user. Therefore, the correlation is stored in the search history storage unit as a search history so that the correlated information can be searched in the subsequent search. In subsequent searches, the higher the degree of correlation, the higher the search result. In this way, the user can easily and quickly search for a desired image from a large number of images using key information adapted to the user's own image. The same applies to an unknown image.

Further, as a selective aspect of the present invention, the image management server outputs an output condition and an image feature amount stored in the storage medium in association with the image with respect to the image selected by the first selection unit. And an image processing unit that performs image processing based on information that associates the output device with the feature amount of the image, and a history of image processing performed by the image processing unit on an image selected by a specific client An image processing history storage unit for storing the image processing unit, and the image processing unit may execute image processing of another client based on the history of the image processing.
The user's image processing technology is superior or inferior depending on the experience of each person, and for inexperienced users, it is well understood what image processing should be performed to correct an unsatisfactory image to a satisfactory state. There's nothing to do. Therefore, in this configuration, the history of image processing performed by a specific user who uses a specific output device is stored, and when other users try to output using the specific output device, Image processing using this history is made available. The specific user is preferably a person skilled in image processing technology. For example, a person who has used the camera for a long time or a person who has frequently performed image processing can be selected. In this way, even a user unfamiliar with image processing can perform appropriate image processing on an appropriate image on the management server.

  As an optional aspect of the present invention, the image management server includes a profile storage unit that stores a profile of the user of the client, and the image processing unit determines the intensity of image processing based on the profile. You may make it do. The profile contains information (such as age and gender) that represents the user's preferences, and the user can prepare appropriate combinations of image processing for each of the preferences expressed by the profile. A combination of image processing that can obtain a correction result that matches the user's preference when trying to perform image processing can be output.

  As another optional aspect of the present invention, the client may be a printing device, and the printing device may print the image searched by the search unit. By providing a printing device as the client, the stored image can be printed using the search function provided in the management server.

  Further, as a selective aspect of the present invention, the client may be a printing device, and the printing device may be configured to print an image that has been subjected to image processing by the image processing unit. By providing a printing apparatus as the client, it is possible to print the stored image using the image processing function provided in the management server.

  The image management system described above includes various modes such as being implemented in a state of being incorporated in another device or system, or being implemented together with another method. Further, by changing the output device, it also functions as a photo ordering system, an exhibition system, an image processing system, an image organization system, and an image search system. Further, the present invention provides a control method having steps corresponding to the configuration of the image management system, a program for causing a computer to realize a function corresponding to the configuration of the image management system, a computer-readable recording medium storing the program, and the like Can also be realized. The invention of the image management method, the image management program, and the medium on which the image management program is recorded also has the above-described operations and effects. Of course, the configurations described in claims 2 to 9 are also applicable to the above method, the above program, and the above recording medium.

It is a block diagram which shows the structure of an image management system. It is a flowchart which shows the whole flow of an image management process. It is a figure which shows the example of the screen of the user interface of an image management process. It is a figure which shows an example of the user management table UT. It is a flowchart which shows the flow of an image analysis process. It is a flowchart which shows the flow of a data utilization process. It is a flowchart which shows the flow of a query search process. It is a figure which shows an example of UI screen displayed by a query search process. It is a table explaining an example of a search history database of keyword search. It is a table explaining an example of the search history database of key image search.

Hereinafter, embodiments of the present invention will be described in the following order.
(1) Configuration of the present embodiment:
(2) Image management processing:
(3) Image analysis processing:
(4) Data utilization processing:
(5) Various modifications:
(6) Summary:

(1) Configuration of the present embodiment:
FIG. 1 is a block diagram showing a configuration of an image management system according to the present embodiment. In FIG. 1, an image management system 100 includes a personal computer 10 (PC 10) as an input device, a printer 20 as an output device, and a management server 30. The PC 10 and the printer 20 are connected via a network NT and are clients that communicate with the management server 30. Although only one input device or output device is shown in the figure, it depends on the number of user registrations or device registrations with respect to the management server 30 or the connection status of the input device or output device to the network NT. The number of input devices and output devices is variously changed.

  In the present embodiment, the input device and the PC 10 are described, and the output device is described as the printer 20. However, the present invention is not limited to this. For example, as an input device, a printer (printing device) having a reader function for reading image data from an external storage medium (memory card, USB memory, etc.), a mobile phone, a camera function for taking a still image or a moving image It is possible to employ a digital camera, a mobile phone, a scanner device that acquires image data on the surface of a medium, and the like. In other words, various devices can be used as the input device as long as the device can upload image data to the management server 30 via the network NT. For example, a mobile phone, a photo image viewer, a personal computer, a projector device, a printer, or the like can be used as the output device. That is, as the output device, various devices can be adopted as long as they can output (display and print) an image based on the image data.

  In FIG. 1, the management server 30 displays a control unit 31 that controls the entire management server 30, an operation input unit 32 for an administrator to input an operation, and various information related to the management server 30 by an administrator. Display unit 33, a storage unit 34 composed of a large-capacity storage medium such as a hard disk (HD), an I / F 35 functioning as an interface with the network NT, and a hard disk (HD ) And the like. The storage unit 34 and the storage medium 36 may be integrated. The control unit 31 can be configured in a program execution environment including a CPU, a RAM, and a ROM, for example. Of course, the control unit 31 may be configured by a custom IC such as an ASIC (Application Specific Integrated Circuit).

  The management server 30 implements functions corresponding to the setting registration unit M1, the image upload unit M2, the image analysis unit M3, and the data utilization unit M4 shown in FIG. The setting registration unit M1 accepts user registration / cancellation, device registration / cancellation, and various settings. The image upload unit M2 accepts image upload from the PC 10. The image analysis unit M3 analyzes the uploaded image, extracts the feature amount of the image, and creates a feature amount database DB. The data utilization unit M4 provides various services to the user using the feature amount database DB or the like.

(2) Image management processing:
FIG. 2 is a flowchart showing an overall flow of image management processing executed by the image management program, and FIG. 3 is a diagram showing an example of a user interface screen displayed on the PC 10 during the image management processing. . The process shown in FIG. 2 is executed independently for each user (or for each output device), and a plurality of image management processes can be executed simultaneously. The image management process is started by performing user registration and output device registration with the management server 30.

  In the following description, the case where the processes are executed in the order of steps S100 to S160 will be described. However, these processes can be executed in any order except for user registration in step S100. User registration must be performed first, but processing to be performed after other user registration is completed (additional registration and deletion of new devices, uploading of images, processing of using uploaded images, cancellation of user registration) This is because it can be executed even if no other processing is performed.

  In addition, the image analysis processing for analyzing the uploaded image and extracting the feature amount can be executed at an appropriate timing by the management server 30 until the processing for using the uploaded image is performed. Extraction of image feature values only needs to be completed by the time when the user executes processing that uses the image, and when image uploads from multiple users are accepted, priority is given over execution in real time. This is because it is more efficient to determine the order and execute it between other processes.

  When the process is started, in step S100, the setting registration unit M1 accepts registration of a new user. User registration can be performed, for example, online via the PC 10 or by submitting an application form or email to a predetermined submission destination. If the former, for example, as shown in FIG. 3, when the user accesses the setting registration unit M1 via the PC 10, "new user registration" is selected on the "login screen" displayed on the display unit 11 of the PC 10, Enter various user information (desired user ID, desired password, information on payment method of image management system usage fee (credit payment, bank transfer, cash, etc.) etc.) on the "New User Registration Screen" displayed at this time Can be done. In the latter case, the same information as the former can be filled in and submitted on an application form and the workers of the organization operating the management server 30 can register the various user information in the management server 30.

  In step S110, the setting registration unit M1 accepts a new device registration. The device to be registered here is the output device described above. Device registration may be performed online via the PC 10 as with user registration, or may be performed by submitting an application form or email. In the example shown in FIG. 3, the user logs in from the “login screen”, proceeds from the “use purpose selection screen” to the “registration / cancellation screen”, and enters “device registration” as device identification information (model and manufacturer of the printer 20 Name etc.). Of course, the device identification information can be entered in an application form and submitted, and an operator of the organization operating the management server 30 can register the device identification information in the management server 30. Note that the user can also register a plurality of devices as output devices. If the user has already been registered, step S110 can be performed without performing step S100. The above steps S100 and S110 are preferable in that they can be performed smoothly when the user purchases an output device such as the printer 20.

  FIG. 4 is a diagram showing an example of the user management table UT that manages the data input in the above steps S100 and S110. The user management table UT is data indicating a correspondence relationship between user information, device identification information of the output device, and available service types. The service types that can be used are the service types that can be used by the user and the service types that can be used by each output device, and the device characteristics of the output device (such as being able to “print” but not “display” with a printer) ) And the type of service contracted by the user. For example, the printer A registered by the user A can print and download, but the printer A registered by the user C can only print. In this way, even if the same output device is used, services that the user has not contracted cannot be received. The user management table UT is stored in the storage medium 36 of the management server 30.

  In step S120, the user performs use environment setting. In the example of FIG. 3, the setting of the use environment is performed on the “setting change screen”, and “security setting”, “image processing setting”, and “printing condition setting” can be performed on the “setting change screen”. it can. In the “security setting”, it is possible to set the disclosure range of the data uploaded in the upload process described later. The disclosure range includes, for example, “no restriction”, “only browsing is possible”, “public to a specific user”, “private”, and the like, and is a setting related to access restriction of an uploaded image. The disclosure range may be set uniformly for all uploaded images, or may be set for each image. Further, the disclosure range set in the “security setting” may be set as a default disclosure range, and the default disclosure range may be applied when uploading without specifying. In “image processing setting”, the type and degree of image processing to be applied to an image when the image is output to the output device can be set. In this setting, for example, it is possible to set whether or not to perform small face correction and memory color correction (processing for correcting skin color, sky color, lawn color, etc. to a human preferred color) and the degree of correction. it can. In “print condition setting”, print settings for outputting an image to the printer 20 such as paper size, print quality, and layout printing can be performed. Of course, when a plurality of output devices are registered, printing conditions and output conditions can be set for each output device. When the output device is not a printer, set the resolution and the number of images to be displayed on one screen, or set the display time per image when outputting multiple images to a projector or photo frame. Can be.

  In step S130, the user registers a profile. In the example of FIG. 3, the profile is input on the “profile input screen”, and the sex, age, and camera use level can be set on the “profile input screen”. Age and gender can be used in image processing to be described later. The camera usage level can be used as an index for discriminating the skill level of image processing, and by using the history of users with high skill levels preferentially when effectively using the search history and image processing history described below, Setting is easy even for users who are not proficient in handling images. The profile may be registered at the time of user registration performed in step S100.

  In step S140, the user uploads an image to the management server 30. For example, the image is uploaded by specifying information specifying the file name and storage location of the image to be stored in the management server 30 on the “upload screen” in FIG. When the designation of information is completed and the user instructs uploading, the designated image is transmitted to the management server 30 via the network NT, and the management server 30 stores the transmitted image in the area of each user in the storage unit 34. Is done. When the image is uploaded, the image analysis in step S150 and the data in step S160 can be used.

  In step S150, the image analysis unit M3 extracts a feature amount from the image, and stores the extracted feature amount in the storage unit 34 in association with the image. Details of this image analysis processing will be described in section (3). The image analysis unit M3 may extract feature amounts from the images in the order in which the images are uploaded, or may extract feature amounts from the images when selected as a search target in a data utilization process described later. Good.

  In step S160, the user receives various services (search process, image process, output process (print process, display process, save process, etc.)) provided by the data utilization unit M4 using the uploaded image. The data utilization unit M4 uses the feature amount extracted by the image analysis unit M3, the image processing setting set by the user, and the printing condition setting (output condition setting) to perform printing, photo printing, exhibition, image processing, and organization.・ Provide various services such as search. Also, as will be described later, use history such as image processing settings and search condition settings made by each user, and use history such as image processing settings and search condition settings when other users use the service are also used. Service is provided. Details of these various processes will be described in section (4).

  In step S170, the setting registration unit M1 cancels user registration and device registration. In the example of FIG. 3, the user can cancel the user registration by selecting “cancel user registration” from the “registration / cancellation screen”. When the user registration is canceled, information regarding the corresponding user is deleted from the management server 30. In addition, the registration can be canceled for each device by selecting “cancel registration of printer A” or the like. Of course, the user can also cancel the registration by submitting the user information and device identification information on the cancellation application form.

(3) Image analysis processing:
FIG. 5 is a flowchart showing the flow of the image analysis process executed in step S150 described above. In order to create the feature database DB, the image analysis unit M3 first scans the data of the analysis target image and acquires metadata (Exif (Exchangeable image file format) or the like) (step S150a). Next, the image analysis unit M3 performs various image analysis processes on the analysis target image, thereby calculating “color feature amount”, “composition feature amount”, and “subject feature amount” (step S3). S150b). Then, the image analysis unit M3 stores the various feature amounts calculated in steps S150a and 150b in the feature amount database DB in association with the images (step S150c).

  “Color characteristic amount” is a characteristic amount related to the brightness, contrast, saturation, color balance, etc. of the image. For example, the RGB histogram or luminance histogram created from the target image, or the maximum or minimum value in these histograms. It can be calculated as a value, an average value, a mode value, the number of colors, the number of luminances, and the like. The “composition feature amount” is a feature amount such as an edge distribution that represents the relationship between the pixel of interest and its neighboring pixels. For example, a density histogram, texture information obtained from a co-occurrence matrix, difference statistics, It can be calculated based on a run length matrix or the like. “Subject feature value” is a principal component parameter of a person or object contained in an image, and represents a low-dimensional feature space that can express the distribution well from many learning input images distributed in a multidimensional feature value space. It can be calculated by the principal component analysis to be obtained. Note that when detecting principal component parameters, prior to this, object detection processing such as face detection processing and face organ detection processing is performed, and principal component analysis is performed on the detected object.

Then, the image analysis unit M3 stores the various feature amounts calculated in steps S150a and 150b in the feature amount database DB in association with the images (step S150c). The feature amount database DB is stored in the storage unit 34. The feature amount database DB is data in which an index (for example, a file name) of each image, metadata such as Exif included in the image, and the various image feature amounts described above are associated with each other. Note that the image feature amount in the present embodiment may be normalized to a predetermined range (for example, 0 to 100, etc.) so that the image feature amount can be easily compared in the search process described later.

  In step S150d, the image analysis unit M3 displays the calculated feature amount on the PC 10 for confirmation by the user. If the displayed feature value is incorrect or unexpected, the user operates the operation input unit 12 while looking at the screen of the PC 10 to correct the feature value, or to recalculate the feature value in the image analysis unit M3. Or instruct. When the recalculation of the feature amount is instructed, the image analysis unit M3 repeats the processing of steps S150a to S150d for the same image.

(4) Data utilization processing:
FIG. 6 is a flowchart showing the flow of data utilization processing. As shown in the figure, in the data utilization process, a service and an output device are selected (steps S200 and S205). In the example shown in FIG. 3, the selection of the service and the output device can be performed on the “service selection screen” and the “output device selection screen”. Of course, the service and the output device may be selected after the image to be output is determined. This is because it is possible to execute without specifying the output device until the image to be output is determined, and the subsequent image processing and output processing are processing performed by specifying the output device.

  Next, the user selects an image to be output. At this time, the user needs to select a desired image from a plurality of images stored in the storage unit 34. As a selection method, a method of displaying all uploaded images as thumbnails or a list and selecting them one by one is possible, but it is troublesome to select one by one from a vast number of images in the storage unit 34. In particular, when an extraction source including an image publicized by another user is used, the labor is very difficult. Therefore, the data utilization unit M4 includes a search unit, and the search unit M41 can perform a query search process for searching based on a query such as a keyword or a key image. In this embodiment, the means for the user to select an image from a plurality of images stored in the storage unit 34, including a method of selecting a desired image while narrowing down the search result using the search unit. A first selection unit is configured. In addition, means for selecting a desired image from images whose search results have been narrowed down using the search unit constitutes a second selection unit.

(4-1) Query search processing:
FIG. 7 is a flowchart showing the flow of query search processing. When the process is started, in step S210A, a counter C indicating a search stage is initialized, and the process proceeds to step S210B. The role of the counter C will be described later. In step S210B, a UI screen for accepting search conditions and displaying a list of search results is displayed on the display. When any operation input is performed on this UI screen, the process proceeds to step S210C.

  FIG. 8 is a diagram illustrating an example of a UI screen displayed in step S210B. The UI screen shown in the figure includes a query setting area A1 for setting various queries, a search range designation area A2 for setting a range of images as a search mother set, a search result display area A3, and a search And an execution button B.

The search range designation area A2 displays a list of names indicating the image search range. A check box is provided beside each search range name, and when the user checks the check box, images belonging to the checked search range are selected as a search mother set in the search process. Checks can be included in multiple search ranges.
In the query setting area A1, a key image KI is designated, a neighborhood range designation value Dx is designated, and a keyword KW is designated. The key image KI can be specified by, for example, dragging and dropping an image displayed in file browsing software, Internet browsing software, or the like into the query setting area A1. Of course, the key image KI may be specified by the file path or URL of the image, or a list of images included in the search range selected in the search range specification area A2 can be displayed, and an image can be selected and specified from the list. You may do it. This query setting area A1 constitutes a reception unit in the present embodiment.

  When the key image KI is designated, a thumbnail image of the designated key image KI is displayed in the query setting area A1, and a slider for designating a neighborhood range designation value Dx for the key image KI is displayed in the query setting area A1. Is displayed. In the example shown in FIG. 8, a thumbnail image indicating the key image KI is displayed in the query setting area A1, and a slider for setting the neighborhood range designation value Dx is displayed below the thumbnail image.

  The user can set the neighborhood range specification value Dx by moving the slider with a mouse or cursor keys. In FIG. 8, the settable range of the neighborhood range specified value Dx is “10” to “80”, and “40” is displayed as the neighborhood range specified value Dx corresponding to the current position of the slider. Since the user may not assume a clear search range, an intermediate “40” or the like may be displayed as an initial value in the initial state of the slider. This neighborhood range designation value Dx corresponds to the predetermined distance of the present embodiment.

  In the example described above, an image is used as a query, but it is also possible to input a keyword KW in the query setting area A1. In the present embodiment, the key image KI and the keyword KW can be exclusively input to the query setting area A1, and the keyword search and the key image KI search are executed exclusively. Of course, when the logical search, logical product, exclusive logical sum, etc. of the keyword search and the key image KI search are used as the search results, the keyword KW and the key image KI may be simultaneously input to the query setting area. These keywords and key images constitute key information in this embodiment.

  In step S210C, it is determined whether search execution is instructed. That is, it is determined whether or not the search execution button B is clicked. If the search execution button B is clicked, the process proceeds to step S210D. If the search execution button B is not clicked, the process proceeds to S210K. If the search has not been performed, the process proceeds to step S210M without displaying the search result even if the process proceeds to step S210K.

  In step S210D, it is specified which search method the keyword search or image search is desired by the user. For example, when the search execution is instructed, if the query input in the query setting area A1 is the keyword KW, it is determined that the keyword search is selected. If the key image KI is input, the key image search is performed. Is determined to have been selected. Of course, search execution buttons for keyword search and key image search may be provided separately, and the search method desired by the user may be specified according to the clicked button. If keyword search is selected, the process proceeds to step S210L, and if image search is selected, the process proceeds to step S210E. When the process proceeds to step S210L, an image including the keyword KW is searched from the search range selected in the search range specifying area A2, the search result list data is created and temporarily stored in the RAM, and the process proceeds to step S210K.

  In step S210E, it is determined whether or not the key image KI has been selected from the search result display area A3, or whether or not the key image in the previous search has been selected again as the key image KI. When a new key image KI is selected from the search results displayed in the search result display area A3, or when the key image in the previous search is selected again as the key image KI, the process proceeds to step S210G. On the other hand, if the new key image KI is an image selected from a key image or a search result other than the previous search, the process proceeds to step S210F to initialize the counter C to 0, and then proceeds to step S210G.

  In step S210G, the image analysis unit M3 is caused to calculate the image feature amount of the key image KI. The image analysis unit M3 calculates an image feature amount from the key image KI by the same process as the image analysis in step S150 described above, and transfers it to the search unit M41.

In step S210H, an image in the search range is searched based on the image feature amount of the type corresponding to the value of counter C. Specifically, when the counter C is “1”, the “color characteristic amount” CA01a, CA01b, CA01c,... Of the key image KI and each image In (n = 1, 1) included in the search range. Euclidean distance Dn (n = 1, 2, 3,...) With “color characteristic amount” CAn1a, CAn1b, CAn1c,. ..) are respectively calculated in the feature space. The distance Dn between each image In and the key image KI can be calculated by the following equation (1).

  When the image In having the distance Dn equal to or smaller than the neighborhood range specified value Dx is specified, list data in which the specified images are arranged in ascending order of the distance D0n is created and temporarily stored in the RAM. When the counter C is “2”, the Euclidean distance Dn is calculated including the image feature amount related to the “composition feature amount” in addition to the “color feature amount”. Further, when the counter C is “3”, the Euclidean distance Dn is calculated including the image feature amount related to the “subject feature amount” in addition to the “color feature amount” and the “composition feature amount”. Become. When the search in step S210G is completed, the counter C is incremented in step S210I and the process proceeds to step S210K.

Here, the counter C will be described. In this embodiment, when the next search is performed within a range where the search results already performed can be used effectively, the search stage is advanced.
Specifically, in the next search, an image included in the previous search result is selected as the key image KI, or an image that was the key image KI in the previous search is selected again as the key image KI. If so, the search phase proceeds. In other words, if the next key image KI is selected from the previous search result or the key image KI in the previous search, a search different from the previously executed search result is started. Consider and initialize the search phase. The counter C is for managing this search stage, and counts the number of searches in a group of searches. That is, when the next key image is selected from the search results, the search stage proceeds, so the counter C is incremented, and the user drags and drops an image such as a web page or the like to the query setting area, and the key image KI. When set to, the counter C is initialized. The search unit that manages the counter C constitutes a counting unit in the present embodiment.

  In step S210K, the search results of steps S210H and S210L are displayed in the search result display area A3 in the UI of FIG. At this time, if the process proceeds from step S210L to step S210K, a list of search results is displayed in the search result display area A3 in the order of the search hit. On the other hand, if the process proceeds from step S210H to step S210K, search results are displayed in the search result display area A3 in the order of similarity with the key image KI, that is, in the order of closer Euclidean distance to the key image KI in the image feature amount space. Listed. When the search result is displayed, the process proceeds to step S210M, and it is determined whether or not the user has selected any of the search results for data utilization. When the user selects any or all of the search results, the process proceeds to step S225 in FIG. 6, and information regarding the search parameters and the selected search results is stored in the storage unit 34 as a search history. The search history will be described in (4-3). On the other hand, if the user does not select any of the search results, the process returns to step S210B to display a UI display and wait for an operation input to the UI.

  As described above, as the search stage progresses, the amount of information used for the search is increased by increasing the types of image feature quantities used for the search. When the search is in the initial stage, the search is performed with a rough image. The result is output, and the search result with higher similarity to the key image is output as the search stage proceeds. In other words, the closer the key image is to the user's image, the more detailed search can be executed. Therefore, even if the user has only a vague image of the image he wants to search at the start of the search, the image can be gradually clarified from the search results as the search proceeds, so the user can Can be guided to. Further, it is possible to easily and quickly search a desired image closest to the user image from the database, starting from an image close to the user image.

  Note that the resolution of the image feature amount may be increased as the search stage proceeds in the sense of increasing the information amount. The precision mentioned here means the degree of quantization in the gradation of the image (the number of gradations of color and the gradation of luminance) and the resolution. Note that the gradation number is a numerical value indicating the number of steps for dividing the lightness and darkness of the image, and the resolution is a sampling interval indicating the number of sampling points per unit length. In other words, if a search is performed based on an image feature amount with low accuracy, a search can be performed with a rough image. If a search is performed based on an image feature amount with high accuracy, a search is performed with a clear image. Can do.

(4-2) Tag search:
In this embodiment, tag search can also be performed. The tag is a category name created by the user, and the user creates it with his / her classification method and attaches it to the image. A plurality of different tags may be assigned to one image, or there may be an image to which no tag is assigned. When a tag search is performed, an image that is totally similar to a plurality of images to which a tag is assigned can be extracted from the storage unit 34.

  In order to perform tagging, the data utilization unit M4 includes a tag providing unit. The tag assigning unit can generate a new tag, delete an existing tag, assign a tag to an image, and delete a tag attached to the image. The tag adding unit can be called and executed at an arbitrary timing in the image management process. For example, by executing the tag assignment unit when performing the upload process, uploading while adding a tag to the image to be uploaded, or tagging the search results obtained when executing the search process described later Or deleting a specific tag from the obtained search result. If a specific tag is specified, it is possible to extract an image to which the tag is assigned.

When a tag is created, a tag feature TC is created for each tag. The tag feature value TC is a comprehensive image feature value of the feature values of a plurality of images to which the same tag is assigned, and can be a feature value distribution of each image, for example.
By generating the tag feature quantity TC, tag search can be performed in the search processing described later. For example, in tag search, a search result similar to the entire image to which a tag is attached can be obtained. That is, it becomes possible to search for an image that is totally similar to a plurality of images to which a specific tag is assigned from among images uploaded by the user and images published by other users. That is, the tag feature TC is an index that characterizes the user classification method, and by extracting a search result in accordance with the concept of the user to which the tag is added by performing the tag search from the image to which the tag is not added. Will be able to.

(4-3) Search history:
Further, in the present embodiment, a search that is weighted by learning data can be performed. The learning data here is a search history, a tag attached to the image by the user, and the like. Below, each learning data utilized for a search is demonstrated first.

  First, the search history will be described. The search history is information indicating a user's search tendency, and indicates a correlation between a query (keyword or key image) used as a search key by the user and a search result actually selected from the search result by the user. It is information. This is because when the search is performed, the query and the image selected by the user should be correlated in the user's image. By using this correlation, the search accuracy can be improved. Since the search history is automatically saved each time a search is performed, the user's search tendency is gradually learned, and a statistically accurate search can be performed. In the present embodiment, the following search history databases KWDB and KIDB as data related to the search history are stored in the storage unit 34. In the present embodiment, the storage unit 34 constitutes a search history storage unit.

FIG. 9 is a table for explaining an example of a search history database KWDB for keyword search. The search history shown in the figure is configured by the correspondence relationship between the search date and time, the keyword, and a plurality of image feature amounts. The plurality of image feature amounts are image feature amounts of images selected by the user from the search results when searching with corresponding keywords.
After that, when performing a keyword search using the search history, the search history database KWDB is referred to, and if the same keyword exists, an image similar to the feature amount of the image associated with the keyword is searched. . At this time, if the images selected by the user when searching with the same keyword are different, the history of the oldest search date is deleted and the search history database KWDB is updated with the new search history, or the selected old and new images It is also possible to obtain a new search history by arithmetically averaging or weighting the feature quantities. By searching for keywords using the history in this way, you can search more accurately reflecting the user's image.

FIG. 10 is a table for explaining an example of the search history database KIDB for key image search. The search history shown in the figure is composed of a search date and a weight coefficient for each image feature amount. This weighting coefficient indicates the degree of coincidence between the key image used as the query and the selected image selected by the user from the search results. That is, the feature amount is compared between the key image and the selected image, and the image feature amount with a higher degree of overlap is determined to be the feature amount that the user is paying attention to. The image feature amount with a higher degree of overlap has a larger weight coefficient. Then, the weighting coefficient is reduced as the image feature amount has a lower degree of overlap.
Based on the weighting coefficient calculated in this way, the image feature amount of the key image used in the subsequent search is weighted and used for the search, and the search reflecting the user image can be performed more accurately. It becomes like this. In the keyword search, the weighting coefficient may be weighted to the feature amount of the image associated with the keyword before searching.

(4-4) Image processing:
When the image to be processed is extracted as described above, various image processes are executed on the image along the flow of FIG. 5 and output. First, a feature amount is acquired from the extracted image (S230). Next, an image processing policy is determined based on the extracted feature amount, and image processing is performed on the image according to the policy (S235). The data utilization unit M4 that executes the above step S235 constitutes an image processing unit in the present embodiment.

  Then, the image after image processing is output to the display unit 11 of the PC 10 (S240), and the user confirms it (S245). In step S245, if the user is satisfied with the image processed image, the process proceeds to step S255 (S245: OK). If the user is not satisfied with the image processed image, the process proceeds to S250 to adjust the image processing parameters. The UI is displayed on the display unit 11 of the PC 10 (S245: NG). In step S250, the user operates the UI with the operation input unit 12 and adjusts the image processing parameters by himself until a desired result is obtained.

  In step S255, the image processing parameter is stored in the corresponding user area of the management server 30. That is, image processing parameters are accumulated every time output is performed. Therefore, by changing the image processing based on the accumulated image processing parameters, it is possible to perform image processing according to each user's preference. The image processing parameters for each user created in this way are stored in the storage unit 34. In the present embodiment, the storage unit 34 constitutes an image processing history storage unit.

  In step S260, the image processed image is output to the output device selected in step S205. At this time, color conversion is performed in consideration of the output characteristics of the display unit 11 of the PC 10 and the output characteristics of the printer 20. For example, an ICC profile of the PC 10 is used as a source profile, an ICC profile of the printer 20 is used as a destination profile, and a color conversion profile in which these profiles are connected with a type of gamut mapping specified as an image processing parameter by an image processing setting is used. Perform conversion. In this way, the data is converted into data conforming to the gamut of the output device, and the data is further converted into a data format that can be accepted by the output device and output.

  In the image processing described above, for example, image correction may be performed in consideration of the user profile input on the “profile input screen” in FIG. That is, the level of correction having a difference in direction depending on sex and age is determined based on the profile. For example, if you are a woman, make your skin smoother or make the skin whiter, or increase the intensity of skin correction or whitening, and if you are a man, you may or may not reduce the intensity of these corrections. To do. In this way, by changing the strength of the correction using the profile, it is possible to realize a correction that can provide an output result preferable for the user. The profile input by the user is stored in the storage unit 34. In the present embodiment, the storage unit 34 constitutes a profile storage unit.

(5) Various modifications:
The management server 30 described above may store data copies in a distributed manner, for example, by raid or mirroring in order to prevent the loss of images stored in the storage unit 34. By doing so, the user can accumulate images with peace of mind.

  Further, the above-described output device may not be a user's individual. For example, a photo server is prepared on the network NT, and the user obtains access authority for the photo server from the photo server management company. By registering the photographic server as an output device, the photographic server can be used as an output device from various input devices. Of course, such a licensing-type output device is not limited to a printing device that prints a photograph, and can be performed by the various output devices described above.

  In the search processing described above, search using a keyword, a key image, or a tag has been described. However, the search may be performed using a time stamp of the image. In this way, images created during a specific period can be extracted and processed together.

  Although the above-described printing condition parameters and image processing parameters are set by each user, the parameters of other users may be applied to these parameters. For example, a user with a long camera use history is selected as an expert based on the profile input by the user, and the printing condition parameters and image processing parameters of the expert are collected. For a user whose input / output environment matches the expert, the expert's printing condition parameters and image processing parameters can be applied as default parameters. In this way, even a beginner can obtain an appropriate output subjected to image processing that reflects the know-how of an expert.

(6) Summary:
According to the embodiment described above, in the image management system including the image management server 30 including the storage unit 34 for storing a plurality of images and the PC 10 accessible to the image management server 30 via the network NT, The PC 10 can input search conditions to the image management server 30 via the UI screen evaluated by the display unit 11 by the image management server 30, and the image management server 30 associates the image with the image from a plurality of images. Based on the search history and image feature quantity stored in the storage unit 34, search for an image that matches the search condition, or based on information that associates the search history, image feature quantity, output device and image feature quantity. A data utilization unit M4 that executes image processing. Therefore, it is possible to perform processing in consideration of the output on the server while accumulating images on the server.

  Note that the present invention is not limited to the above-described embodiments and modifications, and the configurations disclosed in the above-described embodiments and modifications are mutually replaced, the combinations are changed, the known technology, and the above-described implementations. Configurations in which the configurations disclosed in the embodiments and modifications are replaced with each other or combinations are also included.

DESCRIPTION OF SYMBOLS 10 ... Personal computer, 11 ... Display part, 12 ... Operation input part, 20 ... Printer, 30 ... Management server, 31 ... Control part, 32 ... Operation input part, 33 ... Display part, 34 ... Storage part, 35 ... I / F, 36 ... storage medium, 100 ... image management system, M1 ... setting registration unit, M2 ... image upload unit, M3 ... image analysis unit, M4 ... data utilization unit, NT ... network

Claims (9)

An image management system comprising an image management server comprising a storage medium for storing a plurality of images, and a client accessible to the image management server via a network,
The client includes an input unit for inputting search conditions to the image management server,
The image management server includes a search unit that searches for an image that matches the search condition based on the output condition and the feature amount of the image stored in the storage medium in association with the image from the plurality of images. An image management system comprising: An image management system comprising an image management server comprising a storage medium for storing a plurality of images, and a client accessible to the image management server via a network,
The client includes a first selection unit that selects an image of the storage medium,
The image management server associates an output condition stored in the storage medium in association with the image, an image feature amount, and an output device and the image feature amount with respect to the image selected by the first selection unit. An image management system comprising an image processing unit that executes image processing based on the received information. The image management server includes: a search unit that searches for an image based on an output condition, an image feature amount, and the search condition stored in the storage medium in association with the image from the plurality of images; A reception unit for receiving information,
The retrieval unit retrieves, from a plurality of images stored in the storage medium, an image whose distance from an image feature amount created based on the key information is within a predetermined distance in an image feature amount space. The image management system according to claim 1 or 2. The image management server further includes a counting unit that counts the number of searches of the search unit,
The image management system according to claim 3, wherein the search unit increases an information amount of the image feature amount used for the search as the number of searches increases. The client includes a second selection unit that selects an image from the search result of the search unit,
The image management server includes a search history storage unit that stores a correlation between the key information and the image selected by the second selection unit as a search history,
The image management system according to any one of claims 1 to 4, wherein the search unit searches for an image having a high degree of correlation in the search history. The image management server associates an output condition stored in the storage medium in association with the image, an image feature amount, and an output device and the image feature amount with respect to the image selected by the first selection unit. An image processing unit that executes image processing based on the received information, and an image processing history storage unit that stores a history of image processing performed by the image processing unit on an image selected by a specific client,
The image management system according to claim 1, wherein the image processing unit executes image processing of another client based on the history of the image processing. The image management server includes a profile storage unit that stores a profile of the user of the client,
The image management system according to claim 1, wherein the image processing unit determines an intensity of image processing based on the profile.   The image management system according to claim 1, wherein the client is a printing device, and the printing device prints an image searched by the search unit.   The image management system according to claim 2, wherein the client is a printing apparatus, and the printing apparatus prints an image that has been subjected to image processing by the image processing unit.

Images