JP7375091B2 - Image processing device, image processing method, program and recording medium - Google Patents

Description

The present invention relates to an image processing device, an image processing method, a program, and a recording medium that search for an image from a group of images based on search conditions and display the image as a result of the search on a display unit.

For example, when a user searches for a desired image from among a group of images held on a smartphone, the user displays a list of images (thumbnail images) included in the image group in chronological order on the display section, and A method is used in which a desired image is searched for while going back through the list in chronological order. Furthermore, a method is known in which a keyword for searching for an image from a group of images is input, and additional information of the image such as tag information is used to search for an image matching the keyword.

However, with the method of displaying a list of images and going back, it is extremely difficult to search for a desired image from among a large number of images included in a group of images. In addition, with the method of searching for images by inputting keywords, if it is clear what kind of keywords to use, it may be possible to properly search for the desired image, but if the keywords are clearly Otherwise, it is very difficult to properly search for a desired image.

Here, as prior art documents relevant to the present invention, there are Patent Documents 1 and 2.

Patent Document 1 describes that image data is searched by performing a character string search based on additional information added to the image data. For example, by performing an AND/OR search between the Exif character string you want to search and the sticky note character string you want to search, you can search for image data that has additional information that matches the input character string, It is stated that image data corresponding to the search results will be displayed.

Patent Document 2 discloses that a plurality of attributes and attribute values related to a digital image are displayed on a label, and a user selects another set of images by selecting one of the attribute values displayed on the label. is listed. Also, while viewing a selected set of images, the user can search for all images of a particular person, display those images in chronological order, and display their associated labels. Are listed.

Japanese Patent Application Publication No. 2009-124270 Patent No. 5216591

Patent Document 1 describes performing an AND search of multiple character strings of additional information, but the user has to input the character string he or she wants to search, making it impossible to search for images intuitively. .
Further, Patent Document 2 describes that by selecting one of the attribute values of one digital image, a set of other images can be viewed, but images that correspond only to one attribute value You can only view a collection of images, and cannot narrow down the images by, for example, multiple attribute values.

Furthermore, some conventional online photo services have a function of sending a user a notification urging the user to view images, and urging the user to view past images that the user owns.

After a notification prompting the user to view images is sent to the user, when the user opens this notification, for example, from among the images of "2015/01/10" owned by the user, as shown in Figure 20. A collage image created using the selected several images is displayed on the display section.

Next, when the user presses the button "Show other photos" displayed at the bottom of the collage image, as shown in Figure 21, the user will be able to see the photos taken on the same day as the images used in the collage image. A group of images for "January 10, 2015 (Saturday)" are displayed on the display unit in descending order of the image shooting time.

Furthermore, by moving the image group displayed on the display unit upward or downward, the user can sequentially display the image groups taken on the previous and subsequent shooting dates in the order of the image shooting time. can.
For example, when the group of images displayed on the display is moved upward, the most recent shooting date before "Saturday, January 10, 2015", for example, "January 9, 2015 ( A group of images of "Gold" are displayed on the display section. On the other hand, when the group of images displayed on the display section is moved downward, the most recent shooting date after "Saturday, January 10, 2015", for example, "January 18, 2015 ( A group of images of ``day)'' are displayed on the display section.

In this way, with conventional online photo services, by following the "Show other photos" button from a collage image, you can view a group of images taken on the same day as the images used in this collage image. There is a function to display in series.

The collage image included in the notification prompting to view the image was searched using the search condition related to the time the image was taken, ``Saturday, January 10, 2015,'' and the date of the image was ``Saturday, January 10, 2015.'' It was created using several images selected from a group of images. Therefore, from a collage image, it is possible to display in chronological order a group of images with the shooting date "Saturday, January 10, 2015" that were taken before and after the shooting time of the image used in this collage image. can.
However, conventional online photo services display images taken before and after the shooting time of one of the images searched based on arbitrary conditions other than the shooting time of the image in chronological order. There is no function to do so.

In addition, in conventional online photo services, when searching for an image that matches a keyword from among a group of images, the group of images that match that keyword are displayed on the display in descending order of the shooting time of the image by shooting date. Ru.
For example, if you search for images that match the keyword "soccer" from the image group, as shown in Figure 22, from the image group "December 5, 2015 (Saturday)", 9 that match the keyword "soccer" will be found. 2 images are hit.

Here, if the "down arrow" button displayed on the right side of the display of "Saturday, December 5, 2015" is pressed, the "December 5, 2015" button will be displayed, including images unrelated to soccer. All images included in the "Sunday (Saturday)" image group are displayed, and the "downward arrow" button is changed to an "upward arrow" button. When the "upward arrow" button is pressed, nine images corresponding to soccer are displayed again, and the display returns to the "downward arrow" button.

As shown in Figure 22, when searching for the keyword "soccer", images matching "soccer" were found before "Saturday, December 5, 2015" when images matching "soccer" were found. The shooting date that became a hit is "November 28, 2015."

Now, return to the chronological image list screen where all images included in the image group are displayed in order of shooting time, and if you look at the image group of "Saturday, December 5, 2015", the chronological In the image list, the shooting date of the next image after "Saturday, December 5, 2015" is "Sunday, December 6, 2015" and "Saturday, December 5, 2015" )” is assumed to be “December 4, 2015 (Friday)”.

In this way, in conventional online photo services, when you search for images from a group of images using the keyword "soccer," you can search for images that have images corresponding to "soccer" for each shooting date. There is a function to display.

However, with conventional online photo services, with a single action such as pressing a button, images taken before and after the shooting time of one of the images are displayed in chronological order from a list of images taken on hit shooting dates. No display function is provided. In other words, the user must memorize the shooting date that the user wants to display in the list of time-series images, display the list of time-series images, and move to the shooting date that the user wants to display.

An object of the present invention is to display, in chronological order, images taken before and after the photographing time of one image in a list of images searched based on conditions other than the photographing time of the image. The object of the present invention is to provide an image processing device, an image processing method, a program, and a recording medium that can perform the following steps.

In order to achieve the above object, the present invention includes an instruction acquisition unit that acquires instructions input by a user;
an image search unit that searches, as a search image, an image matching a search condition input in response to an instruction from a user from among the image group;
an image display unit that displays images included in the image group on a display unit;
an image specifying section for specifying one image selected from among the images displayed on the display section according to an instruction from a user;
The search condition is a condition for searching for an image from a group of images based on conditions other than the shooting time.
The image designation unit designates one search image selected from among the search images displayed on the display unit according to an instruction from the user as a designated search image;
When the specified search image is specified, the image display section displays only the specified search image on the display section, and then displays the specified search image and the specified search images included in the image group in accordance with instructions from the user. To provide an image processing device that displays at least a part of images photographed before and after a photographing time as time-series images on a display unit in chronological order.

Here, after displaying only the specified search image on the display section, the image display section selects a photograph from among the search images before or after the shooting time of the specified search image every time an instruction is input from the user. It is preferable to display only one search image that has been searched in sequence on the display unit in the order of search images shot in chronological order starting from the shooting time of the specified search image.

Further, the image specifying unit specifies one time-series image selected from among the time-series images displayed on the display unit according to an instruction from the user as a designated time-series image,
When the specified time-series images are specified, the image display section displays only the specified time-series images on the display section, and then displays the specified time-series images from among the image group each time an instruction from the user is input. It is preferable that only one image photographed before or after the photographing time of the image is sequentially displayed on the display unit in the order of images photographed in chronological order starting from the photographing time of the designated time-series images.

In addition, the image search section
a tag information display unit that displays tag information attached to images included in the image group on a display unit;
a tag information designation section that designates tag information selected from among the tag information displayed on the display section as selected tag information according to an instruction from a user;
an image extraction unit that extracts an image to which selection tag information is attached from the image group as a search image;
The tag information display section causes the display section to display at least part of all the tag information attached to all the images included in the image group,
The tag information designation unit designates the first tag information selected from among the tag information displayed on the display unit according to an instruction from the user as the first selected tag information,
The image extraction unit extracts an image to which the first selection tag information is attached from the image group as a first search image;
The image display unit displays at least a portion of all the first search images on the display unit,
It is preferable that the tag information display section displays at least part of all the tag information given to all the first search images on the display section.

Further, in addition to the first selected tag information, the tag information specifying section selects second selected tag information from among the tag information displayed on the display section according to an instruction from the user. Specify at the same time as information,
The image extraction unit extracts the first search image to which the second selection tag information is attached as a second search image from among the first search images;
The image display unit displays at least a portion of all the second search images on the display unit,
It is preferable that the tag information display section displays at least part of all the tag information given to all the second search images on the display section.

The tag information designation unit simultaneously designates two or more pieces of tag information selected from among the tag information displayed on the display unit as selected tag information according to instructions from the user;
The image extraction unit extracts, as a search image, an image to which all two or more pieces of selection tag information are attached from the image group;
The image display unit displays at least a portion of all the search images on the display unit,
It is preferable that the tag information display section displays at least part of all the tag information given to all the search images on the display section.

The present invention also provides a step in which the instruction acquisition unit acquires instructions input by the user;
a step in which the image search unit searches the image group for an image that matches a search condition input in response to an instruction from a user as a search image;
a step in which the image display unit displays an image included in the image group on the display unit;
The image specifying unit specifies one image selected from among the images displayed on the display unit in response to an instruction from a user,
The search condition is a condition for searching for an image from a group of images based on conditions other than the shooting time.
From among the search images displayed on the display unit, one search image selected according to instructions from the user is designated as a specified search image,
When a specified search image is specified, only the specified search image is displayed on the display section, and then, according to instructions from the user, images are displayed before and after the shooting time of the specified search image and the specified search images included in the image group. To provide an image processing method for displaying at least a part of images photographed on a display unit in chronological order as time-series images.

Further, the present invention provides a program for causing a computer to execute each step of the above-described image processing method.

Further, the present invention provides a computer-readable recording medium on which a program for causing a computer to execute each step of the above-described image processing method is recorded.

The present invention also provides an instruction acquisition unit that acquires an instruction input by a user;
an image search unit that searches, as a search image, an image matching a search condition input in response to an instruction from a user from among the image group;
an image display unit that displays images included in the image group on a display unit;
an image specifying section for specifying one image selected from among the images displayed on the display section according to an instruction from a user;
The instruction acquisition unit, image search unit, image display unit, and image designation unit are hardware or a processor that executes a program,
The search condition is a condition for searching for an image from a group of images based on conditions other than the shooting time.
The image designation unit designates one search image selected from among the search images displayed on the display unit according to an instruction from the user as a designated search image;
When the specified search image is specified, the image display section displays only the specified search image on the display section, and then displays the specified search image and the specified search images included in the image group in accordance with instructions from the user. To provide an image processing device that displays at least a part of images photographed before and after a photographing time as time-series images on a display unit in chronological order.

Here, the image search section is
a tag information display unit that displays tag information attached to images included in the image group on a display unit;
a tag information designation section that designates tag information selected from among the tag information displayed on the display section as selected tag information according to an instruction from a user;
an image extraction unit that extracts an image to which selection tag information is attached from the image group as a search image;
The tag information display section, the tag information specification section, and the image extraction section are hardware or a processor that executes a program,
The tag information display section causes the display section to display at least part of all the tag information attached to all the images included in the image group,
The tag information designation unit designates the first tag information selected from among the tag information displayed on the display unit according to an instruction from the user as the first selected tag information,
The image extraction unit extracts an image to which the first selection tag information is attached from the image group as a first search image;
The image display unit displays at least a portion of all the first search images on the display unit,
It is preferable that the tag information display section displays at least part of all the tag information given to all the first search images on the display section.

According to the present invention, in order to display images shot before and after the shooting time of the first search image, the user does not need to return to the image list screen once, but can display the specified images according to the user's instructions. Images taken before and after the time when the searched image was taken can be displayed in chronological order for viewing.

1 is a block diagram of an embodiment showing the configuration of an image processing system according to the present invention. FIG. 2 is a block diagram of one embodiment representing the configuration of a client. 3 is a flowchart of one embodiment depicting the operation of a client. FIG. 2 is a conceptual diagram of an embodiment showing a home screen of an application displayed on a display unit of a client. FIG. 2 is a conceptual diagram of an embodiment showing a designated image viewing screen. FIG. 2 is a conceptual diagram of an embodiment showing a viewing screen for information on a designated image. FIG. 2 is a conceptual diagram of an embodiment showing an editing screen for information on a designated image. FIG. 2 is a conceptual diagram of an embodiment showing an input screen for new tag information. FIG. 2 is a conceptual diagram of an embodiment showing a tag information selection screen. FIG. 2 is a conceptual diagram of an embodiment showing how the first tag information is selected on the tag information selection screen. FIG. 3 is a conceptual diagram of an embodiment showing how the content of tag information is updated according to the first tag information on the tag information selection screen. FIG. 7 is a conceptual diagram of an embodiment showing how second tag information is selected on the tag information selection screen. FIG. 2 is a conceptual diagram of an embodiment showing an image list screen. FIG. 2 is a conceptual diagram of an embodiment showing a state in which one search image is selected on an image list screen. FIG. 2 is a conceptual diagram of an embodiment showing a viewing screen for designated search images. FIG. 2 is a conceptual diagram of an embodiment showing a list screen of time-series images. FIG. 2 is a conceptual diagram of an embodiment showing a state in which a "search tag" button is tapped on a tag information selection screen. FIG. 2 is a conceptual diagram of an embodiment showing a search screen for tag information. FIG. 2 is a conceptual diagram of an embodiment showing a tag information selection screen. It is a conceptual diagram of an example showing a collage image created using several images selected from a group of images of “2015/01/10” owned by a user. It is a conceptual diagram of an example showing a group of images taken on "Saturday, January 10, 2015", the same shooting date as the images used in the collage image. FIG. 2 is a conceptual diagram of an example of an image that corresponds to the keyword "soccer."

DESCRIPTION OF THE PREFERRED EMBODIMENTS An image processing apparatus, an image processing method, a program, and a recording medium of the present invention will be described in detail below based on preferred embodiments shown in the accompanying drawings.

FIG. 1 is a block diagram of an embodiment showing the configuration of an image processing system according to the present invention. The image processing system 10 shown in FIG. 1 includes a server 12 and a plurality of clients 14 each connected to the server 12 via a network 16.

The server 12 acquires a group of images owned by each of the plurality of users who use the image processing system 10, holds each image included in the image group owned by each user, and allows the images to be downloaded by each user. It has a function to share only with specified sharing partners.
The server 12 is not limited to one, but may be a plurality of servers, and is constituted by a workstation or the like having a control device, a storage device, a communication device, and the like.

The client 14 is an image processing device according to the present invention, and searches images from a group of images owned by the user based on a search condition, more specifically, using tag information attached to an image as a search condition for the image. It has a function of searching and displaying the image of the search result on the display unit.
The client 14 is configured by a desktop PC (personal computer), a notebook PC, a tablet PC, or a mobile terminal such as a mobile phone or a smartphone, which has a control device, an input device, a storage device, a communication device, a display, and the like.
Hereinafter, a case where the client 14 is a smartphone will be described as an example.

FIG. 2 is a block diagram of one embodiment representing the configuration of a client. The client 14 shown in FIG. 2 includes an instruction acquisition section 18, an image holding section 20, an image analysis section 22, an evaluation value assignment section 24, a tag information assignment section 26, a weighting setting section 28, and a tag information display section. 30, tag information specifying section 32, image extracting section 34, tag information searching section 36, image display section 38, image specifying section 40, image information display section 42, image information editing section 44, display A section 46 is provided.

In the client 14, the instruction acquisition unit 18 first acquires an instruction input by the user of the client 14.

The user can input various instructions by touching the touch panel of the smartphone. Note that instructions can be input using not only the touch panel but also input devices such as a mouse, keyboard, and track pad.
Instructions from the user include instructions to select tag information, instructions to select images, instructions to input tag information to be added to images, instructions to input search conditions (keywords) for tag information, although details will be described later. This includes instructions for displaying an image on the display unit 46, and the like.

Next, the image holding unit 20 is a storage device such as a semiconductor memory, and holds a group of images.

The image group is taken by a user using a smartphone camera and is held inside the smartphone. Note that the image group is not particularly limited, and may include, for example, a group of images imported into a smartphone from the outside, a group of online images stored in SNS (Social Network System), online storage, etc. It may also be a combination of these image groups.

For example, images taken with a smartphone camera can be set to be automatically uploaded from the smartphone to the server 12. Images uploaded to the server 12 are held in each user's storage area within the storage device of the server 12. The images uploaded to the server 12 can be set to remain in the image holding section 20 of the smartphone, or can be set to be deleted.
In addition, for example, images taken with a digital camera, images stored on a PC, notebook PC, tablet PC, etc., and images scanned from photo prints (digital image data), etc. cannot be imported into the smartphone. can. Images imported into the smartphone from the outside are held in the image holding unit 20 and similarly handled as part of the image group.

Subsequently, the image analysis unit 22 analyzes each image included in the image group.

Image analysis items are not particularly limited, but include, for example, detection of a subject in an image (person detection and object detection), detection of a person's face, detection of a person's facial expression, detection of a person's actions, and scene detection (night scene detection). , sea, beach, sky,...), event detection (sports day, wedding, graduation ceremony,...), and analysis of image content, such as detecting user preferences (hobbies), image brightness, hue, contrast, and blurring. This includes analysis of the image quality, such as the degree of image quality, and analysis of time information (shooting time) and location information (shooting location) included in additional image information such as Exif (Exchangeable Image File Format).

Subsequently, the evaluation value assigning unit 24 assigns an evaluation value to each image based on the results of the analysis of each image.

The method by which the evaluation value assigning unit 24 assigns an evaluation value to an image is not particularly limited, but for example, it assigns a score to the result of each analysis of the image, calculates a total score of the scores of the results of a plurality of analyses, and calculates the total score. can be given as the evaluation value of the image.
For example, an image in which a person is smiling is given a higher score than a crying face, an angry face, and the like. Furthermore, an image with appropriate brightness is given a higher score than an image that is too bright or an image that is too dark.
In addition, the evaluation value assigning unit 24 assigns tag information selected later than an image assigned tag information selected earlier in terms of time, according to settings for weighting tag information to be described later. set the evaluation value of the image to be high.

Subsequently, the tag information adding unit 26 adds tag information to each image based on the results of the analysis of each image.

Tag information is a variety of information related to images, such as the date and location of the image, the name of the subject in the image, the name of the subject's actions, scenes, events, and emotions of the person that can be read from the image. A name representing a user's preference, a name representing a user's preference, etc. can be exemplified. Tag information is text information, and it is possible to search for tag information using keywords.
The tag information may include tag information of a superordinate concept that has tag information of a plurality of subordinate concepts. For example, it may include tag information for the superordinate concept "sweets" that has tag information for two subordinate concepts "Mont Blanc" and "strawberry shortcake."
Further, the tag information may include reliability information indicating the reliability of the tag information. Although the content of automatically assigned tag information may not be accurate, it is possible to judge the accuracy of the content of tag information based on reliability information.
Although the tag information adding unit 26 is not an essential component, it may add additional tag information to the tag information that has been added to the image in advance, or it may add new tag information to an image to which no tag information has been added. You may.

Subsequently, the weighting setting unit 28 sets weighting for the tag information.

The method by which the weighting setting unit 28 sets weighting for tag information is not particularly limited, but when a user searches for an image using two or more pieces of tag information, the tag selected earlier in terms of time is used. It is considered that there is a high possibility that the tag information selected later is more important than the tag information. In response to this, the weighting setting section 28 selects one of the two or more pieces of tag information simultaneously selected from among the tag information displayed on the display section 46 according to an instruction from the user, which is earlier in time. It is possible to set the weighting of tag information selected later to be heavier than the selected tag information.

Subsequently, the tag information display section 30 causes the display section 46 to display the tag information added to the images included in the image group.

Although the method by which the tag information display section 30 displays tag information is not particularly limited, for example, the tag information displayed on the display section 46 can be classified into a plurality of categories and displayed. As a result, the tag information can be classified and displayed for each type of tag information, so the user can easily select the tag information that he/she wishes to use as an image search condition from among the tag information displayed on the display section 46. can be found in.

In this case, the tag information display section 30 can display the tag information included in at least one of the plurality of categories in descending order of the number of images to which tag information has been added. Thereby, tag information attached to many images can be displayed preferentially. Note that tag information represented by consecutive numbers, such as the photographing date of an image, may be displayed in descending or ascending numerical order.

Further, the tag information display section 30 may display tag information included in at least one of a plurality of categories in descending order of the number of times the tag information has been used as a keyword for searching for an image. When having reliability information, tag information included in at least one of the plurality of categories may be arranged and displayed in descending order of reliability based on reliability information included in the tag information.

Furthermore, when the tag information of a superordinate concept is selected in response to an instruction from the user, the tag information display section 30 causes the display section 46 to display tag information of a plurality of subordinate concepts included in the tag information of the superordinate concept. be able to.
Further, the tag information display section 30 can display the tag information as a button (link) on the display section (touch panel) 46. This allows the user to easily select tag information.

Subsequently, the tag information specifying unit 32 specifies the tag information selected from among the tag information displayed on the display unit 46 as the selected tag information according to an instruction from the user.
Further, the tag information specifying section 32 specifies search tag information selected from among the search tag information displayed on the display section 46 in response to an instruction from the user as selected tag information.

Subsequently, the image extracting unit 34 extracts, as a search image, an image to which the selected tag information specified by the tag information specifying unit 32 is added from the image group.

Here, the tag information display unit 30, the tag information designation unit 32, and the image extraction unit 34 are configured to search for images that match the search conditions input in response to instructions from the user as search images from among the image group. This constitutes an image search unit of the invention.

Next, the tag information search unit 36 selects tag information that at least partially matches the keyword input in response to an instruction from the user, from among all the tag information added to all the images included in the image group. Search as search tag information.

Subsequently, the image display section 38 causes the display section 46 to display the images included in the image group.

The image display unit 38 displays, for example, each image included in the image group, a list of images included in the image group (thumbnail images), a list of searched images (thumbnail images), and the like.
When displaying a list of images or a list of searched images on the display unit 46, the image display unit 38 can display the images chronologically in the order of their photographing times. Further, when each image included in the image group has information on the image evaluation value, the image display section 38 can display the images on the display section 46 in order of the image evaluation value.

Subsequently, the image specifying section 40 specifies one image selected from among the images displayed on the display section 46 in response to an instruction from the user.

The user can select one image that he/she wishes to view from the list of images displayed on the display section 46. When one image is selected, the image display section 38 displays only the selected one image on the display section 46.

Subsequently, the image information display section 42 causes the display section 46 to display information about the one image specified by the image specifying section 40, that is, the one image selected according to the instruction from the user.

The image information includes various types of information regarding the image. In the case of this embodiment, the image capture time and tag information attached to the image are included. Note that the information is not limited to this, and may include information such as the image file name, image size, resolution, and shooting location.

Subsequently, the image information editing section 44 edits the information of one image displayed on the display section 46 by the image information display section 42 in accordance with instructions from the user.

The user can edit tag information etc. given to the image. There is a possibility that the tag information automatically added by the tag information adding section 26 includes inaccurate tag information. The user can modify or delete tag information that is considered inaccurate.
The user can also add new tag information to images. In this case, the image information editing section 44 adds new tag information input in response to an instruction from the user to one image as information about the one image displayed on the display section 46.

Subsequently, the display section 46 displays various information.

The display unit 46 displays images, image information, a list of images, a list of tag information, and the like. The display unit 46 is a liquid crystal panel, an organic EL (Electro Luminescence) panel, or the like, and in the case of this embodiment, is a touch panel.

Next, with reference to the flowchart shown in FIG. 3, the operation when the client 14 searches for a desired image from a group of images using tag information attached to an image as a search condition will be explained. .

First, the operation when the first tag information is selected will be explained.
In this case, first, the tag information display section 30 displays on the display section 46 at least a portion of all the tag information added to all the images included in the image group (step S1).
When the tag information is displayed on the display section 46, the tag information specifying section 32 then selects the first tag from among the tag information displayed on the display section 46 according to an instruction from the user. The information is designated as first selected tag information (selected tag information) (step S2).
When the first selection tag information is specified, the image extraction unit 34 extracts the image to which the first selection tag information has been added from the image group as a first search image (search image). Step S3).
After the first search images are extracted, the image display section 38 displays at least a portion of all the first search images on the display section 46 (step S4).
Subsequently, the tag information display section 30 displays at least a portion of all the tag information added to all the first search images on the display section 46 (step S5).

Next, the operation when the second tag information is simultaneously selected in addition to the first tag information will be described.
In this case, the tag information specifying section 32 then selects the second tag selected from the tag information displayed on the display section 46 in addition to the first selected tag information according to the instruction from the user. The information is simultaneously designated as second selected tag information (selected tag information) (step S6).
Subsequently, the image extraction unit 34 extracts the first search image to which the second selection tag information has been added as a second search image (search image) from among the first search images (step S7).
Subsequently, the image display section 38 displays at least a portion of all the second search images on the display section 46 (step S8).
Subsequently, the tag information display unit 30 displays at least a portion of all the tag information added to all the second search images on the display unit 46 (step S9).

Note that the operation is similar when three or more pieces of tag information are selected at the same time. In other words, the operation when two or more pieces of tag information are selected at the same time is as follows.
First, the tag information specifying section 32 simultaneously specifies two or more pieces of tag information selected from among the tag information displayed on the display section 46 as selected tag information according to an instruction from a user.
Subsequently, the image extraction unit 34 extracts, from the image group, an image to which all two or more pieces of selection tag information have been added as a search image.
Subsequently, the image display section 38 displays at least a portion of all the search images on the display section 46 .
Subsequently, the tag information display section 30 displays at least a portion of all the tag information added to all the search images on the display section 46.

The image processing apparatus of the present invention will be described below, taking as an example a case where the client 14 is implemented by an application program (hereinafter also simply referred to as an application) running on a smartphone.

FIG. 4 is a conceptual diagram of an embodiment showing a home screen of an application displayed on a display unit of a client. The display unit 46 of the smartphone shown in FIG. 4 is composed of a touch panel, and by touching the touch panel, the user can use the display unit (touch panel) 46 not only to display various information but also to receive input from the user. It can also be used as an instruction acquisition unit 18 that acquires various instructions.

A notification and status area 50 is displayed at the top of the home screen, and a menu area 52 is displayed below it. Furthermore, a navigation area 54 is displayed at the bottom of the home screen.

Within the navigation area 54, buttons ``Search,''``Collect,''``Home,''``Share,'' and ``Various'' are displayed in order from left to right.
When the "Search" button is tapped (pressed down), a tag information selection screen is displayed on the display unit 46.
Furthermore, when the "Combine" button is tapped, a composite image such as a photo album is automatically created using the images included in the image group.
When the "home" button is tapped, a home screen, that is, a list of images included in the image group is displayed on the display unit 46. In the list of images, the images are arranged and displayed, for example, in the order of shooting time (descending order or ascending order).
When the "Share" button is tapped, the selected image is shared with other users in accordance with the user's instructions.
When the "Various" button is tapped, various other functions not displayed in the navigation area can be selected and executed.

A list of images is displayed in the area between the menu area 52 and the navigation area 54. Before searching for an image, if no tag information is selected as a search condition, the image list screen will display a list of images, for example at least some of all the images included in the image group will be displayed when the image was taken. The images are displayed in chronological order from the top to the bottom of the image list screen in descending order of shooting time in this embodiment.

As shown by the finger mark 56 in FIG. 4, the user taps one image that he or she wishes to view from among the images displayed on the image list screen, and displays only the tapped image on the display unit 46. You can also view it.

In this case, the image specifying section 40 specifies one image selected from the list of images displayed on the display section 46 in response to an instruction from the user, and the image display section 38 specifies the specified image. Only the image is displayed on the display section 46.

FIG. 5 is a conceptual diagram of one embodiment showing a designated image viewing screen. The designated image viewing screen is displayed on the entire surface of the display unit 46. As shown in FIG. 5, an "x" button 58 is displayed at the upper left of the designated image viewing screen, and an "information" button 60 is displayed at the lower right. At the bottom of the designated image viewing screen, a "trash can" button 62, a "share" button 64, and the like are displayed.

For example, by tapping the "x" button 58, the user can close the designated image viewing screen and return to the image list screen (home screen). The user can also tap the "trash can" button 62 to delete the designated image, or tap the "share" button 64 to share the designated image with other users. Further, by tapping the "Information" button 60, as shown by the finger mark 56 in FIG. 5, the user can flip over the designated image and view the information of the designated image.

FIG. 6 is a conceptual diagram of an embodiment showing a viewing screen for information on a designated image. An "x" button 66 is displayed at the top left of the viewing screen for information on the specified image, and an "edit" button 68 is displayed at the top right. In addition, a thumbnail image of the specified image is displayed at the top of the screen for viewing information about the specified image, and below it, the shooting time of the specified image, tags added automatically, tags added by yourself, and " "Add Tag" buttons 70 are displayed in this order.

Automatically added tags are categorized into calendar, person, shooting location, automatically added tags, and service tags, while tags you add yourself are categorized into manually added tags.
The calendar category includes buttons for tag information of "2018" and "December" as the year and month of photography of the specified image.
The person category includes a button for tag information of "Taro."
The photography location category includes tag information buttons for "park,""MinatoWard," and "Odaiba Seaside Park."
The automatically assigned tag category includes tag information buttons for "outdoors,""boy," and "sunny."
The scan tag category includes a "scan service" tag information button.
The manually assigned tag category includes a button for tag information of "Kids Coordination".

For example, the user can close the designated image information viewing screen and return to the designated image viewing screen by tapping the "x" button 66. Further, the user can edit the information of the specified image by tapping an "edit" button 68, as indicated by the finger mark 56.

FIG. 7 is a conceptual diagram of an embodiment showing an editing screen for information on a designated image. A "Cancel" button 72 is displayed at the upper left of the information editing screen for the designated image shown in FIG. 7, and a "Save" button 74 is displayed at the upper right.

When the "edit" button 68 is tapped on the specified image information viewing screen, an "x" button is displayed among the buttons for tag information that can be deleted on the specified image information editing screen. . When the user taps the "X" button among the tag information buttons, the image information editing unit 44 deletes the tag information button. Further, as shown by the finger mark 56 in FIG. 7, when the user taps the "Add tag" button 70, a new tag information input screen is displayed.

If the "Cancel" button 72 is tapped on the specified image information editing screen, the edited content for the specified image information is discarded, and the screen returns to the specified image information viewing screen. When the "save" button 74 is tapped, the edited content of the information of the specified image is saved.

FIG. 8 is a conceptual diagram of an embodiment showing a new tag information input screen. At the top of the new tag information input screen shown in FIG. 8, a thumbnail image of the specified image is displayed, and below that, a new tag information input field 76 is displayed. In the input field 76 for new tag information, a message "Add tag" is displayed to prompt input of new tag information. A "Cancel" button 78 is displayed on the right side of the input field 76 for new tag information.

When the user taps the new tag information input field 76, the history of recently added new tag information is displayed below the new tag information input field 76, and the new tag information input field 76 is displayed. A software keyboard is displayed at the bottom of the screen. The user can input the name of new tag information in the new tag information input field 76 using the software keyboard.

In this case, each time the user inputs characters into the input field 76 for new tag information, a partial match search is performed to search for new tag information from among all tag information attached to all images included in the image group. Tag information that partially matches the characters input in the tag information input field 76 is searched and displayed in a new tag information input screen. The user can also select new tag information from the list of tag information displayed in the new tag information input screen.

When the user taps the "Cancel" button 78, the new tag information input screen is closed and the screen returns to the specified image information editing screen.

Next, the tag information selection screen will be explained.
As described above, when the "Search" button is tapped, a tag information selection screen is displayed on the display unit 46.

FIG. 9 is a conceptual diagram of an embodiment showing a tag information selection screen. The tag information selection screen is displayed so that the list of images in the lower layer can be seen through at a constant transmittance by being superimposed on an area other than the upper part of the image list screen. In a state where no tag information is selected, on the tag information selection screen, a portion of all the tag information added to all the images included in the image group is displayed, classified into a plurality of categories. Each piece of tag information is displayed as a button on the tag information selection screen. Further, a "search tag" button 80 is displayed at the bottom of the tag information selection screen.

The multiple categories include calendar, person, shooting location, automatically assigned tag, manually assigned tag, and service tag.
The calendar category includes buttons for tag information of the year and month the image was taken.
The person category includes a button for tag information of the name of the person in the image.
The photography location category includes buttons for tag information of the location where the image was taken.
The category of automatically assigned tags includes buttons for tag information automatically assigned by the tag information assigning unit 26.
The category of manually added tags includes buttons for tag information manually added by the user, that is, buttons for tag information added by the image information editing section 44.
The service tag category includes buttons for tag information related to services provided by the image processing system 10 according to the present invention, such as scanning a print to create digital image data.

On the tag information selection screen, in the calendar category, tag information buttons for "2018,""2017," and "2016" are displayed in this order from the left to the right as the year the image was taken. Tag information buttons for "December,""November,""October," and "September" are displayed in this order from the left to the right as shooting months.
In the person category, tag information buttons for “Takashi,” “Akira,” and “Fuji Hanako” are displayed as person names in this order from the left to the right.
The shooting location category does not include tag information.
In the automatically assigned tag category, tag information buttons for "bicycle,""ramen," and "drinking party" are displayed in this order from the left to the right.
In the manually assigned tag category, buttons for tag information of "Circle,""Today'sLunch," and "Scenic View" are displayed in this order from the left to the right.
In the service tag category, a tag information button for "scanned image" is displayed.

Buttons for tag information included in each category other than the calendar category are displayed in a horizontal line. In the calendar category, buttons for tag information for the year and month of image capture are arranged horizontally in a row, and are displayed in descending numerical order of the year and month of image capture. The tag information buttons included in each of the other categories are displayed in descending order of the number of images to which each tag information has been added.

In addition, if each category contains tag information that cannot be displayed in the left or right direction, the user can swipe left or right while tapping the tag information button included in each category. By doing this, the tag information button displayed on the tag information selection screen is pushed outward, and the tag information button that is not displayed on the tag information selection screen is moved and displayed within the tag information selection screen. be able to.

In order to search for an image, the user does not have to type in a keyword; instead, the user taps and selects the tag information corresponding to the desired image to be searched from among the tag information displayed on the display section 46. This allows for intuitive searching.

Here, when the user has not selected any tag information as a search condition, as shown by the finger mark 56 in FIG. Assume that a button with tag information of "bicycle" is tapped as the tag information of "bicycle".

In this case, the tag information specifying unit 32 specifies the tag information of "bicycle" corresponding to the button of the tag information of "bicycle" selected according to the instruction from the user as the first selected tag information. In addition, the tag information display unit 30 changes the color of the tag information button for "bicycle" and displays "bicycle" on the left side in the upper area of the image list screen that is not covered by the tag information selection screen. is displayed.

Subsequently, the image extracting unit 34 extracts the image to which the tag information of “bicycle”, which is the first selection tag information, is added from the image group as the first search image, and the image display unit 38 At least a portion of the first search image is displayed on the image list screen.

Further, as shown in FIG. 11, the tag information display unit 30 displays at least a portion of all the tag information added to all the first search images on the tag information selection screen. That is, the content of the tag information displayed on the tag information selection screen is updated.

In the case of this embodiment, the calendar category is changed from "2018,""2017," and "2016" to buttons with tag information of "2018" and "2017," as the year in which the image was taken.
Furthermore, the person categories are changed from "Takashi,""Akira," and "Fuji Hanako" to tag information buttons for "Takashi,""Akira," and "Mr. Yamada."
The automatically assigned tag categories are changed from "bicycles,""ramen," and "drinking parties" to tag information buttons for "bicycles,""mountains," and "guardrails."
The manually assigned tag categories are changed from "circle,""today'slunch," and "scenic view" to tag information buttons for "circle,""scenicview," and "bicycle trip."

In other words, if only the tag information button for "bicycle" is selected from among the tag information buttons displayed on the tag information selection screen, the tag information for "bicycle" is assigned from among the images. All the tags added to all the search images that have been narrowed down to search images and have tag information of "bicycle" added to them from all the tag information added to all the images included in the image group. Narrowed down to information only.

Next, as shown by the finger mark 56 in FIG. 12, the user selects the second tag information button in addition to the first "bicycle" tag information button from among the tag information buttons displayed on the tag information selection screen. Assume that the tag information button for "chari trip" is tapped as the second tag information. That is, suppose that in a state where the tag information button for "bicycle" is selected, a button for tag information for "charitary trip" is additionally selected, and two pieces of tag information are selected at the same time.

In this case, in addition to the first "bicycle" tag information, the tag information specifying unit 32 selects a second "bicycle" tag information button that corresponds to the "chari trip" tag information button selected in response to the user's instruction. The tag information of "chari trip" is specified as the second selected tag information. In other words, both the tag information of "bicycle" and "charitary trip" are designated as the selected tag information at the same time. In addition, the tag information display unit 30 changes the color of the tag information button for "chari trip" in addition to the tag information button for "bicycle", and also changes the color of the tag information button for "chari trip" and displays images that are not covered by the tag information selection screen. ``Chari trip'' is displayed to the right of ``bicycle'' displayed in the upper area of the list screen.

Subsequently, the image extraction unit 34 extracts the first search image to which the tag information of "chari trip" is attached as a second search image from among the first search images to which the tag information of "bicycle" is attached. , as shown in FIG. 13, the image display unit 38 displays at least a portion of the second search image on the image list screen.
Although not shown, the tag information display unit 30 displays at least a portion of all the tag information added to all the second search images on the tag information selection screen.

In other words, if the tag information buttons for both "bicycle" and "chari trip" are selected at the same time, only the searched images with tag information for both "bicycle" and "chari trip" will be selected from the image group. Among all the tag information added to all the images included in the image group, all the searched images with tag information of both "bicycle" and "chari trip" are added. The information is narrowed down to all tag information.

The user can close the tag information selection screen at any time by swiping downward on the downward arrow 92 displayed at the top of the tag information selection screen. When the tag information selection screen is closed, as shown in FIG. 13, a list screen of images of the area covered by the tag information selection screen is displayed. This allows the user to view a list of search images.

If the image list screen contains too many search images to display all of them, the user can swipe the search images displayed on the image list screen up or down to move the search images up or down. This allows you to push out a part of the search image displayed on the image list screen, and move a part of the search image that is not displayed on the image list screen into the image list screen for display. .

Furthermore, when the tag information selection screen is closed, an upward arrow 94 is displayed at the top of the tag information selection screen, as shown in FIG. The user can open the tag information selection screen at any time by swiping upward on this upward arrow 94. In this embodiment, the tag information selection screen can be switched to three stages: a fully open state shown in FIGS. 9 to 12, a fully closed state shown in FIG. 13, and an intermediate position. Note that the tag information selection screen may be switched to three or more stages.

Furthermore, the user can display the tag information selection screen at any time by tapping the "Search" button while the image list screen is displayed. Alternatively, with the two tag information "bicycle" and "chari trip" selected, the user can select the button for other tag information to further narrow down the tag information and view the image further. You can also narrow it down.

The two tag information "bicycle" and "chari trip" designated as the selected tag information are selected by the tag information specifying unit 32 until the designation as the selected tag information is canceled in response to an instruction from the user. The designation as tag information is maintained.
For example, if a "cancellation" button is provided on the tag information selection screen, and the user taps the "cancellation" button, the two tags "bicycle" and "chari trip" specified as the selected tag information can be removed. It may also be possible to cancel selection of information all at once.

Furthermore, each time the user taps a tag information button, the user can switch between selecting and canceling the tag information as selected tag information.
In this case, the tag information specifying unit 32 specifies the tag information corresponding to the button as the selected tag information and the selected tag of the tag information corresponding to the button each time the button is pressed, in accordance with an instruction from the user. You can switch between canceling the designation as information and canceling the designation as information.

In other words, the selection of tag information is maintained until it is canceled, so that when the tag information selection is maintained, that is, after the image search is performed using the selected tag information, other When the screen is switched to the display screen and then switched back to the tag information selection screen, the search image that is the search result using the selected tag information can be viewed at any time.

In addition, in a state where the tag information is displayed as a button on the display unit 46, the user can press and hold (long tap) a button for tag information 1, thereby obtaining a superordinate concept of the tag information of the long-pressed button 1. tag information can be selected.

In this case, when one tag information button selected from among the tag information buttons displayed on the display section 46 in response to an instruction from the user is pressed for a certain period of time or more, the tag information specifying section 32 , all of the tag information of the subordinate concepts included in the tag information of the superordinate concept of the tag information corresponding to the pressed one tag information button is designated as the selected tag information.
For example, when a button for tag information of "Mont Blanc" is pressed for a long time, all tag information of "sweets" which is a superordinate concept of "Mont Blanc" is selected.

Alternatively, when one piece of tag information is selected by the tag information specifying unit 32 from among the tag information displayed on the display unit 46 in response to an instruction from the user, other tag information that has already been specified is selected. The designation may be canceled and only the selected tag information may be designated as the selected tag information.

Subsequently, as shown by the finger mark 56 in FIG. 14, the user taps the first search image that he or she wishes to view from among the search images displayed on the image list screen, and selects only the tapped one search image. can be displayed on the display section 46 for viewing.

In this case, the image designation section 40 designates one search image selected from among the search images displayed on the display section 46 in response to an instruction from the user as a specified search image, and the image display section 38 specifies, Only the specified search images are displayed on the display section 46.

FIG. 15 is a conceptual diagram of an embodiment showing a viewing screen for designated search images. The viewing screen for the specified search image is displayed on the entire surface of the display unit 46. At the top left of the specified search image viewing screen shown in FIG. 15, an "x" button 58 is displayed, at the bottom left a "calendar" button 82, and at the bottom right, an "information" button 58 is displayed. A button 60 is displayed. At the bottom of the designated search image viewing screen, a "trash can" button 62, a "share" button 64, and an "other" button 84 are displayed.
The designated search image viewing screen has the same configuration as the designated image viewing screen, except that a "calendar" button 82 is displayed at the lower left.

For example, by tapping the "X" button 58, the user can close the specified search image viewing screen and return to the search image list screen. The user can also tap the "trash can" button 62 to delete the specified search image, tap the "share" button 64 to share the specified search image with other users, or tap the "other" button 64 to share the specified search image with other users. 84 can be tapped to perform other functions. Further, the user can view information on the designated search image by tapping the "Information" button 60.

Furthermore, the user can view the searched images by sequentially displaying them one by one.
In this case, after only the designated search images are displayed on the display section 46, the image display section 38 selects from among all the search images a photographing time of the designated search image. Only one search image photographed before or after is sequentially displayed on the display unit 46 in the order of search images photographed in chronological order starting from the photographing time of the specified search image.

For example, when the user swipes the designated search image to the right, only one search image photographed at a shooting time one before the designated search image is displayed from among all the search images. From now on, every time the search image is swiped to the right according to the user's instructions, only the previous search image will be displayed one by one in the order of the search images taken in chronological order from the shooting time of the specified search image. Is displayed.

On the other hand, when the user swipes the specified search image to the left, only one search image photographed at a shooting time one after the specified search image is displayed from among all the search images. From now on, every time the search image is swiped to the left according to the user's instructions, only the search image that is one after the other in the order of the search images taken in chronological order from the shooting time of the specified search image will be displayed. Is displayed.

In addition, as shown in FIG. 15, by tapping the "calendar" button 82 displayed on the search image viewing screen, the user can select images taken before and after the shooting time of the designated search image from among the images. The captured image can be displayed on the display unit 46 and viewed.
In this case, when a specified search image is specified, only the specified search image is displayed on the display unit 46, and then the image display unit 38 displays the specified search image and images included in the image group according to instructions from the user. At least some of the images photographed before and after the photographing time of the specified search image are displayed on the display unit 46 in chronological order as time-series images.

In this way, the user does not have to return to the image list screen, such as the home screen, in order to display images taken before and after the shooting time of the first search image. In the case of the present embodiment, with a single action of tapping the "calendar" button 82, it is possible to display and view images photographed before and after the photographing time of the specified search image in chronological order.

Here, the same thing can be done by returning to the image list screen, such as the home screen, in response to an instruction from the user, and displaying images taken before and after the shooting time of the specified search image in chronological order. However, with this operating method, if a smartphone has a large number of images stored over several years, you can view the image list from many years ago on the image list screen such as the home screen until the shooting date of the desired image. This forces the user to move between

In contrast, by using the image processing device of this embodiment, images taken before and after the shooting time of the specified search image can be displayed in chronological order with a single action of tapping the "calendar" button 82. Using conventional technology, as described above, you can return to the image list screen such as the home screen and view images taken before and after the specified search image in chronological order. This is very convenient compared to displaying it.

FIG. 16 is a conceptual diagram of an embodiment showing a list screen of time-series images. A list screen of time-series images is displayed in an area between the menu area 52 and the navigation area 54.

Similarly, by tapping the first time-series image that the user wants to view from among the time-series images displayed on the time-series image list screen, the user can display only the tapped one time-series image on the display unit 46. It can be displayed and viewed.
In this case, the image designation unit 40 designates one time-series image selected from among the time-series images displayed on the display unit 46 as a designated time-series image according to an instruction from the user, and 38, only the specified time-series images are displayed on the display unit 46.

Further, when designated time-series images are designated, after only the designated time-series images are displayed on the display section 46, the image display section 38 selects one of the images from among the images each time an instruction from the user is input. , only one image photographed before or after the photographing time of the designated time-series image is sequentially displayed on the display unit 46 in the order of images photographed in chronological order from the photographing time of the designated time-series image.
The operation when the specified time series image is swiped to the right or left is the same as the operation when the specified search image is swiped to the right or left.

Note that instead of providing the "calendar" button 82, the same function as the "calendar" button 82 may be performed by a specific touch operation.

The user can return to the search image list screen by tapping the "Home" button on the time series image list screen.
As mentioned above, the selection of tag information is maintained until it is canceled, so when you return to the search image list screen, the list of search images that are the search results using the selected tag information will be displayed. Displayed on the image list screen.

Note that, before displaying the time-series images on the display unit 46, the search conditions for searching for an image from the image group are those for searching for an image from the image group based on conditions other than the shooting time of the image. There are no particular limitations as long as the conditions are met.
The conditions for the image capture time include not only the image capture time, but also various conditions related to the image capture time, such as the image capture year, capture month, capture week, capture date, and capture time. The conditions other than the photographing time include all conditions other than the conditions regarding the photographing time of the image.
As search conditions, the content of image analysis by the image analysis unit 22 and the name of tag information assigned by the tag information assignment unit 26 can be used, and the name of the subject in the image (person name and Examples include object names), names of people's facial expressions, names of people's actions, names of people's emotions, scene names, event names, names of user preferences, and names of shooting locations.

Furthermore, if the user cannot see the button for the tag information that he or she wants to use as a search condition on the tag information selection screen, the user taps the "Search for tags" button 80, as shown by the finger 56 in FIG. 17. This allows you to search for tag information by inputting keywords.

When the "Search tag" button 80 is tapped, a tag information search screen is displayed on the display unit 46, as shown in FIG.

The tag information search screen is displayed so that the list of search images in the lower layer can be seen through at a constant transmittance by being superimposed on an area other than a part of the upper part of the image list screen. A keyword input field 86 is displayed at the top of the tag information search screen. In the keyword input field 86, a message "Search for tags" is displayed to prompt the user to input a keyword. At the right end of the keyword input field 86, an "x" button 88 is displayed, and at the right side of the keyword input field 86, a "cancel" button 90 is displayed.

Suppose that the user then inputs a keyword into the keyword input field 86.

In this case, the tag information search unit 36 determines that at least a portion of all the tag information added to all the images included in the image group matches the keyword input in response to an instruction from the user. Tag information is searched as search tag information.
Subsequently, the tag information display section 30 displays at least a portion of the search tag information on the display section 46.
Subsequently, the tag information specifying section 32 specifies one piece of search tag information selected from among the search tag information displayed on the display section 46 in response to an instruction from the user.
Subsequently, the image extraction unit 34 extracts an image to which one search tag information has been added from the image group as a search image.
Subsequently, the image display section 38 displays at least a portion of all the search images on the display section 46 .
Subsequently, the tag information display section 30 displays at least a portion of all the tag information added to all the search images on the display section 46.

For example, as shown in FIG. 18, when "a" is entered in the keyword input field 86 to search for tag information of "ice cream," a partial match search is performed, and all Tag information that partially matches "a" is searched as search tag information from among all the tag information added to the image.

Next, as shown in FIG. 18, as search tag information candidates, on the tag information search screen, for example, the tag information buttons of "Ai-chan", "A-chan", "Amaou", and "Ice cream" are selected in this order. are displayed from the top to the bottom.

If the tag information search screen contains too many search tag information to display all of them, the user can swipe up or down the search tag information displayed on the tag information search screen to see search tag information candidates. By moving the list up and down, part of the search tag information displayed on the tag information search screen is pushed outward, and a part of the search tag information that is not displayed on the tag information search screen is pushed out. You can move to the information search screen and display it.

Further, the user can delete keywords input in the input field 86 all at once by tapping the "x" button 88. Furthermore, by tapping the "Cancel" button 90, the tag information search screen can be closed and the tag information selection screen can be returned. Further, the user can close the tag information search screen at any time by swiping the downward arrow 96 displayed at the top of the tag information search screen.

Next, as shown by the finger mark 56 in FIG. 18, the user taps, for example, a button for tag information of "ice cream" from among the search tag information candidate buttons displayed on the tag information search screen. do.

In this case, as shown in FIG. 19, a tag information selection screen is displayed on the display unit 46. The subsequent operations are the same as in the case after the "bicycle" tag information button is selected in a state where no tag information is selected.

In other words, the tag information of "Ice Cream" is specified as the selected tag information, and as shown in FIG. Is displayed.
Subsequently, an image to which tag information of "ice cream", which is the selected tag information, is added is extracted from the image group as a search image, and the search image is displayed on the image list screen.
Furthermore, all the tag information added to the search image to which the tag information of "ice cream", which is the selected tag information, is added is displayed on the tag information selection screen.

The operation after this is the same as when the tag information button for "bicycle" is selected and the tag information button for "chari trip" is additionally tapped. You can select additional buttons for the tag information you want.

Furthermore, if the user takes a new image, adds new tag information to the image, or deletes the tag information already added to the image, these new images, new tag information, etc. Based on the tag information, deleted tag information, etc., the reliability of the tag information may be improved by, for example, machine learning using AI (Artificial Intelligence).

Further, the server 12 may include at least a part of the functions of the client 14, such as image analysis, assigning evaluation values, assigning tag information, setting weighting, extracting images, and searching tag information, and the server 12 may perform at least some of the functions.

In the apparatus of the present invention, for example, an instruction acquisition section 18, an image analysis section 22, an evaluation value assignment section 24, a tag information assignment section 26, a weighting setting section 28, a tag information display section 30, a tag information specification section 32, an image extraction section 34 The hardware configuration of the processing unit that executes various processes, such as the tag information search section 36, image display section 38, image specification section 40, image information display section 42, and image information editing section 44, is as follows. It may be dedicated hardware, or it may be a variety of processors or computers that execute programs.

Various types of processors include CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays), which are general-purpose processors that execute software (programs) and function as various processing units.The circuit configuration can be changed after manufacturing. This includes programmable logic devices (PLDs), which are processors, and dedicated electric circuits, which are processors with circuit configurations specifically designed to perform specific processing, such as ASICs (Application Specific Integrated Circuits). .

One processing unit may be configured with one of these various processors, or a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU. It may also be configured by Further, the plurality of processing units may be configured using one of various processors, or two or more of the plurality of processing units may be configured using one processor.

For example, as typified by computers such as servers and clients, there is a configuration in which a single processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Furthermore, as typified by a system on chip (SoC), there is a form of using a processor that realizes the functions of an entire system including a plurality of processing units with one IC (Integrated Circuit) chip.

Furthermore, the hardware configuration of these various processors is, more specifically, an electric circuit (Circuitry) that is a combination of circuit elements such as semiconductor elements.

Furthermore, the method of the present invention can be implemented, for example, by a program that causes a computer to execute each step. Further, a computer-readable recording medium on which this program is recorded can also be provided.

Although the present invention has been described in detail above, the present invention is not limited to the above embodiments, and it goes without saying that various improvements and changes may be made without departing from the gist of the present invention.

10 Image processing system 12 Server 14 Client 16 Network 18 Instruction acquisition unit 20 Image holding unit 22 Image analysis unit 24 Evaluation value assignment unit 26 Tag information assignment unit 28 Weighting setting unit 30 Tag information display unit 32 Tag information specification unit 34 Image extraction unit 36 Tag information search section 38 Image display section 40 Image specification section 42 Image information display section 44 Image information editing section 46 Display section 50 Notification and status area 52 Menu area 54 Navigation area 56 Fingerprint 58, 66, 88 "X" button 60 "Information" button 62 "Trash" button 64 "Share" button 68 "Edit" button 70 "Add tag" button 72, 78, 90 "Cancel" button 74 "Save" button 76 New Tag information input field 80 "Search tag" button 82 "Calendar" button 84 "Other" button 86 Keyword input field 92, 94, 96 Arrows

Claims (9)

an instruction acquisition unit that acquires instructions input by the user;
an image search unit that searches, as a search image, an image matching a search condition input in response to an instruction from the user from among the image group;
an image display unit that causes a display unit to display images included in the image group;
an image specifying section for specifying one image selected from among the images displayed on the display section according to an instruction from the user;
The search condition is a condition for searching for an image from the image group based on conditions other than shooting time,
The image designation unit designates one search image selected from among the search images displayed on the display unit in response to an instruction from the user as a specified search image;
When the specified search image is specified, the image display unit displays only the specified search image on the display unit, and then displays the specified search image and the specified search image in response to a one-action instruction from the user. , displaying at least some of the images included in the image group that were photographed before and after the photographing time of the specified search image as time-series images on the display unit in chronological order;
The image processing apparatus wherein the one-action instruction from the user is input by pressing a button displayed on the viewing screen of the specified search image. After displaying only the specified search image on the display unit, the image display unit selects an image from among the search images that is earlier than the shooting time of the specified search image every time an instruction from the user is input. The image processing apparatus according to claim 1, wherein only one search image taken later is displayed on the display unit in the order of search images taken in chronological order starting from the shooting time of the specified search image. The image specifying unit specifies one time-series image selected from among the time-series images displayed on the display unit according to an instruction from the user as a specified time-series image,
The image display unit displays only the specified time-series images on the display unit when the specified time-series images are specified, and then displays the image group each time an instruction from the user is input. Among them, only one image photographed before or after the photographing time of the specified time-series image is displayed on the display unit in sequence in the order of images photographed in chronological order from the photographing time of the specified time-series image. , The image processing device according to claim 1 or 2. The image search unit includes:
a tag information display unit that causes the display unit to display tag information added to images included in the image group;
a tag information designation unit that designates tag information selected from among the tag information displayed on the display unit as selected tag information in response to an instruction from the user;
an image extraction unit that extracts an image to which the selection tag information is added from the image group as the search image;
The tag information display section causes the display section to display at least part of all tag information attached to all images included in the image group,
The tag information designation unit designates first tag information selected from among the tag information displayed on the display unit in response to an instruction from the user as first selected tag information,
The image extraction unit extracts an image to which the first selection tag information is attached from the image group as a first search image;
The image display unit displays at least a portion of all the first search images on the display unit,
The image processing device according to any one of claims 1 to 3, wherein the tag information display section causes the display section to display at least a part of all the tag information added to all the first search images. . In addition to the first selected tag information, the tag information specifying section may include second tag information selected from among the tag information displayed on the display section according to an instruction from the user. Specify at the same time as selected tag information,
The image extraction unit extracts a first search image to which the second selection tag information is attached from among the first search images as a second search image;
The image display unit displays at least a portion of all the second search images on the display unit,
The image processing device according to claim 4, wherein the tag information display section causes the display section to display at least a part of all the tag information added to all the second search images. The tag information designation unit simultaneously designates two or more pieces of tag information selected from among the tag information displayed on the display unit according to an instruction from the user as the selected tag information,
The image extraction unit extracts, from the image group, an image to which all of the two or more selection tag information is attached as the search image;
The image display unit displays at least a portion of all the search images on the display unit,
The image processing device according to claim 4 , wherein the tag information display unit displays at least a part of all the tag information added to all the search images on the display unit. a step in which the instruction acquisition unit acquires instructions input by the user;
a step in which the image search unit searches for an image matching the search condition input in response to an instruction from the user from among the image group as a search image;
an image display unit displaying an image included in the image group on a display unit;
an image specifying unit specifying one image selected from among the images displayed on the display unit in response to an instruction from the user;
The search condition is a condition for searching for an image from the image group based on conditions other than shooting time,
designating one search image selected from among the search images displayed on the display unit in response to an instruction from the user as a specified search image;
When the specified search image is specified, after displaying only the specified search image on the display unit, the specified search image and the images included in the image group are displayed in response to a one-action instruction from the user. displaying at least some of the images photographed before and after the photographing time of the designated search image as time-series images on the display unit in chronological order;
An image processing method , wherein the one-action instruction from the user is input by pressing a button displayed on the viewing screen of the specified search image. A program for causing a computer to execute each step of the image processing method according to claim 7 . A computer-readable recording medium on which a program for causing a computer to execute each step of the image processing method according to claim 7 is recorded.