JP6513009B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, PROGRAM, AND RECORDING MEDIUM - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium, which apply tag information corresponding to an event to an image included in an image group captured in the event.

  Conventionally, there is a technique of detecting an object present in an image by image analysis technology and automatically applying tag information corresponding to the object to each image. The tag information attached to the image by this technology is used to store and organize the images for each tag information in a storage service or the like that stores digital data such as image data corresponding to the image in a storage on the network. It is used.

  On the other hand, this tag information is used not only for organizing images but also for creating creation of value-added products such as photo books (electronic albums) (automatic creation of product data, selection assistance for images used for products, etc. It is conceivable to use it for

  In this case, the conventional tag information adding technology can automatically add tag information only from information that can be acquired from an object present in an image, and so image assist for creating more information and creation of value-added materials are needed. There is a new problem of not being able to

  For example, you want to classify images of pet dogs, you want to create value-added goods using images of pet dogs, and you want to classify images of food you ate in a restaurant, you use images that you ate in a restaurant In the case of applications where it is desired to create value-added commercial materials, it is possible to apply tag information corresponding to dogs and dishes to images in which dogs and dishes are present, using conventional tag information technology. It can correspond.

  However, if you want to classify travel images, you want to use them to create value-added goods, or you want to classify children's one-year school event images, use those images to add value-added goods In the case of the purpose of wanting to make, classifying a wedding image of a couple, and wanting to make a value-added product using that image, the objects present in each image are disjointed, so tag information is disjointed. Can not cope with these applications.

  That is, there is a problem that it is necessary for the user to manually organize (organize) the image group in applications for processing images of events that can not be determined from images, such as travel, school events, weddings, etc. The

  Patent documents 1 to 3 are prior art documents relevant to the present invention.

  In Patent Document 1, the photographed image (search target image) and each registered image of the image DB are collated, and a predetermined number of upper registered images similar (including the case of matching) to the photographed image are extracted, When acquiring the candidate name tag based on the result of the recognition processing, or when the subject of the photographed image is a temple, the candidate of the name of the temple similar (matching) to the subject of the photographed image is acquired as a candidate name tag It is stated that it is done.

  In patent document 2, when the keyword "sea" is extracted from the image A, the photographing date and / or position information is acquired from the metadata of the images B, C, D, E, F other than the image A and the image A If the shooting date and / or position information acquired from the images B, C, D, E, and F is the same as or close to the shooting date and / or position information acquired from the image A, the images B, C, and It is described that D, E, F are assigned the same "sea" as in the case of the image A.

  Patent Document 3 manages an event having an event date and time and an event name, and sets an event name as image metadata for an image having shooting date and time data coincident with or close to the event date and time linked to the event name. Alternatively, it is described that images of matching dates or in the vicinity of the shooting date of images are searched to determine images of matching or close proximity of shooting positions of the images.

JP, 2010-218372, A JP, 2009-271752, A JP 2012-194800 A

  Although Patent Documents 1 and 2 disclose that tag information corresponding to an object present in an image is attached to an image, it is possible to attach tag information corresponding to an event to an image corresponding to the event. Not listed. Although tag information corresponding to an object is tag information representing the object, tag information corresponding to an event is tag information representing the event, which is different from tag information corresponding to an object.

  An object of the present invention is an image processing apparatus that can solve the problems of the prior art and can add tag information corresponding to each event to an image included in an image group captured in each event, image processing Abstract: A method, program and recording medium are provided.

In order to achieve the above object, the present invention associates an event with an image of a first object corresponding to the event and an image of one or more first objects, and stores the event table;
An image analysis unit that analyzes an image of the first object and an image included in the image group, and detects only an image in which the second object matching the first object is present among the image group;
An event detection unit for detecting an event stored in association with the first object matching the second object present in the detected image based on the event table when the image in which the second object exists is detected; ,
An event image group specifying unit for specifying an event image group corresponding to a detected event from among the image group based on incidental information of the image included in the image group;
Based on the analysis result of the image, tag information having the same name as at least one of the detected event name and the detected event related name, and the first object name and the first object associated with the first object the tag information of different names from at least one of names of sex, there is provided an image processing apparatus including a tag information assigning unit that applies to all images included in an event image group.

Here, the apparatus further comprises an instruction acquisition unit that receives an instruction input by the user,
The event table preferably stores an event specified in response to an instruction and an image of a first object and an image of the first object specified in response to the instruction.

Further, the event image group specifying unit, based on the image shooting time of the image included in the supplementary information, sets an image group included in a certain shooting period including the shooting time of the detected image out of the image group to the event Identified as an image group,
The tag information adding unit corresponds to the detected event, and is an image included in the event image group, and the tag information corresponding to a shooting date common among the images shot within a predetermined shooting period. It is preferable to assign to the images included in the event image group.

  Further, it is preferable that the fixed photographing period is determined by a fixed number of days including the photographing date of the detected image and the photographing date of the detected image.

  Further, it is preferable that the certain number of days is a preset number of days.

  Moreover, it is preferable that a fixed number of days is the number of days designated according to the instruction.

  In addition, it is preferable that the tag information adding unit adds tag information corresponding to a photographing place representing an image included in the event image group to an image included in the event image group in response to the detected event.

  Further, it is preferable that the tag information adding unit determine that the imaging location represents the area name to which the imaging location having a ratio equal to or more than the threshold belongs to the imaging locations of the images included in the event image group.

The event table further stores the sub-event in association with the images of the third object and the third object corresponding to the sub-event,
The image analysis unit further analyzes the image included in the third object and the event image group, and detects only the image in which the fourth object matching the third object is present from the event image group,
The event detection unit detects a sub-event stored in association with the third object matching the fourth object based on the event table when an image in which the fourth object exists is detected.
Tag information assigning unit, as the tag information of at least one of names of names that are related to the same name and the detected event and the name of the detected event, which is the same name and detecting the name of the detected sub-event It is preferable to add tag information of at least one name of a name associated with the sub-event to all the images included in the event image group.

  Further, it is preferable that the event table stores the sub-event specified according to the instruction and the images of the third object and the third object specified according to the instruction in association with each other.

The event table further stores an event and event information representing the feature of the event in association with each other,
The image analysis unit further detects an image matching the event information from the image group,
The event detection unit stores an image in which the second object is detected and an image that matches the event information is detected in association with the first object that matches the second object based on the event table. It is preferable to detect the stored event in association with the event information.

  The event table preferably stores an event and event information specified according to an instruction in association with each other.

  Further, the event information preferably includes at least one of the shooting date of the image, the shooting location of the image, and the person present in the image.

  Further, it is preferable that the event image group specifying unit specifies, as an event image group, an image having the same shooting date as the image in which the second object is present among the image group.

  In addition, when the images included in the image group are arranged in chronological order in order of shooting time, the event image group specifying unit includes an image in which the second object is present, and a shooting interval between two adjacent images. It is preferable to identify a series of images in which is less than or equal to a threshold as an event image group.

  In addition, when the images included in the image group are arranged in time series in the order of shooting time, the event image group specifying unit includes a second object and is included between two images of different shooting times. It is preferable to identify the series of images to be generated as an event image group.

Also, in the present invention, the event table stores an event and an image of a first object and one or more first objects corresponding to the event in association with each other;
The image analysis unit analyzes an image of the first object and an image contained in the image group, and detects, from the image group, only an image in which a second object matching the first object is present;
The event detection unit detects an event stored in association with the first object matching the second object present in the detected image based on the event table when the image in which the second object exists is detected Step to
Identifying an event image group corresponding to the detected event from among the image group based on incidental information of the image included in the image group, the event image group specifying unit;
The tag information adding unit is tag information having the same name as at least one of the name of the detected event and the name having relevance to the detected event based on the analysis result of the image, the name of the first object and the first Providing tag information having a name different from at least one of names associated with one object to all the images included in the event image group.

Here, in the step of identifying the event image group, the image group included in a certain imaging period including the imaging time of the detected image among the image group based on the imaging time of the image included in the supplementary information , As an event image group,
The step of applying tag information corresponds to a detected event and is an image included in an event image group, the tag corresponding to a shooting date common among images shot within a certain shooting period It is preferable to add information to the images included in the event image group.

  In addition, the step of attaching tag information may include attaching tag information corresponding to a shooting location representing an image included in the event image group to an image included in the event image group in response to the detected event. preferable.

Further, the event table stores the sub-event in association with the third object and the image of the third object corresponding to the sub-event;
The image analysis unit further analyzes the image included in the image of the third object and the event image group, and detects only the image in which the fourth object matching the third object is present among the event image group Including and
The step of detecting an event detects a sub-event stored in association with the third object matching the fourth object based on the event table when an image in which the fourth object exists is detected.
In the tag information adding step, tag information of at least one of the same name as the name of the detected event and / or a name related to the detected event has the same name as the name of the detected sub-event and detection It is preferable to add tag information of at least one of the names having a name associated with the recorded sub-event to all the images included in the event image group.

Further, the event table associates and stores an event and event information representing the feature of the event;
The image analysis unit further includes the step of detecting an image matching the event information from the image group,
In the step of detecting an event, when an image in which the second object is present is detected and an image matching the event information is detected, based on the event table, it is associated with the first object matching the second object Preferably, an event stored and stored in association with the event information is detected.

  The present invention also provides a program for causing a computer to execute the steps of the image processing method described above.

  The present invention also provides a computer readable recording medium having recorded thereon a program for causing a computer to execute the steps of the image processing method described above.

  In the present invention, an event is stored in association with a first object corresponding to the event and its image, and an image in which a second object matching the first object is detected is detected from the image group and detection is performed. The event stored in association with the first object matching the second object present in the selected image is detected, and tag information corresponding to the detected event is added to the image included in the event image group.

  In this way, the user simply attaches the tag information corresponding to the event to the image included in the event image group captured in the event, only by capturing the first object stored in association with the event in the event. can do. Tag information corresponding to an event can be used, for example, to assist in creation of a value-added item such as a photo book (electronic album).

It is a block diagram of one embodiment showing composition of an image processing system concerning the present invention. It is a block diagram of one Embodiment showing the structure of the server shown in FIG. 5 is a flow chart of an embodiment depicting operation of the image processing system. 5 is a flowchart illustrating the operation of the image processing system following FIG. 4; FIG. 6 is a flowchart illustrating the operation of the image processing system following FIG. 5. (A) And (B) is a conceptual diagram of an example showing "my car" and a "school building" as an image of a 1st object. (A) and (B) are conceptual diagrams of an example showing an event image group including an image in which "my car" and "school building" exist. It is a conceptual diagram of an example showing the event image group of travel. It is a conceptual diagram of an example showing the event picture group of an athletic meet.

  Hereinafter, an image processing apparatus, an image processing method, a program, and a recording medium of the present invention will be described in detail based on preferred embodiments shown in the attached drawings.

  FIG. 1 is a block diagram of an embodiment showing the configuration of an image processing system according to the present invention. The image processing system 10 shown in the figure specifies an event image group corresponding to each event from among the image groups owned by the user, and sets tag information corresponding to each event to an image included in the event image group. It is given. The image processing system 10 includes a server 12 and one or more clients 14 connected to the server 12 via a network 16 such as the Internet.

  FIG. 2 is a block diagram of an embodiment representing the configuration of the server shown in FIG. The server 12 shown in the figure is an example of the image processing apparatus according to the present invention, and includes an instruction acquisition unit 18, an event table 20, an image group acquisition unit 22, an image analysis unit 24, an event detection unit 26, and an event. An image group identification unit 28 and a tag information addition unit 30 are provided.

  In the server 12, the instruction acquisition unit 18 receives an instruction input by the user.

  The instruction acquisition unit 18 acquires various instructions input by the user at the client 14 from the client 14 via the network 16. As the user's instruction, an instruction to specify an image to be processed, an event stored in the event table 20, an instruction to specify the first object (subject) and the image thereof, a sub event stored in the event table 20, 3 includes an instruction to specify an object and its image, an image capturing period, a sub event, an instruction to specify event information, and the like.

  The event table 20 associates and stores an event, a first object corresponding to the event, and one or more images thereof.

An event is an event that has a certain meaning for people, such as travel, school events, birthdays, weddings, etc., and is an event that occurs over a certain period of time, an event that ends in one day, or more than two days Includes events.
The first object is, for example, a car when the event is a trip, and a school building when the event is a school event, so that objects related to the event can be freely set.

  Also, when there is a sub-event included in the event, the event table 20 further stores the sub-event, the third object corresponding to the sub-event, and the image thereof in association with each other.

For example, when the event is a school event, the sub-event is an event such as an athletic meet, an entrance ceremony, or a graduation ceremony, which is one of the school events.
For example, when the sub-event is an athletic meet, the user can freely set an object present in the image captured in the sub-event, such as a red-and-white hat.

  Subsequently, the image group acquisition unit 22 acquires an image group owned by the user based on an instruction from the user.

  In the case of the present embodiment, the image group acquisition unit 22 acquires an image group uploaded from the client 14 of the user to the server 12 via the network 16 in accordance with an instruction from the user.

  Subsequently, the image analysis unit 24 analyzes the image of the first object stored in the event table 20 and the images included in the image group acquired by the image group acquisition unit 22.

  The image analysis performed by the image analysis unit 24 includes the detection of the position and the area of the area of the person and the object in the image, as well as the detection of the person and the object other than the person present in the image. Further, as the image analysis, for example, the image analysis unit 24 compares the image of the first object with the image included in the image group, and a second object matching the first object exists in the image group. Detect an image. Furthermore, as the image analysis, the image analysis unit 24 compares the image of the third object with the image included in the image group, and an image in which the fourth object matching the third object exists among the event image group To detect

  If the first object is a car, the second object is a car that matches the car of the first object, and if the first object is a school building, the second object is a school building that matches the school of the first object. The second object may be any object that matches the first object, and the size, position, orientation, image quality, etc. of the object in the image may be different.

  Subsequently, when the image analysis unit 24 detects an image in which the second object is present, the event detection unit 26 matches the second object present in the detected image based on the event table 20. It detects events stored in association with objects.

  The event detection unit 26 detects, for example, the event table 20 when an image is detected in which a second object matching the first object stored in the event table 20 is detected by the image analysis unit 24 from among the image group. Among the first objects stored in the second image, a first object that matches the second object present in the detected image is detected, and an event stored in association with the detected first object is acquired.

  Further, when the image analysis unit 24 detects an image in which a fourth object matching the third object is detected from the event image group, the event detection unit 26 determines a fourth object based on the event table 20. Detect a sub-event stored in association with a third object that matches.

  The relationship between the third object and the fourth object is similar to the relationship between the first object and the second object. For example, if the third object is a red-and-white hat, the fourth object is a red-and-white hat that matches the red-and-white hat of the third object. The fourth object may be any object that matches the third object, and the size, position, orientation, image quality, etc. of the object in the image may be different.

  Subsequently, the event image group specifying unit 28 specifies an event image group corresponding to the event detected by the event detection unit 26 from the image group based on the incidental information of the image included in the image group. is there.

  The incidental information is, for example, Exif (Exchangeable Image File Format) information incidental to the image as header information of the image. The incidental information includes information such as an imaging time and an imaging position (GPS (Global Positioning System) information) of an image.

  Subsequently, the tag information adding unit 30 adds one or more pieces of tag information corresponding to one or more objects present in the image to the image based on the analysis result of the image by the image analysis unit 24.

  For example, when it is detected that “car” is present in the image as an object, the tag information adding unit 30 adds tag information of “car” to the image. As the name of tag information corresponding to an object, for example, the same name as the name of the object or a name associated with the object is used. In addition, tag information of an arbitrary name designated according to a user's instruction may be added to the image as the name of tag information of the object.

  Further, the tag information attaching unit 30 attaches tag information different from the tag information corresponding to the first object as the tag information corresponding to the event detected by the event detecting unit 26 to the image included in the event image group . Further, the tag information adding unit 30 adds tag information corresponding to the detected sub-event to an image included in the event image group as tag information corresponding to the detected event.

  The name of tag information corresponding to an event is, for example, the same name as the name of the event or a name associated with the event, and is different from the name of tag information corresponding to the first object. For example, when the event is travel and the first object is a car, the name of tag information corresponding to the event is, for example, the same travel as the name of the event and is different from the car that is the name of the first object. The same applies to sub-events.

  On the other hand, the client 14 is an example of a terminal device owned by each user, such as a PC (personal computer), a tablet PC, and a smartphone. The client 14 has, for example, an input unit such as a keyboard, a mouse, and a touch panel, a liquid crystal display, a display unit such as a touch panel, a process of acquiring an instruction input by the user from the user, and a process of the display unit displaying various information A control unit or the like to control is provided.

  Although the server 12 and the client 14 each include a transmission / reception unit that transmits / receives various data between the two via the network 16, in the present embodiment, descriptions of the transmission / reception unit and the like are provided to prevent complexity. It is omitted.

  Next, the operation of the image processing system 10 will be described with reference to the flowcharts shown in FIGS.

  First, the user inputs an event, and an instruction to specify the first object corresponding to the event and its image in the client 14. This indication is sent from the user's client 14 to the server 12 via the network 16.

  In the case of the present embodiment, the user designates two “travel” and “school event” as events. When "travel" is specified as an event, "my car" and one or more images thereof are specified as the first object corresponding to "travel" and its image as shown in FIG. 6 (A). Also, when "school event" is specified as an event, "school building" and one or more images thereof are specified as the first object corresponding to the "school event" and its image as shown in FIG. 6 (B). .

  In the server 12, the instruction acquisition unit 18 acquires the user's instruction transmitted from the user's client 14 via the network 16.

  Subsequently, the event specified in accordance with the user's instruction acquired by the instruction acquiring unit 18, the first object corresponding to the event specified in accordance with the user's instruction, and the image thereof are associated with each other and the event table 20 is displayed. It is stored (step S1).

  Subsequently, the user captures an image at various events.

  In each event, the user captures an image of the first object stored in the event table 20 in association with the event. When the event is "travel", the "my car" stored in the event table 20 is photographed as a first object corresponding to "travel". When the event is a "school event", the "school building" stored in the event table 20 is photographed as a first object corresponding to the "school event".

  Subsequently, an instruction to specify an image group to be processed is input by the user at the client 14.

  In the case of the present embodiment, the user designates an image group including images taken in “travel” and “school event” as the image group to be processed.

  Similarly, the instruction acquisition unit 18 acquires the user's instruction, and the image group acquisition unit 22 acquires an image group owned by the user based on the user's instruction acquired by the instruction acquisition unit 18 (step S2). ).

  Subsequently, as image analysis, detection of persons and objects present in the image included in the image group and detection of positions and areas of areas of the persons and objects in the image are performed by the image analysis unit 24 (step S3).

  Subsequently, based on the analysis result of the image by the image analysis unit 24, the tag information attachment unit 30 attaches tag information corresponding to an object present in the image included in the image group to the image (step S4).

  Further, the image analysis unit 24 compares the image of the first object stored in the event table 20 with the image included in the image group as the image analysis, and matches the first object out of the image group. An image in which an object is present is detected (step S5).

  In the case of the present embodiment, the image analysis unit 24 determines that “my car” matching the “my car” stored in the event table 20 is included in the image group as the image in which the second object matching the first object exists. The present image and an image in which a “school building” matching the “school building” stored in the event table 20 is detected are detected.

  Subsequently, it is confirmed whether or not an image in which a second object matching the first object is detected is detected from the image group.

  In the case of the present embodiment, first, it is confirmed by the event detection unit 26 whether or not an image having “my car” matching the “my car” stored in the event table 20 is detected from the image group. (Step S6).

As a result, when an image having "my car" matching the "my car" stored in the event table 20 is not detected from the image group (No in step S6), the process proceeds to step S13.
On the other hand, when an image having "my car" matching the "my car" stored in the event table 20 is detected from the image group (Yes in step S6), the event detection unit 26 An event stored in association with the "my car" stored in the event table 20 that matches the "my car" present in the detected image as shown by the dotted line in FIG. "Travel" is detected as (step S7).

  Subsequently, the event image group identification unit 28 detects the event detection unit 26 from among the image group as shown by the solid line in FIG. 7A based on the supplementary information of the images included in the image group. As the event image group corresponding to “travel” which is the event that has been performed, an image group having a shooting date of 2014/3/8 is specified (step S8).

  Subsequently, as shown in FIG. 7A, the tag information adding unit 30 sets “travel” as tag information corresponding to “travel” which is an event detected by the event detection unit 26; Is attached to the image included in the specified event image group (step S9).

Here, the event image group identification unit 28 is an image included in a certain imaging period including the imaging time of the detected image among the image group based on the imaging time of the image included in the supplementary information, for example. A group can be identified as an event image group (step S10).
In this case, the tag information adding unit 30 corresponds to the detected event, and is an image included in the event image group, and corresponds to a common shooting date among the images shot within a certain shooting period. Tag information to be added to the image included in the event image group (step S11).

  In addition, a fixed imaging | photography period is determined by the fixed number of days including the imaging | photography day of the detected image, and the imaging | photography date of the detected image. Further, the certain number of days may be the number of days set in advance in the image processing system 10, or may be the number of days specified in accordance with the user's instruction. In addition, the image capturing period may be a certain number of days before or a certain number of days after the image capturing date, or may be a certain number of days before or after the image capturing date.

  In addition, the tag information adding unit 30 can add tag information corresponding to a shooting location representing an image included in the event image group to an image included in the event image group in response to the detected event. (Step S12).

  The tag information adding unit 30 determines that the area name to which the shooting location having a ratio of the threshold or more belongs to the shooting locations of the images included in the event image group is the shooting location representing the image included in the event image group be able to.

  For example, as shown by the dotted line in FIG. 8, the shooting time (shooting date) of the detected image is 2014/8/2, and most of the images included in the event image group, that is, the event image group Among the images included, for example, corresponds to the shooting position (shooting location) of an image shot at a travel destination other than an image of “My Car” shot at home and an image shot on the way from home to a travel destination It is assumed that the latitude and longitude to be included are included in latitude: XX to YY, longitude: AA to BB.

  In this case, the tag information adding unit 30 further sets the shooting date 2014/8 common to the images captured within the shooting period of 2014/8/2 ± α (α is an allowance for shooting time). The tag information of is added to the images included in the event image group of “travel”, and the images included in the event image group captured within the shooting location range of latitude: XX to YY, longitude: AA to BB are represented When the shooting place is in Nagano prefecture neighborhood, tag information of "Nagano" is added to the images included in the "travel" event image group.

  As a result, as shown in FIG. 8, tag information of “2014/8 Nagano travel” is added to the images included in the “travel” event image group. Thus, as tag information corresponding to an event, it is possible to add more detailed tag information than “travel”.

  Subsequently, the event detection unit 26 checks whether an image having “school buildings” matching the “school buildings” stored in the event table 20 is detected from the image group (step S13). .

As a result, when an image having “school buildings” matching the “school buildings” stored in the event table 20 is not detected from the image group (No in step S13), the process is ended.
On the other hand, when an image having a "school building" matching the "school building" stored in the event table 20 is detected from the image group (Yes in step S13), the event detection unit 26 An event stored in association with a "school building" stored in the event table 20, which matches the "school building" present in the detected image, as shown by a dotted line in FIG. As a "school event" is detected (step S14).

  Subsequently, the event image group identification unit 28 detects the event detection unit 26 from among the image group as shown by a solid line in FIG. 7B based on the supplementary information of the image included in the image group. An image group having a shooting date of 2014/4/4 is specified as an event image group corresponding to the “school event” that is the event (step S15).

  Subsequently, as shown in FIG. 7B by the tag information adding unit 30, the “school event” is specified as the tag information corresponding to the “school event” which is the event detected by the event detection unit 26. It is attached to the image included in the event image group specified by the unit 28 (step S16).

  Subsequently, as the image analysis, the image analysis unit 24 compares the image of the third object stored in the event table 20 with the image included in the image group, and matches the third object in the event image group. An image in which the fourth object is present is detected (step S17).

  In Japan, children wearing red and white hats are often seen in athletic meet, so in the event table 20, a sub-event "sports meeting" is associated in association with "red and white caps". The image analysis unit 24 detects an image in which “red and white hat” matching the “red and white hat” stored in the event table 20 is present in the event image group.

  Subsequently, it is confirmed by the event detection unit 26 whether or not an image having a “red and white cap” matching the “red and white cap” stored in the event table 20 is detected from the event image group. (Step S18).

As a result, when an image having a "red and white cap" matching the "red and white cap" stored in the event table 20 is not detected from the image group (No in step S18), the process is performed. finish.
On the other hand, when an image having a “red and white cap” matching the “red and white cap” stored in the event table 20 is detected from the image group (Yes in step S18), the event detection unit 26 is performed. Performs detection of sub-events that match the "red hat". That is, based on the event table 20, as shown in FIG. 9, "sports day" is detected as a sub-event stored in association with the "red and white cap" (step S19).

  Subsequently, as tag information corresponding to the detected event, the tag information adding unit 30 adds tag information corresponding to the detected sub-event “athletic event” to the image included in the event image group (see FIG. Step S20).

  For example, as shown in FIG. 9, when the shooting date of the images included in the event image group is 2014/5/27, the tag information adding unit 30 sets the shooting date 2014/5/27 and the sub-events. Tag information corresponding to a certain "sports meeting" is added to the images included in the "school event" event image group.

  As a result, instead of the tag information of "school event", tag information of "2014/5/27 athletic meet" is given to the images included in the event image group of "school event" as shown in FIG. Ru. Thus, it is possible to attach tag information corresponding to a sub-event instead of an event.

  As described above, in the image processing system 10, the event, the first object corresponding to the event, and the image thereof are stored in association with each other, and the first object corresponding to the first object is selected from the group of images owned by the user. 2 detects an image in which an object is present, detects an event stored in association with a first object matching a second object present in the detected image, detects tag information corresponding to the detected event, an event image Assign to the images included in the group.

  In this way, the user simply attaches the tag information corresponding to the event to the image included in the event image group captured in the event, only by capturing the first object stored in association with the event in the event. can do. Tag information corresponding to an event can be used, for example, to assist in creation of a value-added item such as a photo book (electronic album).

  The event table 20 may further store an event and event information representing the feature of the event in association with each other.

  In this case, the image analysis unit 24 further detects an image that matches the event information from the image group, and the event detection unit 26 detects an image in which the second object is present, and matches the event information. When an image to be detected is detected, an event stored in association with the first object matching the second object based on the event table 20 and stored in association with the event information is detected.

  Further, as the event information, for example, information including at least one of a shooting date of an image, a shooting location of the image, and a person present in the image can be used.

  When the event is "travel", as the event information, an image with three or more family members whose shooting date of the images included in the "travel" event image group is weekends, holidays, summer holidays, winter holidays is Certain things can be illustrated. In addition, when the event is a "school event", the child's own child is shown where the shooting location (GPS information) of the image included in the event image group of the "school event" is the event information as the event information. It can be exemplified that there is an image.

  As described above, by using the event information, it is possible to improve the determination accuracy of each event.

  The event table 20 may store an event and event information set in advance in the image processing system 10 in association with each other, or associate the event with event information specified according to a user's instruction. May be stored.

  Further, the event table 20 may store the event set in advance in the image processing system 10 in association with the first object and the image thereof, and in response to the user's instruction acquired by the instruction acquiring unit 18 The designated event may be stored in association with the designated first object and its image according to the user's instruction. The same applies to sub-events, third objects and their images.

  In addition, when the place where the image group acquisition unit 22 acquires the image group to be processed (a specific folder or the like on the client 14) is determined in advance, the image group acquisition unit 22 stores the image group according to the user's instruction. You may acquire the image group stored in the place.

  In addition, the user may shoot an image in which a second object corresponding to the event exists, as the first image shot in the event, or as the last image, or in the middle of the event You may In addition, the user may shoot one or more images in which the second object is present. For example, the user may capture two images in which the first object is present as the first image and the last image captured in the event.

  The event image group specifying unit 28 may specify an image having the same shooting date as the image in which the second object exists from among the image group as the event image group. In addition, when the images included in the image group are arranged in time series in the order of shooting time, a series of images including the image in which the second object exists and the shooting interval between two adjacent images is equal to or less than the threshold Images may be identified as a group of event images. Furthermore, in the case where two images in which the first object exists are captured as the first image and the last image captured in an event, the images included in the image group are arranged in chronological order in the order of capturing time Alternatively, a series of images included between two images in which the second object exists and the shooting times are different may be specified as the event image group.

  Further, even if the event image group specifying unit 28 specifies an image whose shooting location is within a certain range as an event image group based on the shooting location of the image in which the second object is present among the image groups. Good. For example, in the event table, "work" as the event, "the persimmon tree (planted in the courtyard of the company)" as the first object, and an image of that persimmon tree as the image of the first object, The image location information of the images included in the image group identifies an image within a certain range around the image location of the image where “tree of oak” exists as an event image group corresponding to the event “job” It is also good.

  In addition, it is not essential that the tag information adding unit 30 add tag information corresponding to an object to an image, and an image group including an image to which tag information corresponding to an object is added in advance may be used.

  In the above embodiment, all the functions corresponding to the image processing apparatus of the present invention are arranged on the server 12 side, but the present invention is not limited to this. For example, even if some functions are arranged on the client 14 side Alternatively, all functions may be located on the client 14 side.

In the device of the present invention, each component of the device may be configured by dedicated hardware, or each component may be configured by a programmed computer.
The method of the present invention can be implemented, for example, by a program for causing a computer to execute each of the steps. It is also possible to provide a computer readable recording medium in which the program is recorded.

  The present invention has been described above in detail, but the present invention is not limited to the above embodiment, and it goes without saying that various improvements and changes may be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 10 Image processing system 12 Server 14 Client 16 Network 18 Instruction acquisition part 20 Event table 22 Image group acquisition part 24 Image analysis part 26 Event detection part 28 Event image group identification part 30 Tag information attachment part

Claims (23)

An event table that stores an event and a first object corresponding to the event and an image of the one or more first objects in association with each other;
An image analysis unit that analyzes an image of the first object and an image included in an image group, and detects only an image in which a second object matching the first object is present among the image group;
An event for detecting an event stored in association with a first object matching a second object present in the detected image based on the event table when an image containing the second object is detected A detection unit,
An event image group specifying unit for specifying an event image group corresponding to the detected event from the image group based on incidental information of an image included in the image group;
The tag information of the same name as at least one of the name of the detected event and the name associated with the detected event based on the analysis result of the image , and the name of the first object and the name of the first object An image processing apparatus, comprising: a tag information adding unit that adds tag information of a name different from at least one of names having relevance to the first object to all the images included in the event image group. And an instruction acquisition unit configured to receive an instruction input by the user.
The image processing apparatus according to claim 1, wherein the event table associates and stores an event designated according to the instruction, and a first object designated according to the instruction and an image of the first object. The event image group identification unit selects an image group included in a certain imaging period including the imaging time of the detected image among the image group based on the imaging time of the image included in the supplementary information. , Identify as the event image group,
The tag information adding unit corresponds to the detected event, and further corresponds to an image included in the event image group and common to the images captured within the predetermined imaging period. The image processing apparatus according to claim 2, wherein the tag information to be added is added to an image included in the event image group.   The image processing apparatus according to claim 3, wherein the predetermined photographing period is determined by a photographing date of the detected image and a predetermined number of days including a photographing date of the detected image.   The image processing apparatus according to claim 4, wherein the predetermined number of days is a preset number of days.   The image processing apparatus according to claim 4, wherein the certain number of days is a number of days specified according to the instruction.   The tag information adding unit is further configured to add tag information corresponding to an imaging location representative of an image included in the event image group to an image included in the event image group, corresponding to the detected event. An image processing apparatus according to any one of 3 to 6.   8. The image according to claim 7, wherein the tag information adding unit determines that an area name to which an imaging location having a ratio equal to or greater than a threshold belongs to the imaging location representing the imaging location of the images included in the event image group. Processing unit. The event table further stores a sub-event in association with a third object corresponding to the sub-event and an image of the third object.
The image analysis unit further analyzes the images contained in the third object and the event image group, and from among the event image group, only the image in which the fourth object matching the third object is present Detect
The event detection unit detects a sub-event stored in association with a third object matching the fourth object based on the event table when an image in which the fourth object is present is detected.
Said tag information assigning unit, as at least one of the name tag information of the name that is relevant to the same name and the detected event as the detected event name, the same name as the name of the detected sub-event The image according to any one of claims 2 to 8, wherein tag information of at least one name having a name associated with the detected sub-event is added to all the images included in the event image group. Processing unit.   10. The image processing apparatus according to claim 9, wherein the event table stores the sub-event specified according to the instruction and the third object specified according to the instruction and the image of the third object in association with each other. . The event table further stores the event and event information representing a feature of the event in association with each other.
The image analysis unit further detects an image matching the event information from the image group,
The event detection unit is configured to match the second object based on the event table when an image including the second object is detected and an image matching the event information is detected. The image processing apparatus according to any one of claims 2 to 10, which detects an event stored in association with an object and stored in association with the event information.   The image processing apparatus according to claim 11, wherein the event table stores the event and event information specified according to the instruction in association with each other.   The image processing apparatus according to claim 11, wherein the event information includes at least one of an image shooting date, an image shooting location, and a person present in the image.   The event image group specifying unit according to any one of claims 1 to 13, wherein an image having the same shooting date as the image on which the second object exists is specified as the event image group from among the image group. Image processing device.   The event image group identification unit, when the images included in the image group are arranged in time series in order of shooting time, includes an image in which the second object is present, and shooting of two adjacent images The image processing apparatus according to any one of claims 1 to 13, wherein a series of images having an interval equal to or less than a threshold is specified as the event image group.   The event image group identification unit, when the images included in the image group are arranged in time series in the order of shooting time, the second object is present, and between the two images at different shooting times. The image processing apparatus according to any one of claims 1 to 13, wherein a series of images included is specified as the event image group. Storing an event in association with an event, and a first object corresponding to the event and an image of the one or more first objects;
The image analysis unit analyzes an image of the first object and an image included in the image group, and detects only an image in which a second object matching the first object is present from the image group; ,
The event detection unit is stored in association with the first object matching the second object present in the detected image based on the event table when the image in which the second object exists is detected. Detecting an event;
Identifying an event image group corresponding to the detected event from the image group based on incidental information of the image included in the image group, the event image group specifying unit;
The tag information adding unit is, based on the analysis result of the image , tag information having the same name as at least one of the name of the detected event and the name having relevance to the detected event , the first object Applying tag information having a name different from at least one of the first object and the name associated with the first object to all the images included in the event image group. The step of specifying the event image group is an image included in a certain imaging period including the imaging time of the detected image among the image group based on the imaging time of the image included in the supplementary information Identify a group as the event image group,
The step of applying the tag information corresponds to the detected event, and is an image included in the event image group, and a common photographing date among the images photographed within the certain photographing period The image processing method according to claim 17, wherein tag information corresponding to is attached to an image included in the event image group.   The step of attaching the tag information applies tag information corresponding to a photographing place representing an image included in the event image group to the image included in the event image group, the tag information corresponding to the detected event. The image processing method according to claim 18. And storing the event table in association with a sub-event and a third object corresponding to the sub-event and an image of the third object.
The image analysis unit further analyzes an image of the third object and an image included in the event image group, and an image in which a fourth object matching the third object is present among the event image group And detecting only the
The step of detecting the event detects a sub-event stored in association with a third object matching the fourth object based on the event table when an image in which the fourth object is present is detected. And
The step of attaching the tag information may include the name of the detected sub-event as tag information of at least one of a name having the same name as the name of the detected event and a name associated with the detected event . 20. The tag information according to any one of claims 17 to 19, wherein tag information of at least one of names having the same name and a name associated with the detected sub-event is added to all the images included in the event image group. Image processing method. Further, the event table associates and stores the event and event information representing a feature of the event;
The image analysis unit further detecting an image that matches the event information from the image group,
The step of detecting the event may match the second object based on the event table when an image in which the second object is present is detected and an image matching the event information is detected. The image processing method according to any one of claims 17 to 20, which detects an event stored in association with the first object and stored in association with the event information.   A program for causing a computer to execute each step of the image processing method according to any one of claims 17 to 21.   A computer readable recording medium having recorded thereon a program for causing a computer to execute the steps of the image processing method according to any one of claims 17 to 21.

Images