JP2015070505A - Photographing apparatus, system, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for efficiently taking a picture profitable for a subject, a photographer, and the other person concerned.SOLUTION: A photographing apparatus comprises: accumulated information acquisition means which acquires accumulated information indicating accumulated values obtained by accumulating, in each photographing, scores according to how well the subject has been photographed; determination means which, on the basis of the accumulated information acquired by the accumulated information acquisition means, determines a subject to be photographed preferentially from among plural subjects of photographing targets; and presentation means which presents instruction information for giving an instruction to photograph the subject determined by the determination means.

Description

  The present invention relates to an imaging device, a system, an information processing method, and a program.

There is a growing demand for automatic photography. However, in an automatic photographing device that only finds and shoots a person, there may be a case where an unintended bias or the like of the user of the automatic photographing device, a subject, or a related person occurs in the photographed photograph. For example, there may be variations in the number of shots between subjects, omissions that have not been shot despite being at the spot, and the like. Therefore, by registering participants in the event in advance and analyzing the photographed photos, it is determined who has been photographed so far, who is not yet photographed, and who should be photographed. Technology has been proposed.
For example, Patent Document 1 discloses a technique for creating a person list in advance in a photographing apparatus that automatically performs photographing at a party venue, etc., determining whether or not photographing is performed for the persons in the list, and preventing omissions in photographing. ing.
Patent Document 2 discloses a technique (shooting assist method) that notifies a photographer of a shooting tendency and can shoot a plurality of persons in a balanced manner.
Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for automatically photographing people having high relationships by analyzing human relationships.

JP 2005-10244 A JP 2011-211695 A JP 2011-82913 A

However, the technique disclosed in Patent Document 1 is a technique that only prevents face omissions by collating a face image registered in advance with a face image included in the photographed photograph. The technique disclosed in Patent Document 2 is a technique that only supports shooting by counting the number of shots for each person and notifying the shooting tendency. For this reason, with the techniques disclosed in Patent Document 1 and Patent Document 2, there is a possibility that the image quality will vary greatly depending on the subject. For example, there may be a case where a certain person appears large in the center of the photograph, but a certain person only appears small toward the edge of the photograph. It is also conceivable that the eyes are closed in all the photos of a person.
Moreover, in the technique disclosed in Patent Document 3, it is possible to obtain an image in which good friends are photographed together by analyzing human relationships and automatically photographing people with high relationships. Depending on the scene, there is a case where it is desired to photograph a person who is the main character of the scene.
An object of the present invention is to provide a technique for efficiently taking a photograph useful for a subject, a photographer, and other related parties.

  Therefore, the imaging apparatus of the present invention is acquired by the cumulative information acquisition unit that acquires cumulative information indicating a cumulative value obtained by accumulating the score corresponding to the captured state of the photographed subject for each shooting, and the cumulative information acquisition unit. Determination means for determining a subject to be preferentially photographed among a plurality of subjects to be photographed based on the accumulated information, presentation means for presenting instruction information for instructing photographing of the subject determined by the determination means, Have

  According to the present invention, it is possible to provide a technique for efficiently taking a photograph useful for a subject, a photographer, and other related parties.

It is a figure which shows an example of the hardware constitutions of an imaging device. FIG. 3 is a diagram illustrating an example of a functional configuration of an imaging apparatus according to Embodiment 1. 1 is a diagram illustrating an example of an imaging device in Embodiment 1. FIG. 5 is a flowchart illustrating an example of processing in the first embodiment. 4 is a diagram illustrating an example of a shooting scene in Embodiment 1. FIG. FIG. 6 is a diagram (part 1) illustrating an example of shooting data. It is a figure (the 2) which shows an example of imaging | photography data. FIG. 6 is a third diagram illustrating an example of shooting data. FIG. 6 is a diagram (part 1) illustrating an example of a subject score result. It is a figure which shows an example of the list which matched the face of a person and the name. FIG. 10 is a diagram (part 1) illustrating an example of a cumulative subject result; It is FIG. (1) which shows an example of social interest data. FIG. 10 is a diagram illustrating an example of shooting priority (part 1); It is FIG. (4) which shows an example of imaging | photography data. 6 is a diagram illustrating an example of a functional configuration of an imaging apparatus according to Embodiment 2. FIG. It is a figure which shows an example of the imaging device in Embodiment 2. FIG. 10 is a flowchart illustrating an example of processing in the second embodiment. 6 is a diagram illustrating an example of a shooting scene in Embodiment 2. FIG. It is FIG. (5) which shows an example of imaging | photography data. It is FIG. (6) which shows an example of imaging | photography data. It is FIG. (2) which shows an example of social interest data. It is FIG. (7) which shows an example of imaging | photography data. FIG. 10 is a second diagram illustrating an example of a subject score result. FIG. 10 is a second diagram illustrating an example of a cumulative subject result; FIG. 11 is a second diagram illustrating an example of shooting priority; 6 is a diagram illustrating an example of a functional configuration of an imaging apparatus according to Embodiment 3. FIG. It is a figure which shows an example of the imaging device in Embodiment 3. 10 is a flowchart illustrating an example of processing in the third embodiment. FIG. 10 is a diagram illustrating an example of a shooting scene in the third embodiment. It is FIG. (8) which shows an example of imaging | photography data. It is FIG. (3) which shows an example of social interest data. FIG. 10 is a third diagram illustrating an example of a cumulative subject result; It is FIG. (4) which shows an example of social interest data.

The best mode for carrying out the present invention will be described below with reference to the drawings.
<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a hardware configuration of an imaging (imaging) apparatus 1000.
The photographing apparatus 1000 includes a CPU 1001, a RAM 1002, a ROM 1003, an HD 1004, a display unit 1005, a photographing unit 1006, an input unit 1007, and a network I / F 1008. Note that in the case where the photographing apparatus 1000 is a pan-tilt drive camera, an autonomously movable camera, or the like, it may have a drive device that drives the own apparatus.
A CPU 1001 controls the entire photographing apparatus 1000. When the CPU 1001 executes a program stored in the HD 1004 or the like, processing (information processing) according to a functional configuration and flowchart of the photographing apparatus 1000 described later is realized.
The RAM 1002 functions as a work area for the CPU 1001.
The storage areas of the ROM 1003 and the HD 1004 store various data including various programs required when the CPU 1001 executes processing, data related to predetermined values and mathematical expressions, and the like.
A display unit 1005 is a display or the like that displays information on the image capturing apparatus 1000. The display unit 1005 may be a touch panel integrated with an input unit 1007 described later.
The photographing unit 1006 photographs a subject to be photographed. The shooting here includes both moving image shooting and still image shooting.
The input unit 1007 includes keys, buttons, and the like that accept input operations by the operator.
A network I / F 1008 communicates with devices that can communicate via a network. For example, the network I / F 1008 can communicate with a server device or the like designated via the input unit 1007.

FIG. 2 is a diagram illustrating an example of a functional configuration of the imaging apparatus 1000.
The photographing unit 1006 photographs a subject to be photographed. The shooting data shot by the shooting unit 1006 is used when a shooting priority setting unit A12006 (to be described later) sets shooting priority (shooting order).
The calculation unit A12000 performs various calculations in the photographing process. Each function of the calculation unit A12000 is a function realized by the CPU 1001 executing a program stored in the HD 1004 or the like. The calculation unit A12000 includes an aesthetic evaluation unit A12001, a subject detection unit A12002, a subject score calculation unit A12003, a cumulative subject score calculation unit A12004, a social interest data acquisition unit A12005, and a photographing priority setting unit A12006. Further, the calculation unit A12000 includes, as storage areas, an aesthetic evaluation result holding unit A12010, a subject score holding unit A12030, a cumulative subject score holding unit A12040, a social interest data holding unit A12050, and a photographing priority holding unit A1060.

The aesthetic evaluation unit A12001 evaluates the aesthetics of the photographic data acquired by the imaging unit 1006, and outputs data relating to the aesthetic evaluation result (hereinafter simply referred to as aesthetic evaluation result). Details of processing by the aesthetic evaluation unit A12001 will be described later with reference to FIG. The esthetic evaluation result holding unit A12010 holds the aesthetic evaluation result evaluated by the aesthetic evaluation unit A12001, and outputs it on demand.
A subject detection unit A12002 detects a subject from the shooting data acquired by the shooting unit 1006, and outputs a subject list. Details of processing by the subject detection unit A12002 will be described later with reference to FIG.
The subject score calculation unit A12003 determines each of the data on the state of reflection of each subject detected by the subject detection unit A12002 (hereinafter simply referred to as the state of reflection) and the aesthetic evaluation result held in the aesthetic evaluation result holding unit A12010. The subject score of the subject is calculated. The subject score calculation unit A12003 outputs data relating to the calculated subject score (hereinafter simply referred to as subject score). Details of processing by the subject score calculation unit A12003 will be described later with reference to FIG. The subject score holding unit A12030 holds the subject score calculated by the subject score calculation unit A12003, and outputs it when requested.

The cumulative subject score calculation unit A12004 performs calculation processing based on the subject score for each photographing of each subject held in the subject score holding unit A12030 and the data related to the cumulative subject score held in the cumulative subject score holding unit A12040. . In the following description, data relating to the cumulative subject score is simply referred to as a cumulative subject score. More specifically, the cumulative subject score calculation unit A12004, based on the subject score and the cumulative subject score, cumulative subject score (cumulative value) obtained by cross-evaluating photographing data photographed in the past for each subject. Is calculated. Then, the cumulative subject score calculation unit A12004 holds the calculated cumulative subject score in the cumulative subject score holding unit A12040. Details of processing by the cumulative subject score calculation unit A12004 will be described later with reference to FIG. The accumulated subject score is an example of accumulated information. The process by the cumulative subject score calculation unit A12004 is an example of the cumulative information acquisition process. The cumulative subject score holding unit A12040 holds the cumulative subject score calculated by the cumulative subject score calculation unit A12004, and outputs it when requested.
The social interest data acquisition unit A12005 acquires (receives) social interest data from a server device or the like that provides a social networking service (hereinafter simply referred to as a social networking service). The social networking service transmits social interest data in response to a request. And social interest data acquisition part A12005 hold | maintains the acquired social interest data in social interest data holding | maintenance part A12050. Details of the processing by the social interest data acquisition unit A12005 will be described later with reference to FIG. The social interest data here is data relating to relationship information representing the human relationship between the subject and the person (user) in the place, interest information representing the target and degree of interest of each user, and the like. The social networking service is a service for building a social network on the Internet. The social interest data holding unit A12050 holds the social interest data acquired by the social interest data acquisition unit A12005, and outputs it in response to a request.

The shooting priority setting unit A12006 calculates a shooting priority regarding which subject is to be preferentially shot from the cumulative subject score held in the cumulative subject score holding unit A12040. The shooting priority setting unit A12006 holds data relating to the calculated shooting priority (hereinafter simply referred to as shooting priority) in the shooting priority holding unit A12060. Details of processing by the shooting priority setting unit A12006 will be described later with reference to FIG. The shooting priority holding unit A12060 holds the shooting priority for each subject and outputs it on demand.
The shooting priority presentation unit A13000 acquires the shooting priority for each subject held in the shooting priority holding unit A12060 and presents it to the user via the display unit 1005. Further, the photographing priority presentation unit A13000 presents the user with advice related to the selection of the subject via the display unit 1005 based on the social interest data held in the social interest data holding unit A12050. Information presented to the user via the display unit 1005 by the shooting priority presentation unit A13000 is an example of instruction information related to shooting of a subject. The function of the photographing priority presentation unit A13000 is a function realized by the CPU 1001 executing a program stored in the HD 1004 or the like. Details of the processing by the photographing priority presentation unit A13000 will be described later with reference to FIG.

FIG. 3 is a diagram illustrating an example of the imaging apparatus 1000 according to the present embodiment. In the present embodiment, a digital camera 2000 shown in FIG.
In the present embodiment, a digital camera 2000 including the above-described photographing unit 1006, calculation unit A12000, and photographing priority presentation unit A13000 is used. The digital camera 2000 evaluates the aesthetics of the photographed photos, detects the subject, and calculates the subject score of each subject. Then, the digital camera 2000 calculates the cumulative subject score of each subject from the subject score of each subject, and sets the shooting priority by acquiring social interest data. Furthermore, the digital camera 2000 presents the photographing priority on the display 2001 corresponding to the display unit 1005.
Further, as described above, the digital camera 2000 is equipped with a memory that can be used as the aesthetic evaluation result holding unit A12010, the subject score holding unit A12030, and the cumulative subject score holding unit A1240. Furthermore, the digital camera 2000 includes a memory that can be used as the social interest data holding unit A12050 and the shooting priority holding unit A12060.
The above is an example of the configuration of the digital camera 2000 in the present embodiment.

The processing flow of this embodiment is demonstrated using FIG.
FIG. 4 is a flowchart illustrating an example of processing in the present embodiment.
In step S3100, the aesthetic evaluation unit A12001 acquires the shooting data shot by the shooting unit 1006, evaluates aesthetics, and holds the aesthetic evaluation result holding unit A12010.
In step S3200, the subject detection unit A12002 detects a subject from the shooting data shot by the shooting unit 1006. The subject score calculation unit A12003 acquires the photographing data photographed by the photographing unit 1006 and the aesthetic evaluation result held in the aesthetic evaluation result holding unit A12010, and each subject detected by the subject detection unit A12002 The subject score is calculated. Further, the subject score calculation unit A12003 holds the calculated subject score in the subject score holding unit A12030 for each subject.
In step S3300, the cumulative subject score calculation unit A12004 updates the cumulative subject score for each subject held in the cumulative subject score holding unit A12040 from the subject score held in the subject score holding unit A12030. Then, the cumulative subject score calculation unit A12004 holds the updated cumulative subject score in the cumulative subject score holding unit A12040.

In step S3400, the social interest data acquisition unit A12005 acquires social interest data from the social networking service, and stores the acquired social interest data in the social interest data holding unit A12050.
In step S3500, the shooting priority setting unit A12006 sets the shooting priority from the accumulated subject score held in the accumulated subject score holding unit A12040.
In step S3600, the shooting priority presentation unit A13000 acquires the shooting priority from the shooting priority holding unit A12060, and presents the shooting priority to the photographer. In addition, the shooting priority presentation unit A13000 presents advice regarding the selection of the subject from the social interest data held in the social interest data holding unit A12050.
When the above processing ends, the digital camera 2000 deletes the shooting data used to determine the shooting priority in the processing of FIG. 4 from the storage area. As a result, the user can effectively use the storage area of the digital camera 2000.
The processing flow of the image capturing apparatus 1000 in the present embodiment has been described above.

Hereinafter, the process of FIG. 4 will be described using a more specific example.
FIG. 5 is a diagram showing an example of a shooting scene in which a photographer 4000 photographs a plurality of subjects 4001 to 4004 with a digital camera 2000 held at a party venue.
In the present embodiment, when the photographer 4000 performs photographing with the digital camera 2000 held by the photographer 4000, the digital camera 2000 presents the photographing priority to the photographer 4000 via the display 2001. More specifically, the digital camera 2000 presents to the photographer 4000 via the display 2001 the shooting priority regarding which of the subjects 4001 to 4004 is to be preferentially shot. As a result, the photographer 4000 can efficiently take photographs useful for the photographer 4000 and the subject 4001 to the subject 4004. The photographer 4000 uses the digital camera 2000 to photograph the subject.

FIG. 6 is a diagram illustrating an example of shooting data 5000 acquired by the digital camera 2000.
The digital camera 2000 processes the shooting data 5000 according to the flow shown in FIG.
In step S <b> 3100, the aesthetic evaluation unit A12001 performs aesthetic evaluation on the shooting data 5000. In the present embodiment, the aesthetic evaluation unit A12001 evaluates the aesthetics of the shooting data 5000 using a composition related to the appearance of the subject. For example, the aesthetic evaluation unit A12001 uses a three-division method or the like for composition evaluation. The three-division method is a method of drawing a line that divides a photograph into three parts vertically and horizontally as shown in FIG. 7, and arranging a main subject (usually a face in a person photograph) where these lines intersect. The aesthetic evaluation unit A12001 detects a face area from the photographing data 5000 and specifies a position in order to evaluate by a three-division method.
FIG. 8 is a diagram illustrating an example of photographic data in which the aesthetic evaluation unit A12001 performs face detection processing on the photographic data 5000 and draws lines that are divided into three parts vertically and horizontally.
A region surrounded by a rectangle 7002 is a face region of the subject 4002, and a region surrounded by the rectangle 7003 is a face region of the subject 4003. In this embodiment, the aesthetic evaluation unit A12001 detects a face by detecting a known face-specific image feature pattern area stored in advance in the storage area from the image data 5000. When there is one detected face, the aesthetic evaluation unit A12001 obtains the distance between the face and the intersection of the lines to be divided into three. In addition, when there are two detected faces, the aesthetic evaluation unit A12001 obtains each distance and obtains an average. In addition, when there are three or more detected faces, the aesthetic evaluation unit A12001 obtains the distance from the nearest intersection in each of the face closest to any intersection and the next two faces closest to the intersection, Use the average distance as a score.

ここで、Ａは、顔と、写真を縦横に三分割する線の交点との距離である。Ｂは、撮影データの対角線の長さである。Ｃは、審美性スコアである。検出された顔が写真を縦横に三分割する線の交点と重なっていればスコアは１００となり、検出された顔が撮影データの隅にあるとすればスコアは０となる。以降、本実施形態では、審美性評価部Ａ１２００１が撮影データ５０００に対して求めたスコアが６０であったものとして説明を行う。 The aesthetic evaluation unit A12001 calculates the score C using the following formula (1).

Here, A is the distance between the face and the intersection of lines that divide the photograph into three parts vertically and horizontally. B is the length of the diagonal line of the photographic data. C is an aesthetic score. The score is 100 if the detected face overlaps the intersection of the lines that divide the photo vertically and horizontally, and the score is 0 if the detected face is in the corner of the captured data. Hereinafter, in the present embodiment, description will be made assuming that the score obtained by the aesthetic evaluation unit A12001 for the photographing data 5000 is 60.

  The esthetic evaluation result calculated as described above is held in the aesthetic evaluation result holding unit A12010. The method for evaluating the composition is not limited to the evaluation method based on the position of the face. For example, a method of detecting the eye of the subject and evaluating the composition based on the distance from the eye to the intersection of the lines to be divided into three may be used. . In the present embodiment, the composition is determined based on the three-division method. However, the aesthetic evaluation unit A12001 may evaluate the quality of the composition using another composition. In addition, the aesthetic evaluation unit A12001 does not necessarily have to determine whether the composition is good or bad based on the composition of the score that is better than a predetermined score. For example, the aesthetic evaluation unit A12001 uses the Internet or the like to search for a photograph (image data) having a composition similar to the composition of the photographed data, and uses the evaluation attached to the image as a reference. You may make it judge good and bad. Further, in this embodiment, even when three or more faces are detected, the composition is evaluated using two faces close to the intersection of lines that divide the photograph vertically and horizontally. The sex evaluation unit A12001 may evaluate the composition using three or more faces. Also, the score calculation method need not be limited to the method described above. In the present embodiment, composition is used as a method for evaluating aesthetics. However, the aesthetic evaluation unit A12001 searches for similar images using the color or the Internet, and evaluates the images. You may evaluate aesthetics by the method etc. which evaluate aesthetics on the basis.

Returning to the description of FIG.
In step S3200, the subject detection unit A12002 detects a subject from the shooting data 5000. Then, the subject score calculation unit A12003 calculates a subject score that is a value that quantitatively indicates the quality of the image for each detected subject. First, the subject detection unit A12002 detects a subject by detecting a face area from the captured data. The subject detection unit A12002 recognizes information on the size of the target person in the image, the number of persons present in the image, the presence / absence of a smile, the orientation of the face, the presence / absence of eyebrows, etc. (hereinafter referred to as subject information) To do. The subject score calculation unit A12003 calculates the score of each subject from the subject information recognized by the subject detection unit A12002 and the aesthetic evaluation result held in the aesthetic evaluation result holding unit A12010, and holds it in the subject score holding unit A12020. To do.

ここで、Ｄは、画像中に占める被写体の顔の面積の割合で０．０から１．０までの値を取り、大きく写っているほど被写体スコアは高くなる。Ｅは、画像中に存在する人物の数であり、少人数で写っている写真ほど被写体スコアは高くなる。撮影データ５０００の場合、写真中の人物数は体の一部が写っている人物を含めれば４人であるが、顔の有無により人物検出する検出器の検出結果によれば、図８に示す通り２つの顔が検出されることになるので、人物数は２とカウントされる。Ｆは、笑顔であるか否かを示す値であり、笑顔の場合は２、笑顔が検出されなかった場合は１とする。本実施形態のデジタルカメラ２０００は、笑顔の判定をするために、笑顔画像から笑顔検出用の辞書データ（以下、単に辞書という）を作成する。Ｇは、顔の向きであり、正面を向いている場合は２、それ以外の場合は１とする。本実施形態のデジタルカメラ２０００は、顔の向きを検出するために、様々な方向の顔画像から顔の向き毎に辞書を作成し、それを用いる検出器を利用して検出する。前記検出器は、様々な方向の顔画像から作成された辞書を用いて検出処理を行うことによって、該当する顔の向きに対応した辞書で顔を検出することができるので、顔の向きを判定することができる。Ｈは、目つむりであるか否かを示す値であり、被写体が目をつむっている場合は１、目を開けている場合は２とする。本実施形態のデジタルカメラ２０００は、目をつむっている場合と、開けている場合との辞書を作成し、他のパラメータと同様に検出処理を行う。Ｉは、審美性評価結果保持部Ａ１２０１０に保持される審美性評価結果である。 In the present embodiment, the subject score calculation unit A12003 calculates the subject score J using the following equation.

Here, D is a ratio of the area of the face of the subject in the image, and takes a value from 0.0 to 1.0, and the subject score increases as the image is larger. E is the number of persons present in the image, and the subject score is higher for photographs taken with a small number of people. In the case of the shooting data 5000, the number of persons in the photograph is four if the person including a part of the body is included, but according to the detection result of the detector that detects the person based on the presence or absence of the face, it is shown in FIG. Since two faces are detected, the number of persons is counted as two. F is a value indicating whether or not a smile is present, and is 2 when a smile is detected and 1 when a smile is not detected. The digital camera 2000 of this embodiment creates smile detection dictionary data (hereinafter simply referred to as a dictionary) from a smile image in order to determine smile. G is the direction of the face, and is 2 when facing the front, and 1 otherwise. In order to detect the face orientation, the digital camera 2000 according to the present embodiment creates a dictionary for each face orientation from face images in various directions, and detects it using a detector using the dictionary. The detector can detect a face with a dictionary corresponding to the face direction by performing a detection process using a dictionary created from face images in various directions. can do. H is a value indicating whether or not the eyes are closed, and is 1 when the subject has eyes closed and 2 when the eyes are open. The digital camera 2000 of this embodiment creates a dictionary for when the eyes are closed and when the eyes are open, and performs the detection process in the same manner as other parameters. I is an aesthetic evaluation result held in the aesthetic evaluation result holding unit A12010.

The parameters used for the calculation process of Expression (2) need not be limited to the above parameters. For example, parameters such as the presence or absence of focusing on the subject may be used. It is desirable to use parameters suitable for the situation. For example, when the detector detects a smile in a scene where the smile is not suitable, the subject score calculation unit A12003 may give a lower evaluation than when the smile is not smile. Further, the subject score calculation unit A12003 may search for a similar image related to the face portion using the Internet and evaluate the appearance of the subject based on the evaluation of the image. FIG. 9 is a diagram illustrating an example of the result of the subject score calculated by the subject score calculation unit A12003 in the present embodiment.
The subject score calculation unit A12003 in the present embodiment detects a person name using the face area of each subject detected by the subject detection unit A12002 as a query, and associates the subject name with the subject score calculated by the subject score calculation unit A12003. A12030 is held.

The digital camera 2000 tags the subject of the photograph taken in the past in accordance with the photographer's instruction received via the input unit 1007 or the like, and holds it as shown in FIG. Thereby, the digital camera 2000 can search for a similar face and acquire a person name. FIG. 10 is a diagram illustrating an example of a list in which human faces and names are associated with each other.
As a method of determining whether or not they are the same person, a method of searching for an image having a matching feature by using the relative position and size of facial parts, the shape of eyes, nose, cheek bones, and jaws as features. Etc. The main algorithms include eigenfaces using principal component analysis, linear discriminant analysis, elastic bunch graph matching, hidden Markov models, etc., but any method may be used. In addition, it is not necessary to perform person recognition processing in this apparatus, and a face recognition service on the cloud may be used. The photograph taken in the past does not need to be a photograph taken by the photographer himself, but some database may be used as long as the face and the name can be matched. Further, the information to be matched does not need to be the name of the subject, and may be associated with an account such as a social networking service. When the subject detection unit A12002 cannot specify information about a person such as a subject name or an account of a social networking service from the face, a subject-specific ID that can identify the person may be assigned to identify the subject. Furthermore, the subject detection unit A12002 may identify a person using information other than the face, such as a name tag and an RFID tag, without using the face information.

Returning to the description of FIG.
In step S3300, the cumulative subject score calculation unit A12004 adds a subject score related to the new shooting data held in the subject score holding unit A12030 to the cumulative subject score for each subject already held in the cumulative subject score holding unit A12040. Thereby, the cumulative subject score calculation unit A12004 updates the cumulative subject score. Then, the cumulative subject score calculation unit A12004 holds the updated new cumulative subject score in the cumulative subject score holding unit A12040. The accumulated subject score calculation unit A12004 obtains a list of accumulated subject scores held in the accumulated subject score holding unit A12040, and whether or not the subject 4002 and the subject 4003 of the newly obtained shooting data 5000 are already included in the list. Determine.
The list of accumulated subject scores in this embodiment includes the names of the subjects and the accumulated subject scores as shown in FIG. FIG. 11 is a diagram illustrating an example of the cumulative subject score result.
If the subject 4002 and the subject 4003 for which the subject score has been calculated already exist in the list, the cumulative subject score calculation unit A12004 adds the new subject score to the cumulative score of the subject and updates the cumulative subject list. On the other hand, when the subject 4002 and the subject 4003 do not exist in the list, the cumulative subject score calculation unit A12004 adds them to the list as new subjects. In the present embodiment, the cumulative subject score calculation unit A12004 creates a list using the same subject name as a unique ID. However, the cumulative subject score calculation unit A12004 may hold the subject scores calculated in the past without collecting the same subjects, and add and use the subject scores of the same subject when necessary. Thus, there is no need to limit the score holding method. In this embodiment, the cumulative subject score calculation unit A12004 simply adds and updates the list if the subject in the acquired shooting data is in the list held by the cumulative subject score holding unit A1240. doing. However, the cumulative subject score calculation unit A12004 may weight and add according to the number of photographs in which the subject is photographed.

Returning to the description of FIG.
In S3400, the social interest data acquisition unit A12005 acquires social interest data and holds it in the social interest data holding unit A12050. In the present embodiment, it is assumed that social interest data is constructed using information on the number of common friends between subjects (hereinafter simply referred to as the number of common friends between subjects). In this embodiment, the social interest data acquisition unit A12005 acquires the number of common friends of each subject from an external social network service. Note that social interest data representing the relationship between subjects such as the number of common friends between the subjects is an example of relationship information. Further, the process of S3400 is an example of a related information acquisition process.
FIG. 12 is a diagram illustrating an example of the number of common friends between the subjects.
FIG. 12 shows that the subject 4001 and the subject 4002 have one common friend, and the subject 4001 and the subject 4003 have 32 common friends, which is considered as a weighted undirected graph. be able to. In general, it is considered that the subject is intimate with each other when the number of the common friends is larger than the number of the common friends, and the subject 4001 is estimated to be closer to the subject 4003 than the subject 4002. The social interest data acquisition unit A12005 holds the social interest data acquired and constructed as described above in the social interest data holding unit A12050.

  In the present embodiment, the social interest data acquisition unit A12005 estimates the intimacy between subjects from the number of common friends. However, the social interest data acquisition unit A12005 may estimate the familiarity using other parameters. For example, the social interest data acquisition unit A12005 may estimate the intimacy using information such as whether the region of origin or university is the same, or whether the workplace is the same. Also, the social interest data acquisition unit A12005 does not necessarily need to acquire social interest data from an external social networking service. For example, the social interest data acquisition unit A12005 may estimate the relationship between the subjects from information such as having a conversation together or being closer than a predetermined distance. Also, the social interest data acquisition unit A12005 does not necessarily have to estimate social interest data from past or current friendship information. For example, the social interest data acquisition unit A12005 may generate the social interest data by estimating that the subjects are likely to be intimate because the interested objects are similar. In this embodiment, the social interest data acquisition unit A12005 acquires the friendship from the social network service using the subject name as a query. Good. Further, the social interest data acquisition unit A12005 does not need to automatically acquire or generate social interest data. For example, the digital camera 2000 or the like may generate social interest data based on a user operation (instruction) received via the input unit 1007 or the like.

Returning to the description of FIG.
In step S3500, the shooting priority setting unit A12006 sets a shooting priority for the subject. In this embodiment, the shooting priority setting unit A12006 sets a higher shooting priority for a subject with a lower cumulative subject score. For example, in the case of the cumulative subject score shown in FIG. 11, since the cumulative subject score of the subject 4002 is the lowest, the photographing priority setting unit A12006 sets the photographing priority of the subject 4002 to the highest.
FIG. 13 is a diagram illustrating an example of shooting priority for each subject set by the shooting priority setting unit A12006.
In the example of FIG. 13, the shooting priority setting unit A12006 assigns numbers in order from 1 so that the shooting priority increases in order from the lowest cumulative subject score. Here, it is assumed that the shooting priority is higher as the number is smaller, such as 1 being the highest shooting priority and 2 being the next highest shooting priority. In the present embodiment, the shooting priority setting unit A12006 sets the shooting priority of a subject that does not have a cumulative subject score to be the highest, but for all subjects that do not have a cumulative subject score, all shooting priority levels are set. You may make it set the average value of. Further, the shooting priority setting unit A12006 may be set according to the situation, such as not setting the shooting priority of a subject for which no cumulative subject score exists. The shooting priority setting unit A12006 sets the shooting priority in the above-described order regardless of the difference in the cumulative subject score. However, the shooting priority may be set according to the difference in the cumulative subject score. . Further, the photographer may correct the shooting priority according to his / her preference via the input unit 1007.

Returning to the description of FIG.
In S3600, shooting priority presentation unit A13000 presents the subject with the highest shooting priority to photographer 4000. In the present embodiment, the photographing priority presentation unit A13000 displays on the display 2001 of the digital camera 2000. More specifically, the shooting priority presentation unit A13000 shoots which subject should be preferentially shot by displaying a square frame superimposed on the subject 4002 as shown by a frame 13002 in FIG. Present it to the person. FIG. 14 is a diagram illustrating an example of shooting data in which shooting priority is presented. Further, there are a subject 4001 and a subject 4003 next to the subject 4002, but it is assumed that only one of the subject 4001 or the subject 4003 can be photographed together in addition to the subject 4002 due to the problem of angle of view. In that case, based on the friendship relationship between the subjects indicated by the social interest data held in the social interest data holding unit A12050, the shooting priority presentation unit A13000 is happy with which subject to shoot the subject 4002. judge.
According to the social interest data shown in FIG. 12, there is one common friend between the subject 4002 and the subject 40001, and there are 80 common friends between the subject 4002 and the subject 4003. That is, the subject 4002 has more friends with the subject 4003 than with the subject 4001. Therefore, it is considered that there is a high possibility that the subject 4002 and the subject 4003 are photographed together. Therefore, the shooting priority presentation unit A13000 presents to the photographer which subject is to be placed in the viewfinder by displaying a round frame superimposed on the subject 4003 to be shot like a frame 13003. In this way, the shooting priority presentation unit A13000 determines and presents a shooting subject to be shot with priority. As a result, the photographer 4000 can take a picture that is appreciated by the subject.

The method of presenting the photographing priority of the subject to the photographer 4000 by the photographing priority presentation unit A13000 is not necessarily a method of displaying a rectangular frame in an overlapping manner, and is indicated by an arrow, a number, a character, a symbol, or the like. The method may be used. The same applies to the case of instructing a subject to be photographed together. Further, the shooting priority presentation unit A13000 may cause the photographer 4000 to search for a subject by displaying a face photograph of the subject with the highest shooting priority on the display 2001. Further, the shooting priority presentation unit A13000 may present not only the shooting priority of the subject but also advice regarding the composition and the like.
In this embodiment, the photographing priority presentation unit A13000 selects subjects to be photographed together based only on the social interest data held in the social interest data holding unit A12050. However, based on other criteria. A subject may be selected. For example, it is assumed that there are more photographs than the predetermined number of photographs in which the subject 4001 and the subject 4002 were taken at the same time in the past. In this case, the photographing priority presenting unit A13000 selects the subject 4002 and the subject 4003 to photograph, so that the subject photographed together with the subject 4002 can have diversity.
The image capturing apparatus 1000 according to the present embodiment is not only a digital camera that captures images by hand, but also a digital camera that performs remote control, a camera that is installed in an environment and that can be autonomously pan-tilt driven, a camera that can move autonomously, and the like There may be. Further, the size, shape, method, and the like of the photographing apparatus 1000 are not particularly limited, and there is no need to limit the subject, the scene, and the like.

  As described above, according to the present embodiment, the imaging apparatus 1000 evaluates the aesthetics and the degree of reflection of each subject with respect to the imaging data, calculates the cumulative subject score across the past imaging data, and accumulates it. The photographer is presented to preferentially photograph the subject with the lowest subject score. By doing so, the photographing apparatus 1000 can average the degree of reflection of each subject. That is, the photographer can efficiently take pictures that are appreciated by the participants and other related parties using the photographing apparatus 1000.

<Embodiment 2>
The main difference between the present embodiment and the first embodiment is that shooting is performed using an imaging apparatus capable of autonomously driving the pan / tilt instead of a handheld digital camera. Another difference is that the imaging device does not acquire social interest data from the social networking service, but generates social interest data of the subject from the imaging data. Another difference is that the social interest data generated by the photographing apparatus is used to set the main subject and perform aesthetic evaluation, and the social interest data is used when setting the photographing priority.

FIG. 15 is a diagram illustrating an example of a functional configuration of the imaging apparatus 1000.
As described above, the image capturing apparatus 1000 according to the present embodiment is an image capturing apparatus capable of autonomous pan / tilt driving.
The imaging unit 1006 is as described in the first embodiment.
The calculation unit A142000 performs various calculations in the photographing process. Each function included in the calculation unit A142000 is a function realized by the CPU 1001 executing a program stored in the HD 1004 or the like. The calculation unit A142000 includes a social interest data generation unit A142001, a main subject setting unit A142002, an aesthetic evaluation unit A142003, and a subject detection unit A142004. Furthermore, the calculation unit A142000 includes a subject score calculation unit A142005, a cumulative subject score calculation unit A142006, a shooting priority setting unit A142007, and a shooting control unit A142008. The calculation unit A142000 includes a social interest data holding unit A142010, an aesthetic evaluation result holding unit A142030, a subject score holding unit A142050, a cumulative subject score holding unit A12040, and a shooting priority holding unit A142070 as storage areas.

The social interest data generation unit A142001 estimates the connection between users and the interests of each user from the shooting data acquired by the shooting unit 1006, generates social interest data, and holds it in the social interest data holding unit A142010. Details of the processing by the social interest data generation unit A142001 will be described later with reference to FIG. The social interest data holding unit A142010 holds the social interest data generated by the social interest data generation unit A142001, and outputs it in response to a request.
The main subject setting unit A142002 acquires the social interest data held in the social interest data holding unit A142010, sets the main subject in the captured image from the structure of the acquired social interest data, and outputs it. Details of processing by the main subject setting unit A142002 will be described later with reference to FIG.
The aesthetic evaluation unit A142003 evaluates the aesthetics of the shooting data acquired by the shooting unit 1006 in consideration of the main subject set by the main subject setting unit A142002, and outputs an aesthetic evaluation result. Details of the processing by the aesthetic evaluation unit A142003 will be described later with reference to FIG. The esthetic evaluation result holding unit A142030 holds the esthetic evaluation result evaluated by the aesthetic evaluation unit A142003, and outputs it on demand.

A subject detection unit A142004 detects a subject from the shooting data acquired by the shooting unit 1006, and outputs a subject list.
The subject score calculation unit A142005 is based on the photographing data acquired by the photographing unit 1006, the aesthetic evaluation result held in the aesthetic evaluation result holding unit A142030, and the reflection of each subject detected by the subject detection unit A142004. A subject score for each subject is calculated. Then, the subject score calculation unit A142005 outputs the calculated subject score of each subject and holds it in the subject score holding unit A142050. Details of the processing performed by the subject score calculation unit A142005 will be described later with reference to FIG. The subject score holding unit A142050 holds the subject score calculated by the subject score calculation unit A142005 and outputs it when requested.
The cumulative subject score calculation unit A142006 performs a calculation process based on the subject score for each photographing of each subject held in the subject score holding unit A142050 and the cumulative subject score held in the cumulative subject score holding unit A142060. More specifically, the cumulative subject score calculation unit A142006 calculates a cumulative subject score obtained by transversely evaluating shooting data shot in the past for each subject from the subject score and the cumulative subject score. It is held in the cumulative subject score holding unit A142060. Note that the processing by the cumulative subject score calculation unit A142006 is an example of cumulative information acquisition processing. Details of the processing performed by the cumulative subject score calculation unit A142006 will be described later with reference to FIG. The cumulative subject score holding unit A142060 holds the cumulative subject score calculated by the cumulative subject score calculation unit A142006 and outputs it when requested.

The shooting priority setting unit A142007 calculates the shooting priority from the cumulative subject score held in the cumulative subject score holding unit A142060 and the social interest data held in the social interest data holding unit A142010. Then, the shooting priority setting unit A142007 holds the calculated shooting priority in the shooting priority holding unit A142070. Details of the processing by the shooting priority setting unit A142007 will be described later with reference to FIG. The shooting priority holding unit A142070 holds shooting priority for each subject and outputs it when requested.
The shooting control unit A142008 acquires the shooting priority for each subject held in the shooting priority holding unit A142070, and controls the shooting unit 1006 based on the acquired shooting priority.

FIG. 16 is a diagram illustrating an example of the imaging apparatus 1000 according to the present embodiment. In the present embodiment, description will be given using an imaging device 15000 capable of autonomous pan / tilt driving shown in FIG. 16 as an example of the imaging device 1000.
The photographing apparatus 15000 generates social interest data from the photograph taken by the photographing unit 1006, sets a main subject, performs aesthetic evaluation, detects a subject, and calculates a subject score for each subject. Further, the photographing apparatus 15000 calculates a cumulative subject score from the subject score of each subject, sets a photographing priority, and controls the photographing unit 1006.
The photographing apparatus 15000 includes a memory that can be used as a social interest data holding unit A142010, a main subject holding unit A142020, and an aesthetic evaluation result holding unit A142030. Further, the photographing apparatus 15000 includes a memory that can be used as a subject score holding unit A142050, a cumulative subject score holding unit A142060, and a photographing priority holding unit A142070.
The above is the configuration example of the imaging device 15000 in the present embodiment.

The processing flow of this embodiment will be described with reference to FIG.
FIG. 17 is a flowchart illustrating an example of processing in the present embodiment.
In S16100, the social interest data generation unit A142001 estimates the friendship and interest between the subjects from the shooting data, generates social interest data, and holds the social interest data in the social interest data holding unit A142010.
In step S16200, the main subject setting unit A142002 acquires information about friendship and interest between subjects from the social interest data held in the social interest data holding unit A142010. The main subject setting unit A142002 sets a main subject based on the acquired information, and holds information related to the setting of the main subject (hereinafter simply referred to as a main subject) in the main subject holding unit A142020. Note that the process of S16200 is an example of a main subject determination process.
In step S16300, the aesthetic evaluation unit A142003 acquires the shooting data shot by the shooting unit 1006 and the main subject held in the main subject holding unit A142020, evaluates the aesthetics, and stores the aesthetic evaluation result holding unit. A142030.

In step S <b> 16400, the subject detection unit A142004 detects the subject from the shooting data shot by the shooting unit 1006. Then, the subject score calculation unit A142005 acquires the photographing data photographed by the photographing unit 1006 and the aesthetic evaluation result held in the aesthetic evaluation result holding unit A142030, and the subject score detected by the subject detection unit A142004. Is calculated. Further, the subject score calculation unit A142005 holds the calculated subject score in the subject score holding unit A142050.
In step S16500, the cumulative subject score calculation unit A142006 updates the cumulative subject score held in the cumulative subject score holding unit A12040 from the subject score held in the subject score holding unit A142050, and holds it in the cumulative subject score holding unit A12040. .
In S16600, the shooting priority setting unit A142007 sets shooting priority from the cumulative subject score held in the cumulative subject score holding unit A142060 and the social interest data held in the social interest data holding unit A142010. Then, the shooting priority setting unit A142007 holds the set shooting priority in the shooting priority holding unit A142070.
In step S16700, the shooting control unit A142008 acquires the shooting priority from the shooting priority holding unit A142070 and controls the shooting unit 1006.
When the above processing ends, the imaging device 15000 deletes the imaging data used for determining the imaging priority in the processing of FIG. 17 from the storage area. Accordingly, the user can effectively use the storage area of the photographing apparatus 15000.
The above is the processing flow of the photographing apparatus 15000 in the present embodiment.

Hereinafter, the process of FIG. 17 will be described using a more specific example.
FIG. 18 is a diagram illustrating an example of a shooting scene in which a shooting device 15000 capable of autonomous pan / tilt drive shooting of a plurality of subjects 17001 to 17004 at a party venue.
The photographing apparatus 15000 in the present embodiment efficiently photographs photographs useful for the subject, the operator of the photographing apparatus 15000, and other related parties by setting a photographing priority for each subject when photographing.
FIG. 19 is a diagram illustrating an example of shooting data 18000 shot by the shooting apparatus 15000.
The imaging device 15000 processes the imaging data 18000 according to the flow shown in FIG.
In S16100, the social interest data generation unit A142001 generates social interest data from the shooting data 18000. In the present embodiment, the social interest data generation unit A142001 generates social interest data using the face orientation of each subject. The social interest data generation unit A142001 detects a face by detecting a known face-specific image feature pattern area stored in advance in a storage area, and determines the orientation of the detected face. In order to detect the face orientation, the social interest data generation unit A142001 creates a dictionary for each face orientation from face images in various directions, and detects it using a detector using the dictionary.

FIG. 20 is a diagram illustrating an example of imaging data indicating an object of interest between subjects.
As shown in FIG. 20, in the shooting data 18000, the subject 17002 and the subject 17004 are oriented in the direction of the subject 17003. Therefore, it is considered that the subject 17002 and the subject 17004 are interested in the subject 17003.
FIG. 21 is a diagram illustrating an example of social interest data regarding the interest between the above-described subjects.
In FIG. 21, a neutral state that is not of special interest and interest is expressed as 1, and a state of interest and interest is expressed as 2. FIG. 21 shows that the subject 17002 and the subject 17004 are interested in the subject 17003. The social interest data generation unit A142001 holds the social interest data generated in this way in the social interest data holding unit A142010. Social interest data representing interest between subjects is an example of interest information. Further, the process of S16100 is an example of the interest information generation process.

  The method for generating social interest data is not necessarily limited to the above method. The social interest data generation unit A142001 may use the direction of the line of sight instead of the orientation of the face of the subject in order to generate social interest data. Also, the social interest data generation unit A142001 does not determine from a single still image, but determines whether or not there is an interest and a degree of interest in consideration of the time when the face is facing a specific direction from a moving image or a plurality of still images. It may be estimated. Further, the social interest data generation unit A142001 may estimate by combining the facial direction and line of sight, the facial expression, the body direction, and the distance between subjects. Further, the social interest data generation unit A142001 may use a representation method that weights the interest and degree of interest of each subject with a numerical value or a representation method using a directed graph as a representation method of the social interest data. Further, the social interest data generation unit A142001 generates social interest data from single shooting data, but it is not necessary to be limited to this. For example, the social interest data generation unit A142001 may generate a combination of previously captured image data, previously generated social interest data, data acquired from the outside using a social networking service, and the like.

Returning to the description of FIG.
In step S16200, the main subject setting unit A142002 sets the main subject in the shooting data 18000 based on the social interest data held in the social interest data holding unit A142010. In the present embodiment, the main subject setting unit A142002 sets, as a main subject, a subject that has many subjects and is interested. The social interest data held in the social interest data holding unit A142010 is as shown in FIG. According to FIG. 21, the subject 17002 and the subject 17004 are interested in the subject 17003. Therefore, the main subject setting unit A142002 sets the subject 17003 in the shooting data 18000 as the main subject. However, two or more main subjects may exist. As a main subject setting method, not only the face direction but also a setting method using unique knowledge (data) according to the scene may be used. For example, the main subject setting unit A142002 may recognize a subject that is spotlighted on the stage of the play as a main subject, or may recognize a subject having a microphone as a main subject at a party venue. The main subject setting unit A142002 may recognize a subject wearing a wedding dress as a main subject at a wedding. Further, the main subject setting unit A142002 may determine the main subject using not only shooting data but also a sensor and other methods, and may set it as a main subject in response to an appeal such as a gesture from the subject. . The main subject setting unit A142002 may set the main subject by combining these various main subject setting methods and the degree of interest of each subject.

ここで、Ｋは対象の被写体が主被写体かどうかによって決定される値である。例えば、主被写体の場合は１、それ以外の場合は０．５とする。即ち、被写体スコア算出部Ａ１４２００５は、被写体が主被写体か否かに応じた重み付けをして被写体スコアを算出する。 Returning to the description of FIG.
In S16300, the aesthetic evaluation unit A142003 evaluates the aesthetics of the shooting data 18000. The aesthetic evaluation method by the aesthetic evaluation unit A142003 is the same as that of the first embodiment, except that not all the faces detected from the shooting data are used but the main subject is used. More specifically, the aesthetic evaluation unit A142003 draws a line that divides the photographing data 18000 vertically and horizontally into three parts as shown in FIG. 22, and depends on the distance between the intersection of these lines and the face area of the main subject. Assess aesthetics. The esthetic evaluation unit A142003 holds the esthetic evaluation result calculated in this way in the esthetic evaluation result holding unit A142030. Note that FIG. 22 is a diagram illustrating an example of imaging data regarding aesthetic evaluation.
Returning to the description of FIG.
The process of S16400 is the same as the process of S3200 in the first embodiment, but the mathematical formula used for calculating the subject score is different. The subject score calculation unit A142005 in the present embodiment calculates the subject score J using the following equation (3).

Here, K is a value determined by whether or not the subject is the main subject. For example, 1 is set for the main subject, and 0.5 is set otherwise. That is, the subject score calculation unit A142005 calculates the subject score by weighting according to whether or not the subject is the main subject.

FIG. 23 is a diagram illustrating an example of the subject score calculated by the subject score calculation unit A142005.
Note that the formula and parameters described in this embodiment do not necessarily have to be used for the subject score calculation method, and other mathematical formulas and parameters may be used.
Returning to the description of FIG.
The process of S16500 is the same as the process of S3300 in the first embodiment. The cumulative subject score calculation unit A142006 updates the cumulative subject score by adding the subject score related to new shooting data held in the subject score holding unit A142050 to the cumulative subject score already held in the cumulative subject score holding unit A12040. Then, the cumulative subject score calculation unit A142006 holds the updated new cumulative subject score in the cumulative subject score holding unit A142060. FIG. 24 is a diagram illustrating an example of the cumulative subject score obtained by the cumulative subject score calculation unit A142006.

Returning to the description of FIG.
The process of S16600 is the same as the process of S3500 in the first embodiment, except that the result of the social interest data held in the social interest data holding unit A142010 is considered. More specifically, the shooting priority setting unit A142007 sets the shooting priority so that a subject that is noticed by another person is shot with priority. In the present embodiment, the shooting priority setting unit A142007 allows an (n + 1) times cumulative subject score to be received every time attention is received from n people. In other words, the shooting priority setting unit A142007 determines that a person who is attracting attention from many people is a subject to be shot even if the cumulative subject score is high.
For example, according to FIG. 24, the subject 17003 has the second highest cumulative subject score among the four subjects. Here, FIG. 24 is a diagram illustrating an example of the cumulative subject score result. Further, according to the social interest data shown in FIG. 21 generated by the social interest data generation unit A142001, it can be seen that the subject 17003 is an object of interest from two subjects. For this reason, the subject 17003 is allowed to have a cumulative subject score up to 3 times. That is, in the present embodiment, the shooting priority setting unit A142007 multiplies the cumulative subject score of the subject 17003 by (1/3) to obtain the shooting priority. Therefore, assuming that the cumulative subject score calculated without considering the attention level is as shown in FIG. 24, the shooting priority setting unit A142007 sets the cumulative subject score of the subject 17003 when considering the attention level. 260/3 = 866.6666, ie 87. In this case, the cumulative subject score of the subject 17003 is the third largest among the four subjects.

FIG. 25 is a diagram illustrating an example of photographing priority.
Since the shooting priority setting unit A142007 sets the shooting priority as described above, the shooting priority is as shown in FIG. The shooting priority setting unit A142007 holds the shooting priority set as shown in FIG. 25 in the shooting priority holding unit A142070. As described above, in this embodiment, the shooting priority setting unit A142007 calculates the shooting priority by biasing (weighting) the cumulative subject score based on how many subjects the subject is interested in. However, the method for calculating the shooting priority need not be limited to this. For example, the imaging priority setting unit A142007 may weight the subject in order from the most interested subject, or the most interested person is interested. You may make it give the largest weighting to a person's accumulated subject score.

In step S16700, the shooting control unit A142008 searches for a subject with the highest shooting priority held in the shooting priority holding unit A142070, and performs control related to shooting. The shooting control unit A142008 performs pan-tilt driving at random while constantly detecting and identifying a person with respect to the live view video of the shooting device 15000, and searches for a subject with the highest shooting priority.
If the subject cannot be found even when the photographing device 15000 is driven to all angles capable of pan / tilt driving, the photographing control unit A142008 searches for the subject with the second (second) highest photographing priority. . At this time, the imaging control unit A142008 may move the imaging device 15000 by a predetermined method instead of randomly moving the imaging device 15000. In addition, the shooting control unit A142008 may skip to the next search when the subject having the highest shooting priority cannot be found continuously for a predetermined number of times. Also, the shooting control unit A142008 randomly performs pan-tilt driving, and if a subject to be searched cannot be found after a predetermined time has elapsed, the shooting control unit A142008 searches for a subject with the next highest shooting priority. You may do it. As a method for searching for a subject, an active RFID tag is attached to the subject, and the photographing control unit A142008 scans when photographing to grasp the position of the subject in the space so that the subject can be photographed. Alternatively, a method of performing pan / tilt driving may be used. Furthermore, as a method for searching for a subject, a method may be used in which a subject to be photographed is searched with a subject search camera and the location of the subject to be photographed is notified to the photographing camera.

The photographing control unit A142008 may perform control so that photographing is performed with the highest priority when a subject having no accumulated subject score is detected. In the present embodiment, the shooting control unit A142008 preferentially captures the subject with the highest shooting priority held in the shooting priority holding unit A142070, but performs shooting regardless of the shooting priority depending on the situation. You may do it. For example, the shooting control unit A142008 may shoot a subject when the subject is requested to be shot by a gesture or the like. In addition, when the shooting control unit A142008 finds a subject with the highest shooting priority but is predicted not to be able to take a picture well due to occlusion or the like, the shooting control unit A142008 may preferentially shoot other subjects.
In addition, the image capturing apparatus 15000 in the present embodiment may not only perform pan / tilt driving but also have a moving function. In addition, the photographing device 15000 may be a device in which a digital camera used with a hand gives a movement instruction to a photographer. Further, the size, shape, method, and the like of the photographing apparatus 15000 are not particularly limited, and it is not necessary to limit the subject, the scene, and the like.

  As described above, according to the present embodiment, the photographing apparatus 15000 sets the main subject by estimating the interest of the subject feeling from the photographing data, considers the main subject, the aesthetics, and how each subject is reflected. And the cumulative subject score is calculated across the past shooting data. In addition, the imaging device 15000 calculates the imaging priority based on the social interest data and the cumulative subject score, and performs imaging based on the calculated imaging priority. As a result, the image capturing device 15000 can capture an image of a subject in which the subject is interested. In other words, the photographer can efficiently take pictures that are pleased by the subject, the participants, and other parties using the photographing apparatus 15000.

<Embodiment 3>
The main difference between this embodiment and Embodiment 1 and Embodiment 2 is that shooting is performed using an autonomously moving imaging device. Another difference is that the photographing device generates social interest data from the image data, and corrects and sets the photographing priority of the subject focused on by the subject having the highest cumulative subject score based on the generated social interest data. .
FIG. 26 is a diagram illustrating an example of a functional configuration of the photographing apparatus 1000.
As described above, the imaging device 1000 in the present embodiment is an autonomous mobile imaging device.
The imaging unit 1006 is as described in the first embodiment.
The calculation unit A252000 performs various calculations in the photographing process. Each function of the calculation unit A252000 is a function realized by the CPU 1001 executing a program stored in the HD 1004 or the like. The calculation unit A252000 includes an aesthetic evaluation unit A252001, a subject detection unit A252002, and a subject score calculation unit A252003. Furthermore, the calculation unit A252000 includes an accumulated subject score calculation unit A252004, a social interest data generation unit A252005, and a shooting priority setting unit A252006. Further, the calculation unit A252000 includes an aesthetic evaluation result holding unit A252010 and a subject score holding unit A252030 as storage areas. Furthermore, the calculation unit A252000 includes a cumulative subject score holding unit A252040, a social interest data holding unit A252050, and a photographing priority holding unit A252060 as storage areas.

The aesthetic evaluation unit A252001 evaluates the aesthetics of the shooting data acquired by the shooting unit 1006, and outputs an aesthetic evaluation result. Details of processing by the aesthetic evaluation unit A252001 will be described later with reference to FIG. The esthetic evaluation result holding unit A252010 holds the aesthetic evaluation result evaluated by the aesthetic evaluation unit A252001, and outputs it as required.
A subject detection unit A252002 detects a subject from the shooting data acquired by the shooting unit 1006, and outputs a subject list. Details of processing by the subject detection unit A252002 will be described later with reference to FIG.
The subject score calculation unit A252003 calculates a subject score for each subject detected by the subject detection unit A252002 from the aesthetic evaluation result held in the aesthetic evaluation result holding unit A252010 and the degree of reflection of each subject. ,Output. Details of the processing performed by the subject score calculation unit A252003 will be described later with reference to FIG. The subject score holding unit A252030 holds the subject score calculated by the subject score calculation unit A252003, and outputs it when requested.

The cumulative subject score calculation unit A252004 performs calculation processing based on the subject score of each subject for each shooting held in the subject score holding unit A252030 and the cumulative subject score held in the cumulative subject score holding unit A252040. More specifically, the cumulative subject score calculation unit A252004 calculates a cumulative subject score obtained by cross-evaluating shooting data shot in the past for each subject from the subject score and the cumulative subject score. It is held in the cumulative subject score holding unit A252040. Note that the processing by the cumulative subject score calculation unit A252004 is an example of cumulative information acquisition processing. Details of the processing by the cumulative subject score calculation unit A252004 will be described later with reference to FIG. The cumulative subject score holding unit A252040 holds the cumulative subject score calculated by the cumulative subject score calculation unit A252004, and outputs it when requested.
The social interest data generation unit A252005 generates social interest data from the shooting data acquired by the shooting unit 1006 and the cumulative subject score of each subject held in the cumulative subject score holding unit A252040. The social interest data generation unit A252005 holds the generated social interest data in the social interest data holding unit A252050. Details of the processing by the social interest data generation unit A252005 will be described later with reference to FIG. The social interest data holding unit A252050 holds the social interest data generated by the social interest data generation unit A252005 and outputs it in response to a request.

The shooting priority setting unit A252006 calculates shooting priority regarding which subject will be preferentially shot from the social interest data held in the social interest data holding unit A252050, and holds it in the shooting priority holding unit A252060. . Details of the processing by the photographing priority setting unit A252006 will be described later with reference to FIG. The shooting priority holding unit A252620 holds shooting priority for each subject and outputs it when requested.
The shooting control unit A252007 acquires the shooting priority for each subject held in the shooting priority holding unit A252620 and controls the shooting unit 1006. Details of the processing by the imaging control unit A252007 will be described later with reference to FIG.

FIG. 27 is a diagram illustrating an example of the imaging apparatus 1000 according to the present embodiment. In the present embodiment, description will be made using an autonomous mobile imaging device 26000 shown in FIG. 27 as an example of the imaging device 1000.
In this embodiment, the autonomous mobile imaging device 26000 having the imaging unit 1006 and the calculation unit A252000 described above is used. The autonomous mobile imaging device 26000 evaluates aesthetics for the photos taken by the imaging unit 1006, detects subjects, and calculates subject scores for each subject. Then, the autonomous mobile imaging device 26000 calculates a cumulative subject score from the subject score of each subject. Further, the autonomous mobile imaging device 26000 generates social interest data from the imaging data and the cumulative subject score, sets the imaging priority, and controls the imaging unit 1006.
Further, as described above, the photographing apparatus 15000 is equipped with a memory that can be used as the aesthetic evaluation result holding unit A252010, the subject score holding unit A252030, and the cumulative subject score holding unit A252040. Further, the photographing apparatus 15000 is equipped with a memory that can be used as the social interest data holding unit A252050 and the photographing priority holding unit A252060.
The above is the configuration example of the autonomous mobile imaging device 26000 in the present embodiment.

FIG. 28 is a flowchart illustrating an example of processing in the present embodiment.
In step S27100, the aesthetic evaluation unit A252001 acquires the shooting data shot by the shooting unit 1006, evaluates aesthetics, and holds the aesthetic evaluation result holding unit A252010.
In step S27200, the subject detection unit A252002 detects a subject from the shooting data shot by the shooting unit 1006. Then, the subject score calculation unit A252003 acquires the shooting data shot by the shooting unit 1006 and the aesthetic evaluation result held in the aesthetic evaluation result holding unit A252010, and calculates the subject score of each detected subject. To do. The subject score calculation unit A252003 holds the calculated subject score in the subject score holding unit A252030.
In S27300, the cumulative subject score calculation unit A252004 updates the cumulative subject score held in the cumulative subject score holding unit A252040 from the subject score held in the subject score holding unit A252030, and holds it in the cumulative subject score holding unit A12040. .

In step S27400, the social interest data generation unit A252005 calculates and acquires social interest data from the information on the friendship and interest between the subjects acquired from the shooting data and the cumulative subject score. And social interest data production | generation part A252005 hold | maintains the acquired social interest data in social interest data holding | maintenance part A252050.
In step S27500, the shooting priority setting unit A252006 sets the shooting priority from the social interest data held in the social interest data holding unit A252050.
In step S27600, the shooting control unit A252007 acquires the shooting priority from the shooting priority holding unit A252060 and controls the shooting unit 1006.
When the above processing is completed, the autonomous mobile imaging device 26000 deletes the imaging data used for determining the imaging priority in the processing of FIG. 28 from the storage area. Thereby, the user can effectively utilize the storage area of the autonomous mobile imaging device 26000.
The above is the processing flow of the autonomous mobile imaging device 26000 in the present embodiment.

Hereinafter, the process of FIG. 28 will be described using a more specific example.
FIG. 29 is a diagram illustrating an example of a shooting scene in which the autonomous mobile imaging device 26000 captures a plurality of subjects 28001 to 28004 at a party venue.
In the present embodiment, when the autonomous mobile imaging device 26000 performs imaging, the imaging priority is set for each subject and imaging is performed. Thereby, the autonomous mobile photographing device 26000 can efficiently photograph a photograph useful for an operator, a subject, and other related parties.
FIG. 30 is a diagram illustrating an example of imaging data captured by the autonomous mobile imaging device 26000.
The autonomous mobile imaging device 26000 processes the imaging data 29000 according to the flow shown in FIG.

Returning to the description of FIG.
The process of S27100 is the same as the process of S3100 in the first embodiment.
The process of S27200 is the same as the process of S3200 in the first embodiment.
The process of S27300 is the same as the process of S3300 in the first embodiment.
In step S27400, the social interest data generation unit A252005 generates social interest data from the shooting data 29000. First, the social interest data generation unit A252005 generates social interest data using the face orientation of each subject as in the processing of S16100 in the second embodiment. Here, it is assumed that the social interest data generation unit A252005 can obtain the social interest data shown in FIG. FIG. 31 is a diagram showing an example of social interest data in the present embodiment. Next, the social interest data generation unit A252005 acquires the cumulative subject score held in the cumulative subject score holding unit A252040. Here, it is assumed that the social interest data generation unit A252005 has obtained the cumulative subject score shown in FIG. FIG. 32 is a diagram illustrating an example of a cumulative subject score result.

The social interest data generation unit A252005 corrects the generated and acquired social interest data using the acquired cumulative subject score. That is, the social interest data generation unit A252005 multiplies the cumulative subject score of each subject and a value (interest value) indicating the degree of interest in each subject. By this correction, the subject interested in the subject having the highest cumulative subject score has a higher interest value than before the correction, and is considered to have more interest than before the correction.
FIG. 33 is a diagram illustrating an example of social interest data corrected by the above-described method.
The social interest data generation unit A252005 holds the social interest data shown in FIG. 33 acquired by correction in the social interest data holding unit A252050.
The social interest data correction method is not necessarily limited to the method of multiplying the cumulative subject score by the interest value of the social interest data, and other calculation methods may be used. Further, the social interest data generation unit A252005 may hold past social interest data and update and use the data.

Returning to the description of FIG.
The process of S27500 is the same as the process of S3500 in the first embodiment. However, the difference is that the shooting priority setting unit A252006 sets the shooting priority from the social interest data held in the social interest data holding unit A252050. According to FIG. 33, it can be seen that the subject 28003 is attracting the most interest from other subjects. Therefore, the shooting priority setting unit A252006 sets the shooting priority so that the subject 28003 is preferentially shot. Note that the shooting priority setting unit A252006 in the present embodiment calculates the shooting priority based on how much the subject is interested from other subjects. However, the method for calculating the shooting priority need not be limited to this method. For example, the shooting priority setting unit A252006 may consider the number of interested people in the social interest data, not the total score of interested interests, or may be inclined instead of simply summing them. May be attached.

In step S27600, the shooting control unit A252007 searches for a subject held in the shooting priority holding unit A252620 and performs shooting. Since the search method has been described above, a description thereof will be omitted. In the present embodiment, the shooting control unit A252007 preferentially captures the subject with the highest shooting priority held in the shooting priority holding unit A252620, but as described above, the shooting priority is not limited to the shooting priority. You may take a picture.
Note that the autonomous mobile imaging device 26000 in the present embodiment may be one that performs only pan / tilt driving without having a moving function. In addition, the moving method may be a method in which a digital camera that is used by hand gives a moving instruction to the photographer. In addition, the size, shape, method, and the like of the autonomous mobile imaging device 26000 need not be limited, and the subject, scene, and the like need not be limited.

  As described above, according to the present embodiment, the autonomous mobile imaging device 26000 evaluates the aesthetics and the degree of reflection of each subject with respect to the imaging data, and calculates the cumulative subject score across the past imaging data. To do. Then, the autonomous mobile imaging device 26000 generates social interest data from the cumulative subject score and the imaging data. Furthermore, the autonomous mobile photographing device 26000 calculates photographing priority based on the generated social interest data, and preferentially photographs a subject with a low photographing priority. As a result, the autonomous mobile imaging device 26000 can shoot a subject whose subject is interested in the scene, and photographs that are appreciated by the subject, participants, and other related parties. It becomes possible to shoot efficiently.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  As described above, according to each of the above-described embodiments, it is possible to provide a technique for efficiently capturing a photograph useful for a subject, a photographer, and other related parties.

  The preferred embodiment of the present invention has been described in detail above, but the present embodiment is not limited to the specific embodiment, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

Claims (16)

Cumulative information acquisition means for acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the degree of reflection of the photographed subject for each photographing;
A determination unit that determines a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information acquired by the cumulative information acquisition unit;
Presenting means for presenting instruction information for instructing photographing of the subject determined by the determining means;
A photographing apparatus having Score calculating means for calculating the score based on the composition related to the image of the photographed subject;
The photographing apparatus according to claim 1, wherein the cumulative information acquisition unit calculates a cumulative value obtained by accumulating the score calculated by the score calculation unit for each photographing, and acquires cumulative information indicating the calculated cumulative value.   3. The photographing apparatus according to claim 1, wherein the determining unit determines a subject having the lowest cumulative value indicated by the cumulative information among the plurality of subjects as a subject to be preferentially photographed among the plurality of subjects to be photographed. It further has relationship information acquisition means for acquiring relationship information indicating the relationship between subjects from the server device,
The determination means determines a subject to be preferentially photographed from a plurality of subjects to be photographed based on the accumulated information, and photographs with the subject based on the relationship information acquired by the relationship information acquisition means. The imaging device according to claim 1, wherein the subject is determined. Cumulative information acquisition means for acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the degree of reflection of the photographed subject for each photographing;
A determination unit that determines a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information acquired by the cumulative information acquisition unit;
A photographing control unit configured to control the device so as to photograph the subject determined by the determining unit; Interest information generating means for generating interest information indicating an object of interest between the captured subjects from the imaging data,
6. The photographing apparatus according to claim 5, wherein the determining unit determines a subject to be preferentially photographed from among a plurality of subjects to be photographed based on the accumulated information and the interest information generated by the interest information generating unit. . Main subject determining means for determining a subject as a main subject among the plurality of subjects based on the interest information generated by the interest information generating means;
Score calculating means for calculating the score by weighting the subject determined by the main subject determining means among the plurality of subjects;
Further comprising
The photographing apparatus according to claim 6, wherein the cumulative information acquisition unit calculates a cumulative value obtained by accumulating the score calculated by the score calculation unit for each photographing, and acquires cumulative information indicating the calculated cumulative value.   The photographing apparatus according to claim 6 or 7, wherein the determining unit determines a subject having the lowest cumulative value indicated by the cumulative information among the plurality of subjects as a subject to be preferentially photographed among the plurality of subjects to be photographed. The interest information generating means generates the interest information corrected based on the accumulated value indicated by the accumulated information,
The determination unit preferentially selects a subject having the highest product of the cumulative value specified based on the corrected interest information and an interest value indicating a degree related to the subject of interest from among a plurality of subjects to be photographed. The photographing apparatus according to claim 6, wherein the photographing apparatus is determined as a subject to be photographed.   The photographing apparatus according to claim 5, wherein the photographing control unit performs control related to pan / tilt driving of the own apparatus or control related to movement of the own apparatus so as to photograph the subject determined by the determining unit. . A system including a photographing device and a server device,
The imaging device
An acquisition means for acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the image of the photographed subject for each photographing;
Receiving means for receiving relationship information indicating the relationship between subjects from a server device;
A determining unit that determines a subject to be preferentially photographed among a plurality of subjects to be photographed based on the accumulated information obtained by the obtaining unit and the relationship information received by the receiving unit;
Presenting means for presenting instruction information for instructing photographing of the subject determined by the determining means;
Have
The server device
The system which has a transmission means to transmit the said relationship information to the said imaging device according to the request | requirement from the said imaging device. An information processing method executed by a photographing apparatus,
A cumulative information acquisition step of acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the reflected state of the photographed subject for each photographing;
A determination step for determining a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information obtained in the cumulative information obtaining step;
A presenting step for presenting instruction information for instructing photographing of the subject determined in the determining step;
An information processing method including: An information processing method executed by a photographing apparatus,
A cumulative information acquisition step of acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the reflected state of the photographed subject for each photographing;
A determination step for determining a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information obtained in the cumulative information obtaining step;
A shooting control step of controlling the own device so as to shoot the subject determined in the determination step;
An information processing method including: An information processing method in a system including an imaging device and a server device,
An acquisition step in which the photographing apparatus obtains cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the degree of reflection of the photographed subject for each photographing;
A transmission step in which the server device transmits relationship information indicating a relationship between subjects to the imaging device in response to a request from the imaging device;
A receiving step in which the imaging device receives the related information from a server device;
A determination step in which the photographing apparatus determines a subject to be preferentially photographed among a plurality of subjects to be photographed based on the accumulated information obtained in the obtaining step and the relationship information received in the receiving step; ,
A presenting step in which the photographing device presents instruction information for instructing photographing of the subject determined in the determining step;
An information processing method including: On the computer,
A cumulative information acquisition step of acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the reflected state of the photographed subject for each photographing;
A determination step for determining a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information obtained in the cumulative information obtaining step;
A presenting step for presenting instruction information for instructing photographing of the subject determined in the determining step;
A program for running On the computer,
A cumulative information acquisition step of acquiring cumulative information indicating a cumulative value obtained by accumulating a score corresponding to the reflected state of the photographed subject for each photographing;
A determination step for determining a subject to be preferentially photographed among a plurality of subjects to be photographed based on the cumulative information obtained in the cumulative information obtaining step;
A shooting control step of controlling the own device so as to shoot the subject determined in the determination step;
A program for running

Images