JP5173666B2 - camera - Google Patents

Description

The present invention also relates to a camera and, more particularly, the present invention relates to a camera that allows photo viewing with a narrative.

  With the widespread use of digital cameras, images are digitized and can be easily transferred, and various image viewing possibilities such as the use of slideshows and automatic layout using electronic albums are expanding.

  Also, anyone can add music to BGM or synthesize characters into images if it takes time. Even in the above-mentioned slide show, it is possible to convey a richer feeling through interaction using a plurality of images.

  In this trend, if you get used to elaborate images, even if you can show only the images taken with the camera, there are an increasing number of viewers who feel monotonous or tasteless. On the other hand, even if a casual comment is included, the photographer may feel familiar and the atmosphere of the photograph may change. Therefore, a technique for easily adding comments to images is being sought.

In addition, there is a need for an expression method that allows a story to be created as a whole by adding comments to individual images in the aforementioned slide show, electronic album, and the like. For example, there are technologies such as Patent Document 1 and Patent Document 2 described below for simply adding a comment to an image. For example, Patent Document 1 below discloses a digital still camera and a control method thereof that can easily add specific identification information useful for rearrangement / retrieval to a captured image. Patent Document 2 below discloses a technique for associating a photograph with a haiku using a seasonal word or the like.
JP 2007-228608 A JP 2004-240579 A

  However, the camera described in Patent Document 1 is troublesome because the user must select icons displayed on the touch panel one by one. Moreover, since the technique described in Patent Document 2 is completed with a single photograph, it is impossible to obtain a narrative from the photographed photograph group.

Accordingly, the present invention has been made in view of the above circumstances, and its purpose is to provide a keyword that advises shooting and a comment that modifies the keyword from the contents of an image obtained as a result of shooting according to the keyword. synthesized and to provide a camera capable of displaying an image.

According to the first aspect of the present invention, there is provided a photographing unit for photographing a subject, a theme extracting unit for extracting and outputting information on a candidate for a subject to be photographed next for a photographer, and the theme extracting unit. Based on the information, notification means for notifying the photographer of information relating to a candidate for a subject to be photographed next as advice, and feature detection means for detecting the characteristics of the image photographed based on the advice by the photographing unit; , to create a phrase combined and wording extracted based on the output of the feature detection means wording of candidates of related phrases to the output of the theme extracting unit, the created phrase was the shot Phrase synthesizing means for notifying the photographer together with the image.

The invention described in claim 2 further comprises a position acquisition unit that acquires position information at the time of shooting, and a time detection unit that acquires time information at the time of shooting in the invention of claim 1. The theme extracting unit extracts information on a candidate for a subject to be photographed next based on at least one of the position information and time information at the time of photographing.

According to a third aspect of the present invention, in the second aspect of the present invention, at least one of the positional information at the time of photographing acquired by the position acquisition unit and the time information at the time of photographing acquired by the time detection unit. And further comprising a communication unit for acquiring specific information from an external device.

According to a fourth aspect of the present invention, in the second aspect of the present invention, the phrase synthesizing unit adds the phrase candidate word to the phrase when the position information is acquired by the position acquisition unit. The phrase is created by synthesizing words extracted based on position information.

The invention according to claim 5 is the invention according to claim 2 , wherein when the time information is acquired by the time detection unit, the phrase synthesizing unit adds the phrase candidate words to the phrase candidate words. The phrase is created by synthesizing words extracted based on time information.

The invention according to claim 6 is the invention according to claim 2 , wherein when the color information is acquired by the feature detecting means, the phrase synthesizing means adds the phrase candidate words to the phrase candidate words. The phrase is created by synthesizing words extracted based on color information.

The invention according to claim 7, in the invention described in claim 2, if the photographed image matches the phrase that is created by the phrase combining means records 該Fu phrase and the image The recording apparatus further includes recording means for performing the above-described operation.

The invention according to claim 8, in the invention described in claim 1, further characterized by comprising a display means for displaying the phrase.

The invention according to claim 9, in the invention described in claim 8, wherein the display means, a camera according to claim 8, characterized in that displaying the phrase together with the captured image.

According to the present invention, and keywords to advise the shooting, from the contents of the image obtained as a result of the shooting in accordance with the keyword, the camera capable of displaying the image by combining the comments that modified the keyword Can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram illustrating a configuration of a digital camera 10 to which an imaging system is applied according to an embodiment of the present invention.

  In FIG. 1, a digital camera 10 includes a position detection unit 11, a time detection unit 12, a communication unit 13, a photographing unit 14, a feature detection unit 15, a database (DB) unit 16, and a scene extraction. A unit 18, a candidate display unit 19, a phrase synthesis unit 20, a display unit 20 including a candidate display unit 20 a, and a control unit 22 are configured.

  The position detection unit 11 is a position acquisition unit for receiving radio waves from GPS and detecting / acquiring position information of the digital camera 10 at the time of shooting. The time detection unit 12 has a clock function built in the digital camera 10 and acquires time information at the position acquired by the position detection unit 11. The communication unit 13 is for communication with an external device (not shown), for example, for acquiring specific information such as shooting spot information.

  The photographing unit 14 is photographing means for obtaining a photographic image at the time of photographing. The feature detection unit 15 is a feature detection unit that detects the feature of the photograph by analyzing the photograph image obtained from the photographing unit 14, and has an image processing function. For this feature detection, for example, color distribution or face detection is used. The database unit 16 is a recording unit that stores the shooting position, time, and photograph characteristics.

  The scene extraction unit 18 prompts the user who is a photographer to select a shooting theme from the position information and time information acquired first, and further extracts a next shooting scene from the characteristics of the photographed photograph and the shooting position information. Extraction means. The display unit 20 includes a candidate display unit 20a as a notification unit, and is a display unit such as a display for displaying an image (photograph) acquired by the photographing unit 14. The candidate display unit 20a displays the scene information extracted by the scene extraction unit 18 as advice. The phrase synthesizing unit 21 composes a phrase synthesizing unit by creating and synthesizing a phrase from the scene information displayed on the candidate display unit 20a and the feature of the photograph after photographing. The control unit 22 performs control in accordance with the user's operation, and controls the entire digital camera 10.

  Next, a method for associating a phrase with a photographed picture in the digital camera 10 according to an embodiment of the present invention will be described with reference to FIG.

  The season and time are specified based on the time information detected by the time detector 12. Specific keywords that represent the season include, for example, “Early Spring”, “Equinox”, “Spring”, “Early Summer”, “Summer”, “Autumn”, “Autumn”, “Winter”, “Winter Solstice”, etc. . Further, as keywords representing the specified time, for example, “early morning”, “morning”, “morning”, “noon”, “daytime”, “afternoon”, “takata”, “night”, “midnight” and so on.

  Based on the position information detected by the position detector 11, the position on the map is specified. Here, as a keyword indicating the position on the specified map, a rough thing indicating the place, for example, “mountain”, “field”, “sea”, and a mark indicating the place, For example, there are “school” and “company”. Phrases are synthesized from keywords representing seasons and keywords representing places and associated with photos.

  Furthermore, the feature of the subject is specified based on the information detected by the feature detection unit 15. The keywords representing the identified subject include “color (red, blue, yellow, green, etc.)”, “shape (circle, triangle, square, long, short, etc.)” and “estimated subject ( Flowers, people, cars, etc.) ".

  Phrases are synthesized based on the phrase template from the season, place, and subject keywords described above, and associated with the photo.

  For example, as shown in FIG. 2A, a hiking scene is estimated by a combination of “mountain” from position information, “autumn” from time information, and “person” from subject feature information. The phrase “hiking” 26a is synthesized and associated with the photograph.

  In addition, as shown in FIG. 2 (b), a red flower was photographed in a mountain by a combination of “mountain” as position information, “spring” as time information, and “red / flower” from subject feature information. Is estimated and the phrase 26b "I found a red flower" is synthesized and associated with the photograph.

  Next, referring to FIGS. 3 and 4, prompting the user to select a shooting theme will be described.

  FIG. 3 is a diagram showing a theme for prompting the user to select.

  In FIG. 3, a keyword representing a season specified from information acquired by the time detection unit 12 and a keyword indicating a place specified from information acquired by the position detection unit 11 are transmitted via the communication unit 13. An inquiry is made to a server (not shown), and a large frame of several themes associated with the picked-up keywords is acquired from the server.

  The main theme of the picked up theme is “Family Wild Mountain Hiking” and “Wild Bird Observation”. When the outline of the picked theme is selected by the user and “family mountain hiking” is selected by the user, an inquiry is made to a server (not shown) via the communication unit 13 based on the theme information. Next, scenario data, shooting spot information, phrase templates, keywords, and image feature data according to the theme are acquired from the server. Then, along with the scenario data, a shooting scene along the theme is advised.

  FIG. 4 is a diagram for explaining a shooting theme and advice.

  First, a shooting theme selection menu as shown in FIG. 4B is displayed on the display (display unit 20) of the digital camera 10 shown in FIG. Before the user starts photographing, a desired theme is selected by following a menu hierarchy as shown in FIG. In the example shown in FIG. 4A, first, “Private” is selected in the first menu level, “Excursion” is selected in the second menu level, and “Day trip” is selected in the third menu level. The

  Each menu in each selected menu hierarchy is used as a keyword. A candidate for a photographic scene is extracted from this keyword, position information at the start of photographing, and time information. For example, when the above example is selected, the position information is “mountain”, the time information is “spring”, and the combination of the selected keywords, as shown in FIG. In the category “”, advice on shooting scenes “1. Typical landscape”, “2. Beautiful flowers”, “3. With the whole family” is extracted.

  Next, with reference to FIG. 5, description will be given of the wording of the synthetic phrase candidate, the display advice, and the association of the photographed photograph group.

  FIG. 5A to FIG. 5C are diagrams for explaining the association of candidate phrases, display advice, and photographed photograph groups.

  The advices A1 to A3 as shown in FIG. 5B are related to the phrases M1, M2, M11, M12, M21, and M22 as shown in FIG. When the user takes a picture, the digital camera 10 displays advices A1 to A3 as shown in FIG. 5B on the candidate display unit (display device) 20a. The user takes a picture in consideration of the displayed advice. At this time, photographs P1 to P5 as shown in FIG. 5C are related to the advice. Therefore, the photograph associated with the advice and the phrase associated with the advice are combined.

  Next, the control operation of the digital camera 10 in this embodiment will be described with reference to the flowchart of FIG. Here, a case where the user selects a shooting theme, a case where the user takes a photograph, and a case where the photograph taken by the user is reproduced will be described.

  This control operation is mainly performed by the control unit 22.

  First, when the camera is activated, the shooting mode is confirmed in step S1. If it is the photographing mode, the process proceeds to step S2, and if not, the process proceeds to step S11 described later.

  In step S2, the state is changed to a state where photographing is possible. Here, when the shutter button (not shown) is not pressed (when a through image is displayed), the process proceeds to step S3. In step S3, shooting is performed and an image is recorded in the database unit 16.

  Next, in step S4, based on the data extracted from the photographed photograph, such as characteristics, photographing position, photographing time, etc., it is determined whether or not the relevant data is obtained and the data can be obtained. . If the data can be acquired, the process proceeds to step S5, and the acquired data is associated with the image. On the other hand, if the data cannot be acquired, this sequence ends.

  In step S6, a process for determining whether the image matches the photographing advice is performed. The detailed operation of the subroutine “advice image determination processing” in step S6 will be described later. As a result of the determination process in step S6, if the photographing advice is met in step S7, the process proceeds to step S8 and is recorded in the database unit 16 as an advice image. On the other hand, if the image does not match the photographing advice, the process proceeds to step S4, and data acquisition is performed again.

  If it is determined in step S2 that a shutter button (not shown) is not pressed (through image display is in progress), the process proceeds to step S9. Here, if a shooting theme is designated, the process proceeds to step S10 and advice display is performed on the candidate display unit 20a of the digital camera 10. Thereafter, the process proceeds to step S9.

  On the other hand, if no shooting theme has been selected in step S9, the process proceeds to step S1 to prompt the user to select a shooting theme.

  Next, an operation for reproducing a photograph taken by the user will be described.

  If it is determined in step S1 that the shooting mode is not set, it is determined in step S11 whether or not the playback mode is set. Here, if it is the reproduction mode, the process proceeds to step S12. If not, the process proceeds to step S19 described later.

  If it is determined in step S11 that the playback mode is selected, an image to be played back is selected in step S12. Next, in step S13, it is determined whether or not a reproduction image has been selected. If no image is selected, the process proceeds to step S11 to prompt the user to input again whether or not to shift to the reproduction mode.

  If it is determined in step S13 that a reproduced image has been selected, the process proceeds to step S14 to determine whether the selected image is based on shooting advice. Here, if it is a photograph based on photographing advice, the process proceeds to step S15, and a word (phrase) is synthesized based on the photographing advice and data at the time of photographing. The detailed operation of the subroutine “Combine comments from advice and shooting data” in step S15 will be described later. Next, in step S16, the synthesized word (phrase) is reproduced as a comment in association with the photographed photo.

  On the other hand, if it is determined in step S14 that the selected image is not based on shooting advice, the process proceeds to step S17, where normal image reproduction is performed.

  If all the selected images have been reproduced, the process proceeds to step S18 to determine whether or not to end the reproduction mode. As a result, in the case of end of playback, the playback mode ends and this sequence ends. On the other hand, if it is not the end of reproduction, the process proceeds to step S13, and image selection is performed again.

  Next, the shooting theme selection operation will be described.

  If it is determined in step S11 that the mode is not the playback mode, the process proceeds to step S19 to determine whether the mode is the theme designation mode. If the theme is not designated, the process proceeds to step S1 described above, and if the theme is designated, the process proceeds to step S20. In step S20, selectable themes are displayed on the candidate display unit 20a, and the user is prompted to input. Here, when the user selects a theme, the theme designated by the user is recorded in the digital camera 10 in the subsequent step S21. Thereafter, this sequence ends.

  7 is a flowchart for explaining the detailed operation of the subroutine “advice image determination processing (one-day trip) in step S6 of the flowchart of FIG. 6. Here, the photograph taken is a photograph according to the photographing advice. It is determined whether or not.

  Here, as shown in FIG. 4C, a case where “day trip” is selected as the theme will be described. In this case, there is a phrase “with the whole family” as shooting advice. Therefore, first, in step S31, whether or not a subject in the captured image has a human face is detected by the feature detection unit 15 using face recognition technology, and a determination is made. As a result, if a face exists in the subject, the process proceeds to step S32, and the number of smiles facing the front is detected by using a smile detection technique.

  Next, in step S33, it is determined whether or not the number of detected smiles is the largest among the photographs that have been photographed so far by the photographing advice of “all the family”. Here, if there are the largest number of photographed photographs, the process proceeds to step S34, and this image is recorded as an advice image “with the whole family”. Then, in step S35, what was previously recorded as the advice image of “all the family” at this time is excluded from the candidates. Thereafter, the subroutine is exited and the process proceeds to step S7 in the flowchart of FIG.

  On the other hand, if a human face cannot be detected in the captured image in step S31, the process proceeds to step S36. Here, in order to determine whether or not the shooting advice “beautiful flower” has been shot, it is determined whether or not a flower pattern exists in the image. A flower pattern uses a feature extraction technique (processing such as edge, contour, line component extraction, region segmentation, texture extraction, etc.) in an image, and is extracted and detected by the feature detection unit 15. Is called.

  If a flower pattern is extracted in step S36, then in step S37, it is compared with a photograph previously recorded as a “beautiful flower” advice image based on the characteristics of the extracted image. It is determined whether the image is larger and more vivid. As a result of this determination, if the image is large and vivid, the process proceeds to step S38, and the photograph is recorded as a “beautiful flower” advice image. In step S39, the previously recorded “beautiful flower” advice image is excluded from the candidates. Thereafter, the subroutine is exited and the process proceeds to step S7 in the flowchart of FIG.

  On the other hand, if it is determined in step S37 that the extracted image is not large and vivid, steps S38 and S29 are skipped, and the process proceeds to step S7 in the flowchart of FIG.

  If no flower pattern is detected in the photographed image in step S36, the process proceeds to step S40. In this step S40, it is determined whether or not the shooting advice “representative landscape” has been shot. Therefore, it is determined whether or not the compressed data size of the photographed image is larger than a photograph recorded as an advice image of “representative landscape” so far. Since landscape photographs have complicated color information, when image compression such as JPEG is performed, since the compression rate is low, the compressed image data size becomes large.

  If the compressed data is large in step S40, the process proceeds to step S41, where an advice image “representative landscape” is recorded. At this time, in the subsequent step S42, what was previously recorded as the “representative landscape” advice image is excluded from the candidates. Thereafter, the subroutine is exited and the process proceeds to step S7 in the flowchart of FIG.

  On the other hand, if it is not determined in step S40 that the amount of compressed data is large, steps S41 and S42 are skipped, and the process proceeds to step S7 in the flowchart of FIG.

  Next, with reference to the flowchart of FIG. 8, the detailed operation of the subroutine “Combine comments from advice and data at the time of shooting” in step S15 of the flowchart of FIG. 6 will be described. Here, an operation for synthesizing a phrase from shooting advice and shooting data will be described.

  When this subroutine is entered, first, in step S51, it is determined whether or not there is location data that can be acquired from the position information acquired at the time of shooting. The place data means a keyword in which position information (GPS data) is converted into a place name, topography, and keywords (mountains, rivers, seas) that mean topography. If the location data exists, the process proceeds to step S52, and a comment for modifying the location is extracted. Here, the extracted comments are, for example, “mountain”, “river”, “sea”, and the like. On the other hand, if no location data exists in step S51, step S52 is skipped.

  Next, in step S53, it is determined whether date / time data (time information) exists. If there is no date / time data, the process proceeds to step S54. On the other hand, if there is date and time data, the process proceeds to step S57, and the season is determined from the month information. If the season is determined in step S57, the process proceeds to step S58 to extract a comment that modifies the season. Comments that modify the season include, for example, “early spring”, “spring equinox”, “spring”, “early summer”, “summer solstice”, “summer”, “residual heat”, “early autumn”, “autumn”, “autumn”, “ “Winter Solstice”, “Early Winter”, “Winter” and so on. If the season is not determined in step S57, step S58 is skipped.

  In step S59, holiday determination is performed. Here, if it is a holiday, it is presumed that the user is taking an action different from that of daily life. Therefore, the process proceeds to step S60, and a comment that modifies the holiday is extracted. If it is a holiday, the type of holiday is extracted. Here, the extracted comments are “Saturday”, “Sunday”, “New Year”, “Spring Day”, “Green Day”, “National Holiday”, “Sea Day”, “End of the War”, “ Equinox Day, Respect for the Aged Day, Work Thanks Day, Emperor's Birthday, etc. After the holiday decoration comment is extracted, the process proceeds to step S54.

  In step S54, color determination, which is one of feature data extracted from the photographed photograph, is performed. In this case, image features are extracted from the photographed image (processing such as edge, contour, line component extraction, region division extraction, texture extraction, etc.). If a symbolic color is extracted in the image, the process moves to step S55 to extract a comment that modifies the color. The comments to be modified are “red”, “blue”, “yellow”, “green”, “white”, “black”, and the like. On the other hand, if it is not possible to determine each condition from each photographing data in step S54, the process proceeds to step S56, and no modification comment is extracted. Thereafter, the subroutine is exited, and the process proceeds to step S16 in the flowchart of FIG.

  FIG. 9 shows a diagram in which the position is associated on the map 30 based on the position information acquired by the position detection unit 11 of the digital camera 10.

  Even if position information is acquired by the position detection unit 11 at a place where shooting is performed, a keyword indicating the place, such as a place name or topography on the map 30, may not be detected. In this case, by acquiring the position information of the home 31 in advance and registering it in the digital camera 10, for example, when the shooting location 32 is away from the home, a modification comment such as “I came all the way” is given. Is done.

  As described above, according to the digital camera of the present embodiment, a comment that modifies the keyword is synthesized and displayed on the image from the keyword that advises shooting and the content of the image obtained as a result of shooting according to the keyword. can do.

  Although the embodiments of the present invention have been described above, the present invention can be variously modified without departing from the gist of the present invention other than the above-described embodiments.

  Further, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention Can be extracted as an invention.

1 is a block diagram illustrating a configuration of a digital camera 10 to which an imaging system is applied according to an embodiment of the present invention. It is a figure for demonstrating the method of associating a phrase with the image | photographed photograph. It is the figure which showed the theme for prompting a user to select. It is a figure for demonstrating a photography theme and advice. It is a figure for demonstrating the wording of the candidate of a synthetic phrase, display advice, and the correlation of the image | photographed photograph group. 4 is a flowchart for explaining a control operation of the digital camera 10 according to the embodiment of the present invention. FIG. 7 is a flowchart for explaining a detailed operation of a subroutine “advice image determination process (day trip) in step S6 of the flowchart of FIG. 6; FIG. 7 is a flowchart for explaining detailed operations of a subroutine “Combine a comment from advice and photographing data” in step S15 of the flowchart of FIG. 6; It is the figure which showed the example which linked | related the position on the map 30 based on the positional information acquired in the position detection part 11 of the digital camera 10. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 11 ... Position detection part, 12 ... Time detection part, 13 ... Communication part, 14 ... Imaging | photography part, 15 ... Feature detection part, 16 ... Database (DB) part, 18 ... Scene extraction part, 20 ... Display Part, 20a ... candidate display part, 21 ... phrase synthesizing part, 22 ... control part.

Claims (9)

A shooting section for shooting the subject;
Theme extracting means for extracting and outputting information on a candidate for a subject to be photographed next for the photographer;
Based on the information output from the theme extracting means, a notifying means for notifying the photographer of information on candidate subjects to be photographed next as advice,
A feature detecting means for detecting a feature of an image photographed based on the advice by the photographing unit;
The theme extraction means to create a phrase by combining the words that have been extracted based on the wording of the candidates related phrase and the output to an output of said characteristic detection means, the created phrase, the photographed image Together with phrase synthesizing means for notifying the photographer,
A camera comprising: A position acquisition unit that acquires position information at the time of shooting; and a time detection unit that acquires time information at the time of shooting,
The camera according to claim 1, wherein the theme extraction unit extracts information related to a candidate for a subject to be photographed next based on at least one of position information and time information at the time of photographing. A communication unit is further provided for acquiring specific information from an external device based on at least one of the position information at the time of shooting acquired by the position acquisition unit and the time information at the time of shooting acquired by the time detection unit. The camera according to claim 2. The phrase synthesizing unit, when the position information is acquired by the position acquisition unit, synthesizes a phrase extracted based on the position information and a phrase of the phrase candidate to create the phrase The camera according to claim 2, wherein When the time information is acquired by the time detection unit, the phrase synthesizing unit generates the phrase by synthesizing a phrase extracted based on the time information with a phrase of the phrase candidate. The camera according to claim 2, wherein When the color information is acquired by the feature detection unit, the phrase synthesizing unit synthesizes the phrase extracted based on the color information and the phrase candidate word to create the phrase The camera according to claim 2. The camera according to claim 2, further comprising a recording unit that records the phrase and the image when the photographed image matches the phrase created by the phrase synthesizing unit. The camera according to claim 1, further comprising display means for displaying the phrase. The camera according to claim 8, wherein the display unit displays the phrase together with the photographed image.

Images