JP2014175733A - Image reproduction apparatus and image reproduction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To retrieve an image with predetermined atmosphere, impression, emotion, etc. to a predetermined extent.SOLUTION: An image reproduction apparatus comprises: storage means which associates and stores plural pieces of color information and plural emotional features respectively expressing the atmospheres of images with respective pieces of the color information; selection means which selects at least one of the emotional features and the extent of the emotional feature; detection means which detects color information in plural images; level determination means which determines, on the basis of the detection result of the detection means, the extent of color information corresponding to the emotional feature selected by the selection means included in the respective images; and display means which on the basis of the determination result of the level determination means, displays an image corresponding to the extent of the emotional feature selected by the selection means, among the plural images.

Description

  The present invention relates to an image reproduction device and an image reproduction program.

  A technique is known in which a feature amount of emotional features is assigned to an image, and an unorganized image data set is grouped based on the feature amount (Patent Document 1).

JP 2011-154687 A

  A technique for searching for an image having a predetermined atmosphere, impression, emotion or the like is desired. With the technique described in Patent Document 1, an image having a predetermined emotional feature can be searched, but the degree of the atmosphere, impression, emotion, etc. expressed in the image cannot be known.

  The image reproducing device according to the first aspect of the present invention includes a storage unit that stores a plurality of color information in association with a plurality of emotional features each representing an atmosphere of an image having each color information, and a plurality of emotional features Selection means for selecting at least one of the emotional characteristics, detection means for detecting color information in a plurality of images, and a plurality of color information corresponding to the emotional characteristics selected by the selection means. Level determination means for determining the degree of inclusion in each image based on the detection result of the detection means, and the degree of the emotional feature selected by the selection means from among the plurality of images based on the determination result of the level determination means Display means for displaying an image corresponding to.

An image reproduction program according to a second aspect of the present invention includes a storage unit that stores a plurality of color information and a plurality of emotional features that respectively represent atmospheres of an image having each color information, and stores a plurality of emotional features. An image reproduction program applied to an image reproduction apparatus comprising: selection means for selecting at least one and a degree of the emotional feature; and display means for displaying an image,
The color information included in each of the plurality of images is detected, and the degree to which the color information corresponding to the emotional feature selected by the selection unit is included in each of the plurality of images is determined based on the detection result. Based on the above, the image reproducing apparatus is operated so that an image corresponding to the degree of the emotional feature selected by the selection means among the plurality of images is displayed on the display means.

  By using the image reproduction apparatus and the image reproduction program according to the present invention, the user can search for an image having a predetermined degree of a predetermined atmosphere, impression, emotion, and the like.

It is a block diagram which shows the structural example of the image reproduction apparatus by one embodiment of this invention. It is a figure which shows the format example of the data contained in an atmosphere database. It is a functional block diagram of the control part of the imaging device by one embodiment of the present invention. It is a figure which shows an example of the metadata contained in image data. It is a figure which shows an example of an image selection screen. It is a flowchart which shows the operation | movement for displaying an image selection screen. It is a flowchart regarding the process of a fitness determination part. It is a figure which shows an example of the screen which a display control part displays.

  FIG. 1 is a block diagram showing a configuration example of an image reproduction apparatus according to an embodiment of the present invention. FIG. 1 illustrates a digital camera 100 as an example of an image reproducing device. The digital camera 100 includes a photographing lens 10, an imaging unit 11, a control unit 12, a storage unit 13, a display unit 14 including a display monitor, an operation member 15, an interface unit 16, and a GPS module 18. Is provided.

  The digital camera 100 has a plurality of scene modes such as portrait, landscape, and sunset. The portrait mode is a scene mode suitable for photographing a person. The landscape mode is a scene mode suitable for landscape photography. The sunset mode is a scene mode suitable for photographing a subject that is reddish due to sunset or sunrise.

  The taking lens 10 forms a subject image on the image pickup surface of the image pickup device 11a. The imaging unit 11 captures a subject image and outputs an RGB color system digital image signal to the control unit 12. In FIG. 1, the photographing lens 10 is illustrated as a single lens for the sake of simplicity, but actually includes a plurality of optical systems such as a focusing lens and a zoom lens (not illustrated).

  The control unit 12 includes, for example, a CPU and an image processing unit, and controls each unit of the digital camera 100 by executing a program stored in the storage unit 13. Details of each process executed by the control unit 12 will be described later.

  The storage unit 13 includes a volatile recording medium such as a buffer memory 13a and a non-volatile recording medium such as a flash memory 13b and a ROM (not shown). The buffer memory 13a temporarily stores the digital image signal output from the imaging unit 11 and various image data being processed by the control unit 12. The flash memory 13b stores an atmosphere database 130 whose details will be described later. A program executed by the control unit 12 and map data are stored in a nonvolatile recording medium such as a flash memory 13b or a ROM (not shown).

  The display unit 14 includes a display monitor such as an LCD, and is provided on the back surface of the digital camera 100. The display unit 14 displays a menu screen and an image reproduction screen based on the image data input under the control of the control unit 12.

  The operation member 15 includes, for example, a release switch 15a for release operation including a half-press switch that is turned on in response to a half-press operation of a release button and a full-press switch that is turned on in response to a full-press operation, and a cross switch used for various selection operations. An input device such as 15b. The cross switch 15b is an aggregate of a plurality of switches, and includes, for example, a switch that controls the vertical and horizontal directions, and a determination switch. The operation member 15 transmits an operation signal from each input device to the control unit 12.

The interface unit 16 is an interface for recording image data on an external recording medium such as a memory card 17 that is detachably attached to the digital camera 100 and reading the image data from the memory card 17 or the like.
The GPS module 18 detects a location where the digital camera 100 exists based on a signal transmitted from a GPS satellite, and outputs information regarding the location to the control unit 12.

  FIG. 2 is an example of data (data table) stored in the atmosphere database 130. As shown in FIG. 2, the atmosphere database 130 includes n pieces of atmosphere data including color information including ranges of hue, saturation, and brightness in image data, a scene mode, and an atmosphere keyword. D1 to Dn are included. Each of the atmosphere data D1 to Dn is data that defines the relationship between the color and the scene mode, the atmosphere that the color brings, the impression that the color gives to the person, or the emotion expressed or associated with the color. Hereinafter, the atmosphere created by the color, the impression given to the person who sees the color, and the emotion expressed or associated with the color are collectively referred to as the atmosphere.

  In the atmosphere data D1 to Dn, a color range that brings about the atmosphere is defined in the HSV color space. That is, in the atmosphere data D1 to Dn, the color range that brings about the atmosphere is defined by the hue (H), saturation (S), and lightness (V) ranges. Each scene mode constituting the atmosphere data D1 to Dn is a scene mode that can be used by the digital camera 100, that is, portrait, landscape, sunset, or the like.

The atmosphere keyword is character string type data expressing the atmosphere and emotional characteristics. In FIG. 2, adjective verbs such as “exhilarating”, “refreshing”, and “clear” are exemplified as atmosphere keywords.
As is clear from D1 and D2 in FIG. 2, the atmosphere keywords applied according to the scene mode are set in advance on the data table even in the same hue, saturation and lightness range colors. (In the landscape mode, it is “exhilarating”, but in the portrait mode, “fresh” is preset in the data table).
As is clear from D1 and D3 in FIG. 2, even in the same scene mode (landscape mode), the colors (hue (H), saturation (S), and lightness (V)) defined by the atmosphere are defined. If the color is different, the applied atmosphere keyword is set differently on the data table ("exhilarating" and "clear" are preset).

Although not shown, in addition to the above, the atmosphere database 130 also includes the following data as relation data between the color brought about by the atmosphere, the scene mode, and the atmosphere keyword. For example, “here is good” is set in the atmosphere database 130 as the atmosphere keyword when the color is green and the scene mode is the landscape mode. Also, “beautiful” is set in the atmosphere database 130 as the atmosphere keyword when the color is marine blue and the scene mode is the landscape mode.

  Note that the atmosphere keyword is not limited to adjective verbs. For example, nouns such as “Exhilarating”, adjectives such as “Kokochiyo”, and adjective verbs with adverbs such as “Very Exhilarating” and “A little good here”. Or an adjective may be used. Note that the atmosphere keyword may be determined for the atmosphere keyword with a word indicating the “degree” of the situation indicated by the atmosphere keyword. For example, when an atmosphere keyword with an adverb is used, the presence or absence of an adverb and the adverb to be added may be determined based on the color saturation that brings the atmosphere. As an example, words indicating the degree (degree) of exhilaration (such as “very” or “little”), such as “very exhilarating”, “exhilarating”, “a little exhilarating” in order from the atmosphere data with high saturation ) With the atmosphere keyword. In FIG. 2, the atmosphere keyword of the atmosphere data D1 is set to “exhilarating” (expression indicating normal exhilaration), and the atmosphere keyword of the atmosphere data D2 is “refreshing” (expression indicating normal refreshing) Is set to

  FIG. 3 is a functional block diagram of the control unit 12. The control unit 12 executes the program stored in the storage unit 13 to thereby execute the image processing unit 120, the scene determination unit 121, the representative color determination unit 122, the atmosphere determination unit 123, and the pixel detection unit 124. , Functions as a level determination unit 125 and a display control unit 126.

  The image processing unit 120 performs various image processing on the digital image signal. For example, the image processing unit 120 performs white balance processing, gamma correction processing, color interpolation processing, and the like on the digital image signal input from the imaging unit 11 to generate image data. The image data generated by the image processing unit 120 is data in an image format to which various metadata such as the EXIF format can be added. Further, the image processing unit 120 performs color space conversion processing, and converts the color space of the image signal from the RGB color space to the HSV color space.

  Based on the digital image signal output from the image capturing unit 11 (image data output from the image processing unit 120), the image data recorded on the memory card 17, or the like, the scene determination unit 121 corresponds to the scene mode corresponding to those images. Is automatically determined (judged).

  Based on the scene mode determined by the scene determination unit 121, the representative color determination unit 122 represents the subject image included in the image signal from the image signals converted into the HSV color space by the image processing unit 120. Determine the representative color. For example, the representative color determining unit 122 determines the color that occupies the widest area (number of pixels) as the representative color in the subject image corresponding to the scene mode included in the input image. The representative color determination unit 122 outputs color information of the determined representative color, that is, information on the hue, saturation, and brightness of the representative color.

  The atmosphere determination unit 123 searches the atmosphere database 130 based on the scene mode determined by the scene determination unit 121 and the information on the hue, saturation, and brightness of the representative color output by the representative color determination unit 122. Thus, the atmosphere keyword corresponding to the image is determined.

  The pixel detection unit 124 obtains predetermined color information corresponding to a predetermined atmosphere from each of the image data stored in the memory card 17 (or from each of the image data output from the image processing unit 120). A pixel having the same is detected. Specifically, the pixel detection unit 124 sets the hue, saturation, and brightness within the hue, saturation, and brightness ranges that are associated with predetermined atmosphere keywords (atmosphere and emotional characteristics) in the atmosphere database 130. A certain pixel is detected. For example, the pixel detection unit 124 detects pixels whose hue is in the range of H1 to H2, saturation is in the range of S1 to S2, and brightness is in the range of V1 to V2.

  The level determination unit 125 determines, based on the number of pixels detected from the image data by the pixel detection unit 124, the degree to which color information corresponding to a predetermined atmosphere keyword (atmosphere, emotional feature) is included in the image data. To do. In other words, the level determination unit 125 determines the degree of a predetermined atmosphere that the image has. The level determination unit 125 represents the degree to which color information corresponding to a predetermined atmosphere keyword (atmosphere, emotional feature) is included in the image data, for example, at 0 to 10 levels.

  The display control unit 126 generates an image selection screen, which will be described later, and causes the display unit 14 to display the image selection screen. The user uses the image selection screen displayed by the display control unit 126 to search for an image corresponding to a desired atmosphere (emotional feature) level.

  FIG. 4 is a diagram illustrating an example of a data format of image data recorded on the memory card 17 by the digital camera 100. The image data 500 illustrated in FIG. 4 includes a metadata data area 501, and includes data 511 regarding the atmosphere (emotional characteristics), data 512 regarding the shooting date and time, and data 513 regarding the shooting location as metadata. It is. The data 511 regarding the atmosphere is information regarding the atmosphere keyword determined by the atmosphere determination unit 123. Data 513 regarding the shooting location is acquired from the GPS module 18.

  FIG. 5 is a diagram illustrating a display example of the image selection screen. The image selection screen 20 illustrated in FIG. 5 is a screen for searching for a desired image from the image data recorded on the memory card 17, and includes a graph display area 21, an image display area 22, and a map display area 23. And have.

  In the graph display area 21, a line graph in which information regarding each piece of image data recorded on the memory card 17 is plotted is displayed. In the line graph displayed in the graph display area 21, the horizontal axis represents the image recording date and time, and the vertical axis represents the result of the level determination unit 125 determining the atmosphere “exhilarating”. The atmosphere indicating the level on the vertical axis of the line graph can be selected by the user using the operation member 15.

  In the graph display area 21, each image data recorded on the memory card 17 is represented by a black circle or a circled numerical mark. Regarding the atmosphere represented by the vertical axis of the graph, image data having a predetermined level is represented by a circled number mark, and other than that, it is represented by a black circle. The numbers indicated by the circled number marks are different from each other so that each image data can be identified.

  Two reference lines 25 are displayed in the graph display area 21. The reference line 25 represents the level (degree) desired by the user for the atmosphere represented by the vertical axis of the graph. When searching for an image, the user sets at least one reference line 25 using the operation member 15. The user can add a new reference line 25 to the graph display area 21, delete the reference line 25 from the graph display area 21, or move the reference line 25 up and down by various operations using the operation member 15. You can make it.

  In the image display area 22, thumbnail images are displayed with the respective circled marks for the image data displayed in the graph display area 21 with circled numerical marks. The image display area 22 is divided for each level indicated by the reference line 25. In the upper area of the image display area 22, thumbnail images related to 7-level image data are displayed for the atmosphere “exhilarating”. Similarly, in the area below the image display area 22, thumbnail images relating to 10-level image data are displayed for the atmosphere “exhilarating”. When the user changes the atmosphere indicating the level on the vertical axis of the line graph displayed in the graph display area 21, or when the user moves the reference line 25 up and down, the display of the image display area 22 also changes. It changes according to.

  In the map display area 23, information regarding the shooting location of the image data is displayed on the map for the image data displayed by the circled numerical marks in the graph display area 21. The map is displayed based on the map data stored in the flash memory 13b or the like.

FIG. 6 is a flowchart regarding the operation for displaying the image selection screen 20.
In step S <b> 200, the control unit 12 reads image data from the memory card 17 via the interface unit 16.
In step S <b> 201, the control unit 12 causes the user to select an atmosphere indicating a level on the vertical axis of the line graph in the graph display area 21. The user selects a desired atmosphere by a predetermined operation using the operation member 15.
In step S202, the control unit 12 causes the user to select a level for displaying the reference line 25. The user selects a desired level by a predetermined operation using the operation member 15.

In step S203, the control unit 12 uses the level determination unit 125 to determine the level for the atmosphere selected in step S201 for each image data read in step S200.
In step S204, based on the level determined in step S203, the control unit 12 extracts image data corresponding to the level selected in step S202 from the image data read in step S200.

  In step S <b> 205, the control unit 12 generates the image selection screen 20 using the display control unit 126 and displays it on the display unit 14. For example, a line graph to be displayed in the graph display area 21 is generated based on the data 512 related to the shooting date and time included in the metadata of the image data read in step S200 and the information related to the level determined in step S203, and in step S204. A thumbnail image to be displayed in the image display area 22 is generated based on the extracted image data, and the map display area 23 is generated based on the data 513 relating to the imaging location and the map data included in the metadata of the image data extracted in step S204. Generate an image to display.

  By viewing the image selection screen 20 displayed in this manner, the user can confirm the search result of image data having a desired atmosphere (emotional feature) in a desired degree. The user can select one of the image data obtained as a result of the search and reproduce the image.

  FIG. 7 is a flowchart relating to the processing of the level determination unit 125 used in the processing of step S203 of FIG. In step S301, the control unit 12 searches the atmosphere database 130 for atmosphere data using the atmosphere selected in step S201 of FIG. 6 as the atmosphere keyword, and the hue range, saturation range, Get information about the brightness range.

  In step S302, the control unit 12 uses the pixel detection unit 124 to select the hue range, saturation range, and the like acquired in step S301 from each of the image data read in step S200 of FIG. Pixels having color information included in the brightness range are detected.

  In step S303, the control unit 12 calculates the number of pixels detected in step S302 for each image data, and determines the level based on the number of pixels. For example, the control unit 12 calculates a ratio m = m1 / m2 between the calculated number of pixels m1 and the total number of pixels m2 of the image data, and when the ratio m is 0, 0 level, 0 <m ≦ 0. It may be determined such that 1 level is 1 level, 0.1 <m ≦ 0.2 is 2 level,..., 0.9 <m ≦ 1 is 10 level.

According to the embodiment described above, the following operational effects can be obtained.
The digital camera 100 includes a flash memory 13b and a control unit 12. The flash memory 13b stores an atmosphere database 130 in which a plurality of pieces of color information and a plurality of emotional features (atmosphere keywords) each representing the atmosphere of an image having each piece of color information are associated with each other. The control unit 12 accepts a manual selection operation using the operation member 15 and selects at least one of the plurality of emotional features and the degree of the emotional feature (step S201 and step S202 in FIG. 5). Further, the control unit 12 detects color information in each of the plurality of images read from the memory card 17 (pixel detection unit 124, step S302 in FIG. 6), and corresponds to the emotional feature selected in step S201. The degree (number of pixels) that the color information (hue, saturation, brightness) is included in each of the plurality of images is determined (step S303 in FIG. 6). Based on the determination result of the level determination unit 125, the control unit 12 displays an image corresponding to the degree of the emotional feature selected in step S <b> 202 in the image display area 22 among the plurality of image data. By having such a configuration, the digital camera 100 can search for an image having a predetermined atmosphere, impression, emotion, and the like. In the digital camera 100, a plurality of reference lines 25 can be set and images of a plurality of types of fitness can be displayed in the image display area 22. The user can learn the composition for photographing an image having a predetermined atmosphere by comparing and examining these images of a plurality of types.

The embodiment described above can be implemented with the following modifications.
[Modification 1] In the above embodiment, the digital camera 100 has been described as an example of the image reproducing device, but the scope of application of the present invention is not limited to the digital camera 100 alone. A display monitor, a function for reproducing and displaying an image on the display monitor, a storage unit for storing a program for executing the above-described operation (the operation of the flowcharts shown in FIGS. 6 and 7), and a control unit for controlling operation according to the program The present invention can be applied to any electronic device having the following. For example, the present invention can also be applied to a personal computer in which the above-described image reproduction program (FIGS. 6 and 7) is installed.

[Modification 2] The image selection screen 20 generated by the display control unit 126 is not limited to the screen of the format illustrated in FIG. For example, the image selection screen 20 may be changed as follows.
(1) The map display area 23 may not be displayed.
(2) The graph display area 21, the image display area 22, and the map display area 23 may not be displayed on the display unit 14 at the same time. For example, the graph display area 21, the image display area 22, and the map display area 23 may be sequentially switched using a predetermined operation member 15. When the graph display area 21 or the map display area 23 is displayed on the display unit 14, a thumbnail image in which the level determined in step S203 matches the level selected in step S202 may be further displayed. .
(3) A plurality of line graphs related to a plurality of atmospheres may be displayed in the graph display area 21. At this time, a plurality of line graphs may be superimposed and displayed, or the reference line 25 may be shared by the line graphs.
(4) A graph other than the line graph may be displayed in the graph display area 21. For example, a bar graph in which the horizontal axis is information for identifying an image and the vertical axis is a level determined by the level determination unit 125 for a predetermined atmosphere may be displayed.

[Modification 3] The image displayed on the display unit 14 by the display control unit 126 is not limited to the image selection screen 20 alone. For example, an atmosphere distribution screen 30 as shown in FIG. 8 may be displayed. The atmosphere distribution screen 30 includes a radar chart 31 representing the number of image data of each atmosphere stored in the memory card 17. Around the radar chart 31, an image having the highest degree of suitability of each atmosphere is displayed.

[Modification 4] The control unit 12 reads out the atmosphere (emotional feature) opposite to the atmosphere (emotional feature) selected in step S201 using the level determination unit 125 in step S200. The level may be determined for each image data. Then, the control unit 12 may perform correction to reduce the level for the opposite atmosphere (emotional feature) with respect to the level determined in step S203 for each image data. Opposite atmospheres are, for example, “depressed” for “exhilarating”, “dirty” for “clean”, “stressing” for “fun”, It's dark. When the portrait mode is selected by the scene determination unit 121, the control unit 12 determines that the atmosphere (emotional feature) selected in step S201 and the facial expression of the person determined by face recognition (expression recognition). The level correction may be performed based on the above. For example, when “Refreshing” is selected as the atmosphere (emotional feature), the level is increased if the person ’s facial expression is smiling, and the level is decreased if the person ’s eyes are closed. Also good.

[Modification 5] In the above-described embodiment, as described with reference to FIGS. 6 and 7, the level determination unit 125 determines the level of each image during image reproduction. However, during the imaging operation, the level determination unit 125 described above performs the bell determination operation (the operation described above with reference to FIG. 7) on the image temporarily stored in the buffer memory 13a (before recording the image on the memory card 17). You may make it do. In this case, if the level determination result is recorded in the memory card 17 as metadata accompanying the captured image (level determined) with the image, a search process based on the metadata at the time of image reproduction is possible. By performing the above, it is possible to perform a search process similar to that described in the above embodiment.

  The embodiments and modifications described above are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Further, the embodiments and modifications described above may be combined and executed as long as the features of the invention are not impaired.

12 control unit 13 storage unit 14 display unit 15 operation member 16 interface unit 17 memory card 20 image selection screen 21 graph display region 22 image display region 23 map display region 25 reference line 30 atmosphere distribution screen 100 digital camera 120 image processing unit 121 scene Determination unit 122 Representative color determination unit 123 Atmosphere determination unit 124 Pixel detection unit 125 Level determination unit 126 Display control unit 130 Atmosphere database

Claims (4)

Storage means for storing a plurality of pieces of color information in association with a plurality of emotional features respectively representing the atmosphere of the image having each piece of color information;
A selection means for selecting at least one of the plurality of emotional characteristics and the degree of the emotional characteristics;
Detecting means for detecting color information included in each of the plurality of images;
Level determination means for determining the degree to which color information corresponding to the emotional feature selected by the selection means is included in each of the plurality of images based on a detection result of the detection means;
Display means for displaying an image corresponding to the degree of the emotional feature selected by the selection means among the plurality of images based on the determination result of the level determination means;
An image reproducing apparatus comprising: The image reproduction apparatus according to claim 1,
The display means further has a numerical axis related to the degree of the emotional feature selected by the selection means, and further displays a graph on which the result determined by the level determination means is displayed for each of the plurality of images. An image reproducing apparatus characterized by: The image reproduction apparatus according to claim 1 or 2,
Representative color determining means for determining color information representing each of the plurality of images from each of the plurality of images;
Atmosphere determination for determining an emotional feature associated with color information determined by the representative color determination unit from each of the plurality of images from the plurality of emotional features stored in the storage unit Means,
Further comprising
The image reproducing apparatus, wherein the display means further displays the number of images whose emotional characteristics determined by the atmosphere determination means are predetermined emotional characteristics among the plurality of images. A storage means for storing a plurality of color information and a plurality of emotional features each representing an atmosphere of an image having each color information, and selecting at least one of the plurality of emotional features and the degree of the emotional feature An image reproduction program applied to an image reproduction apparatus comprising a selection means for displaying and a display means for displaying an image,
Detect color information contained in each of multiple images,
The degree of color information corresponding to the emotional feature selected by the selection means is included in each of the plurality of images based on the detection result,
Based on the result of the determination, the image reproduction device is operated so that an image corresponding to the degree of the emotional feature selected by the selection unit among the plurality of images is displayed on the display unit. An image reproduction program characterized by the above.

Images