JP4743097B2 - Camera and image search program - Google Patents

Description

  The present invention relates to a camera and an image search program.

2. Description of the Related Art Conventionally, it is known to automatically determine camera shooting parameters based on the result of image recognition of a shot image, whether the subject is a moving object, the shooting magnification, the subject distance, or the like (for example, Patent Document 1).
JP 2003-344891 A

However, the above-described conventional technique has a problem in that the processing load required for image recognition must be increased.
Therefore, an object of the present invention is to provide a technique for appropriately determining shooting parameters with a simple configuration.

<< 1 >> The camera of the present invention includes a setting unit, a sensor unit, a storage unit, a search unit, a parameter determination unit, and a photographing unit.
The setting unit sets camera functions.
The sensor unit detects the state of the subject or the operating state of the camera.
The storage unit classifies and stores combinations of information corresponding to the function set by the setting unit and the detection result of the sensor unit for each type of subject.
The search unit searches the storage unit for a combination corresponding to the function set by the setting unit and the detection result of the sensor unit.
The parameter determination unit determines a shooting parameter of the camera based on the combination searched by the search unit.
The photographing unit performs photographing according to the photographing parameter determined by the parameter determining unit.
<< 2 >> Preferably, when there are a plurality of corresponding combinations, the parameter determination unit creates the second imaging parameter based on the imaging parameters corresponding to each of the plurality of combinations.
<< 3 >> Preferably, a recording processing unit that records attribute information corresponding to a corresponding combination in association with a photographed image photographed by the photographing unit is provided.
<< 4 >> Preferably, an image search unit for searching for a captured image recorded by the recording processing unit based on the attribute information is provided.
<< 5 >> The image search program of the present invention is a program for causing a computer to function as an image acquisition unit, an input unit, and an image search unit.
The image acquisition unit captures the captured image and attribute information recorded in the camera described in << 3 >> or << 4 >>.
The input unit accepts input of attribute information.
The image search unit searches for a captured image corresponding to the input attribute information.

In the present invention, the shooting parameter is determined based on the function set in the camera and the detection result of the state of the subject or the operating state of the camera. In general, the setting state of the camera function reflects the user's intention to shoot the subject type. On the other hand, objective information regarding the subject type is obtained from the sensor information. For this reason, it is possible to appropriately determine the shooting parameters without requiring a load-intensive process such as image recognition.
As a result, it is possible to determine various shooting scenes and determine shooting parameters.

<< Configuration of Embodiment >>
FIG. 1 is a block diagram showing the configuration of the present embodiment.
In FIG. 1, the camera 11 includes a setting unit 12 that sets a function in accordance with a user operation input. For example, the setting unit 12 can set the following functions.
(1) AF mode: It can be selected from AF-S mode in which focus control is prohibited (focus lock) once focused, AF-C mode in which focus control is continued after focusing, and the like.
(2) Exposure mode: Selectable from program exposure mode, aperture priority exposure mode, shutter speed priority mode, etc.
(3) Metering mode: Selectable from split metering mode, center-weighted metering mode, spot metering mode, etc.
(4) Drive mode: Selectable from single frame shooting mode, continuous shooting mode, etc.

Furthermore, the camera 11 includes a sensor unit 14 that detects the state of the subject and the operation state of the camera 11. For example, the sensor unit 14 includes the following sensor group.
(a) Photometric sensor 14a: Sensor for subjecting a subject to divided photometry / colorimetry
(b) Temperature sensor 14b: Sensor for detecting environmental temperature
(c) Acceleration sensor 14c... Acceleration acting on the camera 11 is detected. Also, the direction and camera posture are detected.
(d) Distance measuring sensor 14d: detects the distance to the subject or the defocus amount.
(e) Distance encoder 14e: The distance to the in-focus position is detected based on the rotation angle of the focus ring of the photographing lens and the lens position of the focus adjustment lens.
(f) Zoom encoder 14f... The focal length of the photographic lens is detected based on the rotation angle of the zoom ring of the photographic lens and the lens position of the zoom adjustment lens.
(g) Timer 14g... Date / time, morning / day / night difference, season difference, etc. are detected by date / time measurement.

  The camera 11 also has a shooting condition pattern file that is a combination of features corresponding to a shooting scene and information detected by a sensor, and control variables for controlling each control unit such as exposure and autofocus. A storage unit 15 in which pattern files are registered in advance is provided. In addition, the camera 11 is also provided with a microprocessor 16, a photographing unit 17, a recording processing unit 18, and an input unit 19. The microprocessor 16 includes functions of a search unit 20, a parameter determination unit 21, and an image search unit 22.

<Creation procedure of shooting condition pattern file>
The photographing condition pattern file stored in the storage unit 15 is created in advance by a camera designer or a camera user. Hereinafter, this creation procedure will be described.

(Procedure 1) First, information regarding functions set in the setting unit 12 for various shooting scenes and output information obtained from the sensor unit 14 are acquired. These types of information are as shown in Table 1.

(Procedure 2) Among these pieces of information, the function symbolizing the shooting scene and the output information of the sensor are extracted as feature quantities and stored for each subject type indicating each shooting scene.

(Procedure 3) An imaging condition pattern file is created by extracting highly similar items and condition ranges from information groups obtained from the same type of subjects and their feature quantities.

(Procedure 4) The obtained shooting condition pattern file is registered in the storage unit 15 in association with each subject type (shooting scene).

《Procedure for creating control variable pattern file》
On the other hand, the control variable pattern file stored in the storage unit 15 is also created in advance by the camera designer or the camera user. Hereinafter, this creation procedure will be described.

(Procedure 1) First, the camera 11 shoots while changing individual shooting parameters for various shooting scenes. For example, the shooting parameters as shown in Table 2 are changed.

(Procedure 2) The image quality of the captured image obtained by this shooting and the accuracy of the camera operation are evaluated, and appropriate shooting parameters are determined for each subject type (shooting scene).

(Procedure 3) The shooting parameters determined for each subject type (shooting scene) are registered in the control variable pattern file of the storage unit 15.

<Explanation of shooting operation>
FIG. 2 is a flowchart for explaining the photographing operation of the camera 11. Hereinafter, this operation will be described along the step numbers shown in FIG.

Step S1: The user performs an operation to set the function of the camera 11 (AF mode, exposure mode, photometry mode, drive mode, etc.) according to the intention of shooting. The setting unit 12 accepts this operation and sets the function of the camera 11.

Step S2: The sensor unit 14 detects the state of the subject or the operation state of the camera as sensor information based on the various sensors 14a to 14g.

Step S <b> 3: The search unit 20 acquires information related to the function set by the setting unit 12 and sensor information obtained by the sensor unit 14. The search unit 20 collates the combination of the setting state of the function and the sensor information with the shooting condition pattern file in the storage unit 15 and searches for a matching or similar shooting condition pattern file.

Step S4: The parameter determination unit 21 obtains the subject type associated with the searched shooting condition pattern file from the storage unit 15. The parameter determination unit 21 collates this subject type against the control variable pattern file in the storage unit 15 and determines the corresponding shooting parameter.
For example, in the case of a sunset subject, the exposure control is underexposed due to the influence of the brightness of the sun. Therefore, the imaging parameters are set so as to exclude the area corresponding to the sun from the photometric area used for exposure control. Also, shooting parameters are set to a higher color temperature so that the redness at sunset is not automatically corrected by the auto white balance function. Further, the shooting parameters are set so as to emphasize the red saturation.
In addition, for example, when the movement speed of the subject varies greatly and the maximum value of the movement speed is relatively low, it can be determined that the subject is a non-powered moving subject such as a human being or an animal. In this case, shooting parameters such as increasing sensitivity of focus adjustment control and expanding the area of the AF area to be used are set in accordance with the non-powered moving subject.

Step S5: The parameter determination unit 21 determines whether or not there are a plurality of shooting condition pattern files searched in step S3. When a plurality of shooting condition pattern files are searched, the parameter determination unit 21 moves the operation to step S6. If only one shooting condition pattern file is searched, the parameter determination unit 21 shifts the operation to step S8.

Step S6: It is preferable that the function and sensor information at the time of shooting match one shooting condition pattern file. This is because it is necessary to uniquely determine appropriate shooting parameters for the function and sensor information at the time of shooting. However, it is difficult to determine and store shooting parameters by classifying every shooting scene in advance.
Therefore, the shooting scene is divided into several tendencies, and the adaptability to various shooting scenes is enhanced by their combination (combination).

  Therefore, the parameter determination unit 21 creates a new shooting condition pattern file (1 + 8 + 11 shown in FIG. 3) by combining a plurality of shooting condition pattern files 1, 8, and 11 having relatively high matching degrees as shown in FIG. To do. In this case, when there are no contradictions or conflicts in the items (features) of the combined shooting condition pattern file, it is preferable to combine the items of the plurality of shooting condition pattern files into a new shooting condition pattern file. On the other hand, when contradiction or conflict occurs, it is preferable to define a new shooting condition pattern file by processing such as giving priority to one of a plurality of shooting condition pattern files or taking the middle of both. The parameter determining unit 21 registers the newly created shooting condition pattern file in the storage unit 15 as corresponding to a scene in which subject types are combined.

  For example, as a shooting condition pattern file, a new shooting condition pattern file corresponding to landscape + sunset + backlight (backlighted landscape object including sunset) is registered by combining a landscape subject, a sunset subject, and a backlight subject. The

  By generating and registering this new shooting condition pattern file, the generated shooting condition pattern file is preferentially searched for similar function settings and sensor information from the next time. . Note that the user may arbitrarily define the subject type corresponding to the newly created shooting condition pattern file.

Step S7: As shown in FIG. 4, the parameter determination unit 21 defines a new shooting parameter corresponding to the newly registered composite scene. For example, when there is no contradiction or competition among the plurality of shooting parameters obtained for each subject type in step S4, it is preferable to combine the plurality of shooting parameters into a new shooting parameter. Further, for example, when contradiction or competition occurs in a plurality of shooting parameters, it is preferable to define a new shooting parameter by processing such as giving priority to one or taking the middle of both. The parameter determination unit 21 registers the newly created shooting parameter in the control variable pattern file of the storage unit 15 as corresponding to the newly created shooting condition pattern file. The user may arbitrarily redefine the shooting parameters corresponding to the newly created composite scene later.

Step S8: The photographing unit 17 performs a photographing operation according to the photographing parameter determined in step S7 (step S4 when step S7 is not executed).

Step S9: The recording processing unit 18 acquires the attribute information indicating the photographing condition pattern file determined in step S6 (step S3 when step S6 is not executed) from the search unit 20. The recording processing unit 18 records this attribute information as metadata of the captured image captured in step S8 (data describing the characteristics of the image). For example, this attribute information is recorded in the EXIF information of the image file.

Step S10: The recording processing unit 18 records the image analysis result of the captured image as metadata of the captured image. For example, the following is preferable as the image analysis in this case.
(1) Face recognition / face judgment: Recognize the face in the image. Face recognition recognizes who is shown.
(2) Judgment of symmetry: Judges the symmetry of the image with respect to the horizon, and determines the reflection of the water surface.
(3) Point light source judgment: Judges whether the scene is dark or night scene with many illuminations (point light sources).
(4) Character recognition: Estimate the shooting scene from the result of character recognition. Extract text information as metadata.

  However, it is difficult to perform image analysis for every kind of subject type every time due to the problem of processing time. Therefore, as shown in FIG. 4, it is preferable to narrow down the types of subject types to be subjected to image analysis each time by using the subject types estimated from the photographing condition pattern file as reference information.

<< Description of Image Search Operation by Camera 11 >>
FIG. 5 is a flowchart showing an image search operation in the camera 11. Hereinafter, this operation will be described along the step numbers shown in FIG.

Step S41: The user inputs a keyword related to the subject to the input unit 19 in order to search for image data showing the desired subject. The input unit 19 transmits this input keyword to the image search unit 22.

Step S42: The image search unit 22 refers to the input keyword in the data table shown in Table 3 as an example, and selects attribute information (imaging condition pattern file) corresponding to the input keyword. The image search unit 22 refers to the attribute information of the recorded captured image for the attribute information, and selects a captured image that matches or is similar to the attribute information.

Step S43: The image search unit 22 arranges the thumbnails of the photographed images selected in Step S42 and creates a linked thumbnail page. The image search unit 22 displays the created thumbnail page data on the monitor screen (not shown) of the camera 11 as an image search result. The user can access the photographed image as a search result by browsing the thumbnail page and operating the link of the thumbnail page.
With this operation, it is possible to quickly search for an image showing a subject related to the input keyword from a large number of photographed images.

<Description of image search operation by computer>
The image search process described above can be executed not only on the camera 11 but also on a computer. FIG. 6 is a diagram for explaining image search processing by this computer. The computer 30 functions as the image acquisition unit 31, the input unit 32, and the image search unit 33 illustrated in FIG. 6 by starting the image search program.
The image acquisition unit 31 captures the captured image and attribute information (corresponding to the capturing condition pattern file) recorded by the camera 11 described above via a communication medium or a recording medium.

On the other hand, the user inputs a keyword related to the subject to the input unit 32 in order to search for image data showing the desired subject. The image search unit 33 refers to the keyword in the data table as shown in Table 3 above, and obtains attribute information (imaging condition pattern file) corresponding to the input keyword. The image search unit 33 collates this attribute information with the attribute information acquired from the image acquisition unit 31, and selects a captured image that matches or is similar to the attribute information.
Thus, attribute information has already been added in the camera 11 for the captured image recorded by the camera 11. For this reason, it is possible to immediately execute the above-described image search process simply by taking these captured images and attribute information into the computer 30.

<< Specific examples of sensor information processing >>
7 to 11 are diagrams for explaining a subject feature calculation algorithm.
First, FIG. 7A shows a state in which shooting direction information is newly created from sensor information of the direction sensor included in the acceleration sensor 14c of FIG. First, the angle θ1 of the optical axis of the camera 11 with respect to the north is detected from the output of this direction sensor. On the other hand, the horizontal angle of view ωh of the camera 11 is detected from the zoom encoder 14f shown in FIG. In this case, if the optical axis angle θ1 is in the range of (270 ° + ωh / 2)>θ1> (270 ° −ωh / 2), the true west falls within the horizontal angle of view of the camera 11. At this time, if the date and time information of the timer 14g is at the sunset time, the probability that the high-intensity subject in the captured image is sunset is high. It is also possible to obtain an accurate sun sunset direction based on the date and time information obtained from the timer 14g. In this case, it is possible to accurately determine the sunset based on whether or not the sunset direction falls within the horizontal angle of view ωh and corresponds to the sunset time.

  FIG. 7B shows a state in which the position in the screen of the horizontal line (horizon line) is detected based on the direction of gravity acceleration detected by the acceleration sensor 14c. From the output of the acceleration sensor, the elevation angle θ2 of the camera 11 is detected. On the other hand, the vertical angle of view ωv of the camera 11 is detected from the zoom encoder 14f. In this case, if the elevation angle θ2 is in the range of + ωv / 2> θ2> −ωv / 2, a horizontal line enters the vertical angle of view ωv of the camera 11. Further, the position of the horizontal line in the screen can be detected from the position of the elevation angle θ at the vertical field angle ωv.

  Further, FIG. 8 shows a divided photometric / colorimetric result of the photometric sensor 14a shown in FIG. In this case, the divided photometric / colorimetric results can be roughly divided into a ground area and an aerial area using the horizontal line position obtained in FIG. 7B. FIG. 9A shows the color appearance frequency of the ground area. FIG. 9B shows the lightness appearance frequency in the ground area. For example, when the ground area has a lot of green, it can be determined that the probability of grassland or artificial grass is high. Such a determination can be substituted by a color histogram for each RGB instead of a histogram for color and lightness.

  FIG. 10A shows the image plane moving speed obtained by time differentiation of the defocus amount detected by the distance measuring sensor 14d. Even for a subject moving at a constant speed, the image plane moving speed changes in a quadratic function depending on the characteristics of the optical system. FIG. 10B shows the image plane moving speed converted into the subject moving speed in the object space. From this subject moving speed and its speed fluctuation, the characteristics of the subject moving can be obtained.

<< Specific example of combination judgment of function and sensor information >>
FIG. 11 is a diagram comprehensively showing a function setting state and sensor information. From the combination of these pieces of information, the following subject feature calculation algorithm can be implemented.

(1) The angle of view and the shooting distance are detected from the focal length of the shooting lens (detected by the zoom encoder 14f), the focus position (detected by the distance encoder 14e), and the imaging size. Based on these sensor information, the field size (image area) can be calculated. From this field size, the actual size of the subject can be estimated. For example, this actual size can be used to assist the determination of face detection or object detection.

(2) The color / brightness information of the object scene is obtained from the output of the photometric sensor 14a. Based on the color / lightness information, the luminance distribution, color distribution, temporal luminance change, and the like of the object scene can be calculated.

(3) Defocus amounts at a plurality of positions in the photographing screen can be obtained by the distance measuring sensor 14d. By using this defocus amount, a rough distance distribution of the object scene can be detected. In addition, when the distance measuring operation is continuously performed, the tendency of the distance variation can be detected.

(4) The azimuth sensor and acceleration sensor can detect the angle from the north of the optical axis and the elevation angle with respect to the ground. Further, the inclination of the photographing screen can be detected by the vertical position / horizontal position sensor of the camera 11. Based on these sensor information, the position and inclination of the horizon in the captured image can be estimated.

(5) By detecting the time change of the acceleration of the acceleration sensor 14c, it is also possible to detect blurring or movement of the camera 11.

(6) The environmental temperature can be detected by the temperature sensor 14b. On the other hand, the timer 14g can detect date, season, and calendar information.

(7) The zoom lens 14f determines that the photographing lens is on the wide-angle side, the distance sensor 14d biases the distance distribution of the object field to infinity, and the photometric sensor 14a has a large blue area (sky) at the top of the screen. Is set to single frame shooting, and the AF mode is set to single area and single AF (AF-S).

(8) When the distance encoder 14e brings the shooting distance into the close-up area, the distance measurement sensor 14d has a narrow movement width of the subject, and the photometric sensor 14a has a feature that the color gamut detected is green, red, or yellow. There is a high probability of being close-up shooting.

(9) The photographing lens is determined to be on the telephoto side by the zoom encoder 14f, the AF mode is set to continuous AF (AF-C), the drive mode is set to the continuous shooting mode, and the photographing distance is set to 20 by the distance encoder 14e. When the subject speed is within 50 m and the subject speed is in the human travel speed range by the distance measuring sensor 14 d and fluctuates frequently (stops or runs), there is a high probability of shooting sports such as soccer.

(10) When it is detected by the acceleration sensor that the camera posture is downward and the shutter button is not half-pressed, it is determined that the photographing is not performed, and the detection operation can be stopped. If the camera posture changes suddenly with the shutter half-pressed, it can be determined that the composition has changed.

(11) In a situation where a landscape including a sunset is a subject, if a symmetry is recognized in subject brightness and subject color with the horizon as a symmetry axis by image analysis, it can be determined that the landscape is a sunset reflected on the water surface.

(12) It can be determined that it is night sports from the state of the subject such as green at the bottom of the screen and the light source color temperature of artificial lighting. In this case, a team name, a player name, etc. can be specified by character recognition of a uniform number or image recognition of a uniform. These name information can also be added to the captured image as attribute information.

<< Effects of this embodiment >>
In the present embodiment, the function setting of the camera 11 and the sensor information are combined and used as the determination material. The function of the camera 11 is set appropriately by the user in accordance with the main subject, and the situation of the main subject is strongly reflected. On the other hand, an objective subject situation is reflected in the sensor information. By combining both pieces of information having different characteristics in this way, the subject type can be estimated from more various viewpoints.

  In the present embodiment, the subject type is estimated by checking the function setting state and sensor information against a preset shooting condition pattern file. Such processing has less processing load than the case where subject determination by image analysis is performed one by one, and the subject type can be determined at high speed.

  Furthermore, in the present embodiment, when a plurality of shooting condition pattern files are searched, new shooting parameters are created by combining shooting parameters corresponding to the plurality of shooting condition pattern files. As a result, it is possible to flexibly cope with unregistered subject types and determine shooting parameters.

  In the present embodiment, the attribute information (shooting condition pattern file) obtained for determining the shooting parameter is recorded as metadata in association with the shot image. Therefore, it is possible to easily execute the image search process on the camera 11 or the computer 30 based on the attribute information.

  As described above, the present invention is a technique that can be used for a camera or the like.

It is a block diagram which shows the structure of this embodiment. 4 is a flowchart for explaining a photographing operation of the camera 11. It is a figure explaining the composite process of an imaging condition pattern file. It is a figure explaining the composite process of an imaging parameter, and the production | generation process of metadata. 4 is a flowchart showing an image search operation in the camera 11. It is a figure which shows the image search process by the computer. It is a figure explaining an imaging azimuth | direction and a detection process of a horizon. It is a figure explaining the field judgment of a ground area and an aerial area. It is a figure explaining the histogram analysis of a terrestrial area. It is a figure explaining the moving speed detection of a to-be-photographed object. It is a figure which shows the setting state and sensor information of a function comprehensively.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Camera, 12 ... Setting part, 13 ... Operation member, 14 ... Sensor part, 15 ... Memory | storage part, 16 ... Microprocessor, 17 ... Shooting part, 18 ... Recording process part, 19 ... Input part, 20 ... Search part, DESCRIPTION OF SYMBOLS 21 ... Parameter determination part, 22 ... Image search part, 30 ... Computer, 31 ... Image acquisition part, 32 ... Input part, 33 ... Image search part

Claims (5)

A setting section for setting camera functions;
A sensor unit for detecting a state of a subject or an operating state of the camera;
A storage unit for storing a combination of information corresponding to the function set by the setting unit and a detection result of the sensor unit in advance for each subject type;
A search unit that searches the storage unit for the combination corresponding to the function set by the setting unit and the detection result of the sensor unit;
A parameter determining unit that determines a shooting parameter of the camera based on the combination searched by the search unit;
A camera comprising: a photographing unit that performs photographing according to the photographing parameter determined by the parameter determining unit. The camera of claim 1,
The said parameter determination part produces the 2nd imaging | photography parameter based on the imaging | photography parameter corresponding to each of the said some combination, when the said corresponding combination exists in multiple numbers. The camera characterized by the above-mentioned. The camera according to claim 1 or 2,
A camera comprising: a recording processing unit that records attribute information corresponding to the combination in association with a photographed image photographed by the photographing unit. The camera according to claim 3.
A camera comprising: an image search unit that searches for the captured image recorded by the recording processing unit based on the attribute information. Computer
An image acquisition unit that captures the captured image and the attribute information recorded in the camera according to claim 3 or 4,
An input unit for receiving input of the attribute information;
An image search program for functioning as an image search unit for searching for the photographed image corresponding to the input attribute information.

Images