JP2003046848A - Imaging system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging system for generating appropriate images without being conscious of the control in an imaging system and without depending on the brightness of a subject. SOLUTION: The imaging system for handing a moving image as a picture comprises a photograph-switching section 8 for selecting moving image photograph or still image photograph, a still image generation section 1 for generating the picture information of the still image from picture information that is acquired as a moving image when the still image photograph is selected, a photograph condition estimation section 28 for estimating photograph conditions based on picture information that is acquired as a moving image or the picture information of a still image that is generated by the still image generation section 21, a gradation characteristic modification control section 29 for controlling the change in the gradation characteristics of picture information based on the estimated photograph condition, a gradation characteristic generation section 25 for generating the gradation characteristics of picture information according to the modification control of gradation characteristics, and a picture generation section 26 for generating reproduced picture information according to the gradation characteristics.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup system and an image pickup program, and more specifically, to effectively utilize the dynamic range of an input image by controlling gradation according to the condition of a subject in the image. The present invention relates to an imaging system and an imaging program capable of appropriately reproducing gradation.

[0002]

2. Description of the Related Art In an image pickup system such as a video camera capable of shooting a moving image, it is important to properly reproduce the gradation of a recorded image in every application. In particular, the subject of interest in the image (for example,
(People photographed with a home video movie camera, abnormal intruders in surveillance cameras, affected areas in endoscopes, etc.)
With respect to, it must be possible to reproduce without causing a loss of gradation and without causing a feeling of strangeness in the entire image.
For that purpose, it is necessary to reproduce the gradation according to the situation of the image.

Conventionally, the following techniques are known in relation to the gradation reproduction in the video as the moving image.

For example, in Japanese Patent No. 2951909, regarding a gradation correction device and a gradation correction method of an image pickup device, two types of video information having different exposure amounts in one field are used as input images, and based on the brightness information. It is disclosed that the input image is divided into areas, the gradation is corrected for each area, and the combined areas are combined to realize gradation correction adapted to a subject.

Further, Japanese Patent Application Laid-Open No. 2000-253386 discloses a method of controlling a surveillance television camera and a recording device, in which a shutter speed or an aperture of a camera is changed when an entering object is found in a television camera image. It is disclosed that an image having no blur and proper brightness is recorded.

Further, in Japanese Patent Laid-Open No. 2001-16500, in the electronic endoscope, the gain of the image information on the central area of the screen is amplified by a predetermined ratio from the gain of the image information on the peripheral area, so that It is described that the brightness of the area is made appropriate.

[0007]

However, considering that gradation reproduction is performed from the viewpoint of the subject of interest, the above-mentioned conventional techniques have the following problems, respectively.

That is, in the technique disclosed in Japanese Patent No. 2951909, area division is performed based on luminance information of an input image based on two types of image information with different exposures, and independent gradation correction is performed for each area. Therefore, when the subject of interest exists over the divided areas, it is considered that the subject is converted due to the respective gradation correction characteristics, and discomfort occurs such that discontinuous gradation occurs in the subject of interest.

Further, in the technique disclosed in Japanese Unexamined Patent Publication No. 2000-253386, the image pickup system of the camera is controlled when the entering object is detected. However, when the entering object moves at high speed in the area where the luminance change is large (that is, When the change in the brightness of the approaching object changes greatly), it becomes difficult to control the imaging system of the camera in real time by tracking the movement of the approaching object.

Further, in the technique disclosed in Japanese Unexamined Patent Publication No. 2001-16500, the gain is corrected and controlled so that the brightness of the central portion is always appropriate, and no consideration is given to image information. For this reason, the adaptability to the imaging situation is lacking, and for example, when the observed organ moves and the affected part is reflected in the periphery of the screen, it may be overlooked.

Therefore, the first object of the present invention made in view of the above point is to control the image pickup system by controlling the gradation necessary for image reproduction according to the situation of the subject in the image. It is an object of the present invention to provide an imaging system and an imaging program that can generate an appropriate image without depending on the brightness of a subject.

Further, a second object of the present invention is to provide an image pickup system and an image pickup program capable of reproducing the gradation of the whole image by effectively utilizing the dynamic range of the input image itself. It is in.

[0013]

According to a first aspect of the present invention, there is provided an image pickup system according to claim 1, wherein a moving image or a still image is selected in an image pickup system capable of handling a video as a moving image. And a still image generation unit that generates video information of a still image from video information acquired as a moving image when still image shooting is selected by the shooting switching unit, and a still image generation unit acquired as a moving image. A shooting situation estimation unit that estimates the shooting situation based on the video information or the video information of the still image generated by the still image generation unit, and the gradation of the video information based on the shooting situation estimated by the shooting situation estimation unit A gradation characteristic change control unit for controlling the change of characteristics; a gradation characteristic generation unit for generating the gradation characteristic of the video information according to the control of the gradation characteristic change control unit;
And a video generation unit for generating reproduced video information based on the gradation characteristic of the video information.

According to the first aspect of the present invention, video information of a moving image acquired by moving image shooting by switching in the shooting switching unit, or video of a still image generated by the still image generating unit by still image shooting. Based on the information, the situation (for example, the subject's situation) at the time of being photographed is estimated by the photographing situation estimation unit, and based on the estimation result, the gradation characteristic change control unit controls the gradation characteristic change. The gradation characteristic is generated in consideration of the shooting condition estimated by the gradation characteristic generating unit according to the change control of the gradation characteristic, and the image reproduced by the image generating unit is reproduced based on the gradation characteristic. (Image information) is generated, and as a result, reproduced image information considering the shooting situation is generated.

As described above, according to the first aspect of the invention, the situation at the time of shooting is estimated from the image, the change of the gradation characteristic is controlled in consideration of the situation, and the gradation characteristic is generated as necessary. ,
Since the image to be played back is generated by using the gradation characteristics in consideration of the shooting situation, both moving images and still images can be generated without considering the control of the imaging system and regardless of the brightness of the subject. In, it is possible to generate and reproduce an image in which the gradation of the subject is appropriately reproduced according to the shooting situation.

The invention of an image pickup system according to claim 2 for achieving the above-mentioned second object, in an image pickup system capable of handling a video as a moving image, includes a shooting switching section for selecting a moving image shooting or a still image shooting. A moving image generation unit that generates wide dynamic range image information of a moving image by synthesizing images with different exposures in a fixed time unit at a predetermined time when moving image shooting is selected in the shooting switching unit,
When still image shooting is selected in the shooting switching unit, a still image generation unit that synthesizes images with different exposures to generate wide dynamic range image information of the still image, and a moving image generated by the moving image generation unit Of the image information based on the image information of the image or the image information of the still image generated by the still image generating unit, and the image information of the image information based on the image estimated by the image estimating section. A gradation characteristic change control unit that controls a change in gradation characteristic, a gradation characteristic generation unit that generates a gradation characteristic of the video information under the control of the gradation characteristic change control unit, and a floor of the video information. And a video generation unit that generates reproduced video information based on the key characteristic.

According to the second aspect of the present invention, in moving image shooting by switching in the shooting switching unit, the moving image generating unit generates video information of a moving image having a wide dynamic range by different exposures in fixed time units at a predetermined time. Further, in still image shooting, video information of a still image with a wide dynamic range is generated by different exposures in the still image generation unit.
With respect to the generated video information having a wide dynamic range, the situation (for example, the situation of the subject) when the video information was photographed is estimated by the photographing situation estimation unit, and the gradation characteristic change control unit based on the estimation result. The control relating to the change of the gradation characteristic is performed, and the gradation characteristic generation unit generates the gradation characteristic in consideration of the estimated shooting condition according to the change control of the gradation characteristic, and based on the gradation characteristic, Since the video (playback video information) to be played back is generated by the video generation unit, as a result, the playback video information considering the shooting situation is generated.

As described above, according to the second aspect of the present invention, since image information obtained by different exposures is acquired, it is possible to generate an image (moving image, still image) having a wide dynamic range. Therefore, it is possible to reproduce an image in which gradation is not lost even if the subject is present in a portion that is too dark or too bright in normal shooting. In addition, the situation at the time of shooting is estimated from the image, the change of the gradation characteristic is controlled in consideration of the situation, the gradation characteristic is generated as necessary, and the gradation characteristic considering the shooting situation is used. Since the video to be played back is generated in this way, the dynamic range that the input video itself can have is effectively used in both moving images and still images, and the gradation of the subject can be changed according to the shooting conditions. A properly reproduced image can be generated and reproduced.

According to a third aspect of the present invention, in the image pickup system according to the first or second aspect, the photographing condition estimating section extracts a feature from the video information by the feature extracting section and the feature extracting section. An attention information setting unit that sets information regarding a subject of interest in the video information based on the above-described characteristics.

According to the third aspect of the present invention, the photographing condition estimating section extracts the characteristic from the video information by the characteristic extracting section,
Since the information about the subject of interest in the video information is set by the attention information setting unit using the extracted features, the information about the subject of interest can be set according to the situation of the features in the video information. Therefore, since the gradation can be controlled on the basis of the information on the target object, the gradation reproducibility of the target object can be improved in addition to the effect of the invention according to claim 1 or 2.

According to a fourth aspect of the present invention, in the image pickup system according to the first or second aspect, the photographing condition estimation section is displayed on the image display section for displaying the image information and on the image display section. An attention information setting unit that sets information regarding a subject of interest in the image information based on the image information.

According to the fourth aspect of the present invention, the photographing condition estimating unit displays the image information on the image display unit, and the attention information setting unit displays information on the subject of interest in the image information from the displayed image information. Since the setting is made, it is possible to set the information on the subject of interest while observing the displayed video information. Therefore, since the gradation can be controlled on the basis of the information on the target object, the gradation reproducibility of the target object can be improved in addition to the effect of the invention according to claim 1 or 2.

According to a fifth aspect of the present invention, in the image pickup system according to the first or second aspect, the photographing situation estimating section includes a knowledge information storing section for storing knowledge information about a photographing environment, and the knowledge information storing section. A shooting environment estimation unit that estimates the shooting environment of the video information based on the knowledge information of
And an attention information setting unit that sets information regarding a subject of interest in the video information based on the estimation result of the photographing environment estimation unit.

According to the fifth aspect of the present invention, in the photographing situation estimation unit, the photographing environment estimation unit estimates the photographing environment of the video information based on the knowledge information about the photographing environment stored in the knowledge information storage unit, Based on the result, the attention information setting unit sets the information regarding the subject of interest in the video information, so that it is possible to determine which subject the video information is photographing when the photographing target is predetermined. . Therefore, since the gradation can be controlled on the basis of the information on the target object, the gradation reproducibility of the target object can be improved in addition to the effect of the invention according to claim 1 or 2.

According to a sixth aspect of the present invention, in the image pickup system according to the first or second aspect, the photographing situation estimating section includes a knowledge information storing section for storing knowledge information about a photographing environment and the knowledge information storing section. A shooting environment estimation unit that estimates the shooting environment of the video information based on the knowledge information of
A gradation characteristic control unit that generates control information for controlling the gradation characteristic of the video information based on the estimation result of the photographing environment estimation unit and outputs the control information as the estimation result of the photographing situation. To do.

According to the sixth aspect of the present invention, in the photographing situation estimation unit, the photographing environment estimation unit estimates the photographing environment of the image information based on the knowledge information about the photographing environment stored in the knowledge information storage unit, Based on the result, the gradation characteristic control unit generates the control information for controlling the gradation characteristic of the video information and outputs the control information as the estimation result of the photographing situation, so that the photographing target is predetermined. In this case, it is possible to determine which object the video information is shooting, and to control the gradation characteristics according to the shooting target. Therefore, in addition to the effect of claim 1 or claim 2, The gradation reproducibility of the target can be improved.

According to a seventh aspect of the present invention, in the image pickup system according to the third, fourth or fifth aspect, the attention information setting section determines whether or not the subject of interest exists in the video information. A subject presence / absence determining unit, and a second level in which the image information is equal to or higher than a first predetermined value or smaller than a first predetermined value when there is no subject in the subject presence / absence determining unit.
Exposure for acquiring the video information when a level detection unit that detects a portion having a predetermined value or less and a part where the level of the video information is the first predetermined value or more is detected as a result of the level detection unit And an exposure control unit that controls so as to increase exposure when acquiring the video information when detecting a portion where the level of the video information is equal to or less than the second predetermined value. And a target position setting unit that sets a portion corresponding to the subject as a target portion when the subject is present in the subject presence / absence determining unit.

According to the seventh aspect of the present invention, in the attention information setting unit, the subject presence / absence determining unit determines whether or not the subject of interest exists in the image information. If there is no subject, the level of the image information is determined. Is the first predetermined value (saturation level)
The level detection unit detects a portion that is equal to or more than or equal to or less than a second predetermined value (blackened level) smaller than the first predetermined value, and as a result of the detection, when the level of the video information is equal to or higher than the first predetermined value (saturation level). When a certain portion is detected, the exposure for obtaining the image information is reduced, and when the portion where the image information level is equal to or lower than the second predetermined value (blackout level) is obtained, the image information is obtained. The exposure control unit controls the exposure when acquiring the image information so that the exposure is increased, and when the subject exists, the attention position setting unit sets the corresponding portion as the attention portion. In addition to the effects of the inventions according to items 3 to 5, when the subject of interest in the image information is not seen due to saturation or blackout, it is possible to control the exposure when obtaining the image information so that the subject can be seen. Can Consequently it is possible to reproduce an image including the subject without underexposure or overexposure.

The invention according to claim 8 is the image pickup system according to claim 3, 4 or 5, wherein the attention information setting section uses at least the luminance level information of the subject as the information on the subject to be set. When the luminance of the subject of interest is darker than a predetermined range from the luminance level information, the exposure of the image information is controlled to be increased, and the luminance of the subject of interest is within the predetermined range. It is characterized in that it includes an exposure control section that controls so as to reduce the exposure when acquiring the image information when it is on the brighter side.

According to the invention of claim 8, the brightness information of the object is included as the information about the object of interest, and when the brightness of the object is on the darker side than the predetermined range, the video information is acquired from this brightness information. By including an exposure control unit that controls each of the exposure control unit so as to increase the exposure and reduce the exposure when acquiring image information when the brightness of the subject is on the brighter side than the same predetermined range, Even when the brightness cannot be reproduced properly, the exposure can be controlled to keep the subject within the range of appropriate brightness. Therefore, in addition to the effects of the inventions according to claims 3 to 5, as a result, it is possible to reproduce an image including a subject without blackout or whiteout.

According to a ninth aspect of the present invention, in the image pickup system according to the seventh or eighth aspect, the exposure control section is
When changing the exposure when acquiring the image information, the generation of the wide dynamic range image information is controlled so as to change the exposure for one or more images of the images with different exposures in the fixed time unit. To do.

According to the ninth aspect of the invention, the exposure control section controls the generation of the wide dynamic range image information so as to change the exposure for one or more images among the images with different exposures in a fixed time unit. The exposure can be controlled so that the brightness of the subject is appropriate at the stage of the image input for the range image information. Therefore, in addition to the effect of the invention according to claim 7 or 8,
It is possible to reproduce the subject corresponding to the image of each exposure without blackout or blown-out highlights.

According to a tenth aspect of the present invention, in the image pickup system according to the fifth or sixth aspect, the estimation result generated by the photographing environment estimating unit is the knowledge stored in the knowledge information storage unit. Among the information, the information corresponds to the above-mentioned video information.

According to the tenth aspect of the present invention, the photographing environment estimating section outputs knowledge information corresponding to the image information as an estimation result, so that the object of photographing of the image information can be determined, and the gradation is determined based on the result. Can be controlled. Therefore, in addition to the effect of the invention according to claim 5 or 6, it is possible to improve the gradation reproducibility of the determined subject to be photographed.

The invention described in claim 11 is the image pickup system according to claim 5, wherein the attention information setting unit compares the estimation result output from the photographing environment estimation unit with the video information and makes a difference. It is characterized in that processing is performed on a portion where the level of is equal to or higher than a predetermined value.

According to the eleventh aspect of the present invention, the attention information setting unit compares the estimation result output from the photographing environment estimation unit with the image information and processes the portion having a large difference. Can be set to control the gradation. Therefore, in addition to the effect of the invention according to claim 5, the gradation reproducibility of the subject of interest can be improved.

According to a twelfth aspect of the present invention, in the image pickup system according to the first or second aspect, the photographing condition estimating section is configured to focus position estimated from focus information at the time of shooting and photometric information at the time of shooting. It is characterized in that the shooting situation is estimated based on the subject brightness distribution estimated from the.

According to the twelfth aspect of the invention, the photographing situation is estimated based on the focus position estimated from the focus information at the time of photographing and the subject brightness distribution estimated from the photometric information at the time of photographing. In addition to the effect of the invention according to Item 1 or 2, it is possible to adaptively estimate the shooting situation with respect to the shooting situation, and as a result, generate an image in which the gradation of the subject is appropriately reproduced according to the shooting situation. Can be played.

According to a thirteenth aspect of the present invention, in the image pickup system according to the first or second aspect, the still image generating section has a level of the video information or the reproduced video information on a screen which is equal to or higher than a third predetermined value. When a still image level detection unit that detects a portion that is smaller than a third predetermined value and equal to or less than a fourth predetermined value and a portion that is detected by the still image level detection unit are present, video information or reproduced video as still image information. A continuous shooting unit that continuously acquires information, a position correction unit that aligns the still image information group acquired by the continuous shooting unit on the screen, and the still image information group corrected by the position correction unit And a still image synthesizing unit for synthesizing one still image information to generate one still image information.

According to the thirteenth aspect of the present invention, in the still image generation unit, the still image level detection unit has a third predetermined value (saturation level) or more or a fourth predetermined value smaller than the third predetermined value (blackened level) or less. When a part is detected, the continuous shooting part continuously shoots to acquire a group of still image information when that part exists, the position correction part aligns the positions of the still image information groups, and the still image composition part combines them. For example, when a still image is acquired from a video in which there is a saturated portion due to the effect of specular reflection, a still image information group that is not saturated in the same portion is synthesized. One still image can be generated. Therefore, in addition to the effect of the invention according to claim 1 or 2, it is possible to generate a still image without saturation or blackout.

According to a fourteenth aspect of the present invention, in the image pickup system according to the thirteenth aspect, the still image synthesizing unit is
A reference image designation unit that designates still image information serving as a reference in the still image information group, and the level of the still image information designated by the reference image designation unit is equal to or higher than the third predetermined value or equal to or lower than the fourth predetermined value. And a reference image correction unit that corrects the portion using the still image information group other than the designated still image information.

According to the fourteenth aspect of the present invention, in the still image synthesizing unit, the reference image specifying unit specifies the reference still image information in the still image information group, and the reference image correcting unit saturates or blackens the reference still image information. Since the correction is performed using the remaining still image information group for the part where the
In addition to the effect of the invention according to 3, the still image information group can be effectively used to generate a still image without saturation or blackout.

According to a fifteenth aspect of the present invention, in the image pickup system according to the first or second aspect, the gradation characteristic change control unit changes the gradation characteristic when a change in the photographing condition occurs in the video information. It is characterized in that change control information for controlling the change is output.

According to the fifteenth aspect of the present invention, the gradation characteristic is updated by outputting the change control information for controlling the gradation characteristic to be changed when the photographing condition is changed in the video information. In addition to the effect of the invention according to 1 or 2, it is possible to control the change of the gradation characteristic according to the change of the shooting condition. Moreover, since the gradation characteristic is changed only when the photographing situation changes, it is not necessary to constantly update the gradation characteristic, and the processing load can be reduced.

According to a sixteenth aspect of the present invention, in the image pickup system according to the first or the second aspect, the gradation characteristic change control section is such that the gradation characteristic is changed when still picture photographing is selected by the photographing switching section. Is output as change control information.

According to the sixteenth aspect of the present invention, in still image shooting, change control information for controlling to change the gradation characteristic is output to update the gradation characteristic, and the gradation characteristic dedicated to the still image is generated. In addition to the effect of the invention according to claim 1 or 2, it is possible to realize appropriate gradation reproducibility for a subject of a still image.

According to a seventeenth aspect of the present invention, in the image pickup system according to the first or second aspect, the gradation characteristic change control unit calculates an evaluation value representing a temporal change of information regarding the subject of interest. It is characterized in that change control information for controlling to change the gradation characteristic is output when the evaluation value determines that the temporal change is large.

According to the seventeenth aspect of the present invention, the evaluation value representing the temporal change of the information on the subject of interest is calculated, and the gradation characteristic is controlled to be changed when it is determined that the temporal change is large. Since the control information is output, in addition to the effect of the invention according to claim 1 or 2, it is possible to control the change of the gradation characteristics according to the change of the subject of interest. Moreover, since the gradation characteristics are changed only when there is a change in the subject of interest, it is not necessary to constantly update the gradation characteristics, and the processing load can be reduced.

According to the eighteenth aspect of the present invention, in the image pickup system according to the first or second aspect, the gradation characteristic generating unit is configured to detect the image when the subject of interest is detected by the photographing condition estimating unit. In the information, a weighting pattern is set that has a characteristic such that the portion corresponding to the subject of interest is weighted more than other portions, and if the subject of interest is not detected by the shooting situation estimation unit, the The present invention is characterized in that a weighting pattern that gives a uniform weight to the entire screen is set, and gradation characteristics are generated using the set weighting pattern.

According to the eighteenth aspect of the present invention, when a subject of interest is detected by the photographing condition estimating unit, a characteristic is set so that a portion corresponding to the subject of interest in the video information is weighted more than other portions. By setting a weighting pattern to be held, and if the subject of interest is not detected, by setting a weighting pattern that uniformly weights the entire screen of the video information, the presence or absence of the subject of interest, the position,
It is possible to set an adaptive weighting pattern according to the characteristics in the video information such as, and by using the weighting pattern for the generation of the gradation characteristics, the gradation characteristics considering the characteristics of the image, especially the subject of interest. Can be generated. Therefore, in addition to the effect of the invention according to claim 1 or 2, due to the gradation characteristic that attaches importance to the subject of interest, an image excellent in tone reproducibility of the subject of interest (without gradation collapse or extreme emphasis) is produced. Can be generated.

According to a nineteenth aspect of the present invention, in the image pickup system according to the eighteenth aspect, the gradation characteristic generating section is
A gradation characteristic feature for generating a gradation characteristic from the video information is detected, a histogram relating to the video information is created using the gradation characteristic feature and the weighting pattern, and a floor is calculated based on the histogram. It is characterized by generating a tonality characteristic.

According to the nineteenth aspect of the present invention, a histogram relating to the image information is created from the characteristics for gradation characteristic detected from the image information (for example, the frequency characteristic represented by an edge and the sharpness characteristic) and the weighting pattern. Since the gradation characteristic is generated from the histogram, in addition to the effect of the invention according to claim 18, it is possible to generate the gradation characteristic adaptive to the characteristic of the video information, and the gradation of the characteristic portion is not impaired. Video can be generated.

The invention according to claim 20 is the invention according to claims 3 and 4.
Alternatively, in the image pickup system according to the fifth aspect, the gradation characteristic generation unit determines that the target subject is set when the ratio of the information regarding the target subject set in the target information setting unit to the screen is equal to or more than a predetermined value. It is characterized in that the gradation characteristics are generated by limiting the applicable gradation.

According to the twentieth aspect of the invention, since the gradation characteristic is generated by limiting the gradation corresponding to the object of interest when the ratio of the object of interest occupying the screen exceeds a predetermined value, In addition to the effects of the inventions of 3 to 5, it is possible to prevent the gradation of the subject of interest from being unnecessarily increased and the quality of the video information from being deteriorated.

According to a twenty-first aspect of the present invention, in the image pickup system according to the fifth or sixth aspect, the gradation characteristic generating section is associated with the knowledge in the knowledge information storage section corresponding to the estimation result of the photographing situation estimating section. It is characterized in that gradation characteristics are generated based on information.

According to the twenty-first aspect of the invention, the gradation characteristic generating section generates the gradation characteristic based on the knowledge information corresponding to the estimation result of the photographing situation estimating section.
In addition to the effect of the invention according to the above, in addition to the knowledge information or the control information for controlling the gradation characteristic reflecting the knowledge information, the gradation characteristic suitable for the knowledge information, that is, the gradation suitable for the image information The characteristics can be generated and the video can be played.

According to a twenty-second aspect of the present invention, in the image pickup system according to the first or the second aspect, the image generation unit outputs the image information based on the gradation characteristic generated by the gradation characteristic generation unit. The luminance information is converted, and the color difference information of the video information is converted based on the luminance information before and after the conversion by the gradation characteristic and the theoretical limit characteristic of color reproduction, and the reproduced video information is converted from the converted luminance information and the color difference information. Is generated.

According to the twenty-second aspect of the invention, the luminance information of the image information is converted by the gradation characteristic, and the color difference information of the image information is converted by the luminance information before and after conversion by the gradation characteristic and the theoretical limit characteristic of color reproduction. Since the reproduced video information is generated by the reproduced video information, it is possible to generate the reproduced video information with good gradation reproducibility and color reproducibility in addition to the effect of the invention according to claim 1 or 2.

Further, in the invention of an image pickup program according to claim 23, which achieves the first object, the computer for controlling the operation of the image pickup system capable of handling a video as a moving image shoots a moving image or A shooting switching function for selecting still image shooting, a still image generating function for generating video information of a still image from video information acquired as a moving image when still image shooting is selected by the shooting switching function, and a moving image. Based on the shooting situation estimation function that estimates the shooting situation based on the video information acquired as an image or the video information of the still image generated by the still image generation function, and the shooting situation estimated by the shooting situation estimation function The gradation characteristic change control function for controlling the change of the gradation characteristic of the image information, and the level of the image information according to the control by the gradation characteristic change control function. The present invention is characterized by realizing a gradation characteristic generating function for generating a gradation characteristic and a video generating function for generating reproduced video information based on the gradation characteristic of the video information.

According to the invention of claim 23, claim 1
Similar to the invention according to, estimating the situation at the time of shooting from the video,
In consideration of the situation, the change of the gradation characteristic is controlled, the gradation characteristic is generated as needed, and the image to be reproduced is generated by using the gradation characteristic in consideration of the shooting situation. It is possible to generate and reproduce an image in which the gradation of the subject is appropriately reproduced according to the shooting condition in both the moving image and the still image regardless of the control of the subject and regardless of the brightness of the subject. .

Further, according to the invention of an image pickup program described in claim 24, which achieves the second object, a computer for controlling an operation of an image pickup system capable of handling a video as a moving image shoots a moving image or When video shooting is selected by the shooting switching function that selects still image shooting and the above shooting switching function, the wide dynamic range video information of a moving image is created by synthesizing videos with different exposures in fixed time units at a predetermined time. A moving image generation function to generate,
When still image shooting is selected by the shooting switching function, a still image generation function that synthesizes images with different exposures to generate wide dynamic range image information of the still image, and a moving image generated by the moving image generation function Of the image information based on the image information of the image or the image capturing condition estimated by the image capturing condition estimating function for estimating the image capturing state based on the image information of the still image generated by the image capturing function. A gradation characteristic change control function for controlling the change of the gradation characteristic, a gradation characteristic generation function for generating the gradation characteristic of the video information according to the control by the gradation characteristic change control function, and a floor of the video information. It is characterized by realizing a video generation function for generating reproduced video information based on the key characteristic.

According to the invention of claim 24, claim 2
Similarly to the invention according to, the image information obtained by different exposures can be obtained, so that an image with a wide dynamic range (moving image, still image) can be generated, and therefore, the subject is located in a portion that is too dark or too bright in normal shooting. Even if there is, it is possible to reproduce an image without gradation loss. In addition, the situation at the time of shooting is estimated from the image, the change of the gradation characteristic is controlled in consideration of the situation, the gradation characteristic is generated as necessary, and the gradation characteristic considering the shooting situation is used. Since the video to be played back is generated in this way, the dynamic range that the input video itself can have is effectively used in both moving images and still images, and the gradation of the subject can be changed according to the shooting conditions. A properly reproduced image can be generated and reproduced.

[0063]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of an imaging system and an imaging program according to the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 shows the first embodiment of the present invention.
FIG. 1 is a block diagram showing a basic configuration of a video camera (handheld video camera) as an imaging system according to an embodiment. This video camera is composed of a single-plate color CCD or the like having an electronic shutter function, and an image pickup device 1 for photoelectrically converting a subject image and outputting it as image information, and a subject image is formed on the image pickup device 1. Lens 2, a diaphragm / shutter mechanism 3 for controlling the passage range and passage time of the light flux that has passed through this lens 2, and a noise component by a correlated double sampling circuit (not shown) after being output from the image sensor 1. Amplifier for amplifying the video information from which the noise has been removed, an A / D converter 5 for converting the analog information amplified by this amplifier 4 into digital information, and digitization by this A / D converter 5. The AF (autofocus) information and the AE (autoexposure) information are received by the video information processing circuit 6 that performs various processes on the processed information and the digital output from the A / D converter 5. Ja) information, AW
A for detecting B (auto white balance) information
An F, AE, AWB detection circuit 7, a compression circuit 9 for performing information compression processing such as JPEG compression or MPEG compression on the video information output from the video information processing circuit 6, and a compression circuit 9.
Of the video information compressed by, the still image recording medium I / F 14 for performing control for recording the video information relating to the still image on a still image recording medium 15 such as a memory card described later, and the still image recording medium I / F 14. Of the still image recording medium 15 for recording the image information of the still image under the control of the recording medium I / F 14 for recording, and the image information compressed by the compression circuit 9.
A moving image recording medium I / F 16 for performing control for recording video information relating to a moving image on a moving image recording medium 17 such as a digital video (DV) tape or a versatile disc (DVD) described later, and this moving image recording medium I / F 16. Video recording medium 1 for recording video information of a moving image by controlling the recording medium I / F 16
7, a DRAM 11 used as a working memory when performing color processing of video information, a memory controller 10 for controlling the DRAM 11, and a monitor 13 described later.
And a display circuit 12 for controlling the display circuit 12, and by controlling the display circuit 12, the video information recorded in the still image recording medium 15 or the moving image recording medium 17 is reproduced and displayed, and various kinds of shooting related to this video camera are performed. Monitor 1 for displaying status etc.
3, a timing generator (TG) 18 that generates a timing pulse for driving the image sensor 1, a strobe 19 that emits illumination light for illuminating a subject during still image shooting, and various shooting modes are set. Input key 20 having a switch for switching and a trigger switch for inputting a photographing operation, a video information processing circuit 6, a compression circuit 9, a memory controller 10, a display circuit 12, a still image recording medium I / F14, moving image recording medium I / F16
It is connected via a bus line with the AF, AE, AWB detection circuit 7, the light emission information by the strobe 19, the input information by the input key 20, etc., and receives the lens 2, the diaphragm / shutter mechanism 3, the TG 18, the strobe. 1
9, a video information processing circuit 6, an input key 20, and a CPU 8 for controlling the entire video camera including each circuit connected to the bus line.

With this video camera, a wide shooting mode in which normal shooting mode in which image information is obtained by normal shooting and images with different exposures are shot and these images are combined to obtain one wide dynamic range (DR) image information Shooting mode,
Is manually selected by operating the input key 20, or the CPU 8 automatically makes a judgment by detecting a whiteout of image information from the image sensor 1 to select a shooting mode. C depending on the selected shooting mode
The shooting operation is controlled by the PU 8.

That is, when the normal photographing mode is selected, image information for one screen in one frame (two fields) is acquired from the image sensor 1 by the photographing operation,
On the other hand, when the wide DR photographing mode is selected, a known photographing means using the electronic shutter function of the image pickup device 1 or a combination of this electronic shutter function and the diaphragm / shutter mechanism 3 (for example, photographing means by double speed field drive). ), Image information of a plurality of screens (for example, screens corresponding to two types of exposures) having different exposure amounts is obtained from the image sensor 1 by shooting one frame, and an image corresponding to the shooting mode is obtained in the video information processing circuit 6. Process data.

In this video camera, a moving image shooting mode for obtaining video information as a moving image and a still image shooting mode for obtaining video information as a still image are selected by operating the input keys 20, and the selected shooting is performed. The operation obtained by the CPU 8 is controlled according to the mode.

That is, when the moving image shooting mode is selected, the CPU 8 controls the image pickup device 1, the diaphragm / shutter mechanism 3, and the image information processing circuit 6 so as to shoot a moving image (for example, one frame at a time). When the still image shooting mode is controlled, the CPU 8 controls the image sensor 1 and the aperture / shutter mechanism 3 to shoot a still image.
The video information processing circuit 6 is controlled.

The shutter release in the still image shooting mode may be provided with a shutter release key dedicated to the still image shooting mode, or the shutter release is controlled when the still image shooting mode is selected by the input key 20. You may. At that time, in the case where the entire body is dark such as at night or inside the organ, and the subject is dark such as backlight, the strobe 19 emits light to capture a still image.

As a result, the present embodiment is provided with four shooting modes: moving image shooting by normal shooting, still image shooting by normal shooting, moving image shooting by wide DR shooting, still image shooting by wide DR shooting. Become.

In this embodiment, the moving image recording medium and the still image recording medium are separately prepared.
One recording medium I / F is controlled by the instruction of PU8,
The moving image and the still image may be recorded on the same recording medium.

FIG. 2 shows the video information processing circuit 6 shown in FIG.
3 is a block diagram showing the configuration of FIG. The video information processing circuit 6 includes a wide DR still image video information generation circuit 21, a wide DR video image information generation circuit 22, a luminance / color difference information separation circuit 23,
It has an edge detection circuit 24, a gradation characteristic generation circuit 25, a video generation circuit 26, a subject search processing circuit 27, a shooting condition estimation circuit 28, and a gradation characteristic change control circuit 29.
Switching between movie shooting mode and still image shooting mode
Control is performed according to the moving image / still image shooting mode information nn output from No. 8.

The digital video information aa output from the A / D converter 5 is either the wide DR still picture video information generation circuit 21 or the wide DR motion picture video information generation circuit 22 according to the moving picture / still picture shooting mode information nn. Supply to crab. That is, when the moving image / still image shooting mode information nn indicates moving image shooting, the digital image information aa is supplied to the wide DR moving image image information generating circuit 22, and the moving image / still image shooting mode information nn indicates still image shooting. In this case, the digital video information aa is supplied to the wide DR still image video information generation circuit 21.

The wide-DR still image video information generation circuit 21 and the wide-DR video image information generation circuit 22 respectively generate wide-DR still-picture and moving-picture video information by known techniques and output the wide-DR video information bb. To do. Also, wide DR
The still image video information generation circuit 21 outputs still image gradation characteristic instruction information hh for controlling the generation of gradation characteristics in addition to the wide DR video information bb.

The wide DR video information bb is, for example, information about video information of different exposures obtained from the shooting means by the double speed field drive control in the video shooting mode, in the present embodiment, video information corresponding to two types of exposures. It is generated by combining the exposure ratios between them. Even in the still image shooting mode, the drive control of the shooting means is changed to generate the image by the same method.

The generated wide DR video information bb is supplied to the luminance / color difference information separation circuit 23, where it is separated into luminance information dd and color difference information cc. Of these, the luminance information dd is supplied to the edge detection circuit 24, where an edge is detected by an existing filter (Laplacian, Sobel, etc.) and output as edge information ee. In this embodiment, the edge information ee
Outputs as binary information indicating whether or not each pixel is an edge.

On the other hand, the wide DR still picture image information generation circuit 21
Alternatively, the wide DR video image information bb output from the wide DR video image information generation circuit 22 is also supplied to the subject search processing circuit 27, and attention information ii indicating information regarding a subject of interest and a photographing time are captured by a method described below. The exposure change control information pp for instructing the change of the exposure is generated and output.

The exposure change control information pp is transmitted to the CPU 8 via the bus line, and the attention information ii is supplied to the photographing situation estimation circuit 28 together with the moving image / still image photographing mode information nn.
The shooting situation estimation circuit 28 outputs shooting situation information kk indicating an estimation result regarding the shooting situation by a method described later.

The photographing condition information kk and the still image gradation characteristic instruction information hh are supplied to the gradation characteristic change control circuit 29. In the gradation characteristic change control circuit 29, gradation characteristic change instruction control is performed by a method described later, and at the same time, gradation characteristic instruction information mm indicating information for reflecting the preliminarily estimated photographing condition on the gradation characteristic is given. Is output.

The luminance information dd, the edge information ee, and the gradation characteristic instruction information mm are supplied to the gradation characteristic generating circuit 25, and the gradation characteristic is generated by the method described later to generate the gradation characteristic information ff.
Output as. The gradation characteristic information ff is supplied to the video generation circuit 26 together with the luminance information dd and the color difference information cc. The video generation circuit 26 converts the luminance information and the color difference information based on the gradation characteristics, synthesizes the luminance information and the color difference information, and generates and outputs the converted video information gg.

In the present embodiment, the gradation characteristic generation circuit 25
The wide DR video information bb supplied to the
Binary edge information ee (that is, high frequency component information) is handled, but as another feature,
On the contrary, it is also possible to use information on low frequency components and information on sharpness characteristics.

FIG. 3 is a diagram for explaining a method of generating a wide dynamic range (DR) image in the first embodiment. In the present embodiment, for example, in the moving image shooting mode, in one field unit time (1/60 seconds), information for two screens of a short exposure (SE) image and a long exposure (LE) image is acquired, A wide DR image for one frame is generated by combining these.

As a synthesis method, known techniques include
For an overexposed portion of an LE image due to a large luminance value, the same portion of the SE image is adjusted to the brightness of the LE image and then combined. As a result, it has the effect of being enlarged by the exposure time ratio of the SE image and the LE image as compared with the DR that the image sensor can have. For example, the exposure time of SE image is 1/1000 second, and the exposure time of LE image is 1/1000 second.
If it is set to 125 seconds, by combining these, it becomes possible to capture an image having a DR that is eight times as large as the DR of the image sensor.

In the still image shooting mode, it is possible to capture the video information for two screens without setting the time limit of one field unit period.

FIG. 4 is a block diagram showing the configuration of the wide DR still image / video information generating circuit 21 shown in FIG. Wide DR
The still image / video information generation circuit 21 is a moving image from the CPU 8 /
When the still image shooting mode information nn represents still image shooting, in response to this, two types of still image images with different exposures are taken in, and if necessary, to reproduce the part not reflected due to the influence of specular reflection or the like. Wide DR still image synthesizing circuit 31, level detecting circuit 32, first switch 33, second switch 34, continuous shooting image control circuit 35, continuous shooting image position correcting circuit 36, reference image estimating circuit 37, It has a reference image correction circuit 38 and a gradation characteristic change instruction output circuit 39.

The digital video information aa output from the A / D converter 5 is sent to the wide DR still image synthesizing circuit 31 in response to the content of the still image shooting in the moving image / still image shooting mode information nn from the CPU 8. The wide DR video information bb as a still image is generated and output by, for example, the method shown in FIG. The output wide DR video information bb is supplied to the level detection circuit 32, where the wide DR video information bb may have DR.
Find out about. Specifically, a dark area threshold ThB1 indicating a blackout condition and a bright area threshold ThW1 indicating an overexposed area (ThW1> Th).
B1) is preset and T is set for the entire wide DR video information bb.
Check whether it exists between hB1 and ThW1. As a result, level appropriate information rr is output.

The level adequacy information rr is information for driving the first switch 33 and the second switch 34. That is, in the level detection circuit 32, wide DR video information
If all the information is present in the range of bb between the thresholds ThB1 and ThW1, it is determined that it is an “appropriate level”, and the wide DR video information is obtained.
bb is passed through the first switch 33 and the second switch 34 to the luminance / color difference information separation circuit 23 and the subject search processing circuit 2
Directly to 7.

On the other hand, in the level detection circuit 32, wide D
When the R video information bb exists outside the range between the thresholds ThB1 and ThW1, it is determined that “there is an inappropriate level”, and the wide DR video information bb is set to the first switch 33, the continuous shooting control circuit 35,
The continuous shooting image position correction circuit 36, the reference image estimation circuit 37, the reference image correction circuit 38, and the second switch 34 are passed through the luminance / color difference information separation circuit 23 and the subject search processing circuit 2.
Transmit to 7.

The continuous shooting control circuit 35 obtains a plurality of still image video information by the continuous shooting function in order to obtain the information of the suitability level for the portion of the wide DR video information bb which is the "unsuitable level". To control.

By the way, when the cause of the improper level is due to the influence of specular reflection or the like, the exposure is performed with the wide DR image information bb.
It is not possible to reproduce it as an appropriate level simply by shooting differently from the time of acquisition, and it is necessary to change the position (viewpoint) of the imaging system to shoot. On the other hand, when hand-held shooting with a video camera, "camera shake" occurs, so that viewpoints do not always match exactly when continuous shooting is performed.

Therefore, in the present embodiment, this "camera shake" is positively used to perform continuous shooting at a predetermined timing in accordance with the camera shake cycle. As a result, it is possible to realize the operation of continuous shooting while shifting the viewpoint.

In this way, the video information of the still image photographed by the function of the continuous shooting control circuit 35 (images by two kinds of exposure) is passed through the wide DR still image synthesizing circuit 31 and the continuous shooting control circuit 35. Supply to. Here, the aim is to acquire a plurality of still image video information so that the “inappropriate level” in the first still image video information can be obtained as the “appropriate level”. As a result, the continuous shooting control circuit 35 outputs the wide DR continuous shooting video information ss including the wide DR video information bb of the still image.

Wide D output from the continuous shooting control circuit 35
The R-continuous-shooting video information ss is then supplied to the continuous-shooting image position correction circuit 36, where the position correction processing is performed on the wide DR continuous-shooting video information ss. Specifically, wide DR video information bb for still images
Is fixed and the position of other wide DR video information (video information captured by the continuous shooting control circuit 35) on the screen is corrected. Regarding the position correction, a method (template matching or the like) that is performed in a known matching process is used, and the position can be corrected by shifting the position to the maximum correlation. Then, these are generated and output as position-corrected wide DR continuous shooting video information tt.

The position-corrected wide DR continuous-shooting video information tt output from the continuous-shooting image position correcting circuit 36 is then supplied to the reference image estimating circuit 37, where the reference image to be generated as a still image is generated. presume. What is estimated as the reference image is, for example, the wide DR video information bb that is the first still image video information, but it is not particularly limited to this. Inside,
It is conceivable that the video information having the smallest portion indicating the "inappropriate level" is used as the reference image. As a result, the reference image information uu is generated and output.

The position-corrected wide DR continuous shooting image information tt output from the continuous shooting image position correction circuit 36 and the reference image information uu output from the reference image estimation circuit 37 are supplied to the reference image correction circuit 38. Here, a process of correcting the portion indicating “inappropriate level” in the reference image information uu by using the video information other than the reference image is executed. This correction process is
The portion indicating the improper level of the reference image is replaced by matching it with the level of the reference image by using video information other than the reference image information uu that is at the proper level, and the like. As a result, the inappropriate level is corrected to an appropriate level, that is, the wide DR video information bb as a still image without blackout or overexposure is generated.

The wide DR video information bb of the still image generated by the reference image correction circuit 38 is passed through the second switch 34, and the luminance / color difference information separation circuit 23, the subject search processing circuit 27 shown in FIG. And gradation characteristic change instruction output circuit 39
Supply to. In the gradation characteristic change instruction output circuit 39, the wide D
Upon receiving the R video information bb, still image gradation characteristic instruction information hh is generated and transmitted to the gradation characteristic change control circuit 29 shown in FIG. That is, the gradation characteristic change instruction output circuit 39 generates still image gradation characteristic instruction information hh as control information for instructing the gradation characteristic change control circuit 29 to newly generate the gradation characteristic of the still image. Output.

FIG. 5 is a diagram for explaining the processing in the wide DR still image video information generation circuit 21 described above. Wide DR
The still image synthesizing circuit 31 obtains wide DR video information bb of a still image as shown in FIG. 5A, for example. Note that FIG.
Is a wide DR for the lower part of the face and the left chest (oval).
It indicates that the image information is not obtained due to reflection even though the image is taken.

Therefore, this reflection portion is connected to the level detection circuit 3
2, and the continuous shooting control circuit 35 performs continuous shooting of wide DR. As a result, a still image video as shown in FIG. 5B is acquired. In FIG. 5 (b), the left still image is obtained from the information about the lower part of the face of the person, which is not obtained in FIG. 5 (a), and the right still image in FIG. 5 (b) is obtained. Similarly, information on the left chest of the person, which has not been obtained in FIG. 5A, is obtained. Note that FIG.
Although (b) acquires two still image videos, the number of still images is not limited to this, and three or more still image videos may be acquired.

Next, regarding the still picture image of FIG.
Assuming that the position is corrected by the continuous image position correction circuit 36, a still image as shown in FIG. 5C is acquired. The still image video shown in FIG. 5C is obtained by deleting the portion which does not exist in FIG. 5A by referring to the screen of the still image video shown in FIG. The still image is shown in Figure 5 (a).
The position is corrected so as to form a partial still image. This facilitates the correlation calculation.

Next, in the reference image estimation circuit 37, as shown in FIG. 5D, the still image of FIG. 5A is used as a reference image, and a portion of the reference image to be corrected, Estimate the portion of the still image other than the reference image that corresponds to the correction target of the reference image. That is, of the reflection parts to be corrected in the reference image on the left side of FIG. 5D, the lower part of the human face (“A”) corresponds to “A” of the still image in the center of FIG. Figure 5 for left chest ("B")
(D) It is estimated that it corresponds to "B" of the still image on the right side. Then, the reference image correction circuit 38 corrects the reference images for "A" and "B", and as a result, a wide still image (corrected) from which the influence of reflection is removed as shown in FIG. The DR video information bb is generated.

FIG. 6 is a block diagram showing the structure of the subject search processing circuit 27 shown in FIG. As described above, the subject search processing circuit 27 searches for a subject of interest with respect to the video information on the screen, generates and outputs information regarding the subject of interest if there is a subject of interest, and there is no subject of interest. In this case, the information for generating the image information and controlling the exposure at the time of shooting is generated and output. The image feature extraction circuit 41, the subject presence / absence determination circuit 42,
It has a subject position setting circuit 43, an exposure change control circuit 44, and an attention information setting circuit 45.

In FIG. 6, the wide DR video information bb output from the wide DR still image video information generation circuit 21 or the wide DR moving picture video information generation circuit 22 is supplied to the video feature extraction circuit 41, where the video-related features are extracted. Is extracted and output as video feature information ww.

For example, when the wide DR image information bb is a moving image, the difference information between the image at the current time (field unit or frame unit) and the image at the immediately previous time, similarly the image at the current time. Information such as motion vector information corresponding to the video at the immediately preceding time and information indicating temporal change of the video is extracted as a feature. Also, wide DR video information bb
If the image is a still image, information indicating frequency characteristics by filtering (high-pass filter for detecting high-frequency components, low-pass filter for detecting low-frequency components, etc.) and information on color components of characteristics (for example, skin color detection in human photography) Etc.) are extracted as features. It should be noted that a plurality of methods may be used to detect the features, and the detected features may be integrated to generate and output the video feature information ww.

The image feature information ww is supplied to the subject presence / absence determining circuit 42 together with the wide DR image information bb, and it is determined here whether or not the subject is present in the screen. The determination condition regarding the presence / absence of a subject is set according to the image feature information ww. For example, the following conditions are conceivable when the subject may be present. (I) There is a portion on the screen where the degree of the image feature information ww is remarkably large (for example, a portion where the feature indicating the degree of skin color is remarkably large, such as a person's face). (Ii) The screen is divided into predetermined blocks, and the blocks showing a tendency different from the surrounding blocks in the video feature information ww for each block (for example, in the case of a moving image, a person when the camera is stationary and a person is moving) , A block corresponding to a subject when the camera is tracking a moving subject, a portion showing a tendency that the motion vectors on the screen are different, and the like).

Various other conditions can be considered, but when there are a plurality of judgment conditions, the judgment conditions are weighted and integrated, or the judgment conditions are prioritized. By this determination, information indicating whether or not a subject is present on the screen is output as subject presence / absence information xx. The subject presence / absence information xx is, for example, information indicating a range in which the subject is present (coordinate values indicating the range), information obtained by adding information indicating the presence / absence of the subject (1 bit information, etc.) to the image feature information ww, And so on.

The subject presence / absence information xx output from the subject presence / absence determination circuit 42 is supplied to the subject position setting circuit 43 or the exposure change control circuit 44 according to the content thereof. That is, when the subject presence / absence information xx indicates “subject present”, the subject presence / absence information xx is supplied to the subject position setting circuit 43, where the position of the subject on the screen (the subject portion within the range in which the subject exists). (Position) and output as subject position information zz. Note that the subject presence / absence information xx
When the feature in the image represented by the image feature information ww is included in the subject position information zz, the content of the image feature information ww may be included in the subject position information zz.

On the other hand, when the subject presence / absence information xx indicates "no subject", the subject presence / absence information xx is wide DR.
The exposure change control information pp is supplied together with the image information bb to the exposure change control circuit 44, and the control information for changing the exposure at the time of photographing is set by the method described later, and the exposure change control information pp for instructing to change the exposure at the time of photographing is set. CPU via bus line as
8 is transmitted. This is because when the subject presence / absence determination circuit 42 determines “no subject”, the presence of the subject cannot be confirmed because the exposure during shooting is not performed properly. This is done to make it possible to more accurately determine the presence of the subject by making it appropriate.

If the presence of the subject is not seen due to the exposure change, it is determined that there is no subject (that is, the entire screen is the target portion), and the content is transmitted to the subject position setting circuit 43 as the total target information yy. To do. When the entire target information yy is supplied, the subject position setting circuit 43 sets the entire screen as the subject position and reflects the result in the subject position information zz.

The object position information zz output from the object position setting circuit 43 is supplied to the attention information setting circuit 45 together with the wide DR image information bb, and the object position and wide DR image on the screen indicated by the object position information zz are displayed here. Based on the information bb, attention information ii is generated as information regarding the subject of interest, and is transmitted to the shooting situation estimation circuit 28. This attention information
ii can be represented by, for example, luminance information, color information of the image at the subject position, and feature information at the subject position if the subject position information zz includes the feature in the image.

FIG. 7 shows the exposure change control circuit 4 shown in FIG.
4 is a flowchart of the exposure change algorithm in 4.
The exposure change control circuit 44 controls the exposure change using the wide DR video information bb when the subject presence / absence determination circuit 42 determines "no subject". In the generation, two types of exposure image information (long-time exposure (LE) image and short-time exposure (SE) image) are used.

First, the level of the entire image is calculated from the wide DR image information bb (step S1). The level calculated here is, for example, luminance information or the like, and is expressed in pixel units or in block units composed of a plurality of local pixels.

Next, the calculated video level is set to "L", and the level L of each unit such as a pixel or a block is compared with a threshold value "ThB2" indicating a dark portion, and a level at which "ThB2>L" is satisfied. Is checked (step S)
2).

When it is determined in step S2 that there is a level L smaller than the dark area threshold ThB2, it is estimated that there is a blackout portion in the target video information, and the exposure condition for the LE video is changed to a 1EV bright exposure condition. The exposure change information to be set is set (step S3). That is, by making the long-time exposure 1EV brighter, the blackout portion is made brighter and the image information included therein is reproduced.

Next, the level L of each unit such as a pixel or a block is compared with a threshold "ThW2" indicating a bright portion,
It is checked whether or not there is a level at which "ThW2 <L" is satisfied (step S4).

If it is determined in step S4 that there is a level L that is larger than the bright area threshold ThW2, it is estimated that there is an overexposed portion in the target image information, and the exposure condition for the SE image is set to the 1EV dark exposure condition. The exposure change information to be changed is set (step S5). That is, by making the short-time exposure 1 EV dark, the overexposed areas are darkened and the image information included therein is reproduced.

Next, based on the determination results of steps S2 and S4, whether exposure change has been performed for at least one of SE and LE (whether at least one of step S3 and step S5 has been executed). Is determined (step S6).

When it is determined in step S6 that the exposure change has been performed, the exposure change control information pp for controlling the exposure change is output based on the exposure change information generated in step S3 or step S5. Then, control is performed so that the image is captured again in the changed exposure state (step S
7). By executing this step S7, the wide-range DR video information bb obtained by taking the image again is similarly set in the subject search processing circuit 27.
When it is determined that there is no subject, the exposure changing process described above is executed again.

On the other hand, if it is determined in step S6 that the exposure has not been changed, it is determined that there is no problem with the exposure state at present. In this case, the subject of interest does not exist in a certain part of the screen but exists in the entire screen. Therefore, it is determined that the entire screen is focused, and the entire attention The information yy is generated and output (step S8).

FIG. 8 is a diagram for explaining a subject position (attention position) setting operation by the subject position setting circuit 43 shown in FIG. 6, and FIGS. 8A to 8D show an operation relating to a moving image. 8 (e) to 8 (f) show the operation relating to the still image.

In the case of a moving image, the subject position of interest is set by utilizing the temporal change of the image between adjacent frames. For example, one frame of wide DR video information (FIG. 8A) captured and combined at time n−1 and the next time n
When the motion vector is detected in block units based on the difference (temporal difference) between the wide-frame DR video information (FIG. 8B) for one frame captured and synthesized in FIG.
It becomes like (c). Here, looking at the state of the motion vector on the entire screen, the area where the motion vector exists is smaller than the area where the motion vector does not exist. In other words, it is possible that the imaging system is stationary and you are shooting a moving subject.
It can be estimated that the part in which the motion vector exists is the subject of interest (it may be estimated by considering the screen position). Therefore, by setting the portion where the motion vector exists as the subject position,
A portion corresponding to a person can be set as in (d).

In the case of a still picture image, color information, which is a characteristic of the image information itself, is used. For example, with respect to the wide DR video information of FIG. 8E, a “skin color” portion is detected, and a correlation with a reference skin color (correlation with a reference color component of the skin color) is obtained and there is a correlation. The part that is equal to or greater than the predetermined value is detected. Since it can be considered that the detected part is "human skin", it can be estimated that the skin-colored part is the subject of interest (similar to the video image, it is possible to estimate by considering the screen position. ). Therefore, by setting the part detected as the skin color as the subject position, the part corresponding to the skin color part (face, neck, arm, etc.) of the person can be set as shown in FIG.

FIG. 9 is a circuit diagram of the photographing situation estimating circuit 2 shown in FIG.
8 is a flowchart showing the operations of 8 and gradation characteristic change control circuit 29. Here, finally the gradation characteristic instruction information
Although the process up to outputting mm is performed, the process is branched depending on the content of the moving image / still image shooting mode information nn output from the CPU 8 (whether the target wide DR image information bb is a moving image or a still image). .

First, the shooting situation estimation circuit 28 receives the attention information ii output from the subject search processing circuit 27 and generates a shooting situation pattern depending on the screen position (step S11). In the shooting situation pattern generated here, the screen is divided into a plurality of blocks (the block size does not necessarily correspond to the block size in the attention information ii), and the position to be noticed according to the block is divided. This is information for distinguishing the position from the other position. The value indicating the pattern may be a binary value (1 is assigned to the block at the position of interest, 0 is assigned to the other block), but the attention information ii is more detailed information (for example, the feature amount in the video information). Itself, etc.), a multi-valued pattern may be shown according to the degree of the attention information.

Next, the generated shooting situation pattern is output as shooting situation information kk (step S12). As a result, the shooting situation information kk becomes information indicating the position of the subject of interest on the screen and the degree of attention (in the case of a multivalued pattern).

Next, it is judged from the contents of the moving image / still image shooting mode information nn whether the target image is a moving image or a still image (step S13).

If it is determined that the target image is a moving image, the gradation characteristic change control circuit 29 receives the photographing condition information kk output from the photographing condition estimating circuit 28,
A comparison is made with the already-stored shooting status information before the predetermined time (step S14). In this case, the shooting situation information to be compared is the shooting situation information generated in the processing performed before the predetermined time. Here, it is examined how a temporal change occurs between the shooting condition information stored in the gradation characteristic change control circuit 29 and the shooting condition information kk newly output from the shooting condition estimation circuit 28. In particular,
The difference is calculated by comparing the values indicating the patterns of the blocks at the same position on the screen, and the difference between the blocks is weighted according to the screen position and the arithmetic mean is taken as the evaluation value.
Etc. are processed. However, if the shooting situation information is not stored immediately after the start of shooting, step S14 is not executed.

Next, if it is determined in step S14 that there is a temporal change in the shooting status information (the evaluation value representing the difference between the patterns described above is equal to or greater than a predetermined threshold value), or immediately after the start of shooting, shooting is performed. If the status information is not stored, the latest shooting status information kk is used as the gradation characteristic change control circuit 2
9 (step S15). Then, the gradation characteristic instruction information mm for instructing the change of the gradation characteristic is generated based on the latest photographing situation information kk (step S16).

The gradation characteristic instruction information generated here need only include the content for instructing to change the gradation characteristic, but in the present embodiment, the photographing condition information kk itself is indicated as the gradation characteristic instruction. Information in mm. This is because when the gradation characteristics are generated in the gradation characteristics generation circuit 25 later, the shooting condition information
This is because the pattern indicating the degree of attention in block units defined by kk is used.

Next, if it is determined from the processing in step S14 that there is no temporal change in the shooting status information (the evaluation value representing the difference between the above patterns is less than a predetermined threshold value), the gradation Gradation characteristic instruction information mm representing the content of instructing not to change the characteristic is generated (step S17). When the gradation characteristic instruction information at this time is expressed by a pattern indicating the degree of attention in block units as described above, a pattern indicating the content in which the gradation characteristic is not changed (for example, 0 is shown for all blocks). Generate as.

On the other hand, if it is determined in step S13 that the target video is a still video, the shooting condition estimation circuit 2
8, receiving the shooting status information kk output from the shooting status estimation circuit 28 and the still image gradation characteristic instruction information hh output from the wide DR still image video information generation circuit 21, and based on the shooting status information kk Gradation characteristic instruction information mm for instructing to change the tone characteristic is generated (step S18). That is, in the case of a still image video, the gradation characteristic is generated according to the shooting situation of the still image video, instead of controlling the change of the gradation characteristic according to the temporal change of the shooting situation information like the video image. In order to control so as to execute, the processing is executed in this manner. Here, as in the case of a moving image, shooting status information kk (a pattern indicating the degree of attention in block units)
It is generated as the gradation characteristic instruction information mm.

Then, step S16 and step S17
The gradation characteristic instruction information mm generated in both (in the case of moving image) or in step S18 (in the case of still image) is output to the gradation characteristic generating circuit 25 (step S19).

FIG. 10 is a block diagram showing the structure of the gradation characteristic generating circuit 25 shown in FIG. As described above, the gradation characteristic generation circuit 25 generates gradation characteristics based on the luminance information dd, the edge information ee, and the gradation characteristic instruction information mm, and the weighting pattern setting circuit 51 and the edge histogram calculation circuit 52. , And a gradation characteristic calculation circuit 53.

First, the gradation characteristic instruction information mm output from the gradation characteristic change control circuit 29 is supplied to the weighting pattern setting circuit 51 to set the weighting pattern for generating the gradation characteristic. The weighting pattern here is to set the weight of the part at the subject (attention) position larger than the weight of the parts other than the subject (attention), and make the gradation at the subject (attention) position more appropriate. To control it. In the present embodiment, gradation characteristic instruction information mm
The weighting pattern is set by using the pattern shown by the above, which indicates the degree of attention in block units. The set weighting pattern is output as weighting pattern information AA.

The weighting pattern information AA output from the weighting pattern setting circuit 51 is the edge histogram calculation together with the luminance information dd output from the luminance / color difference information separating circuit 23 and the edge information ee generated in the edge detecting circuit 24. It is supplied to the circuit 52, where a histogram relating to the luminance information of the edge portion is calculated. In calculating this histogram, considering the weighting pattern information AA,
The increment value of the frequency of the luminance information is controlled according to the weight. The edge histogram calculated here is output as edge histogram information BB.

The edge histogram information BB is supplied to the gradation characteristic calculation circuit 53, where the edge histograms are accumulated to obtain a cumulative edge histogram, which is normalized according to the relationship between the input luminance information and the output luminance information. By doing so, the gradation characteristic is calculated and transmitted as the gradation characteristic information ff to the video generation circuit 26.

FIG. 11 shows the gradation characteristic generation circuit 25 described above.
6 is a diagram for explaining the operation of FIG. In FIG. 11, wide DR
The case where the image information bb is a moving image is shown. Figure 1
As shown in FIG. 1 (a), the luminance / color difference information separation circuit 23 generates the luminance information of the wide DR video information for one frame at time n, and the edge detection circuit 24 has the luminance information as shown in FIG. 11 (b). The edge information shown in FIG. 11A is generated.
The edge information is held as binary information indicating the presence / absence of an edge by a known filter operation such as Laplacian or Sobel depending on whether the operation value is a threshold value or more.

On the other hand, subject position (attention) information (FIG. 8) set based on the motion vector shown in FIG.
The weighting pattern setting circuit 51 sets the weighting pattern as shown in FIG. 11D based on (a) to (d). This weighting pattern controls gradation characteristics in the subsequent generation of gradation characteristics. In addition to the subject position, the weighting pattern is set by considering the type of the shooting scene (short-distance subject shooting, long-distance landscape shooting, etc.) and integrating them. In FIG. 11D, since the shooting scene is a person at a short distance, a weighting pattern that gives the largest weight to the subject position is set based on the weighting pattern that gives greater weight to the central portion than to the peripheral portion. It is set. Also, the screen is divided into 9 blocks in the horizontal direction and 5 blocks in the vertical direction to set weights.

Then, the luminance information of FIG.
The edge histogram calculation circuit 52 calculates the edge histogram by integrating the edge information shown in FIG. 11B and the weighting pattern shown in FIG. The edge histogram referred to here is generated by counting the frequency of the luminance information of the portion where the edge exists according to the edge information of FIG. 11B by the corresponding weight from the weighting pattern of FIG. 11D. This is the histogram. Therefore, in FIG. 11D, the frequency of the histogram regarding the luminance information of the edge corresponding to the person is counted most. The gradation histogram of the calculated edge histogram is calculated by the gradation characteristic calculation circuit 53, and the normalized edge histogram is further normalized according to the relationship between the input luminance value and the output luminance value. Such gradation characteristics are generated.

From FIG. 11 (e), regarding the "person part" in the part having a relatively small brightness value and the "background part" in the part having a relatively large brightness value, the gradation is as shown by the hatched lines. Although it is assigned, especially regarding the “person part”, FIG.
Since the weight is large in (d), the range of gradation allocation in the output luminance value is wide. As a result, it is possible to improve the gradation reproducibility of the “portrait part” while maintaining the gradation of the “background part”. Note that the contents of calculating the gradation characteristics from the histogram relating to the luminance information of the edge portion in consideration of the weight are described in, for example, Japanese Patent Laid-Open No. 2000-2000.
No. 228747 discloses that the contrast of an image can be improved, especially when the histogram of the original image is concentrated in a specific range because the histogram is flattened. It is described in detail.

FIG. 12 shows the video generation circuit 26 shown in FIG.
6 is a diagram showing the limit characteristics of color difference information used in the image generation processing in FIG. In the video generation circuit 26, the luminance information dd and the color difference information
After receiving cc and the gradation characteristic information ff, the luminance information is first converted based on the gradation characteristic. Here, assuming that the luminance information before conversion is Y, the luminance information after conversion is Y ′, and the gradation characteristic of the information x is Trs (x), the relationship between Y and Y ′ is expressed by the following equation.

## EQU1 ## Y '= Trs (Y) (1)

Next, similar conversion is performed on the color difference information. Regarding the conversion of the color difference information, the brightness information before and after the conversion is used, but it is possible that the range of the color difference information that can be normally reproduced is exceeded by simply multiplying the ratio of the brightness information before and after the conversion. It is necessary to consider the reproducible range. Therefore, the color difference information is converted using the limit characteristic indicating the reproduction range of the color difference information as shown in FIG.

Specifically, the limit characteristic generated from the luminance information before conversion is Lmt (Y), and the limit characteristic generated from the luminance information after conversion is Lmt (Y '). The ratio GC is defined by the following equation.

[Equation 2] GC = Lmt (Y ') / Lmt (Y) (2)

This ratio GC is treated as a conversion coefficient of color difference information. That is, by multiplying the color difference information Cr, Cb relating to the information having the luminance information Y before conversion by GC, the color difference information Cr ', Cb' corresponding to the luminance information Y'after conversion is generated. The converted color difference information Cr ′, Cb ′ is calculated by both the gradation characteristics relating to the luminance information and the limit characteristics indicating the reproduction range of the color difference information, so that the gradation conversion is appropriately performed within the reproduction range. Will be done. Then, the converted luminance information Y'and the color difference information Cr ', Cb' are combined and output as converted video information gg.

Although the first embodiment has been described above, the present embodiment can be variously applied. For example,
The features of the image used in the subject search process, the attention information set for the subject of interest, and the like are not limited to those described in the present embodiment, and each information may be applied in block units, pixel units, or the like. it can. Also, wide DR
The still image / video information generation circuit 21 realizes continuous shooting with different viewpoints at a timing (shooting interval) matched with the cycle of “camera shake”. Instead, a sensor for detecting camera shake is provided, and the sensor output The timing of continuous shooting may be set according to the above. On the other hand, if "camera shake" does not occur due to the video camera being fixed, even if continuous shooting is performed while moving the image pickup system by a drive system that can minutely move the image pickup system such as an image stabilization lens. Good, specular reflection by using a control method such as moving the image pickup system minutely while paying attention to the "inappropriate level" portion existing in the wide DR video information bb of the still image, and shooting at the appropriate level It is also possible to obtain a wide-DR still image without any image.

(Second Embodiment) FIG. 13 is a block diagram showing the basic structure of a video camera as an image pickup system according to the second embodiment of the present invention. After that, the second
With regard to the embodiments, those having the same functions as those of the first embodiment are designated by the same reference numerals, description thereof will be omitted, and only different portions will be described.

In the first embodiment, wide DR video information is generated from videos shot a plurality of times with different exposures, but in the second embodiment, video information with wide DR is acquired by one shot with video input. Use an image sensor that can be used. For example, if the image information obtained from an image sensor capable of inputting 12 bits is 8
When outputting to the bit output system, etc.

Further, the video camera according to the second embodiment has a map information memory 62 which is connected to the bus line and stores map information relating to the photographing environment, and is stored in the map information memory 62. The image information processing circuit 61 reads the existing map information via the bus line and performs a process using the map information. In the second embodiment, for example, an endoscopic device for observing the inside of the human body (holding map information about the internal structure of the human body), or observing the inside of the building or the external environment in a predetermined process (mobile robot or automobile) It is assumed to be used for a mobile surveillance camera (which is installed in the vehicle) (which holds map information about the inside and outside environment of the building).

FIG. 14 is a block diagram showing the structure of the video information processing circuit 61 shown in FIG. The video information processing circuit 61 searches for a subject of interest based on the map information in the map information memory 62, estimates the shooting condition of the subject, and uses the map information to determine the gradation characteristics suitable for the subject of interest. A switch 71, a still image / video information generation circuit 72, a subject search processing circuit 73, a gradation characteristic generation circuit 74, and a luminance / color difference information separation circuit 23 that performs the same operation as that shown in FIG. Edge detection circuit 2
4, an image generation circuit 26, a shooting condition estimation circuit 28, and a gradation characteristic change control circuit 29.

The digital image information aa output from the A / D converter 5 (unlike in the first embodiment, the image information of a wide DR is already obtained by one shot) is stored in the switch 71.
In, distribution is performed according to the type of video information. When the switch 71 receives the moving image / still image shooting mode information nn from the CPU 8 and the digital image information aa is a moving image, the wide DR image information bb of the moving image (digital image information aa
(Same as the above), and is directly transmitted to the luminance / color difference information separation circuit 23 and the subject search processing circuit 73.

On the other hand, when the digital image information aa is a still image image, it is supplied to the still image image information generation circuit 72, where the wide DR image information bb as a still image image is generated,
It is transmitted to the luminance / color difference information separation circuit 23 and the subject search processing circuit 73. In the still image / video information generation circuit 72, a wide D is added from the configuration of the still image / video information generation circuit 21 shown in FIG.
With the configuration and processing excluding the R still image synthesis circuit 31,
Wide DR image information bb of a still image image is generated.

In the subject search circuit 73, the wide DR image information bb of the moving image or the still image and the map information memory 6
2 receives the environment map information DD transmitted via the bus line, the attention information ii indicating the information about the subject of interest, the exposure change control information pp for instructing the exposure change at the time of photographing, and the environment by the method described later. The shooting position information EE indicating the shooting position based on the map information DD is output. The exposure change control information pp is transmitted to the CPU 8 via the bus line, the attention information ii is supplied to the photographing condition estimation circuit 28, and the photographing position information EE is supplied to the gradation characteristic generation circuit 74. The shooting condition estimation circuit 28 and the gradation characteristic change control circuit 29 execute the same processing as that in the first embodiment.

The gradation characteristic generation circuit 74, in addition to the luminance information dd, the edge information ee, the gradation characteristic instruction information mm, the environment map information DD from the map information memory 62, and the subject retrieval circuit 7
The photographing position information EE from 3 is input, and the gradation characteristic information ff indicating the information regarding the gradation characteristic is generated and output by the method described later. The video generation circuit 26 generates and outputs the converted video information gg by the same configuration and processing as in the first embodiment.

As in the first embodiment, the characteristics of the wide DR video information bb supplied to the gradation characteristic generating circuit 74 are not limited to the binary edge information ee relating to the luminance information dd, but other frequency components and sharpness. It is also possible to use information on degree characteristics.

FIG. 15 is a block diagram showing the structure of the subject search processing circuit 73 shown in FIG. The subject search processing circuit 73 includes a video feature extraction circuit 41, a shooting position estimation circuit 81, a subject presence / absence determination circuit 82, a subject position setting circuit 83, an exposure change control circuit 84, and an attention information setting circuit 85. The function is the same as that of the subject search processing circuit 27 in the first embodiment shown in FIG.
In the present embodiment, using the environment map information DD, a process of recognizing the current shooting position and searching for a subject of interest therein is performed. Here, only points (numbers) different from those in the first embodiment will be shown.

In the photographing position estimation circuit 81, the wide DR video information output from the A / D converter 5 via the switch 71 or from the still picture video information generation circuit 72 is output.
By receiving bb and the environmental map information DD output from the map information memory 62, the position on the map at the time of shooting is estimated. In the present embodiment, it is premised that the predetermined environment (inside the human body, inside the building, etc.) is photographed as described above. Therefore, by referring to the map information corresponding to the photographing environment, It is possible to estimate which part of the image is captured.

Therefore, the photographing position estimating circuit 81 estimates which portion of the environment map information DD is photographed from the wide DR video information bb as the photographing position. As an estimation method, for example, landmarks (characteristic parts in the environment; characteristic parts of organs inside the human body, plates indicating room numbers inside the building, etc.) that serve as landmarks in the environment map information DD are preliminarily set. The setting is performed and the shooting position is estimated from the type and positional relationship of the landmarks in the wide DR video information bb. In addition to the wide DR video information bb, a method of estimating the shooting position by using a sensor that separately detects information regarding movement may be adopted. Then, as a result, the shooting position information indicating the shooting position based on the environmental map information DD
Generate and output EE.

In this embodiment, the photographing position information EE
The image information includes image information generated from the map information about the periphery of the photographing position (image information reproduced according to the exposure at the time of photographing based on the map information corresponding to the photographing position). On the other hand, with respect to the wide DR video information bb, when the shooting position cannot be estimated because the exposure state is not appropriate, the shooting position information EE indicating that is generated and output.

The subject presence / absence determination circuit 82 receives the image feature information ww output from the image feature extraction circuit 41, the wide DR image information bb, and the shooting position information EE output from the shooting position estimation circuit 81, and receives the object The presence or absence is judged. Here, first, the level (for example, brightness information, etc.) of the video information and the position of the video information around the shooting position and the wide DR video information bb included in the shooting position information EE are adjusted and then compared with each other. The part where the difference in the level of 1 is considered to be large is detected, and this is set as the “subject existence candidate region” in which the subject of interest is considered to exist. Then, wide DR video information corresponding to the “subject existence candidate area”
Based on bb and the image feature information ww, the presence or absence of a subject is determined according to the feature situation.

Here, the "subject existence candidate area" is detected using the shooting position information EE and the wide DR video information bb by comparing the information on the map with the video actually shot,
There is something to be noticed in the different part, that is, it does not exist in the map information but exists in the actual image for a certain subject, or exists in the map information but does not exist in the actual image. Then, the presence / absence of a subject is determined using the image feature information ww in this “subject existence candidate region”.
The subject presence / absence information xx generated and output by the subject presence / absence determination circuit 82 has the same contents as in the first embodiment.

In the subject position detection circuit 83, subject presence / absence information xx output from the subject presence / absence determination circuit 82 (information indicating that "subject exists"), and overall attention information yy output from an exposure control change circuit 84 described later. By receiving (the information indicating that the entire screen is the subject) and the shooting position information EE (when the shooting position can be estimated) output from the shooting position estimation circuit 81, the position of the subject on the screen is determined. Set. Here, by using the map information indicated by the shooting position information EE, the subject position information zz including the shooting position information EE is generated and output. This makes it possible to recognize the subject position in the environment indicated by the map information in association with the shooting position of the map information.

In the exposure control change circuit 84, the subject presence / absence information xx (the information indicating that "there is no subject") output from the subject presence / absence determination circuit 82, the wide DR video information bb, and the shooting position estimation circuit 81 are output. Shooting location information EE
In response (when the shooting position could not be estimated), the control information for changing the exposure during shooting is set. The process for changing the exposure is executed when either the subject presence / absence information xx or the photographing position information EE is received. In addition, in this embodiment, since the wide DR video information bb is obtained by one shot, the brightness information of the wide DR video information bb is slightly different from the processing by the algorithm flow shown in FIG. Based on this, the process of changing the exposure condition once to the bright side or the dark side is performed. The resulting exposure change control information pp (when an exposure condition change is instructed), and general attention information yy (when an exposure condition change is not instructed)
Shows the same contents as in the first embodiment.

The attention information setting circuit 85 receives the subject position information zz and the wide DR video information bb output from the subject position detection circuit 83, and from the subject position on the screen or the map information indicated by the subject position information zz. The attention information ii is generated and output. In the present embodiment, the contents of the attention information ii include the subject position information zz in addition to the contents shown in the first embodiment.
It is possible to use the shooting position information corresponding to (i.e., the map information corresponding to the shooting position).

FIG. 16 is a block diagram showing the structure of the gradation characteristic generating circuit 74 shown in FIG. The gradation characteristic generation circuit 74 has a weighting setting circuit 91 corresponding to the photographing environment, a weighting pattern setting circuit 92, an edge histogram calculation circuit 52 and a gradation characteristic calculation circuit 53 having the same configurations as those shown in FIG. There is. This gradation characteristic generation circuit 7
The generation of the gradation characteristic by 4 is the same as that in the first embodiment, but in the present embodiment, information on the shooting environment (environment map information DD and shooting position information EE) is used together,
The gradation characteristics are controlled according to the shooting environment. Here, only the parts (numbers) different from those in the first embodiment will be shown.

In the shooting environment-corresponding weight setting circuit 91,
Environment map information DD output from the map information memory 62 and shooting position information output from the subject search processing circuit 73
EE is used to set the weighting pattern corresponding to the shooting environment. Here, a weighting pattern corresponding to the environment map information DD is prepared in advance, the environment map information DD corresponding to the photographing position information EE is searched, and the corresponding weighting pattern is selected from the search result.

The weighting pattern corresponding to the photographing environment is
For example, if you are worried about whether there is an affected area around the center but the center is dark like an endoscope, prepare a weighting pattern that gradually reduces the weight from the center to the surroundings. When it is desired to make uniform observation over the entire screen, there is a method of preparing a weighting pattern for making the weights of the entire screen the same. The weighting pattern set by the weighting setting circuit 91 corresponding to the shooting environment is output as the shooting environment weighting pattern information FF.

In the weighting pattern setting circuit 92, the gradation characteristic instruction information output from the gradation characteristic change control circuit 29 is output.
mm, and the shooting environment weighting pattern information FF output from the shooting environment corresponding weight setting circuit 91, and sets a weighting pattern for generating gradation characteristics. The processing here is to generate the final weighting pattern information AA by combining the pattern indicating the degree of attention in block units represented by the gradation characteristic instruction information mm and the shooting environment weighting pattern information FF. Output. As a method of combination, the pattern of the gradation characteristic instruction information mm is superimposed and normalized on the photographing environment weighting pattern information FF, or the pattern of the photographing environment weighting pattern information FF and the gradation characteristic instruction information mm is normalized respectively. It is conceivable that the larger one is used as the weighting pattern as it is. With this, it is possible to control the gradation of the subject (attention) position to be appropriate while corresponding to the shooting environment.

Although the second embodiment has been described above, the various applications described in the first embodiment are also possible in this embodiment. Further, in addition to the above, as the form of map information, for example, image information of a still image is held for each fixed position (distance), or image information is held based on landmarks as described above. You can also Further, as a method of storing the environment map information DD in the map information memory 62, the video information obtained by once taking a picture of the photographing environment may be expanded and stored in the memory.
Further, the memory may be updated (rewritten) sequentially according to changes in the environment due to the growth of the human body and the layout change in the building.

(Third Embodiment) FIG. 17 is a block diagram showing the structure of a video information processing circuit according to the third embodiment of the present invention. Here, components having the same functions as those in the first embodiment are designated by the same reference numerals, and only different portions will be described. Note that, in the present embodiment, similar to the first embodiment, wide DR image information is generated in the wide DR image capturing mode in which image information for two screens with different exposures is obtained from the image sensor 1 in one field image capturing.

In the third embodiment, focusing information, photometric information, etc. are used in searching for a subject and estimating a shooting condition.
The subject position of interest in the image is set based on the information required when the image is captured by the video camera. Further, the gradation characteristic generation circuit 104 controls the generation of the gradation characteristic using the color difference information cc from the luminance / color difference information separation circuit 23.

In the object retrieval processing circuit 101, the CPU 8
Wide DR video information bb output from the wide DR still image video information generation circuit 21 or the wide DR video image information generation circuit 22 and A according to the moving image / still image shooting mode information nn from
CPU receives the information from F, AE, AWB detection circuit 7
Upon receiving the focus / photometric information GG generated and output by
Attention information ii indicating information about a subject of interest and exposure change control information pp for instructing change of exposure at the time of shooting are generated and output. Here, the focusing / photometric information GG is AF in the current shooting when the exposure change control information pp is generated,
It is used as information indicating AE. That is, the exposure change control information pp is generated as information for controlling how to change the current AF and AE. The focusing / photometric information GG in the present embodiment may be information relating to one of the two types of exposure images.

The photographing situation estimation circuit 102 receives the attention information ii from the subject search processing circuit 101 and the focus / photometry information GG output from the CPU 8, and generates photographing situation information kk representing the estimation result regarding the photographing situation. And output. Here, focusing / photometric information G
The shooting situation is classified into a plurality of patterns based on G, and the patterns are output as shooting situation information kk.

The gradation characteristic change control circuit 103 receives the photographing condition information kk output from the photographing condition estimating circuit 102 and the still image gradation characteristic instruction information hh output from the wide DR still image video information generating circuit 21. , And generates and outputs gradation characteristic instruction information mm that represents information for instructing to change the gradation characteristic or instructing to maintain the previously generated gradation characteristic. Here, the gradation characteristic instruction information mm is generated according to the change of the shooting condition information kk. That is, when there is no change in the shooting condition information kk (no change in focus / photometry information GG), the gradation characteristic instruction information mm representing the content for maintaining the current gradation characteristic is generated, When there is a change in the shooting condition information kk (a change in the focus / photometry information GG is found), the gradation characteristic instruction information mm indicating the content for instructing the change of the gradation characteristic is generated.

In the gradation characteristic generation circuit 104, the luminance information dd and the color difference information cc outputted from the luminance / color difference information separation circuit 23, the edge information ee outputted from the edge detection circuit 24, and the gradation characteristic change control circuit. Upon receiving the gradation characteristic instruction information mm output from 103, the gradation characteristic information ff indicating the gradation characteristic is generated and output. Here, the color difference information cc controls the generation of gradation characteristics so as to limit the allocation of gradation characteristics. For example, a change in the gradation of the luminance information dd (gradient of gradation characteristics) relating to a portion indicating a skin color is checked by the color difference information cc, and when this gradient is equal to or greater than a predetermined value, the control is performed to reduce the value to a predetermined value. In this way, too many gradations are assigned so that the contrast of the skin color part does not become stronger than necessary, and the gradation reproduction that makes the gradation reproducibility of the subject of the person making up the skin color part more natural. Characteristic information
generate ff.

FIG. 18 is a photograph of the third embodiment.
Evaluation for setting the photometric information used to determine the situation
It is a figure which shows the example of a photometric division pattern. Here the screen
A ₁~ A_ThirteenIt is divided into 13 photometric areas of
From area photometric information such as brightness measured by
The photometric evaluation values S1 to S3 shown are calculated.

Equation 3] _{S1 = | A 2 -A 3 |} ··· (3) S2 = max (| A 4 -A 6 |, | A 4 -A 7 |) ··· (4) S3 = max (A ₁₀ , A ₁₁ ) −ΣA _i / 13 (5)

Here, the expression (3) evaluates whether there is only one subject or a plurality of subjects at the center of the screen when close-up photography is performed as in close-up, and (4)
The formula is when you are shooting a person like a portrait,
Evaluating whether one or more subjects are present in the central part of the screen, the expression (5) evaluates whether or not there is a sky at the top of the screen when the entire screen is photographed when shooting the entire landscape. . Then, these evaluation values are set as photometric information.

FIG. 19 is a diagram showing an example of a classification pattern of shooting conditions based on focus information and photometric information in the third embodiment. Here, AF information for estimating the distance to the subject is used as the focus information. In the present embodiment, the shooting situation of the image information is classified into six patterns (shooting situation type) by the combination of the focus information and the photometric information. Each shooting situation type is defined as follows. -Type 1: Focus information is 5 m to ∞ (landscape shooting) and photometric information S3 is at or above the threshold Th21 (the sky is present at the top of the screen) -Type 2: Focus information is 5 m to ∞ (landscape shooting) If the photometric information S3 is less than the threshold Th21 (there is no sky at the top of the screen, or there is little sky) -Type3: Focus information is 1 m to 5 m (photographing person) and photometric information S2 is the threshold Th22. If it is above (portrait photographing of one person) -Type 4: Focus information is 1 m to 5 m (portrait photographing) and photometric information S2 is less than the threshold Th22 (multiple portrait photographing) -Type 5: When the focus information is 1 m or less (close-up shooting) and the photometric information S1 is the threshold Th23 or more (a close-up of a single object) ・ Type6: Focus information is 1 m or less (close-up shooting) and the photometric information S1 is the threshold value If it is less than Th23 ( It is a close-up of multiple objects)

FIG. 20 is a diagram showing an example of a shooting situation pattern based on the shooting situation type shown in FIG. Type
Reference numeral 1 is a shooting situation pattern when it is determined to be "a landscape with a sky above", and attention is paid to a portion other than the sky. Type 2 is a shooting situation pattern when it is determined that there is "no sky at the top or a landscape with few sky parts", and the entire screen is focused. Type 3 is a shooting situation pattern when it is determined that "one person is shooting portrait", and the upper part of the screen center is focused more than other parts. Type 4 is a shooting situation pattern when it is determined that “portrait shooting of a plurality of persons” is performed, and attention is paid to the center of the screen and the left and right sides of the center as compared with Type 3. Type5
Is a shooting situation pattern when it is determined that “a single object is taken up in close-up”, and attention is paid to the central part of the screen rather than the peripheral part. Typ
e6 is a shooting situation pattern when it is determined that “a plurality of objects are being taken in close-up shooting”.
Compared to e5, the degree of attention at the center of the screen is reduced.

In the photographing situation pattern of FIG. 20 determined in this embodiment, the degree of attention is numerically changed according to the position on the screen. Therefore, it is possible to use the shooting situation pattern itself as a weighting pattern in the subsequent generation of the gradation conversion characteristic.

Although the third embodiment has been described above, the present embodiment can be applied to various applications. For example,
In the present embodiment, the focus information and the photometric information are used as the information required when capturing an image with the video camera, but the present invention is not limited to this, and zoom position information, multi-spot photometric information, and line-of-sight input information It can be realized by using one or more of them.

(Fourth Embodiment) FIG. 21 is a block diagram showing the basic arrangement of a video camera as an image pickup system according to the fourth embodiment of the present invention. After that, the 4th
In the embodiments, those having the same functions as those in the second embodiment are designated by the same reference numerals, and only different points will be described. Note that, in the fourth embodiment, as in the second embodiment, an image sensor that can acquire video information with a wide DR in one shot by video input is used, and a video is generated using map information related to the shooting environment. To do.

In this embodiment, the monitor 13 for displaying an image is provided with the monitor input driver 112 for controlling the input of the screen position by the touch panel system, and the screen position input information HH from the monitor input driver 112 is provided.
Upon receiving (not shown), the image information processing circuit 111 performs gradation reproduction processing to generate converted image information gg.

FIG. 22 is a block diagram showing the structure of the video information processing circuit 111 shown in FIG. In this video information processing circuit 111, the screen position input information HH output from the monitor input driver 112 is changed to the wide DR video information as a difference from the configuration of the video information processing circuit 61 in the second embodiment shown in FIG. bb and environmental map information DD
It is also supplied to the subject search processing circuit 121 and subject search processing is performed by the method described later. The attention information output from the subject search processing circuit 121
ii, exposure change control information pp, and shooting position information EE have the same contents as in the second embodiment.

FIG. 23 is a block diagram showing the structure of the subject search processing circuit 121 shown in FIG. here,
Only parts different from the subject search processing circuit 73 in the second embodiment shown in FIG. 15 will be described.

The screen position input information HH output from the monitor input driver 112 is supplied to the subject presence / absence determining circuit 131 together with the image feature information ww, the wide DR image information bb, and the photographing position information EE to determine the presence / absence of a subject. To do.
Here, the presence / absence of a subject is determined from the wide DR video information bb and the video characteristic information ww around the screen position indicated by the screen position input information HH. The subject presence / absence information xx generated and output by the subject presence / absence determination circuit 131 has the same contents as in the second embodiment.

The screen position input information HH includes subject presence / absence information xx (information indicating that "the subject is present"), general attention information yy (information indicating that the entire screen is the subject), and photographing. The position information EE is supplied to the subject position detection circuit 132 to set the position of the subject on the screen. Here, when the subject presence / absence information xx is received, the screen position input information HH itself is set as the subject position, and when the overall attention information yy is received, the entire screen is set as the subject position, and as a result Shooting position information
The subject position information zz including EE is generated and output.

The attention information setting circuit 85 receives the subject position information zz output from the subject position detection circuit 83 and the wide DR video information bb and receives the subject position on the screen or the map information indicated by the subject position information zz, That is, the attention information ii is generated and output from the screen position indicated by the screen position input information HH. As a result, the attention information for the screen position designated by the input from the monitor input driver 112 can be obtained, and as a result, the gradation characteristics corresponding to the screen position are generated and applied to the video information to input. It is possible to generate image information that provides appropriate gradation reproduction for different screen positions.

[Brief description of drawings]

FIG. 1 is a block diagram showing a basic configuration of a video camera as an imaging system according to a first embodiment of the present invention.

2 is a block diagram showing a configuration of a video information processing circuit shown in FIG.

FIG. 3 is a diagram illustrating a method of generating a wide dynamic range (DR) image according to the first embodiment.

FIG. 4 is a block diagram showing a configuration of a wide DR still image video information generation circuit shown in FIG.

FIG. 5 is also a diagram for explaining processing in the wide DR still image video information generation circuit.

6 is a block diagram showing a configuration of a subject search processing circuit shown in FIG.

7 is a flowchart of an exposure change algorithm in the exposure change control circuit shown in FIG.

FIG. 8 is also a diagram for explaining a subject position (attention position) setting operation by the subject position setting circuit.

FIG. 9 is a flowchart showing the operations of the shooting condition estimation circuit and the gradation characteristic change control circuit shown in FIG. 2.

10 is a block diagram showing a configuration of a gradation characteristic generation circuit shown in FIG.

FIG. 11 is also a diagram for explaining the operation of the gradation characteristic generation circuit.

12 is a diagram showing limit characteristics of color difference information used in the image generation processing in the image generation circuit shown in FIG.

FIG. 13 is a block diagram showing a basic configuration of a video camera as an image pickup system according to a second embodiment of the present invention.

14 is a block diagram showing a configuration of the video information processing circuit shown in FIG.

15 is a block diagram showing a configuration of a subject search processing circuit shown in FIG.

16 is a block diagram showing a configuration of a gradation characteristic generation circuit shown in FIG.

FIG. 17 is a block diagram showing a configuration of a video information processing circuit according to a third embodiment of the present invention.

FIG. 18 is a diagram showing an example of an evaluation photometric division pattern for setting photometric information used to determine a shooting situation in the third embodiment.

FIG. 19 is a diagram showing an example of a classification pattern of shooting conditions based on focus information and photometric information in the third embodiment.

20 is a diagram showing an example of a shooting situation pattern based on the shooting situation type shown in FIG.

FIG. 21 is a block diagram showing a basic configuration of a video camera as an imaging system according to a fourth embodiment of the present invention.

22 is a block diagram showing a configuration of the video information processing circuit shown in FIG.

FIG. 23 is a block diagram showing the configuration of the subject search processing circuit shown in FIG. 22.

[Explanation of symbols]

1 Image sensor 2 lens 3 Aperture / shutter mechanism 4 amplifier 5 A / D converter 6 Video information processing circuit 7 AF, AE, AWB detection circuit 8 CPU 9 compression circuit 10 memory controller 11 DRAM 12 Display circuit 13 monitors 14 Recording medium I / F for still images 15 Recording media for still images 16 Video recording medium I / F 17 Video recording media 18 Timing generator (TG) 19 Strobe 20 Input key 21 Wide DR still image information generation circuit 22 Wide DR video image information generation circuit 23 Luminance / color difference information separation circuit 24 Edge detection circuit 25 gradation characteristic generation circuit 26 Video generation circuit 27 Subject Search Processing Circuit 28 Shooting situation estimation circuit 29 gradation characteristic change control circuit 31 Wide DR still image synthesis circuit 32 level detection circuit 33 First switch 34 Second switch 35 Continuous shooting control circuit 36 Continuous shooting image position correction circuit 37 Reference image estimation circuit 38 Reference image correction circuit 39 Tone characteristic change instruction output circuit 41 Video feature extraction circuit 42 Subject Presence / Absence Determination Circuit 43 Subject position setting circuit 44 Exposure change control circuit 45 Attention information setting circuit 51 Weighting pattern setting circuit 52 Edge Histogram Calculation Circuit 53 gradation characteristic calculation circuit 61 Video information processing circuit 62 Map information memory 71 switch 72 Still image video information generation circuit 73 Subject Search Processing Circuit 74 gradation characteristic generation circuit 81 Shooting position estimation circuit 82 Subject presence / absence determination circuit 83 Subject position setting circuit 84 Exposure change control circuit 85 Attention information setting circuit 91 Weighting setting circuit for shooting environment 92 Weighting pattern setting circuit 101 Subject Search Processing Circuit 102 Shooting situation estimation circuit 103 gradation characteristic change control circuit 104 gradation characteristic generation circuit 111 Video information processing circuit 112 Monitor input driver 121 Subject Search Processing Circuit 131 Subject existence determination circuit 132 Subject position setting circuit

Continued front page    F term (reference) 5B057 AA20 CA01 CA08 CA12 CB01                       CB08 CB12 CC01 CE08 CE11                       CE16 CH01 CH11 CH18 DA16                       DB02 DB06 DB09                 5C022 AA11 AB17 AB22 AC69                 5C065 AA01 AA03 BB48 CC02 CC03                       DD02 GG32                 5C066 AA01 CA21 GA02 GA05 GB01                       KE17

Claims (24)

[Claims] 1. In an imaging system capable of handling video as a moving image, a shooting switching unit for selecting moving image shooting or still image shooting, and a moving image if still image shooting is selected by the shooting switching unit. A still image generation unit that generates video information of a still image from the video information acquired as, and a shooting situation based on the video information acquired as a moving image or the video information of the still image generated by the still image generation unit. A shooting condition estimation unit for estimating, a gradation characteristic change control unit for controlling a change in gradation property of video information based on the shooting condition estimated by the shooting condition estimation unit, and a control for the gradation characteristic change control unit. And a video generation unit that generates reproduction video information based on the gradation characteristics of the video information. Imaging system to be. 2. In an imaging system capable of handling video as a moving image, a shooting switching unit for selecting moving image shooting or still image shooting, and a predetermined time when moving image shooting is selected by the shooting switching unit. In the above, when a still image shooting is selected by the shooting switching unit, a moving image generation unit that synthesizes images with different exposures in a certain time unit to generate wide dynamic range image information of a moving image, and images with different exposures are displayed. A still image generation unit that synthesizes and generates wide dynamic range video information of a still image, and video information of the moving image generated by the moving image generation unit or video information of the still image generated by the still image generation unit. A shooting situation estimation unit that estimates the shooting situation based on the shooting situation, and a change in the gradation characteristics of the video information based on the shooting situation estimated by the shooting situation estimation unit. A gradation characteristic change control unit, a gradation characteristic generation unit that generates the gradation characteristic of the video information under the control of the gradation characteristic change control unit, and a reproduced video based on the gradation characteristic of the video information. An image pickup system, comprising: a video generation unit that generates information. 3. The image capturing situation estimating unit extracts a feature extracting unit that extracts a feature from the image information, and information about a subject of interest in the image information based on the feature extracted by the feature extracting unit. The imaging system according to claim 1 or 2, further comprising: an attention information setting unit to be set. 4. The image capturing status estimating unit sets an image display unit for displaying the image information, and information about a subject of interest in the image information based on the image information displayed on the image display unit. 3. The imaging system according to claim 1 or 2, further comprising: 5. The shooting condition estimation unit estimates a shooting environment of the video information based on the knowledge information storage unit that stores knowledge information about a shooting environment, and the knowledge information in the knowledge information storage unit. The image capturing apparatus according to claim 1 or 2, further comprising: a section, and an attention information setting section that sets information regarding a subject of interest in the video information based on an estimation result of the imaging environment estimating section. system. 6. The shooting situation estimation unit estimates a shooting environment of the video information based on the knowledge information storage unit that stores knowledge information about the shooting environment and the knowledge information in the knowledge information storage unit. Section, and a gradation characteristic control section that generates control information for controlling the gradation characteristic of the video information based on the estimation result of the photographing environment estimating section and outputs the control information as the estimation result of the photographing situation. The imaging system according to claim 1 or 2, characterized in that. 7. The attention information setting unit determines whether or not the subject of interest exists in the image information, and the image presence / absence determination unit determines whether or not the image exists. A level detection unit that detects a portion where the information level is equal to or higher than a first predetermined value or equal to or lower than a second predetermined value that is smaller than the first predetermined value; and as a result of the level detection unit, the level of the video information is the first predetermined value. When detecting the above part, it controls so as to reduce the exposure when acquiring the video information,
When a portion where the level of the image information is equal to or lower than the second predetermined value is detected, an exposure control unit that controls to increase exposure when acquiring the image information, and an object exists in the object presence / absence determination unit The imaging system according to claim 3, 4 or 5, further comprising: an attention position setting unit that sets a portion corresponding to the subject as an attention portion in the case of performing. 8. The attention information setting unit includes at least the luminance level information of the subject as the information about the subject to be set, and the luminance of the subject of interest is darker than a predetermined range from the luminance level information. Control to increase the exposure when acquiring the video information, and control to reduce the exposure when acquiring the video information when the brightness of the subject of interest is on the brighter side than the predetermined range. The imaging system according to claim 3, 4 or 5, further comprising an exposure control unit that controls the exposure. 9. The wide dynamic range, wherein the exposure control unit changes the exposure for one or more images among the images with different exposures in the fixed time unit, when changing the exposure for acquiring the image information. 8. The method for controlling generation of video information.
Or the imaging system according to 8. 10. The estimation result generated by the photographing environment estimation unit is information corresponding to the video information among the knowledge information stored in the knowledge information storage unit. The imaging system according to 5 or 6. 11. The attention information setting unit compares the estimation result output from the photographing environment estimation unit with the video information and performs processing on a portion where a difference level is a predetermined value or more. The imaging system according to claim 5. 12. The shooting condition estimation unit estimates the shooting condition based on a focus position estimated from focus information at the time of shooting and a subject brightness distribution estimated from photometric information at the time of shooting. The imaging system according to claim 1 or 2. 13. The still image generating section is characterized in that on the screen, a level of the video information or the reproduced video information is equal to or higher than a third predetermined value or smaller than a third predetermined value.
A still image level detection unit that detects a portion that is less than or equal to a predetermined value, and a sequence that continuously acquires video information or playback video information as still image information when the portion detected by the still image level detection unit exists. A copying unit, a position correction unit that performs on-screen alignment of the still image information group acquired by the continuous shooting unit, and the still image information group corrected by the position correction unit are combined to form one still image. The image pickup system according to claim 1, further comprising: a still image synthesizing unit that generates information. 14. The still image synthesizing unit has a reference image designation unit that designates still image information serving as a reference in the still image information group, and the level of the still image information designated by the reference image designation unit is the above level. A reference image correction unit that corrects a portion having a third predetermined value or more or the fourth predetermined value or less by using the still image information group other than the designated still image information. Item 13. The imaging system according to item 13. 15. The gradation characteristic change control unit outputs change control information for controlling the gradation characteristic to be changed when a change in a shooting condition occurs in the video information. Or the imaging system according to 2. 16. The gradation characteristic change control unit outputs change control information for controlling to change the gradation characteristic when still image shooting is selected by the shooting switching unit. The imaging system according to 1 or 2. 17. The gradation characteristic change control unit calculates an evaluation value representing a temporal change of information on the subject of interest, and if the evaluation value determines that the temporal change is large, the gradation characteristic is changed. The image pickup system according to claim 1, wherein the image pickup system outputs change control information for controlling to change the characteristic. 18. The gradation characteristic generating unit, when the subject of interest is detected by the photographing condition estimating unit, weights a portion of the image information corresponding to the subject of interest to a greater weight than other portions. If a subject of interest is not detected by the shooting situation estimation unit, a weighting pattern that sets uniform weight to the entire screen of the video information is set and set. The image pickup system according to claim 1, wherein the weighting pattern is used to generate gradation characteristics. 19. The gradation characteristic generation unit detects a characteristic for gradation characteristic for generating a gradation characteristic from the video information, and utilizes the characteristic for gradation characteristic and the weighting pattern. The imaging system according to claim 18, wherein a histogram regarding information is created, and gradation characteristics are generated based on the histogram. 20. The gradation characteristic generation unit is a gradation corresponding to a subject of interest when the ratio of the information regarding the subject of interest set in the attention information setting unit on the screen is equal to or more than a predetermined value. 4. The gradation characteristic is generated by limiting the above-mentioned gradation characteristics.
The imaging system according to 4 or 5. 21. The gradation characteristic generation unit generates the gradation characteristic based on the knowledge information in the knowledge information storage unit corresponding to the estimation result of the photographing condition estimation unit. The imaging system according to 6. 22. The image generation unit converts the luminance information of the image information based on the gradation characteristic generated by the gradation characteristic generation unit, and the luminance information and color before and after the conversion by the gradation characteristic. 3. The imaging system according to claim 1, wherein the color difference information of the video information is converted based on a theoretical limit characteristic of reproduction, and the reproduced video information is generated from the converted luminance information and color difference information. 23. A computer for controlling the operation of an imaging system capable of handling a video as a moving image, and a shooting switching function for selecting moving image shooting or still image shooting, and still image shooting by the shooting switching function. When selected, the still image generation function that generates the video information of the still image from the video information that is acquired as the moving image, and the video information that was acquired as the moving image or the still image generated by the above still image generation function. A shooting condition estimation function for estimating a shooting condition based on the video information; a gradation characteristic change control function for controlling a change in the gradation property of the video information based on the shooting condition estimated by the shooting condition estimation function; Based on the gradation characteristic generation function for generating the gradation characteristic of the video information according to the control by the gradation characteristic change control function and the gradation characteristic of the video information And a video generation function for generating reproduction video information.
Imaging program. 24. A computer for controlling the operation of an imaging system capable of handling video as a moving image, and a shooting switching function for selecting video shooting or still image shooting, and video shooting selected by the shooting switching function. If
A moving image generation function that generates a wide dynamic range image information of a moving image by synthesizing images with different exposures in a fixed time unit at a predetermined time, and a different exposure when still image shooting is selected by the shooting switching function. Of a still image generated by the above-mentioned moving image generation function or a still image generated by the above-mentioned moving image generation function, and a still image generated by the above moving image generation function. A shooting condition estimation function for estimating a shooting condition based on the video information; a gradation characteristic change control function for controlling a change in the gradation property of the video information based on the shooting condition estimated by the shooting condition estimation function; A gradation characteristic generation function for generating the gradation characteristic of the video information according to the control by the gradation characteristic change control function, and a gradation characteristic of the video information Based on imaging program characterized thereby realizing a video generation function of generating a reproduction video information.