JP4139432B1 - Imaging method, imaging apparatus, and image information processing apparatus - Google Patents

Abstract

An object of the present invention is to realize an imaging method capable of shortening the time required for image processing when the low resolution image or the high compression rate image is photographed and shortening the time until the next photographing is possible. In an imaging method for setting a resolution of a photographed image as a condition for creating a photographed image and creating the photographed image, an image formed by an imaging means on an image sensor having a color filter for color separation. When transferring the charges of all the pixels accumulated according to the above to generate a captured image, the charge data transfer is divided into M (≧ 3) fields, and color signals for all colors of the color filter are obtained. It is determined whether or not transfer data for m (<M) fields in which the required number of pixels and the required number of pixels are aligned. Do Number of pixels necessary to create an image with a set resolution: When transfer data for n (m ≦ n ≦ M) fields are prepared, image processing is performed using the transfer data for n fields, and the captured image Create
[Selection] Figure 6

Description

  The present invention relates to an imaging method, an imaging apparatus, and an image information processing apparatus.

  An imaging device that transfers the charges of all the pixels accumulated according to the image formed by the imaging unit to an imaging device having a color filter for color separation, and performs predetermined image processing to create an image. It is known as an electronic still camera or the like, and is being incorporated as a part of the functions of a portable information terminal such as a mobile phone or a computer device.

  In addition, in order to meet the demand for an increase in the number of pixels and downsizing, the image pickup element “interlace transfer type image pickup in which charge data transfer is divided into a plurality of fields when transferring the charges of all accumulated pixels”. "Elements" are becoming common.

  For example, in a color electronic still camera using such an interlaced transfer type image sensor, the color filters for color separation are of the primary colors of red (R), green (G), and blue (B). Information contained in one field is red (R) / green (G) information or blue (B) / green (G) information, and information necessary to create a color image is obtained in one field. I can't.

  Conventionally, this type of electronic still camera does not perform any image processing until the transfer of all fields is completed, in other words, until the transfer of all charges accumulated in the image sensor is completed (patented). Reference 1), for example, has a problem that “time required to display a photographed confirmation image” becomes long.

  Further, in the electronic still camera as described above, since all charge information is subjected to image processing, the image processing time requires a certain time regardless of the set resolution and compression rate. Even when shooting an image that is originally sufficient with a small amount of image information, such as an image for low resolution or a high compression ratio, there is a problem that it takes time until the next shooting can be performed and that it is not easy to use. It was.

JP2001-285688

In light of the above-described circumstances, the present invention can shorten the time required for image processing when shooting a low-resolution image or a high compression rate image. It is an object of the present invention to realize an imaging method capable of shortening the time until the first imaging becomes possible.
Another object of the present invention is to realize an imaging apparatus for carrying out the imaging method and an image information processing apparatus having at least a part of the imaging apparatus.

The imaging method of the present invention is “an imaging method for setting a resolution of a captured image as a condition for creating a captured image and creating a captured image”.
The imaging method according to claim 1 has the following characteristics (claim 1).
In other words, when the “charges of all pixels accumulated in accordance with the image formed by the image forming means on the image pickup device having a color filter for color separation” are transferred in order to create a photographic image, Whether or not transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned is prepared. Make a decision.
Then, when “determined to be complete”, a “photographing confirmation image” is created using transfer data for m fields, and the number of pixels necessary for creating an image with a set resolution: n (m ≦ n ≦ M ) When the transfer data for the field is prepared, image processing is performed using the transfer data for n fields, and the captured image is created . Then, the aspect ratio is adjusted at the time of creating the shooting confirmation image, and “when the shot image is created using the transfer data of the number of fields: n (<M)” is created by adjusting the aspect ratio. .

The imaging method according to claim 2 has the following characteristics.
In other words, when the “charges of all pixels accumulated in accordance with the image formed by the image forming means on the image pickup device having a color filter for color separation” are transferred in order to create a photographic image, Is divided into M (≧ 3) fields, and determination is made as to whether transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are the same have been prepared. I do.
Then, when it is determined that the images have been prepared, a “thumbnail image” is created using transfer data for m fields, and the number of pixels necessary for creating an image with a set resolution: n (m ≦ n ≦ M) field When the transfer data for the same amount are prepared, image processing is performed using the transfer data for n fields to create an image to be created.
Then, when the thumbnail image is created, the aspect ratio is adjusted, and when the image to be created is created using transfer data with the number of fields: n (<M), the aspect ratio is adjusted and created .
In this case, a thumbnail image can be created together with the shooting confirmation image, or only a thumbnail image can be created.

3. The imaging method according to claim 1, wherein the number of fields: M is an odd number of 3 or more and m = 1 (Claim 3), and the number of fields: M is an even number of 4 or more, m = 2 (claim 4).
The imaging method according to claim 1 or 2, wherein M = 3, a low-resolution image is created for the number of fields: n = 1, and a medium-resolution image is created for the number of fields: n = 2. In addition, a high-resolution image can be created for the number of fields: n = M (Claim 5), and a low-resolution image is created for the number of fields: n = 2 at M = 4. It is also possible to create a medium-resolution image and / or a high-resolution image for the number of fields: n = M (Claim 6), and further, for M = 6, for the number of fields: n = M A high-resolution image can be created, and a low-resolution image can be created for the number of fields: n = 2, or a medium-resolution or low-resolution image can be created for the number of fields: n = 4 ( Claim 7).

An imaging apparatus according to an eighth aspect includes a setting unit, an imaging element, an imaging unit, an imaging processing unit, and an image processing unit.
“Setting means” is means for setting the resolution of an image as a condition for creating a captured image.

The “imaging device” has a color filter for color separation.
The “imaging unit” is a unit that causes the image sensor to form an image to be captured.
The “imaging processing unit” performs an imaging process for converting the electric charge accumulated and transferred in the imaging element into a digital signal.

The “image processing means” performs image processing using transfer data transferred via the imaging processing unit.
The image sensor then captures the interlaced transfer image by dividing the charge data transfer into M (≧ 3) fields when “transferring the accumulated charges of all the pixels to create a captured image”. The image processing means performs “determination as to whether transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned” has been prepared ”. In this case, a shooting confirmation image is created by the image processing means using the transfer data for m fields, and the number of pixels necessary for creating an image with a set resolution: n (m ≦ n ≦ M) fields When transfer data is prepared, image processing is performed using the transfer data for n fields to generate an image to be generated.
The image processing means also adjusts the aspect ratio when creating the shooting confirmation image, and adjusts the aspect ratio when the image to be created is created using transfer data of the number of fields: n (<M). Create .

An imaging apparatus according to a ninth aspect includes a setting unit, an imaging element, an imaging unit, an imaging processing unit, and an image processing unit.
[Setting means] is means for setting the resolution of an image as a condition for creating a captured image.

The “imaging device” has a color filter for color separation.
[Image forming means] is means for forming an image to be imaged on the image sensor.
The “imaging processing unit” performs an imaging process for converting the electric charge accumulated and transferred in the imaging element into a digital signal.

The [image processing means] performs image processing using transfer data transferred via the imaging processing unit.
The image sensor then captures the interlaced transfer image by dividing the charge data transfer into M (≧ 3) fields when “transferring the accumulated charges of all the pixels to create a captured image”. The image processing means performs “determination as to whether transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned” has been prepared ”. In this case, thumbnail images are created by image processing means using transfer data for m fields, and the number of pixels necessary for creating an image with a set resolution: transfer for n (m ≦ n ≦ M) fields. When the data is ready, image processing is performed using the transfer data of the n field sentence, and an image to be created is created.
The image processing means also adjusts the aspect ratio when creating a thumbnail image, and adjusts the aspect ratio when creating an image to be created using transfer data with the number of fields: n (<M). To do.

  The “color filter of the imaging device” in the imaging apparatus according to claim 8 or 9 can be “primary color system” of red (R), green (G), and blue (B). ), Or “complementary colors” of yellow (Y), cyan (C), magenta (M), and green (G) (claim 11).

  The imaging apparatus according to claim 8, 9, 10, or 11, “M in the interlace transfer type imaging device is 3, and a low-resolution image is created for the number of fields: n = 1, and the number of fields: n Can create a medium-resolution image for = 2 and a high-resolution image for the number of fields: n = M (claim 12). “M is 4 in the element, and a low resolution image is created for the number of fields: n = 2, and a medium resolution image and / or a high resolution image is created for the number of fields: n = M” Further, “M in the interlaced transfer type imaging device is 6 and a high-resolution image is created for the number of fields: n = M, and the number of fields: n = Vs 2 To create an image of low resolution Te or the number of fields: creating an image of medium resolution or low resolution with respect to n = 4 may be "those (claim 14).

The image pickup apparatus according to any one of claims 8 to 14 can "detach and attach an external storage unit that stores a created image" (claim 15), and any one of claims 8 to 15. The imaging apparatus described in (1) may have an “image holding area for storing a created image”.
The image holding area can also be provided in the “imaging device to which the external storage unit can be attached and detached”.

  The image information processing apparatus according to the present invention includes at least part of the imaging device according to claims 8 to 16 (claim 17). That is, the image information processing apparatus can be the imaging apparatus itself, or a more general apparatus in which this is incorporated. As an image information processing apparatus which is the image pickup device itself, an “electronic still camera” can be cited.

  Examples of the image information processing apparatus in which the imaging apparatus is incorporated in a part of the apparatus include a “portable information device such as a portable computer or a mobile phone” in which the imaging apparatus is incorporated, and a monitoring camera. Can do. An image created by image processing by the image sensor can be transmitted as image information.

  The “imaging means” in the above is generally an imaging optical system (such as a lens system, an imaging mirror system, or a combined system of a lens system and a mirror system), and a shutter that passes and blocks the imaging light flux with respect to the image sensor. And a drive unit for driving opening and closing of the shutter.

  The image processing performed by the image processing means may include “at least YUV conversion (image processing for converting transferred charge data into a YUV signal (luminance color difference signal)”).

  Further, although the upper limit of M depends on the manufacturing conditions of the image sensor, it is considered to be about 10, and M = 3, 4, 5, and 6 are practically considered.

  For example, when the imaging method of the present invention is applied to an electronic still camera, “continuous shooting” may be performed in the electronic still camera. In such a case, the data of information by continuous shooting cannot be combined as raw data. After storing, image processing can be performed using all color signals and the required number of pixels (less than the total number of pixels).

  The “imaging device” may include a display unit that displays an imaging confirmation image and / or a thumbnail image. The display unit is not provided in the imaging apparatus, and appropriate external display means (computer display or the like) can be used.

  As described above, according to the imaging method of the present invention, when transferring the charge accumulated in the imaging device, image data can be created without waiting for the transfer of data of all pixels. The processing time until confirmation and the start of the next shooting can be shortened. Therefore, the imaging apparatus and the image information processing apparatus that implement the imaging method of the present invention are convenient for the user.

  Embodiments of the invention will be described below.

  FIG. 1 is a block diagram showing an outline of a system of an “electronic still camera (usually called a digital camera)” as an embodiment of an imaging apparatus.

  An “imaging device” denoted by reference numeral 1 has a color filter for color separation. Specifically, it is a CCD using a color filter for color separation.

  The imaging lens group indicated by reference numeral 01, the shutter 02, and the drive unit 03 constitute "imaging means" that forms an "image to be imaged" on the imaging element 1. The driving unit 03 is controlled by the control unit 5 to drive the opening and closing of the shutter 02 and to displace and drive the lens position of the imaging lens group in accordance with autofocus (AF) control and camera shake correction control.

  An “imaging processing unit” indicated by reference numeral 2 performs an imaging process for converting the charges accumulated and transferred in the imaging device 1 into a digital signal. The imaging processing unit 2 includes a CDS (correlated double sampling circuit) 2a, an AGC (automatic gain control circuit) 2b, an A / D (analog / digital conversion circuit) 2c, a TG (timing generation circuit), and the like.

  An “image processing unit” denoted by reference numeral 3 includes a CCD-I / F (CCD interface) 3a, a memory controller 3b, a YUV conversion unit 3c, a compression processing unit 3d, and a display unit I / F (display unit interface) 3e. Have.

  A “storage unit” indicated by reference numeral 4 stores “raw data” necessary for image processing and image-processed data, and includes a RAW-RGB area 4a and a YUV area 4b for storing raw data. JPEG region 4c. The storage in the storage unit 4 is “volatile”.

  In this embodiment, an external storage unit 4 '(specifically, a memory card) can be attached to and detached from the image pickup device body from the outside. The external storage unit 4 ′ stores “various images created by image processing”. Instead of making the external storage unit 4 ′ detachable as described above, or making the external storage unit 4 ′ detachable, an image holding area 4 ″ is provided in the imaging device, and the above-mentioned portion is provided in this portion. Various images may be stored. Storage in the external storage unit and the image holding area is “nonvolatile”.

  A “control unit (CPU)” denoted by reference numeral 5 controls the exchange of data with each unit of the image processing unit 3 and the storage unit 4 via the memory controller 3 b in the image processing unit 3, and controls the entire imaging apparatus. To do. A “display unit” indicated by reference numeral 6 is a part for displaying thumbnail images, shooting confirmation images, and the like. An “operation unit” denoted by reference numeral 7 sets shooting conditions, created image conditions, and the like.

  Image processing means for performing image processing including at least “YUV conversion” using transfer data (digital signal) transferred via the imaging processing unit 2 includes an image processing unit 3, a storage unit 4, and a control unit. 5.

  Incidentally, the “electronic still camera” whose system is shown in FIG. 1 has a form as shown in FIG. 2A is a front view, FIG. 2B is a side view, and FIG. 2C is a top view. Reference numeral 20 denotes a “camera body”, reference numeral 21 denotes an “upper lid” attached to the camera body 20 by a hinge 22 so as to be freely opened and closed, and FIG. 2 shows a state in which the upper lid 21 is opened. Reference numeral 01 in the front view indicates the “imaging lens group” described in FIG. 1, reference numeral 23 indicates “strobe”, and reference numeral 24 indicates “finder”.

  In FIG. 2C, reference numeral 21A indicates a “liquid crystal monitor”, and reference numeral Sw indicates an “upper lid opening / closing detection switch”. That is, the upper lid open / close detection switch Sw is a switch for detecting whether the upper lid 21 is open or closed. Reference B0 indicates “barrier opening knob”.

  A “date switch” indicated by a symbol SD sets the date and time for the electronic still camera. A “frame advance switch” indicated by reference numeral SF is a switch for performing frame advance of a thumbnail image displayed on the liquid crystal monitor 21A, for example. A “recording mode switching switch” denoted by reference numeral SM1 is a switch for switching “recording modes” such as a color shooting mode, a monochrome shooting mode, a single shooting mode, and a continuous shooting mode.

An “image mode switch” indicated by reference numeral SM2 is a switch for setting a captured image size and the like. Reference numeral SP denotes a “display switch” that designates whether or not to display on the liquid crystal monitor 21A, and reference numeral SE denotes an execution switch (enter switch).
Symbol SM indicates a “menu switch”.

  Reference numeral RL denotes a “release switch” for performing a shutter operation for photographing. These various switches constitute the operation unit 7 of FIG.

  In FIG. 1, an image pickup device 1 is an “interlace transfer type image pickup device that performs charge data transfer divided into M (≧ 3) fields when transferring the accumulated charges of all pixels”. As described above, the image sensor 1 has the “color filter for color separation”. As the color filter, the above-mentioned primary color system can be used, and a complementary color system can also be used.

  3A shows a part of a primary color filter, and FIG. 3B shows a part of a complementary color filter. In the primary color filter, R represents a “red filter”, G represents a “green filter”, and B represents a “blue filter”. These three types of filters R, G, and B are two-dimensionally arranged as shown in the figure. . Looking at the arrangement of the filters in the horizontal direction, the filter R and the filter G are alternately arranged in a certain column, and the filter B and the filter G are alternately arranged in another column. Therefore, in the entire color filter, the filter G occupies 1/2 of the total number of filters, and the filter R and the filter B occupy 1/4.

  In the complementary color filter, Y represents a “yellow filter”, C represents a “cyan filter”, M represents a “magenta filter”, and G represents a “green filter”. These four types of filters Y, C, M, and G are illustrated. As shown in FIG. Looking at the arrangement of the filters in the horizontal direction, the filter M and the filter G are alternately arranged in a certain column, and the filter Y and the filter C are arranged alternately in another column. Therefore, each filter occupies 1/4 of the total number of filters.

  As described above, either a primary color type or a complementary color type can be used as the color filter. However, for the sake of concreteness of description, a “primary color type color filter” as shown in FIG. The case where it is used will be described.

  As described above, the image pickup device 1 is an “interlace transfer type image pickup device that performs charge data transfer divided into M (≧ 3) fields when transferring the accumulated charges of all pixels”. is there. First, the case where M = 3 will be described first.

  4A, 4 </ b> B, and 4 </ b> C illustrate an arrangement of “two columns of pixels (light receiving elements)” in the vertical direction (up and down direction in the drawing) of the M = 3 interlace transfer type image sensor. Show. R, Gr, Gb, and B indicate filters arranged corresponding to each pixel. In the horizontal direction, which is the horizontal direction in the figure, the pixels of the filter R and the pixels of the filter Gr are alternately arranged in a certain column, and the pixels of the filter B and the pixels of the filter Gb are alternately arranged in another column. is doing. The filter Gr is “a green filter on the arrangement line of the filter R”, and the filter Gb is “a green filter on the arrangement line of the filter B”.

  For example, in FIG. 4A, the pixel arrangement in the vertical direction (vertical direction) in the drawing is divided into three groups. That is, numbers 1, 2, and 3 are repeatedly written on the left side of the pixel array in the vertical direction, and one of the numbers represents one column of the pixel array in the horizontal direction. One column of such a pixel array in the horizontal direction is a “field”. These fields are represented by numbers written on the left side of the pixel, and are called a first field, a second field, and a third field.

  As shown in FIG. 4A, the pixel array in the vertical direction of the image sensor is a repetition of the first field, the second field, and the third field. Therefore, the first, second, and third fields constituting one cycle of field repetition are collectively referred to as a “field group”.

In FIG. 4, the transfer of charges accumulated in all pixels of the image sensor is performed as follows.
That is, when the transfer is started, first, the charges of the pixels belonging to the first field in each field group are transferred. FIG. 4A shows this state as an explanatory diagram. This transfer is referred to as “primary transfer (charges of pixels in the first field of each field group are transferred)” for convenience.

  When the primary transfer ends, next, the charges of the pixels belonging to the second field in each field group are transferred. FIG. 4B shows this state, and this transfer is called “secondary transfer”. When the secondary transfer is completed, “tertiary transfer” is performed in which the charges of the pixels belonging to the third field in each field group are transferred (FIG. 4C).

  Therefore, at the time when the tertiary transfer is completed, the “transfer of all charges” is completed. Note that data transferred by “same order transfer” in the above-described primary, secondary, and tertiary transfers is referred to as “same order transfer data”.

  Hereinafter, when the image pickup device 1 transfers the accumulated charges of all the pixels, the image data of the interlace transfer method in which the charge data transfer is divided into three fields (first to third fields). Image processing and image creation in the case of “element” will be described with reference to the flowchart of FIG.

  In this embodiment, the resolution of the photographed image can be set to three levels of “low / medium / high” as photographing conditions, and a thumbnail image is always created, and whether or not a photographing confirmation image is created is determined. It can be set selectively.

  Then, “shooting condition setting” is first performed. This setting is performed by operating the operation unit 7 in FIG. More specifically, as shown in FIG. 2, the upper cover 21A is opened with respect to the electronic still camera body 21, and the liquid crystal monitor 21A is brought into a display state by the display switch SP.

  With the menu switch SM, a setting screen as shown in FIG. 5 is displayed on the liquid crystal monitor 21A to perform recording setting. In the example of FIG. 5, “AF (autofocus)” is set as the focus control, and the display of the shooting confirmation image on the liquid crystal monitor 21 </ b> A is “off” or the display time is “1 second” or “3 seconds”. For example, as shown in the figure, “1 second” is set, and the resolution of the recorded image is set to “medium” among “high”, “medium”, and “low”, for example. When the setting is completed in this way, the setting content is confirmed by the execution switch SE. Instead of setting the resolution, a similar display can be performed for the image compression rate, and the setting can be made by user selection.

  Shooting is performed in this state. In the start state in the flowchart of FIG. 6, “Electrification is accumulated in all pixels of the image sensor by photographing”.

  When the transfer is started, the charge data accumulated in the image sensor 1 is divided into three fields according to the image to be photographed, and the primary transfer to the tertiary transfer are sequentially performed. Data to be transferred is “correlated double sampling” by the CDS 2 a in the imaging processing unit 2, automatically gain controlled by the AGC 2 b, converted to a digital signal (digital data) by the A / D 2 c, and the CCD− of the signal processing unit 3. The data is stored in the storage unit 4 through the I / F 3a and the memory controller 3b (S1).

  At that time, when the control unit 5 confirms “completion of data transfer in the first field” (S2), it confirms whether or not “setting of photographing confirmation image display” exists at that time (“Yes” in S2) (S3). . If the shooting conditions are set as shown in FIG. 5, a shooting confirmation image is displayed for 1 second.

  Here, when the state in which the primary transfer is completed is seen, R information (red component information), G information (green component information), and B information (blue component information) are displayed as color information in the first field in successive field groups. When the primary transfer is completed, all the information necessary to create a color image is included in the “data transferred next time” in the primary transfer. It also includes the number of pixels required to create a low resolution color image.

  Accordingly, image processing can be performed at this stage.

  When the “setting for displaying the shooting confirmation image” is confirmed in the processing S3 (“Yes” in S3), an image for creating the “shooting confirmation image” based on the transfer data of the first field transferred next. Processing is performed (S4). The photographing confirmation image created in this way is displayed on the display unit 6 via the display unit I / F 3e.

  Subsequently, image processing (S5) for creating a thumbnail image is performed. If it is confirmed in the process S3 that “the setting is not to display the shooting confirmation image” (“No” in S3), image processing for creating a thumbnail image is performed by omitting “creation of the shooting confirmation image”.

  Next, it is confirmed whether or not the resolution of the image to be created is set to “low” (S6). If “low resolution” is set (“Yes” in S6), the first field is checked. Image processing is performed using the transfer data (same transfer data by the primary transfer) (S7), and a "low resolution image" is created.

  When the set resolution is not “low” (“No” in S6), the completion of the data transfer (secondary transfer) in the second field is confirmed (S8), and the resolution of the image to be created is “medium” It is confirmed whether it is set to "" (S9).

  When “medium resolution” is set (FIG. 5 corresponds to this case) (“Yes” in S9), transfer data in the first and second fields (data for primary transfer and secondary transfer) Is used to perform image processing for creating an “intermediate resolution image” (S10).

  In the process S9, when the set resolution is not “medium” (“No” in S9), the completion of the third field data transfer (tertiary transfer) is confirmed (S11), and the first, second, Image processing for creating a “high resolution image” using transfer data (data of all pixels) in the third field is performed (S12).

  Referring to FIG. 1, transfer data from the image sensor 1 passes through the CCD-I / F 3a of the signal processing unit 3 and the memory controller 3b via the imaging processing unit 2 in the order of the first, second, and third fields. It is stored as “raw data” in the RAW-RGB area 4 a of the storage unit 4.

  As described above, the R / G / B information data stored in the RAW-RGB area 4a of the storage unit 4 includes the presence / absence of display of the shooting confirmation image, the resolution “low”, “medium”, and “high”. After the completion of transfer of each field is detected in accordance with the “user setting” and the like, it is sent to the YUV conversion unit 3c via the memory controller 3b, converted into YUV signal data by YUV conversion, and the YUV area of the storage unit 4 4b.

  As described above, the shooting confirmation image and the thumbnail image created by the transfer data of the first field (same transfer data of the primary transfer) are displayed on the display unit 6 via the display unit I / F 3e. When the set resolution is “low”, the data of the YUV signal is read again from the YUV area 4b of the storage unit 4 and sent to the compression processing unit 3d via the memory controller 3b. The compression processing unit 3d performs compression processing to the JPEG format, and the compressed data is stored in the JPEG area 4c of the storage unit 4.

  When storage in the JPEG area 4c of the storage unit 4 is performed, header data or the like is added to the JPEG-compressed data and saved as “JPEG image data” in the external storage unit 4 ′ under the control of the control unit 5. If there is an image storage area 4 ″, it may be stored in the image storage area 4 ″.

  When the set resolution is “medium”, the completion of the data transfer (secondary transfer) in the second field is detected, and the image processing is performed using the transfer data in the first and second fields. When the resolution is “high”, the completion of data transfer in the third field is detected, and image processing using the transfer data (data of all pixels) in the first to third fields is performed.

  As a specific example, in the case of using an image sensor (CCD) of 3 million pixels of horizontal: 2048 pixels and vertical: 1536 pixels, in the interlaced transfer method in which divided transfer (primary to tertiary transfer) is performed three times, one time. Thus, data of horizontal 2048: pixels and vertical: 512 pixels (same transfer data) is transferred.

  Accordingly, when recording the size of horizontal: 640 pixels and vertical: 480 pixels (VGA), the number of transfer pixels in the first field is sufficient to create an image. Furthermore, the number of transfer pixels in the first and second fields (the sum of the same transfer data in the primary transfer and the same transfer data in the secondary transfer) is horizontal: 2048 pixels and vertical: 1024 pixels. An image of horizontal: 1024 pixels, vertical: 768 pixels (XGA) size, horizontal: 1280 pixels, vertical: 1024 pixels can be generated.

When the transfer data of the first field or “first and second fields” is used to create the captured image, the aspect ratio of the horizontal pixel and the vertical pixel is different, so the image becomes a “long image in the horizontal direction” . When the conversion processing is performed in the YUV conversion unit 3c, the image is formed by adjusting the aspect ratio from the RAW-RGB area 4a of the storage unit 4 by a process such as “reading with the horizontal direction thinned out” .

  For the purpose of “image recording of VGA size” similar to the shooting confirmation image and the thumbnail image, the transfer processing of the first field (same transfer data of the primary transfer) is sufficient for the captured image processing. By stopping the data transfer process of the second and third fields and moving to the next shooting, the process can be speeded up.

  In other words, using the transfer data of the number of fields necessary for image processing according to the set resolution (low / medium / high), without performing unnecessary field data transfer, the next shooting is started and the processing is performed at high speed. Can be achieved.

  In addition, the image processing performed by selecting the transfer data of the first to third fields transferred from the image sensor 1 may be performed in accordance with the compression rate setting.

  For example, in the case of “I want to record early and send quickly”, a high compression rate is often selected. Therefore, when the high compression rate is set, image processing is performed using the transfer data of the first field, and the transfer data of the first and second fields is used for the medium compression rate, and the transfer data of the first to third fields is used for the low compression rate. The selection of use can be performed.

  In the above, the number of field divisions in the image sensor (the number of fields constituting one field group): M has been described. However, in the future, the number of pixels of the image sensor will increase, and the total number of pixels will be transferred. The time may be increased. In such a case, when an image of VGA size can be obtained from the divided transfer pixel number of one field (same transfer data of primary transfer), data transfer is performed by dividing into 5 fields (M = 5) Even when M is an odd number larger than 3, image processing similar to the above can be performed.

  Hereinafter, the case where the number of field divisions M in the image sensor 1 of the interlace transfer method is 4 will be described with reference to FIG.

  7A, 7 </ b> B, and 7 </ b> C show an arrangement of “two columns of pixels (light receiving elements)” in the vertical direction (up and down direction in the figure) in an M = 4 interlaced transfer type image sensor. FIG. 4 is shown following FIG. In order to avoid complications, the same reference numerals as those in FIG. Therefore, the portions denoted by the same reference numerals as in FIG. 4 are the same as those described with reference to FIG.

  In the horizontal direction, which is the horizontal direction in the figure, in one column “filter R pixels and filter Gr pixels alternate” and in another column “filter B pixels and filter Gb pixels alternate” Are arranged.

  For example, in FIG. 7A, the pixel arrangement in the vertical direction (vertical direction) in the figure is divided into four groups. That is, numbers 1, 2, 3, and 4 are repeatedly written on the left side of the pixel array in the vertical direction, and each of these numbers represents a field. That is, in FIG. 7, the first field, the second field, the third field, and the fourth field constitute one “field group”.

  In FIG. 7, the transfer of the charges accumulated in all the pixels of the image sensor is performed as follows. That is, when transfer is started, first, charge transfer (primary transfer) of pixels belonging to the first field in each field group is performed (FIG. 7A). Charge transfer (secondary transfer) of pixels belonging to the second field is performed (FIG. 7B). When the secondary transfer ends, charge transfer (tertiary transfer) of the pixels belonging to the third field in each field group is performed (FIG. 7C), and finally, the pixels belonging to the fourth field in each field group are transferred. Charge transfer (quaternary transfer) is performed (FIG. 7D).

  Accordingly, when the quaternary transfer is completed, the “transfer of all charges” is completed.

  Hereinafter, image processing and image creation using such an image sensor will be described with reference to the flowchart of FIG.

  In this embodiment, the resolution of the photographed image can be set to three levels of “low / medium / high” as photographing conditions, and a thumbnail image is always created, and whether or not a photographing confirmation image is created is determined. It can be set selectively.

  First, shooting conditions are set. This setting is the same as in the case of M = 3 described above, and the setting state is, for example, as shown in FIG.

  Shooting is performed in this state. In the “start” state in the flowchart of FIG. 8, electrification is accumulated in all the pixels of the image sensor by photographing.

  When the transfer is started, the charge data accumulated in the image sensor 1 according to the image to be captured is divided into four fields, and the primary transfer to the quaternary transfer are sequentially performed. The data to be transferred is “correlated double sampling” by the CDS 2a in the image pickup processing unit 2 of FIG. 1, is automatically gain controlled by the AGC 2b, is converted into a digital signal (digital data) by the A / D 2c, and the signal processing unit 3 The CCD-I / F 3a and the memory controller 3b are stored in the storage unit 4 (S21).

  At this time, when the control unit 5 confirms “completion of data transfer in the first and second fields” (S22), it confirms whether or not “setting of photographing confirmation image display” is present at that time (“Yes” in S22). (S23). If the shooting conditions are set as shown in FIG. 5, a shooting confirmation image is displayed for 1 second.

  Here, looking at the state where the primary transfer is completed, the first field includes only R information (red component information) and G information (green component information) as color information, and B information (blue component information). Not included. For this reason, in the state where the primary transfer is completed, the “data transferred next time” does not contain all the information necessary to create a color image.

  However, when the secondary transfer is completed, R information (red component information), G information (green) are included as color information in the data transferred in the primary transfer and the data transferred in the secondary transfer. Component information) and B information (blue component information) are all included. Therefore, when the secondary transfer is completed, the transferred data includes all the “information necessary to create a color image”. In addition, the number of pixels necessary to create a low-resolution color image is also included.

  Accordingly, image processing can be performed at this stage.

  When “setting to display the shooting confirmation image” is confirmed in the process S23 (“Yes” in S23), the image processing for creating the “shooting confirmation image” based on the transfer data of the first field and the second field is performed. Perform (S24). The photographing confirmation image created in this way is displayed on the display unit 6 via the display unit I / F 3e.

  Subsequently, image processing (S25) for creating a thumbnail image is performed. If it is confirmed in the process S23 that “the setting is not to display the shooting confirmation image” (“No” in S23), image processing for creating a thumbnail image is performed by omitting “creation of the shooting confirmation image”.

  Next, it is confirmed whether or not the resolution of the image to be created is set to “low” (S26). If “low resolution” is set (“No” in S26), the first and second Image processing is performed using transfer data of two fields (sum of the same transfer data by primary and secondary transfer) (S27), and a "low resolution image" is created.

  When the set resolution is not “low” (“Yes” in S26), the completion of the third and fourth field data transfer (third and fourth transfer) is confirmed (S28), and the completion is confirmed. (Yes in S28), image processing is performed using the transfer data (third transfer and fourth transfer data) in the third and fourth fields.

  Whether the set resolution is “medium” or “high”, image processing is performed using the transfer data after the fourth transfer. When creating a high-resolution image, image processing is performed using all pixel data (all data) after the fourth transfer, and when creating a medium-resolution image, it is necessary to thin out the data from all the data. Pixel number data is obtained, and image processing is performed on this data (S29).

  Referring to FIG. 1, the transfer data from the image pickup device 1 is transmitted in the order of the first, second, third, and fourth fields via the image pickup processing unit 2 to the CCD-I / F 3 a of the signal processing unit 3 and the memory controller. 3b and stored as “raw data” in the RAW-RGB area 4a of the storage unit 4.

  As described above, the R / G / B information data stored in the RAW-RGB area 4a of the storage unit 4 includes the presence / absence of display of the shooting confirmation image, the resolution “low”, “medium”, and “high”. After the completion of transfer of each field is detected in accordance with the “user setting”, it is sent to the YUV conversion unit 3c via the memory controller 3b, converted into YUV signal data by YUV conversion, and YUV in the storage unit 4 Stored in area 4b.

  As described above, the shooting confirmation image and the thumbnail image created by the transfer data of the first and second fields (the same transfer data in the primary transfer and the secondary transfer) are transmitted via the display unit I / F 3e. It is displayed on the display unit 6. When the set resolution is “low”, the data of the YUV signal is read again from the YUV area 4b of the storage unit 4 via the memory controller 3b and sent to the compression processing unit 3d. The compression processing unit 3d performs compression processing into the JPEG format, and the compressed data is stored in the JPEG area 4c of the storage unit 4.

  When storage in the JPEG area 4c of the storage unit 4 is performed, header data or the like is added to the JPEG-compressed data and saved as “JPEG image data” in the external storage unit 4 ′ under the control of the control unit 5. If there is an image storage area 4 ″, it may be stored in the image storage area 4 ″. When the set resolution is “medium” or “high”, the completion of the data transfer in the third and fourth fields (tertiary and quaternary transfer) is detected, and the transfer data in the first to fourth fields are used. Perform image processing.

  As a specific example, when an image sensor (CCD) of 3 million pixels of horizontal: 2048 pixels and vertical: 1536 pixels is used, in the interlace transfer method in which divided transfer (primary to quaternary transfer) is performed four times, once. Thus, data of horizontal 2048: pixels and vertical: 384 pixels (transfer data of the same order) is transferred.

  Therefore, when recording the size of horizontal: 640 pixels and vertical: 480 pixels (VGA), the number of transfer pixels in the first and second fields (horizontal: 2048 pixels, vertical: 768 pixels) is “Create Image It is possible to generate an image having a horizontal size of 1024 pixels and a vertical size of 768 pixels (XGA).

When the transfer data of the “first and second fields” is used for image formation , the horizontal pixels and the vertical pixels have different aspect ratios, resulting in an “horizontal image” . When the conversion process is performed in the YUV conversion unit 3c, the aspect ratio is adjusted from the RAW-RGB area 4a of the storage unit 4 by a process such as “reading in which the horizontal direction is thinned out” to form an image.

  For the purpose of “image recording of VGA size” similar to the shooting confirmation image and the thumbnail image, the transfer data of the first and second fields is sufficient for the shot image processing. By stopping the data transfer process and moving to the next shooting, the process can be speeded up.

  In other words, using the transfer data of the number of fields necessary for image processing according to the set resolution (low / medium / high), without performing unnecessary field data transfer, the next shooting is started and the processing is performed at high speed. Can be achieved.

  In addition, the image processing performed by selecting the transfer data of the first to fourth fields transferred from the image sensor 1 may be performed in accordance with the compression rate setting.

  For example, in the case of “I want to record early and send quickly”, a high compression rate is often selected. Therefore, when the high compression rate is set, the image data is processed using the transfer data of the first and second fields, and the transfer data of the first to fourth fields is used at the medium compression rate or the low compression rate. It can be carried out.

  In the above, the number of field divisions in the image sensor (the number of fields constituting one field group): M has been described. However, in the future, the number of pixels in the image sensor will increase, and the total number of pixels will be transferred. Time is expected to increase.

  In that case, when M is an even number greater than 4, for example, M = 6, and data transfer is performed by dividing into six fields, a shooting confirmation image, a thumbnail image, or a recording image with a low resolution or a high compression rate is created. When the image is created using the transfer data obtained by the transfer of the first and second fields, and the recording image having the medium resolution or the medium compression rate is created, the image is obtained by the transfer of the first to fourth fields. When an image is created using the transfer data and a high resolution or low compression ratio recorded image is created, the image can be created using the transfer data obtained by the transfer in the first to sixth fields.

  In the embodiment described above with reference to FIGS. 1 to 7, the charges of all the pixels accumulated in the imaging device 1 having the color filter for color separation according to the image formed by the imaging unit. Is transferred to M (≧ 3) fields, and image processing is performed using transfer data for m (<M) fields in which all color signals and the required number of pixels are aligned. I do.

  In the embodiment described with reference to FIGS. 4 and 6, the number of fields: M is an odd number of 3 or more and m = 1. In the embodiment described with reference to FIGS. Number: M is an even number of 4 or more, and m = 2.

  In each of the embodiments described above, image processing including at least YUV conversion is performed immediately after transfer data for m fields is prepared, and a shooting confirmation image is generated by image processing including YUV conversion. A thumbnail image is created.

  In each of the above embodiments, a resolution that is a condition of an image to be created is set, image processing is performed using transfer data for n (m ≦ n ≦ M) fields, and an image to be created is created. n is the number of fields in which the number of pixels necessary for creating an image with the set resolution is aligned.

  In the embodiment described with reference to FIGS. 4, 5, and 6, a low-resolution image is created for M = 3, the number of fields: n = 1, and the number of fields: n = 2. A medium resolution image is created, and a high resolution image is created for the number of fields: n = M.

  In the embodiment described with reference to FIGS. 5, 7, and 8, a low-resolution image is created for M = 4, the number of fields: n = 2, and the number of fields: n = M. On the other hand, a medium resolution image and / or a high resolution image are created. As described above, M = 6, a high resolution image is created for the number of fields: n = M, and the number of fields: n. A low-resolution image can be created for = 2, or a medium-resolution or low-resolution image can be created for the number of fields: n = 4.

  Further, in each of the embodiments described above, “image compression rate” is used as a condition for an image to be created, and n is “the number of fields in which the number of pixels necessary for creating an image with a set compression rate is equal”. It can also be. In the embodiment described with reference to FIGS. 4, 5, and 6, image compression processing is performed in the JPEG format for M = 3 and the number of fields: n = 1, and FIGS. 5, 7, and 8. In the embodiment described with reference to the above, image compression processing can be performed in the JPEG format with M = 4 or 6 and the number of fields: n = 2.

  The imaging apparatus shown in the embodiment in FIG. 1 captures images on a setting unit 7 that sets a resolution as a condition of an image to be created, an image sensor 1 having a color filter for color separation, and the image sensor 1. Image forming means 01, 02, 03 for forming a power image, an image pickup processing unit 2 for performing image pickup processing for converting electric charges accumulated and transferred in the image pickup device 1 into a digital signal, and the image pickup processing unit. Data processing means 3, 4, and 5 for performing image processing including YUV conversion using the transfer data, and when the image sensor 1 transfers the charges of all the accumulated pixels, the charge data transfer Is an interlaced transfer method in which M is divided into M (≧ 3) fields.

  The color filter of the image sensor is of the primary color system of red (R), green (G), and blue (B). However, as described above, instead of this, yellow (Y), cyan (C), magenta ( M) and green (G) complementary colors may be used.

  In the image pickup apparatus whose embodiment has been described with reference to FIGS. 1, 4, 5, and 6, M in the interlace transfer type image pickup device 1 is 3, and FIGS. 1, 5, 7, and 8. In the image pickup apparatus whose embodiment has been described according to the above, M in the interlace transfer type image pickup device 1 is 4 or 6.

  Furthermore, the imaging apparatus described in the above embodiment has a display unit 6 that displays a captured confirmation image and / or a thumbnail image, and an external storage unit 4 ′ that stores the created image is detachable. An image holding area 4 '' that stores the created image can be provided.

  That is, the electronic still camera whose embodiment has been described above is an example of an image information processing apparatus having at least a part of the imaging apparatus of the present invention.

  For example, in the case of displaying a shooting confirmation image, when image processing is performed after data of all pixels in the image sensor is transferred, the conventional three million pixel display is 0.3 seconds before the shooting confirmation image is displayed. However, when M = 3 and m = 1, 1/3 of that is about 0.1 second, and when M = 4 and m = 2, 1 The shooting confirmation image can be displayed in about 0.15 seconds of / 2, which improves usability.

It is a figure which shows the system configuration | structure of the electronic still camera which is one Embodiment of an imaging device or an image information processing apparatus. It is a figure which shows an example of the form of the said electronic still camera. 2A and 2B are diagrams for explaining a color separation color filter included in an image sensor, in which FIG. 1A shows a primary color system and FIG. 2B shows a complementary color system. It is a figure for demonstrating the interlace transfer system in case M = 3. It is a figure for demonstrating an example of the setting of imaging conditions. FIG. 5 is a flowchart for explaining one embodiment of an imaging method using the interlaced imaging device of FIG. 4. It is a figure for demonstrating the interlace transfer system in case M = 4. FIG. 8 is a flowchart for explaining one embodiment of an imaging method using the interlaced imaging device of FIG. 7.

Explanation of symbols

1 Image sensor (CCD)
2 Imaging processing unit 3 Signal processing unit 4 Storage unit (SDRAM)
4 'External storage (memory card)
5 Control unit (CPU)
6 Display section 7 Operation section

Claims (17)

In the imaging method of setting the resolution of the captured image as a condition for creating the captured image and creating the captured image,
When transferring the charges of all the pixels accumulated according to the image formed by the imaging means to the image pickup device having a color filter for color separation in order to transfer the charge data, the data transfer of the charges is performed. Is divided into M (≧ 3) fields,
It is determined whether transfer data for m (<M) fields in which color signals for all colors of the color filter and the required number of pixels are aligned are prepared,
When it is determined that all the images have been prepared, a shooting confirmation image is created using the transfer data for the m fields.
Number of pixels required to create an image with a set resolution: When transfer data for n (m ≦ n ≦ M) fields are prepared, image processing is performed using the transfer data for n fields, and the above shooting is performed. Create an image ,
Adjust the aspect ratio when creating the above shooting confirmation image,
An imaging method comprising: adjusting the aspect ratio when generating the captured image using transfer data having the number of fields: n (<M) . In the imaging method of setting the resolution of the captured image as a condition for creating the captured image and creating the captured image,
When transferring the charges of all the pixels accumulated according to the image formed by the imaging means to the image pickup device having a color filter for color separation in order to transfer the charge data, the data transfer of the charges is performed. Is divided into M (≧ 3) fields,
A determination is made as to whether transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned are prepared,
If it is determined that the images are complete, thumbnail images are created using the transfer data for the m fields.
Number of pixels required to create an image with a set resolution: When transfer data for n (m ≦ n ≦ M) fields are prepared, image processing is performed using the transfer data for n fields, and the above shooting is performed. Create an image ,
Adjust the aspect ratio when creating the above thumbnail image,
An imaging method comprising: adjusting the aspect ratio when generating the captured image using transfer data having the number of fields: n (<M) . The imaging method according to claim 1 or 2,
Number of fields: M is an odd number greater than or equal to 3, and m = 1. The imaging method according to claim 1 or 2,
Number of fields: M is an even number greater than or equal to 4, and m = 2. The imaging method according to claim 1 or 2,
A low-resolution image is created for M = 3 and the number of fields: n = 1, and the number of fields:
An imaging method, wherein a medium resolution image is created for n = 2, and a high resolution image is created for the number of fields: n = M. The imaging method according to claim 1 or 2,
M = 4,
An imaging method, wherein a low-resolution image is created for the number of fields: n = 2, and a medium-resolution image and / or a high-resolution image is created for the number of fields: n = M. The imaging method according to claim 1 or 2,
M = 6,
A high resolution image is created for the number of fields: n = M, and a low resolution image is created for the number of fields: n = 2, or a medium or low resolution for the number of fields: n = 4. An imaging method characterized by creating an image. Setting means for setting the resolution of the image as a condition for creating a captured image;
An image sensor having a color filter for color separation;
Imaging means for forming an image to be imaged on the imaging element;
An imaging processing unit that performs an imaging process of converting the charge accumulated and transferred in the imaging element into a digital signal;
Using transfer data transferred via the imaging processing unit, and image processing means for performing image processing,
When the image pickup device transfers the accumulated charges of all the pixels to create the captured image, the charge transfer is performed by dividing the charge data into M (≧ 3) fields. Element,
The image processing means determines whether the transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned, and if determined to be aligned, Using the transfer data for the m fields, a shooting confirmation image is created by the image processing means,
Number of pixels required to create an image with a set resolution: When transfer data for n (m ≦ n ≦ M) fields are prepared, image processing is performed using the transfer data for n fields. Create an image
Adjust the aspect ratio when creating the above shooting confirmation image,
An image pickup apparatus, wherein the image to be created is created by adjusting the aspect ratio when creating the transfer data having the number of fields: n (<M) . Setting means for setting the resolution of the image as a condition for creating a captured image;
An image sensor having a color filter for color separation;
Imaging means for forming an image to be imaged on the imaging element;
An imaging processing unit that performs an imaging process of converting the charge accumulated and transferred in the imaging element into a digital signal;
Using transfer data transferred via the imaging processing unit, and image processing means for performing image processing,
When the image pickup device transfers the accumulated charges of all the pixels to create the captured image, the charge transfer is performed by dividing the charge data into M (≧ 3) fields. Element,
The image processing means determines whether the transfer data for m (<M) fields in which the color signals for all the colors of the color filter and the required number of pixels are aligned, and if determined to be aligned, Using the transfer data for the m fields, a thumbnail image is created by the image processing means,
Number of pixels necessary for creating an image with a set resolution: When transfer data for n (m ≦ n ≦ M) fields are prepared, image processing is performed using the transfer data of the n field sentence. Create an image
Adjust the aspect ratio when creating the above thumbnail image,
An image pickup apparatus, wherein the image to be created is created by adjusting the aspect ratio when creating the transfer data having the number of fields: n (<M) . The imaging device according to claim 8 or 9,
An image pickup apparatus, wherein the color filter of the image pickup element is of a primary color system of red (R), green (G), and blue (B). The imaging device according to claim 8 or 9,
An image pickup apparatus, wherein the color filter of the image pickup element is of a complementary color system of yellow (Y), cyan (C), magenta (M), and green (G). In the imaging device according to claim 8, 9 or 10 or 11,
M in an interlace transfer type image sensor is 3,
A low resolution image is created for the number of fields: n = 1, a medium resolution image is created for the number of fields: n = 2, and a high resolution image is created for the number of fields: n = M. An imaging apparatus characterized by that. In the imaging device according to claim 8, 9 or 10 or 11,
M in an interlace transfer type image sensor is 4,
An image pickup apparatus that creates a low-resolution image for the number of fields: n = 2 and creates a medium-resolution image and / or a high-resolution image for the number of fields: n = M. In the imaging device according to claim 8, 9 or 10 or 11,
M in an interlace transfer type image sensor is 6,
A high resolution image is created for the number of fields: n = M, and a low resolution image is created for the number of fields: n = 2, or a medium or low resolution for the number of fields: n = 4. An image pickup apparatus that creates an image of The imaging device according to any one of claims 8 to 14,
An imaging apparatus, wherein an external storage unit for storing a created image is detachable. The imaging device according to any one of claims 8 to 15,
An image pickup apparatus having an image holding area for storing a created image.   An image information processing apparatus having the imaging apparatus according to claim 8 as at least a part thereof.

Images