JP2011151454A - Image processor, imaging apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, an imaging apparatus and an image processing method, wherein a processing amount for generating images emphatically displaying a high luminance area like a cross filter or the like is drastically reduced. <P>SOLUTION: The image processor includes: a binarization circuit means for binarizing an input image by a predetermined threshold; a high luminance area specifying means for specifying a high luminance area from the binarized image; a priority degree determination means for determining a priority degree on the basis of a predetermined condition for the specified high luminance area; a selection means for selecting the predetermined number in the order from the high luminance area of the high priority degree when there are the predetermined number or more of the specified high luminance areas; and an image processing means for generating an emphatic image for emphatically displaying the selected high luminance area and combining the emphatic image with the input image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an imaging device, and an image processing method for generating a processed image that highlights a high-luminance region included in an image.

  2. Description of the Related Art Conventionally, a cross filter that generates a stripe on a high-luminance region of a subject image is known. This cross filter is a type of lens filter that creates a light stripe in the high brightness area of a subject image by engraving a thin groove on the surface of a transparent optical glass. Techniques for performing processing have been developed (see, for example, Patent Documents 1 and 2). Such a technique of emulating the effect of the cross filter by digital image processing is generally called cross filter processing.

  Cross filter processing by digital image processing extracts a high brightness area from a subject image and applies a light stripe to the subject image with respect to the high brightness area. A light stripe pattern is applied to each pixel in the high brightness area. The product-sum operation processing is performed.

  This product-sum operation process requires a huge number of operations and is difficult to process in real time. For this reason, it has been marketed as retouching software for still image processing, but recently, products installed in imaging devices such as digital cameras have been commercialized.

JP-A-6-30373 JP 2006-340070 A

  In an imaging device such as a digital camera, it is necessary to suppress the cross filter processing time within a time period during which the user can wait for the time until generation of a captured image is completed.

  However, since the conventional cross filter processing technology cannot perform the cross filter processing at high speed, it is difficult to perform the cross filter processing on a moving image or the like. This is not limited to the cross filter process, and the same applies to the case of generating an image in which a high luminance area is highlighted.

  Therefore, the present invention provides an image processing apparatus, an imaging device, and an image processing method that can significantly reduce the amount of processing for generating an image in which a high luminance region is highlighted, such as a cross filter. Objective.

  Accordingly, in order to solve the above-mentioned problem, the invention according to claim 1 is an image processing device in which binarization means for binarizing an input image with a predetermined threshold and high luminance from the binarized image. High brightness area specifying means for specifying an area, priority determining means for determining priority for the specified high brightness area based on a predetermined condition, and priority when the specified high brightness area is a predetermined number or more Selecting means for selecting the predetermined number in order from a high-intensity area having high degree; and an image processing means for generating an emphasized image for highlighting the selected high-intensity area and synthesizing the emphasized image with the input image. It was decided.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, the priority determination unit includes, as the predetermined condition, a size of the high luminance region and the high luminance region for the input image. The priority of the high luminance region is determined based on the position of the high luminance region and the hue of the high luminance region.

  According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect, the image processing means is assigned an enhanced image having a certain number of pixels to one high luminance region, The size of the enhanced image is changed according to the area of the luminance region.

  According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the image processing means reduces the input image and then increases the height of the reduced input image. An enhanced image that highlights the luminance region is generated, and the enhanced image is enlarged and then combined with the input image.

  Further, the invention according to claim 5 is the image processing apparatus according to any one of claims 1 to 4, wherein the image processing means generates the enhanced image in units of a plurality of frames of the input image, The same emphasized image is synthesized with the input images of the plurality of frames.

  According to a sixth aspect of the present invention, in the image processing device according to any one of the first to fifth aspects, the input image is a color image, and the image pattern of the emphasized image is two or more. Assigned to each of the colors, the binarizing means extracts a component of each of the two or more colors from the input image to generate a single color image, and the high-luminance region specifying means The high-luminance area is specified for each color image, the priority determination means determines the priority for each single-color image, and the selection means is the high-luminance area for each single-color image. The image processing means generates an enhanced image that highlights the selected high-intensity area for each single color image, and synthesizes the enhanced image with the input image.

  The invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 6, further comprising face image detection means for detecting a face image of a person from the input image. The means overwrites the face image extracted by the face detection means on the composite image generated by combining the emphasized image with the input image.

  The invention according to claim 8 is the image processing method, wherein the input image is binarized with a predetermined threshold, the high-luminance region is identified from the binarized image, and the identified high Determining a priority for a luminance area based on a predetermined condition; selecting a predetermined number in order from a high-luminance area having a higher priority when there are a predetermined number or more of the specified high-luminance areas; Generating an emphasized image for highlighting the high-intensity region, and synthesizing the emphasized image with the input image.

  According to a ninth aspect of the present invention, in the imaging device, the image processing apparatus includes an image processing device that processes an input image from the solid-state imaging device, and the image processing device binarizes the input image with a predetermined threshold value. A high brightness area specifying means for specifying a high brightness area from the binarized image, a priority determining means for determining a priority for the specified high brightness area based on a predetermined condition, When there are a predetermined number or more of the specified high-luminance areas, a selection unit that selects the predetermined number in order from the high-luminance areas with higher priority, and an emphasized image that highlights the selected high-luminance areas are generated, Image processing means for synthesizing the input image.

  According to the present invention, it is possible to provide an image processing apparatus, an imaging device, and an image processing method capable of significantly reducing the amount of processing for generating an image in which a high luminance region is highlighted such as a cross filter. it can.

It is a figure which shows the structure of the imaging device which concerns on one Embodiment of this invention. It is a figure which shows the circuit structure which performs the cross filter process for moving images. It is a figure which shows the circuit structure which performs the cross filter process for still images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is explanatory drawing of the cross filter process for moving images. It is a figure which shows the example of the face exclusion synthetic image which overwrited the person's face image on the synthetic image. It is explanatory drawing of the processing amount in a vector type cross filter process. It is explanatory drawing of a vector type cross filter process. It is explanatory drawing of a raster type cross filter process. It is a figure which shows the actual difference of the cross filter process result of a vector type and a raster type. It is explanatory drawing of the cross filter process in a 1st modification. It is explanatory drawing of the cross filter process in a 1st modification. It is explanatory drawing of the cross filter process in a 1st modification. It is explanatory drawing of the cross filter process in a 1st modification. It is explanatory drawing of the cross filter process in a 2nd modification. It is explanatory drawing of the cross filter process in a 2nd modification. It is a figure which shows the image of other pattern data. It is a figure which shows the image of other pattern data.

Hereinafter, modes for carrying out the invention (hereinafter referred to as “embodiments”) will be described. The description will be given in the following order.
1. Configuration of imaging device 2. Configuration of image processing apparatus 3. Processing operation of image processing apparatus 4. Operation of the cross filter circuit 5. First modification of image processing apparatus Second modification of image processing apparatus

[1. Configuration of imaging equipment]
First, the configuration of the imaging device according to the present embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration of an imaging device according to an embodiment of the present invention.

  As shown in FIG. 1, the imaging device 1 includes a solid-state imaging device 2, an A / D (analog / digital) conversion circuit 3, an image processing device 4, a system controller 5, an input unit 6, and an optical block 7. . The imaging device 1 is provided with a driver 8 for driving a mechanism in the optical block 7, a timing generator (TG) 9 for driving the solid-state imaging device 2, and the like. For example, a CMOS solid-state image sensor or a CCD solid-state image sensor is used as the solid-state image sensor 2.

  The optical block 7 includes a lens for condensing light from the subject onto the solid-state imaging device 2, a driving mechanism for moving the lens to perform focusing and zooming, a mechanical shutter, a diaphragm, and the like. The driver 8 controls driving of the mechanism in the optical block 7 in accordance with a control signal from the system controller 5.

  The solid-state imaging device 2 is driven based on the timing signal output from the TG 9 and converts incident light from the subject into an electrical signal. The TG 9 outputs a timing signal under the control of the system controller 5.

  The A / D conversion circuit 3 performs A / D conversion on the image signal output from the solid-state imaging device 2 and outputs a digital image signal. The image signal is composed of pixel signals output from each pixel of the solid-state imaging device 2.

  The image processing device 4 performs AF (Auto Focus), AE (Auto Exposure), defective pixel detection processing and correction on the digital image signal output from the A / D conversion circuit 3 in addition to cross-filter processing described later. Various camera signal processing such as processing, white balance adjustment, and matrix processing are executed.

  The system controller 5 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU executes the programs stored in the ROM, thereby controlling the respective units of the imaging device 1 in an integrated manner and executing various calculations for the control. The input unit 6 includes operation keys, dials, levers, and the like that receive user operation inputs, and outputs a control signal corresponding to the operation inputs to the system controller 5.

  In this imaging device 1, pixel signals received and photoelectrically converted by the respective photoelectric conversion units of the solid-state imaging device 2 are sequentially supplied to the A / D conversion circuit 3 and converted into digital signals, and the image processing device 4 performs defect processing. Correction processing and the like are performed and output as image data.

  The image data output from the image processing device 4 is supplied to a graphic interface circuit (not shown) and converted into an image signal for display, whereby a camera-through image is displayed on a monitor (not shown). When the system controller 5 is instructed to record an image by a user input operation to the input unit 6 or the like, the image data from the image processing device 4 is supplied to a CODEC (enCOder, DEcoder) and compressed and encoded. The processing is performed and recorded on the recording medium. The imaging apparatus 1 according to the present embodiment has a still image mode for recording a still image and a moving image mode for recording a moving image, and the user can select either mode by an input operation to the input unit 6 or the like. Can be selected. When recording a still image in the still image mode, image data for one frame is supplied from the image processing device 4 to the CODEC. When recording a moving image in the moving image mode, the processed image is processed. Data is continuously supplied to the CODEC.

[2. Configuration of the image processing apparatus 4]
Next, the configuration of the image processing apparatus 4 according to the present embodiment will be described with reference to the drawings. The image processing apparatus 4 performs processing speed priority processing when recording a moving image in the moving image mode, and performs image quality priority processing when recording a still image in the still image mode. FIG. 2 is a diagram showing a circuit configuration for performing moving image cross-filter processing applied when recording a moving image or monitoring a moving image. FIG. 3 is a diagram showing a circuit configuration for performing a still image cross-filter process applied at the time of recording a still image. The image processing apparatus 4 performs various other processes and actually has other circuit configurations, but here, the cross filter process will be mainly described.

  First, a circuit configuration for performing moving image cross filter processing will be described with reference to FIG.

  As shown in FIG. 2, the image processing apparatus 4 includes a frame interpolation processing circuit 11, an enhanced image generation circuit 12, a face detection circuit 13, and a synthesis processing circuit 14. Perform filter processing. For example, the emphasized image generation circuit 12 and the composition processing circuit 14 perform cross filter processing, and the face detection circuit 13 and the composition processing circuit 14 perform cross filter exclusion processing.

  The frame interpolation processing circuit 11 performs processing for interpolating images between frames. Specifically, the frame interpolation processing circuit 11 detects a motion vector between images in units of frames (hereinafter referred to as “input images”) formed by input image signals sequentially input in units of frames. An interpolated image signal is generated based on the vector. The interpolated image signal generated in this way is mixed with the input image signal by the frame interpolation processing circuit 11.

  The emphasized image generation circuit 12 generates raster data of an emphasized image for performing an emphasis display on the input image. The emphasized image is specifically a streak image, and is combined with the input image to perform cross filter processing.

  The enhanced image generation circuit 12 includes a gray scale conversion circuit 21, a reduction scaling circuit 22, a binarization circuit 23, a labeling circuit 24, a priority determination circuit 25, a selection circuit 26, and a cross filter processing circuit 27. And an expansion scaling circuit 28 and a pattern data storage unit 29. The pattern data storage unit 29 stores light stripe pattern data (hereinafter also referred to as “pattern data”) used by the cross filter processing circuit 27.

  When the input image is a color image, the gray scale conversion circuit 21 performs processing for changing the input image to a gray scale. That is, the gray scale conversion circuit 21 expresses the input image only by contrasting from white to black, and converts it into an image that does not include color information. It should be noted that the grayscale conversion of the input image can be performed before the processing by the reduction scaling circuit 22, thereby reducing the processing amount in the reduction scaling circuit 22.

  The reduction scaling circuit 22 performs a process of reducing the size of the gray scaled input image at a predetermined reduction rate. The reduction ratio can be changed, and the smaller the reduction size, the faster the cross filter process can be performed. The reduction rate can be changed by, for example, a user operation from the input unit 6.

  The binarization circuit 23 performs a process of generating a binarized image by binarizing the grayscale and reduced input image with a set threshold as binarization means. This threshold value is set to a value that can specify a high-luminance region of the input image. This threshold value can be adjusted, and can be changed, for example, by a user operation from the input unit 6. By adjusting the threshold value, it is possible to adjust the area and range to which the cross filter process is performed.

  The labeling circuit 24 performs a process of specifying a high brightness area from the binarized image as a high brightness area specifying means. Specifically, the labeling circuit 24 identifies a region in which high-luminance pixels of the binarized image are gathered as a single high-luminance region, and identifies each of the identified high-luminance regions as unique identification information. Give a number. The labeling circuit 24 does not specify a small high luminance area that is smaller than a preset threshold area as a high luminance area. The setting of the threshold area for specifying the high luminance region can be adjusted, and can be changed by, for example, an operation by the user from the input unit 6. In this way, it is possible to reduce the processing time in the subsequent stage by not specifying a high luminance area less than the threshold area. Note that by increasing the threshold area, it is possible to reduce the number of high-luminance areas to be specified, and it is possible to further reduce the processing time in the subsequent stage.

  The priority determination circuit 25 performs a process of determining the priority of the high luminance area specified by the labeling circuit 24 and assigning priority to the high luminance area as priority determination means. The priority determination by the priority determination circuit 25 is performed based on the size of the high luminance area. For example, the priority is set higher as the size of the high luminance area is larger. Note that the priority can be determined based on the size of the high-luminance area, the position of the high-luminance area with respect to the input image, the hue of the high-luminance area, and the like. For example, the priority is set higher as the position of the high luminance region is near the center of the input image. Further, the priority may be determined by combining a plurality of conditions by setting by an input operation or the like to the input unit 6 such as based on the size of the high luminance area and the position of the high luminance area with respect to the input image. it can. Note that the accuracy of the priority of the high-brightness area can be improved by detecting the hue of the high-brightness area instead of the binarized image. However, in the example shown in FIG. The priority of the high brightness area is detected from the binarized image.

  When there are n or more high-luminance areas specified by the labeling circuit 24, the selection circuit 26 performs a process of selecting n in order from the high-luminance areas with the highest priority. n is set to a number with which the cross filter process can be performed quickly. This n can be changed by, for example, a user operation from the input unit 6. As n increases, more high-luminance regions are subjected to the cross-filter process, and the cross-filter process can be performed with high accuracy, but the cross-filter process takes time. On the other hand, as n is smaller, the number of high-luminance regions that are subjected to cross filter processing decreases, and the accuracy of the cross filter processing decreases, but the time for cross filter processing can be reduced. It should be noted that the maximum number of n that can be subjected to the cross filter process for the input image of each frame is an upper limit value, and the setting cannot be changed beyond that.

  The cross filter processing circuit 27 is an image processing unit that performs raster-type cross filter processing, and performs cross filter processing on the high-luminance region selected by the selection circuit 26 based on the pattern data stored in the pattern data storage unit 29. Generate raster data to be applied. The processing of the cross filter processing circuit 27 will be described in detail later.

  The enlargement / scaling circuit 28 serves as an image processing means for converting the enhanced image generated by the cross filter processing circuit 27 into an enhanced image enlarged at a predetermined enlargement ratio. Arithmetic processing is performed on the raster data. The enlargement ratio in the enlargement scaling circuit 28 corresponds to the reduction ratio in the reduction scaling circuit 22 and is set to be an enhanced image having a size corresponding to the input image. For example, when the reduction ratio is 20% (1/5 times), the enlargement ratio is 500% (5 times).

  The face detection circuit 13 performs a process of detecting a human face image from an input image and outputting face position and size information as a face detection unit. Extraction of a face image is a well-known technique, and can be performed, for example, by detecting a skin color area or detecting a motion.

  The synthesis processing circuit 14 is a process for generating a synthesized image by synthesizing the enhanced image with the input image subjected to frame interpolation based on the raster data of the enhanced image output from the enlargement scaling circuit 28 as image processing means. I do.

  The synthesis processing circuit 14 detects a human face image based on the face position and size information output from the face detection circuit 13, overwrites the face image on the synthesized image, and outputs the image as an output image. Thus, by overwriting the face image on the synthesized image, the enhanced image can be removed from the person's face even when the enhanced image is synthesized with the person's face. In addition, it is possible to avoid the problem that the face of the person cannot be recognized due to the emphasized image. Before combining the enhanced image with the input image, after excluding the raster data of the person's face area from the raster data of the enhanced image based on the face position and size information output from the face detection circuit 13, Based on the raster data of the enhanced image, the enhanced image can be combined with the input image.

  As described above, in the image processing device 4 at the time of recording a moving image, the enhanced image generation circuit 12 and the synthesis processing circuit 14 perform the cross filter process, so that an image in which a high luminance area is highlighted as in a cross filter or the like. The processing amount for generating can be significantly reduced. Further, by performing cross filter exclusion processing by the face detection circuit 13 and the synthesis processing circuit 14, a human face can be excluded from the processing, and a cross filter effect that cannot be realized by a lens filter can be obtained.

  Next, a circuit configuration for performing the still image cross filter processing will be described with reference to FIG. In addition, the element which has the same function as the element mentioned above attaches | subjects the same code | symbol, and shall abbreviate | omit description below.

  As shown in FIG. 3, the image processing apparatus 4 includes an enhanced image generation circuit 12 ′, a synthesis processing circuit 14 ′, and a face detection circuit 13, and performs frame interpolation processing, cross filter processing, and the like. . For example, cross filter processing is performed by the enhanced image generation circuit 12 ′ and the synthesis processing circuit 14 ′, and cross filter exclusion processing is performed by the face detection circuit 13 and the synthesis processing circuit 14 ′.

  The enhanced image generation circuit 12 ′, like the enhanced image generation circuit 12, generates enhanced image vector data for emphasizing the input image. The emphasized image is specifically a streak image, and is combined with the input image to perform cross filter processing.

  The emphasized image generation circuit 12 ′ includes a gray scale conversion circuit 21, a binarization circuit 23, a labeling circuit 24, a cross filter processing circuit 27 ′, and a pattern data storage unit 29. Unlike the above-described enhanced image generation circuit 12, the enhanced image generation circuit 12 ′ does not perform processing such as reduction of the input image or enlargement of the enhanced image, and limits the number of high luminance areas to which the enhanced image is added. Absent. Further, in the cross filter processing circuit 27 ', vector type cross filter processing is performed as described later. As a result, when this still image is recorded, although processing time is required, a fine cross filter process is performed compared to the cross filter process for a moving image.

  Based on the pattern data stored in the pattern data storage unit 29, the cross filter processing circuit 27 'generates vector data for performing the cross filter process on the high luminance area specified by the labeling circuit 24. Specifically, the cross filter processing circuit 27 ′ assigns pattern data to each of the pixels constituting the high luminance area specified by the labeling circuit 24, and assigns the pattern data to all the pixels. Generate vector data for the enhanced image.

  As an image processing unit, the synthesis processing circuit 14 ′ performs a process of generating a synthesized image by synthesizing the enhanced image with the input image based on the vector data of the enhanced image output from the cross filter processing circuit 27 ′. Similarly to the composition processing circuit 14, the composition processing circuit 14 ′ detects a human face image based on the face position and size information output from the face detection circuit 13, and overwrites the face image on the composite image. And output as an output image. By overwriting the face image on the composite image in this way, even when the emphasized image is combined with the person's face, the emphasized image can be removed from the person's face. It is possible to avoid the problem that the face of a person cannot be understood by the image.

  As described above, the image processing apparatus 4 at the time of recording a still image does not perform processing such as reduction of the input image or enlargement of the emphasized image, and high brightness for adding the emphasized image, compared with the case of recording the moving image. There is no limit on the number of areas. Therefore, it is possible to output a still image that takes time but is subjected to beautiful cross filter processing. Further, by performing the cross filter exclusion process by the face detection circuit 13 and the synthesis processing circuit 14, a cross filter effect that cannot be realized by the lens filter can be obtained by the function of excluding a human face from the process.

[3. Processing operation of image processing apparatus 4]
The processing operation of the image processing apparatus 4 configured as described above will be specifically described with reference to FIGS. Here, the processing operation of the image processing apparatus 4 at the time of recording a moving image will be described.

  When an input image is input to the image processing device 4, processing by the image processing device 4 is started. As an input image, for example, it is assumed that a color image having a night lighthouse as a subject and an image having a high luminance area are input to the image processing device 4.

  The input image input to the image processing apparatus 4 is input to the gray scale conversion circuit 21 and converted into a gray scale image expressed only by light and dark from white to black, as shown in FIG.

  As shown in FIG. 5, the input image data subjected to the gray scale conversion is reduced by the reduction scaling circuit 22 at a predetermined reduction rate and is input to the binarization circuit 23. The binarization circuit 23 binarizes the input image that has been grayscaled and reduced with a set threshold value to generate a binarized image as shown in FIG. Output to the labeling circuit 24.

  The labeling circuit 24 performs a process of specifying a high luminance area from the binarized image. Specifically, the labeling circuit 24 identifies and extracts a region having a predetermined threshold area or more as a high-luminance region among regions where the high-luminance pixels of the binarized image are gathered in a lump. Further, as shown in FIG. 7, the labeling circuit 24 assigns a unique identification number to each identified high-luminance area as identification information. The identification number to be assigned is identification information for identifying the high-luminance region, and may be a symbol instead of a number if there is no overlap.

  The labeling circuit 24 associates the identified high-brightness area raster data with the identification number and outputs it to the priority determination circuit 25. The priority determination circuit 25 determines the priority of the high luminance area specified by the labeling circuit 24, and gives priority to the high luminance area. Here, the priority is determined based on the size of the high luminance area. In the example shown in FIG. 8, 6, 4, 5, 3, 1, 2, 8, 9, A priority of 7 is assigned. In this example, the lower the number, the higher the priority.

  The priority determination circuit 25 adds priority information to the raster data of the high luminance region specified by the labeling circuit 24 and associated with the identification number, and outputs the raster data to the selection circuit 26.

  When the selection circuit 26 determines that there are n or more high-luminance areas specified by the labeling circuit 24 based on the information input from the priority determination circuit 25, the selection circuit 26 selects n in order from the high-luminance areas with the highest priority. In the example shown in FIG. 9, n is set to 3, and the high-brightness areas with the identification numbers 4, 5, and 6 are selected as high-brightness areas with high priority.

  The selection circuit 26 associates the raster data of the selected high-luminance area with the identification number and outputs it to the cross filter processing circuit 27. The cross filter processing circuit 27 generates raster data for performing the cross filter process on the high luminance region selected by the selection circuit 26 based on the data of the light stripe pattern (see FIG. 10) stored in the pattern data storage unit 29. To do. That is, the cross filter processing circuit 27 generates raster data of an emphasized image as shown in FIG. 11 by raster type cross filter processing described later.

  The raster data of the emphasized image generated by the cross filter processing circuit 27 is input to the enlargement scaling circuit 28. As shown in FIG. 12, the enlargement scaling circuit 28 uses the enhanced image generated by the cross filter processing circuit 27 in order to make the enhanced image generated by the cross filter processing circuit 27 enlarged at a predetermined enlargement ratio. The arithmetic processing is performed on the raster data.

  The synthesis processing circuit 14 synthesizes the enhanced image with the input image subjected to frame interpolation based on the raster data of the enhanced image output from the enlargement / scaling circuit 28, and generates a synthesized image as shown in FIG. . Note that the synthesis processing circuit 14 has a function of overwriting a person's face image on the synthesized image based on the face position and size information output from the face detection circuit 13, but the image shown in FIG. Since no human image is included, the detection by the face detection circuit 13 is not performed, and the face image is not overwritten. Note that if the input image includes a human face image, the synthesis processing circuit 14, for example, as shown in FIG. 14, the human face based on the face position and size information output from the face detection circuit 13. A face-excluded composite image is generated by overwriting the image on the composite image.

  As described above, in the image processing apparatus according to the present embodiment, the emphasized image is generated based on the image obtained by reducing the input image by the reduction scaling circuit 22 and the cross filter process is performed. In addition, high-luminance area selection processing, enhanced image generation processing, and the like can be performed at high speed.

  Further, since the labeling circuit 24 does not specify a small high-brightness area less than the threshold area as a high-brightness area, the high-brightness area selection process, the enhanced image generation process, and the like are also performed at high speed. be able to.

  Further, in the selection circuit 26, when there are n or more high-luminance regions specified by the labeling circuit 24, a process of selecting n in order from the high-luminance region with the highest priority is performed, so there are many high-luminance regions. Even at times, the enhanced image generation process can be performed within a certain period of time. Therefore, even when the imaging device 1 is in the moving image mode, the cross filter process can be executed in real time.

[4. Operation of cross filter circuit]
Next, the operation of the cross filter circuit will be described more specifically with reference to the drawings. Here, in order to facilitate understanding of the drawings, for the sake of convenience, the emphasized image is represented as a star-shaped image instead of a light stripe image.

  Conventional cross-filter processing is vector-type cross-filter processing, in which pattern data is assigned to each of the pixels constituting the high-luminance region, and pattern data is assigned to all the pixels, thereby creating a stripe image. To generate an enhanced image. That is, the integration process is performed for each pixel constituting the high luminance area by the number of times corresponding to the area of the light stripe pattern. For this reason, the processing amount increases as the number of pixels constituting the high luminance region increases. Further, as shown in FIG. 15, when processing for changing the enlargement ratio of the emphasized image is performed according to the area of the high luminance region, the processing amount increases in proportion to the enlargement area.

  Therefore, in the image processing apparatus 4 according to this embodiment, the cross filter processing circuit 27 performs raster type cross filter processing so that the processing amount can be reduced.

  FIG. 16 is an explanatory diagram of conventional vector type cross filter processing, and FIG. 17 is an explanatory diagram of raster type cross filter processing.

  In the conventional vector type cross filter processing, as shown in FIG. 16, the cross filter processing is performed on all regions of the target image. On the other hand, in the raster-type cross filter processing shown in FIG. 17, the cross filter processing is performed in a unit of enlargement ratio on the target area of the target image in consideration of the enlargement ratio of the cross filter for the pattern data. That is, in the cross filter processing circuit 27, integration processing is performed for the number of pixels constituting the pattern data for one high-luminance region regardless of the area of the high-luminance region. Further, the cross filter processing circuit 27 performs a process of enlarging the pattern data so that the size of the pattern data becomes larger as the area of the high luminance region becomes larger.

  In this way, the cross filter processing circuit 27 performs raster type cross filter processing, so that the emphasis image becomes jagged, but the processing amount per pixel does not depend on the enlargement rate of the cross filter. Cross filter processing can be executed at high speed.

  FIG. 18 shows the actual difference between the vector type and raster type cross filter processing results. FIG. 18A shows a vector type cross filter processing result, and FIG. 18B shows a raster type cross filter processing result.

  In the imaging device 1 according to the present embodiment, for example, in the still image mode, at the time of moving image monitoring, the circuit shown in FIG. 2 performs the raster-type cross filter processing shown in FIG. The processing result is displayed on a monitor (not shown). The monitor of the imaging device 1 uses a monitor having a resolution lower than the resolution of the image to be recorded, and cross filter processing is performed in real time by using a raster type cross filter. Further, when a still image is recorded, a beautiful cross filter processing result can be obtained by performing the vector type cross filter processing shown in FIG. 16 over the processing time shown in FIG.

[5. First Modification of Image Processing Device]
In the image processing apparatus 4 described above, the cross filter process is performed on the input image for each frame, but the same enhanced image is applied to the input image in units of a plurality of frames by a user input operation to the input unit 6 or the like. The cross filter processing by may be performed.

  That is, the enhanced image generation circuit 12 generates an enhanced image in units of a plurality of frames of the input image, and the synthesis processing circuit 14 synthesizes the same enhanced image with the input images of the plurality of frames. A frame buffer circuit 31 that holds the enhanced image generated by the cross filter processing circuit 27 is provided between the cross filter processing circuit 27 and the synthesis processing circuit 14. In addition, a frame buffer circuit 32 that holds an input image subjected to the interpolation processing by the frame interpolation processing circuit 11 is provided between the frame interpolation processing circuit 11 and the composition processing circuit 14.

  In this way, even when the cross filter processing cannot keep up with the frame rate of the moving image, the cross filter processing can be performed even on a moving image with a high frame rate.

  Further, the process of synthesizing the same emphasized image with each of the input images of a plurality of frames (hereinafter referred to as “multiple frame unit process”) has a processing quality of cross filter processing in the case of a moving image with little motion. Not so bad. Therefore, for example, a detection unit that detects the motion of a moving image is provided, and when the motion of the moving image is small, the power consumption can be reduced by performing the cross filter processing in units of frames instead of the cross filter processing in units of frames. It can also be reduced.

  Hereinafter, with reference to FIGS. 19 to 22, the above-described multi-frame unit processing will be specifically described. Here, a description will be given of multi-frame unit processing using three frames as one unit. 19 to 22, some circuits are omitted for easy understanding.

  As shown in FIG. 19, the input image of the first frame is input to the cross filter processing circuit 27 and further input to the frame buffer circuit 32 via the frame interpolation processing circuit 11. At this time, the input image of the first frame is accumulated in the frame buffer circuit 32 and is not input to the synthesis processing circuit 14, and the synthesized image is not output.

  Next, when the input image of the second frame is input, this input image is not input to the cross filter processing circuit 27 but is input only to the frame buffer circuit 32. At this time, the input image of the first frame and the input image of the second frame are accumulated in the frame buffer circuit 32 and are not input to the synthesis processing circuit 14, and the synthesized image is not output.

  As shown in FIG. 20, until the input image of the third frame is input, the generation of the emphasized image in the cross filter processing circuit 27 is completed, and this emphasized image is accumulated in the frame buffer circuit 31. When the input image of the third frame is input, the composition processing circuit 14 adds and combines the input image of the first frame read from the frame buffer circuit 32 and the emphasized image read from the frame buffer circuit 31. And output as an output image.

  Next, when the input image of the fourth frame is input, the input image of the fourth frame is similar to the input image of the first frame, as shown in FIG. 32. The synthesis processing circuit 14 performs addition synthesis processing on the input image of the second frame read from the frame buffer circuit 32 and the emphasized image read from the frame buffer circuit 31, and outputs the result as an output image. The enhanced image at this time is an enhanced image generated based on the input image of the first frame.

  Next, when the input image of the fifth frame is input, the input image of the fifth frame is not input to the cross filter processing circuit 27 as in the case of the input image of the second frame, as shown in FIG. The signal is input only to the frame buffer circuit 32. The synthesis processing circuit 14 performs addition synthesis processing on the input image of the third frame read from the frame buffer circuit 32 and the emphasized image read from the frame buffer circuit 31, and outputs the result as an output image. The enhanced image at this time is an enhanced image generated based on the input image of the first frame.

  Also, as shown in FIG. 22, the generation of the emphasized image in the cross filter processing circuit 27 is completed until the input image of the sixth frame is input, and this emphasized image is accumulated in the frame buffer circuit 31. The enhanced image at this time is an enhanced image generated based on the input image of the fourth frame. Then, when the input image of the sixth frame is input, the synthesis processing circuit 14 adds and combines the input image of the fourth frame read from the frame buffer circuit 32 and the emphasized image read from the frame buffer circuit 31. And output as an output image.

  Thereafter, the same processing as the processing in the 4th to 6th frames is repeatedly performed to execute the cross filter processing once in 3 frames. By performing this process, it is possible to perform the cross filter process even on a moving picture with a high frame rate even if the cross filter process cannot keep up with the frame rate of the moving picture. When the same cross filter processing is performed on an input image of a plurality of frames as described above, the upper limit value n of the high luminance region selected by the selection circuit 26 is the time when the input image of the plurality of frames is input. And the maximum number that can be subjected to the cross filter processing.

[6. Second Modification of Image Processing Device]
In the image processing apparatus described above, the pattern data used for the cross filter processing is not provided for each color, but in the image processing apparatus of the second modification, pattern data is provided for each color. For example, as shown in FIG. 23, each pattern data of R (red), G (green), B (blue), and Y (yellow) is stored in the pattern data storage unit 49.

  In the image processing apparatus of the second modified example, the gray scale conversion circuit 21 is not provided, and a color input image is input to the binarization circuit 43. In the binarization circuit 43, threshold conditions are determined in the color space as binarization means. That is, threshold values for R, G, B, and Y are set in the binarization circuit 43, and the binarization circuit 43 sets the colors of R, G, B, and Y in the input image. The luminance is extracted, and single color images of R, G, B, and Y are generated. Then, the binarization circuit 43 generates a binarized image based on each single color image of R, G, B, and Y based on each threshold value. Thereafter, the binarization circuit 43 outputs each binarized image data to the labeling circuit 24.

  The labeling circuit 44 specifies a high luminance area for each of the binary color images of R, G, B, and Y as a high luminance area specifying means. The labeling circuit 44 associates the identified high-luminance area raster data with the identification number and outputs it to the priority determination circuit 25. The method for specifying the high luminance region is the same as that of the labeling circuit 24. It should be noted that the threshold area can be changed for each binarized color image by a user operation from the input unit 6 or the like.

  The priority determination circuit 45 determines the priority of the high luminance area for each raster data of each of the high luminance areas R, G, B, and Y as the priority determination means, and the priority for each raster data. Is added to the selection circuit 26. The priority determination process is the same as that of the priority determination circuit 25. Note that the priority determination condition can be changed for each binarized color image by a user operation from the input unit 6.

  The selection circuit 46 selects a high luminance area for each single color image as selection means. That is, the selection circuit 46 determines whether there are n or more high-luminance regions specified by the labeling circuit 24 for each of R, G, B, and Y colors based on the priority information input from the priority determination circuit 25. If there are n or more high-luminance regions, n are selected in order from the high-luminance region with the highest priority.

  The cross filter processing circuit 47 stores, as an image processing means, an enhanced image that highlights the high-intensity area selected by the selection circuit 46 for each of R, G, B, and Y in the pattern data storage unit 49. Generated based on the pattern data of each color. The enhanced image of each color generated by the cross filter processing circuit 47 is input to the synthesis processing circuit 50. In the synthesis processing circuit 50, the enhanced image of each process is added and synthesized to the input image to generate a synthesized image.

  Various emphasis display can be performed by changing the color, the emphasis degree, the binarization threshold value, and the like of the emphasis image for each color. For example, in FIG. 24A, the threshold value for binarization of B and Y colors is lowered, and these colors are highlighted in B (blue). In FIG. 24B, the threshold value for binarizing the R and Y colors is lowered, and these colors are highlighted with R (red). In FIG. 24C, the threshold for binarization of G and Y is lowered, and these colors are highlighted in G (green). In FIG. 24D, the threshold for binarizing the Y color is lowered, and this color highlighting is performed in Y (yellow).

  As described above, by performing separate threshold determination for each of two or more colors and applying a pattern table for each color, it is possible to widen variations of the cross filter.

  In the above description, the streak image is taken as an example of the enhanced image. However, the enhanced image is not limited to this, and may be, for example, a character as shown in FIG. In addition, as shown in FIG. 26, by using color pattern data as a character, it is possible to obtain an effect that the light source is a character.

  Although some of the embodiments of the present invention have been described in detail with reference to the drawings, these are exemplifications, and the present invention is implemented in other forms with various modifications and improvements based on the knowledge of those skilled in the art. Is possible.

  For example, each circuit described above may be configured by hardware or may be processed by software.

1 Imaging equipment 1
2 Solid-state imaging device 3 A / D conversion circuit 4 Image processing circuit 5 System controller 6 Input unit 7 Optical block 8 Driver 9 Timing generator 11 Frame interpolation processing circuit 12 Enhanced image generation circuit 13 Face detection circuit 14 Composition processing circuit 21 Grayscale conversion Circuit 22 Reduction scaling circuit 23 Binary circuit 24 Labeling circuit 25 Priority determination circuit 26 Selection circuit 27 Cross filter processing circuit 28 Expansion scaling circuit 29 Pattern data storage unit

Claims (9)

Binarization means for binarizing the input image with a predetermined threshold;
High luminance area specifying means for specifying a high luminance area from the binarized image;
Priority determining means for determining a priority based on a predetermined condition for the identified high-luminance region;
A selection means for selecting the predetermined number in order from a high-luminance area having a higher priority when the specified high-luminance area is a predetermined number or more;
An image processing apparatus comprising: an image processing unit that generates an emphasized image that highlights the selected high-luminance region, and combines the emphasized image with the input image. The priority determination means determines the priority of the high brightness area based on the size of the high brightness area, the position of the high brightness area with respect to the input image, and the hue of the high brightness area as the predetermined condition. The image processing apparatus according to claim 1. 3. The image processing according to claim 1, wherein the image processing unit is assigned an enhanced image having a certain number of pixels to one high brightness area, and changes the size of the enhanced image in accordance with the area of the high brightness area. apparatus. The image processing means generates an emphasized image that highlights a high-intensity region of the reduced input image after reducing the input image, and combines the input image after enlarging the emphasized image. The image processing apparatus according to any one of the above. 5. The image according to claim 1, wherein the image processing unit generates the enhanced image in units of a plurality of frames of the input image, and synthesizes the same enhanced image with the input image of the plurality of frames. Processing equipment. The input image is a color image;
The image pattern of the emphasized image is assigned to each of two or more colors,
The binarizing means extracts a component of each of the two or more colors from the input image to generate a single color image,
The high luminance area specifying means specifies the high luminance area for each single color image,
The priority determination means determines the priority for each single color image,
The selection means performs the selection of the high luminance region for each single color image,
The said image processing means produces | generates the highlight image which highlights the said selected high-intensity area | region for every said single color image, and synthesize | combines the said highlight image with the said input image. Image processing apparatus. Comprising face detection means for detecting a face image of a person from the input image,
The image processing apparatus according to claim 1, wherein the image processing unit overwrites the face image extracted by the face detection unit on a composite image generated by combining the emphasized image with the input image. . Binarizing an input image with a predetermined threshold;
Identifying a high brightness region from the binarized image;
Determining a priority for the identified high brightness area based on a predetermined condition;
When there are a predetermined number or more of the specified high-intensity areas, the step of selecting the predetermined number in order from the high-intensity areas with high priority;
Generating an emphasized image that highlights the selected high-luminance region, and synthesizing the emphasized image with the input image. An image processing device for processing an input image from the solid-state image sensor;
The image processing apparatus includes:
Binarization means for binarizing the input image with a predetermined threshold;
High luminance area specifying means for specifying a high luminance area from the binarized image;
Priority determining means for determining a priority based on a predetermined condition for the identified high-luminance region;
A selection means for selecting the predetermined number in order from a high-luminance area having a higher priority when the specified high-luminance area is a predetermined number or more;
An imaging apparatus comprising: an image processing unit that generates an emphasized image that highlights the selected high-luminance region and combines the emphasized image with the input image.