JP5058128B2 - Imaging device and portable device - Google Patents

Description

  The present invention relates to an imaging device having a function of capturing a color image, a symbol recognition function such as a two-dimensional code, and a character recognition function, and a portable device including the imaging device.

  In recent years, many portable devices such as mobile phones that have rapidly spread are equipped with a small camera. Many of such small cameras have a function of recognizing a symbol such as a two-dimensional code such as a barcode or QR code (registered trademark) or a function of recognizing character information in addition to a function of capturing a normal color image. Etc. are installed.

  A small camera includes an image sensor such as a CCD or a CMOS, and a color photographed image includes, for example, Bayer data including luminance information output from each of R, G, and B pixels arranged in a Bayer array as RGB signals and YUV signals. It is output through signal processing for conversion.

  A camera having a symbol recognition function or character recognition as described above is conventionally recognized by using a color image captured in the same manner as when performing normal color shooting even when performing symbol recognition or character recognition. Processing was in progress.

  On the other hand, when recognizing a QR code or a barcode, an image pickup apparatus that performs a recognition process using only a signal output from a specific pixel, for example, a G pixel, among R, G, and B pixels. Is disclosed (see Patent Documents 1 and 2).

In addition, a solid-state imaging device using white pixels in addition to RGB pixels for obtaining a video signal excellent in color reproducibility under low illuminance has been disclosed (see Patent Document 3).
JP 2005-286536 A JP 2006-195961 A JP 2008-22521 A

  However, when a camera that captures a color image using the techniques of the conventional patent documents 1 to 3 is also used for symbol recognition or character recognition such as a barcode or QR code, it cannot be recognized with high accuracy. There is.

  For example, in a low illuminance environment, since the difference in signal level between the bright part and the dark part is small, it becomes difficult to distinguish between light and dark. Further, in the case of an image sensor having an RGB pixel with a Bayer array, an output signal in a certain pixel (any one of R, G, and B) needs to interpolate a color from the surrounding pixels of the pixel, and the output signal is low-pass This is equivalent to passing through a filter, and further reduces the difference between light and dark levels. Further, when performing symbol recognition or character recognition, when the user focuses on the imaging target and performs imaging, the user may not be in focus or the entire imaging target within the imaging range may not be included. Furthermore, the frame rate of the signal output from the image sensor is slow, and it takes a relatively long time to perform recognition processing. These factors are the main causes of the decrease in the recognition rate of symbol recognition and character recognition. Yes.

  Hereinafter, an example in which symbol recognition or character recognition is performed using an output signal from an image sensor with a Bayer array will be described. FIG. 9 is an explanatory view showing an example of a conventional primary color type Bayer array image sensor. As shown in FIG. 9, the primary color Bayer array has sensitivity in only one wavelength region per pixel, and therefore it is necessary to interpolate with peripheral pixels to make the entire visible wavelength region. FIG. 9A shows a case where the center pixel is a B pixel having sensitivity to light having a wavelength near 436 nm, and FIG. 9B shows that the center pixel has sensitivity to light having a wavelength near 546 nm and is horizontal. FIG. 9C shows the case of a Gb pixel adjacent to the B pixel in the direction (horizontal direction). FIG. 9C shows the sensitivity of the Gr pixel adjacent to the R pixel in the horizontal direction (horizontal direction). FIG. 9D shows a case where the center pixel is an R pixel having sensitivity to light having a wavelength of around 700 nm, and in each case, interpolation is performed using peripheral pixels of 3 × 3 pixels.

  In the case of FIG. 9A, R value, G value, and B value are R = (R1 + R2 + R3 + R4) / 4, G = (G1 + G2 + G3 + G4) / 4, B = B1. It can ask for.

  In the case of FIG. 9B, the R value, G value, and B value can be obtained by R = (R1 + R2) / 2, G = G3, and B = (B1 + B2) / 2.

  In the case of FIG. 9C, the R value, G value, and B value can be obtained by R = (R1 + R2) / 2, G = G3, and B = (B1 + B2) / 2.

  In the case of FIG. 9D, R value, G value, and B value are R = R1, G = (G1 + G2 + G3 + G4) / 4, B = (B1 + B2 + B3 + B4), respectively. / 4.

  In the case of converting the RGB value obtained for each pixel into a YUV value (luminance and color difference), Y = 0.299 × R + 0.587 × G + 0.114 × B, U = −0.147 × R-0. 289 × G + 0.436 × B, V = 0.615 × R−0.515 × G−0.100 × B.

  In the symbol recognition process and the character recognition process, the bright part and the dark part are represented by, for example, “1” and “0” data. Therefore, in the recognition process, a predetermined value for the luminance value Y obtained by the above formula is used. Binarization is performed with a threshold value, and it is determined whether the acquired data is a bright part or a dark part. Therefore, in the process of converting the Bayer data output from the image sensor to RGB, the level difference between the bright part and the dark part is the same as the state where the low-pass filter is applied by interpolation using the peripheral pixels of the upper, lower, left, and right pixels. The symbols and characters cannot be recognized with high accuracy.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an imaging apparatus capable of accurately recognizing symbols and characters and a portable device including the imaging apparatus.

An image pickup apparatus according to the present invention includes an image pickup device having an image pickup element having a specific color pixel in which a plurality of color filters of a specific color are arranged in units of elements on a plurality of photoelectric conversion elements. A display means for displaying a color image to be picked up using an output signal from a pixel that includes at least a part of the specific color pixel and an output signal from the white pixel. Extraction means for extracting information included in the target monochrome image, a plurality of first pixel lines in which white pixels and specific color pixels of the first color are alternately arranged in the longitudinal direction, and the first of the white pixels A specific color pixel of a second color provided at a position corresponding to a position on one pixel line, and a white pixel provided at a position corresponding to a position on the first pixel line of the specific color pixel of the first color Alternately A plurality of second pixel lines arranged in the longitudinal direction of the first pixel line, and switching means for switching an output destination of output signals from the pixels of the first pixel line and the second pixel line to either the display means or the extraction means with the door, said switching means constitutes an output destination to switch alternately in frame units of the captured image, wherein the tare Rukoto.

  An imaging apparatus according to the present invention includes binarization means for binarizing the monochromatic image, and the extraction means is based on the binarized image binarized by the binarization means. It is configured to extract character information.

  An imaging apparatus according to the present invention includes binarization means for binarizing the monochromatic image, and the extraction means is based on the binarized image binarized by the binarization means. It is configured to extract information included in the two-dimensional code.

Imaging device according to the present invention, prior Symbol display means, by using the output signals from the pixels of the first pixel line and the second pixel line, Yes and configured to display a color image of the imaging object, wherein The extraction means is configured to extract information using an output signal from a white pixel among the pixels of the first pixel line and the second pixel line.

  The imaging apparatus according to the present invention is characterized in that the display means is configured to display a color image having an image size smaller than that of the monochrome image.

  The imaging device according to the present invention is characterized in that the plurality of specific colors are red, green and blue or their complementary colors.

  A portable device according to the present invention includes an imaging device according to any one of the above-described inventions.

In the present invention, a white pixel that is not provided with a color filter, a display unit that displays a color image to be imaged using an output signal from a pixel that includes at least a part of a specific color pixel, and a white pixel Extraction means for extracting information contained in a monochrome image to be imaged using an output signal from The information included in the monochromatic image is, for example, information included in a two-dimensional code such as a QR code or a barcode, character information, and the like. That is, for symbol recognition and character recognition, the output signal from a white pixel without a color filter is used as it is, so that it is not necessary to perform interpolation processing using peripheral pixels, and a color filter is provided. Therefore, it is possible to perform high-sensitivity imaging as compared with the case where RGB pixels are used, and it is possible to recognize symbols and characters with high accuracy. In addition, since the color image of the imaging target can be displayed, the user can adjust the focus of the imaging target while viewing the displayed color image, or adjust the entire imaging target to be within the imaging range. The recognition rate of symbols and characters can be further increased.
Also, a plurality of first pixel lines in which white pixels and a specific color pixel of the first color are alternately arranged in the longitudinal direction, and a first pixel line provided at a position corresponding to a position on the first pixel line of the white pixel. A plurality of second pixel lines in which white pixels provided at positions corresponding to positions on the first pixel lines of the two-color specific color pixels and the first color specific color pixels are alternately arranged in the longitudinal direction; Is provided. Then, the output destination of the output signal from the pixels of the first pixel line and the second pixel line is alternately switched to either the display unit or the extraction unit in units of frames of the captured image. For example, the first frame is a display image for color display, and the second frame is a monochrome image for character or symbol recognition. Alternatively, the first frame is a display image for color display, and the second and third consecutive frames are monochromatic images for character and symbol recognition. Thereby, the user can confirm the imaging target, and the recognition rate of characters and symbols can be increased .

  In the present invention, the monochrome image based on the output signal from the white pixel is binarized, and character information in the imaging target is extracted based on the binarized binarized image. Thereby, characters can be recognized with higher accuracy than in the case of using RGB pixels.

  In the present invention, a monochromatic image based on an output signal from a white pixel is binarized, and based on the binarized binarized image, a two-dimensional code (for example, a barcode or a QR code) in an imaging target is obtained. Information included in (symbol). As a result, the symbols can be recognized with higher accuracy than when RGB pixels are used.

In the present invention, by using the output signals from the pixels of the first pixel line and the second pixel line, and displays a color image of the imaging object, white among the pixels of the first pixel line and the second pixel line Information included in the monochrome image to be imaged is extracted using an output signal from the pixel. For example, the first color is R, the second color is B, and the first pixel line is arranged along the line direction as W pixel, R pixel, W pixel, R pixel, etc., and the second pixel line is , B pixels, W pixels, B pixels, W pixels... Along the line direction. In this case, an output signal from the W pixel is used to extract information included in the monochrome image to be imaged. In order to display a color image, output signals from the W pixel, the R pixel, and the B pixel are used. In this case, the G value can be obtained by G = (W−j × R−k × B) / l. However, j, k, and l are predetermined constants. Thus, the same pixel line can be used for displaying a color image and extracting information, for example, a color imaging mode for acquiring a color image and a monochrome imaging mode for recognizing characters and symbols. As described above, it is not necessary to change the imaging mode, and it can be realized with a simple configuration, and the control of the output signal can be simplified.

  In the present invention, the display means displays a color image having an image size smaller than that of the single color image. This makes it possible to display a color image of a size necessary for the user to confirm the imaging target, shorten the time required for display, and improve the real-time property felt by the user.

  In the present invention, the plurality of specific colors are red (R), green (G), and blue (B) or their complementary colors (for example, Y, M, C). If the white pixel is W, the color image can use, for example, output signals from RGB, RGW, GBW, and RBW pixels, and information such as symbols and character recognition can be extracted from the W pixel. A signal can be used.

  In the present invention, a portable device includes the above-described imaging device. Thereby, in addition to the imaging function of a color image, the portable apparatus provided with the function which can recognize a symbol and a character accurately can be provided.

  According to the present invention, symbols and characters can be recognized with high accuracy.

  Hereinafter, the present invention will be described with reference to the drawings illustrating embodiments thereof. FIG. 1 is a block diagram illustrating an example of a main configuration of a mobile phone 100 as a mobile device including an imaging apparatus according to the present invention. The mobile phone 100 includes an optical system 10 including a lens for forming an image of incident light from the outside, an imaging element 11 for converting light passing through the optical system 10 into an electrical signal corresponding to the amount of light, and the imaging element 11. A signal processing unit that controls the AFE (analog front end) 12 that converts the output analog electrical signal into digital data (image data), the image sensor 11 and the AFE 12, and performs predetermined processing on the image data output from the AFE 12. 13, CPU 14 for controlling the operation of the signal processing unit 13, an image memory 15 for storing image data in units of frames of the captured image, a display unit 16 such as a liquid crystal display, the CPU 14 and the signal processing unit 13, the image memory 15, and a display A data bus 20 for transmitting / receiving data to / from the unit 16 is provided.

  The signal processing unit 13 performs various processes such as a shading correction process and a gamma correction process.

  The CPU 14 generates RGB data for displaying a color image, YUV data, an NTSC video signal, and the like based on image data in units of frames stored in the image memory 15. In this case, the CPU 14, as will be described later, out of the image data extracted from the image memory 15, an output signal (a pixel line of the pixel line) including a part of a specific color pixel (may include a W pixel described later). Data).

  The CPU 14 extracts a luminance signal from the image data in units of frames stored in the image memory 15, binarizes the extracted luminance signal with a predetermined luminance threshold, and based on the binarized binary image, character information Extraction (character recognition) and / or extraction (symbol recognition) of information included in a two-dimensional code (symbol) such as a QR code (registered trademark) or a barcode. In this case, the CPU 14 uses an output signal (pixel line data) from a W pixel as a white pixel, which will be described later, among the image data extracted from the image memory 15.

  The CPU 14 also serves as a switching means for switching whether to use the image data in units of frames stored in the image memory 15 for color image display processing or recognition processing such as symbol recognition or character recognition in units of frames. It has a function.

  FIG. 2 is an explanatory diagram showing a main configuration of the image sensor 11 of the present embodiment. The image sensor 11 includes a plurality of photoelectric conversion elements arranged in a planar shape. A plurality of red (R), green (G), and blue (B) color filters (wavelength selection filters) as specific colors are arranged on the photoelectric conversion element in units of pixels. The red color filter has high sensitivity to light in the red wavelength region (near 700 nm), transmits only red light to the photoelectric conversion element, and constitutes an R pixel together with the photoelectric conversion element. Similarly, the green color filter has high sensitivity to light in the green wavelength range (near 546 nm), transmits only green light to the photoelectric conversion element, and constitutes a G pixel together with the photoelectric conversion element. The blue color filter has high sensitivity to light in the blue wavelength region (near 436 nm), transmits only blue light to the photoelectric conversion element, and constitutes the B pixel together with the photoelectric conversion element. The R pixel, the G pixel, and the B pixel described above are specific color pixels. Instead of the R, G, and B color filters, these complementary color (for example, Y, M, and C) color filters may be used.

  Further, the image sensor 11 has W pixels as white pixels not provided with the above-described color filter. That is, the W pixel has higher sensitivity to light in all visible regions, and is more sensitive than red pixels to light in the red wavelength region than G pixels to light in the green wavelength region. , It has higher sensitivity than the B pixel with respect to light in the blue wavelength region. Since the W pixel has no wavelength selection filter at the top, it can detect light in the visible region with good S / N.

  In addition, although the difference in wavelength cannot be detected, the difference between the bright and dark areas can be increased even with a colorful imaging target (colorful subject) if the incident light has a spectrum with a peak in a certain wavelength range. Can be identified by sensitivity. Speaking of normal monochrome symbol recognition and character recognition, it is a black and white object, but even a colorful single-color imaging object can be recognized. In the present embodiment, a symbol of a specific color is also a monochrome symbol. For example, even with a green QR code, the sensitivity of the W pixel to green (G) light is higher than the sensitivity of the G pixel. Therefore, it is possible to realize a higher recognition rate than monochrome symbol recognition using G pixels.

  Further, since the signal output from the W pixel is a luminance signal, symbol recognition and character recognition can be performed without performing the color interpolation processing performed in the case of a normal primary color Bayer array. Therefore, it is possible to eliminate the influence of the low-pass filter due to the interpolation process, which is the current problem.

  FIG. 3 is an explanatory diagram illustrating an example of an array of pixels of the image sensor 11 according to the present embodiment. In the figure, W, R, G, and B represent the above-described W pixel, R pixel, G pixel, and B pixel, respectively. FIG. 3 shows a part of the pixels constituting the image sensor 11. In the line a (first pixel line), two pixels of W pixel, R pixel, W pixel, R pixel,... Are alternately arranged along the line direction (lateral direction, horizontal direction). In addition, the line b has two pixels B, G, B, G,... Arranged alternately along the line direction (lateral direction, horizontal direction), and the line c is the line direction. Two pixels of G pixel, R pixel, G pixel, R pixel,... Are alternately arranged along (lateral direction, horizontal direction). In the line d (second pixel line), two pixels of B pixel, W pixel, B pixel, W pixel,... Are alternately arranged along the line direction (lateral direction, horizontal direction). In this case, the B pixel of the line d is provided at a position corresponding to the position on the line of the W pixel of the line a. The W pixel on the line d is provided at a position corresponding to the position on the line of the R pixel on the line a. The lines a to d are repeatedly arranged along the vertical direction (vertical direction, direction orthogonal to the line direction).

  An output signal from the image sensor 11 composed of W pixels, R pixels, G pixels, and B pixels is temporarily stored in the image memory 15 as image data for each frame under the control of the CPU 14. The CPU 14 retrieves the image data once stored from the image memory 15 and performs color image display processing and recognition processing such as symbol recognition and character recognition.

  When the lines b, c, b (or the lines c, b, c) are taken out, the pixels constituted by the lines are in a Bayer array.

  The CPU 14 extracts the data of the line a as the first pixel line and the data of the line d as the second pixel line from the above-described lines of the image sensor 11, and uses output signals from the pixels of each line. Generating color image data for displaying a color image and monochrome image data for symbol recognition or character recognition. Hereinafter, a method for generating color image data and monochrome image data using the pixel configuration of the present embodiment will be described.

  FIG. 4 is an explanatory diagram illustrating an example of a pixel configuration according to this embodiment. As shown in FIG. 4, the pixel configuration of the present embodiment includes a line a as the first pixel line and a line d as the second pixel line, and the lines a and d are repeated in this order. It is composed of W pixels, R pixels, and B pixels. That is, the line a as the first pixel line and the line d as the second pixel line are alternately arranged along the direction orthogonal to the line direction. FIG. 4 shows only a 4 × 4 pixel block for the sake of simplicity.

  FIG. 5 is an explanatory diagram illustrating a 3 × 3 pixel block having a pixel configuration according to the present embodiment. The pixel block shown in FIG. 5 is obtained by extracting 3 × 3 pixels from the pixel configuration shown in FIG. 5A shows a case where the central pixel is a B pixel, FIG. 5B shows a case where the central pixel is a W pixel, and a Wb pixel adjacent to the B pixel in the line direction (lateral direction). 5C shows the case where the central pixel is a W pixel and the Wr pixel is adjacent to the R pixel in the line direction (lateral direction), and FIG. 5D shows the case where the central pixel is the R pixel.

  In the case of FIG. 5A, the R value, W value, and B value at the center pixel are R = (R1 + R2 + R3 + R4) / 4, W = (W1 + W2 + W3 + W4) / 4, B = B1. (18)

  In the case of FIG. 5B, the R value, W value, and B value at the center pixel are R = (R1 + R2) / 2, W = W3, and B = (B1 + B2) / 2, respectively. Can be sought.

  In the case of FIG. 5C, the R value, W value, and B value at the center pixel are R = (R1 + R2) / 2, W = W3, and B = (B1 + B2) / 2, respectively. Can be sought.

  In the case of FIG. 5D, the R value, W value, and B value at the central pixel are R = R1, W = (W1 + W2 + W3 + W4) / 4, and B = (B1 + B2), respectively. + B3 + B4) / 4.

  That is, the image data of a single color image can be obtained from each W value described above. Further, since the W value described above is a luminance value, the G value (green of the three primary colors) can be obtained by G = (W−jR−kB) / l. Here, W, R, and B are output values from the W pixel, R pixel, and B pixel, respectively, and j, k, and l are predetermined constants.

  Furthermore, each of the color difference signals U and V can be obtained by U = 0.492 × (B−W) and V = 0.877 × (R−W) using the W value, the R value, and the B value.

  Information included in a two-dimensional code such as a QR code or a bar code can be extracted (symbol recognition) or character information can be extracted (character recognition) using image data of a single color image. That is, in symbol recognition and character recognition, an output signal from a W pixel without a color filter is used as it is, so that it is not necessary to perform interpolation processing using peripheral pixels, and a color filter is provided. Therefore, it is possible to perform high-sensitivity imaging as compared with the case where RGB pixels are used, and it is possible to recognize symbols and characters with high accuracy.

  In particular, by binarizing a monochromatic image based on an output signal from a W pixel and extracting character information in the imaging target based on the binarized binarized image, accuracy is higher than when using RGB pixels. Can recognize characters well.

  In addition, the monochrome image based on the output signal from the W pixel is binarized and included in the two-dimensional code (for example, a symbol such as a barcode or QR code) in the imaging target based on the binarized binarized image. By extracting the information, it is possible to recognize the symbols with higher accuracy than when using RGB pixels.

  Further, as described above, since the G value can be obtained from the W, R, and B values, the color image to be captured can be displayed, and the user can focus on the captured image while viewing the displayed color image. And the entire imaging target can be adjusted within the imaging range, and the recognition rate of symbols and characters can be further increased.

  Thus, in order to extract the information included in the monochrome image to be imaged, the output signal from the W pixel can be used. In order to display a color image, output signals from the W pixel, the R pixel, and the B pixel can be used. As a result, the same pixel line can be used both when displaying a color image and when extracting information such as symbol recognition and character recognition. For example, a color imaging mode and a character or symbol for acquiring a color image can be used. Unlike the monochrome imaging mode for recognizing the image, it is not necessary to change the imaging mode, and it can be realized with a simple configuration, and the control of the output signal can be simplified.

  In the above example, the G pixel is replaced with the W pixel, but the B pixel or the R pixel may be replaced with the W pixel. Further, a configuration in which a part of another type of image sensor (for example, a complementary color image sensor) is replaced with a W pixel may be used.

  When generating color image data, data of another pixel line can be used instead of using the data of line a and line d described above. FIG. 6 is an explanatory diagram showing an example of a pixel configuration for generating color image data. As shown in FIG. 6, when color image data is generated, the line b and the line c are repeated in this order, and are composed of R pixels, G pixels, and B pixels. Further, in FIG. 6, only a 4 × 4 pixel block is shown for simplification.

  In the case of this configuration, for symbol recognition and character recognition, as shown in FIG. 4, monochromatic image data is generated from output signals from W pixels on line a and line d. On the other hand, in order to display a color image, as shown in FIG. 6, color image data is generated by output signals from the lines b and c. Thereby, the display of a color image can be processed with the same algorithm as the conventional pixel configuration.

  Even in a pixel configuration including W pixels, there are cases where a large amount of data is required for color image display. FIG. 7 is an explanatory diagram showing an example of an RGB pixel configuration including W pixels. In the pixel configuration as shown in FIG. 7, although symbol recognition and character recognition can be performed using the output signal from the W pixel, the pixel line is a combination of the W pixel and the R pixel, or the G pixel and the B pixel. For example, when data is taken out (read) every other line, color information cannot be obtained. That is, in reading G pixels and B pixels, R pixel components are lost, and in reading W pixels and R pixels, G pixel and B pixel components cannot be separated. Therefore, information of at least three areas (for example, lines e, f, and g) is required for color image display among the four wavelength areas (W pixel, G pixel, B pixel, and R pixel). In this regard, there is no such problem in the example of FIG.

  Next, the timing of color image display processing and recognition processing such as symbol recognition and character recognition will be described. When recognizing symbols or characters, the recognition target is imaged to check whether the recognition target (imaging target) is within the imaging range or whether the recognition target is in focus. Need to be displayed. If the displayed image is not a color image according to the recognition target (subject) that the user actually sees, the user will feel confused and uncomfortable. Therefore, it is necessary to use a binarized single color image for symbol recognition and character recognition, and to display a color image similar to the normal camera mode for image display. Switch.

  In order to improve the recognition rate of symbol recognition and character recognition, there are a method of improving the recognition rate per frame, a method of increasing the number of frames used for recognition processing, a method of improving user operability, etc. is there.

  In order to increase the recognition rate by improving user operability, for example, there is a method of performing recognition processing while displaying a recognition target (subject) on a display unit. By displaying the recognition target as a color image, it is possible to easily confirm whether the recognition target is in focus or whether the recognition target is correctly within the imaging range. Even if the recognition target is an achromatic color such as black and white, the display is preferably a color image so that it can be easily confirmed what type of recognition target is in the imaging range. .

  FIG. 8 is a timing chart showing the relationship between display processing and recognition processing. In FIG. 8, the upper stage is an output signal (image data) from the image sensor 11, the middle stage is a display signal (display image) output to the display unit 16, and the lower stage is a recognition signal (recognition image, Monochrome image). As shown in FIG. 8, the output signal output in units of one frame is switched between a display image and a recognized image in units of frames. Thereby, the user can confirm the imaging target (subject, recognition target) and can increase the recognition rate of characters and symbols.

  In the example of FIG. 8, the display signal and the recognition signal are alternately switched in units of frames, but the present invention is not limited to this. As a method of alternately switching the output destination of the output signal for each frame of the captured image, the first frame is a display image for color display, and the second frame is a single color image for character or symbol recognition. In addition, the first frame may be a display image for color display, and the second and third consecutive frames may be single-color images for character and symbol recognition. As a result, the number of recognition frames can be increased, the recognition time can be shortened, and the recognition rate can be improved.

  Further, instead of alternately switching output destinations in units of frames, output signals in units of frames can be simultaneously used as display signals (display images) and recognition signals (recognition images).

  Also, since the image size that can be displayed on the display unit is usually smaller than the number of pixels used for symbol recognition and character recognition, only the output signal from the image sensor 11 to the W pixel is used for symbol recognition and character recognition. When displaying on the display unit 16, the image size is further reduced, and the image size necessary for display on the display unit 16 is increased, thereby increasing the number of frames output from the image sensor 11 and reducing the recognition time. It is also possible to improve the real-time property felt by the user by the display.

  As described above, according to the present invention, peripheral pixels as in the past can be used for symbol recognition and character recognition by directly using output signals from white pixels (W pixels) that are not provided with a color filter. Since no interpolation processing is required and no color filter is provided, it is possible to capture images with higher sensitivity than in the case of using RGB pixels, and it is possible to recognize symbols and characters with high accuracy. In addition, since the color image of the imaging target can be displayed, the user can adjust the focus of the imaging target while viewing the displayed color image or adjust the entire imaging target within the imaging range. And the recognition rate of symbols and characters can be further increased.

  In the above-described embodiment, a mobile phone has been described as an example. However, the imaging device according to the present invention is not limited to a mobile phone, and may be applied to any mobile device equipped with a camera. it can. Thereby, in addition to the imaging function of a color image, the portable apparatus provided with the function which can recognize a symbol and a character accurately can be provided.

It is a block diagram which shows an example of a principal part structure of a mobile telephone as a portable apparatus provided with the imaging device which concerns on this invention. It is explanatory drawing which shows the principal part structure of the image pick-up element of this Embodiment. It is explanatory drawing which shows an example of the arrangement | sequence of the pixel of the image pick-up element of this Embodiment. It is explanatory drawing which shows an example of the pixel structure of this Embodiment. It is explanatory drawing which shows the pixel block of 3x3 pixel of the pixel structure of this Embodiment. It is explanatory drawing which shows an example of the pixel structure for producing | generating color image data. It is explanatory drawing which shows the example of the RGB pixel structure containing W pixel. It is a timing chart which shows the relationship between a display process and a recognition process. It is explanatory drawing which shows an example of the image pick-up element of the conventional primary color type Bayer arrangement.

Explanation of symbols

10 Optical System 11 Image Sensor 12 AFE
13 Signal processor 14 CPU
15 Image memory 16 Display section

Claims (7)

In an image pickup apparatus including an image pickup element having a specific color pixel in which a plurality of color filters of a specific color are arranged in units of elements on a plurality of photoelectric conversion elements,
A white pixel not provided with the color filter;
Display means for displaying a color image to be imaged using an output signal from a pixel including at least a part of the specific color pixel;
Extraction means for extracting information contained in a monochrome image to be imaged using an output signal from the white pixel ;
A plurality of first pixel lines in which white pixels and specific color pixels of a first color are alternately arranged in the longitudinal direction;
A specific color pixel of a second color provided at a position corresponding to a position on the first pixel line of the white pixel, and a position corresponding to a position on the first pixel line of the specific color pixel of the first color A plurality of second pixel lines in which the white pixels provided in are alternately arranged in the longitudinal direction;
Switching means for switching an output destination of output signals from the pixels of the first pixel line and the second pixel line to either the display means or the extraction means ;
The switching means is
Configuration imaging apparatus according to claim tare Rukoto an output destination to switch alternately in frame units of the captured image. Binarizing means for binarizing the monochromatic image;
The extraction means includes
The imaging apparatus according to claim 1, wherein character information in the imaging target is extracted based on the binarized image binarized by the binarization unit. Binarizing means for binarizing the monochromatic image;
The extraction means includes
The imaging according to claim 1, wherein information included in a two-dimensional code in an imaging target is extracted based on the binarized image binarized by the binarization unit. apparatus. Before Symbol display means,
It is configured to display a color image to be imaged using output signals from the pixels of the first pixel line and the second pixel line,
The extraction means includes
The information is extracted by using an output signal from a white pixel among the pixels of the first pixel line and the second pixel line. The imaging apparatus according to item 1. The display means includes
The imaging apparatus according to any one of claims 1 to 4 , wherein the imaging apparatus is configured to display a color image having a smaller image size than the monochrome image. The plurality of specific colors are:
Red, green and blue or the image pickup apparatus according to any one of claims 1, characterized in that it is these complementary to claim 5. A portable device comprising the imaging device according to any one of claims 1 to 6 .

Images