JP4355305B2 - Image processing apparatus, computer program, and recording medium - Google Patents

Description

  The present invention relates to image data processing technology, and more particularly to an image processing apparatus for optimizing image data obtained by photographing a screen of a matrix type display device composed of a plurality of dots arranged in a matrix, and for the same The present invention relates to a computer program and a recording medium.

  In recent years, the technology for converting visual information such as images and characters into digital data by digitalizing information has made remarkable progress and widespread use. Along with this, the opportunity to record images and characters converted into digital data and the opportunity to display images and characters converted into digital data both increased.

  As an example of the spread of technology for recording images and characters as digital data, the spread of digital cameras can be mentioned. With the high performance and widespread use of image sensors and the development of signal processing technology, digital cameras that convert captured images into digital data for recording have become widespread. Not only a digital camera as a device dedicated to photographing but also a digital camera built in an information device such as a mobile phone has already been generalized. In addition, image sensors made up of CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) have become more sophisticated, and high-resolution image sensors with millions of millions of pixels have become commonplace. . It is expected that the number of pixels of the image sensor will increase further in the future. By taking a picture with a digital camera equipped with an image sensor with an enormous number of pixels in this way, it is easy to obtain high-definition or higher-quality digital data that is comparable to silver halide photography. Obtainable.

  In addition to simply reproducing and browsing digital data (hereinafter referred to as “image data”) of an image obtained by photographing, a technique for obtaining other information from the image data is becoming widespread. Patent Document 1 discloses a technique for performing character recognition using a photographing function of a camera-equipped mobile phone. In the technique described in Patent Document 1, a medium such as paper and a blackboard on which characters are described (hereinafter referred to as “character medium”) is photographed with a camera-equipped mobile phone, and the image data is transmitted to a character image processing apparatus. To do. When receiving the input of image data, the character image processing device performs character recognition based on the image data, encodes characters included in the image, and obtains character code data. Japanese Patent Application Laid-Open No. H10-228561 also discloses correcting distortion generated in an image due to lens aberration or the like during character recognition.

  As an example of the spread of technology for displaying digitalized images, characters, and the like, a matrix display device including a plurality of pixels (dots) arranged in a matrix can be cited. Due to the high performance and widespread use of display panels such as liquid crystal display panels, thin and high-definition display devices have become widespread. Among them, particularly recent liquid crystal display panels have improved contrast and increased reaction speed, and are widely used as display devices for television receivers, mobile phones, personal computers, and the like. Since a display device using a liquid crystal display panel is thin, even a display device having a large display surface can be installed in various forms such as hanging on a wall. Therefore, a display device using a liquid crystal display panel is often used as a medium for guidance and advertisement in public places. In the future, it is expected that liquid crystal display panels will be used for displaying images and characters in various places.

JP 2004-118563 A

  Current digital camera shooting targets are mainly landscapes and people. However, in view of the current state of widespread use of liquid crystal display panels, it is expected that opportunities for photographing a display screen on the liquid crystal display panel using a digital camera will increase in the future. Furthermore, it is expected that other information is obtained from image data obtained by photographing.

  FIG. 15 shows an image 600 obtained by photographing the screen of the color liquid crystal display panel with a digital camera equipped with a CCD. FIG. 16 shows an image 610 in which the vicinity of a portion 602 surrounded by a circle in the photographed image 600 shown in FIG. 15 is enlarged. Referring to FIGS. 15 and 16, images 600 and 610 have many blurred outlines and noise, and have a periodic pattern in the background, making them difficult to see.

Hereinafter, with reference to FIGS. 17 to 20, a periodic pattern in an image obtained by photographing a display on the liquid crystal display panel with a camera equipped with a CCD will be described. FIG. 17 schematically shows a captured image 610 of the color liquid crystal display panel shown in FIG. FIG. 18 shows an enlarged image 620 near a portion 612 surrounded by a circle in FIG. FIG. 19 schematically shows an image 630 obtained by photographing the same display as the image 610 shown in FIG. 16 on the monochrome liquid crystal display panel. FIG. 20 shows an enlarged image 640 in the vicinity of the portion 632 corresponding to the portion 612 surrounded by circles of Oite Figure 17 in an image 630 shown in FIG. 19. In these figures, the outline blur and noise seen in FIG. 16 are removed, and the background pattern is emphasized.

  In the background portion of the image 610 shown in FIG. 17, black lines extending in the horizontal direction are regularly displayed vertically. In addition, a lattice-like pattern is shown in the background portion of the image 630 shown in FIG. These black lines or grid patterns are images of the black mask of the liquid crystal display panel. The black mask is a protective mask for protecting the signal wiring and the semiconductor.

  In the image 610 shown in FIG. 17, an image in which a set of three color images of red (R), green (G), and blue (B) is arranged is shown between the images of the black mask. A set of three colors of red (R), green (G), and blue (B) is an image of a pixel (dot) of a color liquid crystal display panel. A dot is a minimum unit in display on a liquid crystal display panel. The dots of the color liquid crystal display panel have a structure in which three display elements composed of light emitting portions corresponding to three additive primary colors red (R), green (G), and blue (B) are combined. . The dots of the color liquid crystal display panel express one color based on the principle of additive color mixing when each display element emits light appropriately. In the image 630 shown in FIG. 19, the images between the lattices are dot images. The dots of the monochrome liquid crystal display panel have a display element composed of one light emitting unit. The dots of the black-and-white liquid crystal display panel express one gradation of light and dark when the light emitting unit emits light appropriately.

  The liquid crystal display panel has been improved in definition, and normally the display elements of the black mask and the dot are not recognized independently by human eyes. However, an image sensor such as a CCD has been improved in resolution as described above, and an extremely fine image can be expressed by image data obtained by photographing. Referring to FIGS. 18 and 20, the fine lattice patterns in these images 620 and 640 are image ranges represented by each of the pixel value data (pixels) which is the minimum unit of image representation in the image data. Is schematically represented. An actual image does not have such a lattice pattern. As shown in FIGS. 18 and 20, in the images 620 and 640, one dot on the liquid crystal display panel is expressed by a large number of pixels. The black mask and the dot are represented by separate pixels, and the black mask and the display element are recorded so as to be distinguishable. Therefore, there arises a problem that the recorded image becomes a difficult-to-see image as shown in FIGS.

  In addition, there is a problem that the information amount of the photographed image increases with respect to the information amount displayed on the liquid crystal display panel. The liquid crystal display panel expresses one color or one gradation per dot. However, in image data obtained by photographing, as shown in FIGS. 18 and 20, one dot of liquid crystal is expressed by a plurality of pixels. In FIG. 18 and FIG. 20, one dot of the liquid crystal appears in the range of 9 pixels wide and 9 pixels high in the image data. That is, a pixel value for 81 pixels is used to express one piece of information. Further, even when characters are displayed on the liquid crystal screen using two gradations of black and white, generally, on the image data output by the CCD, 24 bits (red (R), green (G), and Blue (B) is 8 bits each), and further information enlargement occurs.

  It is possible to reduce the information amount of the image data by general processing such as reduction and compression. However, in general processing, the black mask and color information of the liquid crystal display panel are also reflected in the processed image data, so that a clear image cannot be obtained and there is a limit in reducing the amount of information. is there.

  Furthermore, there is a problem that it is difficult to obtain other information from image data obtained by photographing. Consider a case in which character recognition is performed using an image of characters displayed on a liquid crystal display panel. When character recognition is performed, image binarization is generally performed as preprocessing. However, since an image obtained by photographing the screen of the liquid crystal display panel has a black mask and a dot pattern, it is difficult to perform appropriate binarization. For example, if the image 630 shown in FIG. 19 is binarized as it is, the pattern of the black mask is converted to the same gradation (black) as the character, which hinders character recognition. Patent Document 1 does not mention that character recognition is performed using an image obtained by photographing a screen of a liquid crystal display panel. For this reason, the technique described in Patent Document 1 cannot solve the above-described problem that occurs when an image obtained by photographing the screen of the liquid crystal display panel is binarized, and an appropriate binarized image can be obtained. Therefore, the accuracy of character recognition may be reduced.

  Therefore, an object of the present invention is to provide an image processing apparatus that can obtain a clear image from image data obtained by photographing a screen of a liquid crystal display panel or the like.

  Another object of the present invention is to provide an apparatus capable of optimizing image data by reducing the data amount of the image without impairing information on the image obtained by photographing a screen of a liquid crystal display panel or the like. .

  Still another object of the present invention is to provide an apparatus capable of recognizing characters displayed on a liquid crystal display panel or the like with high accuracy.

  An image processing apparatus according to a first aspect of the present invention is an apparatus for encoding an image obtained by photographing a screen of a matrix type display device composed of a plurality of dots arranged in a matrix. In response to the image being given, the image processing apparatus detects a periodic pattern that repeatedly appears on the image, and detects the periodic pattern in the image based on the periodic pattern detected by the pattern detecting means. A feature amount extracting means for specifying a region corresponding to each dot image and extracting a predetermined feature amount for each region, and a feature amount extracted by the feature amount extracting means for each dot And data generation means for generating encoded data of the image.

  In an image obtained by photographing the screen of the matrix display device, a periodic pattern may appear repeatedly on the screen. When such an image is given to the pattern detection means, the pattern detection means detects the periodic pattern. The detected periodic pattern is a periodic pattern formed by images of a plurality of dots arranged in a matrix. The feature amount extraction unit specifies an area corresponding to each dot image in the image based on the periodic pattern. Further, a predetermined feature amount is extracted for each region. A dot is a minimum unit of information displayed on a matrix type display device. Since the feature amount is extracted from the area corresponding to the dot image, each feature amount corresponds to information expressed by dots arranged in a matrix. Therefore, the encoded data generated by the data generation unit based on the feature amount is data that appropriately represents the feature of the information displayed by the matrix display device without excess or deficiency.

  Preferably, in the image, a region corresponding to each dot image of the matrix type display device is expressed by a plurality of pixels, and the feature amount extraction unit applies each of the plurality of dot images based on the periodic pattern. A region determining unit for determining a region on the corresponding image, and, for each of the regions determined by the region determining unit, a pixel of the region based on pixel values of a plurality of pixels forming the region Representative value generating means for generating a representative value representing the value as a feature quantity of the region.

  The representative value generating means generates a representative value representing the dot image based on the pixel values of a plurality of pixels forming one dot image of the display device. As a result, information for one dot expressed by a plurality of pixels is collected into a representative value. Therefore, the encoded data generated by the data generation unit based on the representative value appropriately represents the image displayed by the matrix type display device, and becomes data with little useless information.

  More preferably, the representative value generating unit is configured to generate, for each of the regions determined by the region determining unit, a gray scale representing the pixel value of the region based on the pixel values of a plurality of pixels forming the region. Means for generating a value as said representative value.

  Accordingly, information for one dot expressed by a plurality of pixels is collected into a gray scale value of almost one pixel. Therefore, the encoded data generated by the data generation means based on the representative value of the gray scale appropriately represents the image displayed by the matrix type display device in the gray scale, and is data with less useless information. Become.

  More preferably, each of the dots may include a plurality of display elements for expressing the color of the image, and the region determining means may apply each of the plurality of display element images based on a periodic pattern. A first determining means for determining a first region on the corresponding image, and a set of the first region on the image corresponding to each of the plurality of dot images based on the periodic pattern; Second determining means for determining. The representative value determining means generates, for each of the first regions, element-specific representative values that represent the pixel values of the first region based on the pixel values of a plurality of pixels that form the first region. In accordance with the set determined by the first generation means and the second determination means, the pixel value of the region is represented based on a plurality of representative values corresponding to each of the plurality of dot images. Second generating means for generating a representative value.

  In a color display device or the like in which one dot is composed of a plurality of display elements, the color information for one dot is first aggregated into element-specific representative values corresponding to the plurality of display elements in the dot. , It is collected into a representative value of almost one pixel generated based on the representative value for each element. Therefore, the encoded data generated by the data generation unit based on the representative color components appropriately represents the color of the dot image displayed by a plurality of display elements, and has little useless information. It becomes.

  The data generation means includes image data generation means for reconstructing a representative value representative of the pixel value of the region relating to each of the dots based on the periodic pattern and generating new image data corresponding to the original image. But you can.

  The image data generation means reconstructs the representative value and generates new image data. The pixel value of one pixel in the new image data is a representative value that represents the pixel value of the area corresponding to one dot of the display device, and the information for one dot of the display device is collected into the representative value. Therefore, in the new image data, the black mask portion is removed from the display of the display device, and one dot is expressed by almost one pixel. Further, in a display device having dots composed of a plurality of display elements, information expressed by a plurality of display elements within one dot is also expressed by almost one pixel. Therefore, the periodic pattern resulting from the difference in the characteristics of the display elements is excluded. Therefore, it is possible to generate image data with less noise, a clear image, and a small size.

  The feature amount extraction means further includes binarization processing means for binarizing each of the representative values generated by the representative value generation means, and the data generation means performs binarization processing means for each dot. Binary image data generation means for reconstructing the binarized representative value based on the periodic pattern and generating new binary image data corresponding to the original image may be included.

  The binarization processing unit binarizes each representative value generated by the representative value generation unit. The pixel value of one pixel in the new binary image data is obtained by binarizing the representative value representing the pixel value in the region corresponding to one dot of the matrix type display device, and one pixel of the matrix type display device. Minute information is collected into representative values. Therefore, in the new binary image data, the black mask portion is removed from the display of the display device, and one dot is represented by almost binary values. Further, in a display device having dots composed of a plurality of display elements, information expressed by a plurality of display elements within one dot is also expressed by binary values. Therefore, the periodic pattern resulting from the difference in the characteristics of the display elements is excluded. Therefore, clear binary image data with less noise can be obtained.

  The image may include an image obtained by photographing the characters displayed on the screen of the matrix display device, and the image processing device is further provided with the encoded data generated by the data generation means. In response, character recognition means for performing character recognition on the image based on the encoded data and outputting character information representing the recognition result may be included.

  The encoded data faithfully reflects the content displayed by the display device and has a small amount of data. Therefore, when the character recognition means performs character recognition using this encoded data, character recognition can be performed at a high speed and with a high recognition rate from an image obtained by photographing the screen of the display device. Since the recognition result is a character, the amount of information is smaller than that of an image representing the character. Therefore, the characters displayed on the display device can be expressed with a small amount of information.

  The image processing apparatus may further include data correction means for correcting the encoded data based on a periodic pattern that repeatedly appears on the image due to the presence of dots of the display device.

  Appropriate encoded data can be generated by correcting the encoded data corresponding to the image based on the inclination, distortion, etc. of the periodic pattern that appears repeatedly in the image.

  The image processing apparatus further includes photographing means for photographing the screen of the matrix type display device and providing an image obtained by photographing to the pattern detecting means.

  When the photographing unit photographs the screen of the matrix type display device and gives the obtained image to the pattern detection unit, encoded data corresponding to the photographed image is generated. Therefore, a screen of a desired display device can be photographed using the photographing means, and the information displayed on the display device can be extracted and encoded as encoded data without leaving all the information.

  When executed by a computer, the computer program according to the second aspect of the present invention causes the computer to operate as any one of the image processing apparatuses according to the first aspect of the present invention. Therefore, by executing this computer program on a computer, the operation and effect of the first aspect of the present invention described above can be realized on a computer.

The recording medium according to the third aspect of the present invention is a computer-readable recording medium on which the computer program according to the second aspect of the present invention is recorded. The operation and effect of the first aspect of the present invention can be realized by reading and executing the computer program recorded on the recording medium by a computer.

According to the present invention, an image obtained by capturing a screen of a matrix type display device is removed from the periodic pattern by using a periodic pattern generated in the image due to the structure of the display device. It is possible to generate encoded data that reflects the displayed contents and has little useless information. Furthermore, a plurality of pixels can be aggregated into one pixel, and clear image data with less noise can be generated with a small amount of information. The coded data to create the original binary image data, by character recognition by using the binary image data, it is possible to perform character recognition with high accuracy and high speed, further, the amount of information by encoding It is possible to reduce or to make the data searchable. Furthermore, correction can be performed based on the periodic pattern, and appropriate encoded data can be generated.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the drawings used for the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated. In the following description, a pixel such as a liquid crystal display panel is referred to as “dot”, and pixel value data for one pixel in image data is referred to as “pixel”.

<Constitution>
FIG. 1 shows a functional configuration of the image processing apparatus according to the present embodiment. Referring to FIG. 1, image processing apparatus 100 according to the present exemplary embodiment captures an image displayed on liquid crystal display panel 102 according to a user operation on input device 104 and processes the obtained image data. Thus, the information designated by the operation is generated and output to the output device 106.

  The image processing apparatus 100 captures an image of the liquid crystal display panel 102 and generates an image data thereof, and the image data generated by the image input unit 108 is converted into structural features of the liquid crystal display panel 102. The image processing unit 110 for processing and encoding using the image, the image storage unit 112 for storing the image data encoded by the image processing unit 110, and the image data stored by the image storage unit 112 And a character recognition unit 114 for generating character string data, a recognition result storage unit 116 for storing character string data generated by the character recognition unit 114, and an operation on the input device 104. According to the input, a request is given to the image input unit 108, the image processing unit 110, and the character recognition unit 114, and the image storage unit 11 is operated by an operation according to the request. And the data stored in the recognition result storage unit 116, a control unit 118 for outputting to the output device 106.

  The control unit 118 has a function of giving a shooting command to the image input unit 108. The control unit 118 further has a function of giving information regarding whether or not image processing is necessary to the image processing unit 110 and a function of giving information about whether or not character recognition is necessary to the character recognition unit 114. The image input unit 108, the image processing unit 110, and the character recognition unit 114 each operate according to information provided from the control unit 118.

  FIG. 2 shows the configuration of the image input unit 108. Referring to FIG. 2, an image input unit 108, when driven, generates a CCD 130 for generating an analog signal of an image of the liquid crystal display panel 102 formed on the surface via an optical system (not shown), and an analog signal generated by the CCD. In accordance with a request from the control unit 118, an A / D converter 132 for converting the image data into digital image data, an image memory 134 for temporarily storing the image data converted by the A / D converter 132, And a photographing control unit 136 for driving the CCD 130 and the A / D converter 132. Similar to a general digital camera, the image input unit 108 generates an image signal converted into digital data, and stores this as image data in an image memory. Therefore, when the display on the liquid crystal display panel 102 is photographed, a pattern (hereinafter referred to as “periodic pattern”) that repeatedly appears in the image due to the structure of the liquid crystal display panel 102, as in the images shown in FIGS. ”Is stored in the image memory 134.

  FIG. 4 shows an example of a periodic pattern. Referring to FIG. 4, periodic pattern 160A is a periodic pattern in image 620 shown in FIG. The periodic pattern 160B is a periodic pattern in the image 640 shown in FIG. The periodic pattern 160A is a pattern that appears in an image obtained by photographing a color liquid crystal display panel. The black mask image 162 and red (R), green (G), and blue (B) display elements that form one dot. Images 164A, 164B, and 164C. The periodic pattern 160B is a pattern that appears in an image obtained by photographing a monochrome liquid crystal display panel, and includes a black mask image 162 and a display element image 164D.

  FIG. 3 shows a functional configuration of the image processing unit 110. Referring to FIG. 3, image processing unit 110 reads image data from image memory 134 (see FIG. 2) in image input unit 108 in response to information from control unit 118, and is represented by the image data. An operation for determining whether or not a periodic pattern is projected in the image, and for storing the image data in the image storage unit 112 based on the determination result and for applying the image data to other functional units of the image processing unit 110 And a pattern interval detection unit 142 for specifying the size of the periodic pattern on the image based on the image data supplied from the periodic pattern detection unit 140. The tilt detector 144 for detecting the tilt of the image based on the image data given from the periodic pattern detector 140, and one dot in the liquid crystal display panel 102. A selected pattern shape determining unit 146 for determining the shape of a pattern (hereinafter referred to as a “selected pattern”) used for extracting the minute information based on the size and inclination of the pattern, and the pattern width and inclination. Based on this, the position of the area where each dot appears in the image data is detected, pixels are extracted from the area in a shape corresponding to the shape of the selected pattern, and the detected position information and the pixel of the pixel extracted from the position are extracted. A selected pattern position determining unit 148 for outputting value data, a representative value determining unit 150 for determining a representative value of the pixel value from the pixel value data output by the selected pattern position determining unit 148, and a representative value 2 for comparing the representative value determined by the determination unit 150 with a predetermined threshold value and outputting a value of 0 or 1 based on the comparison result Pixel value obtained by binarizing the image for one dot based on the position information detected by the binarization processing unit 152 and the selection pattern position determination unit 148 and the value output by the binarization processing unit 152 And a reconstructing unit 154 for generating image data composed of pixels expressed by the above and storing the image data in the image storage unit 112.

<Control Structure of Image Processing Apparatus 100>
As will be apparent from the following description, the control unit 118, the image processing unit 110, and the character recognition unit 114 of the image processing apparatus 100 shown in FIG. 1 are all hardware for program execution similar to a general computer. and wear, can be implemented by a computer program executed on the it.

  FIG. 5 shows a control structure of image processing apparatus 100 by control unit 118 according to the present embodiment. Referring to FIG. 5, this control is started in response to a request given to control unit 118. When the process is started, it is determined in step 200 whether or not an image input request has been made. If there is a request for image input, the process proceeds to step 202. Otherwise, the process proceeds to step 214, and other processes corresponding to the given request are executed, and the process is terminated.

  In step 202, the image input unit 108 is commanded to shoot. The image input unit 108 performs shooting in response to the command. In step 204, it is determined whether the image data obtained by the photographing in step 202 is to be processed based on the periodic pattern. If the image processing is requested in the request given to the control unit 118, the process proceeds to step 206. Otherwise, go to step 216. Note that even if processing of the image is required, if the periodic pattern cannot be detected in the image, the process proceeds to step 216.

  In step 206, the image processing unit 110 processes the image based on the image data obtained by the shooting in step 202, generates a new image, and encodes the new image obtained by the processing. An encoding process for storing the image data in the image storage unit 112 is executed.

  In step 208, it is determined whether character recognition is requested based on the new image data stored in the image storage unit 112 by the encoding process in step 206. If character recognition is requested in the given request, the process proceeds to step 210. Otherwise, the process ends.

In step 210, the control unit 118 gives information indicating that character recognition is necessary to the character recognition unit 114, and causes the character recognition unit 114 to perform character recognition from the encoded image data. In step 212, in response to the character recognition unit 114 storing the result obtained by character recognition in the recognition result storage unit 116 , the control unit 118 outputs the character information of the recognition result and ends the process.

  In step 216, the image data is stored in the image storage unit 112 without performing the encoding process, and the process ends.

  FIG. 6 shows a control structure of the encoding process executed by the image processing unit 110 in step 206 of FIG. Referring to FIG. 6, when the encoding process is started, in step 220, the image data is sequentially scanned in the vertical direction and the horizontal direction, and based on the arrangement of pixels having pixel values satisfying a predetermined condition. The size of the periodic pattern in the image data is specified.

In step 222, the image data is sequentially scanned in the vertical direction and the horizontal direction, the deviation of the generation position of the pixel having the pixel value satisfying the predetermined condition is measured, and the image corresponding to the dot arrangement on the liquid crystal display panel 102 is measured. Detect the slope of.

  In step 224, based on the size of the periodic pattern and the inclination of the image, the shape of the selected pattern is determined in the image for one periodic pattern.

  In step 226, an area for storing the image data after the encoding process is prepared and initialized.

  In step 228, based on the size of the periodic pattern and the inclination of the image, an area corresponding to one dot on the screen of the liquid crystal display panel is detected from the image data, and selection within the area is performed with reference to the center pixel of the area. Determine the position of the pattern. In step 230, the pixel values in the pixels belonging to the selection pattern determined in step 228 are averaged, and the representative value of the pixel value in the image of the area detected in step 228 is determined. In step 232, the representative value determined in step 230 is compared with a predetermined threshold value, and a value of 0 or 1 is generated according to the comparison result. In step 234, the pixel value of the pixel at the position corresponding to the area selected in step 228 in the image data initialized in step 226 is set to the value generated in step 232.

  In step 236, it is determined whether or not there is an area that has not been selected in step 228. If it exists, the process returns to step 228. Otherwise, go to step 238.

  By repeating a series of steps from step 228 to step 236, each dot image is converted into binarized information for one pixel. In step 238, the binarized image is stored in the image storage unit 112, and the encoding process ends.

<Operation>
The image processing apparatus 100 according to the present embodiment operates as follows. Referring to FIG. 1, image processing apparatus 100 is assumed to be installed at a position suitable for photographing the screen of liquid crystal display panel 102 by the user, and when image input unit 108 photographs, Assume that the screen of the display panel 102 is photographed.

The user operates the input device 104 and requests the image processing apparatus 100 to perform processing. The processing required here is one of the following processing. That is,
(1) Processing for taking an image, performing character recognition using the image, and outputting the result;
(2) Processing to capture an image, encode and output based on a periodic pattern,
(3) a process of taking an image and outputting the image as it is, and (4) other processes.

  When one of these four types of processing requests is given from the input device 104 to the image processing device 100, the control unit 118 shown in FIG. 1 determines the requested content. If the request corresponds to the above (1) to (3), the control unit 118 indicates the content of the requested processing to the image input unit 108, the image processing unit 110, and the character recognition unit 114. Give information and command execution of the process. If the request corresponds to the above (4), processing according to the request is executed. For example, when data output is requested, the data stored in the image storage unit 112 and the recognition result storage unit 116 is output to the output device 106 in accordance with the request.

(Input image data)
When an instruction corresponding to the above requests (1) to (3) is given from the control unit 118 to the image input unit 108, the imaging control unit 136 shown in FIG. 2 of the image input unit 108 causes the CCD 130 and the A / D. The converter 132 is driven. The CCD 130 generates an analog signal of an image formed on the CCD area and applies it to the A / D converter 132. The A / D converter 132 converts the applied analog signal into a digital signal, generates image data, and stores the image data in the image memory 134.

(Processing of image data)
When a command is given from the control unit 118 (see FIG. 1) to the image processing unit 110, the periodic pattern detection unit 140 shown in FIG. 3 determines the content of the command. If the command corresponds to the above request (1) or (2), the pixel of the image data is scanned to determine whether or not a periodic pattern appears in the image. For example, if periodicity is recognized in the pixel value when scanning vertically and horizontally, it is determined that a periodic pattern exists. In this case, an image data encoding process is executed. When it is determined that the periodic pattern is not shown, or when the command corresponds to (3), the image data stored in the image memory 134 is read and the image storage unit 112 is performed without performing the encoding process. To store.

  Hereinafter, the encoding process by the image processing unit 110 will be described. First, an encoding process for an image obtained by photographing a monochrome liquid crystal display panel will be described. FIG. 7 shows an outline of the encoding process by the image processing unit 110. Referring to FIG. 7, an image 300 is an enlarged image of a black and white liquid crystal display panel screen, similar to the image 640 shown in FIG. 20. In this image, the periodic pattern 160B of the monochrome liquid crystal display panel shown in FIG. 4 is generated, and one of the periodic patterns (for example, the periodic pattern 302) is expressed by a rectangle of 9 pixels. The image processing unit 110 performs processing for generating a 1-pixel binarized image from images of one periodic pattern for each periodic pattern, and rearranges the generated images according to the arrangement of the periodic patterns. Then, a processed image is generated. A portion corresponding to the image 300 in the processed image is an image 310. In this image, the periodic pattern 302 is represented by one pixel 312. When the encoding process of the image data is started, the periodic pattern detection unit 140 reads the image data stored in the image memory 134, and the pattern interval detection unit 142, the inclination detection unit 144, the selected pattern shape determination unit 146, and This is given to the selection pattern position determination unit 148.

  The pattern interval detection unit 142 scans the pixels of the image data in the vertical direction and the horizontal direction, detects pixels having a similar characteristic pixel value and existing at a fixed interval, or a group of the pixels, and the interval of the group To specify the width and height of the pattern.

  For example, pixels of image data are scanned in the vertical and horizontal directions to detect pixels having pixel values that are considered to be black mask images. Subsequently, a portion where the detected pixels continuously form a line is checked. Further, the interval between the portions forming the line is obtained. In the image 300 illustrated in FIG. 7, a line 306 </ b> A having a width of 1 pixel and composed of pixels having pixel values of gradations (represented in black) similar to each other at intervals of 9 pixels in the vertical direction. ..., D is detected. That is, a black mask image is detected at intervals of 9 pixels in the vertical direction and 1 pixel in width. Further, in the image 300 shown in FIG. 7, a black mask image is similarly detected in the horizontal direction at intervals of 9 pixels and at a width of 1 pixel. Therefore, it is detected that the size of the periodic pattern is 9 pixels in both width and height.

  FIG. 8 shows another example of an image in which a line that seems to be an image of a black mask is shown. Referring to FIG. 8, in image 320 shown in FIG. 8, dark (low gradation) lines are shown at intervals of 11 pixels in the horizontal direction and 23 pixels in the vertical direction. Further, in the image 320, thin (high gradation) lines having a thickness of 2 pixels extending in the vertical direction are shown at intervals of 11 pixels in the horizontal direction. The image 320 further shows a thin line having a thickness of 2 pixels extending in the horizontal direction at intervals of 23 pixels in the vertical direction. Further, a thin line having a width of 1 pixel is shown between a dark line and a thin line extending in the horizontal direction. This is an interference pattern that occurs when the dot size does not become an integral multiple of the size of one pixel in the image data.

  FIG. 9 schematically shows the positional relationship between the image of the black mask on the CCD 130 and each element of the CCD 130 when the image data as shown in FIG. 8 is obtained. Referring to FIG. 9, CCD 130 has a structure in which a large number of elements are arranged in a matrix. A black mask image (eg, image 332A) formed near the center of the element in the CCD 130 becomes a dark line when converted to image data. A black mask image (for example, image 332B) formed at a portion off the center of the element in the CCD 130 becomes a lighter line than the black mask image formed near the center when converted to image data. Since the image of the black mask (for example, the image 332C) formed in the vicinity of the element boundary in the CCD 130 is detected by two adjacent elements, when it is converted into image data, it becomes a thin line extending over two pixels. .

  Referring to FIG. 8 again, in the image 320, dark lines are arranged at intervals of 11 pixels in the horizontal direction, and there is one thin line between them. For this reason, it can be estimated that two dots exist within an interval of 11 pixels. Therefore, the width of the periodic pattern is set to 11/2 = 5.5 pixels. In the vertical direction of the image 320, there are one dark line, one thin and thick line, and two thin lines between 23 dots, so the height of the periodic pattern is 23/4 = 5.75 pixels. And

  FIG. 10 shows another example of an image in which a line that seems to be an image of a black mask is shown. In the image 340 shown in FIG. 10, the dots on the image rotate counterclockwise with respect to the pixel arrangement. Therefore, short lines having a length of 8 pixels are arranged at intervals of 10 pixels in the vertical direction and the horizontal direction. In this case, the size of the periodic pattern is specified from the interval between adjacent lines extending in the same direction. In the example shown in FIG. 10, the size of the periodic pattern is specified to be 10 pixels in both width and height.

  The pattern interval detection unit 142 provides the selection pattern shape determination unit 146 and the selection pattern position determination unit 148 with information on the width and height of the periodic pattern specified as described above.

  With reference to FIG. 3, the inclination detection unit 144 detects the inclination of the captured image data. For example, the pixel of the image data is scanned, a line that seems to be an image of a black mask is detected, and the inclination is specified from the deviation of the line. For example, in the image 300 shown in FIG. 7, the black mask images are arranged at intervals of 9 pixels both vertically and horizontally as described above, and there is no deviation in the vertical or horizontal direction. In this case, it is specified that the image 300 has no inclination. On the other hand, in the image 340 shown in FIG. 10, the line that seems to be the image of the black mask is shifted by 1 pixel per 8 pixels, so that the inclination is specified as 1/8. The inclination detection unit 144 provides the specified inclination information to the selection pattern shape determination unit 146 and the selection pattern position determination unit 148.

  The selected pattern shape determination unit 146 receives image data from the periodic pattern detection unit 140, and further responds to reception of periodic pattern interval information and inclination information from the pattern interval detection unit 142 and the inclination detection unit 144, respectively. Then, the shape of the selected pattern is determined.

  For example, the center pixel and the neighboring pixels in the image for one dot are determined as the selection pattern. In the image 300 shown in FIG. 7, the size of the periodic pattern is 9 pixels vertically and horizontally, so that information displayed by 1 dot can be extracted from a collection of pixels arranged in a square of 9 pixels. . However, in the image 300 shown in FIG. 7, a black mask image is shown with a width of 1 pixel, one for each periodic pattern, one for the vertical and one for the horizontal pattern. The black mask image is an image irrelevant to the information displayed on the liquid crystal display panel 102. For this reason, the selected pattern shape is, for example, a 7-pixel square that excludes the surrounding 1 pixel from a 9-pixel square.

  Furthermore, in order to eliminate the influence of the boundary portion in the image for one dot, margins are taken in the vertical and horizontal directions. FIG. 11 shows an outline of a selection pattern excluding the influence of the boundary portion. FIG. 11A schematically shows a one-dot image formed on the CCD 130 element. Referring to FIG. 11A, the element located at the boundary portion of 1-dot image 360 generates an image signal composed of a part of the image of dot 360 and an image other than dot 360.

  FIG. 11B schematically shows image data obtained based on the image of the dot 360. In FIG. 11B, a pixel value corresponding to the display on the liquid crystal display panel 102 is obtained from each pixel of the portion 370 other than the boundary. However, since the boundary portions 372 and 374 are affected by both a part of the image of the dot 360 and an image other than the dot 360, a blur occurs and an appropriate pixel value cannot be obtained. In such a case, in order to eliminate the influence of the boundary portion, the shape of the selected pattern is a rectangle 380 of 4 pixels square. For example, in the image 320 shown in FIG. 8, it is detected that the periodic pattern interval is 5.5 pixels in the horizontal direction and 5.75 pixels in the vertical direction. The shape of the selection pattern is a rectangle 3 pixels wide by 3 pixels high. In many cases, if a rectangle having a size two or more smaller than the size of the periodic pattern is set as the shape of the selection pattern, a selection pattern that eliminates the influence of the black mask and blur can be obtained. The selection pattern shape determination unit 146 provides information indicating the shape of the selection pattern to the selection pattern position determination unit 148.

  When image data is provided to the selection pattern position determination unit 148 and further information regarding the size of the periodic pattern, the inclination of the image, and the shape of the selection pattern is provided, the selection pattern position determination unit 148, the representative value determination unit 150, The binarization processing unit 152 and the reconstruction unit 154 convert each dot image in the image data into a binarized pixel by sequential processing.

  The selection pattern position determination unit 148 determines the position of each selection pattern as follows, for example.

  First, the number of vertical and horizontal dots appearing in the image is estimated from the number of vertical and horizontal pixels in the image data and the size of the periodic pattern. Further, the number of dots is corrected in consideration of the inclination. Further, based on the corrected number of dots, the height and width of the encoded image data are obtained and given to the reconstruction unit 154 shown in FIG. In addition, the center pixel in the shape of the selected pattern is determined.

  Subsequently, the image of the uppermost dot in the image is specified. This dot image can be identified by detecting a line that seems to be a black mask image. Further, the center pixel of the dot image is specified. Then, the central pixel of the dot is determined as the central pixel of the selection pattern corresponding to the dot image. Further, the selection pattern position determination unit 148 notifies the representative value determination unit 150 of the pixel value of each pixel at the position of the determined selection pattern, and the reconstruction unit 154 indicates the position of the dot in the encoded image data. To give.

  When the selection pattern position determination unit 148 determines the position of the selection pattern in one dot in a series of processes, the center pixel in the dot image is sequentially identified based on the height and width of the periodic pattern and the inclination of the image. To do. The selection pattern position specifying unit 148 repeats the above operation to give the reconstruction unit 154 the position of each dot in the image after encoding and represents the pixel value of each pixel in the selection pattern for that dot. The value is given to the value determining unit 150.

  FIG. 12 shows an example of the position of the selection pattern determined by the selection pattern position determination unit 148 for the image 320 shown in FIG. Referring to FIG. 12, selection patterns 402A, B,... Are set in portions other than the black mask image in image 320, respectively. The images in these selection pattern portions are images of display elements of the liquid crystal display panel 102. Therefore, it is possible to estimate the gradation of light and shade expressed by the dots corresponding to the selection pattern from the pixel values of the pixels constituting the selection pattern.

  3 is provided with the pixel value of each pixel included in the selection pattern sequentially from the selection pattern position determination unit 148 for each of the selection patterns 402A, B,... The representative value determining unit 150 calculates and outputs an average value of the pixel values each time a pixel value of each pixel in one selection pattern is given. This average value becomes the representative value of the selected pattern.

  The binarization processing unit 152 compares the representative value output from the representative value determining unit 150 with a predetermined threshold value, and outputs 1 if the representative value is greater than the threshold value and 0 otherwise. To do. For example, if the threshold value is image data of 256 gradations, 128 may be set as the threshold value.

  In response to the height and width of the encoded image data being given from the selection pattern position determining unit 148, the reconstruction unit 154 prepares and initializes the area of the encoded image data. The number of pixels in the prepared image data substantially matches the number of periodic patterns in the image data before encoding, that is, the number of dots appearing in the image.

  The reconstruction unit 154 further receives notification of the position of the dot in the encoded image data from the selection pattern position determination unit 148 and receives an output of a value of 0 or 1 from the binarization processing unit 152. In response, the pixel value of the pixel corresponding to the notified position is set to the value output from the binarization processing unit 152.

  When the selection pattern position determination unit 148, the representative value determination unit 150, the binarization processing unit 152, and the reconstruction unit 154 encode the images of all dots in the image data by the above sequential processing, reconstruction is performed. The unit 154 stores the encoded image data in the image storage unit 112.

  Hereinafter, an encoding process for an image obtained by photographing a color liquid crystal display panel will be described.

  The operation of the encoding process for the image obtained by photographing the color liquid crystal display panel is almost the same as that for the monochrome liquid crystal display panel. However, the process of detecting the periodic pattern, the process of specifying the size thereof, and the process of calculating the representative value are performed by a method that takes into account the structure of the color liquid crystal display panel.

  In the image obtained by photographing the screen of the color liquid crystal display panel, repetition of red (R), green (G), and blue (B) is detected when the periodic pattern is detected. When specifying the size of the periodic pattern, in the periodic pattern of the color liquid crystal display panel, the color of the pixel is referred to sequentially, and red (R), green (G), and blue (B) are repeated. Ask for. For example, in the image 620 shown in FIG. 18, since the images of the black mask extending in the horizontal direction are shown at intervals of 9 pixels, the height of the periodic pattern is set to 9 pixels by the same specific method as in the case of the monochrome liquid crystal display panel. To be identified. Further, when the image 620 is scanned in the horizontal direction, a portion in which three pixels in which the red (R) component has a higher value than the other components continues, and the green (G) component has a higher value than the other components. A portion where 3 pixels are consecutive and a portion where blue (B) component is higher than the other components is continuously detected in this order. Therefore, the width of the periodic pattern in this image 620 is specified as 9 pixels.

  Even for an image obtained by photographing the screen of the color liquid crystal display panel, the shape of the selection pattern is set by the same procedure as that for the image obtained by photographing the screen of the monochrome liquid crystal display panel. That is, the periodic pattern of the three primary colors of red (R), green (G), and blue (B) is regarded as one dot. Therefore, in the image 620 shown in FIG. 18, the selected pattern shape is set to a 7-pixel square. Therefore, images of display elements of red (R), green (G), and blue (B) are mixed in the selection pattern.

  When determining the representative value, one representative value is calculated by combining the red (R), green (G), and blue (B) components of the pixels in the selected pattern. Therefore, in the encoding process according to the present embodiment, binarized image data is obtained based on the image of the color liquid crystal display panel.

  FIG. 13 shows an example of an image obtained by encoding the data of the image 600 shown in FIG. Referring to FIG. 13, when the relationship between the periodic pattern and the pixels in image 600 is as shown in FIG. 18, image 420 is 1/9 in size both vertically and horizontally with respect to image 600 shown in FIG. 15. become. Further, in the image 420, the character portion of the image 600 shown in FIG. 15 is binarized to black, and the other portions are binarized to white, and a periodic pattern due to the influence of dots is removed to obtain a clear image. ing.

(Character recognition)
When the command corresponding to the above (1) is given from the control unit 118 shown in FIG. 1 to the character recognition unit 114 and the image data is stored in the image storage unit 112, the character recognition unit 114 stores the data. Character recognition is performed based on the received image data, and the result is stored in the recognition result storage unit 116. The character recognition process performed here may be a generally known character recognition process.

  When character recognition is performed using image data encoded by the encoding process (for example, the image 420 shown in FIG. 13), the liquid crystal display panel 102 is as clear as the image data that the liquid crystal display panel 102 seems to have used for displaying the image. Character recognition is performed based on image data with a small data amount. By using the periodic pattern in this way, it is possible to reduce the amount of information and generate a clear image. In addition, since it is possible to perform appropriate binarization, an improvement in recognition rate when performing character recognition can be expected.

  In the above embodiment, the correction based on the inclination of the image is performed, but the present invention is not limited to such an embodiment. For example, distortion due to lens aberration can be detected based on the periodic pattern. Therefore, you may make it correct | amend the distortion by these aberrations.

  In the above embodiment, the method for calculating the average of the pixel values is exemplified as the method for calculating the representative value. However, the method for calculating the representative value is not limited to such a method. For example, the distribution of pixel values may be taken, upper or lower pixel values may be removed, and the remaining pixel values may be averaged to eliminate the influence of pixels having extremely different pixel values. Alternatively, the distribution of pixel values may be taken and the center value may be determined as a representative value.

  In the above embodiment, the comparison between the predetermined threshold value and the representative value is sequentially performed and the binarization process is performed. However, the binarization method is not limited to such a method. . For example, representative values for all selection patterns are accumulated, the maximum value and the minimum value of the representative values are examined, the value of (maximum value + minimum value) / 2 is calculated, and the value is set as a threshold value Then, the representative value may be compared with the threshold value.

  In the above embodiment, the image processing unit 110 generates binary image data in which each pixel is monochrome from the original image data. However, the present invention is not limited to such an embodiment. Grayscale image data can be obtained by using the representative value as it is as the pixel value of the corresponding pixel.

  Further, in the case of an image obtained by photographing the screen of the color liquid crystal display panel, red (R), green (G) in pixel values from images of display elements of red (R), green (G), and blue (B), respectively. ) And blue (B) components can be obtained, and a representative value of the color can be obtained by a combination thereof. In this case, when obtaining a representative value of color from an image of one dot on the color liquid crystal display panel, it is desirable to set a selection pattern for each color so that information on the three primary colors in one dot can be extracted for each color. . FIG. 14 shows an example of the shape and position of such a selection pattern. The image shown in FIG. 14 is similar to the image 620 shown in FIG. 18 in that the images of the display elements of red (R), green (G), and blue (B) are 3 pixels wide and 8 pixels high. Forming.

  For image data having such a periodic pattern, selection patterns 510A and 510B each consisting of three rectangles each having a width of 1 pixel and a height of 8 pixels may be determined as the selection pattern shape. Furthermore, red (R), green (G), and blue (B) components are associated with the three rectangular patterns corresponding to one dot, respectively. When determining a representative value, a representative value is obtained for a color component corresponding to the rectangle from one rectangle in one selection pattern. Then, from the obtained representative values of the three components, a color pixel value for expressing the color by additive color mixture is created and used as the representative value of the selected pattern. By using the representative value thus determined as the pixel value of the corresponding pixel as it is, it is possible to generate clear and small-sized color image data from the original image data.

  Note that the periodic pattern in the above-described embodiment is an example, and various periodic patterns are actually assumed depending on the structure of the dots of the liquid crystal display panel. For example, it is assumed that there is a black mask between red (R), green (G), and blue (B) display elements in a color liquid crystal display panel. Even in such a case, the periodic fluctuation of the pixel value can be detected. Therefore, as long as the shape of the selected pattern is set to be suitable for the periodic pattern, appropriate encoding can be performed.

  Further, in the above embodiment, the processing based on the periodic pattern is performed on the image obtained by photographing the liquid crystal display panel 102. However, the image to be processed is not limited to an image obtained by photographing the liquid crystal display panel 102. The above processing can be applied to an image obtained by photographing a screen of a matrix type display device composed of a plurality of dots arranged in a matrix and similar arrangements, and an image obtained by photographing an object having a periodic pattern like a liquid crystal display panel.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is a diagram illustrating a configuration of an image processing apparatus 100. FIG. 3 is a diagram illustrating a configuration of an image input unit 108. FIG. 2 is a diagram illustrating a functional configuration of an image processing unit 110. FIG. It is a schematic diagram which shows the example of a periodic pattern. 3 is a diagram illustrating a control structure of processing executed by the image processing apparatus 100. FIG. It is a figure which shows the flow of the encoding process shown in FIG. It is a schematic diagram for demonstrating the outline | summary of the process by the image process part. It is a schematic diagram which shows an example of the image in which the image of a black mask is reflected. It is a schematic diagram which shows the magnitude | size of the periodic pattern specified by the image shown in FIG. It is a schematic diagram which shows an example of the image which the periodic pattern with inclination generate | occur | produced. It is a schematic diagram which shows the size of a dot, the image of the dot on an image, and the shape of the selection pattern corresponding to the image of the said dot. It is a schematic diagram which shows the selection pattern determined about the image shown in FIG. It is a figure which shows an example of the image produced | generated by an encoding process. It is a schematic diagram which shows an example of the selection pattern corresponding to the image which image | photographed the color liquid crystal panel. It is a figure which shows an example of the image obtained by image | photographing the character displayed on the color liquid crystal panel. It is the elements on larger scale of the image shown in FIG. It is a schematic diagram for demonstrating the periodic pattern which generate | occur | produces in the image shown in FIG. It is a schematic diagram which shows the pixel of the image obtained by image | photographing the image displayed on the color liquid crystal display panel, and the dot of a color liquid crystal display panel. It is a schematic diagram for demonstrating the periodic pattern which generate | occur | produces in the image displayed on the monochrome liquid crystal display panel. It is a schematic diagram which shows the pixel of the image obtained by image | photographing the image displayed on the monochrome liquid crystal display panel, and the dot of a color liquid crystal display panel.

Explanation of symbols

100 image processing device, 102 liquid crystal display panel, 104 input device, 106 output device, 108 image input unit, 110 image processing unit , 112 image storage unit, 114 character recognition unit, 116 recognition result storage unit, 118 control unit, 130 CCD , 132 A / D converter, 134 image memory, 136 shooting control unit, 140 periodic pattern detection unit, 142 pattern interval detection unit, 144 tilt detection unit, 146 selection pattern shape determination unit, 148 selection pattern position determination unit, 150 representative value Determination unit, 152 binarization processing unit, 154 reconstruction unit

Claims (8)

An image processing device for encoding an image obtained by photographing a screen of a matrix type display device composed of a plurality of dots arranged in a matrix,
Pattern detecting means for detecting a periodic pattern formed by images of a plurality of dots arranged in a matrix, which repeatedly appear on the image in response to being given the image;
A feature amount for identifying a region corresponding to each of the dot images in the image based on the periodic pattern detected by the pattern detection means, and extracting a predetermined feature amount for each of the regions Extraction means;
Based on the characteristic amount extracted by the feature quantity extracting means relates each of the dot, look contains a data generation means for generating an encoded data of the image,
In the image, a region corresponding to each dot image of the matrix display device is represented by a plurality of pixels,
The feature amount extraction means includes:
An area determining means for determining an area on the image corresponding to each of the plurality of dot images based on the periodic pattern;
To generate a representative value representing the pixel value of the region as a feature amount for each of the regions determined by the region determination unit based on the pixel values of a plurality of pixels forming the region. Representative value generation means
Each of the dots may include a plurality of display elements for expressing the color of the image,
The region determining means includes
First determining means for determining a first region on the image corresponding to each of the images of the plurality of display elements based on the periodic pattern;
Second determining means for determining a set of first regions on the image corresponding to each of the plurality of dot images based on the periodic pattern;
The representative value determining means includes
For each of the first regions, based on the pixel values of a plurality of pixels forming the first region, a first element-specific value representative of the pixel value of the first region is generated. 1 generating means;
In accordance with the set determined by the second determining means, a representative value representative of the pixel value of the area is generated based on the plurality of element-specific representative values corresponding to each of the plurality of dot images. including, an image processing apparatus and a second generating means for. The data generation means reconstructs a representative value representative of the pixel value of the region for each of the dots based on the periodic pattern, and generates image data for generating new image data corresponding to the image The image processing apparatus according to claim 1 , comprising means. The feature quantity extraction means further includes binarization processing means for binarizing each of the representative values generated by the representative value generation means,
The data generation means reconstructs a representative value binarized by the binarization processing means for each of the dots based on the periodic pattern, and generates new binary image data corresponding to the image. The image processing apparatus according to claim 1 , further comprising binary image data generating means for generating. The image may include an image obtained by photographing characters displayed on the screen of the matrix display device,
Further, in response to the provision of the encoded data generated by the data generation means, character recognition relating to the image is performed based on the encoded data, and character information representing a recognition result is output. including character recognition means, an image processing apparatus according to any one of claims 1 to 3. Furthermore, on the basis of the periodic pattern comprises a data correcting means for correcting the coded data, the image processing apparatus according to any one of claims 1 to 4. Further, taking a screen matrix display device, comprising an imaging means for providing an image obtained by photographing the pattern detection means, the image processing apparatus according to any one of claims 1 to 5. A computer program that, when executed by a computer, causes the computer to operate as the image processing apparatus according to any one of claims 1 to 6 . A computer-readable recording medium on which the computer program according to claim 7 is recorded.

Images