JP2016099771A - Image processor and pattern matching method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable reduction of a processing load for searching for a template image in pattern matching.SOLUTION: An image processor for imaging an object having a detection pattern during movement of the object to acquire movement information of the object includes search parts (903 to 907) and a movement information acquisition part (908). The search part searches for a template image including the detection pattern to be extracted from image data of the object picked up in advance in image data of the object picked up later, specifies a division area smaller than a search area by detecting brightness of the search area in the image data of the object picked up later, and searches for a template image in the division area. The movement information acquisition part acquires the movement information of the object on the basis of a positional relation between the searched template image and a template image extracted from the image data of the object picked up in advance.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an image processing apparatus and a pattern matching method, and more particularly to a technique for obtaining a movement amount of an object by detecting movement of a detection pattern in an object such as a recording medium by pattern matching.

  In Patent Document 1, a detection pattern on a recording medium such as recording paper or a conveyance belt that conveys the recording pattern is imaged at different timings using an image sensor such as an image sensor, and the recording medium or the conveyance belt is detected based on the result. It describes that the amount of movement is calculated. Specifically, a template image including a detection pattern is extracted from image data captured at a certain timing. Then, the position where the template image exists in the captured image at the next timing is searched by the pattern matching technique, and the movement distance of the template image between the two timings is obtained.

JP 2011-93680 A JP-A-10-63842

  However, in the example shown in Patent Document 1, when searching for an image having a high correlation with the template image in the captured image, the area for obtaining the correlation with the template image is moved in the search area in units of pixels, and for each movement. Since the correlation is obtained, there is a problem that a processing load for that is increased.

  On the other hand, Patent Document 2 describes a technique for obtaining a centroid position of a template image (target image) in a captured image, limiting a search range based on the centroid position, and reducing a search load. ing. However, Patent Document 2 is based on the premise that the template image (target image) does not move, thereby allowing the search area to be limited. That is, as in Patent Document 1, when the template image moves, it is clear that the search range of the template image after the movement cannot be limited by the position of the center of gravity before the movement. In addition, the processing for obtaining the position of the center of gravity has a problem that a circuit and processing time for the processing are relatively long.

  An object of the present invention is to provide an image processing apparatus and a pattern matching method that can reduce the processing load for searching for a template image.

  Therefore, in the present invention, an image processing apparatus that obtains movement information of an object by imaging an object having a detection pattern during the movement of the object, and is extracted from the image data of the object that has been previously captured. A means for searching a template image including a detection pattern in image data of an object picked up later, and by identifying the brightness of the search area in the image data of an object picked up later, a divided area smaller than the search area is specified. , Moving to the object based on a positional relationship between search means for searching the template image in the divided area, the searched template image, and a template image extracted from the image data of the previously captured object. It is characterized by comprising movement information acquisition means for obtaining information.

  According to the above configuration, it is possible to reduce a processing load for searching for a template image in pattern matching.

1 is an external perspective view showing a schematic configuration of a recording apparatus according to an embodiment of the present invention. (A) And (b) is a figure which shows the detailed structure of the recording part shown in FIG. (A) And (b) is a figure which shows the detailed structure of the sensor unit shown in FIG. (A) And (b) is a figure explaining the basic process which concerns on one Embodiment of this invention for detecting the moving amount | distance of a recording medium (sheet | seat) based on the image data imaged with the image sensor. It is. (A) And (b) is a figure explaining the conventional pattern matching. It is a figure explaining the search method of the conventional template image. It is a figure explaining the pattern matching process which concerns on one Embodiment of this invention. It is a block diagram which shows the structure for the process which compares the brightness between the divided areas which concern on one Embodiment of this invention. It is a graph which shows the example of the relationship of the correlation value with respect to the pixel position calculated | required in the movement amount calculation part based on one Embodiment of this invention. It is a figure explaining the process which sets a search area over several division areas based on one Embodiment of this invention. It is a flowchart which shows the process until it calculates the movement amount which concerns on one Embodiment of this invention demonstrated above.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is an external perspective view showing a schematic configuration of a recording apparatus according to an embodiment of the present invention. The recording apparatus according to the present embodiment generally includes a recording unit 1, a sheet supply unit 2, a sheet winding unit 3, and a control unit 6. The sheet supply unit 2 holds the roll sheet 4 wound in a roll shape and supplies the continuous sheet 8 as a recording medium to the recording unit 1 while pulling it out from the roll. The recording unit 1 records an image on the continuous sheet 8 as will be described later with reference to FIG. The recorded sheet 8 is wound up as a roll sheet 5 by the sheet winding unit 3. The control unit 6 includes a controller, a memory, and various 1 / O interfaces, and controls the entire recording apparatus. The control unit 6 executes processing according to the embodiment of the present invention, which will be described later with reference to FIG. 11, and may be built in the recording apparatus itself or may be an external host computer connected to the recording apparatus. There may be. In this specification, an apparatus for executing processing according to the embodiment of the present invention described later with reference to FIG. 11 is referred to as an “image processing apparatus” including apparatuses such as a recording apparatus and a host computer. Further, in this specification, at an arbitrary position in the sheet conveyance path, the side closer to the sheet supply unit 2 is referred to as “upstream side” and the far side is referred to as “downstream side”.

  2A and 2B are diagrams showing a detailed configuration of the recording unit 1 shown in FIG. 1, FIG. 2A is a top view of the recording unit, and FIG. 2B is a diagram of the recording unit. Each side view is shown.

  In these drawings, the sheet 8 supplied from the sheet supply unit 2 to the recording unit 1 is conveyed in the arrow A direction in the recording unit 1. As a sheet conveyance mechanism in the recording unit 1, a pair of main conveyance rollers including a conveyance roller 11 and a pinch roller 12 that rotates following the conveyance roller 11 controls conveyance accuracy. On the downstream side of the recording heads 17 to 20 for each ink color, four pairs of sub-conveying roller pairs including a conveying roller 13 and a pinch roller 14 that is driven to rotate are provided. The recording heads 17 to 20 are ink jet recording heads that discharge black, cyan, magenta, and yellow ink, respectively. Each of these recording heads is a line type recording head provided with a nozzle row in which nozzles for ejecting ink are arranged in a range covering the maximum recording width of the sheet 8 assumed to be used. As a method for ejecting ink, a method using a heating element, a method using a piezo element, a method using an electrostatic element, a method using a MEMS element, or the like can be used. The number of ink colors and the number of recording heads are not limited to four, and may be larger or smaller. Each recording head is connected to an ink tube, and ink of a corresponding color is supplied from an ink tank (not shown). Instead of the supply via the tube, the recording head may be a unit integrated with an ink tank for storing the corresponding color ink. Each recording head is integrally held by a head holder 10.

  The sensor unit 701 captures the surface of the sheet on which the detection pattern is recorded in a process of detecting the amount of movement of the sheet, which will be described later. The marking head 500 is an ink jet recording head, and records a detection pattern by this. The marking head 500 and the sensor unit 701 are disposed upstream of the recording unit 1 and substantially at the center of the sheet width. With this arrangement, it is possible to cope with so-called marginless recording with no margin, and it is difficult to be affected by the mist of each color recording head in the recording unit. Further, even if a minute meandering or a minute skew of the sheet occurs, the average conveyance amount can be detected.

  Note that the recording apparatus of the present embodiment is a line printer capable of high-speed recording using a long and continuous sheet as a recording medium, and is suitable for the field of printing a large number of sheets in a print laboratory, for example. . However, the application of the present invention is not limited to such a form, and of course, also for a recording apparatus using a single sheet recording medium or a serial type recording apparatus that performs recording by scanning a recording head. Can be applied.

  3 (a) and 3 (b) are diagrams showing a detailed configuration of the sensor unit 701 shown in FIG. 2. FIG. 3 (a) is a top view of the sensor unit, and FIG. 3 (b) is a side view. FIG. As shown in these drawings, the sensor unit 701 includes an outline, a light emitting unit, a light receiving unit, and an image processing unit, which are integrated as a unit. The light source 703 is a light emitting element such as an LED. Light emitted from the light source 703 is guided by the light guide 702 to illuminate the surface of the sheet 8 from an oblique direction. An image of the illuminated area on the sheet 8 is formed on the image sensor 705 by the lens 706. The transparent protective cover 704 prevents ink mist or the like from entering from the sheet 8 side and stains on the lens 706. An image of the surface of the sheet (object) including the detection pattern recorded by the marking head 500 is acquired by the image sensor 705. The acquired image pickup signal is A / D converted and sent to the image processing unit 707. The image processing unit 707 performs image processing including pattern matching processing, which will be described later, on a plurality of image data obtained by performing imaging a plurality of times as the sheet moves, thereby moving the sheet movement amount (conveyance amount). Is detected (movement information acquisition). Information on the amount of movement of the sheet thus obtained is fed back to the control unit 6.

  4A and 4B illustrate a basic process according to an embodiment of the present invention for detecting the amount of movement of a recording medium (sheet) based on image data captured by an image sensor. It is a figure to do. The recording medium 8 is conveyed in the direction of arrow A shown in FIG. 4A, FIG. 4A shows a state before the recording medium moves by the conveyance, and FIG. 4B shows the recording medium by conveyance. Each of the states after a certain time after moving is shown.

  In FIG. 4A, 401, 413, 414, and 415 respectively indicate detection patterns recorded by the nozzle 411 of the marking head 500. An area 402 indicated by diagonal lines is an imaging area captured by the image sensor 705. In this configuration, an area 400 surrounded by a square shown in FIG. 4A is an area of a template image including a detection pattern (401) extracted from the imaging area 402 by the image sensor 705. In the example shown in FIG. 4, the marking head 500 has two nozzle rows. However, the marking head 500 may be two or more rows or less.

  On the other hand, in FIG. 4B, reference numeral 405 denotes a detection pattern after the detection pattern 401 has moved until the next imaging. Also, an area 407 indicated by hatching is an imaging area imaged by the image sensor 705, as in FIG. Further, an area 408 surrounded by a square is a search area for detecting the area 400 of the template image extracted by the previous shooting in the imaging area 407. The search area 408 is an area that is wider than the area of the template image and for detecting by pixel matching which pixel position in the search area the same image as the template image exists. When performing pattern matching, for example, an interphase value is calculated using an arithmetic expression according to the following NCC method, and a pixel position with the highest correlation is searched.

  The amount of movement of the recording medium is calculated from the search result. For example, if no conveyance deviation originally occurs, the detection pattern should exist at the position of the detection pattern 409 shown in FIG. 4B, but the detection pattern (401) exists at the position of the detection pattern 405. Is detected. In this case, the movement amount of the recording medium includes a detection pattern 401 (template image 400) shown in FIG. 4A and a detection pattern 405 (area 410 having the same size as the template image 400) shown in FIG. It is calculated based on the positional relationship. It should be noted that what is obtained from the positional relationship before and after movement of the template image is not limited to the movement amount, and for example, the movement speed can be obtained using the time required for movement of the movement amount. In this specification, the movement amount, movement speed, and the like obtained based on the positional relationship are referred to as movement information.

  Reference numeral 406 denotes a template image (newly recorded) for calculating the next movement amount. By repeating the same processing for this image, the movement amount of the recording medium can be calculated in real time.

  Here, before describing pattern matching according to an embodiment of the present invention, a conventional pattern matching described in Patent Document 1 will be described as a comparative example. In this pattern matching, an area having the greatest correlation with the template image including the detection pattern is searched from within the search area to detect where the template image exists in the search area, and the amount of movement is determined from the result. Is calculated. 5A and 5B are diagrams for explaining conventional pattern matching. In these drawings, reference numeral 100 denotes an imaging area in which a recording medium is imaged by an image sensor or the like, and 101 denotes a template image including a detection pattern 102. As shown in FIG. 5A, a template image 101 is acquired and held from an imaging area 100 obtained by imaging with an image sensor. After a certain time, as shown in FIG. 5B, the image data of the imaging area 100 is acquired by the image sensor. When the image sensor is fixed and the recording medium moves, the template image including the detection pattern 102 moves to a position such as the image 105 or the image 104 according to the transport state at that time. In contrast, the amount of movement is calculated by searching where the template image has moved in the imaging area 100 shown in FIG. That is, if no transport error occurs, the template image exists at the position of the image 104. If a transport error occurs, the template image moves to the position of the image 105, and this position and the template shown in FIG. The movement amount is calculated based on the positional relationship with the image 101.

  The method for searching where the template image is in the image area is as follows. As shown in FIG. 5B, first, a search area 103 having a predetermined size including a template image is set. At this time, the search area 103 is required to have a template image in this area, and therefore, the search area 103 is set to an area wider than the template image in consideration of a transport error or the like. Thereafter, the position where the template image exists in this search area is searched by image processing (cross-correlation processing).

  FIG. 6 is a diagram for explaining a conventional template image search method. In FIG. 6, reference numeral 110 corresponds to the search area 103 shown in FIG. First, in the search area 110, an area having the same size as the pixel size of the template image is cut out from the upper left pixel position as in the area 106. Then, a correlation value with the data of the template image 101 held in the first imaging is acquired by cross-correlation processing, and the result is held. Next, an area having the same size as the pixel size of the template image is similarly extracted at the extraction position shifted by one pixel in the search area 110 as in the area 107. Then, a correlation value with the template image is acquired and held. The same operation is repeated for regions 108 and 109, each shifted by one pixel, and interphase values at all pixel positions in the search range are acquired. Then, the extracted pixel position of the region having the highest correlation with the template image 101 among the acquired correlation values becomes the pixel position where the template image exists in the image area captured for the second time, and the template image search is performed. Is completed.

  However, the cross-correlation process for obtaining the interphase value can be performed, for example, by a method called an NCC (Normalized Cross-Correlation) method, but the processing of this method has a heavy processing load. In addition, in order to search for a template image, it is necessary to perform the above arithmetic processing at all pixel positions in the search area. For this reason, in order to search a template image from the entire search area within the required search time, for example, it is necessary to implement a plurality of arithmetic circuits for executing the above arithmetic expressions and perform parallel processing, which increases the cost. To do. In addition, if the CPU performs arithmetic processing, the template image itself may not be detected during the required search time.

  On the other hand, according to the pattern matching according to the embodiment of the present invention, it is possible to reduce the processing load for the search.

  FIG. 7 is a diagram for explaining pattern matching processing according to an embodiment of the present invention. In FIG. 7, the recording medium is conveyed in the direction of arrow A. At this time, the image sensor itself is fixed, and the moving recording medium is continuously imaged. Reference numerals 801, 802, and 803 denote template images each including a detection pattern 405 (see FIG. 4B). The template image exists at the position of the image 801 when there is no conveyance deviation or the like. On the other hand, when a conveyance deviation occurs, the image 802 moves to a position different from the position of the image 801, such as the image 803. When searching for a template image (801, 802, 803) from the search area 800 for this transport state, conventionally, as described above, correlation processing is sequentially performed for each pixel from the position of the area 809 shown in FIG. This is performed for the entire search area 408, but the load of correlation processing increases.

  On the other hand, in the present embodiment, the search area 408 (see FIG. 4B) is divided into four areas of areas 804, 805, 806, and 807. Then, the densities of the image data in the four divided areas are compared. In this case, the recording head 500 and the sensor unit 701 that record the detection pattern are located upstream of the recording head that records image data, as described above with reference to FIG. No recording pattern other than the detection pattern is recorded on the recording medium. Accordingly, the area where the detection pattern does not exist is, for example, a paper white area which is the background color of the recording medium. For this reason, in the case of this example, there is a density difference in the image captured by the image sensor between the area where the detection pattern exists and the area where the detection pattern does not exist. Specifically, in the area where the detection pattern exists, there is a pixel of image data whose brightness detected by the image sensor is lower (darker; higher density) than the other pixels depending on the recording density of the detection pattern. For this reason, in the four divided areas, the number of pixels having a lightness value equal to or smaller than a predetermined threshold is compared, and a template image including a detection pattern exists in an area where the number of pixels whose lightness is equal to or smaller than the predetermined threshold is large. Is estimated. Then, this divided area is specified as a search rear.

  FIG. 8 is a block diagram showing a configuration for processing for comparing brightness between divided areas according to the present embodiment. Specifically, FIG. 8 shows a circuit 901 that executes processing for limiting a search area, calculating an interphase value, and calculating a movement amount, and the surrounding configuration. These circuits and the like are configured in the control unit 6 described above with reference to FIG.

  First, a template image is extracted from the image data of the imaging area obtained by imaging first and held in the external memory 902 and stored in the external memory 902. The held template image is used for correlation processing with an image captured later. Then, the image data of each pixel output from the image sensor 705 by the next imaging is compared with a predetermined threshold value relating to brightness in the image level comparison unit 903 of the circuit 901. If there is data below the threshold, the pixel counter 904 increments the count value by 1. Here, the image data obtained by the next imaging to be compared is provided for the comparison and stored in the external memory 902 for each pixel.

  Since the comparison of the image data can be realized by a comparatively simple comparison circuit, the circuit scale does not increase and the processing does not take time. For this reason, the comparison process can be completed during the process in units of pixels in which data is output from the image sensor. That is, this comparison processing does not affect the processing performance until the template image is searched. Also, the predetermined threshold value can be made variable according to the type of the recording medium, the color of the recording head of the detection pattern, and the like.

  The above comparison and counting are performed for all the pixels in each of the four divided areas described above with reference to FIG. 7, and the result is held in the external memory 902 for each of the four divided areas.

  Next, the search area setting unit 905 sets one divided area for searching the template image based on the count value for each of the four divided areas held in the external memory 902. Thereby, in the case of this example, the area to be searched can be made approximately ¼ area, and the search load can be reduced. In the present embodiment, the search area is divided into four areas. However, the number of areas to be divided is not limited to four, and may be changed according to various conditions.

  In the example shown in FIG. 7, when the template image has moved to the position of the image 802, the divided area 804 has a larger number of pixels below the threshold than the other areas, and the search area is shaded in FIG. Is set in an area 811 indicated by. Alternatively, when the template image exists at the position of the image 803, the number of pixels equal to or smaller than the threshold value in the divided area 806 is larger than the other areas, and the search area is set to an area 812 indicated by hatching in FIG. Since the template image is formed by the detection pattern and a predetermined number of pixels around the detection pattern, the detection pattern itself may exist at the boundary of the divided area as in the case of the template image 801. Assuming this case, the original divided area 804 is not set as the search area as it is, but the search area is widened by a predetermined number of pixels around the detection pattern described above, such as areas 813 and 814 shown in FIG. Set.

  When the search area is determined as described above, the interphase value calculation unit 906 reads the image data of the search area set by the search area setting unit 905 from the external memory 902. Then, a correlation value between the read image data and the template image acquired and held in the previous imaging is calculated by an arithmetic expression such as the NCC method described above and held in the interphase value holding unit 907. The correlation value calculation method is not limited to the NCC method, and other calculation methods may be used. The method for obtaining the correlation value itself is the same as the conventional method described above with reference to FIG. That is, in the search area, a region having the same size (shape) as the size (shape) of the template image is cut out from the upper left pixel position, and the correlation value between the image in this region and the template image is acquired by cross-correlation processing. Then, the position of the region to be cut out is shifted by one pixel, and thereafter the same processing is repeated.

  Next, the movement amount calculation unit 908 determines the pixel position having the highest correlation value among the correlation values held in the interphase value holding unit 907, and based on the relationship between the pixel position and the pixel position of the template image, The amount of movement is calculated.

  FIG. 9 is a graph illustrating an example of the relationship of the correlation value with respect to the pixel position obtained by the movement amount calculation unit 908 according to an embodiment of the present invention. As illustrated in FIG. 9, the movement amount calculation unit 908 obtains an approximate curve by curve fitting for the interphase value for each pixel position calculated by the interphase value calculation unit. Then, the pixel position 911 having the highest correlation value in the data of the approximate curve is obtained, and this is finally set as the pixel position to be obtained. Thereby, the movement amount can be calculated in pixel units (sub-pixel level) equal to or lower than the resolution of the image sensor 705.

  By the way, in the above-described process of measuring the number of pixel data below the threshold for each divided area, the number of pixels below the threshold may be the same in a plurality of divided areas. In this embodiment, when there are a plurality of divided areas having the same number of pixels equal to or smaller than the threshold value, the search area is set across the divided areas.

  FIG. 10 is a diagram illustrating processing for setting a search area across a plurality of divided areas according to an embodiment of the present invention, and an example in which a template image including a detection pattern 405 has moved to the position of an image 1002. Show. In this example, the detection pattern 405 (a pattern indicated by four circles in the figure, which actually has a predetermined density of a color that is not white) occupies the same area (number of pixels) in each of the divided area 1006 and the divided area 1007. It is an example. In this case, the number of pixels below the threshold counted in each of the divided area 1006 and the divided area 1007 is the same. In this case, a search area 1008 extending over the image areas 1006 and 1007 is set. When the template image exists at the boundary portion as described above, the search area 1008 to be set does not need to be unnecessarily widened, and may have the same number of pixels as the search areas 811 and 812 described above with reference to FIG. Thereby, it is possible to suppress an increase in processing time without unnecessarily widening the search area.

FIG. 11 is a flowchart showing processing until the movement amount according to the embodiment of the present invention described above is calculated, and is mainly processing executed by the configuration described above with reference to FIG. In FIG. 11, first, in step 1100, the brightness of each pixel input from the image sensor is compared with a predetermined threshold value. As a result of the comparison, if it is equal to or smaller than the threshold value, in step 1102, the number of pixels equal to or smaller than the threshold value is counted up. This count-up process is performed for each divided area. Next, in step 1103, it is determined whether all the pixels in the search area have been compared. If not completed, the process returns to step 1100 to repeat the same processing. Next, in step 1104, the number of pixels equal to or smaller than the threshold value in each divided area is compared, and in step 1105, a search area is set based on the comparison result. In step 1106, in the set search area, the highest pixel position between the template image and the phase is calculated by the cross-correlation process by the process described above with reference to FIG. Then, in step 1107, the movement amount at the sub-pixel level is calculated. As described above, according to the embodiment of the present invention, it is possible to reduce the processing load for searching the template image.

705 Image sensor 902 External memory 903 Image level comparison unit 904 Pixel number counter 905 Search area setting unit 906 Correlation value calculation unit 907 Correlation value holding unit 908 Movement amount calculation unit

Claims (10)

An image processing apparatus that obtains movement information of an object by imaging an object having a detection pattern during the movement of the object,
A means for searching a template image including a detection pattern extracted from the image data of the previously captured object in the image data of the subsequently captured object, and detecting the brightness of the search area in the image data of the subsequently captured object A search means for specifying a divided area smaller than the search area and searching for the template image in the divided area;
Movement information acquisition means for obtaining movement information for the object based on a positional relationship between the searched template image and a template image extracted from the image data of the previously captured object;
An image processing apparatus comprising:   The search means is divided into a plurality of divided areas in advance in the search area, detects the brightness for each of the plurality of divided areas, and specifies the divided area having the lowest brightness as a divided area smaller than the search area. The image processing apparatus according to claim 1.   The search means, when there are a plurality of divided areas having the same brightness detected for each of the plurality of divided areas, specifies an area extending over the plurality of divided areas as a divided area smaller than the search area. The image processing apparatus according to claim 2.   The search means includes means for comparing the brightness of the image data in the search area with a predetermined threshold value for each pixel, and counting the number of pixels that are equal to or less than the predetermined threshold value for each of the plurality of divided areas. The image processing apparatus according to claim 2, wherein the divided area having the largest counted number is specified as a divided area smaller than the search area.   The image processing apparatus according to claim 2, wherein the number of the plurality of divided areas is changeable. A pattern matching method for searching a predetermined template image in image data,
A pattern matching method, wherein a division area smaller than the search area is specified by detecting brightness of the search area, and the template image is searched in the division area.   7. The search area is divided into a plurality of divided areas in advance, brightness is detected for each of the plurality of divided areas, and a divided area having the lowest brightness is specified as a divided area smaller than the search area. The pattern matching method described in 1.   8. When there are a plurality of divided areas having the same brightness detected for each of the plurality of divided areas, an area extending over the plurality of divided areas is specified as a divided area smaller than the search area. Pattern matching method.   The brightness of the image data in the search area is compared with a predetermined threshold value for each pixel, the number of pixels that are equal to or less than the predetermined threshold value is counted for each of the plurality of divided areas, and the divided area having the largest counted number The pattern matching method according to claim 7, wherein: is specified as a divided area smaller than the search area.   The pattern matching method according to claim 7, wherein the number of the plurality of divided areas is changeable.