JP4724799B2 - Image processing system and program - Google Patents

Description

  The present invention relates to an image processing system that performs image processing on an image obtained by photographing an area including one or more identification target areas.

  2. Description of the Related Art Conventionally, a technique has been used in which a region of a two-dimensional code is specified from an image showing a two-dimensional code as an identification target region, and information encoded by the two-dimensional code is identified (see Patent Document 1). .

JP 2001-195536 A

  However, in the conventional technique, an image acquired when identifying information is configured to use an image in which one identification target region is included in a large part of the image region. Each of the identification target areas included in the image cannot be identified from the image in which the plurality of identification target areas are included in the area.

  The present invention has been made to solve such a problem, and an object thereof is to provide a technique for specifying each of one or more identification target regions included in an image.

  In order to solve the above-described problem, an image processing system that performs image processing on an image in which an area including one or more identification target areas is photographed may be configured as shown in a first configuration (claim 1) below. .

In this configuration, an image acquisition unit that acquires an image obtained by capturing an area including one or more identification target regions, and an image acquired by the image acquisition unit is converted into a component that indicates brightness included in the image . Component conversion means for converting into a component image indicating a distribution of a specific component that is a component equal to or greater than a predetermined threshold , and a rule determined in advance from the component image converted by the component conversion means , Region extraction for extracting a region formed according to a rule for determining whether or not to be a candidate region for the identification target region based on a parameter of an outer shape of the identification target region as a candidate region composed of the identification target region And the corresponding region corresponding to the candidate region extracted by the region extracting unit among the images acquired by the image acquiring unit. The formation elements, said pattern being formed in the corresponding area is a pattern for determining whether or not the identification target region characteristic pattern determined pattern determination means for determining whether contains If the corresponding area is determined to contain a characteristic pattern by the pattern determination means, a detection region detecting means as the recognition target area, the conversion of component image of the image by said component conversion means 2 For each component image converted for each component, the candidate region is extracted by the region extracting unit, the characteristic pattern is determined by the pattern determining unit, and the region is detected. Repetitive execution means for repeatedly detecting the identification target area by the means, and for the specific component image, the area Based on both candidate areas, whether each candidate area extracted by the output means matches a candidate area extracted by the area extracting means for another component image (hereinafter referred to as “extracted candidate area”). And a coincidence judging means for judging.

  Then, the repetitive execution means is extracted by the area extracting means for determining whether or not it has a characteristic pattern by the pattern determining means and for detecting the identification target area by the area detecting means. Of the candidate areas, the process is performed only on the candidate areas that are determined not to match the extracted candidate areas by the match determination unit.

  In this configuration, based on a component image generated from an externally acquired image (original image), extraction of candidate regions in the component image and features corresponding to the corresponding regions in the original image corresponding to the extracted candidate regions are performed. Through the determination of whether or not a typical pattern is included, it is possible to detect the area of the identification target area from the original image.

  As described above, in the original image area, the corresponding area including the characteristic pattern among the corresponding areas corresponding to the candidate areas of the component image is detected as the area of the identification target area in the original image. Each of the one or more identification target areas included in the image can be specified.

  Further, in this configuration, for each of a plurality of types of component images generated based on the original image, extraction of candidate regions, determination of whether or not the extracted candidate regions include a characteristic pattern, and identification The target area is repeatedly detected.

  As a result, even if the region of the identification target region cannot be detected based on the specific component image, it can be expected to be detected based on another component image. As a result, the detection accuracy of the identification target region in the original image is expected. Can be increased.

  Also, in this iterative process, the candidate area newly extracted for a specific component image may match the extracted candidate area extracted for the previously referenced component image in the area of the original image. is assumed. However, in the above configuration, for the extracted candidate regions that match in this way, the determination as to whether or not the extracted candidate region includes a characteristic pattern and the detection of the identification target region are not performed.

  Therefore, for each identification target area detected based on each of the plurality of component images, when identifying the information indicated in the identification target area, the identification target area that should originally be the same is repeatedly identified. Unnecessary processing burden can be eliminated.

  Further, in the above configuration, it is possible to extract a candidate area consisting of an identification target area from an image acquired from the outside, and then identify the identification target area from this area to identify information. Therefore, compared to the case where the identification target region is to be detected without extracting the candidate region, the identification target region can be detected from the limited range of the extracted candidate region. The processing load and processing time required until identification can be suppressed.

  In this configuration, the component image converted from the original image only needs to indicate the distribution of specific components included in the original image. For example, specific components such as luminance, brightness, saturation, and specific colors in the original image are included. An image in which only the components above the predetermined threshold value are extracted (specifically, binarized by the threshold value).

  In addition, the “predetermined rule” for extracting candidate regions from the component image only needs to define a candidate region consisting of identification target regions. For example, the outer shape and outer shape of the identification target region are The length of each side when it is a polygon, the angle formed by each side, the ratio of the length of each side when the outer shape is a polygon, and the component when the identification target area is included in the component image The distribution pattern (component distribution state, arrangement of distributed components, etc.), etc. may be specified.

  In addition, the “characteristic pattern” referred to when detecting the area of the identification target area from the original image may be a characteristic pattern determined for the identification target area. In the case of a two-dimensional code, if the two-dimensional code is used, the arrangement pattern of a cell that becomes a coding symbol in the two-dimensional code, the arrangement pattern of a specific symbol arranged to identify the two-dimensional code, or the two-dimensional code A pattern of marks arranged to identify a certain thing.

Moreover, the said structure is good to make it like the 2nd structure (Claim 2) shown below.
In this configuration, for each identification target area detected by the area detection means, information identification means for identifying information indicated in the identification target area is provided.

With this configuration, for each identification target area detected based on each of the plurality of component images, information indicated in the identification target area can be identified.
In the above configuration, the specific configuration for identifying the information indicated in the identification target area is not particularly limited. For example, in the identification target area, a plurality of cells are arranged and each cell is colored. Thus, when the two-dimensional code is obtained by coding each of one or more pieces of information corresponding to the color or the combination of colors, the third configuration (Claim 3) described below may be used.

In the third configuration, the information identification unit identifies information based on a color or a combination of colors assigned to each cell in the two-dimensional code that is the identification target region.
With this configuration, it is possible to identify the information indicated in the identification target area based on the color attached to the two-dimensional code.

  In this configuration, there is no problem as long as the entire two-dimensional code is normally generated as an image. However, depending on the image generation (specifically, shooting) state, an area corresponding to some cells may be missing. It is also possible that the image is not normally generated as an image, and in this case, the information itself cannot be identified.

  For such a problem, it is desirable that even if a region corresponding to a part of cells is missing, this can be compensated so that appropriate information can be identified. . As a specific configuration for this purpose, for example, the fourth configuration (claim 4) shown below can be considered.

  In the fourth configuration, in the identification target area, a color assigned to each of the plurality of cells is information to be encoded in the cell, and a cell adjacent to the cell among a plurality of types of colors. It is determined according to the coloring rules corresponding to the attached colors.

  Further, for each identification target region detected by the region detection means, out of each cell in the two-dimensional code that is the identification target region, the color to be assigned to the cell according to the coloring rule, and the two Color estimation means for estimating a color to be originally assigned to the cell according to the coloring rule for a cell having a color other than the color to be used in the dimension code. Then, the information identifying means is assumed that, among the cells in the two-dimensional code that is the identification target area, the cells whose color is estimated by the color estimating means are assigned with the estimated color. The information is identified based on the color or color combination attached to the.

  With this configuration, even if an area corresponding to some of the cells in the two-dimensional code is missing, the color to be originally assigned to the cell is estimated according to the coloring rule, and the color Since the information can be identified based on the information, it is possible to prevent the information itself from being identified.

  In this configuration, when a plurality of colors are estimated as the cell color estimated according to the coloring rule, any one of the colors is selected based on parameters such as the probability of the estimation result. And then presume that the selected color is attached. When a plurality of colors are estimated, information is identified for each of the patterns with a plurality of colors, and the user is allowed to select one of the identified information. May be.

 Further, when the identification target region is a two-dimensional code, it is necessary to determine the color of each cell. For example, a fifth configuration (Claim 5) shown below can be used for this purpose. It is possible to do.

  In the fifth configuration, for each identification target region detected by the region detection means, a plurality of colors used in the two-dimensional code are assigned to each cell in the two-dimensional code that is the identification target region. Color determination means for determining whether the color is included in a determination region on a color space that extends centering on each of the colors. Then, the information identifying unit assumes that each cell in the two-dimensional code that is the identification target region is assigned the color determined by the color determining unit for the cell, and the color or color assigned to each cell. The information is identified based on the combination.

With this configuration, the color assigned to each cell in the two-dimensional code can be determined based on which determination region in the color space is included.
However, in this configuration, depending on the image generation conditions, it may be considered that colors assigned to many cells are not included in any of the determination regions, and in this case, information identification itself cannot be performed.

  For such a problem, even if the colors given to many cells are not included in any of the determination areas, this is compensated so that appropriate information can be identified. It is desirable. As a specific configuration for this purpose, for example, a sixth configuration (Claim 6) and a seventh configuration (Claim 7) may be considered.

  In the sixth configuration, the color determination unit may determine the determination region for an identification target region including a predetermined number or more of cells that are determined not to be included in any of the determination regions corresponding to a plurality of colors. After expanding a certain area, the color assigned to each cell is determined again.

  With this configuration, even if the image generation conditions are poor and the color cannot be determined properly, the color determination probability can be increased by increasing the determination sensitivity and determining the color again by expanding the determination area. Can be increased.

  In this configuration, the expansion width when expanding the determination area may be expanded up to a range that does not interfere with other determination areas, and the number of cells determined not to be included in any of the determination areas is less than a certain number. The determination area may be gradually expanded until the determination is repeated.

  In the seventh configuration, the color determination unit is configured to determine, for an identification target area where a predetermined number or more cells are determined not to be included in any of the determination areas corresponding to each of a plurality of colors. The determination area is shifted in the color space in accordance with the color distribution in, and the color assigned to each cell is determined again.

  With this configuration, even when the image generation conditions are poor and the color cannot be determined appropriately, the color is determined again by shifting the determination region to an appropriate position according to the color distribution, The color determination probability can be increased.

  In this configuration, the determination area may be shifted to a position on the color space where the deviation of the color component in the identification target area can be canceled. More specifically, when a specific color component is increased as a whole in the identification target region, it can be considered that the determination region is shifted in a direction of decreasing the color component.

  In the sixth and seventh configurations, the “certain number” in “there are more than a certain number of cells determined not to be included in any of the determination areas” is the number of the corresponding cell in all cells. It may be a ratio or the absolute number of corresponding cells. In addition, the number of corresponding cells arranged in a specific pattern in the two-dimensional code may be used.

  In each of the above configurations, whether each candidate area extracted from the component image matches the extracted candidate area in another component image is determined according to, for example, the degree of overlap in both candidate areas. And it is sufficient.

As a configuration for this purpose, each of the above configurations may be configured as an eighth configuration (claim 8) shown below.
In this configuration, the coincidence determining unit determines whether each of the candidate regions extracted by the region extracting unit matches the extracted candidate region in another component image with respect to the specific component image. The determination is made according to the degree of overlap in both candidate areas.

  With this configuration, both candidate areas coincide with each other if the degree of overlap between the candidate area extracted from the specific component image and the extracted candidate area in another component image is greater than or equal to a certain level. Then it can be determined.

  Also, whether each candidate area extracted from the component image matches the extracted candidate area in another component image is determined by one candidate area included in the other candidate area being included in the other candidate area. It may be determined according to whether or not there is.

As a configuration for this purpose, each of the above configurations may be configured as a ninth configuration (claim 9) described below.
In this configuration, the coincidence determining unit determines whether each of the candidate regions extracted by the region extracting unit matches the extracted candidate region in another component image with respect to the specific component image. The determination is made according to whether one candidate region of both identification target regions is included in the other candidate region.

  With this configuration, one candidate area extracted from a specific component image and one candidate area out of the extracted candidate areas in another component image are included in the other candidate area. Can be determined to match.

  In addition, if the candidate area is a polygon, whether each candidate area extracted from the component image matches the extracted candidate area in another component image is determined as a polygon formed as a candidate area. The determination may be made according to whether or not one or more vertices in is located within a certain range from any vertex in the polygon formed as the extracted candidate region.

As a configuration for this purpose, each of the above configurations may be configured as a tenth configuration (claim 10) described below.
In this configuration, the coincidence determination unit is configured such that, for each candidate region extracted by the region extraction unit in the specific component image, one or more vertices in a polygon formed as the candidate region are the extracted candidate region. Each of the candidate areas extracted by the area extracting means is different from the extracted candidate area in another component image depending on whether or not it is located within a certain range from any vertex in the polygon formed as It is determined whether or not they match.

  With this configuration, when one or more vertices in a candidate area extracted from a specific component image are located within a certain range from any one of the extracted candidate areas, the two candidate areas match. Can be determined.

  Here, “within a certain range from any vertex” means a range determined by the positional relationship between a candidate area extracted from a specific component image and an extracted candidate area in another component image. It should be inside.

  As a condition for determining that both candidate areas match, at least one vertex in a candidate area extracted from a specific component image is within a certain range from any vertex in an extracted candidate area in another component image. It only has to be located. Further, when it is determined that both candidate areas match, it may be a condition that a number of vertices are located within a certain range, and depending on the processing load and the processing speed until the identification target area is detected, etc. It is possible to change.

  Further, the area extracting means in the above configuration is converted by the component converting means on the basis of a template that defines rules for forming the identification target area, as in an eleventh configuration (claim 11) described below. An area formed in accordance with a rule defined in the template may be extracted from the component image as a candidate area.

  With this configuration, candidate areas can be extracted in accordance with the rules defined in the template. Therefore, by changing the rules together with this template, it can be detected as an identification target area formed in accordance with an arbitrary rule. be able to.

  In this configuration, as in the twelfth configuration (Claim 12) described below, the template defines a rule for forming a two-dimensional code region obtained by encoding predetermined information as the candidate region. It is possible to use a thing.

With this configuration, it is possible to detect a region of the two-dimensional code as an identification target region and identify information encoded with the two-dimensional code.
Note that the image processing system described above may be configured as a single device, or may be configured as a system in which a plurality of devices operate in cooperation.

  Further, as a configuration for solving the above problems, a program for causing a computer system to execute various processing procedures for causing all the means in any one of the first to twelfth configurations to function as the means (claim 13). ).

A computer system controlled by such a program can constitute a part of the image processing system according to each of the above-described configurations.
Each program described above consists of an ordered sequence of instructions suitable for processing by a computer system, and is provided to the image processing system and users who use it via various recording media and communication lines. It is what is done.

Block diagram showing the overall configuration of the image processing system Diagram showing the structure of a two-dimensional code Flow chart showing information identification processing The figure which shows another method for determining whether each candidate area | region extracted from the component image corresponds with the extracted candidate area | region. Flow chart showing color determination processing

Embodiments of the present invention will be described below with reference to the drawings.
(1) Overall Configuration As shown in FIG. 1, the image processing system 1 is configured as a single server that can communicate with each of one or more cameras 2 via a network 100.

  Each of the cameras 2 is a well-known camera configured to generate image data indicating an image of a predetermined shooting area and sequentially transmit the image data to the image processing system 1. The camera 2 only needs to transmit image data indicating a still image captured at a predetermined timing, and image data indicating each still image continuously captured as a moving image along the time axis. May be transmitted at any time.

  The image processing system 1 includes a control unit 11 that controls the entire image processing system 1, a storage unit 13 that stores various information, a user interface (UI) 15 including an operation unit and a display unit, and input / output of data from the outside. This is a well-known computer system including an input / output interface (I / O) 17 for controlling, a communication unit 19 for controlling communication via the network 100, and the like.

  The image processing system 1 in the present embodiment is configured as a system that identifies and acquires information indicated by an identification target area in the image based on image data indicating an image including one or more identification target areas. It is a thing. In the following description, the system configuration is illustrated by identifying information encoded by a two-dimensional code as an identification target region.

Two-dimensional code in the present embodiment, as shown in FIG. 2, a plurality of cells C11~Cpq (2 ≦ p, 2 ≦ q) are arranged at regular intervals on the background area 2 10 black, the predetermined information It is coded by the color and color combination of each cell. In other words, this two-dimensional code is used for each cell or each group of cells related to a combination of multiple colors (multicolors) or multiple combinations of color values such as numbers and letters. (Multi-value) can be expressed.

Mark where the cell C11~C1q of the first line constituting the two-dimensional code (uppermost row in FIG. 2) is the color of the attaching et Leu Ru mark to each cell C of the two-dimensional code, in order The header area is marked with (cell C is filled).

In the present embodiment, as colors of marks that can be attached to the cell C, one or more colors excluding black (k) are arranged in a set with the cell C marked with black (k).
Also, among the cells C in each row, the cells C11 to Cp1 located at the head (the leftmost column in FIG. 2) are arranged as specific cells for specifying the position of the two-dimensional code. Then, (in the present embodiment, black (k)) obtained color applied to the specific cell by the background area 2 10, a characteristic pattern indicating the area of the two-dimensional code is formed. This characteristic pattern may be realized by another configuration.

The cells C11 to Cpq in the second and subsequent rows (excluding the leading cells C11 to Cp1) are divided into cell groups each including one or more cells C for each row, and the unit to be indicated by the cell group A mark is attached with a color or a combination of colors associated with information (one or more characters). In this way, each cell or each cell group related to one or more combinations can have information consisting of any one of a plurality of types of characters or any value represented by a multi-digit number. Yes.
(2) Processing by Control Unit 11 Hereinafter, procedures of various processes executed by the control unit 11 according to a program stored in the internal memory or the storage unit 13 will be sequentially described.
(2-1) Information Identification Processing First, an information identification processing that is started whenever the image processing system 1 receives image data from the camera 2 or when image data is input via the input / output interface 17. The processing procedure will be described with reference to FIG.

  This information identification processing is based on each component image of an image acquired from the outside, extraction of candidate areas included in the original image, whether or not it is a two-dimensional code, and when it is a two-dimensional code In this process, information is identified through a procedure such as identification of information.

  When this information identification process is activated, first, a template defining information characterizing the two-dimensional code is read from the storage unit 13 (s110). Here, the template indicates a region rule that defines a region composed of a two-dimensional code, a characteristic pattern in the two-dimensional code, a coloring rule of a color (number of colors, types of colors) attached to the two-dimensional code as a mark, and the like. Information.

  Among these, the “region rule” means, for example, the outer shape of the two-dimensional code, the length of each side when the outer shape is a polygon, the angle formed by each side, and the outer shape is a polygon. Specifies one or more of the ratio of the length of each side in the component, the distribution pattern of the component when the component image contains a two-dimensional code (component distribution state, arrangement of distributed components, etc.) Anything to do. However, in order to reduce the processing burden required for the extraction of candidate areas by the subsequent processes, it is desirable that a plurality of items be selected and defined. In this case, the number of extractions as candidate areas is reduced. As a result, a reduction in processing time is realized with a reduction in processing load.

  In addition, the “characteristic pattern” is, for example, a cell arrangement pattern that is a coding symbol in a two-dimensional code, a specific symbol arrangement pattern arranged to identify a two-dimensional code, This is a pattern of marks (a header area mark in this embodiment) arranged to identify a two-dimensional code.

  In addition, the “coloring rule” is a color rule to be assigned to each of a plurality of cells in a two-dimensional code, and depends on information to be coded in the corresponding cell and colors assigned to cells adjacent to the corresponding cell. Can be determined.

Next, variables X and Y to be referred to in the subsequent processing are initialized (0 → X, 1 → Y) (s120).
Next, image data received or input from the outside is acquired (s130). Here, image data is acquired from the outside, and a copy of the image indicated by this image data is created, and the outer shape of the two-dimensional code is formed in accordance with the region rule described above among the regions of the image to be copied. A process of extracting (for example, emphasizing) a boundary portion (a straight line portion as a specific example) is also performed. Here, not only the boundary portion according to the region rule but also the portion formed by the parameter of the error range allowed as the portion to be the outer shape is extracted.

  Thus, after the image data is acquired, an image (original image) that is a copy of the image indicated by the image data is converted into a component image, and the subsequent processing is performed on the component image. This component image is an image obtained by binarizing components included in the original image, and the subsequent processing is performed on each of the first to nth (n is an arbitrary number) component images. become.

In the present embodiment, the component image is an image obtained by binarizing the original image with different threshold values from the first to m-th (m <n) component images, and the m + 1-th to n-th images. The component image is an image obtained by binarizing the original image using only nm color components. Here, “ n ” is a value set in the variable n . In this embodiment, a value (maximum value 6 in this embodiment) corresponding to the detection accuracy of the two-dimensional code requested by the user is arbitrarily set. It is configured to be possible.

  More specifically, the first to third component images are determined as binarized images with thresholds that increase in order, such as luminance 120/255, 150/255, 200/255. The fourth to sixth component images are defined as binarized images in the order of red (R), green (G), and blue (B). This threshold value for binarization is used to extract effective candidate areas in the subsequent processing according to the "region rules" and "characteristic patterns" of the two-dimensional code, and to identify the two-dimensional code appropriately. It is determined as a suitable value.

  In addition, although the example which binarized with the threshold value of a predetermined brightness | luminance and a color component was shown here, it is good also as binarization with the threshold value of brightness and saturation other than a brightness | luminance, The color components may be binarized using color components other than red, green, and blue. In addition to generating a component image by binarization, for example, an image in which only specific components such as luminance, brightness, saturation, and specific color in the original image are distributed may be generated as the component image.

  After the image data is acquired in s130, based on this image data, among the first to n-th component images, the component image having the number indicated by the value currently set in the variable Y (the Y-th component image) Component image) is generated (s140). Here, the Y-th component image is generated by binarizing the original image with a threshold value for realizing conversion to the Y-th component image.

  Next, on the basis of the component image generated in s140, each candidate region consisting of a two-dimensional code is specified from this component image (s150). Here, based on the “region rule” in the template read out in s110, each candidate region (coordinates in the image defining the two-dimensional code) is specified from the Yth component image. . The “region according to the region rule” includes not only a region according to the region rule but also a region formed with parameters of an error range allowed as the region.

  Next, each candidate area specified in s150 is registered in the list (s160). In s160, if no candidate area is specified in s150, no processing is performed.

  Next, it is checked whether or not there is a candidate area that has not been processed (unprocessed) in the subsequent processing among the candidate areas registered in the list in s160 (s170). Here, even if no candidate area is registered in the list, it is determined that there is no unprocessed candidate area.

  If it is determined in s170 that there is an unprocessed candidate area (s170: YES), any candidate area is selected from the unprocessed candidate areas registered in the list (s180).

  Next, it is checked whether the candidate area selected in s180 is valid (s190). Here, when the selected candidate region is a candidate region within an error range arbitrarily set as a region according to the region rule, it is determined that the selected candidate region is effective.

  If it is determined in s190 that the area of the two-dimensional code is not valid (s190: NO), the process returns to s170, and thereafter, the same process is performed for another candidate area.

  On the other hand, if it is determined in s190 that the two-dimensional code area is valid (s190: YES), the candidate area specified in s150 performed immediately before is the above-described candidate area. It is checked whether it is the same as the candidate area specified in s150 (s200).

  Here, the candidate area identified in s150 (new candidate area) is any candidate area (extracted candidate area) identified in s150 performed in the past and stored in the internal memory or the storage unit 13. If a region overlaps with a predetermined ratio or more, or if one of the new candidate region and the extracted candidate region is included in the other, it is determined that the new candidate region is the same as the extracted candidate region. .

  In this embodiment, since the candidate area is a polygon, whether each candidate area extracted from the component image matches the extracted candidate area in another component image is determined as a polygon formed as a candidate area. It is also possible to make a determination according to whether one or more vertices Pn in is located within a certain range (for example, a certain radius r distance) from any vertex Po in the polygon formed as the extracted candidate region Good (see FIG. 4).

  Here, “within a certain range from any one of the vertices” means that both candidate areas when a candidate area extracted from a specific component image matches an extracted candidate area in another component image. May be within a range determined by the positional relationship in which the position can be located, and is not limited to the above-described constant radius r distance.

  If it is determined in s200 that the new candidate area is the same as the extracted candidate area (s200: YES), the process returns to s170, and the same process is performed for another candidate area thereafter.

  On the other hand, when it is determined in s200 that the new candidate area is not the same as the extracted candidate area (s200: NO), the new candidate area at this time is corrected so as to be suitable for information identification. (S210).

  Here, the shape and angle of the two-dimensional code in the new candidate region are corrected. Specifically, for example, when the region to be processed is arranged obliquely or has a distorted shape that is not a square, it is rotated or adjusted to the coordinate axis to correct such a shift. Extensions and shortenings are made. In addition, the area itself is not corrected, but the angle and length of the coordinate axis used as a reference when determining whether or not the code is a two-dimensional code and identifying information from here are corrected. It is good. Further, in this s210, the color component of the two-dimensional code in the new candidate area is corrected so as to enhance the color discrimination in each cell.

  Next, in the original image indicated by the image data acquired in s130, a corresponding pattern corresponding to the candidate area corrected in s210 includes a characteristic pattern as a two-dimensional code. Is checked (s220).

  Here, the candidate area corrected in the above s210 is scanned for each unit area (area composed of one or more dots), and the cell (particularly the header area) and mark pattern thus identified are displayed in the above s110. The code area contains a characteristic pattern as a two-dimensional code when it matches the cell layout pattern or mark pattern specified by the “characteristic pattern” in the template read out It is determined.

  If it is determined in s220 that a characteristic pattern as a two-dimensional code is not included (s220: NO), the process returns to s170, and thereafter the same processing is performed for another code area.

  On the other hand, when it is determined in s220 that a characteristic pattern as a two-dimensional code is included (s220: YES), a new candidate area at this point is detected as a two-dimensional code area (s230).

Next, for the area of the two-dimensional code detected in s230, a color determination process for specifying the color assigned to each cell in the two-dimensional code is performed (s232).
This color determination process is executed using the area of the two-dimensional code detected in s230 as an argument, and the color assigned to each cell in the two-dimensional code becomes a return value. A specific processing procedure in this color determination processing will be described later.

  Next, for the area of the two-dimensional code detected in s230, based on the color specified in s232, the information indicated by the two-dimensional code is the color assigned to each cell C and the combination thereof. (S240), and the information thus identified is stored in the built-in memory of the control unit 11 or the storage unit 13 (s250).

Next, the variable X is incremented (X + 1 → X) (s260).
As described so far, the above-described processing from s180 to s250 means that the candidate region specified in s150 forms a two-dimensional code body. Therefore, in this s260, by incrementing this variable X, the number of candidate areas detected and identified as a two-dimensional code is integrated.

Next, at this time, it is checked whether or not the variable X is equal to or greater than a predetermined number, that is, whether or not a predetermined number or more of candidate areas form a body of a two-dimensional code (s262).
If it is determined in s262 that the variable X is greater than or equal to the predetermined number (s262: YES), the information identification process is terminated.

  On the other hand, when it is determined in s262 that the variable X is not equal to or greater than the predetermined number (s262: NO), the process returns to s170, and thereafter, the same processing is performed for another candidate region.

In this way, the above s170 to s262 are repeated for all the code areas registered in the list unless the number of candidate areas detected and identified as two-dimensional codes exceeds a predetermined number, and thereafter for all candidate areas. When s170 to s262 are performed, it is determined that there is no unprocessed candidate area in s170 (s170: NO), and the value of the variable Y at this time is a value larger than the value n set in the variable n It is checked whether ( n <Y) or not (s270).

If it is determined in s270 that the value of the variable Y is not greater than the value n (s260: NO), the variable X is greater than or equal to a predetermined number at this point, that is, more than a predetermined number of candidates. It is checked whether or not the area has a body of a two-dimensional code (s280).

If it is determined in s280 that the variable X is greater than or equal to the predetermined number (s280: YES), the information identification process is terminated.
On the other hand, when it is determined in s280 that the variable X is not greater than or equal to the predetermined number (s280: NO), after the variable Y is incremented (Y + 1 → Y) (s290), the process returns to s140. The above steps s140 to s290 are performed based on the next component image.

  Thereafter, the above s140 to s290 are performed based on the next component image, and if the variable X becomes a predetermined number or more in the process, it is determined as “YES” in the above s280 and the information identification process is completed. On the other hand, if the variable X does not become a predetermined number or more during the process, the process for the next component image is repeated.

Then, after the s140~s290 is performed based on all of the component images, the value of the variable Y is greater than the value n, the value of the variable Y in the s170 is in the value n greater than In this case, the information identification process is also terminated.

  In this color determination process, first, regarding the area of the two-dimensional code passed from the information identification process, one of the cells C in this two-dimensional code that has not been processed in the subsequent processes (unprocessed) Cell) is selected (s310).

  Next, the color assigned to the cell C selected in s310 is determined (s320). Here, in any of the determination areas on the color space in which the color assigned to the cell C selected in the above s310 spreads around each of a plurality of colors used in the two-dimensional code as a central coordinate, It is determined by whether or not it is included (located in any determination region).

  Then, the cell C selected in s310 and the color determined in s320 are associated with each other and registered in the data table. Here, when a color is determined for the cell C registered in the data table, the color corresponding to the cell C is updated to the latest color thus determined.

  If it is determined in s320 that it is not included in any of the determination areas, the cell C selected in s310 is associated with a color outside the determination area. .

  Next, it is checked whether or not an unprocessed cell C that is not a processing target remains at this time (s330), and when it is determined that an unprocessed cell C remains (s330: YES). ), The process returns to s310, and thereafter s310 to s330 are repeated until there is no unprocessed cell C.

  Thereafter, when it is determined that there is no unprocessed cell C remaining (s330: NO), a certain number of cells C that are colored in accordance with the coloring rule among the cells C whose color is determined in s320. It is checked whether or not it exists (s340). Here, in the data table in which the cells C and colors are registered in association with each other at this time, a certain number (for example, 30% of the total) of cells C associated with colors outside the determination region It is determined that there are more than a predetermined percentage of colored cells C that do not comply with the coloring rules.

  Here, it is configured so as to determine whether or not there are a certain number or more of colored cells C that do not comply with the coloring rule based on the relative number (that is, the ratio) of the cells C in the two-dimensional code. However, the determination may be made based on the absolute number in the cell C of the two-dimensional code, or the cell C that does not comply with the coloring rule may have a specific pattern (for example, two or more consecutive cells C). Or the like) may be determined based on the number of patterns.

  When it is determined in s340 that there are more than a certain number of colored cells C that do not comply with the coloring rule (s340: YES), the determination region can be changed and color determination can be performed again It is checked whether or not (s350). Here, when the determination area referred to in the determination in s320 performed immediately before is changed to a predetermined upper limit, it is determined that the determination area is changed and the color determination is not possible again. Is done.

  This “predetermined upper limit” is defined as a range in which each determination area can be changed so as not to interfere with other determination areas. As will be described later, the determination area is simply expanded. If there is, it can be determined that “the change has been made to a predetermined upper limit” that each determination area has been expanded to such an extent that it interferes with other areas. Further, in the case of shifting the determination area, it is in a state where the color defined in the determination area has shifted to a position (boundary) immediately before it becomes another color component. It has only to be changed to the upper limit.

  If it is determined in s350 that the color determination can be performed again (s350: YES), the determination area referred to in the determination in s320 is changed (s360). Here, each determination area is changed by expanding each determination area by a certain distance in the color space or by shifting each determination area by a certain distance in the color space.

  In the latter case, the determination area may be configured to shift to a position on the color space where the deviation of the color component in the image area of the two-dimensional code can be canceled. Specifically, referring to the color distribution in the image area of the two-dimensional code, when a specific color component is increased overall in the distribution, each determination area is shifted in the direction of decreasing this color component. Let

Thus, after finishing s360, the process returns to s310, and the color of each cell is determined again.
Thereafter, until it is determined that there are not a certain number or more of cells C that are not colored according to the coloring rule in each cell whose color has been determined again (s340: NO), the determination region is changed and again. The color determination is repeated (s310 to s360).

  However, when it is determined in s350 that the color determination is impossible again (s350: NO), after the error process is performed (s370), the main color determination process is ended and the information identification process ( Returning to s240) of FIG. In this error process, it is assumed that the color of any cell C could not be determined, and all the correspondence relationships registered in the data table are deleted, and thus an empty data table is returned to the information identification process as a return value. In this case, in the information identification processing, the processing after s260 is performed without performing any processing in s240 and s250.

  During the processing from s310 to s360, if it is determined in s340 that there are no more than a certain number of colored cells C that do not comply with the coloring rule (s340: YES), coloring is performed. It is checked whether there is a colored cell C that does not comply with the rules (s380).

  If it is determined in s380 that the corresponding cell C exists (s380: YES), the corresponding cell C is specified from the two-dimensional code (s390). Here, among the cells C registered in the data table at this time, a cell C that is colored that does not comply with the coloring rule is specified, and this cell C is specified from the two-dimensional code.

  Next, for each cell C specified in the above s390, the color to be originally attached is estimated according to the coloring rule (s400). Here, for the cell C identified in the above s390, a color that is considered in relation to the color assigned to the cell C adjacent to the cell C is selected as a candidate for the color that should be originally assigned according to the coloring rule. Then, the most probable color among the candidates is estimated as the color to be assigned to the cell C.

  Next, for each cell C identified in s390, it is determined that the color of the cell C is the color estimated in s400 (s410). Here, the color in the data table is updated only for each cell C identified in s390.

Then, after s410 is completed or when it is determined in s380 that the corresponding cell C does not exist (s380: NO), the color determination process ends, and the process proceeds to the information identification process (s240 in FIG. 3). Return. In these cases, the data table in which the correspondence relationship between the cell C and the color is registered at this point returns to the information identification process as a return value. In this case, in the information identification process, information is identified by s240 after recognizing the color assigned to each cell C based on the data table.
(3) Action and Effect In the image processing system 1 in the above embodiment, based on the component image generated from the image (original image) acquired from the outside (s140 in FIG. 3), the candidate area in this component image 2 (s150 in the figure), and whether or not each of the corresponding areas in the original image corresponding to the extracted candidate area includes a characteristic pattern (s220 in the figure) The area of the dimension code can be detected (s230 in the figure).

  In this way, by detecting a corresponding area including a characteristic pattern among the corresponding areas corresponding to the candidate areas of the component image in the original image area as a two-dimensional code area in the original image. Each of the one or more two-dimensional codes included in the original image can be specified.

  In the above embodiment, for each of a plurality of types of component images generated based on the original image, extraction of candidate regions, determination of whether or not a characteristic pattern is included in the extracted candidate regions, and The detection of the two-dimensional code area is repeatedly performed (s140 to s290 in FIG. 3).

  As a result, even if the region of the two-dimensional code cannot be detected based on the specific component image, it can be expected to be detected based on another component image, and as a result, the detection accuracy of the two-dimensional code in the original image Can be increased.

  Also, in this iterative process, the candidate area newly extracted for a specific component image may match the extracted candidate area extracted for the previously referenced component image in the area of the original image. is assumed. However, in the above configuration, for such an extracted region, it is not determined whether or not a characteristic pattern is included in the extracted candidate region, and detection of the region of the two-dimensional code is not performed (s200 in FIG. YES ").

  Therefore, when identifying the information encoded by the two-dimensional code for each two-dimensional code area detected based on each of the plurality of component images, the two-dimensional code that should originally be the same is repeatedly identified. It is possible to eliminate an unnecessary processing burden such as.

Further, in the above embodiment, after extracting a candidate area consisting of a two-dimensional code from the original image (s150 in FIG. 3), the two-dimensional code is detected from the candidate area to identify information (same as above). Figure s17 0 -s240). Therefore, compared with the case where a two-dimensional code is to be detected without extracting a candidate area, a two-dimensional code can be detected from a limited range of extracted candidate areas. The processing load and processing time required until identification can be suppressed.

  Further, in the above-described embodiment, information encoded by the two-dimensional code can be identified for each two-dimensional code region detected based on each of the plurality of component images (s240 in FIG. 3).

  Further, in the above-described embodiment, both candidate areas have a certain degree of overlap according to the degree of overlap between a candidate area extracted from a specific component image and an extracted candidate area in another component image. Can be determined to match.

In the embodiment described above, the candidate region extracted from a particular component image, and, among the already extracted candidate region in another component image, with that one of the candidate region is subsumed in the other candidate region, It can be determined that both candidate areas match.

  Further, in the above embodiment, both candidate regions are located when one or more vertices in the candidate region extracted from the specific component image are located within a certain range from any vertex in the extracted candidate region. It can also be determined that they match.

  Further, in the above embodiment, candidate areas can be extracted according to the rules defined in the template. Therefore, by changing the rules together with this template, it can be detected as an identification target area formed according to an arbitrary rule. can do.

In the above-described embodiment, a two-dimensional code area is detected as an identification target area, and information encoded by the two-dimensional code can be identified.
Moreover, in the said embodiment, the information shown by a two-dimensional code can be identified based on the color attached | subjected to each cell C of a two-dimensional code (s232-s240 of FIG. 3 ).

  In the above embodiment, even if an area corresponding to a part of the cells C is missing among the cells C in the two-dimensional code due to the generation (specifically, shooting) state of the image, the cells The color that should be originally attached to the image can be estimated according to the coloring rule (s400 in FIG. 5), and information can be identified based on the color, so that it is possible to prevent the information itself from being identified. .

Further, in the above embodiment, the color assigned to each cell C in the two-dimensional code can be determined based on which determination region in the color space is included (s320 in FIG. 5 ).

  Further, in this configuration, even when the color of each cell C is determined, even if the color assigned to many cells C is not included in any determination region, and the information itself cannot be identified. It is configured to prevent such a situation.

Specifically, when there are a predetermined number or more of cells that are determined not to be included in any of the determination regions corresponding to each of the plurality of colors (s340 “YES” in FIG. 5 ), the determination region is only the fixed region. After expanding or shifting, the color assigned to each cell is determined again (s360 to s320 in the figure).

With this configuration, even if the image generation conditions are poor and the color cannot be determined properly, the determination area can be expanded to increase the determination sensitivity and determine the color again. By shifting to an appropriate position according to the distribution of the color and determining the color again, the color determination probability can be increased.
(4) Modifications The embodiment of the present invention has been described above. However, the present invention is not limited to the above embodiment, and can take various forms as long as it belongs to the technical scope of the present invention. Needless to say.

  For example, in the above-described embodiment, the case where the image processing system 1 is configured by a single device has been exemplified. However, the image processing system 1 may be configured as a system in which a plurality of devices operate in cooperation. .

  Moreover, in the said embodiment, although the case where the identification object area | region said by this invention was a two-dimensional code was illustrated, as an identification object area | region, not only a two-dimensional code as mentioned above but a specific image (for example, It may be a region showing a logo or the like.

  Further, in the above embodiment, a candidate for a color to be attached according to the coloring rule is selected, and the most probable single color is estimated as a color that should be originally attached according to the coloring rule. At the stage where color candidates are selected, any of these candidates may be selected by the user.

In addition, information may be identified for each color selected as a candidate, and the user may be allowed to select one of the identified information. In this case, no processing is performed in s400 of FIG. 5, and each candidate color is determined as a color assigned to the corresponding cell C in s410. 3 , if there are a plurality of colors determined as the colors assigned to the cell C, information is identified for each color or based on a combination of the colors, and the information thus identified is identified. It is possible to consider a configuration in which the user selects one of the items.
(5) Correspondence with the Present Invention In the embodiment described above, s130 in FIG. 3 is an image acquisition unit in the present invention, s140 in FIG. 3 is a component conversion unit in the present invention, and s150 in FIG. S220 is a pattern determining unit in the present invention, s230 is a region detecting unit in the present invention, and s140 to s290 (especially s140, s270, s290) are repeated in the present invention. FIG. S200 is the coincidence determining means in the present invention, and s240 in the figure is the information identifying means in the present invention.

Further, s400 in FIG. 5 is the color estimation means in the present invention, and s3 20 and s410 in the figure are the color determination means in the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Image processing system, 2 ... Camera, 11 ... Control part, 13 ... Memory | storage part, 17 ... Input-output interface, 19 ... Communication part, 100 ... Network, C ... Cell.

Claims (13)

Image acquisition means for acquiring an image obtained by photographing an area including one or more identification target areas;
Component conversion means for converting the image acquired by the image acquisition means into a component image indicating a distribution of a specific component that is a component equal to or higher than a predetermined threshold among the components indicating brightness included in the image;
In order to determine whether or not to be a candidate area of the identification target area based on a parameter of the outer shape of the identification target area from the component image converted by the component conversion means A region extracting means for extracting a region formed in accordance with the rules as a candidate region consisting of the identification target region;
Among the images acquired by the image acquisition means, for each corresponding area corresponding to the candidate area extracted by the area extraction means, the pattern formed in the corresponding area is an element that forms the corresponding area. Pattern determination means for determining whether or not a characteristic pattern that is a pattern for determining whether or not the identification target region is included;
A region detecting unit for detecting, as the identification target region, a corresponding region determined to include a characteristic pattern by the pattern determining unit;
The conversion of the image into the component image by the component conversion unit is sequentially performed for each of two or more types of components, and for each component image converted for each component, extraction of candidate regions by the region extraction unit and the pattern determination Repetitive execution means for repeatedly performing characteristic pattern determination by the means and detection of the identification target area by the area detection means;
Whether each of the candidate areas extracted by the area extracting unit for a specific component image matches the candidate area extracted by the area extracting unit for another component image (hereinafter referred to as “extracted candidate area”) A determination unit for determining whether or not based on both candidate areas,
The repetitive execution means includes a candidate area extracted by the area extracting means for determining whether or not the pattern determining means has a characteristic pattern and detecting the identification target area by the area detecting means. Of these, only the candidate area determined not to match the extracted candidate area by the match determination means is executed. The image processing system according to claim 1, further comprising: an information identification unit that identifies information indicated by the identification target region for each identification target region detected by the region detection unit. The identification target area is a two-dimensional code in which a plurality of cells are arranged, and each cell is colored to code one or more pieces of information according to the color or a combination of colors,
The image processing system according to claim 2, wherein the information identification unit identifies information based on a color or a combination of colors assigned to each cell in the two-dimensional code that is the identification target region. In the identification target area, the color assigned to each of the plurality of cells corresponds to information to be encoded in the cell and the color attached to the cell adjacent to the cell among the plurality of types of colors. It is determined according to coloring rules,
further,
For each identification target region detected by the region detection means, out of each cell in the two-dimensional code that is the identification target region, the color to be assigned to the cell according to the coloring rule, and the two-dimensional code A color estimation means for estimating a color to be originally assigned to the cell according to the coloring rule for a cell having a color other than the color to be used in the cell,
The information identification unit is attached to each cell on the assumption that the color estimated by the color estimation unit is attached to each cell in the two-dimensional code that is the identification target region. The image processing system according to claim 3, wherein the information is identified based on the selected color or combination of colors. For each identification target area detected by the area detection means, the color assigned to each cell in the two-dimensional code that is the identification target area is spread around each of a plurality of colors used in the two-dimensional code. Color judging means for judging according to which of the judgment areas on the color space is included,
The information identification unit assumes that each cell in the two-dimensional code that is the identification target region is given the color determined by the color determination unit for the cell, and the color or combination of colors assigned to each cell. The image processing system according to claim 3 or 4, wherein the information is identified based on the information. The color determination means expands the determination area by a certain area with respect to an identification target area where a certain number or more of cells determined not to be included in any of the determination areas corresponding to a plurality of colors are present. The image processing system according to claim 5, wherein the color assigned to each cell is determined again. The color determination means responds to a color distribution in the identification target region for a recognition target region having a predetermined number or more of cells determined not to be included in any of the determination regions corresponding to each of a plurality of colors. The image processing system according to claim 5, wherein the color assigned to each cell is determined again after the determination area is shifted in a color space. The match determination unit determines whether each candidate region extracted by the region extraction unit matches the extracted candidate region in another component image with respect to a specific component image. The image processing system according to claim 1, wherein the determination is made according to a degree of duplication. The coincidence determining unit determines whether each of the candidate regions extracted by the region extracting unit matches the extracted candidate region in another component image with respect to a specific component image. The image processing system according to any one of claims 1 to 8 , wherein one of the regions is determined according to whether or not one of the candidate regions is included in the other candidate region. In the case where the identification target area is a polygon,
The coincidence determining unit has one or more vertices in the polygon formed as the candidate region formed as the extracted candidate region for each candidate region extracted by the region extracting unit in the specific component image. Whether each of the candidate areas extracted by the area extracting means matches the extracted candidate area in another component image, depending on whether or not it is located within a certain range from any vertex in the polygon The image processing system according to claim 1, wherein the image processing system is determined. The area extracting means is configured to extract an area formed according to a rule defined in the template from a component image converted by the component converting means based on a template that defines a rule relating to the formation of the identification target area. It extracts as a candidate area | region. The image processing system as described in any one of Claim 1 to 10 characterized by the above-mentioned. The image processing system according to claim 11, wherein the template defines a rule for forming a two-dimensional code region obtained by encoding predetermined information as the candidate region.   A program for causing a computer system to execute various processing procedures for causing a function to be specified as all means according to any one of claims 1 to 12.