JP5463269B2 - Feature figure addition method, feature figure detection method, feature figure addition device, feature figure detection device, and program - Google Patents

Description

  The present invention relates to a feature figure addition method, a feature figure detection method, and a feature figure addition device for adding a figure to an image and detecting the given figure from a photographed image obtained by photographing the image to which the figure is added. , A characteristic figure detection apparatus, and a program.

As a technique for detecting a graphic region such as a predetermined rectangle or circle from an image on a terminal with limited processing capability, the techniques described in Patent Documents 1 and 2 and References 1 and 2 are known. It has been.
In the inventions of Patent Documents 1 and 2, in order to detect a rectangular area at high speed on a terminal with limited processing capability, an edge in an image is searched using a filter for a minute area. A method of detecting an edge candidate by tracing an edge in a direction orthogonal to the edge search direction and evaluating whether the combination of edge candidates represents a rectangular area is shown.

  These techniques are also used to detect a rectangular region in which a digital watermark is embedded in a system that detects a digital watermark embedded in an image such as a photograph or video (for example, References 3 and 4). reference).

JP 2005-277732 A JP 2007-52670 A

Reference 1: Atsushi Katayama, Takao Nakamura, Masashi Yamamuro, Noboru Sonehara, i-αppli fast corner detection algorithm for digital watermark reading, Science theory D-II, Vol. J88-D-II, No. 6, pp. 1035-1046, 2005 Reference 2: Satoshi Katayama, Ryo Kitahara, Harumi Kawamura, Hideki Koike, Application of high-speed quadrilateral detection algorithm for i-applis, Side Trace Algorithm (STA), circular shape detection application, IEICE National Convention, Communication Proceedings 1, B-15 -13, p.591 (2008). Reference 3: Takao Nakamura, Atsushi Katayama, Masashi Yamamuro, Noboru Sone, High-speed watermark detection method from analog images using a mobile phone with camera, Science theory D-II, Vol.J87-D-II, No. 12, pp. 2145-2155, 2004. Reference 4: Takao Nakamura, Kan Yamamoto, Ryo Kitahara, Kaoru Katayama, Takayuki Anno, Hideki Koike, Noboru Sone, Mobile Digital Watermark for Real-time Detectable Video Based on SFPSS Method, Transactions of Information Processing Society of Japan, Vol.49, No .6, pp.1885-1895 (2008).

However, in the prior art, there is a problem that the target rectangular area cannot be detected correctly as follows.
Problems of the prior art will be described with reference to FIG. FIG. 20 is a diagram for explaining the related art.
In the prior art, when a rectangular shape (2002) different from the target rectangular area (2001) exists in the image, or a straight line (2004) having a strong edge component in the vicinity of the target rectangle exists. In this case, there is a problem that this is erroneously detected as a side candidate or a rectangle candidate. As a result, an erroneous rectangular area (2003, 2005) is detected, and there is a problem that the accuracy of detection of the target graphic area is lowered.

  The present invention has been made in view of the above points, and can reduce false detection caused by rectangles, straight lines, graphics, and the like other than the target graphic region, and can improve the accuracy of detection of the target graphic region. It is an object to provide a figure addition method, a feature figure detection method, a feature figure addition apparatus, a feature figure detection apparatus, and a program.

  In view of the above-described problems, the feature graphic addition method according to the present invention inputs target image data to which a feature graphic is added, and detects the feature graphic from an image to which the feature graphic is added. In the feature figure addition method for adding the feature figure to the image data, a feature amount that is predetermined according to the figure to be added to the target image data and has different pixel values according to a position area in the figure is determined. A feature amount generation step for outputting feature amount information indicating the feature amount; and a feature figure pattern representing a feature figure by a pixel value including the predetermined feature amount based on the input feature amount information; A feature graphic pattern generating step for outputting information indicating the feature graphic pattern, and adding the feature graphic pattern to the target image data Create a form additional image, characterized by comprising a rectangular additional step of outputting the image data of the feature graphic imparting image.

  In the above feature graphic adding method, the feature value generating step is characterized in that the feature value is determined by a series of a plurality of numerical values indicating characteristics predetermined according to the position in the feature graphic. .

  In the above-described feature figure addition method, the feature amount generation step and the feature figure pattern generation step are performed a plurality of times to generate a plurality of feature figure patterns, and in the rectangle addition step, the figures are close to each other The feature graphic additional image is created by adding the plurality of characteristic graphic patterns to the target image data within a proximity range that is predetermined as a recognized range.

  In view of the above-described problems, a feature graphic detection method according to the present invention is characterized by inputting feature graphic detection target image data of an image to which a feature graphic is added and detecting the feature graphic from the feature graphic detection target image data. In the graphic detection method, an edge detection step for detecting an edge candidate area corresponding to an edge of a graphic from the feature graphic detection target image data, and an edge neighboring area adjacent to the edge candidate area and a pixel included in the edge candidate area A pixel value acquisition step of acquiring a pixel value of the feature, a feature amount calculation step of obtaining a feature amount of the figure indicated by the edge candidate region based on the acquired pixel value, and a feature amount of the feature figure determined in advance And an evaluation value indicating the similarity between the feature quantity of the graphic indicated by the edge candidate area and indicated by the edge candidate area And shape and the characteristic figure is characterized in that it comprises a feature amount evaluation step of determining based on whether or not similar to the evaluation value.

  In the pixel value acquisition step of the feature graphic detection method described above, the pixel values of the inner edge and the outer edge of the frame-shaped graphic indicated by the edge candidate area, or the graphic indicated by the edge candidate area The pixel value of the position decided beforehand according to the position in is acquired.

  In the feature amount calculation step of the above-described feature figure detection method, the pixel values of the inner edge and the outer edge of the frame-shaped figure indicated by the edge candidate area, or the figure indicated by the edge candidate area A detection sequence including a plurality of numerical values indicating features of the graphic indicated by the edge candidate region is obtained based on a pixel value at a predetermined position according to a position within the position, and is determined in advance in the feature amount evaluation step. As the feature amount of the feature graphic, the similarity between the numerical series including a plurality of numerical values indicating the characteristics predetermined according to the position in the characteristic graphic and the detection series is represented by a correlation value, and An evaluation value is calculated.

  In the feature amount calculation step of the above-described feature figure detection method, for each of the plurality of figures indicated by the edge candidate regions existing in the proximity range determined in advance as a range in which the figures are recognized as being close to each other, The feature amount is obtained based on the acquired pixel value, and in the feature amount evaluation step, the feature amount of the plurality of feature figures according to a predetermined combination and the plurality of figures indicated by the edge candidate region An evaluation value indicating similarity to the feature amount is calculated.

  In view of the above-described problems, a feature graphic adding apparatus according to the present invention inputs target image data to which a feature graphic is added, and detects the feature graphic from an image to which the feature graphic is added. In the feature figure addition device for adding the feature figure to the image data, a feature amount that is predetermined according to the figure to be added to the target image data and has different pixel values according to the position area in the figure is determined. A feature amount generating means for outputting feature amount information indicating the feature amount; and a feature figure pattern representing the feature figure having the predetermined feature amount based on the input feature amount information; A feature graphic pattern generating means for outputting information indicating a pattern, and a feature graphic additional image is created by adding the characteristic graphic pattern to the target image data. Characterized in that it comprises a rectangular adding means for outputting the image data of the feature graphic imparting images.

  In view of the above-described problems, a feature graphic detection apparatus according to the present invention receives feature graphic detection target image data of an image to which a feature graphic is added, and detects the feature graphic from the feature graphic detection target image data. In the figure detection apparatus, an edge detection unit that detects an edge candidate area corresponding to an edge of a figure from the feature figure detection target image data, and an edge neighboring area adjacent to the edge candidate area and a pixel included in the edge candidate area A pixel value acquisition unit that acquires a pixel value of the image, a feature amount calculation unit that obtains a feature amount of the graphic indicated by the edge candidate region based on the acquired pixel value, and a feature amount of the feature graphic that is determined in advance And an evaluation value indicating a similarity between the feature amount of the graphic indicated by the edge candidate region and the graphic indicated by the edge candidate region and the feature diagram Doo is characterized by comprising a feature amount evaluation means for determining based on whether or not similar to the evaluation value.

  In view of the above-described problems, a program according to the present invention is a program for causing a computer to execute the steps described in any one of the above-described feature graphic addition method or the above-described feature graphic detection method. It is characterized by being.

  According to the present invention, it is possible to reduce false detection caused by rectangles, straight lines, graphics, and the like other than the target graphic area, and to increase the accuracy of detection of the target graphic area.

It is a figure for demonstrating an example of a structure of the image detection system which concerns on this invention. It is a block diagram explaining an example of composition of a rectangle adding device concerning this embodiment. It is a block diagram explaining an example of a structure of the feature-value production | generation means of the rectangle addition apparatus which concerns on this embodiment. It is a block diagram explaining an example of the composition of the rectangle detecting device concerning this embodiment. It is a block diagram explaining an example of a structure of the rectangle feature-value calculation means of the rectangle detection apparatus which concerns on this embodiment. It is a block diagram explaining an example of a structure of the feature-value detection means of the rectangular feature-value calculation means which concerns on this embodiment. It is a block diagram explaining an example of a structure of the rectangle feature-value evaluation means of the rectangle detection apparatus which concerns on this embodiment. It is a flowchart which shows an example of the rectangle addition process in the image detection method which concerns on this embodiment. It is a figure which shows an example of the parameter contained in the numerical series which concerns on this embodiment. It is a figure which shows an example which matched the numerical series with each of the edge area | region in the characteristic rectangle figure which concerns on this embodiment. It is a figure which shows an example which provided the characteristic rectangular figure which concerns on this embodiment to the rectangle addition object image. It is a flowchart for demonstrating an example of the rectangle detection process by the rectangle detection apparatus which concerns on this embodiment. It is a figure for demonstrating an example of the edge tracking which concerns on this embodiment. It is a figure for demonstrating an example of the edge detection filter which concerns on this embodiment. It is a figure for demonstrating an example of the image acquisition by the pixel value acquisition means which concerns on this embodiment. It is a figure for demonstrating an example of the projection transformation parameter calculation and division | segmentation position calculation which concern on this embodiment. It is a figure for demonstrating accumulation | storage for every division position of the pixel value which concerns on this embodiment. It is a figure for demonstrating the edge detection series which concerns on this embodiment. It is a figure which shows the example of the rectangle addition image which drew the several rectangle multiply. It is a figure for demonstrating a prior art.

[First embodiment]
The first embodiment according to the present invention will be described below. In the image detection system according to the first embodiment, a rectangular graphic image is detected as a graphic image to be detected in the image. Note that the figure to be detected by the image detection system according to the present invention is not limited to a rectangle but may be a circle or the like. In the present embodiment, an example of a rectangle will be described below.
In the image detection system according to the present embodiment, a rectangular figure (hereinafter referred to as a characteristic rectangular figure C10) having a predetermined characteristic (hereinafter referred to as a drawing characteristic) is drawn on a target image. An image including the characteristic rectangular figure C10 is captured.
When detecting an area corresponding to a rectangular figure, the image detection system according to the present embodiment detects a rectangular candidate having a feature similar to the rectangular figure based on image data obtained by shooting, and the rectangular candidate is detected. Then, it is determined whether or not the feature rectangular figure has the drawing feature. Note that the image detection system according to the present invention is not limited to one that detects a feature rectangular figure based on image data obtained by photographing. For example, the configuration may be such that image data (still image) or video data (moving image) to which a feature rectangle is added is distributed to the rectangle detection device 300 via a network. Alternatively, the external device may create an image using an image with a feature rectangle graphic added as a material, and distribute the image to the rectangle detection device 300 via a network.
The image detection system determines the likelihood that the rectangle candidate is the feature rectangle graphic by evaluating the similarity between the feature amount of the rectangle candidate and the drawing feature of the feature rectangle graphic.

In order to evaluate the feature of the rectangle candidate, the image detection system calculates the feature amount of the rectangle candidate based on the pixel value of the rectangle candidate detected from the captured image, and is added to the calculated feature amount and the rectangle figure in advance. This is performed by evaluating the similarity with the feature amount of a predetermined drawing feature.
The drawing characteristics given to the rectangles have different pixel values depending on the position area in the rectangular figure. For example, a predetermined position can be obtained by changing a division position on each side obtained by dividing the four sides of the rectangular figure at a plurality of positions, an inner side and an outer side of the edge on each side, and a pixel value according to a pixel channel according to a predetermined numerical series It is configured to have the characteristics of In this specification, the position area represents the concept of at least one of an arbitrary position (point) and area (range) in the figure.
As described above, the position area in the direction parallel to the boundary (in this case, the side) can be changed by changing the division position on each side. Further, by dividing the edge into the inside and the outside, the position area in the direction perpendicular to the boundary (side) can be changed.
The similarity evaluation of the feature amount is performed by dividing the detected rectangular candidates on each side, the inside and outside of the edge of the side, a series of pixel values acquired according to the pixel channel, and a predetermined numerical series. Perform by correlation calculation.
Hereinafter, it will be specifically described with reference to the drawings.

<Configuration of Image Detection System According to First Embodiment>
FIG. 1 is a diagram for explaining an example of the configuration of an image detection system according to the present invention.
As shown in FIG. 1, the image detection system 1 according to the present embodiment includes a rectangular addition device 100 that is a feature graphic addition device, an image display device 200, and a rectangular detection that is a feature graphic detection device built in a portable terminal 400. The apparatus 300 is configured.

  The rectangle adding device 100 inputs image data of a rectangle addition target image C11 that is a target to which the feature rectangle figure C10 is added. The rectangle adding device 100 adds a feature rectangle figure C10 to the rectangle addition target image C11 using information stored in advance in the storage unit, and obtains image data of the rectangle addition image C12 to which the feature rectangle figure C10 is added. Generate. The rectangle adding device 100 outputs the generated image data of the rectangle added image C12 to the image display device 200.

The image display device 200 receives the image data of the rectangular additional image C12, and displays the rectangular additional image C12 on the screen based on this image data. The image display device 200 is a display such as a liquid crystal display device.
The rectangle detection device 300 is built in a portable terminal 400 such as a mobile phone or a camera, and detects a rectangular figure having a drawing feature from the captured image based on image data captured by the camera.

Next, an example of the configuration of the rectangle adding device 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of the configuration of the rectangle adding device 100 according to the present embodiment.
The rectangle adding apparatus 100 according to the present embodiment includes a feature amount generating unit 101, a rectangular pattern generating unit 102, a rectangle adding unit 103, and a storage unit 104. The rectangle adding device 100 inputs image data of the rectangle addition target image C11 from an external device via a wired, wireless, or network, and outputs a rectangle added image C12.

  The feature quantity generation unit 101 refers to the storage unit 104 and generates drawing feature quantity information D11 indicating the drawing characteristics of the rectangle. The feature quantity generation means 101 is determined in advance according to a figure to be added to the image data of the rectangle addition target image C11, determines a feature quantity having a different pixel value according to a position area in the figure, and the feature quantity The drawing feature amount information D11 indicating is output.

  The rectangular pattern generation unit 102 receives the drawing feature amount information D11 from the feature amount generation unit 101, and determines a pixel value corresponding to the position on the side of the rectangle and the pixel channel according to the drawing feature amount information. The rectangular pattern generation unit 102 generates a feature graphic pattern (hereinafter referred to as a rectangular pattern) D12 indicating an image of a characteristic rectangular graphic having drawing characteristics based on the determined pixel value. That is, the rectangular pattern generation unit 102 generates a rectangular pattern D12 representing a feature graphic based on pixel values including a predetermined drawing feature amount based on the input drawing feature amount information D11, and indicates this rectangular pattern D12. Output information.

  The rectangle adding means 103 receives the image data of the rectangle addition target image C11 and information indicating the rectangular pattern D12, draws the feature rectangle figure C10 on the rectangle addition target image C11 according to the information indicating the rectangle pattern D12, and adds the rectangle. Image data of the image C12 is generated. The rectangle adding means 103 outputs the image data of the rectangle added image C12 to the image display device 200.

Next, an example of the configuration of the feature quantity generation unit 101 of the rectangle adding apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating an example of the configuration of the feature amount generation unit 101 of the rectangle adding device 100 according to the present embodiment.
The feature quantity generation unit 101 includes an embedded sequence generation unit 111.
The embedded sequence generation unit 111 generates a predetermined numerical sequence characterizing the rectangle, and outputs the generated numerical sequence as the drawing feature amount information D11. This numerical series is a series of a plurality of numerical values indicating features predetermined according to the position area in the feature graphic.

Next, an example of the configuration of the rectangular detection device 300 according to the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram illustrating an example of the configuration of the rectangle detection apparatus 300 according to the present embodiment.
The rectangle detection apparatus 300 according to the present embodiment includes an edge candidate detection unit 301 that is an edge detection unit, a rectangle candidate calculation unit 302, a rectangle feature amount calculation unit 303, and a rectangle feature amount evaluation unit 304. The rectangle detection apparatus 300 inputs, for example, image data captured by a camera mounted on the mobile terminal 400 as image data of the rectangle detection target image C14. The camera mounted on the portable terminal 400 captures the rectangular additional image C12 displayed on the image display device 200, and outputs the image data of the rectangular image detection image C14 to the rectangular detection device 300.
The rectangle detection apparatus 300 performs a rectangle detection process for detecting a rectangular image similar to the feature rectangle figure C10 having a drawing feature based on the input rectangle image detection image C14. The rectangle detection device 300 outputs a rectangle detection result C15 indicating the result of the rectangle detection process.

  The edge candidate detection unit 301 inputs image data of the rectangular detection target image C14 from, for example, a camera built in the portable terminal 400, and analyzes the image data of the rectangular detection target image C14. The side candidate detection unit 301 detects a straight line portion from the rectangular detection target image C14 and outputs information D31 indicating a side candidate similar to the side of the rectangular figure to the rectangle candidate calculation unit 302. The information D31 indicating the edge candidate is information indicating the pixel value of the edge candidate area corresponding to the edge of the figure.

  The rectangle candidate calculation unit 302 inputs information D31 indicating a side candidate, and determines a combination that may form a rectangle as a rectangle candidate from the side candidates. The rectangle candidate calculation unit 302 outputs information D32 indicating the rectangle candidate to the rectangle feature amount calculation unit 303.

  The rectangle feature amount calculation unit 303 inputs the information D32 indicating the rectangle candidate from the rectangle candidate calculation unit 302 and the image data indicating the rectangle detection target image C14 from the camera built in the portable terminal 400, and the rectangle candidate rectangle. Calculate features. The rectangular feature quantity calculation unit 303 outputs a rectangular feature quantity D33 indicating a rectangular feature corresponding to each rectangle candidate to the rectangular feature quantity evaluation unit 304.

The rectangular feature quantity evaluation unit 304 receives the rectangular feature quantity D33, evaluates the similarity between the rectangular feature quantity D33 and the drawing feature quantity, determines the most appropriate rectangle, and outputs the rectangle detection result C15.
The rectangular feature quantity evaluation unit 304 calculates, for example, an evaluation value indicating the similarity between the rectangular feature quantity D33 and the drawing feature quantity, and the evaluation value falls within a threshold range that is predetermined as a range in which both are similar. It is determined whether or not.
If the calculated evaluation value falls within the threshold range, the rectangular feature amount evaluation unit 304 determines that the rectangle candidate D32 having the rectangular feature amount D33 is a detection target. ) For example to an external output device. In the present embodiment, the rectangular feature amount evaluation unit 304 uses, as the rectangle detection result C15 (result: with rectangle detection), the rectangle candidate D22 determined to be a detection target and the rectangle detection target image corresponding to this rectangle candidate D22. C14 image data is output.

  On the other hand, when the calculated evaluation value does not fall within the threshold range, the rectangular feature value evaluation unit 304 determines that the rectangle candidate D32 having the rectangular feature value D33 is not a detection target, a rectangle detection result C15 (result: rectangle detection). None) is output. In this embodiment, instead of outputting the rectangle detection result C15 (result: no rectangle detection), the rectangle feature amount evaluation unit 304 and the rectangle candidate D32 determined not to be a detection target and the rectangle corresponding to the rectangle candidate D32 The image data of the detection target image C14 is not output.

Next, an example of the configuration of the rectangular feature amount calculation unit 303 of the rectangular detection device 300 according to the present embodiment will be described with reference to FIG. FIG. 5 is a block diagram illustrating an example of the configuration of the rectangular feature amount calculation unit 303 of the rectangular detection device 300 according to the present embodiment.
The rectangular feature amount calculation unit 303 according to the present embodiment includes an edge tracking unit 331, a pixel value acquisition unit 332, and a feature amount calculation unit 333. The rectangular feature amount calculation unit 303 receives information indicating the rectangle candidate D32 from the rectangular feature amount candidate calculation unit 302 and image data of the rectangular detection target image C14 from the camera of the portable terminal 400. Based on the information indicating the rectangle candidate D32 and the image data of the rectangle detection target image C14, the rectangle feature amount calculation unit 103 generates a rectangle feature amount D33 indicating the feature of the rectangle candidate D32. The result is output to the rectangular feature amount evaluation unit 304.

  The edge tracking unit 331 inputs information D32 indicating a rectangle candidate, scans each side candidate constituting the rectangle in a direction parallel to the side, detects the edge position of the side, and information D34 indicating the detected edge coordinates. Is output.

  Based on the information D34 indicating edge coordinates, the pixel value acquisition unit 332 acquires information D35 indicating the pixel value of each point inside and outside the edge at the edge position detected from the rectangular detection target image C14. The pixel value acquisition unit 332 outputs information D35 indicating the pixel value of each point inside and outside the edge to the feature amount calculation unit 333. That is, the pixel value acquisition unit 332 acquires information D35 indicating pixel values of pixels included in the edge vicinity region and the edge candidate region adjacent to the edge candidate region.

  The feature quantity calculation means 333 accumulates information D35 indicating the pixel value of each point inside and outside the edge in a built-in memory. The feature quantity calculation means 333 calculates a feature quantity for each side, and calculates a rectangular feature quantity D33 obtained by integrating the feature quantities for each side. In other words, the feature amount calculation means 333 is based on information D32 indicating a rectangle candidate, information D34 indicating edge coordinates, and information D35 indicating a pixel value, and the feature of the figure indicated by the edge candidate region (region corresponding to the side candidate). Find the amount.

  Next, an example of the configuration of the feature amount detection unit 333 of the rectangular feature amount calculation unit 303 according to the present embodiment will be described with reference to FIG. FIG. 6 is a block diagram illustrating an example of the configuration of the feature amount detection unit 333 of the rectangular feature amount calculation unit 303 according to the present embodiment.

  The feature amount calculation unit 333 according to the present embodiment includes a conversion parameter calculation unit 3331, a division position calculation unit 3332, an edge detection series calculation unit 3333, and a detection series integration unit 3334. The feature amount detection means 333 receives information D32 indicating a rectangle candidate, information D34 indicating edge coordinates, and information D35 indicating a pixel value, and outputs a detection series as a rectangle feature amount.

The conversion parameter calculation means 3331 calculates a conversion parameter for converting the coordinates on the sides of the rectangle candidates into coordinate values representing the rectangular division positions.
The division position calculation means 3332 converts the edge coordinates using the obtained conversion parameter, and calculates the division position on the side.
The edge detection series calculation means 3333 accumulates the pixel values of each point on the detected edge for each division position, and calculates an edge detection series for the edge.
The detection sequence integration unit 3334 integrates the side detection sequences for each side and calculates a detection sequence for the entire rectangle.

Next, an example of the configuration of the rectangular feature amount evaluation unit 304 of the rectangular detection device 300 according to the present embodiment will be described with reference to FIG. FIG. 7 is a block diagram illustrating an example of the configuration of the rectangular feature amount evaluation unit 304 of the rectangular detection device 300 according to the present embodiment.
The rectangular feature quantity evaluation unit 304 according to the present embodiment includes an embedded feature quantity calculation unit 341, a feature quantity similarity calculation unit 342, and a feature quantity similarity comparison unit 343.
This rectangular feature quantity evaluation means 304 receives the detection series D39 output from the detection series integration means 3334 of the feature quantity calculation means 333 as the rectangular feature quantity D33, and outputs a rectangular detection result D40.

The embedded feature amount calculation means 341 calculates an embedded sequence D41 as feature amount information that the target rectangle should have.
The feature amount similarity calculation unit 342 calculates the similarity D42 between the detection series obtained from each rectangular candidate and the embedded series.
The feature amount similarity comparison unit 343 compares the similarities obtained for the respective rectangle candidates, and determines the most appropriate rectangle.

Next, an example of rectangle addition processing in the image detection method according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of a rectangle addition process in the image detection method according to the present embodiment.
As shown in FIG. 8, the rectangle adding apparatus 100 inputs image data of the rectangle addition target image C12 (step ST1). This process is hereinafter referred to as a rectangle addition target image input step.
Next, the rectangle adding apparatus 100 adds a feature rectangle image having a predetermined drawing feature to the image data of the rectangle addition target image C14, and generates a rectangle addition image C12 (step ST2). This process is hereinafter referred to as a feature quantity generation step.
Then, the rectangular pattern generation unit 102 of the rectangle adding device 100 follows the numerical sequence that is the feature amount information D11 generated by the feature amount generation unit 101 (embedded sequence generation unit 111) in the feature amount generation step ST2, and is on the side of the rectangle. A pixel value corresponding to the division position and the pixel channel is determined, and a feature graphic pattern indicating a feature rectangular graphic represented by the determined pixel value is generated (step ST3). This process is hereinafter referred to as a rectangular pattern generation step.
Next, the rectangle adding unit 103 of the rectangle adding apparatus 100 draws the feature rectangle figure C10 on the rectangle addition target image C11 in accordance with the rectangular pattern D12 generated by the rectangle pattern generating unit 102 (step ST4). This process is hereinafter referred to as a rectangle adding step.

<Feature Quantity Generation Step ST2>
Next, the feature quantity generation step of step ST2 shown in FIG. 8 will be described in detail with reference to FIG. FIG. 9 is a diagram for explaining an example of the feature amount generation step by the rectangle adding device 100 according to the present embodiment.

In the feature quantity generation step ST2, the feature quantity generation unit 101 refers to the information stored in the storage unit 104 and generates the feature quantity information that should be included in the rectangle to be added.
In the feature value generation step ST2 of this embodiment, the embedded sequence generation unit 111 generates a predetermined numerical sequence that characterizes a rectangle.

  The embedded sequence generation unit 111 determines the length of the numerical sequence according to the following equation (1). In Equation (1), N is the number of divisions for dividing any one of the four sides constituting the rectangle. c is the number of channels of pixel values used as feature quantities. Note that there are two ways of the inner edge and the outer edge of the side.

The embedded sequence generation unit 111 determines the length of the numerical sequence indicating the drawing feature amount information D11 corresponding to an arbitrary one of the four sides constituting the rectangle as the value shown in the equation (1).
Note that the length L of the numerical sequence corresponding to the entire four sides can be expressed by the following equation (2), where N ₁ , N ₂ , N ₃ , and N ₄ are the number of divisions for each side.

Here, the division number N {number of divisions N ₁ , N ₂ , N ₃ , N ₄ } of each side and the channel number c of the pixel value are values shared by the rectangle adding device 100 and the rectangle detecting device 300, Any integer greater than or equal to 1 is acceptable. Further, the division number N {division number N ₁ , N ₂ , N ₃ , N ₄ } of each side and the channel number c of the pixel value are the rectangular addition target image C11, the rectangular detection target image C14, and the image display device 200. Resolution and pixel conditions, the distance between the image display device 200 and the mobile terminal 400, the angle of view of the camera provided in the mobile terminal 400, and the like, are determined in advance according to the assumed shooting environment and conditions.

  The upper limit of the division number N is, for example, that the resolution of the rectangular addition target image C11, the rectangular detection target image C14, and the image display device 200 is in the longitudinal direction of any one of the four sides constituting the rectangle. If the corresponding number of pixels is W pixels, dividing the sides more finely than W pixels results in a division position that is not reflected in the pixel value and is meaningless. For this reason, the upper limit of the division number N may be considered as W.

Specifically, for example, a rectangle is added and detected by changing the number of divisions N between a lower limit value 1 and an upper limit value W under an environmental condition in which use of the rectangle adding device 100 and the rectangle detecting device 300 is assumed. Evaluate the results. A division number N is determined experimentally in advance so that the detection success rate for successful detection of the target rectangle is maximized or sufficiently accurate (for example, a detection success probability of 80% or more). The user may determine the number of divisions N ₁ , N ₂ , N ₃ , N _{4 for} each side and the channel number c of the pixel value.

  In the present embodiment, the channel number c of the pixel value differs depending on whether the color of the rectangular addition target image C11 is black and white, color, or the color table dimension method. For example, when the rectangular addition target image is a monochrome image, the number of channels c = 1. In the case of a color image, the number of channels c = 3 when the R, G, and B values of the pixels are used, or when the Y, Cb, and Cr values and H, S, and V values of the pixels are used. When only the Cb value of the pixel is used, the channel number c = 1.

The embedded sequence generation unit 111 may use a pseudo-random number sequence or a numerical sequence shared with the rectangle detection apparatus 300 in advance in generating a numerical sequence.
When using a pseudo-random number sequence, the embedded sequence generation unit 111 may share the pseudo-random number seed value with the rectangle detection device 300, for example. This sharing means, for example, that the same pseudo-random seed value is stored in advance in the storage unit of the embedded sequence generation unit 111 and the rectangle detection device 300.

  In the case of using a pseudo-random number sequence, the embedded sequence generation unit 111 may use, for example, a numerical sequence of values shown in (3) below as a length L pseudo-random number sequence taking {1; -1}.

As this pseudo-random number sequence, the embedded sequence generation unit 111 may generate, for example, an M sequence or a GOLD sequence.
When using a numerical sequence shared with the rectangle detection apparatus 300 in advance, for example, a numerical sequence that repeats 1, -1 is used, as shown by {1, -1,1, -1. May be.

  Further, the embedded sequence generation unit 111 may use a predetermined function f (x) and use a numerical string of values shown in (4) below.

  An example of a numerical sequence using a trigonometric function as the predetermined function is shown in Expression (5).

In addition, the embedded sequence generation unit 111 may use an arbitrary function that determines the value f (x) of the xth element of the numerical sequence for the input value x as the predetermined function.
The generated numerical sequence is the feature amount information that the rectangle should have.

<Rectangle pattern generation step ST3>
Next, the rectangular pattern generation step will be described in detail.
In the rectangular pattern generation step ST3, the rectangular pattern generation unit 102 follows the numerical sequence that is the feature amount information D11 generated by the feature amount generation unit 101 (embedded series generation unit 111) in the feature amount generation step ST2 on the side of the rectangle. A pixel value corresponding to the division position and the pixel channel is determined, and a feature graphic pattern indicating a feature rectangular graphic represented by the determined pixel value is generated.

STEP301:
First, the rectangular pattern generating means 102, a numerical sequence m _i of the embedding sequence generation unit 111 is generated by the feature value generating step ST2, the divided positions on each side of the rectangular, inner and outer sides of the edges, the channel of the pixel Arrange them in order so that they correspond to each other.

Here, it demonstrates with reference to FIG. FIG. 9 is a diagram illustrating an example of division positions on each side of the rectangle, inner and outer sides of the edges of the sides, pixel channels, and the like included in the numerical series. FIG. 9 shows a side figure P in which only one arbitrary side is extracted from the four sides constituting the characteristic rectangular figure. The side figure P is a figure that represents one side constituting the rectangular feature figure as a long and narrow rectangular figure having a thickness.
As shown in FIG. 9, the rectangular pattern generation unit 102 divides the side figure P of the feature rectangular figure by the division number N in the longitudinal direction of the side figure P, and the side figure P is arranged in the short direction of the side of the feature rectangular figure. And N × 2 edge regions P _{1 to} P _{N × 2} are determined.

In FIG. 9, the rectangular pattern generation unit 102 applies a numerical sequence {m _i | 1 for each of the three channels Y, Cb, and Cr of the pixels for each of the edge regions P _{1 to} P _{N × 2.} An example in which ≦ i ≦ 6N} is associated is shown.

  The rectangular pattern generation means 102 performs this for the four sides. FIG. 10 shows an example in which the rectangular pattern generation unit 102 thus forms an entire rectangle. FIG. 10 shows an example in which a numerical series is associated with each of the edge regions in the feature rectangle figure for each of the three channels Y, Cb, and Cr of the pixel.

The rectangular pattern generation unit 102 performs the above processing on the entire rectangle. That is, the rectangular pattern generation means 102 divides the side figures P1, P2, P3, and P4 constituting the side figure P of the feature rectangular figure into the number of divisions N ₁ and N in the longitudinal direction of the side figures P1, P2, P3, and P4, respectively. ₂ , N ₃ , and N ₄ , and inside and outside of each edge in the short direction of the side figures P 1, P 2, P 3, and P 4. The rectangular pattern generation unit 102 divides each of the side figures P1, P2, P3, and P4 to thereby obtain edge regions P1 _{1 to} P1 _{N1 × 2} , P2 _{1 to} P2 _{N2 × 2} , P3 _{1 to} P3 _{N3 × 2} , P4 _{1 to} P4 _{N4 × 2} are obtained, and for these, a numerical series {m _i | 1 ≦ i ≦ 6 (N ₁ + N ₂ + N ₃ + N ₄ )} for every three channels Y, Cb, and Cr of the pixel Perform the association to associate.

FIG. 10 illustrates an example in which N ₁ = N ₂ = N ₃ = N ₄ = N.
Further, as shown in FIG. 10, for example, (m ₁ , m ₂ , m ₃ ) corresponds to the value corresponding to the Y channel value m ₁ , the value corresponding to the Cb channel value m ₂ , and the value corresponding to the Cr channel. The value to be expressed is m ₃ .

  In FIG. 10, the rectangular pattern generation unit 102 arranges the numerical series generated by the embedded series generation unit 111 in order in order to correspond to each position (edge area) of the rectangle. The order may be as follows. For example, in accordance with a pseudo-random number shared with the rectangle detection device 300, the corresponding positions may be determined randomly, or every other corresponding position may be determined. It may be arranged in another predetermined order.

STEP302:
Next, the rectangular pattern generation unit 102 determines a pixel value at that position in accordance with the value of the numerical series associated with each position (edge region) of the rectangle. Specifically, the rectangular pattern generation unit 102 sets a reference pixel value for each of the inside and outside of the edge, and determines the pixel value by weighting and adding the value of the numerical series to the value.
An example in which three channels of Y, Cb, and Cr are used will be described below.
The rectangular pattern generation unit 102 sets the reference values of Y, Cb, and Cr for the inside of the edge to values shown in the following (6).

  Further, the rectangular pattern generation means 102 sets the reference values of Y, Cb, and Cr for the outside of the edge to values shown in the following (7).

  Since the rectangular pattern generation unit 102 draws the edge of the side figure P of the characteristic rectangular figure as an edge, the difference between the reference values inside and outside the edge of one or more channels is within a predetermined range. To do. For example, if the value of the luminance Y is large on the inside of the edge and small on the outside, a rectangle having a white inside and a black outside is drawn as in the example of FIG. Conversely, if the value of the luminance Y is small on the inside of the edge and large on the outside, a rectangle with the black inside and the white outside is drawn. Similarly, the rectangular pattern generation unit 102 may have different colors on the inner side and the outer side, for example.

  In addition, if the rectangular pattern generation unit 102 detects a side at the edge of the channel so that the reference value inside and outside the edge has a predetermined difference in one channel, the edge of the other channel is detected. The internal and external reference values may take the same value. For example, the case where the values of the Y, Cb, and Cr channels take values from 0 to 255 will be described as an example. The rectangular pattern generation unit 102 sets the reference value for the luminance Y so that there is a difference between the reference values inside and outside the edge. However, for Cb and Cr, the reference values inside and outside the edge may be set to 127 respectively.

  The difference between the reference values inside and outside the edge is that the calculation result of the filter has an edge component in the edge component determination using the edge detection filter in the edge candidate detection step and the edge tracking step in the rectangle detection apparatus 300 described later. Is determined in advance so as to have a difference that exceeds a threshold value determined as. However, it is necessary to consider that noise is added to the pixel value in the rectangular detection target image due to the difference between the pixel values at the time of adding the rectangle due to image capturing by the camera of the mobile terminal 400.

  Specifically, the rectangular pattern generation unit 102 adds a rectangle so that, for example, when the pixel value takes a value of 0 to 255, the difference between the reference values inside and outside the edge takes a value such as 50 to 255, The threshold value may be obtained experimentally so that the rectangle detection apparatus 300 has appropriate rectangle detection accuracy.

  The rectangular pattern generation unit 102 sets the values of the numerical series associated with each of Y, Cb, and Cr of the inner point at the position (edge region) x on the side as the values shown in (8) below.

  Also, the rectangular pattern generation unit 102 sets the values of the numerical series associated with the outer points Y, Cb, and Cr at the position (edge region) x on the side as the values shown in (9) below. .

At this time, the rectangular pattern generation means 102 obtains the pixel values Z _in ^[Y] , Z _in ^[Cb] , and Z _in ^[Cr] of the inner points Y, Cb, and Cr at the position (edge region) x on the side, respectively. The value is shown in the following formula (10).

Further, the rectangular pattern generation unit 102 obtains the pixel values Z _out ^[Y] , Z _out ^[Cb] , and Z _out ^[Cr] of the outer points Y, Cb, and Cr at the position (edge region) x on the side, The value is shown in the following formula (11).

Here, α and β are parameters for determining a predetermined embedding strength. Note that the rectangular pattern generation unit 102 sets α and β to different values α ^[Y] , α ^[Cb] , α ^[Cr] , β ^[Y] , β ^[Cb according to the human visual characteristics for each channel. ^{] And} β ^[Cr] may be used.

As a specific example, for example, the rectangular pattern generation unit 102 takes C _in ^[Y] = 223, C _out when the pixel value ranges from 0 to 255 and the numerical value series ranges from −1 to +1. ^[Y] = 32, C _in ^[Cb] = C _out ^[Cb] = C _in ^[Cr] = C _out ^[Cr] = 127, and α and β take values from 1 to 32. Good.

  In this case, for example, if α = β = 32, the rectangular pattern generation unit 102 indicates that the pixel value of the inner point of the rectangular pattern and the pixel value of the outer point of the rectangular pattern are shown in (12) and (13) below. Decide to vary in range.

Pixel value of inner point of rectangular pattern

Pixel value of outer point of rectangular pattern

Further, the rectangular pattern generation means 102 may set the values of α and β so that the color change at each rectangular division position is not easily noticeable by humans, and is within 5% of the pixel value range. The width of the fluctuation of the pixel value of the rectangular pattern may be made small.
The rectangular pattern generation unit 102 can generate the rectangular pattern as described above, so that the characteristic rectangular figure has a predetermined characteristic represented by a numerical series.

  In the above description, an example in which the rectangular pattern generation unit 102 generates a rectangular additional pattern using a three-channel color space of Y, Cb, and Cr has been described. However, other color spaces may be used. For example, the color space may be R, G, B (R: red, G: green, B: blue) or H, S, V (H: hue, S: saturation, V: lightness) The color space may be an XYZ color system or L * a * b * color system.

<Rectangle addition step ST4>
In the rectangle addition step ST4, the rectangle addition unit 103 draws the feature rectangle figure C10 on the rectangle addition target image C11 according to the rectangle pattern generated by the rectangle pattern generation unit 102. Drawing of a rectangle by the rectangle adding unit 103 can be realized by the following procedure, for example.
Here, each side of the feature rectangular shape C10, it is assumed that the width of the predetermined pixels with a width W _out of the width W _in the outer edges of the inner sides, having such shape of FIG. In addition, the term “inside and outside of each side or edge” refers to the inside and outside of the boundary between the inside side (white portion) and the outside side (black portion). That is, the characteristic rectangular figure C11 is a rectangular figure on the frame, and has a white pattern on the inside and a black pattern on the outside.

Hereinafter, the rectangle adding means 103 by the rectangle adding means 103 is used.
STEP 401:
The rectangle adding unit 103 selects one pixel on the rectangle addition target image C11.
STEP402:
The rectangle adding means 103 determines whether the coordinates of the selected pixel are on the side of the rectangle to be drawn.
STEP403:
If the coordinate of the selected pixel is not on the side of the rectangle, the rectangle adding unit 103 outputs the value of the pixel of the rectangle addition target image as the pixel value of the coordinate of the rectangle added image, and proceeds to STEP 406. STEP 404:
When the coordinates of the selected pixel are on the side of the rectangle, the rectangle adding unit 103 calculates the division position and the inside and outside types of the side on the rectangle corresponding to the pixel coordinate.

STEP405:
The rectangle adding means 103 outputs the value of the pixel at the position obtained in STEP 404 in the rectangular pattern as the pixel value of the coordinate of the rectangle added image. STEP 406:
The rectangle adding unit 103 repeats STEPs 401 to 405 until all the pixels of the rectangle addition target image C11 are scanned. In addition to the rectangle drawing by the rectangle adding means 103, for example, the Bresenham algorithm for drawing a straight line is used, and the pixel value is set according to the position on the rectangular pattern corresponding to the position of the pixel to be drawn. Also good.
Further, the rectangle adding unit 103 may perform the rectangle addition while performing the enlargement / reduction process so that the entire rectangle addition target image fits inside the inner side on the rectangle addition image C12.

Further, the rectangle adding unit 103 outputs the pixel value at the position obtained in STEP 404 in the rectangular pattern as the pixel value of the coordinate of the rectangle added image in STEP 405 so that the rectangle to be drawn is not visually noticeable. Instead, the pixel value at the position obtained in STEP 404 in the rectangular pattern and the pixel value of the rectangular addition target image are α blended (that is, a value obtained by adding a predetermined weight), and a value is output. May be.
For example, assuming that the value of the pixel at the position obtained in STEP 404 in the rectangular pattern is x and the value y of the pixel of the rectangular addition target image, the output pixel value z can be expressed by the following equation (14).

In addition, the rectangle adding unit 103 may change the value of α according to the coordinates of the pixels so that, for example, the weight of the pixels of the rectangle addition target image increases toward the inside of the image.
Here, the value of α takes a value of 0 to 1.0. When α = 0, the pixel value y of the rectangular addition target image is set to z, and when α is 1.0 to 1.0, the pixel value x of the rectangular pattern is set. As z, when α = 0.5, a pixel value obtained by mixing the rectangular addition target image and the rectangular pattern by 50% is output as z.

<Image output step ST5>
In the image output step ST5, the rectangle adding means 103 outputs the rectangle added image C12 generated in the rectangle adding step ST4 to the image display device 200 as a result of the processing.

<Method for determining image values>
A method for determining the pixel value of the rectangular pattern by the rectangular pattern generation unit 102 will be described below.
The storage unit 104 uses Y, Cb, Cr reference values C _in ^[Y] , C _in ^[Cb] , C _in ^[Cr] for the inside of the edge and Y, C for the outside of the edge used in the rectangular pattern generation step ST3. For the reference values C _out ^[Y] , C _out ^[Cb] , and C _out ^[Cr] of Cb and Cr, the pixel values determined by the rectangular pattern generation unit 102 according to the concept described in the description of the rectangular pattern generation step ST3 described above are used. These may be stored in advance. In this case, the rectangular pattern generation means 102 determines the pixel value which comprises a rectangular pattern with reference to the memory | storage part 104 in rectangular pattern generation step ST3.

In addition, the rectangular pattern generation unit 102 sets the pixel value to a maximum value and a minimum value that do not exceed a predetermined range according to the values of the embedding strengths α and β and the range of values that the numerical series can take. Alternatively, the reference value may be determined.
Specifically, for example, the rectangular pattern generation unit 102 takes a value of −w _max ,..., + W _{max as} the numerical series, the embedding strength α = β, and the range of possible pixel values is c _min. ,..., C _max , the inner and outer reference values C _in and C _out may be determined as shown in the following equation (15).

Here, the rectangular pattern generation means 102 is set to a maximum value in a range where the inner pixel does not exceed c _max and a minimum value in a range where the outer pixel does not fall below c _min .
For example, if c _min = 0, c _max = 255, α = β = 8, and w _max = 1, C _in = 247 and C _out = 8.

In addition, the rectangular pattern generation unit 102 may determine the reference value so that the pixel value in a color space different from the channel of the pixel generated as a rectangular pattern does not exceed a predetermined range, for example. An example in which the reference value is determined so that the pixel value in the R, G, B space does not exceed a predetermined range when the channel is Y, Cb, Cr.
Here, the case where the conversion coefficients of the Y, Cb, Cr and R, G, B color spaces are represented by the following equation (16) will be described as an example.

In this case, the rectangular pattern generation means 102 takes the values of −w _max ,..., + W _max for each of Y, Cb, and Cr channels, and R, G, When the possible range of the value of B is c _min ,..., C _max , for example, by setting the following expression (17), the values of R, G, B do not exceed the predetermined range, and Y The value of can be maximized.

<Rectangle detection procedure according to the first embodiment>
Next, an example of a rectangle detection process performed by the rectangle detection apparatus 300 according to this embodiment will be described with reference to FIG. FIG. 12 is a flowchart for explaining an example of a rectangle detection process by the rectangle detection apparatus 300 according to this embodiment.

In the rectangle detection apparatus 300 according to the present embodiment, a rectangle candidate obtained from the rectangle detection target image C14 is evaluated by determining whether or not the rectangle candidate has a feature given in advance to the feature rectangle figure. Determine the certainty.
FIG. 12 is a flowchart illustrating a procedure of rectangle detection processing by the rectangle detection apparatus 300.

Next, an example of rectangle detection processing in the image detection method according to the present embodiment will be described with reference to FIG. FIG. 12 is a flowchart illustrating an example of a rectangle detection process in the image detection method according to the present embodiment.
As shown in FIG. 12, the rectangle detection apparatus 300 inputs image data of a rectangle detection target image C14 that is a rectangle detection target (step ST11). This process is hereinafter referred to as a rectangle detection target image input step.

<Edge candidate detection step>
Next, the edge candidate detection unit 301 of the rectangle detection apparatus 300 analyzes the rectangle detection target image C14 and detects the edge of the rectangle (step ST12). This process is hereinafter referred to as an edge candidate detection step.
The edge candidate detection unit 301 detects edges in a rectangle by detecting straight lines that are edge candidates.
The detection of a straight line as a side candidate by the side candidate detection unit 301 can be performed at a high speed by using, for example, the methods of Patent Literature 1 and Reference Literature 1.

Specifically, for example, the following procedure can be performed.
STEP1201:
The edge candidate detection unit 301 uses the image center point in the rectangular detection target image C14 as a search start point.
Here, the search start point may be a coordinate other than the image center point, for example, a coordinate other than the center point near the center of the image. In addition, for example, in the case of rectangle detection for a video composed of a plurality of frame images, edge candidate detection is performed on coordinates near the center of the rectangle detected in the immediately preceding frame, for example, the center of gravity of the vertex of the rectangle or the intersection of the diagonal of the rectangle. Means 301 may be obtained and used as a search start point.

STEP1202:
The edge candidate detection unit 301 moves the search position in each of the upper, lower, left, and right search directions.
STEP1203:
The edge candidate detection unit 301 applies an edge detection filter to a region around the search position, and determines whether the search position has an edge component.
STEP 1204:
If the edge candidate detection unit 301 determines that the edge component has an edge component, the search position at that time is set as an edge tracking start point.

STEP 1205:
The edge candidate detection unit 301 moves the edge tracking position in a direction orthogonal to the search direction (for example, the left and right directions when the search direction is upward).
STEP 1206:
The edge candidate detection unit 301 applies an edge detection filter to the surrounding area for each point near the edge tracking position, and determines whether any of the points near the edge tracking position has an edge component.
STEP 1207:
If the edge candidate detection unit 301 determines that the edge component has an edge component, the point 1205 to 1206 is repeated with the point having the strongest edge component among the neighboring points as the edge tracking position. When it is determined that the edge component is not included, the side candidate detecting unit 301 stores the immediately preceding edge tracking position as the end point of the side.
The edge candidate detection unit 301 performs STEPs 1205 to 1207 in both directions of the edge viewed from the edge tracking start point (for example, the left and right directions when the search direction is upward) to obtain end points on both sides of the edge.

STEP1208:
If the search position has not reached the end of the image, the edge candidate detection unit 301 returns to STEP 1202 and repeats the process.
As a result, the edge candidate detection unit 301 has information as the edge candidate as
・) Edge tracking start point that first detected the edge of the corresponding side candidate,
・) The coordinates of the endpoints on both sides that have finished tracking the candidate,)
-The search direction which detected the said edge candidate is obtained by the number of edge candidates.

The edge candidate detecting unit 301 detects, for the obtained edge candidate, the length of the detected edge (for example, the distance between the end point coordinates on both sides, the number of times edge tracking was possible, etc.) and the detected edge Depending on the intensity of the component (for example, the average value or cumulative value of the value of the result of the edge detection filter), it is possible to leave only the edge candidates having sufficient length and intensity and reject others.
The example of edge candidate detection shown here is only an example, and any other method may be used as long as it is a method for detecting a linear component in a rectangular detection target image. For example, the edge candidate detection unit 301 may detect a straight line using Hough transform (see Reference 5).
<Reference 5>: Supervision by Mikio Takagi, New Image Analysis Handbook, pp.1254-1258, ISBN4-
13-061119-4, University of Tokyo Press (2004).

  In step 1201 and 1202, the edge candidate detection unit 301 uses the image center point as a search start point and moves the search position toward the outside of the image. However, the present invention is not limited to this, and for example, the search may be performed toward the center of the image with the points on the four outer edges of the image as search start points, respectively. In this case, it should be noted that the relationship between the search direction and the inside and outside of the side in the following example is reversed.

<Rectangle candidate calculation step>
In the rectangle candidate calculation step ST13, the rectangle candidate calculation unit 302 determines a combination that may form a rectangle from the edge candidates obtained in the edge candidate detection step ST12. The calculation of the rectangle candidates can be performed at high speed by using, for example, the methods of Patent Document 2 and Reference Document 1.

Specifically, for example, the following procedure can be performed.
STEP1301:
The rectangle candidate calculation unit 302 selects a total of four side candidates, one by one from the side candidates detected in the search directions of the top, bottom, left, and right.
STEP1302:
A rectangle formed by straight lines representing the four side candidates is defined as a rectangular candidate, and four intersections of the straight lines are obtained.
STEP1303:
The rectangle candidate calculation unit 302 calculates the lengths of the four sides of the rectangle candidate using the intersection as the estimated vertex coordinates of the rectangle.
STEP 1304:
The rectangle candidate calculation unit 302 calculates the lengths of the four side candidates from the end point coordinates of the four side candidates.

STEP 1305:
The rectangle candidate calculation unit 302 calculates the ratio of the lengths of the four side candidates obtained in STEP 1304 to the lengths of the four sides of the rectangle candidate obtained in STEP 1303, and uses this as the rectangle evaluation value. However, if the length of the side candidate is longer than the length of the side of the rectangle candidate, one of the following methods may be used. The rectangle candidate calculation unit 302 determines the length of the side candidate and the side of the rectangle candidate. A value obtained by subtracting the difference in length from the side length of the rectangle candidate is used as the length of the side candidate.
The rectangle candidate calculation means 302 uses the reciprocal of the obtained ratio as the rectangle evaluation value STEP 1306:
The rectangle candidate calculation means 302 rejects an object whose rectangle evaluation value is a predetermined threshold value or less.

In the present embodiment, the predetermined threshold is a range between 0 and 1, for example. The rectangle candidate calculation unit 302 determines how clear the rectangle to be detected can be captured in the rectangle detection target image and based on a trade-off between calculation processing times.
That is, when the rectangle detection apparatus 300 is used in an environment in which a rectangle can be clearly photographed, the rectangle candidate calculation unit 302 can trace the edge of the correct rectangle candidate to the end point of the actual edge in the edge candidate detection step ST12. . In this case, since the rectangle evaluation value approaches 1, the rectangle candidate calculation unit 302 uses a value close to 1 as the threshold value of the rectangle evaluation value, so that the number of detected rectangle candidates decreases, and the calculation processing time of the subsequent processing Can be shortened.
On the other hand, when the rectangle detection apparatus is used in an environment where the rectangle cannot be clearly captured, in the edge candidate detection step ST12, the rectangle candidate calculation unit 302 extends the edge only to the middle of the actual edge with respect to the correct rectangle candidate. None Or the case where the actual side is exceeded and the side is traced occurs. Therefore, the rectangle candidate calculation unit 302 may set a small threshold value so that the correct rectangle candidate is not overlooked. A specific threshold value may be experimentally determined within a range of about 0.5 to 0.95, for example.

As a result, the rectangle candidate calculation unit 302 has information as a rectangle candidate as
-Four corresponding side candidates-The estimated vertex coordinates of the rectangle obtained in STEPs 1302 and 1303 are obtained by the number of rectangle candidates.
The example of rectangle candidate calculation shown here is only an example, and any other method may be used as long as it is a method for detecting a rectangle in the rectangle detection target image.

<Rectangle feature calculation step>
In the rectangular feature amount calculating step ST14, the rectangular feature amount calculating unit 303 evaluates the rectangular candidate obtained in the rectangular candidate calculating step ST13 to determine whether the rectangular candidate is a target rectangular region. Calculate In the rectangular feature amount calculation step ST14, the rectangular feature amount calculation means 303 sequentially performs an edge tracking step ST141, a pixel value acquisition step ST142, and a feature amount calculation step ST143.

<Edge tracking step>
In the edge tracking step ST141, the edge tracking unit 331 scans the edges constituting the side in a direction parallel to the side for each of the four side candidates included in the rectangle candidate, and calculates the edge coordinates at each point on the side. To detect.

Specifically, for example, edge tracking is performed in the following procedure.
STEP1411:
The edge tracking unit 331 sets an arbitrary point on the target side candidate as the edge tracking current position. As the edge tracking current position, for example, the coordinates of the edge tracking start point detected first in the edge candidate detection step ST12 may be used. In addition, the coordinates of the end point that has finished tracking the target side candidate in the side candidate detection step ST12 may be used.
STEP1412:
The edge tracking unit 331 outputs the coordinates of the current edge tracking position as edge coordinates.
STEP1413:
The edge tracking unit 331 moves the current edge tracking position in a direction parallel to the target side candidate. The direction of movement of the edge tracking current position is, for example, a direction orthogonal to the search direction when the target side candidate is detected in the side candidate detection step ST12 (for example, the left and right directions when the search direction is upward). There may be. Further, for example, the direction of the inclination of the target side candidate calculated from the end point coordinates on both sides after the tracking of the target side candidate in the side candidate detection step ST12 may be used.

STEP 1414:
The edge tracking unit 331 applies an edge detection filter to the surrounding area for each point in the vicinity of the current edge tracking position, and determines whether any of the neighboring points in the current edge tracking position has an edge component.
STEP1415:
If the edge tracking unit 331 determines that it has an edge component, the point having the strongest edge component among the neighboring points is set as the edge tracking current position, and the process returns to STEP 1412. If the edge tracking unit 331 determines that the edge tracking unit 331 has no edge component, the process ends. When the start point of edge tracking is determined to be other than the end points of the side candidates, the edge tracking unit 331 performs the processing of STEPs 1412 to 1415 in both directions of the side from the start point of edge tracking.
STEP1416:
The edge tracking unit 331 applies each procedure of STEPs 1411 to 1415 to each of the four side candidates included in the rectangle candidate, and outputs edge coordinates corresponding to each point on the four side candidates.

Here, the determination of the edge component in STEP 1414 by the edge tracking means 331 will be described below with reference to FIGS.
FIG. 13 is a diagram for explaining an example of edge tracking. FIG. 14 is a diagram for explaining an example of the edge detection filter.
In the example illustrated in FIG. 13, the edge tracking unit 331 sets each of the neighboring positions centered on the current edge tracking position as a target coordinate.
For example, the proximity position may be set in a range of W pixels in the direction orthogonal to the movement direction of the edge tracking current position with respect to the edge tracking current position.

The grid in FIG. 13 represents a row of pixels of the rectangular detection target image, and the shading represents the size of the pixel value.
FIG. 13 shows an example in which a range of 5 × 1 pixels is set as a neighborhood position range Q1 in the left and right Xq directions orthogonal to the edge tracking direction Yq with W = 5 with respect to the edge tracking current position q1. ing.

  Here, the edge tracking unit 331 calculates a weighted sum for a plurality of pixels around the target pixel for an arbitrary target pixel. The weighted sum can be obtained, for example, by multiplying the pixel values of a plurality of pixels around the target pixel by using the edge detection filter coefficient 1401 of FIG.

In the calculation example 1504 of the weighted sum shown in FIG. 14, the grid represents a column of pixels of the rectangular detection target image C14, and the numerical value represents a pixel value.
In the case of the pixel of interest 1402, the edge tracking unit 331 obtains a weighted sum as shown in (18) below by performing a product-sum operation with the edge detection filter coefficient 1401 on the calculation range 1403.

The edge tracking unit 331 sets this value as the edge strength of the target pixel 1402.
The edge tracking unit 331 calculates the edge strength for each target coordinate in the vicinity position, and determines the target pixel having the largest value. Furthermore, the edge tracking unit 331 determines that the target coordinate has an edge component when this value exceeds a predetermined threshold value.

Here, the predetermined threshold is based on the range of pixel values and the range of edge strength determined from the coefficient value of the edge detection filter, and is large enough to prevent detection of portions other than edges. A small value is set so that the edge is not determined to be an edge. Further, the edge tracking means 331 may be obtained experimentally in consideration of the difference between the reference values inside and outside the edge in the above-described rectangular pattern generation step ST3 and the degree of noise addition in the rectangular detection target image C14. Specifically, in the example of the edge detection filter coefficient 1401, when each pixel value can take a value of 0 to 255, the range of values that the edge intensity can take is 0 to 1020, for example, a value such as 10 to 512 May be used as a threshold value.
The edge tracking unit 331 determines a pixel having an edge component by such a method, for example, when a rectangle is drawn as shown in FIG. 11, the boundary position between the white inner side and the black outer side. Will be obtained.

Here, the value of the width W of the neighboring position is set to 5, but it goes without saying that other values may be used.
In addition, although the edge detection filter coefficients having a size of 3 × 3 are used here, other sizes may be used.
For example, the size may be 3 × 1 as an edge detection filter coefficient 1405 that is a weighted sum planning example 1404 that is another example of the edge detection filter coefficient 1401 of FIG.
When the method of Patent Document 1 is used in the edge candidate detection step ST12, the edge tracking unit 331 tracks the edge tracking in the edge candidate detection step ST12 instead of performing edge tracking in the edge tracking step ST141. The edge coordinates may be recorded and used for output.

<Pixel value acquisition step>
In the pixel value acquisition step ST142, the pixel value acquisition unit 332 acquires, from the rectangular detection target image, the pixel values of the points inside and outside the edge of the edge candidate with respect to the edge coordinates obtained in the edge tracking step ST141.

Specifically, the pixel value is acquired by the following procedure.
STEP1421:
The pixel value acquisition unit 332 obtains coordinate values up to a point that is 1 to n pixels away from the edge coordinates obtained in the edge tracking step ST141 in a direction orthogonal to the edge candidate.
The pixel value acquisition unit 332 may use the search direction when the target side candidate is detected in the side candidate detection step ST12 as the direction orthogonal to the side candidate, for example. Further, the pixel value acquisition unit 332 may be, for example, in a direction orthogonal to the inclination of the target side candidate calculated from the end point coordinates on both sides that have finished tracking the target side candidate in the side candidate detection step ST12. Good.

STEP 1422:
The pixel value acquisition unit 332 acquires and outputs a pixel value at a coordinate away from the edge coordinate in the search direction when the side candidate is detected as a pixel value outside the side. FIG. 15 shows an example where n = 2, and the pixel values of two pixels of the pixel 1502 outside the side are acquired and output with respect to the edge coordinate 1501. FIG. 15 is a diagram for explaining an example of image acquisition by the pixel value acquisition unit 332.

STEP 1423:
The pixel value acquisition unit 332 acquires and outputs a pixel value at a coordinate separated from the edge coordinate in the direction opposite to the search direction when the side candidate is detected as a pixel value inside the side. In FIG. 15, similarly, the pixel values of two pixels of the pixel 1503 inside the side are acquired and output with respect to the edge coordinate 1501.
By acquiring the pixel values of the pixels outside and inside the edge coordinates in this way, for example, when a rectangle is drawn as shown in FIG. 12, the pixel values of the black outer side portion are outside the edge coordinates. As the pixel value, the pixel value of the white inner side is acquired as the pixel value inside the edge coordinates.
The edge tracking method described in the above-described edge tracking step ST141 can track edge coordinates at high speed and accurately determine the edge coordinates, so that the pixel values inside and outside the edge are respectively acquired. Suitable for doing.

  The pixel values acquired in STEPs 1422 and 1423 are the luminance values of the target pixels when the rectangular detection target image C14 is a monochrome image. When the rectangular detection target image C14 is a color image, the R, G, and B values of the target pixel may be used according to the color space of the rectangular detection target image. Also, Y, Cb, Cr (Y (: luminance, Cb, Cr: color value)) or H, S, V ((H: hue, S: saturation, V: lightness)) may be used. . Further, the color space of the rectangular detection target image may be converted to a value in another color space that is convenient for calculating the rectangular feature amount.

For example, when the XYZ color system is used in the rectangular pattern generation step ST3 of the rectangle adding apparatus 100, the pixel value acquisition unit 332 may convert the pixel value acquisition unit 332 into the XYZ color system.
The value of n is determined in accordance with conditions such as the thickness of the rectangle to be drawn at the time of drawing the rectangle and the shooting environment and shooting size of the rectangle detection target image. Here, an example in which the value of the target pixel range n is 2 is shown. However, the present invention is not limited to this, and needless to say, other values such as 1 and 3 may be used.

<Feature amount calculation step>
In the feature amount calculation step ST143, the feature amount calculation unit 333 accumulates the acquired pixel values, calculates the feature amounts for the sides, and calculates a rectangular feature amount that integrates the feature amounts of the sides.
Specifically, the rectangular feature amount is calculated by the following procedure.
STEP1431 (conversion parameter calculation step):
The conversion parameter calculation means 3331 calculates a conversion parameter for converting the coordinates on the sides of the rectangle candidates into the coordinate values representing the division positions of the rectangles for the rectangle candidates obtained in the rectangle candidate calculation step ST13. This can be obtained by calculating a projective transformation matrix that is projected onto the rectangular vertex coordinates in the coordinate system that represents the division position of the vertex coordinates of the rectangular candidate.

A specific example will be described with reference to FIG. FIG. 16 is a diagram for explaining an example of projective transformation parameter calculation and division position calculation. Here, description will be made assuming that the division number N = 4.
Assume that the rectangle candidate obtained in the rectangle candidate calculation step ST13 is the rectangle candidate 1601 in FIG. 16, and its vertex coordinates (x ₁ , y ₁ ), (x ₂ , y ₂ ), (x ₃ , y ₃ ), Projective transformation parameters for transferring (x ₄ , y ₄ ) to the vertices of the rectangle 1602 in the coordinate system representing the division position, (0, 0), (4, 0), (0, 4), (4, 4), respectively. Can be calculated as follows: That is, the conversion parameter calculation unit 3331 calculates the matrix H by solving an equation obtained by substituting the four vertex coordinates into the following relational expression (19) representing the projective conversion described in Reference 6 or the like, for example. can do.

Here, x˜ is a homogeneous coordinate representing a vertex before transformation, x˜ ′ is a homogeneous coordinate representing a vertex after transformation, and H is a 3 × 3 matrix representing a projective transformation parameter.
<Reference 6>"Digital image processing", CG-ARTS Association, ISBN4-906665-47-0
Here, the division number N = 4, but it goes without saying that this value may be any other natural number.

STEP1432 (division position calculation step):
The division position calculation unit 3332 converts the edge coordinates obtained by the edge tracking unit 331 using the conversion parameter obtained by the conversion parameter calculation unit 3331 in STEP1431, and calculates the division position on the side.
A specific example of division position calculation will be described with reference to FIG.
For example, it is assumed that the edge coordinates obtained by the edge tracking unit 331 are coordinates (x _E , y _E ) (1603 in FIG. 16). This is converted into a coordinate value representing the division position using the conversion parameter obtained in STEP 1431 to obtain coordinates (ξ ₁ , η ₁ ) (1604 in FIG. 16).

However, ξ ~ and x ~ are homogeneous coordinates representing coordinates (ξ _E , η _E ) and (x _E , y _E ) shown in the following formula (21), respectively.

Since the target edge coordinates (x _E , y _E ) are points on the lower side of the rectangle candidate, the division position calculation means 3332 calculates the X coordinate value ξ _E of the obtained coordinates (ξ _E , η _E ). In this case, the division position 1 can be determined by rounding off the decimal point to an integer value.

When the target edge coordinate is a point on the upper side or the lower side, the division position calculation means 3332 uses the X coordinate value ξ _E of the coordinates (ξ _E , η _E ) as in this example, and the target edge coordinate is the left side or In the case of a point on the right side, an appropriate division position can be determined by using the Y coordinate value η _E of the coordinates (ξ _E , η _E ).

  For example, the edge position tracked when the edge tracking unit 331 outputs the target edge coordinates is the edge candidate. It can be determined by memorizing which of the search directions in the upper, lower, left, and right sides is detected by the detection means 301.

STEP 1433 (edge detection sequence calculation step):
The edge detection series calculation means 3333 uses the division position obtained in STEP 1432 to accumulate the pixel values of each point on the detected edge for each division position, and calculates a detection series for the edge.

A specific example will be described with reference to FIGS.
FIG. 17 is an explanatory diagram of accumulation for each pixel value division position. As shown in FIG. 17, the edge coordinates x _{1 to} x ₁₃ obtained on the lower side of the rectangle candidate are converted into ξ ₁ to ξ _{13 in} STEP 1432 and classified into the division positions shown in the figure. Represents.
Also, an example is shown in which the pixel value acquisition unit 332 acquires Y, Cb, and Cr values as pixel values for the inside and outside of the side, respectively.
Next, the edge detection series calculation means 3333 obtains the average value of each pixel value classified into each division position. That is, the edge detection series calculation means 3333

  I represents the division position, and n represents the number of edge coordinates.

_The Y (x), _the Cb (x), _the Cr (x), each Y with respect to the inner points of the sides of the edge coordinates x, Cb, components Cr, Y _out (x), Cb _out (x), Cr _Outer (x) represents Y, Cb, and Cr components for the outer points of the edge coordinate x. The following (23) represents the maximum integer value not exceeding x.

Specifically, for example, in the example of FIG. 17, the luminance value Y _{in 0} of inner pixels in the divided position 0, the following equation (24) holds.

  The edge detection series calculation means 3333 sets an average of the pixel values for each division position obtained above in order as the edge detection series. The average value may be arranged in any order as long as it matches the correspondence between the position on the numerical value series and the division position on the side in the rectangular pattern generation means 102 in the rectangle adding device 100.

  For example, the edge detection series calculation unit 3333 may arrange the inside and outside values of the sides Y, Cb, and Cr in the order of division positions as indicated by 1801 in FIG. The order may be changed at random using columns. FIG. 18 is a diagram for explaining the edge detection series.

STEP 1434 (edge detection sequence integration step):
The detection sequence integration unit 3334 integrates the edge detection sequences obtained in STEP 1433 to obtain a detection sequence for the entire rectangle candidate.
The detection series integration unit 3334 performs integration of the edge detection series based on the correspondence between the numerical series used in the rectangular pattern generation step ST3 in the rectangle adding device 100 and the position on each side.
For example, when the side detection series of the upper side of the rectangle is obtained as shown in FIG. 18, the numerical series m = {m _i | 1 ≦ i ≦ L} and the position on each side in the rectangular pattern generation step ST3 are If expressed in an example as shown in FIG. 10, the relationship between the detection sequence d = {d _i | 1 ≦ i ≦ L} and the edge detection sequence is shown below in Expression (25).

Detection sequence integration unit 3334 is thus defined, the elements d _i elements m _i and detection sequence of numerical sequence as brute corresponding rectangular pattern generating step ST3, the thereby detected sequence with integrated edge detection sequence d = {D _i | 1 ≦ i ≦ L} is obtained.
It should be noted that the correspondence between the elements of the detection series and the positions on each side is any kind of correspondence as long as the correspondence between the numerical series in the rectangular pattern generation step ST3 and the positions on each side is common. Also good.

  In FIG. 10, the series is arranged in order on each side. However, for example, according to a pseudo-random number shared with the rectangle adding device 100, the correspondence position may be determined at random, or every other position. The corresponding position may be determined quickly.

<Rectangle feature evaluation step>
In the rectangular feature amount evaluation step ST15, the rectangular feature amount evaluation means 304 evaluates the similarity between the rectangular feature amount calculated in the rectangular feature amount calculation step ST14 and the predetermined feature amount, and determines the most appropriate rectangle.
In the rectangular feature amount evaluation step ST15, the rectangular feature amount evaluation unit 304 sequentially performs an embedded feature amount calculation step ST151, a feature amount similarity calculation step 152, and a feature amount similarity comparison step 153.

<Embedded feature amount calculation step>
In the embedded feature amount calculating step ST152, the embedded feature amount calculating unit 341 performs predetermined processing as feature amount information that the target rectangle should have by the same procedure as the feature amount generating step ST2 of the rectangle adding procedure in the rectangle adding device 100. Is calculated as an embedded sequence.
Various parameters necessary for calculation of the numerical series are shared by the rectangle adding device 100 and the rectangle detecting device 300. Specific examples of parameters to be shared include the following.
Number of divisions for dividing each side of the rectangle N ₁ ; N ₂ ; N ₃ ; N ₄
-Number of channels of pixel values used as feature values c
・ Inside and outside edge of edge ・ Generating method and key parameter of pseudo random number sequence used to generate numerical sequence ・ Numeric sequence used as numerical sequence itself

Here, the sharing may be performed by any method as long as a common value is held in the internal storage device in advance by the rectangle adding device 100 and the rectangle detecting device 300. . For example, a common value may be recorded in the storage devices of the rectangle adding device 100 and the rectangle detecting device 300 when the device is manufactured. Means for inputting a shared parameter may be provided in both or one of the rectangle adding device 100 and the rectangle detecting device 300, and the system operator may input a common value. Furthermore, one of the rectangle adding device 100 and the rectangle detecting device 300 may be connected to the other via a network, and the shared parameter may be communicated via the network. Moreover, you may use combining these methods.
The embedded feature amount calculating means 341 generates a numerical sequence {m _i | 1 ≦ i ≦ L} similar to the feature amount generating step ST2 as an embedded sequence as a result of the embedded feature amount calculating step ST151. Here, L is the length of the numeric string.

<Feature amount similarity calculation step>
In the feature quantity similarity calculation step ST152, the feature quantity similarity calculation means 342 calculates the similarity between the detection series obtained by the feature quantity calculation means 333 and the embedded series obtained in the embedded feature quantity calculation step ST151. calculate.
In general, in image processing, various methods exist as a measure of similarity between feature amounts, and the rectangle detection apparatus 300 can also calculate a similarity using these measures of similarity.
Hereinafter, the detection sequence obtained by the feature quantity calculation means 333 is d = {d _i | 1 ≦ i ≦ L}, and the embedded series obtained in the embedded feature quantity calculation step ST151 is m = {m _i | 1 ≦ i ≦. L}.

For example, two numerical sequences are regarded as vectors, values are normalized, and L ₁ norm, L ₂ norm, Camberra distance, Czekanowski's coefficient, normalized cross-correlation, extended L ₀ norm considering degradation model There is a method of using values such as as a measure of similarity. These similarity calculation methods are described in Reference 7.
<Reference 7>: Aiko Oka and Toshikazu Wada, “Study on image dissimilarity that is less susceptible to degradation”, Image Recognition and Understanding Symposium (MIRU2010) OS4-2, pp.683-690 (2010/7)

  In addition, for example, the feature quantity similarity calculation unit 342 may calculate the correlation value ρ by the following product-sum operation and use this as the similarity. When this is used, the calculation amount of the similarity is small and processing can be performed at high speed.

Here, “·” represents an inner product operation when the sequence is regarded as a vector.
Further, for example, the feature quantity similarity calculation unit 342 normalizes the d and m elements in advance so that each element of d and m has an average of 0 and a variance of 1, as shown below. By performing correlation value calculation using, an evaluation index ρ considering the magnitude of deviation from the normal distribution may be obtained and used as the similarity. If this is used, the magnitude of the similarity can be evaluated as an absolute value based on the magnitude of false positives.
The detection reliability evaluation criteria as described in Reference 3 are aligned, and more accurate detection of a rectangle that eliminates the possibility of erroneous detection is possible.

  Although an example using the sample mean and sample variance is shown here, the feature quantity similarity calculation unit 342 may perform normalization using unbiased variance.

<Feature amount similarity comparison step>
In the feature amount similarity comparison step ST153, the feature amount similarity comparison unit 343 obtains each of the rectangle candidates 1,... N selected in the rectangle candidate calculation step ST13 in the feature amount similarity calculation step ST153. The similarities ρ _i ,..., Ρ _n are compared, and the most suitable rectangle candidate i _result is determined.
For example, when the similarity ρ uses the correlation value as shown in the feature amount similarity calculation step ST153, the feature amount similarity comparison unit 343 indicates that the larger the value of ρ, the more similar the ρ is. Therefore, the rectangle candidate i that maximizes ρ can be determined as the most appropriate rectangle candidate.

Here, argmaxρ _i represents that the value of the subscript i that maximizes the value of ρ _i is obtained.

Further, the feature quantity similarity comparison unit 343 determines whether the maximum similarity ρ _result = ρ _iris shown in the following formula (29) exceeds a predetermined threshold, and does not exceed the predetermined threshold. It may be determined that an appropriate rectangle could not be detected.

Further, the similarity ρ feature quantity similarity calculation step ST153, for example, L ₂ as in the norm, if they are obtained as an index value that indicates that similar than smaller the value, the feature amount similarity It goes without saying that the comparing means 343 may select the minimum value instead of the maximum.

Here, the predetermined threshold for the similarity is determined by the similarity calculation method used in the feature quantity similarity calculation step ST153. The feature quantity similarity comparison unit 343 is large enough to exclude a rectangle having no desired feature ((noise component)) according to the range of similarity values obtained, and excludes a rectangle having the desired feature. It is sufficient to set a value that is not so small. The threshold value may be obtained experimentally in an actual environment, or may be quantitatively determined in accordance with a necessary false positive standard according to the similarity calculation method as described below.
That is, when the evaluation index ρ taking into account the magnitude of deviation from the normal distribution is used as the similarity, a rectangle having no target feature is erroneously detected as described in Reference 3. The threshold value may be set so that the probability (false positive) of becoming a target value (for example, less than 10 ⁻² or less than 10 ⁻⁷ ). The target value of the false positive probability may be determined based on the criteria necessary to exclude rectangles (noise components) that do not have the desired special information.

  Hereinafter, an operation that is an alternative of the feature amount similarity comparison step ST153 performed by the feature amount similarity comparison unit 343 will be described. The feature amount similarity comparison unit 343 calculates the feature amount similarity calculation step ST152 without calculating the similarity for all the rectangle candidates instead of the maximum value determination in the feature amount similarity comparison step ST153. The calculation may be terminated when a rectangular candidate whose similarity exceeds a predetermined threshold appears, and the rectangular candidate may be determined as an appropriate rectangular candidate. Thereby, the comparison process can be performed at high speed.

<Rectangle detection result output step>
In the rectangle detection result output step ST16, the feature quantity similarity comparison unit 343 outputs the rectangle candidates determined in the feature quantity similarity comparison step ST145 as detection results of the rectangle detection apparatus 300. The method of outputting the detection result by the feature amount similarity comparison unit 343 may be in any format as long as the information can identify the detected rectangle.

Specifically, for example, the coordinate values of the four corners of the determined rectangle candidate in the input rectangle detection target image may be output as a detection result, or a function expressing the four sides of the determined rectangle candidate These coefficients may be output as detection results, or projective transformation coefficients for converting the four corner points of the rectangle candidates into predetermined rectangles (for example, rectangles) may be output as detection results. An image obtained by projective transformation of the determined rectangle candidate to a predetermined rectangle (for example, a rectangle) may be output as a detection result.
Further, after storing these pieces of information in the storage device, the storage position on the storage device may be output as a detection result.

[Second Embodiment]
In the second embodiment, a process variation of the feature amount similarity comparison step ST153 in the first embodiment will be described.
One processing example of the feature amount similarity comparison step ST153, each rectangle candidate 1 for · · · n, respectively similarities [rho ₁ was determined by the feature amount similarity calculation step ST152, the · · · [rho _n, The evaluation is performed in combination with the rectangle evaluation value used in the rectangle candidate calculation step ST13.

For example, the feature quantity similarity comparison unit 343 determines, as a suitable rectangle candidate, a rectangle candidate having the maximum corresponding rectangle evaluation value r _i for the rectangle candidate i whose similarity ρ _i exceeds a predetermined threshold. It may be. Also, the feature quantity similarity comparison unit 343 determines, as an appropriate rectangle candidate, the rectangle candidate having the maximum corresponding rectangle evaluation value r for the top T rectangle candidates from the highest similarity ρ _i. May be. Also, the feature quantity similarity comparison unit 343 determines the one having the maximum value of f (ρ _i , r _i ) obtained by applying a predetermined function to the similarity ρ _i and the rectangle evaluation value r _i as an appropriate rectangle candidate. You may make it do.
The f (ρ _i , r _i ) may be, for example, a weighted sum shown in Equation (30) or a weighted product shown in Equation (31).

Here, α and β are weighting parameters that indicate which of the similarity ρ _i and the rectangular evaluation value r _i is to be regarded as important. The feature amount similarity comparison unit 343 determines, for example, α> β when importance is attached to the similarity ρ _i and α <β when importance is attached to the rectangular evaluation value r _i . Also, the feature quantity similarity comparison unit 343 may determine that the difference or ratio between α and β is large when the difference in importance is larger, and experimentally improve the accuracy of rectangle detection. Find it.
In the case of weighted product, [rho _i, the range of possible values of r _i is naturally in the case of 0-1 is necessary to reverse the magnitude relationship of α and beta.

Further, the value of T means that the larger the value of T, the more important the value of the rectangular evaluation value r _i , and the smaller the value of T, the more important the value of the similarity ρ _i is determined. . The feature quantity similarity comparison unit 343 may set the value of T in a range between 1 and the number of rectangle candidates calculated in the rectangle candidate calculation step ST13.

  Another processing example of the feature quantity similarity comparison step ST153 is such that the feature quantity similarity comparison unit 343 sequentially inputs each frame as a rectangular detection target image with respect to a video composed of a plurality of continuous image frames. This is an example of a rectangular detection device.

In this case, the feature quantity similarity comparison unit 343 performs the similarity ρ _i ,..., Ρ _n obtained in the feature quantity similarity calculation step ST152 for each of the rectangle candidates 1,. The evaluation is performed in combination with the rectangle detection result for the past processing frame.
For example, the feature quantity similarity comparison unit 343 appropriately selects a rectangle candidate i having a similarity ρ _i exceeding a predetermined threshold value that is most similar to the rectangle of the rectangle detection result in the immediately preceding processing frame. You may make it determine as a rectangle candidate. The similarity with the rectangle of the rectangle detection result in the immediately preceding processing frame is obtained by, for example, comparing the feature amount of the rectangle as shown in Patent Document 3 with the rectangle of the rectangle detection result in the immediately preceding processing frame and the current processing frame. May be calculated for each of the rectangle candidates, and the feature amount similarity comparison unit 343 may determine the comparison by comparing the feature amounts.
<Patent Literature 3>: Japanese Patent Application Laid-Open No. 2008-117228 Rectangular tracking method and apparatus, program, and computer-readable recording medium

Also, the feature quantity similarity comparison unit 343 selects an appropriate rectangle candidate that is the most similar to the rectangle of the rectangle detection result in the immediately previous processing frame from the highest similarity ρ _i to the top T rectangle candidates. You may make it determine as. The feature amount similarity comparison unit 343 applies g (ρ _i , λ _i ) obtained by applying a predetermined function to the similarity evaluation value λ _i between the similarity ρ _i and the rectangle of the rectangle detection result in the immediately preceding processing frame. The one having the maximum value may be determined as an appropriate rectangle candidate.
It goes without saying that a weighted sum or a weighted product may be used as the predetermined function.
Furthermore, it goes without saying that these processing examples may be used in combination.

[Third Embodiment]
In the first embodiment, in the edge detection series calculation step of the rectangular pattern generation step ST3 and the feature amount calculation step ST2, the average values of the pixels inside and outside the edge are individually handled to generate the rectangular pattern and calculate the detection series. Although an example of performing is shown, a difference value between the inner and outer pixel values may be used as described below. This will be specifically described below.
In the rectangle adding device 100, first, in the feature amount calculation step ST2, a numerical sequence of length L = c (N ₁ + N ₂ + N ₃ + N ₄ ) is generated.
In the rectangular pattern generation step ST3, the rectangular pattern generation means 102 associates the numerical series with each position and channel on the four sides. The numerical sequences associated with Y, Cb, and Cr at the position x on the side are w ^[Y] (x), w ^[Cb] (x), and w ^[Cr] (x), respectively, shown below. And

Note that there is no distinction between the inside and outside of the side.
The rectangular pattern generation unit 102 determines values so that the pixel value of the inner point and the pixel value of the outer point at the position x on the side change with different signs. That is, the value is determined so as to change (correlate) in accordance with the values w ^[Y] (x), w ^[Cb] (x), and w ^[Cr] (x) of the numerical series at the position x.
The rectangular pattern generation means 102 outputs the pixel values Z _in ^[Y] (x), Z _in ^[Cb] (x), Z _in ^[Cr] (x) of Y, Cb, Cr of the inner point at the position x on the side. Are shown in the following formula (33).

The rectangular pattern generation unit 102 outputs the pixel values Z _out ^[Y] (x), Z _out ^[Cb] (x), and Z _out ^[Cr] (x) of the outer points Y, Cb, and Cr at the position x on the side. Are determined as shown in the following equation (34).

Note that the sign of the coefficient of w is reversed inside and outside. In addition, you may reverse positive / negative inside and outside.
The feature quantity calculation means 333 of the rectangle detection apparatus 300 obtains the detection series d as a difference between the average value of the inner points of the sides and the average value of the outer points in STEP 1433 (side detection series calculation step) of the feature quantity calculation step ST143. Like that. That is, the feature quantity calculation means 333 determines the detection series d as shown in the following formula (35).

  In addition, the feature amount calculation unit 333 may obtain the average value after obtaining the difference between the pixel values of the inner point and the outer point of each side for each division position. That is, the feature quantity calculation means 333 is defined as the following formula (36).

  Needless to say, when the sign of the coefficient of w is reversed in the rectangular pattern generation step ST3, the sign of the inner and outer values is reversed in the calculation of the detection sequence.

[Fourth Embodiment]
In the first embodiment described above, the detection sequence integration means 3334 integrates the detection sequences of the entire rectangle as in STEP 1434 (side detection sequence integration step) of the feature amount calculation step ST143, and then compares the feature amounts. An example was given. However, the present invention is not limited to this, and the detection sequence integration unit 3334 uses the detection sequence obtained by the side detection sequence calculation unit 3333 for each side of the rectangle candidate, and the feature amount similarity calculation unit 342 It is also possible to perform similarity calculation with the partial sequence of the side, obtain similarity for each side, and perform rectangular feature evaluation using this.
Further, for example, the calculation of the similarity of the feature amount for each side may be combined with the calculation of the similarity of the feature amount integrated in the entire rectangle.

Specifically, for example, if the calculation result of the feature similarity for each side includes a side that is below a predetermined threshold for the target rectangular candidate, the rectangular candidate is rejected and rejected. It is also possible to calculate the similarity of feature amounts integrated into the entire rectangle only for the remaining rectangle candidates, and perform feature amount similarity comparison.
In addition, here, the case where it represents that it is similar as the value of similarity is large was taken as an example. However, in the case of representing that it is similar as the value is small, it includes a side exceeding a predetermined threshold Needless to say, the rectangle candidate should be rejected.

  Here, the predetermined threshold value may be determined based on the same concept as the threshold value used in the feature amount similarity comparison step ST153.

[Fifth Embodiment]
In the fifth embodiment, variations of the color space used for the rectangular feature amount will be described.
In the feature amount calculation step ST143 of the first embodiment, the pixel value acquisition unit 332 acquires the pixel value in the Y, Cb, Cr color space as an example, but is used in the rectangular pattern generation step ST3. Other color spaces may be used as long as they are associated with the color space.
As an example, for example, in the rectangular pattern generation step ST3, a rectangular pattern is generated using the B channels of R, G, and B, and in the feature amount calculation step ST143, a rectangle is generated using the Cb channels of Y, Cb, and Cr. The feature amount may be calculated. This is because the B channel and the Cb channel have a large correlation with each other, and fluctuations given to the B channel can be easily observed even in the Cb channel.

[Sixth Embodiment]
In the sixth embodiment, a plurality of rectangles are added in a multiplex manner, each rectangle is detected as a candidate, and the feature amounts for each rectangle are integrated as the feature amounts for the entire multiple rectangles, so that the target rectangular region can be more accurately obtained. An example of detection will be shown.
In the rectangle adding apparatus 100 of the present embodiment, the feature amount generation step ST2 and the rectangular pattern generation step ST3 in the rectangle adding apparatus 100 according to the first embodiment are each executed a plurality of times, and in the rectangle adding step ST4, the size is slightly increased. Multiple different rectangles are drawn.

FIG. 19 shows an example of a rectangular additional image in which a plurality of rectangles are drawn in a multiplexed manner. FIG. 19 is a diagram illustrating an example of a rectangular additional image in which a plurality of rectangles are drawn in a multiplexed manner.
The rectangle addition image 1901 in FIG. 19 shows an example in which rectangles are added in triplicate, and each rectangle has a predetermined feature.
In the rectangle detection apparatus 300 according to the present embodiment, after the rectangle candidate calculation step ST13 in the rectangle detection apparatus 300 according to the first embodiment, a set of rectangles that are close to each other is determined from among a plurality of rectangle candidates, and multiplexed. A rectangle candidate.

Here, “exists in the vicinity of each other” means
The distance between the centroids of the four vertices of the rectangle is less than a predetermined threshold. The condition is that the ratio of the lengths of the four sides of the rectangle is within a predetermined value range, or the difference is less than the predetermined threshold. Judgment may be made.

  Here, the threshold for the distance between the centroids may be set to a value of 1 to 5% of the length of one side of the rectangular detection target image, for example, according to the image size of the rectangular detection target image. Moreover, the range of the value with respect to the ratio of the lengths of the four sides of the rectangle may be, for example, a range of 0.9 to 1.1 (± 10%). The threshold for the length of the four sides of the rectangle may be set to a value of 1 to 5% of the length of one side of the rectangle detection target image, for example, according to the image size of the rectangle detection target image. In addition, it is not limited to the range of the value mentioned in these examples, For example, it is good also considering the range of the value with respect to the ratio of the length of four sides of a rectangle as the range (± 20%) of 0.8-1.2. Needless to say.

  It may be determined as follows whether or not they are close to each other, or alternatively, in the edge candidate detection step ST12, using edges that are continuously detected in the vertical and horizontal directions.

STEP1201 ':
The sides detected in the upper, lower, left, and right directions are arranged in the order of detection, and E ^upper , E ^lower , E ^left , and E ^right are as shown in Expression (37).

However, n- _up , n- _down , n- _left , and n- _right represent the number of sides detected in each of the vertical and horizontal directions.

STEP1202 ':
According to the multiplicity k of the rectangle added by the rectangle adding device, m continuous sides are extracted in each of the upper, lower, left and right directions, and are ^defined as C ^upper , C ^lower , C ^left , C ^right .
These, ^on C, C ^under, C ^left, for C ^right, formula (38) holds.

STEP1203 ':
K rectangles R ₁ , R ₂ ,..., R _k composed of vertically and horizontally aligned edges are formed, and these sets are candidates for multiple rectangles.

STEP 1204 ':
In the rectangular detector of the present embodiment that repeats STEPs 1202 ′ to 1203 ′ for all combinations that match the detected edge conditions, that is, any combination of i, j, k, and l, in the rectangular detector 300 of the first embodiment. In the rectangular feature amount calculation step ST14, a rectangular feature amount is calculated for each rectangle of the obtained multiple rectangle candidates, and k types of detection sequences obtained as feature amounts are arranged and output as detection sequences for the multiple rectangles. In the quantity evaluation step ST15, the target multiple rectangle is determined by comparing the similarity between the entire detection sequence for the multiple rectangle and the predetermined feature value given by the rectangle adding device 100.

[Seventh Embodiment]
In the first embodiment, each side of the rectangle is divided into N ₁ , N ₂ , N ₃ , and N ₄ , and projected in STEP 1431 (conversion parameter calculation step) and STEP 1432 (division position calculation step) in the feature amount calculation step ST143. Although the configuration is such that the edge coordinates on the side are converted to the coordinates representing the division position of the rectangle using the conversion parameter, the following conversion can also be performed in order to simplify the processing.

From the vertex coordinates of the rectangle candidate, the vertices at both ends of the side candidate may be obtained, and the division position may be obtained from the ratio of the distance from one vertex to the edge coordinates on the side and the distance between the vertex coordinates.
That is, when the coordinates of the vertices at both ends of the rectangle candidate are x ₁ , x ₂ (where x ₁ <x ₂ ), the edge coordinates on the side are x _e , and the number of divisions of the side is N, the division position n (1 ≦ n ≦ N) may be obtained by the following formula (40).

  However, the term shown in the numerical value (41) represents the maximum integer value not exceeding x.

  | X−y | represents a distance between the coordinates x and y.

Further, when the number of divisions of the side is 2, the edge tracking start point detected in STEP 1204 of the side candidate detection step ST12 is used as a boundary to determine which of the two edge coordinate division positions is on the side. You may judge. At this time, since the search start point in STEP 1201 of the edge candidate detection step ST12 is near the expected center of the rectangle, the position that roughly bisects the edge is the edge tracking start point. Further, a line connecting the midpoints of the upper and lower outer edges and the left and right outer edges of the entire rectangular detection target image may be determined as a boundary.
By determining the division position by these conversions, it is possible to calculate the division position at a higher speed although there is a slight error.

[Eighth Embodiment]
In the feature amount calculation step ST143 of the first embodiment, instead of directly obtaining the detection series d from the average value of the pixel values at each division position, a frequency-converted pixel value at each division position is used as the detection series. It may be.

For example, Fourier transform or discrete cosine transform may be used as the frequency transform.
In this case, the rectangular additional device 100 in a rectangular pattern generating step ^{_{ST3, w in [Y] (}} x), w in [Cb] (x), w in [Cr] (x), w out [Y] (x) , W _out ^[Cb] (x), w _out ^[Cr] (x), the frequency coefficient value series ξ _in ^[Y] (w), ξ _in ^[Cb] (w), ξ _in ^{[ Cr]} (w), ξ _out ^[Y] (w), ξ _out ^[Cb] (w), and ξ _out ^[Cr] (w) are obtained, and a rectangular pattern is generated after inverse frequency conversion at frequency w. .

Here, F ⁻¹ [] represents, for example, an inverse Fourier transform.
In the feature quantity calculation step ST143, the rectangle detection apparatus 300 obtains a detection sequence after frequency-converting the following (43) at the position i.

  One example is shown in (44) below.

Here, F [] represents, for example, Fourier transform.
When defining the feature quantity as a series in the frequency domain, it is possible to avoid a frequency band (for example, a low frequency component or a high frequency component) in which image noise is likely to occur, and use only a medium frequency component.

[Ninth Embodiment]
In the first embodiment, in the determination of the edge component in the rectangle detection apparatus 300, the rectangular pattern generation unit 102 of the rectangle adding apparatus 100 determines whether the reference value between the inside and outside of the edge exceeds the threshold value determined to have the edge component. A rectangular pattern was generated with a difference between
On the other hand, in the case where the inner position to which the feature graphic of the rectangle addition target image C11 is added has a sufficiently high luminance or a low luminance, the rectangular pattern generation means 102 determines the reference values inside and outside the edge. The configuration may be set independently without considering the difference. That is, when the rectangular pattern generating unit 102 of the rectangular adding device 100 generates a rectangular pattern and the rectangular adding unit 103 adds the rectangular pattern to the rectangular addition target image C11, an edge is generated with the pixel value component of the rectangular addition target image C11 itself. May be configured.

For example, the case where the inner position to which the feature graphic of the rectangular addition target image C11 is added has sufficiently high luminance will be described. First, in the rectangular pattern generation step ST3, the rectangular pattern generation unit 102 sets the inner luminance reference value to 0 and the outer luminance reference value to a small value as, for example, C _in ^{[ Y]} = 0 and C _out ^[Y] = 10 are set so that the pixel value of the rectangular pattern can take a negative value.

  Further, in STEP 405 of the rectangle addition step ST4, when the coordinates of the rectangle addition image C12 are inside the rectangle side, the pixel value acquisition unit 332 of the rectangle addition unit 103 sets the pixel value of the rectangle addition target image C11 to the pixel value. The pixel values of the rectangular additional image are determined by adding the pixel values in the rectangular pattern. When the coordinate of the rectangular additional image is outside the rectangular side, the pixel value acquiring unit 332 of the rectangular adding unit 103 outputs the value of the pixel in the rectangular pattern.

Further, for example, when the inner position to which the feature graphic of the rectangle addition target image C11 is added has a sufficiently low luminance, the rectangular pattern generation unit 102 conversely increases the reference value for the inner and outer edges, and the reference value for the outer luminance is large. For example, C _in ^[Y] = 0 and C _out ^[Y] = 240 may be set. Needless to say, the color components other than the luminance may be used.
By doing in this way, an edge can be comprised by the pixel value component of rectangle addition object image C11 itself, and the rectangle to draw can be made more inconspicuous.

[Tenth embodiment]
In STEP 1433 (edge detection series calculation step) of feature amount calculation step ST143 in the rectangular detection apparatus 300 according to the first embodiment, instead of obtaining an average value of pixel values for each division position, STEP 1431 (conversion) of feature amount calculation step ST143 is performed. Parameters for reverse projection transformation are obtained in parameter calculation step), coordinates in the rectangular detection target image space corresponding to the coordinates of the division position are obtained in STEP 1432 (division position calculation step), and obtained in STEP 1433 (side detection series calculation step). The pixel value corresponding to the division position may be determined by performing complementary calculation from the pixel value of the edge detection point having the edge coordinates in the vicinity of the coordinate.

[Eleventh embodiment]
In the first embodiment, an example has been shown for a rectangle having four sides, but the same can be applied to other graphic shapes.
For example, when detecting a line segment of only one side, the rectangular pattern generation unit 102 is set based on the feature amount generated in the same procedure as the feature amount generation unit 101 in the line segment addition device similar to the rectangle addition device 100. A line segment detection device similar to the rectangle detection device 300, which generates a line segment pattern using line segment pattern generation means applied only to a line segment on one side, draws a line segment having a feature in the target image, and In this case, the rectangular candidate calculation step ST13 is omitted, the feature amount is directly calculated from the edge candidate, and the similarity with the predetermined feature amount is calculated and compared. At this time, the division position of the side can be determined by the ratio of the distance from both end points of the side, and the positions inside and outside the side are the vertical and horizontal directions in the target image and the center point of the image. It is possible to define a predetermined point and use a side close to the point and a side far from the point.

  Further, for example, even when a polygon having N sides is detected, a polygon having a feature in the target image using a polygon pattern generation unit in which the rectangular pattern generation unit 102 is applied to the polygon is used. In a polygon detection device similar to the rectangle detection device 300, a polygon candidate is extracted, a feature amount is calculated, and a similarity with a predetermined feature amount is calculated and compared. Can respond.

Further, for example, even when a circular shape is detected, a circular pattern generating unit in which the rectangular pattern generating unit 102 is applied to a circular shape is used to draw a circular shape having a feature in the target image, and the rectangular detecting device In the circular detection device similar to 300, circular candidates as edges are extracted instead of edge candidate detection, feature quantities are calculated for these, and similarities with predetermined feature quantities are calculated and compared. Can respond.
At this time, the division positions around the circle can be determined by, for example, the angle from the center of the circle, and the extraction of the circular candidate can be performed by applying the technique of Reference 2.

In addition, for example, even when detecting an arbitrary closed curve, a figure having a feature in the target image is drawn by using a figure pattern generation unit in which the rectangular pattern generation unit 102 is applied to the closed curve, thereby detecting a rectangle. A graphic detection device similar to the device 300 can cope with this by extracting a graphic candidate, calculating a feature value for the graphic candidate, and calculating and comparing a similarity with a predetermined feature value. At this time, the division position around the figure can be determined by, for example, the angle from the center of gravity of the figure. Similarly, an arbitrary curve can be dealt with by determining the division position according to the ratio of the distance from both end points of the curve, and the positions inside and outside the curve are the vertical and horizontal directions in the target image. It is possible to define a predetermined point such as the center point of the image and use the side closer to the point and the side far from the point.
As described above, the present invention can also be applied to a figure adding apparatus, a figure adding method, a figure detecting apparatus, and a figure detecting method for an arbitrary figure shape.

[Twelfth embodiment]
Other embodiments are described below.
The rectangular addition target image that is an input of the rectangular adding device 100 and the rectangular detection target image that is an input of the rectangular detection device 300 may be images including a plurality of frame images.
FIG. 1 illustrates an example in which the rectangular detection device 300 operates by being incorporated in a portable terminal. However, the rectangular detection device 300 may be mounted on another terminal such as a digital camera, a personal computer, or a server. Good.

The lengths of the sides used in each embodiment, the distance between the vertices of the rectangle, the distance between the end points of the side candidates, the distance between the centroids of the rectangle, the distance between the coordinates, and the like may be obtained by the Euclidean distance. However, other distance calculation methods may be used.
For example, a city distance or a chess board distance may be used in order to perform calculation processing at high speed.

The effects obtained by the present invention will be described below.
By adding a figure to an image using the figure adding apparatus of the present invention and detecting the target figure using the figure detecting apparatus of the present invention, a similar figure (for example, a rectangle other than the target figure region) is detected. Even when a similar rectangular area) is present when detecting, or when a straight line having a strong edge component is present in the vicinity of the target graphic, erroneous detection of the graphic area can be prevented. Moreover, the target figure can be detected with high accuracy.

  Further, even in an application where a digital watermark is detected from a detected graphic area, it is not necessary to try digital watermark detection a plurality of times, and high-speed detection processing can be realized.

  In addition, a predetermined feature amount is assigned using a pseudo-random number as in the feature amount generation step ST2, and detection is performed using a correlation with a detection sequence as in the feature amount similarity comparison step ST153. Even with fluctuations in the external environment, such as noise, ambient light, and lighting conditions during shooting, as in the case of detection target images shot using a low-performance camera such as a camera, It is possible to detect a figure robustly.

  Further, as in the pixel value acquisition step ST142, the pixel values inside and outside the edge are separately acquired, and this is used as a detection sequence, so that the detection sequence is simply compared with the case of acquiring the pixel value on the side. Can be doubled, and more robust similarity comparisons can be made, resulting in robust figure detection. In addition, since robust detection is possible, the weight value at the time of determining the pixel value in the rectangular pattern generation step ST3 can be reduced, and as a result, the fluctuation of the pixel value in the figure to be added is reduced. It is small and the feature added to the figure to add can be made not visually conspicuous. Further, by using α blend in the rectangle adding step ST4, the figure itself can be made visually inconspicuous.

  Further, in the edge tracking step ST141, when the pixel values inside and outside the edge are acquired, the edge coordinates can be accurately detected by performing the edge tracking method of the present invention. As a result, a more robust figure can be detected.

  Further, by performing the feature amount similarity comparison process as shown in the second embodiment, a more appropriate figure combined with the rectangular evaluation value can be detected, and the figure is detected from a video composed of a plurality of continuous image frames. In this case, the target graphic can be detected more stably.

  Further, by using the difference value between the inner and outer pixel values of the edge as shown in the third embodiment, local low-frequency noise such as illumination fluctuation is added when an image to be detected as a graphic is captured. In this case, the noise appears as a common noise on the inner side and the outer side and is canceled by the difference. As a result, more robust figure detection can be performed.

  Further, by combining the calculation of the similarity of the feature quantity for each side as shown in the fourth embodiment and the calculation of the similarity of the feature quantity integrated in the entire rectangle, for example, as shown in the right diagram of FIG. Even when a false straight line component is included in the vicinity of only one side of the rectangle and a large difference in similarity does not occur in the feature amount of the entire rectangle, a rectangle candidate including the false straight line component can be excluded. At the same time, it is possible to reduce the calculation of the feature amount comparison for the entire rectangle candidate, so that it is possible to detect a figure with higher speed and higher accuracy.

  In addition, by generating a rectangular pattern using a color space as shown in the fifth embodiment and calculating a feature amount, unnecessary color conversion is omitted by a color space suitable for a shooting environment at the time of detecting a figure. Can be detected at high speed.

  In addition, by using a multiple figure as shown in the sixth embodiment, a target figure area can be detected with higher accuracy.

  Further, by calculating the division positions as shown in the seventh embodiment, the calculation of projective transformation parameters can be omitted, and higher-speed graphic detection is possible.

  Further, by using a sequence value in the frequency domain as shown in the eighth embodiment as a feature quantity, it is possible to perform a feature quantity calculation while avoiding the influence of frequency components that are likely to be subjected to noise, and to calculate a division position. Thus, even if a slight positional deviation occurs, a frequency component that does not increase in phase can be used, and as a result, more robust figure detection can be performed.

  Further, by adding a feature graphic as shown in the ninth embodiment, it is possible to make the drawn graphic less noticeable.

Note that the feature graphic addition device (rectangular addition device) 100, the image display device 200, the feature graphic detection device (rectangular detection device) 300, and the portable terminal 400 in the above-described image detection system may have a computer system therein. Good. The operation process of these devices may be stored in a computer-readable recording medium in the form of a program, and the above-described processing may be performed by the computer system reading and executing this program. . The “computer system” herein includes a CPU, various memories, an OS, and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.

Further, a program for realizing each step is recorded on a computer-readable recording medium, and the functions of the rectangle adding device 100, the image display device 200, the rectangle detecting device 300, and the portable terminal 400 shown in FIG. 1 are realized. A process for calculating an estimated value of the shape information of the detection object by recording the program for recording on a computer-readable recording medium, causing the computer system to read and execute the program recorded on the recording medium You may go.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  DESCRIPTION OF SYMBOLS 100 ... Rectangular addition apparatus, 101 ... Feature-value production | generation means, 111 ... Embedded series production | generation means, 102 ... Rectangular pattern production | generation means, 103 ... Rectangular addition means, 104 ... Memory | storage part, DESCRIPTION OF SYMBOLS 200 ... Image display apparatus, 300 ... Rectangle detection apparatus, 301 ... Edge candidate detection means, 302 ... Rectangular candidate calculation means, 303 ... Rectangular feature-value calculation means, 331 ... Edge tracking Means, 332... Pixel value acquisition means, 333... Feature amount calculation means, 3331... Conversion parameter calculation means, 3332 ... division position calculation means, 3333 ... edge detection series calculation means, 3334. ..Detection series integration means 304 ... rectangular feature quantity evaluation means 341 ... embedded feature quantity calculation means 342 ... feature quantity similarity calculation means 343 ... feature quantity similarity comparison means

Claims (10)

In a feature graphic addition method for inputting target image data, which is a target to which a feature graphic is added, and adding the feature graphic to the target image data in order to detect the feature graphic from an image provided with the feature graphic.
Feature quantity generation that is determined in advance according to a graphic to be added to the target image data, determines a feature quantity having a different pixel value according to a position area in the graphic, and outputs feature quantity information indicating the feature quantity Steps,
A feature figure pattern generating step for generating a feature figure pattern representing a feature figure by a pixel value including the predetermined feature quantity based on the input feature quantity information, and outputting information indicating the feature figure pattern;
A rectangular addition step of creating a feature graphic additional image by adding the characteristic graphic pattern to the target image data, and outputting image data of the characteristic graphic additional image;
A method of adding a feature graphic. In the feature quantity generation step,
2. The feature figure addition method according to claim 1, wherein the feature amount is determined by a series of a plurality of numerical values indicating features that are predetermined according to positions in the feature figure. The feature quantity generation step and the feature figure pattern generation step are performed a plurality of times to generate a plurality of feature figure patterns,
In the rectangle adding step,
A feature graphic additional image is created by assigning the plurality of characteristic graphic patterns to the target image data within a proximity range that is determined in advance as a range in which a graphic is recognized as being close to each other. The method of adding a characteristic figure according to claim 1 or 2. In the feature graphic detection method of inputting feature graphic detection target image data of an image to which a feature graphic is added, and detecting the feature graphic from the feature graphic detection target image data,
An edge detection step of detecting an edge candidate area corresponding to the edge of the figure from the feature figure detection target image data;
A pixel value acquisition step of acquiring pixel values of pixels included in an edge vicinity region adjacent to the edge candidate region and the edge candidate region;
Based on the acquired pixel value, a feature amount calculating step for obtaining a feature amount of a graphic indicated by the edge candidate region;
An evaluation value indicating a similarity between a predetermined feature quantity of the feature graphic and a feature quantity of the graphic indicated by the edge candidate area is calculated, and the graphic indicated by the edge candidate area and the characteristic graphic are A feature amount evaluation step of determining whether or not they are similar based on the evaluation value. In the pixel value acquisition step,
A pixel at a position determined in advance according to the pixel value of the inner edge and the outer edge of the frame-shaped figure indicated by the edge candidate area, or the position in the figure indicated by the edge candidate area 5. The feature graphic detection method according to claim 4, wherein a value is acquired. In the feature amount calculating step,
A pixel at a position determined in advance according to the pixel value of the inner edge and the outer edge of the frame-shaped figure indicated by the edge candidate area, or the position in the figure indicated by the edge candidate area Based on the value, obtain a detection sequence including a plurality of numerical values indicating the features of the graphic indicated by the edge candidate region,
In the feature amount evaluation step,
As a predetermined feature amount of the feature graphic, a similarity between a numerical sequence including a plurality of numerical values indicating a predetermined feature according to a position in the feature graphic and the detection sequence is expressed by a correlation value. The feature graphic detection method according to claim 5, wherein the evaluation value is expressed and expressed. In the feature amount calculating step,
For a plurality of figures indicated by the edge candidate area existing in the proximity range that is determined in advance as a range in which the figures are considered to be close to each other, respectively, obtain a feature amount based on the acquired pixel value,
In the feature amount evaluation step,
The evaluation value indicating the similarity between the feature amount of the plurality of feature figures corresponding to a predetermined combination and the feature amount of the plurality of figures indicated by the edge candidate region is calculated. The characteristic figure detection method according to any one of 4 to 6. In a feature graphic adding apparatus that inputs target image data to which a feature graphic is added, and adds the feature graphic to the target image data in order to detect the feature graphic from an image to which the feature graphic is attached.
Feature quantity generation that is determined in advance according to a graphic to be added to the target image data, determines a feature quantity having a different pixel value according to a position area in the graphic, and outputs feature quantity information indicating the feature quantity Means,
A feature figure pattern generating means for generating a feature figure pattern representing a feature figure having the predetermined feature quantity based on the input feature quantity information and outputting information indicating the feature figure pattern;
A feature figure addition apparatus comprising: a rectangle addition unit that assigns the feature figure pattern to the target image data to create a feature figure addition image and outputs image data of the feature figure addition image. In the feature graphic detection apparatus for inputting the feature graphic detection target image data of the image to which the feature graphic is added, and detecting the feature graphic from the feature graphic detection target image data,
Edge detection means for detecting edge candidate regions corresponding to the edges of the figure from the feature figure detection target image data;
Pixel value acquisition means for acquiring pixel values of pixels included in the edge vicinity region adjacent to the edge candidate region and the edge candidate region;
Based on the acquired pixel value, a feature amount calculating means for obtaining a feature amount of a graphic indicated by the edge candidate region;
An evaluation value indicating similarity between a predetermined feature amount of the feature graphic and a feature value of the graphic indicated by the edge candidate area is calculated, and the graphic indicated by the edge candidate area is similar to the characteristic graphic Feature amount evaluation means for determining whether or not to perform based on the evaluation value;
A feature graphic detection device comprising: The program for making a computer perform the step as described in any one of the characteristic figure addition method of Claims 1-3, or the characteristic figure detection method of Claims 4-7.

Images