JP2021157537A - Two-dimensional marker recognition apparatus, method, program, and system - Google Patents

Abstract

To provide a method and apparatus for preventing false recognition when a distant two-dimensional marker is recognized remotely.SOLUTION: A two-dimensional marker recognition apparatus 1000 includes: an image acquisition unit 200 which acquires an image from a camera device; and a two-dimensional marker recognition unit 400 which recognizes, as a two-dimensional marker, a rectangular area image formed of a peripheral single color area and a feature quantity display area, from the image. The two-dimensional marker recognition unit includes a false recognition prevention unit 410 which determines by a specific determination unit in the two-dimensional marker recognition unit, whether the area image corresponds to a two-dimensional marker.SELECTED DRAWING: Figure 1

Description

The present invention relates to a two-dimensional marker recognition device, method, program and system that contributes to reducing the false recognition rate of the two-dimensional marker.

In recent years, as one of the methods for remotely detecting an article at a distant position, a two-dimensional marker attached to the article is detected from an image acquired by an imaging device, and the article is individually identified from the feature amount of the two-dimensional marker. Equipment, methods, programs, systems, etc. are provided.

Here, the two-dimensional marker is, for example, placed on or printed on a box, a machine, a device, a medicine, a ticket, a pallet, or a container loaded on a pallet, which is an article to be identified. It is a thing. Commonly used types of two-dimensional markers include standards such as PDF417 with one-dimensional barcodes stacked, QR code (registered trademark), ArUco marker, or chameleon code (registered trademark). Individual identification of a two-dimensional marker is performed by associating it with a feature amount represented by a color pattern in a two-dimensional rectangle specified in each standard. Here, the two-dimensional rectangle is often a square or a rectangle. One of the display features of the 2D marker is that it has a fixed width single color (often white) area width on the outer edge of the 2D rectangle, and a color (often) inside the area width. It has a large and small black-and-white rectangular) area (hereinafter referred to as a feature amount display area). The feature amount of the two-dimensional marker is represented as a numerical value, a character string, and a URL (Uniform Resource Locator), which can be obtained from the feature amount display area according to each standard of the two-dimensional marker.

According to Non-Patent Document 1, for the ArUco marker, which is one of the two-dimensional markers, in addition to being able to acquire the feature amount of the article, it is possible to obtain information such as the size and distortion of the two-dimensional code from the image, and to use an imaging device. It is shown that the relative distance to the two-dimensional marker and the orientation (orientation direction in the three-dimensional space) can be estimated. The ArUco markers (large and small black and white rectangles) displayed in the feature amount display area are created from a user-specified dictionary, and the corresponding feature amount is acquired from the dictionary by inputting the image of this feature amount display area. be able to.

According to Patent Document 1, in order to improve the accuracy of the detection position and orientation of the two-dimensional marker, a certain hidden part (for example, hidden by a human hand) is hidden in the two-dimensional marker in the image acquired from the imaging device. If there is a part that is determined to have been removed), the two-dimensional marker may be erroneously recognized. Therefore, a technique relating to not making the two-dimensional marker a recognition target is disclosed.

According to Patent Document 2, in order to detect that the barcode symbol (feature amount display area) is continuous for a certain section, a recognition area and an area adjacent to the recognition area are acquired from the image, and the correlation between them is determined. Techniques have been disclosed that calculate and improve the accuracy of barcode detection (by checking the continuity of barcode symbols).

Japanese Unexamined Patent Publication No. 2005-293141 Japanese Unexamined Patent Publication No. 2014-199487 Japanese Unexamined Patent Publication No. 2011-0843484

OpenCV: “Detection of ArUco Markers” Internet Search Results (February 20, 2020) https://docs.opencv.org/trunk/d5/dae/tutorial_aruco_detection.html

The disclosure of the above prior art documents shall be incorporated into this document by citation. The following analysis was made by the present inventors.

According to Patent Documents 1 and 2, it is required that the two-dimensional markers are sufficiently close to each other so that they can be recognized, or that there is a means for bringing them close to each other. According to Patent Document 1, a method is used in which an object on which a two-dimensional marker is displayed is brought close to the object so as to be lifted and operated by a human hand. Further, according to Patent Document 2, a method is used in which a barcode reader placed in a store or the like scans a barcode-printed card.

On the other hand, in the case of remotely detecting an article at a distant position, Patent Documents 1 and 2 do not always correspond accurately. This is because in this case it is not possible to move the article or bring the camera device close to it. In addition, Non-Patent Document 1 does not sufficiently describe remote detection. For example, in a large warehouse, a large number of boxes (for example, 100 in the vertical direction, 100 in the horizontal direction, and 1 cubic meter sparsely installed in the plane direction) are installed, and a two-dimensional marker is placed on the surface of the boxes. Let us consider a case where this is detected by using an image of a camera device installed at a long distance (for example, installed 100 meters away). At this time, there is a possibility that a marker that is not a two-dimensional marker at a distant position is accidentally recognized as a two-dimensional marker. For example, consider a case where a background screen, such as a checkerboard pattern on a warehouse wall, accidentally matches the features of a two-dimensional marker. In this way, when something that is supposed to be a feature amount display area is reflected in the image, this may be the recognition target. In this case, it will be erroneously recognized that there is a management target at that location (the position of the checkerboard pattern in the background of the warehouse). Alternatively, even if a two-dimensional marker at a distant position actually exists, it may be possible to recognize it by increasing the magnification of the imaging camera, but the image is unclear. There is a possibility that it will be judged as an incorrect target. In such a case, Non-Patent Document 1 does not describe how to determine the object to be recognized.

In Patent Document 3, remote detection is an issue, but here, after identifying a two-dimensional marker position candidate from the entire image, a human (worker) approaches the position candidate and further captures an image. This is a system and method for accurate identification. As described above, although Patent Document 3 requires the work of a human (worker), if this is to be operated as a system without human intervention, it is a special case with a high degree of freedom of movement and rotation. An imaging camera is required. Alternatively, it may be necessary to provide guardrails to move the cameras inside the warehouse. Therefore, Patent Document 3 has a problem that it costs a lot to deal with it.

The present invention has been made in view of the above problems, and addresses the problem of remotely recognizing a two-dimensional marker at a distant position. More specifically, an object of the present invention is to contribute to reduction of the possibility of erroneous recognition (reduction of erroneous recognition rate) when recognizing a two-dimensional marker at a distant position from a distance.

According to the first viewpoint of the present invention, a two-dimensional image acquisition unit that acquires an image from a camera device, and a rectangular area image composed of a peripheral single color area and a feature amount display area from the image. A two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a marker, and whether or not the area image corresponds to the two-dimensional marker in the two-dimensional marker recognition unit is specified. Provided is a two-dimensional marker recognition device, characterized in that it has a false recognition suppression unit, which is determined by a determination unit.

According to the second viewpoint of the present invention, a rectangular area image composed of an image acquisition unit that acquires an image from a camera device and a peripheral single color area and a feature amount display area from the image is two-dimensionally formed. A two-dimensional marker recognition method in a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a marker, wherein the area image corresponds to the two-dimensional marker in the two-dimensional marker recognition unit. Provided is a two-dimensional marker recognition method, characterized in that whether or not it is determined to be erroneous recognition by a specific determination unit.

According to the third viewpoint of the present invention, a two-dimensional image acquisition unit that acquires an image from a camera device, and a rectangular area image composed of a peripheral single color area and a feature amount display area from the image. A computer program installed in a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a marker, and the area image corresponds to the two-dimensional marker in the two-dimensional marker recognition unit. Provided is a two-dimensional marker recognition program, which comprises means for determining whether or not a false recognition is made by a specific determination unit.
Note that this program can be recorded on a computer-readable storage medium. The storage medium may be a non-transient such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. The present invention can also be embodied as a computer program product.

According to the fourth aspect of the present invention, a two-dimensional marker recognition system is provided. This two-dimensional marker recognition system has an image acquisition unit that acquires an image from a camera device, and an image acquisition unit.
A two-dimensional marker having a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the above image. It is a recognition system
In the two-dimensional marker recognition unit of the two-dimensional marker recognition device, there is an erroneous recognition suppression unit that determines whether or not the region image corresponds to the two-dimensional marker by a specific determination unit.

According to each viewpoint of the present invention, there are provided two-dimensional marker recognition devices, methods, programs and systems that contribute to reducing false recognition of two-dimensional markers at distant positions.

It is a figure which shows the whole structure which concerns on one Embodiment of this invention. It is a figure which shows the structure of the erroneous recognition suppression part 410 in one Embodiment of this invention. It is a figure which shows the hardware structure of the 2D marker recognition apparatus in one Embodiment of this invention. It is a flowchart which shows the processing example which performs the false recognition suppression by the plurality of image determination in the 1st Embodiment example of this invention. It is a flowchart which shows the processing example which performs the erroneous recognition suppression by the shortest side determination part in the 1st Embodiment example of this invention. It is a flowchart which shows the processing example which performs the false recognition suppression by the area circumference determination part in the 1st Embodiment example of this invention. It is a flowchart which shows the processing example which performs the false recognition suppression by the identification information determination unit in the 1st Embodiment example of this invention. It is a flowchart which shows the processing example which performs the false recognition suppression by the posture determination part in the 1st Embodiment example of this invention. It is a flowchart which shows the processing example which performs the erroneous recognition suppression by the position determination part in the 1st Embodiment example of this invention. It is a figure explaining the example of performing a plurality of image determination. It is a figure explaining the example of performing the shortest edge determination. It is a figure explaining the example of performing the peripheral area determination. It is a figure explaining the example of performing a posture information determination. It is a figure explaining concretely about the case which performs the posture information determination. It is a figure explaining the case which performs the position information determination. It is a figure which shows the example of the correlation between the threshold value of the shortest side of a region, and the number of false recognitions / the number of omissions of detection. It is a flowchart which shows the operation of the 2D marker recognition apparatus in 2nd Embodiment of this invention. It is a figure which shows the execution example of the false recognition suppression by the 2D marker recognition apparatus. It is a figure explaining the modification [2-2] in the 2nd Example of this invention.

First, an outline of one embodiment will be described. It should be noted that the drawing reference reference numerals added to this outline are added to each element for convenience as an example for assisting understanding, and the description of this outline is not intended to limit anything. Further, the connection line between the blocks in each block diagram includes both bidirectional and unidirectional. The one-way arrow schematically shows the flow of the main signal (data), and does not exclude interactivity.

As shown in FIG. 1, the two-dimensional marker recognition device 1000 according to the embodiment includes an image acquisition unit 200 that captures an image of a two-dimensional marker (group) 100 stretched / printed on an article, and the image acquisition unit 200. Recognizes a storage unit 300 that stores the captured image of the two-dimensional marker (group) 100 acquired by the user, and a numerical value, a character string, a URL, etc., which are feature quantities representing the article, from the stored two-dimensional marker captured image. Includes a two-dimensional marker recognition unit 400. The two-dimensional marker recognition unit 400 includes a false recognition suppression unit 410, a feature amount extraction unit 420, and a position / orientation estimation unit 430 as its partial constituent units.

The image acquisition unit 200 acquires an image by, for example, one or more image acquisition units 200 that photograph a two-dimensional marker 100 stretched on the side surface or the upper surface of an article which is a corrugated cardboard box (group) installed in a warehouse. Of course, it is not intended to limit the article on which the two-dimensional marker 100 is placed / printed to a cardboard box. The article may be a machine, device, ticket, pallet, or container loaded on the pallet, or a combination thereof.

Although the number of image acquisition units 200 is one with reference to FIG. 1, the number of image acquisition units 200 is not limited to one. It is assumed that the images of the two-dimensional markers 100 can be acquired from one or more camera devices installed at distant positions.

FIG. 2 shows the configuration of the false recognition suppression unit 410, which shows one of the technical features of the present invention. The erroneous recognition suppression unit 410 includes the following seven determination units for suppressing (determining) erroneous recognition of the two-dimensional marker 100. The erroneous recognition suppressing unit 410 can be composed of seven determination units individually or arbitrarily in combination.
(1) Multiple image determination unit 411, which determines based on the recognition rate of the two-dimensional marker 100 when a plurality of images are used.
(2) The shortest side determination unit 412, which determines based on the length of the shortest side in the rectangular image of the captured image of the captured two-dimensional marker 100.
(3) Area peripheral determination unit 413, which determines based on the width of the monochromatic area recognized from the captured image of the captured two-dimensional marker 100.
(4) Identification information determination unit 414, which determines based on whether or not the feature amount recognized from the captured image of the two-dimensional marker 100 is included in the feature amount list designated in advance.
(5) Posture determination unit 415, which determines based on the difference between the reference three-dimensional orientation orientation specified in advance and the three-dimensional orientation orientation recognized from the captured image of the two-dimensional marker 100.
(6) Position determination unit 416, which determines based on whether or not the two-dimensional marker 100 can exist in the three-dimensional position region where the two-dimensional marker specified in advance can exist.
(7) Clustering determination unit 417 that determines based on the distance from the cluster center position when the posture / position of the two-dimensional marker using a plurality of images is subjected to clustering statistical processing.

[Hardware configuration]
The hardware configuration of the two-dimensional marker recognition device 1000 according to the present invention will be described with reference to FIG. The two-dimensional marker recognition device 1000 includes one or more camera devices (1101, 1102, 1103, hereinafter collectively referred to as 1100) that image the two-dimensional marker (group) 100, which corresponds to the image acquisition unit 200. Auxiliary storage device 1200 corresponding to unit 300 and storing a two-dimensional marker image, and an image area estimated to be a two-dimensional marker 100 to be recognized handed over from the auxiliary storage device 1200 and two-dimensional marker recognition as needed. The memory 1400 that stores the program that realizes the unit 400, the CPU (Central Processing Unit) 1300 that executes the program that realizes the two-dimensional marker recognition unit 400, the output of the recognition result, and the setting values of the judgment unit and the camera are input. The input / output device 1500 is included. Here, it is clear that one or more camera devices 1100 are not intended to be limited to three, and may be any number of one or more. According to FIG. 3, the auxiliary storage device 1200, the CPU 1300, the memory 1400, and the input / output device 1500 are connected by the internal bus 1600. The camera device 1100 is connected to the internal bus 1600 by using the external connection interface 1110. Here, the external connection interface 1110 includes a USB (Universal Serial Bus), a wired network, a wireless network, and the like. When there is one camera device 1100, the camera device 1100 may be directly connected to the internal bus 1600. The auxiliary storage device 1200 may be realized by a ROM (Read Only Memory) or a hard disk, and the memory 1400 may be realized by a memory element that realizes a RAM (Random Access Memory).

In the following description, one or more camera devices 1100 will be installed at different locations in the warehouse, for example. FIG. 15 shows an installation example of the camera. FIG. 15 is an example showing a plan view of the warehouse. With reference to this, three camera devices (1101, 1102, 1103) are installed on the inner wall 2000 of the warehouse. It is assumed that the position where the goods (here, a cardboard box) is stored is determined in the warehouse (warehouse goods management unit 2100). According to the installation shown in FIG. 15, the cardboard box has four vertical rows and three horizontal rows, and each of the aligned compartments is provided with four corrugated cardboard box installation spaces. Here, it is assumed that the corrugated cardboard boxes are not stacked in the upward (stacked) direction, and the two-dimensional marker 100 is stretched on the upper surface or the side surface of each corrugated cardboard box. It is assumed that the sizes (rectangular width, height) of the two-dimensional markers 100 themselves are all the same (for example, a square of 5 cm × 5 cm). At this time, the three camera devices 1100 will image the two-dimensional marker 100 from their respective installation positions. It is assumed that the degree of freedom of movement and rotation of each camera device 1100 itself is limited (it is not the camera device 1100 having an expensive multi-degree of freedom, and the moving lane in the warehouse is not installed, which is simple. It shall be an installation form). The setting input for the camera device 1100 from a distance includes camera movement, rotation, imaging magnification, focal position, imaging brightness, and the like within a possible range. However, the above description is not intended to limit the present invention. For example, it is possible to stack corrugated cardboard. Further, by storing the size of each two-dimensional marker as the corresponding two-dimensional marker size in the two-dimensional marker recognition device in advance, it is not necessary to unify the size of each marker. Can also be extended and realized.

The absolute position of each two-dimensional marker 100 (position in the warehouse coordinate system, warehouse alignment section, section position) can be obtained from the absolute position where each camera device 1100 is installed and the calculated relative position. .. The relative position (three-dimensional distance vector) from the camera device 1100 is obtained from the direction of the image taken from the camera device 1100 and the size in the captured image of the two-dimensional marker 100 (variable depending on the camera magnification), and the camera device 1100 is set. The absolute position can be obtained from the relative position from the current position. From this, it can be estimated that the same two-dimensional marker 100 is used even if the camera devices 1100 are different.

Further, the warehouse article management unit 2100 may have a structure such as a pillar that obstructs the view from the camera device 1100. In the example of FIG. 15, four pillars exist in the warehouse article management unit 2100. Even in such a case, by installing three camera devices (1101, 1102, 1103), it is possible to reduce the area that becomes a blind spot. Here, in the warehouse article management unit 2100, the area where the actual article can exist (the area excluding the area such as the pillar) is hereinafter defined as the existable area 2200.

The above description of the installation of the camera device 1100 is for the purpose of explanation in the embodiment described below, and is not intended to limit the scope of application of the present invention. The camera device 1100 may be a digital camera having a degree of freedom of movement and rotation, a camera with a built-in smartphone, a camera with a built-in wearable device, or a Web camera, a surveillance camera, and a drive recorder that cannot be moved or rotated remotely. You may.

Further, as one embodiment of the two-dimensional marker recognition device 1000, it is a single board computer having a camera device 1100, or a smartphone terminal alone, a server device to which a surveillance camera is connected by wire, and a Web camera are connected by a wireless network. It may be a server device.

In one embodiment of the invention, the two-dimensional marker 100 is located away from the camera device 1100. The distant position means that the two-dimensional marker 100 cannot always be accurately recognized by one captured image. Since the position is far away, the resolution of the rectangle in the feature amount display area is lowered, the rectangle in the feature amount display area is distorted depending on the position of the camera device 1100 (the rectangle is no longer a rectangle), and there is reflection of lighting. In addition, people and equipment may be reflected at the same time as the goods move, and a pattern similar to the 2D marker 100 may be reflected on the walls and sky of the warehouse and natural objects (trees, buildings, clouds) as the background. , Etc. must be dealt with. Due to these problems, the reflected object is recognized as the two-dimensional marker 100, which causes a problem.

According to many two-dimensional marker 100 standards, an error correction code is included in the feature amount display area. For example, in the QR code (registered trademark), which is one of the two-dimensional markers 100, it is possible to read correctly even if a part of the feature amount display area is dirty or damaged. You can also. The level that can be restored by the error correction code can be selected from four levels: L (7%), M (15%), Q (25%), and H (30%). Although these error correction codes can be applied in the present invention as well, there is a problem that even if an error correction code is used, it is erroneously recognized when trying to recognize something that is not originally a two-dimensional marker 100. Does not always solve.

For example, when recognizing the ArUco marker, which has a relatively low level of error correction code, as a numerical value, if the computer keyboard appears in the image to be recognized, each key may be erroneously recognized with a certain probability. be. For example, as an ArUco marker pre-defined as a size of 4 × 4, the key of [I] may be set to 190, the key of [E] may be set to 944, and the key of [H] may be erroneously recognized as 269.

Further, another problem due to the two-dimensional marker 100 being located at a distant position is that the two-dimensional marker 100 cannot be imaged from the front. In FIG. 14, when a specific QR code (registered trademark) is imaged from the front (A), the box on which the two-dimensional marker 100 is stretched rotates in the three-dimensional space, and the orientation posture of the two-dimensional marker 100 changes. It shows an example of doing. In the case of the QR code (registered trademark), one of the four corners of the two-dimensional marker 100 is not displayed, and the horizontal direction (x direction) and the vertical direction (y direction) can be defined from this feature. The z direction can be the distance direction from the front of the QR code (registered trademark) (indicated by a black circle in (A)). As shown in FIGS. 14 (B), (C), and (D), the rotation of the box causes the x, y, and z axes of the two-dimensional marker 100 to rotate, and the respective rotation angles are generated, but the recognition target is two-dimensional. Since it is the marker 100, it is possible to estimate the rotation angle from the captured image. The rotation angle may be a rotation vector with respect to the x-axis, the y-axis, and the z-axis, or may be an Euler angle, a rotation matrix, or a quaternion. In the following, an example using a rotation vector will be described. Many types of two-dimensional markers 100, including the ArUco marker standard, are defined in the left-right and up-down directions.

According to the above-described embodiment, the determination units (1) to (7) contribute to suppressing erroneous determination of the two-dimensional marker 100.

[Example of the first embodiment]
Next, an example of the first embodiment of the present invention will be described in detail.

In this embodiment, each of the seven determination units will be described. That is, in the present embodiment, in order to clarify the technical features of the invention, the two-dimensional marker 100 is erroneously recognized by the individual determination unit. As a modification of the present embodiment, one or more determination units (1) to (7) may be included. Alternatively, the entire determination unit may be included. Hereinafter, the determination units (1) to (6) of the two-dimensional marker recognition device 1000 in the present embodiment will be described as processing examples with reference to FIGS. 4 to 9. The processing contents of the determination unit (7) will be described without using figures.

[(1) Multiple image determination unit 411]
FIG. 4 shows a processing example of the plurality of image determination units 411. In step S-101, information (camera angle, position, magnification, etc.) of the imaging device (camera device 1100) is set. This information is necessary for determining the position and orientation of the two-dimensional marker 100. That is, the camera angle, position, and magnification are correction information for estimating the three-dimensional orientation direction and the three-dimensional position in the absolute coordinate system of the rectangular region recognized as the two-dimensional marker 100. In particular, in estimating the three-dimensional position, according to the present embodiment in which the size of the two-dimensional marker 100 in the real space is known, the camera magnification information estimates the distance from the camera device 1100 to the two-dimensional marker 100. It is used as a calculation standard value.

Next, the minimum recognition detection rate is set in step S-102. This numerical value serves as a threshold value for whether or not the recognition is successful among the plurality of images described later. In the following embodiment, as an example of the minimum recognition detection rate, 0.8 (80%) will be described.

Next, in step S-103, an imaging operation of the camera device 1100 is performed to generate and acquire an captured image. This step includes acquiring a plurality of images by using some means. As a means for this, the captured images of a plurality of camera devices 1100 are acquired, the camera devices 1100 are operated at regular time intervals to acquire a plurality of captured images, and further, the captured images are processed. Includes generating another captured image. In particular, processing the captured image to generate it is suitable for recognizing the binary marker 100 by performing digital processing such as adding Gaussian noise or a low-pass filter to the captured image and adjusting the captured image resolution. It is also possible to extract the outline of the peripheral outer edge portion and to perform binary (specific color value) imaging suitable for the pattern expression of the feature amount display area. Acquiring a plurality of captured images by operating the camera device 1100 at regular intervals includes acquiring arbitrarily sampled frames when the camera device 1100 is used as a moving image imaging device.

The next step S-104 is an iterative process performed on the plurality of captured images acquired in step S-103. In this step, first, a rectangular region recognized as the two-dimensional marker 100 is extracted from the captured image. However, it should be noted here that the captured image does not always clearly (100% accurately) recognize the two-dimensional marker 100. In this step, as a judgment material for recognizing the two-dimensional marker 100, an attempt is made to extract a feature amount from the feature amount display area. For this purpose, in reality, the rectangle in the captured image that has an orientation angle and is distorted is converted into a rectangle (or a square), and then the feature amount is extracted. For example, in the case of ArUco markers, matching feature quantities are extracted by comparing patterns with the ArUco marker dictionary or a unique dictionary. Since the present invention recognizes from a "distant position", the feature amount (numerical value, character string, URL, etc.) of the two-dimensional marker 100 is recognized with a certain likelihood (plausibility). The likelihood may be a fixed value in this step or may be preset in step S-101. Further, a plurality of regions recognized as the two-dimensional marker 100 may be detected from one captured image. When a group of cardboard boxes in a warehouse is imaged, an area corresponding to the number of two-dimensional markers 100 stretched on the cardboard boxes may be detected. Alternatively, a background image or a reflected image that is erroneously recognized as the two-dimensional marker 100 may be detected. In consideration of these points, this step is described as "area detection that may be recognized as a two-dimensional marker" in the figure. From this point onward, the same description in other criteria shall mean the same as stated here. That is, the "region detection that may be recognized as a two-dimensional marker" is actually a feature quantity in a region that may or may not be the two-dimensional marker 100 (misrecognized). It is a process to extract.

The next step S-105 is a process for each of the regions determined to be the same. The fact that they are the same means that the feature amounts extracted from the feature amount display area of the two-dimensional marker 100 in each area are the same. In this step, the ratio (hereinafter referred to as the recognition detection rate) between the number of region-captured images to be determined and the number of features having the same feature amount is obtained. Here, when the recognition detection rate is lower than the minimum recognition detection rate set in step S-102, it is determined that the area is not the area recognized as the two-dimensional marker 100 (Y determination). If this is not the case, the two-dimensional marker 100 is recognized here, and the process proceeds to the next step (step S-106 or later).

Here, with reference to FIG. 10, an operation example of the plurality of image determination units 411 is shown. In FIGS. 10A and 10B, it is assumed that there are 11 captured images of the "region recognized as the same two-dimensional marker". Of these, in FIG. 10A, the same two-dimensional marker 100 is detected on nine sheets. On the other hand, in FIG. 10B, there are three sheets. That is, the recognition detection rate in the case of FIG. 10A is 9/11, and that in the case of FIG. 10B is 3/11. Here, when the minimum recognition detection rate is set to 80% in step S-102, the recognition detection rate exceeds 80% in FIG. 10 (A). It satisfies the criteria of the dimension marker 100. Since FIG. 10B is less than 80%, it does not satisfy the criteria of (1) determination unit.

[(2) Shortest edge determination unit 412]
Next, a processing example by the shortest side determination unit 412 will be described with reference to FIG. Since the process of step S-201 is the same as that of step S-101 of FIG. 4, the description thereof will be omitted. In step S-202, the “region shortest edge threshold” is set.

The captured image to be recognized in step S-203 is acquired. Here, the captured image may be independently acquired from the camera device 1100 in this processing example, but one of the captured images generated by the processing example (1: multiple image determination unit 411) is selected. It may be a choice.

In step S-204, "extraction of a region that may be recognized as a two-dimensional marker" is performed from the acquired captured image. Here, a region, which may be the two-dimensional marker 100, is extracted from the captured image.

In step S-205, the length of each side of the region (rectangular region) that may be the two-dimensional marker 100 is obtained. Of the four sides, the length of the side that is typically judged to be the shortest is obtained. With reference to FIG. 11, in an imaging example that may be the two-dimensional marker 100, the lengths of the four sides are obtained here (the lengths of the broken line and the solid line having double-headed arrows). In the example of FIG. 11, the shortest of these is the length of the solid line with the double-headed arrow. In this step, this length is compared with the region shortest side threshold set in step S-202, and if it is less than the region shortest side threshold (Y determination), the reference (2: shortest side determination unit 412) is used. Not satisfied. If this is not the case (N determination), it is assumed that the criteria (2) are satisfied, and the process proceeds to steps S-206 and subsequent steps. Alternatively, as the length determined in step S-205, it is conceivable to use the marker dimension length corresponding to the smallest length of the four sides, for example, the diagonal line of a rectangular or quadrilateral region. In this case, for example, it is determined as comparing the smallest length of the lengths from the intersections of the two diagonal lines with the shortest side threshold value.

Here, the "region shortest side threshold" set in step S-202 may be any one representing the length (corresponding to the length), and may be, for example, the number of image elements (pixels) in the image. It may be the actual length (centimeter) in the physical space obtained as a result of image recognition.

Since the acquired image in step S-203 may be one of the captured images generated in the plurality of captured images in the processing example (1), the determination unit in (2) is the determination unit in (1). It may be used in combination with. Returning to FIG. 11, as preprocessing for the same two-dimensional marker 100, (2) the criteria of the judgment unit are applied, and if it is less than the "region shortest edge threshold", the same judgment is recognized. It may not be the target.

[(3) Area circumference determination unit 413]
Next, a processing example by the region periphery determination unit 413 will be described with reference to FIG. Since the process of step S-301 is the same as that of step S-101 of FIG. 4, the description thereof will be omitted. In step S-302, the "monochromatic area width threshold value" is set.

The captured image to be recognized in step S-303 is acquired. Here, as in step S-203 in FIG. 5 in the processing example (2), the captured image may be independently acquired from the camera device 1100 in this processing example, but a plurality of captured images may be generated by the processing example (1). One of the captured images may be selected.

In step S-304, "extraction of a region that may be recognized as a two-dimensional marker" is performed from the acquired captured image. This step is the same as step S-204 in FIG. 5 in the processing example (2).

In step S-305, the captured image area (rectangular area) that may be the two-dimensional marker 100 is corrected to a rectangular shape (square). After this processing, a region having a single color (peripheral single color region), which should exist if it is the two-dimensional marker 100 and is recognized as a monochromatic color in a gradation, is extracted. If it is a QR code (registered trademark) or ArUco marker, the single color is white. Furthermore, the width of the extracted surrounding monochromatic area is calculated. With reference to FIG. 12, the width of the extracted perimeter monochromatic region is indicated by the length of the solid line with double-headed arrows. In this step, this length is compared with the monochromatic region width threshold set in step S-302, and if it is less than the monochromatic region width threshold (Y determination), (3) criterion is not satisfied. If this is not the case (N determination), it is assumed that the criteria (3) are satisfied, and the process proceeds to steps S-306 and subsequent steps.

Here, the monochromatic region width threshold value set in step S-302 may be the number of image elements or the physical space, similarly to the region shortest side threshold value set in step S-202 in (2) determination. It may be the actual length within.

Since the image acquisition in step S-303 may be one of the captured images generated in the plurality of captured images in the process example (1), the determination unit in (3) is the determination unit in (1). It may be used in combination with. This point is the same as (2) the determination unit, and can be (1) preprocessing of the determination unit.

[(4) Identification information determination unit 414]
Next, a processing example by the identification information determination unit 414 will be described with reference to FIG. 7. Since the process of step S-401 is the same as that of step S-101 of FIG. 4 (and step S-201 of FIG. 5), the description thereof will be omitted. In step S-402, a list of feature quantities to be recognition candidates is set. For example, it corresponds to a list having three elements of "[1] food A, [2] food B, and [3] food C" as recognition candidates.

The captured image to be recognized in step S-403 is acquired. Here, as in step S-203 in FIG. 5 of the processing example (2), the captured image may be independently acquired from the camera device 1100 in this processing example, but a plurality of captured images may be generated by the processing example (1). One of the captured images may be selected.

In step S-404, "extraction of a region that may be recognized as a two-dimensional marker" is performed from the acquired captured image. This step is the same as step S-204 in FIG. 5 of the processing example (2).

In step S-405, the feature amount is extracted from the captured image region which may be the two-dimensional marker 100. This corresponds to extracting the feature amount from the feature amount display area after performing rectangle correction, color tone correction, image monochromatic correction, and the like of the two-dimensional marker 100. Then, it is determined whether or not the extracted feature amount is included in the feature amount list set in step S-402. For example, when the extracted feature amount is "food A", it is determined that it is included in the feature amount list (Y determination), and (4) the criteria of the determination unit are satisfied, and step S- Proceed to processing after 406. Alternatively, when the feature amount is "stationery A", it is determined that the feature amount is not included in the feature amount list (N determination), and (4) the criteria of the determination unit are not satisfied.

Since the image acquisition in step S-403 may be (1) one of the captured images generated by the determination unit, (4) the determination unit is combined with (1) the determination unit. May be used. This point is the same as (2) the determination unit, and can be (1) preprocessing of the determination unit.

[(5) Posture determination unit 415]
Next, the process by the posture determination unit 415 will be described with reference to FIG. As stated in advance, the criteria of the three judgment units including (6) and (7) are different from (1) preprocessing by the judgment unit explained in (2) (3) (4) judgment unit, and (2). (3) (4) Positioned as post-processing of the determination unit.

Since the process of step S-501 is the same as that of step S-101 of FIG. 4, the description thereof will be omitted. In step S-502, the posture allowable range of the two-dimensional marker 100 is set. Here, it is assumed that the posture allowable range is set to 0.3 radians as a calculation example.

Step S-503 is a step (1) (2) (3) (4) of "extracting a feature amount in a region that can be recognized as a two-dimensional marker" by the determination unit (FIG. 4: S-104, FIG. 5: S-). 204, FIG. 6: S-304, FIG. 7: S-404). Step S-504 corresponds to the process of "estimating the position and orientation of the two-dimensional marker" in the determination unit (1), (2), (3), and (4) (FIG. 4: S-106, FIG. 5: S-207, FIG. 6: S-307, FIG. 7: S-406).

In step S-504, with respect to the two-dimensional marker image presumed to be the two-dimensional marker 100, the relative three-dimensional orientation direction (attitude) as seen from the camera device 1100 when it is the two-dimensional marker 100 is determined. calculate. Here, in the following description, an example of realizing the three-dimensional orientation direction as a three-dimensional vector is used. In addition, as a numerical value indicating the three-dimensional orientation direction, it is conceivable to use a three-dimensional direction matrix, a quaternion, or the like.

The relative three-dimensional orientation angle seen from the camera device 1100 will be described in detail with reference to FIG. FIG. 14A is an image of the two-dimensional marker 100 (in this case, an example of a QR code (registered trademark)) taken from the front. Here, a square figure can be seen except for one of the four corners (lower right). This is based on the provisions of the QR code (registered trademark), and it is assumed that the coordinate system of the two-dimensional marker 100 shown on the right side of FIG. 14A is set with the lower right part as a mark. That is, the origin is set at the center of the two-dimensional marker 100, the x-axis is set on the right side, the y-axis is set on the upper side, and the z-axis is set in the direction toward the front (indicated by a black circle in the figure). In the case of FIG. 14 (A), the direction vector can be calculated as A = (0.0, 0.0, 1.0) (facing the front direction). 14 (B), (C), and (D) show an example in which the same two-dimensional marker 100 rotates in a three-dimensional space. Here, the respective x, y, and z axes can be calculated from the degree of rectangular distortion of the two-dimensional marker 100. For example, FIGS. 14 (B), (C), and (D) are directions vectors, respectively.
B = (0.0, 0.1, 0.9)
C = (-0.2, -0.1, 0.7)
D = (0.3, 0.3, 0.4)
It is assumed that it is calculated. However, the length (sum of squares of vector elements) of the direction vector can be 1.0, but in this explanation, the sum of absolute values of elements is set to 1.0 for the sake of simplicity. The three-dimensional orientation angle θb between A and B at this time is the following formula A · B = | A || B | cos (θb).
It becomes a suspended angle using. The left side is the inner product of the A vector and the B vector, and | A | on the right side is the length of the vector A. From this relationship, θb is calculated to be 0.11 radians. Similarly, the three-dimensional orientation angle θc between the vectors A and C is calculated to be 0.31 radians, and the three-dimensional orientation angle θd between the vectors A and D is calculated to be 0.81 radians.

In step S-505, it is determined whether or not the posture of the designated two-dimensional marker 100 is within the permissible range. In the above calculation example, (B) is within the posture range, but (C) and (D) are larger than the posture range, as compared with the posture tolerance range (0.3 radians) specified in step S-502. It has become. In this case, (B) is recognized as a two-dimensional marker 100 as being within the posture range (Y determination), and (C) and (D) are considered to be outside the posture range, and the process proceeds to step S-506. In step S-506, the one outside the posture range is revoked from the certification as the two-dimensional marker 100 (even if the two-dimensional marker 100 is once certified in the pretreatment, it is not certified. Cancel as).

The meaning of "cancel" will be explained with reference to another figure. FIG. 13 is an example of an ArUco marker. The two-dimensional marker existence region 2200 in the figure is composed of 4 columns and 5 columns horizontally. Referring to FIG. 13, of these, (1,1) (1,3) (2,1) (2,) at the (horizontal, vertical) position according to the criteria of (1), (2), (3), and (4). 4) It is assumed that (3,2), (3,3) and (5,1) are recognized as the same two-dimensional marker 100. Of these, the ArUco markers other than the (2,4) element are upward, and the (2,4) element is rightward. At this time, if the permissible posture range is "only upward", the certification of (2, 4) will be revoked. Although it is described as the same ArUco marker in FIG. 13, even if it is a different ArUco marker, the definition of the direction is the same. Therefore, similarly, a setting such as "upward only" works effectively.

As described above, the determination by (5: Posture determination unit 415) corresponds to (2), (3), and (4) recertification of the determination result of the determination unit. In this case, if the posture is within the permissible range, the estimation of the two-dimensional marker 100 as the determination result of the determination unit (2), (3), and (4) is maintained.

[(6) Position determination unit 416]
Next, the process by the position determination unit 416 will be described with reference to FIG. This determination unit is positioned as post-processing of (2), (3), and (4) determination units, similarly to (5: posture determination unit 415).

The process of step S-601 is the same as that of step S-101 of FIG. 4, and is therefore omitted. In step S-602, a position region in which the two-dimensional marker 100 can exist is set.

The possible positional region corresponds to, for example, the possible region 2200 illustrated in FIG. 15 which has already been described. Those recognized as the two-dimensional marker 100 and outside the existence area 2200 cancel the estimation. For example, in FIG. 15, since the two-dimensional markers 2301, 2302, 2303 are outside the existing region, the estimation is canceled.

As described above, the position area set in step S-602 is an area (map) that can exist in any space including the warehouse due to business and physical constraints. Returning to the example of FIG. 15, the two-dimensional markers 2301 and 2302 are positions that are impossible in business (outside the warehouse article management unit 2100), and the two-dimensional markers 2303 are impossible due to physical restrictions (existing in the pillar in the warehouse). It is presumed to be).

Step S-603 is a step (1) (2) (3) (4) of "extracting a feature amount in a region that can be recognized as a two-dimensional marker" by the determination unit (FIG. 4: S-104, FIG. 5: S-). 204, FIG. 6: S-304, FIG. 7: S-404). Step S-604 corresponds to the "two-dimensional marker position and orientation estimation" process in (1), (2), (3), and (4) (FIG. 4: S-106, FIG. 5: S-207, FIG. 6). : S-307, FIG. 7: S-406).

In step S-604, with respect to the two-dimensional marker image presumed to be the two-dimensional marker 100, the relative three-dimensional orientation angle and the three-dimensional distance as seen from the camera device 1100 when it is the two-dimensional marker 100. Calculate the vector.

Even with only one camera device (for example, 1103), the relative three-dimensional distance vector of each marker can be obtained. This recognizes the four corners surrounding the rectangle, which is the two-dimensional marker 100, recognizes the intersection of the diagonal lines of the four corners as the center point, corrects it by the magnification of the camera device 1103, and then determines its size (4). It can be calculated from the respective imaging positions of the four corner points of the two-dimensional marker 100 whose physical distance between the corner points) is known. The method for estimating the three-dimensional orientation angle (and three-dimensional distance vector) is mathematically called the Perspective-n-Point method, and the principle and application interface are described in the following documents related to Non-Patent Document 1. It is disclosed. The following documents are incorporated and described in the present invention by reference.
https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html ( February 20, 2020 Internet search results)

As illustrated in FIG. 15, by using a plurality of camera devices (1101, 1102, 1103), the estimation accuracy of the four corner positions of the same two-dimensional marker 100 can be improved, and the camera blind spot can be reduced. It is possible to estimate the three-dimensional position in more accurate absolute coordinates.

In step S-605 of FIG. 9, as described above, it is determined whether or not the position of the estimated two-dimensional marker 100 is within the position region where the estimated position of the two-dimensional marker 100 can exist. When it is within the position region (Y determination), the estimation result is maintained. If it is not within the region (N determination), the estimation result is canceled (step S-606).

As described above, (6) the judgment based on the criteria of the judgment unit corresponds to (2) (3) (4) recertification of the result of the judgment unit, which is the same as (5) the judgment unit.

[(7) Clustering determination unit 417]
The operation of the clustering determination unit 417 will be described without using a diagram.

The determination in the clustering determination unit 417 is performed by further statistically processing the estimation result in (5) posture determination unit 415 (6) position determination unit 416. The statistical method called clustering is a method in which, when an element distribution is given, those that deviate from the overall tendency are excluded from the target as abnormal values. That is, the given threshold is, for example, the distance from the clustering center position.

(5) When applied to the attitude determination unit 415, for example, the distribution of the three-dimensional orientation direction (direction vector) estimated as the two-dimensional marker 100 is used as the statistical population. The B, C, and D vectors described as examples corresponding to (B), (C), and (D) in FIG. 14 are elements of the statistical population. Let's assume that this has 1000 elements. At this time, it is assumed that the average element of the direction vector is (0, 0.1, 0.9) and the standard deviation vector is (0.1, 0.1, 0.1). At this time, for example, it is calculated by calculation that the direction vector elements (0.3, 0.3, 0.4) have an existence probability of 0.01% or less. At this time, this element is excluded from the estimation target.

(6) When applied to the position determination unit 416, for example, the height distribution at the center of the two-dimensional marker 100 is used. When the average height position of the population is 100 cm and the standard deviation is 20 cm, the two-dimensional marker 100 existing at the position where the height position is 300 cm is not estimated because the existence probability is low. can do.

Although the above-mentioned statistical method for posture and position uses mean and variance, clustering analysis is not intended to be limited to this method. Furthermore, the elements are not limited to the direction vector and the existence position. As elements, the degree of distortion, color, lightness, degree of haze, degree of two-dimensional code restoration, and the like of the two-dimensional marker 100 can be added to the population. More generally, clustering, which is mathematically called the K-means method, can also be used. The K-means method is disclosed in the following document. The following documents are incorporated herein by reference.
MacQueen, JB (1967). “Some Methods for classification and Analysis of Multivariate Observations”. 1. Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability. University of California Press. Pp. 281-297

The determination unit (7) can be positioned as the post-processing of the estimation by the (2), (3), and (4) determination units, as in the case of the (5) and (6) determination units. Alternatively, it can be positioned as further post-processing of (5) and (6) determination units.

[Example of Second Embodiment]
Next, an example of the second embodiment of the present invention will be described with reference to FIG. In the first embodiment, the first embodiment shows that the items individually described according to the judgment criteria (1) to (6) are used as a whole. Here, the criteria (1) to (6) shown as one embodiment are shown in the overall flowchart using the corresponding Roman numerals (I to VI) in the figure.

In step S-1001, the information of the imaging device is set. That is, the camera angle, position, magnification, etc. for each camera device 1100 are set and stored.

In step S-1002, a set value for determining false recognition suppression is set. The set values are as follows. That is, (1) the minimum recognition detection rate for the plurality of image determination units 411, (2) the area shortest side threshold value for the shortest side determination unit 412, and (3) the single color area width for the area periphery determination unit 413. The threshold value, (4) a feature amount list for the identification information determination unit 414, (5) an attitude tolerance range for the attitude determination unit 415, and (6) an existence possible area map for the position determination unit 416.

In step S-1003, the camera device 1100 is used to acquire a remotely captured image. Here, it includes taking an image using a plurality of camera devices 1100 and acquiring an arbitrary frame from a moving image of the camera device 1100.

In step S-1004, the target image region presumed to be the two-dimensional marker 100 is extracted. Based on the estimated position on the absolute coordinate axis of the target image area, the area estimated to be the same two-dimensional marker 100 is specified as a partial image of each captured image and stored in the storage unit 300.

Steps S-1005 to S-1013 (or step S-1014) are iteratively processed for each target region extracted in step S-1004.

In step S-1005, the area image is processed with respect to the target area image, the feature amount display area is detected, and the image processing (Gaussian noise or low pass filter) is performed on one captured image from which the feature amount is extracted. It also includes contour edge extraction, monochromaticization (black and white binarization), and resolution change. Each processed image is associated with the partial image stored in step S-1004 and stored in the storage unit 300 as detailed image information.

In step S-1006, preprocessing for erroneous recognition determination is performed. That is, (2) the shortest side determination unit 412, (3) the area surrounding determination unit 413, and (4) the identification information determination unit 414 are performed. The threshold (setting) value for these determinations is selected from those set in step S-1002, and the determination process is performed. If even one is detected as erroneous recognition as a result of the determination (YES branch), the detailed image information is counted as erroneous recognition. If none of them are misrecognized (NO branch), the detailed image information is counted as correctly recognized.

Step S-1007 is a step of estimating the position and the posture from the detailed image information of the two-dimensional marker 100. The estimation here is as described in the criteria of the determination unit (5) and (6) in the first embodiment. That is, the three-dimensional position may use the Perspective-n-Point method. The posture may be obtained from the inner product between the direction vector and the front vector as the three-dimensional orientation direction.

In step S-1008, it is detected and determined whether or not the position and the posture obtained in step S-1007 are within the allowable posture and the two-dimensional marker 100 set in step S-1002 within the existence area 2200. If false recognition is detected in either case (YES branch), the detailed image information is counted as false recognition. If neither is erroneously recognized (NO branch), the detailed image information is counted as correctly recognized as the two-dimensional marker 100. That is, the criteria of the (5) and (6) determination units are post-processing of the two-dimensional marker 100 estimation process based on the criteria of the (2), (3), and (4) determination units.

In step S-1009, the feature amount is extracted for the two-dimensional marker 100 which is not erroneously recognized. As a result, the relationship list between the detailed image information and the feature amount of the two-dimensional marker 100 (a plurality of feature amounts may be detected correspondingly to the same two-dimensional marker 100) is associated with each other. It is stored in the storage unit 300.

Step S-1010 is a step called when the detailed image information is erroneously recognized. The storage unit 300 stores that the detailed image information has been erroneously recognized.

After step S-1009 and step S-1010, it is determined whether or not there is unprocessed detailed image information, and if not, the process proceeds to step S-1011.

In step S-1011, the count of the detailed image information in the same target area stored in the storage unit 300 is compared with the count in which the determination result for the detailed image information is recognized as the two-dimensional marker 100. Here, when the two-dimensional marker 100 includes those recognized as different feature quantities, the count of the two-dimensional marker 100 having the largest number of feature quantities is taken as the count recognized as the two-dimensional marker 100. .. The ratio of both counts is calculated, and this is stored in the storage unit 300 as a ratio (detection rate) having the same feature amount.

In step S-1012, it is determined whether or not the detection rate is less than the minimum detection rate set in step S-1002. This step corresponds to the determination unit (1). If it is less than the minimum detection rate, it is determined that the feature amount of the two-dimensional marker 100 cannot be detected in the target image area (the feature amount of the two-dimensional marker 100 is not recognized) (YES branch). If it is equal to or higher than the minimum detection rate, the process proceeds to step S-1013, where the feature amount of the two-dimensional marker 100 is determined to be certified in the target image area. In this way, the two-dimensional marker 100 only when the two-dimensional marker recognition detection rate in the entire plurality of images (multiple camera devices 1100, moving image frame, processed image from the position image) exceeds the threshold value. Recognize as.

How misrecognition is suppressed through the present embodiment will be described with reference to FIG. 18 by a schematic example. In FIGS. 18 (A), (B), and (C), it is assumed that a 5 × 4 region is a possible region in the horizontal and vertical directions. In FIG. 18A, it is recognized as a two-dimensional marker 100 including those outside the existence area 2200 and those having different postures. In FIG. 18B, the two-dimensional marker 100 is evaluated according to the respective criteria of (5) posture determination unit 415 and (6) position determination unit 416. Here, the element (2, 4) is misrecognized as being out of the posture range, and the upper right element is misrecognized as being out of the position range, assuming that the position is outside the possible existence area. As a result, as shown in FIG. 18C, a total of six elements are recognized as the two-dimensional marker 100.

According to the present embodiment, it is possible to acquire the entire image within the image capture range of the camera device 1100 and identify the target image area which may be the two-dimensional marker 100 in the entire image. Further, the population parameter of the recognition target can be increased by performing image processing on the target image area. After that, the determination units (2), (3), and (4) are executed as preprocessing to count the target image area that may be the two-dimensional marker 100. After that, it is further executed as post-processing whether or not it is the two-dimensional marker 100 from the position / orientation, and similarly, the target area image that may be the two-dimensional marker 100 is counted. (1) The determination unit compares the minimum detection rate with the element ratio counted as the two-dimensional marker 100, and recognizes the target area as the two-dimensional marker 100 only if the detection rate is equal to or higher than the minimum detection rate.

[Modification 2-1]
The case where the criteria of the determination unit from (1) to (6) are applied has been described with reference to the present embodiment. As a modification 2-1 of this embodiment, a case where (7) the criterion of the determination unit is applied will be described.

(7) In recognizing the recognized detailed image information, the determination unit uses the basic value as a statistical population, and cancels the recognition of the recognized ones that are stochastically low from the clustering center distance. This process may be performed in step S-1009 in the modified example of the present embodiment (it is a post-process of the determination units (2), (3), and (4)). Alternatively, it may be performed in step S-1011 (the entire post-processing from the determination unit (1) to (6) is performed).

[Modification 2-2]
In the description of the embodiments so far, the value set in step S-1002 is selected by the user. In this modification, the tendency of this set value is described, and the setting policy is described.

FIG. 16 shows the relationship between the region shortest side threshold value used in (2: shortest side determination unit 412) and the false recognition detection rate among the values set in step S-1002. As is clear from FIG. 16, the false recognition detection rate can be reduced by raising the shortest side threshold value. Here, misrecognition means "not recognizing the wrong thing correctly", but on the contrary, think about "recognizing the right thing as the wrong thing". The opposite can be expressed by a numerical value called the detection omission rate. FIG. 16 also shows the relationship between the region shortest edge threshold and the detection omission rate. As is clear from FIG. 16, it can be seen that there is a problem that the detection omission rate increases in response to the increase in the shortest side threshold value. This relationship (correlation) is not limited to the determination unit (2) the region shortest edge threshold of the determination unit, (1) the minimum detection rate of the determination unit, (3) the single color region width threshold of the determination unit, and (5) the determination unit. A similar correlation can be seen for the postural tolerance.

Therefore, in this modified example, a technique for applying reinforcement learning among machine learning will be described. FIG. 19 is a diagram illustrating a realization example of reinforcement learning. The learning agent 3000 is an agent capable of changing the set values given in steps S-1001 and S-1002 in the second embodiment (FIG. 17). The learning agent 3000 can change the input values in S-1001 and S-1002 in the learning environment 4000 of the two-dimensional marker recognition device, and in the learning environment 4000 of the two-dimensional marker recognition device, each determination unit It is assumed that the number of missed detections can be calculated by subtracting the number of misrecognitions from the number of two-dimensional markers that are misrecognized according to the standard and the number of two-dimensional markers that are correct answers in each learning scene (scene). Here, the learning agent 3000 is given a difference or ratio between the false recognition rate and the number of missed detections as a reward value. During the learning period (for example, for one month), the learning agent 3000 changes the set value so as to increase the reward given. The period for changing can be, for example, every hour. Of course, the study period of 1 month and the change period of 1 hour are not intended to be limited to these values. In reinforcement learning, it is assumed that the higher the reward value is, the more effectively the setting is functioning. The change of the set value may be random within a certain range, or may be changed in the direction of increasing the reward value if the differential value with respect to the reward value is obtained as a tendency. Furthermore, it is useful to study not only in one warehouse scene (scene) but also in several warehouse scenes (for example, possible patterns such as morning and noon, some luggage situation, luggage density, etc.).

After the learning period ends, the learning agent 3000 that has been strengthened and learned in this way is applied to the actual operating environment 4001 of the two-dimensional marker recognition device in the warehouse as the learned agent 3001. The trained agent 3001 operates on the assumption that the false recognition rate and the number of missed detections are appropriately matched according to the environmental conditions (parameters: brightness in the environment, number of corrugated cardboards in the warehouse, degree of density, etc.).

By using the modified example of the present embodiment, the trained agent 3001 can perform system tuning according to the accuracy and speed required by the application to be applied. In particular, in the two-dimensional marker recognition device 1000 in operation, it is premised that the camera device 1100 and the two-dimensional marker 100 are brought close to each other as described in the background technology, and the processing efficiency (number) that can be processed at this time is assumed. / Hour) may cause a delay in human work. In the case of processing from a remote location, according to the modified example of the present embodiment, it is possible to perform recognition more efficiently according to the required accuracy and speed.

Although the industrial application fields of the present invention are clear from the above-described embodiment, some examples are given below (naturally, the invention is not limited to those shown below). The first area is to improve the efficiency of goods management work. The two-dimensional marker 100 can be used to manage storage locations and inventories such as receipt, warehousing, and picking of articles. In addition, the two-dimensional marker 100 can be used to carry out inventory inventory of goods. The second area is the prompt and accurate handling of information in the medical field. The corresponding drug information can be immediately accessed from the two-dimensional marker 100 given to the drug. In addition, medical malpractice can be prevented by reading the two-dimensional marker 100 before dosing. Further, it is possible to accurately and quickly read the prescription information from the two-dimensional marker 100. The third field is the handling of people and information in leisure and amusement museums. It is possible to perform entry / exit management without delay using the two-dimensional marker 100. Further, a temporary lost child tag using the two-dimensional marker 100 can be generated and used to avoid confusion. Alternatively, the two-dimensional marker 100 can be attached around the exhibit to display detailed information on the exhibit. The fourth area is the application to cashless payments, which are difficult to counterfeit, are quick, and have few false positives. As another conceivable field, a browser display by linking to a site URL using the two-dimensional marker 100, or a rental service using the two-dimensional marker 100 as a key can be considered.

[mode]
Some or all of the above embodiments may also be described, but not limited to:
<Mode 1>
This is the same as the two-dimensional marker recognition device according to the first viewpoint described above.
<Mode 2>
Preferably, the two-dimensional marker recognition device according to the mode 1 is the specific determination unit in the false recognition suppression unit.
A plurality of images based on whether or not the ratio of the region images recognized as the same two-dimensional marker among the plurality of region images on which the two-dimensional markers are copied exceeds the first threshold value. Judgment unit,
Whether the minimum length of the four sides recognized from the rectangular image of the two-dimensional marker or the length of the corresponding marker dimension from the region image in which the two-dimensional marker is copied is smaller than the second threshold value. Please use the shortest side judgment unit as a judgment standard,
In the region image on which the two-dimensional marker is copied, the region periphery determination unit, which determines whether or not the width recognized as the peripheral monochromatic region of the two-dimensional marker is larger than the third threshold value,
Identification information based on whether or not the feature amount recognized from the feature amount display area is included in one or more feature amount lists specified in advance in the area image in which the two-dimensional marker is copied. Judgment unit,
The three-dimensional orientation direction of the two-dimensional marker is recognized from the image of the two-dimensional marker, and whether or not the difference from the three-dimensional orientation direction specified in advance is smaller than the fourth threshold value is used as a determination criterion. , Attitude judgment unit, or
A position determination unit, which recognizes the three-dimensional position of the two-dimensional marker from an image of the two-dimensional marker and uses whether or not it is included in a predetermined position range as a determination criterion.
A two-dimensional marker recognition device, characterized in that it is one of the above.
<Mode 3>
Preferably, the two-dimensional marker recognition device according to the mode 1 is used, and the specific determination unit in the false recognition suppression unit is:
A plurality of images based on whether or not the ratio of the region images recognized as the same two-dimensional marker among the plurality of region images on which the two-dimensional markers are copied exceeds the first threshold value. Judgment unit,
The shortest side determination unit, which is used as a criterion for determining whether the minimum length of the four sides recognized from the rectangular image of the two-dimensional marker is smaller than the second threshold value from the area image on which the two-dimensional marker is copied.
In the region image on which the two-dimensional marker is copied, the region periphery determination unit, which determines whether or not the width recognized as the peripheral monochromatic region of the two-dimensional marker is larger than the third threshold value,
Identification information based on whether or not the feature amount recognized from the feature amount display area is included in one or more feature amount lists specified in advance in the area image in which the two-dimensional marker is copied. Judgment unit,
The three-dimensional orientation direction of the two-dimensional marker is recognized from the image of the two-dimensional marker, and whether or not the difference from the three-dimensional orientation direction specified in advance is smaller than the fourth threshold value is used as a determination criterion. , Attitude judgment unit, and
A position determination unit that recognizes the three-dimensional position of the two-dimensional marker from an image of the two-dimensional marker and determines whether or not the two-dimensional marker is included in a predetermined position range.
A two-dimensional marker recognition device comprising.
<Mode 4>
The two-dimensional marker recognition device according to the mode 2 or 3, preferably, the erroneous recognition suppressing unit includes the specific determination unit.
The three-dimensional orientation direction or the three-dimensional position of the two-dimensional marker is recognized from the plurality of region images on which the two-dimensional marker is copied, and clustering statistical processing is performed with the three-dimensional orientation direction or the three-dimensional position as an element. Further includes a clustering determination unit, which is based on whether or not the distance from the calculated cluster median value is smaller than the fifth threshold value.
A two-dimensional marker recognition device characterized by this.
<Mode 5>
The two-dimensional marker recognition device according to the mode 2 or 3, preferably in the plurality of image determination units in the false recognition suppression unit.
The two-dimensional marker in an absolute coordinate system obtained by acquiring a plurality of the region images from a plurality of the camera devices having different installation positions or installation directions and calculating from the installation positions and the installation directions of the plurality of camera devices. Since the three-dimensional orientation orientation and the three-dimensional position are the same, it is determined that the plurality of the region images corresponding to the two-dimensional marker correspond to the same two-dimensional marker.
A two-dimensional marker recognition device characterized by this.
<Mode 6>
Preferably, in the two-dimensional marker recognition device according to the mode 4, Gaussian noise is added to one of the area images as a plurality of the area images in the plurality of image determination units in the false recognition suppression unit. Alternatively, a two-dimensional marker recognition device, characterized in that another image is generated by applying a low-pass filter and added to a plurality of the region images for image determination.
<Mode 7>
Preferably, the two-dimensional marker recognition device according to the mode 4 is used in the posture determination unit and the position determination unit in the false recognition suppression unit.
The three-dimensional orientation direction and the three-dimensional position of the two-dimensional marker are estimated as relative quantities from the camera device by using the Perspective-N-Point method.
A two-dimensional marker recognition device characterized by this.
<Mode 8>
Preferably, the two-dimensional marker recognition device according to mode 4, wherein the false recognition suppression unit has a first threshold value, a second threshold value, a third threshold value, a fourth threshold value, and a fifth threshold value. A learning agent having a set value including a camera direction, an orientation posture, a magnification, and an illuminance of the camera device as an internal value is included.
In the misrecognition suppressing unit, the difference or ratio between the number misrecognized by the determination and the number actually not recognized by the two-dimensional marker is used as the reward value in learning.
As reinforcement learning, the learning agent performs learning for a certain period of time in a two-dimensional marker recognition device learning environment that performs learning on condition that the reward value is increased for the combination of the internal values.
After learning for the fixed period, the learned agent is set as a learned agent, and the learned agent has the internal value and operates as an operating environment of a two-dimensional marker recognition device.
A two-dimensional marker recognition device characterized by this.
<Mode 9>
This is the same as the two-dimensional marker recognition method according to the second viewpoint described above.
<Mode 10>
This is the same as the two-dimensional marker recognition program according to the third viewpoint described above.
<Mode 11>
An image acquisition unit that acquires an image from a camera device, a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the image, and a two-dimensional marker recognition unit. It is a two-dimensional marker recognition system having a two-dimensional marker recognition device, and whether or not the area image corresponds to the two-dimensional marker in the two-dimensional marker recognition unit of the two-dimensional marker recognition device. A two-dimensional marker recognition system characterized by having a false recognition suppression unit that is judged by a specific judgment unit.
It should be noted that modes 9 to 11 can be developed like modes 2 to 8 as in mode 1.

Each disclosure of the above-mentioned patent documents cited shall be incorporated into this document by citation. Within the framework of the entire disclosure (including the scope of claims) of the present invention, it is possible to change or adjust an embodiment or an embodiment based on the basic technical idea thereof. In addition, various combinations of various disclosure elements (including each element of each claim, each element of each embodiment or embodiment, each element of each drawing, etc.) within the framework of all disclosure of the present invention, or Choices (including partial deletion) are possible. That is, it goes without saying that the present invention includes all disclosure including claims, and various modifications and modifications that can be made by those skilled in the art in accordance with the technical idea. In particular, with respect to the numerical range described in this document, it should be interpreted that any numerical value or small range included in the range is specifically described even if there is no other description. Further, each of the disclosed matters of the above-cited documents may be used in combination with the matters described in this document in part or in whole as a part of the disclosure of the present invention, if necessary, in accordance with the gist of the present invention. It is considered to be included in the disclosure matters of the present application.

100: Two-dimensional marker (group)
200: Image acquisition unit 300: Storage unit 400: Two-dimensional marker recognition unit 410: False recognition suppression unit 420: Feature amount extraction unit 430: Position and orientation estimation unit 411: Multiple image determination unit 412: Shortest side determination unit 413: Area Peripheral determination unit 414: Identification information determination unit 415: Attitude determination unit 416: Position determination unit 417: Clustering determination unit 1000: Two-dimensional marker recognition device 1100, 1101, 1102, 1103: Camera device 1110: External connection interface 1200: Auxiliary storage Device 1300: CPU
1400: Memory 1500: Input / output device 1600: Internal bus 2000: Warehouse inner wall 2100: Warehouse article management department 2200: Existence area 2301, 2302, 2303: Uncertified two-dimensional marker 3000: Learning agent 3001: Learned agent 4000: Learning environment of 2D marker recognition device 4001: Actual operation environment of 2D marker recognition device

Claims (10)

An image acquisition unit that acquires images from the camera device,
A two-dimensional marker recognition device including a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the above image.
The two-dimensional marker recognition unit includes an erroneous recognition suppression unit that determines whether or not the region image corresponds to the two-dimensional marker by a specific determination unit.
A two-dimensional marker recognition device characterized by this. What is the specific determination unit in the false recognition suppression unit?
A plurality of images based on whether or not the ratio of the region images recognized as the same two-dimensional marker among the plurality of region images on which the two-dimensional markers are copied exceeds the first threshold value. Judgment unit,
Whether the minimum length of the four sides recognized from the rectangular image of the two-dimensional marker or the length of the corresponding marker dimension from the region image in which the two-dimensional marker is copied is smaller than the second threshold value. Please use the shortest side judgment unit as a judgment standard,
In the region image on which the two-dimensional marker is copied, the region periphery determination unit, which determines whether or not the width recognized as the peripheral monochromatic region of the two-dimensional marker is larger than the third threshold value,
Identification information based on whether or not the feature amount recognized from the feature amount display area is included in one or more feature amount lists specified in advance in the area image in which the two-dimensional marker is copied. Judgment unit,
The three-dimensional orientation direction of the two-dimensional marker is recognized from the image of the two-dimensional marker, and whether or not the difference from the three-dimensional orientation direction specified in advance is smaller than the fourth threshold value is used as a determination criterion. , Attitude judgment part,
A position determination unit, which recognizes the three-dimensional position of the two-dimensional marker from an image of the two-dimensional marker and uses whether or not it is included in a predetermined position range as a determination criterion.
The two-dimensional marker recognition device according to claim 1, further comprising any one or more of the above. The specific determination unit in the false recognition suppression unit includes
The three-dimensional orientation direction or the three-dimensional position of the two-dimensional marker is recognized from the plurality of region images on which the two-dimensional marker is copied, and clustering statistical processing is performed with the three-dimensional orientation direction or the three-dimensional position as an element. Further includes a clustering determination unit, which is based on whether or not the distance from the calculated cluster median value is smaller than the fifth threshold value.
The two-dimensional marker recognition device according to claim 2, wherein the two-dimensional marker recognition device is characterized in that. In the plurality of image determination units in the false recognition suppression unit,
The two-dimensional marker in an absolute coordinate system obtained by acquiring a plurality of the region images from a plurality of the camera devices having different installation positions or installation directions and calculating from the installation positions and the installation directions of the plurality of camera devices. Since the three-dimensional orientation orientation and the three-dimensional position are the same, it is determined that the plurality of the region images corresponding to the two-dimensional marker correspond to the same two-dimensional marker.
The two-dimensional marker recognition device according to claim 3, wherein the two-dimensional marker recognition device is characterized in that. As a plurality of the region images in the plurality of image determination units in the false recognition suppression unit,
Another image is generated by an arbitrary image generation method including a method of adding Gaussian noise to one said region image and a method of applying a low-pass filter, and is added to a plurality of said region images for image determination.
The two-dimensional marker recognition device according to claim 4, wherein the two-dimensional marker recognition device is characterized in that. In the posture determination unit and the position determination unit in the false recognition suppression unit,
The three-dimensional orientation direction and the three-dimensional position of the two-dimensional marker are estimated as relative quantities from the camera device by an arbitrary position estimation method including the Perspective-N-Point method.
The two-dimensional marker recognition device according to claim 4, wherein the two-dimensional marker recognition device is characterized in that. Settings including the first threshold value, the second threshold value, the third threshold value, the fourth threshold value, and the fifth threshold value in the false recognition suppression unit, and the camera direction, orientation posture, magnification, and illuminance of the camera device. Includes a learning agent that has a value as an internal value
In the misrecognition suppressing unit, the difference or ratio between the number misrecognized by the determination and the number actually not recognized by the two-dimensional marker is used as the reward value in learning.
As reinforcement learning, the learning agent performs learning for a certain period of time in a two-dimensional marker recognition device learning environment that performs learning on condition that the reward value is increased for the combination of the internal values.
After learning for the fixed period, the learned agent is set as a learned agent, and the learned agent has the internal value and operates as an operating environment of a two-dimensional marker recognition device.
The two-dimensional marker recognition device according to claim 4, wherein the two-dimensional marker recognition device is characterized in that. An image acquisition unit that acquires images from the camera device,
Two-dimensional marker recognition in a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the above image. It ’s a method,
Whether or not the region image corresponds to the two-dimensional marker in the two-dimensional marker recognition unit is determined by a specific determination unit as erroneous recognition.
A two-dimensional marker recognition method characterized by this. An image acquisition unit that acquires images from the camera device,
A computer installed in a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the above image. It ’s a program,
The two-dimensional marker recognition unit includes means for determining whether or not the region image corresponds to the two-dimensional marker as erroneous recognition by a specific determination unit.
A two-dimensional marker recognition program characterized by this. An image acquisition unit that acquires images from the camera device,
A two-dimensional marker having a two-dimensional marker recognition device having a two-dimensional marker recognition unit that recognizes a rectangular area image composed of a peripheral single color area and a feature amount display area as a two-dimensional marker from the above image. It is a recognition system
The two-dimensional marker recognition unit of the two-dimensional marker recognition device includes an erroneous recognition suppression unit that determines whether or not the region image corresponds to the two-dimensional marker by a specific determination unit.
A two-dimensional marker recognition system characterized by this.

Images