JP5961938B2 - Monospectral marker and method and apparatus for detecting the same - Google Patents

Description

  The present invention relates to a monospectral marker used for identification, identification and the like, and a detection method and apparatus therefor.

  Markers have been used for a long time to identify the location of the object and the position of the object to which it is attached (or represented), to identify the position or object. Recently, a technique for estimating the position and orientation of an observation object and an imaging device by performing image processing on a marker on an image taken by the imaging device has been studied. In particular, the ARToolKit marker is generally used as a marker for positioning the virtual space and the real space (Non-patent Document 1). The ARToolKit marker is constructed by drawing an arbitrary pattern inside a black rectangular frame. The vertex of the four corners of the rectangular frame is detected in the observed image, and the marker position and orientation relative to the camera are detected. The internal pattern is used to identify which marker is observed by comparing the pattern with a known pattern after detecting the position and orientation.

H. Kato, and M. Billinghurst, “Marker tracking and hmd calibration for a video-based augmented reality conferencing system,” International Workshop on Augmented Reality, pp. 85-94, 1999.

  The problem with this black and white marker is that it cannot cope with motion blur and defocusing that occur in the observed image. This is because the edge, which is a high-frequency component, is used to detect the position and orientation of the marker. In other words, when motion blur or defocus occurs in the observed image, high-frequency components on the image are suppressed, and the edge position cannot be detected correctly.

  An object of the present invention is to provide a marker (this is called a monospectral marker) that can be extracted even from an image with motion blur or defocus.

  Another object of the present invention is to provide a method and apparatus for detecting (identifying or identifying) the above-described monospectral marker.

  A monospectral marker according to a first aspect of the present invention is a two-dimensional surface using a color space including a first element representing a value related to brightness and a second element representing a value related to hue. Is a marker represented on a medium having This marker has a marker area having a boundary where the value of the first element is a basic value, and at least two independent position segments are arranged at predetermined positions in the marker area. The position segment has a center position, and in all directions through the center position, the value of the first element shows a peak at the center position and a single value between the value of the peak and the base value. It changes with frequency. The second element of the position segment has a value representing a specific hue.

  A color space (color system) including a first element related to brightness (luminance, brightness) and a second element related to hue (color, color difference) includes, for example, hue (Hue), saturation ( Saturation, Chroma), brightness (Brightness, Lightness, Value) HSV color space (also referred to as HSB color space), brightness Y (luminance) and color difference U, V (chrominance) YUV color space, brightness L Lab color space having red intensity a and yellow intensity b.

  The first element relating to brightness corresponds to brightness V in the HSV color space, brightness Y in the YUV color space, and brightness L in the Lab color space.

  The second element relating to the hue corresponds to the hue H in the HSV color space, the color difference U or V in the YUV color space, and the red intensity a or yellow intensity b in the Lab color space.

  When the HSV color space is used, the third element, saturation S, can also be used.

  Examples of the marker medium include a sheet such as paper and plastic, a flat surface such as a plate-like body and a wall, and one surface of an object (product) (including one surface such as a cardboard box for packing the object). In this case, monospectral markers will be represented by printing, drawing, painting, etc. on these media. The marker medium may be a display screen for displaying the marker.

  The boundary that defines the marker area (boundary area) continuously surrounds the periphery of the marker (for this reason, also referred to as a connected area in the embodiment), and is characterized by the value of the first element being a basic value . The basic value is most typically the minimum value of the value of the first element, but may be a value close to the minimum value.

  The monospectral marker according to the present invention is characterized in that a plurality of mutually independent position segments are arranged in the marker region ("independent" means that they are separated from each other or do not overlap with each other). In addition, the position segment has a center position, and the value of the first element changes at a single frequency (single spatial frequency) in all directions passing through the center position. More specifically, the value of the first element shows a peak at the center position of the position segment. At a position away from the center position, the value of the first element is the above basic value, increases toward the center, and reaches a peak. This change is represented by a single frequency. The peak value is generally the maximum value of the value of the first element, but may be a value close to this.

  The single frequency value (frequency value) may be equal in all directions passing through the center position of the position segment, or may change continuously in the angular direction as shown in the embodiments described later. When the value of the single frequency changes in the angular direction, the spread of the frequency band corresponding to the change of the value of the single frequency must be considered in the pass band of the band-pass filter described later.

  The location segment is further characterized by the value of the second element. The value of the second element may have a range.

  The monospectral marker of the first aspect described above is a monospectral marker characterized by only a plurality of position segments, but the monospectral marker can also include an ID segment different from the position segment. The ID segment is effectively used to distinguish these monospectral markers from each other particularly when there are a plurality of types of monospectral markers (position segments may be the same).

  That is, in the monospectral marker of the second aspect, an ID segment is further arranged in the marker area away from the position segment, and the second element of the ID segment is the second element of the position segment. It is characterized by having a value different from the value. The value of the first element of the ID segment may change with a single frequency or may have an edge.

  As mentioned above, the ARToolKit marker uses edges for its detection. Edges contain high-frequency components, so if there is motion blur or defocus, the high-frequency components are suppressed and the edges are difficult to detect correctly. In the monospectral marker according to the present invention, since the value of at least the first element of the position segment changes with a single frequency (and not a high frequency like an edge, but a low frequency), Even if there is blurring or defocusing, it is not easily affected, and even if there is motion blur or defocusing, it can be detected.

  The present invention provides a method and apparatus for detecting the monospectral marker described above.

  According to the first aspect of the method for detecting a monospectral marker, image data obtained by imaging a scene including the monospectral marker of the first aspect is converted into image data of the color space, and for each pixel, A band path having a passband including a frequency corresponding to the single frequency with respect to the value of the first element, at least obtaining the value of the first element and the value of the second element; Filtering with a filter identifies a pixel that passes through the band pass filter, extracts the marker region (boundary region) for the value of the first element, and the value of the second element is the specific hue. The value of the first element passes through the band pass filter, and the value of the second element falls within the range representing the specific hue. In a pixel, above A pixel whose first element value indicates a peak is identified to determine a center position candidate of the position segment, and among the determined center position candidates, a center position candidate existing in the extracted marker area (boundary area) Is identified as a monospectral marker by matching the predetermined number with a predetermined number.

  The monospectral marker detection apparatus according to the first aspect converts image data obtained by imaging a scene including the monospectral marker according to the first aspect into image data of the color space. Image conversion means for acquiring at least the value of the first element and the value of the second element; and a band pass filter having a pass band including a frequency corresponding to the single frequency. Filtering means for filtering the value of the first element with a filter to identify pixels passing through the band pass filter; a marker area for extracting the marker area (boundary area) for the value of the first element (Boundary region) extraction means, second element determination means for determining whether or not the value of the second element is within the range representing the specific hue, the filtering means Among the pixels specified by (2) and the value of the second element is within the range representing the specific hue by the second element determination means, the value of the first element has a peak. A center position candidate determining means for determining a center position candidate for the position segment by specifying a pixel to be indicated, and a marker area extracted by the marker area extracting means from the center position candidates determined by the center position candidate determining means ( Marker identification means for identifying a monospectral marker by matching the number of center position candidates existing in the boundary region) with a predetermined number.

  The pass band of the band pass filter is a single frequency value (single frequency is the size of the monospectral marker on the image obtained by imaging, the value of the first element representing the position segment of the monospectral marker). The position segment image represented by the value of the first element is passed in consideration of the range of change), the attitude of the monospectral marker in the scene (presence or absence of tilt), the imaging angle of the scene by the imaging means, etc. Determined.

  As described above, the position segment image represented by the value of the first element is not easily affected by motion blur or defocusing. Therefore, the center position of the position segment is correctly extracted and the monospectral marker is selected. Can be identified. The value of the second element contributes to the location segment extraction. Note that the marker area (boundary area) can be extracted by threshold processing at a level near the basic value of the value of the first element, and thus is not easily affected by motion blur or defocus.

  Here, the identification of a monospectral marker means determining the presence and position of at least one monospectral marker. The identification of a monospectral marker, which will be described later, means that when there are at least two types of monospectral markers, the presence and position thereof are determined by distinguishing them. Detection is a concept that includes both identification and identification.

  According to a second aspect of the method for detecting a monospectral marker, image data obtained by capturing a scene including a monospectral marker is converted into image data of the color space, and at least the first spectrum is detected for each pixel. The value of the element and the value of the second element are obtained, and the value of the first element is filtered by a band pass filter having a passband including a frequency corresponding to the single frequency. A pixel that passes through the band pass filter is specified, the marker area (boundary area) is extracted for the value of the first element, and the value of the second element is within a range representing the specific hue. Among the pixels in which the value of the first element passes through the band pass filter and the value of the second element is within the range representing the specific hue. In the first When a pixel whose prime value indicates a peak is specified to determine a center position candidate of the position segment, and the number of center position candidates existing in the extracted marker region matches a predetermined number, The center position candidate is set as the center position of the position segment, and at least the center position of the ID segment is obtained based on the center position. Depending on whether the value of the second element of the obtained center position is within a predetermined range. , It is determined whether it is a specific monospectral marker.

  The monospectral marker detection apparatus according to the second aspect converts image data obtained by capturing a scene including a monospectral marker into image data in the color space, and at least the first spectrum is obtained for each pixel. Image conversion means for acquiring an element value and a value of the second element; and a band pass filter having a pass band including a frequency corresponding to the single frequency. Filtering means for filtering the value of one element to identify pixels passing through the band pass filter, and extracting a marker area (boundary area) for extracting the marker area (boundary area) for the value of the first element Means, second element determining means for determining whether or not the value of the second element is within a range representing the specific hue, specified by the filtering means Among the pixels that are determined by the second element determination means and the value of the second element is within the range representing the specific hue, the pixel having the peak value of the first element Center position candidate determination means for determining a center position candidate of the position segment by specifying the position segment, and the number of center position candidates existing in the marker area (boundary area) extracted by the marker area extraction means is predetermined. If the number matches the number, the center position candidate is set as the center position of the position segment, and at least the center position of the ID segment is obtained based on the center position. The value of the second element of the obtained center position is a predetermined value. Marker identifying means for determining whether or not a specific monospectral marker is determined depending on whether it is within the range or not.

  If the value of the second element of the ID segment is used in a plurality of monospectral markers having the same position segment (the same configuration and position) because the value of the second element of the ID segment is used, This makes it possible to identify a plurality of monospectral markers. Since the center position of the ID segment is determined based on the center position of the position segment, even if the first element of the ID segment does not have a single frequency, the value of the second element of the ID segment It can be acquired and is not easily affected by motion blur or defocus.

  In any of the above detection methods and devices, if the distance from the imaging device to the monospectral marker is fixed, a single band-pass filter is sufficient, but the distance from the imaging device to the monospectral marker is sufficient. If the distance varies or varies depending on the monospectral marker, prepare multiple band pass filters with different passbands depending on the distance from the imaging device to the monospectral marker. Those band pass filters are used to filter the values of the first element to identify pixels that pass through one of the band pass filters.

(A) shows an example of a monospectral marker, and (B), (C), and (D) show changes in values of lightness V, hue H, and saturation S along the L1 line of (A), respectively. (A) shows an example of a monospectral marker, and (B), (C), and (D) show changes in values of lightness V, hue H, and saturation S along the line L2 in (A), respectively. It is a graph which shows the spectrum of the brightness V along the L1 line of the monospectral marker shown to FIG. 1 (A). Another example of a monospectral marker is shown. Still another example of a monospectral marker is shown. Still another example of a monospectral marker is shown. Still another example of a monospectral marker is shown. Still another example of a monospectral marker is shown. An example of an image obtained by imaging a scene in which a monospectral marker is arranged is shown. It is a functional block diagram showing the whole monospectral marker detection system. (A) shows the pass frequency band of the band pass filter in the frequency domain, and (B) shows in the spatial domain. It is a flowchart which shows the process (monospectral marker identification process) which calculates | requires the center pixel of a position segment. FIG. 13 is a functional block diagram for explaining a process of extracting a first candidate pixel of a position segment in step S12 of FIG. It is a flowchart which shows the detection process of the monospectral marker using an ID segment. A process for obtaining the center position of the ID segment will be described.

  First, the configuration of the monospectral marker will be described.

  The monospectral marker of this embodiment is expressed by the HSV color space (HSV color system). The HSV color space is composed of three components of hue (Hue), saturation (Saturation, Chroma), and brightness (Brightness, Lightness, Value) (also referred to as HSB color space).

  The brightness V is the first element (representing a value related to brightness), and the value is represented by 0 to 100%.

  Hue H is the second element (representing a value related to color), and the value is represented by 0 to 360 °.

  Saturation S is the third element (representing a value related to vividness), and the value is represented by 0 to 100%.

  FIG. 1A and FIG. 2A show an example of a monospectral marker according to an embodiment of the present invention (indicated by reference numeral MK1). FIGS. 1A and 2A show exactly the same monospectral marker (only the positions where lines L1 and L2 are drawn are different).

  The monospectral marker MK1 has a square boundary. A narrow square frame-shaped boundary region (connection region) BD is provided inside the boundary. The boundary area BD is a black area (V = 0, H = 0, S = 0). The boundary area BD and the inside thereof are marker areas. The boundary area BD is a closed area surrounding the marker area.

  A total of nine segments are arranged inside the marker area surrounded by the boundary area BD. The four segments arranged at the four corners of the square are referred to as position segments and are represented by Sp1, Sp2, Sp3, and Sp4, respectively. The ID segments Sd1, Sd2, Sd3, Sd4, and Sd5 are arranged in the middle of these position segments and in the center of the marker area (ID = Identification).

  FIGS. 1B, 1C, and 1D show brightness V, hue H, and saturation S along line L1 passing through the centers of the top three segments Sp1, Sd1, and Sp2 in FIG. Shows the change in the value of. Similarly, FIGS. 2B, 2C, and 2D show brightness V, hue H, and saturation along line L2 passing through the centers of the three segments Sd2, Sd3, and Sd4 in the middle of FIG. The change of the value of degree S is shown.

  In FIG. 1 (B), the brightness V smoothly changes in a sine wave (cosine wave) along the line L1 from the value 0 (minimum value, basic value) in the boundary region BD at the left end, and the segments Sp1, Sd1 respectively. , Sp2 shows a peak, shows a minimum value in the middle of these segments, and reaches the rightmost boundary region BD. That is, the change in value V is represented by a single frequency (single spatial frequency) along the line L1. This is the significance of the monospectral marker.

  The frequency spectrum of this change in brightness V is shown in FIG. Since the marker MK1 is large enough to be visually recognized, the change in the brightness V is relatively moderate spatially, and the single frequency spectrum appears in a relatively low frequency region.

  Looking at the hue (H) in FIG. 1 (C), the hue (H) value of the position segments Sp1 and Sp2 is 60 ° (yellow), and the hue (H) value of the central ID segment Sd1 is 0 °. (Red).

  Looking at the saturation S in FIG. 1D, the maximum value 100 is uniformly shown inside the marker area excluding the boundary area BD. The saturation S is not necessarily the maximum value, but is preferably as high as possible.

  In FIG. 2 (B), the brightness V along the second line L2 of the monospectral marker MK1 also shows a change depicting a single frequency, and the frequency value is the same as that shown in FIG. 1 (A). is there. In FIG. 2C, the hues H of the ID segments Sd2, Sd3, and Sd4 are 120 ° (green), 180 ° (blue), and 0 ° (red), respectively. In FIG. 2D, the saturation S shows the maximum value.

  For the lowermost segments Sp3, Sd5, Sp4 of the monospectral marker MK1, the change in brightness V along the line passing through the center thereof is a sine wave having the same frequency as that shown in FIGS. 1 (B) and 2 (B). It is. The hues of these segments Sp3, Sd5, and Sp4 are 60 ° (yellow), 180 ° (blue), and 60 ° (yellow), respectively. Saturation S indicates the maximum value.

  The change in brightness V along the vertical direction of each segment of the monospectral marker MK1 (sequence of Sp1, Sd2, Sp3, sequence of Sd1, Sd3, Sd5, sequence of Sp2, Sd4, Sp4) is also a single sinusoidal change. The frequency value of this single frequency is the same as that shown in FIGS. 1 (B) and 2 (B).

  The change in brightness V along the array of diagonal segments (sequence of Sp1, Sd3, Sp4, sequence of Sp2, Sd3, Sp3) is also a sine wave at a single frequency, but the frequency is shown in FIG. It is 1 / √2 of the frequency shown in (B).

  In each segment (Sp1 to Sp4, Sd1 to Sd5) of the monospectral marker MK1, two directions (directions of the segments L1 and L2) and a vertical direction (direction orthogonal to the lines L1 and L2) In two directions orthogonal to the center, the value of brightness V changes at a single frequency with the same frequency value. In all directions passing through the center of each segment, the value of brightness V changes at a single frequency, but that frequency decreases with increasing angular distance from the two orthogonal directions, and in the rotated direction as described above. Is 1 / √2 of the frequency in the two orthogonal directions.

  Thus, each segment changes its brightness V at a single frequency in all directions passing through the center of each segment. The frequency changes continuously at different angles, but the frequency change is relatively narrow.

  All the segments Sp1 to Sp4 and Sd1 to Sd5 are independent from each other (separate or do not overlap). In other words, there is at least one pixel having a value of segment brightness V of 0 between the segments.

  For the ID segments Sd1 to Sd5, the change in the value of brightness V has a single frequency. As shown in an example later (FIG. 8), the brightness V of the ID segment is not necessarily required to have a single frequency. Alternatively, it may be a discontinuous change such as a square wave.

  Such a monospectral marker MK1 is represented on the surface of the medium having a two-dimensional surface or directly on the surface of the observation object. The medium includes a sheet or plate such as paper, plastic, a flat surface such as a wall, and a display screen.

  FIG. 4 shows another example of a monospectral marker. The monospectral marker MK2 is obtained by taking out the bottom three segments Sp3, Sd5, Sp4 of the monospectral marker MK1 and placing them in a border region having a very narrow width.

  FIG. 5 has an equilateral triangle frame-like boundary area, and three position segments Sp7, Sp8, Sp9 (the hue is yellow) are arranged at positions near the apex of the equilateral triangle in the boundary area. ID segments Sd7 (blue), Sd8 (red), and Sd9 (blue) are arranged between them. In every segment, its brightness shows a single frequency change in all directions through the center of each segment. The lightness peak value is the center of each segment, and the frequency value of a single frequency is equal in all directions through the center.

  FIG. 6 shows another example of a monospectral marker. This monospectral marker MK4 includes only two position segments Sp11 and Sp12. The lightness value V of each position segment Sp11, Sp12 shows a peak value at the center thereof, changes in all directions of 360 ° at the same single frequency, and the lightness value is 0 at the outermost periphery. The portion where the brightness value is 0 is the boundary region. The value of hue H of position segments Sp11 and Sp12 is 60 ° (yellow), and the value of saturation S is 100%. As described above, the monospectral marker may not be provided with the ID segment but may be configured only with the position segment.

  FIG. 7 shows another example of a monospectral marker. The monospectral marker MK5 includes a rectangular frame boundary region (connected region) BD and two position segments Sp13 and Sp14 provided in the boundary region BD apart from the boundary region. The values of the brightness V of the position segments Sp13 and Sp14 both change at the same single frequency in the orthogonal direction (there is a peak at the center), and a single value smaller than the single frequency as they rotate from these directions. It changes at one frequency (it becomes 1 / √2 times smaller at a position rotated by 45 °). The value of hue H of position segments Sp13 and Sp14 is 60 ° (yellow), and the value of saturation S is 100%. As described above, the boundary region BD and the position segments Sp13 and Sp14 may be separated from each other.

  In the monospectral marker MK6 shown in FIG. 8, two ID segments Sd11 and Sd12 are provided between the two position segments Sp13 and Sp14. The value V of the brightness V of these ID segments Sd11 and Sd12 does not change at a single frequency in a sine wave shape but discontinuously changes in a square wave shape and is constant at 100%. The ID segment Sd11 has a hue H value of 0 ° (red), and the ID segment Sd12 has a hue H value of 180 ° (blue). The value of the saturation S of these ID segments Sd11 and Sd12 is 100%. The relative positions of the two ID segments Sd11 and Sd12 with respect to the position segments Sp13 and Sp14 are predetermined (known).

  FIG. 10 shows an entire system for detecting a monospectral marker by imaging a scene (background, landscape, scene, field) including the above-described monospectral marker and analyzing image data obtained thereby.

  This system basically includes a digital camera 70 and a marker detection device 10.

  The marker detection device 10 is generally realized by a computer system (personal computer PC or other computer device). In FIG. 10, the functions (processes) performed by this computer system are represented in the form of blocks. That is, image conversion processing (means) 21, DC component removal processing (means) 22, n band pass filter processing (means) 31, 32,..., 3n (hereinafter abbreviated as BFP1, 2,..., N) ), The marker extraction process (means) 60 is all realized by software for operating a computer. In addition, the V memory 23, the V memory 41, 42,..., 4n (hereinafter abbreviated as V memory 1, 2,..., N), the H memory 51, and the S memory 52 are only described functionally. However, these memories are practically realized by one or several memories.

  FIG. 9 shows an example of a scene image captured by the digital camera 70. In this scene, a telephone 81, a book 82, a coffee cup 83, etc. are placed on a desk 80, and two monospectral markers MK11 and MK12 are further placed.

  RGB image data representing an image as shown in FIG. 9 is output from the digital camera 70 and applied to the marker detection device 10.

  In the marker detection device 10, RGB image data for one image input from the digital camera 70 is converted into V (lightness), H (hue), and S (saturation) data in the HSV color space by image conversion processing 21. Is done. Among the image data of lightness V, the direct current component removal processing 22 removes the direct current component, and the lightness V data after removal of the direct current component is stored in the V memory 23. This is because, as will be described later, in the process of comparing the brightness data before and after filtering by the BPF, the influence due to the presence of the DC component is removed in advance. Image data of hue H and saturation S are stored in the H memory 51 and S memory 52, respectively.

  BPF1 to n having a plurality of different passbands are prepared. This is due to the following reason. Although the size of the monospectral marker is constant, the size of the monospectral marker that appears on the captured image varies depending on the distance between the monospectral marker and the camera 70. As described above, the value V of the brightness V of at least the position segment of the monospectral marker has a single spatial frequency. On the captured image, the value of the monospectral marker varies depending on the size of the monospectral marker image. One frequency changes. Since the size of the monospectral marker image on the captured image is often unknown in advance, a BPF having a plurality of different passbands can be used in order to be able to cope with various image sizes of the monospectral marker. Is prepared. If the size of the monospectral marker image on the captured image is constant and known in advance, one BPF may be used.

  Even at least in the position segment of the monospectral marker, the value of the single frequency representing the change in the value of brightness V varies depending on the direction, and the single frequency of one monospectral marker has a certain width (band). ing. Therefore, each BPF 1, 2,..., N has a fixed pass band as shown in FIG. 11 (A) in correspondence with the single frequency band of the monospectral marker. In the case of the monospectral marker shown in FIG. 1 (A) and FIG. 2 (A), the single frequency of one monospectral marker is calculated from the single frequencies in two different directions as described above. √2 changes. Considering the case where the scene (monospectral marker) is photographed diagonally or the monospectral marker is tilted obliquely in the image (XY plane), the minimum frequency f1 and maximum frequency f3 of the passband of the BPF As a general guide, the ratio will be about 1 to 2.

  Filtering by BPF is performed while scanning the window WD (see FIG. 9) on the V image data. As shown in FIGS. 11A and 11B, the BPF window size (window size) (size of one side) corresponds to the minimum frequency f1 of the BPF pass band (one period M1 in the spatial domain). ) And the maximum frequency f3 (corresponding to one period M3 in the spatial region) is defined as one period M2 of the average value f2.

  The V data in the V memory 23 is filtered by a plurality of BPFs 1 to n, and the filtering results are stored in the V memories 1 to n, respectively.

  Using the data stored in the V memories 1 to n, the H memory 51, and the S memory 52, the monospectral marker is extracted by the marker extraction process 60 as follows.

  Referring to FIG. 12, as described above, V data is filtered by BPF1 to n having a plurality of different passbands (step S11, filtering means). The result is stored in the V memories 1 to n for all pixels for one screen (one image).

  Using all the data stored in these V memories 1 to n, the monospectral marker position segment candidate (first candidate) pixel is extracted (step S12). This process will be described in detail with reference to FIG.

  The difference between the V data before filtering by BPF1 (stored in the V memory 23) and the V data after filtering by BPF1 (stored in the V memory 1) is calculated, and a difference image is obtained ( This is the difference image 1, and the difference image data of a certain pixel is d1 (i, j)). Similarly, the difference between the V data before filtering by BPF2, ..., n and the V data after filtering by BPF2, ..., n (stored in V memory 2, ..., n) is calculated, The respective difference images are obtained (these are set as difference images 2,..., N, and the difference image data of a certain pixel is set as d2 (i, j),..., Dn (i, j)).

  Focusing on a specific pixel (i, j) of the difference images 1 to n, the smallest one of the difference data is obtained. Then, the minimum difference data is compared with a predetermined threshold (a value as small as possible), and a pixel smaller than the predetermined threshold is set as a position segment candidate (first candidate) with respect to the brightness V. This process is performed for all pixels of one image (captured image).

  A small difference in V data before and after filtering by BPF means that the V data of that pixel has passed through BPF. As described above, a plurality of BPFs having passbands corresponding to various sizes of monospectral markers on the captured image are prepared. For each pixel, it is determined whether the pixel has passed any BPF.

  Since the position segment has a value of brightness V that changes at a single frequency as described above, any BPF passes through any BPF if it is V data of a pixel representing the position segment. As a result, pixels that are likely to belong to the position segment are extracted. This is hardly affected by motion blur or defocus.

  The position segment has a specific hue value (H = 60 ° (yellow) in the above example) as described above. For the first candidate pixel that may belong to the position segment, it is checked whether the H data (stored in the H memory 51) is within a predetermined allowable range (step S13, second element). Determination means). If the H data is within a predetermined range of a predetermined H value, the pixel is determined as a candidate (second candidate) that may belong to the position segment. This check eliminates most of the image portion other than the monospectral marker included in the background image.

  Further, it is checked whether the pixel belongs to the position segment using the value of saturation S. The saturation of the pixels in the position segment is set to the maximum value, 100%, as described above. Therefore, it is checked whether the S data of the second candidate pixel exceeds a predetermined threshold (a value close to 100%), and the exceeding pixel is set as a candidate (third candidate) belonging to the position segment (step 3). S14). This eliminates image portions having properties similar to the position segments present in the background. The process of step S14 is not necessarily performed.

  In this way, since a set of pixels belonging to the position segment (collected in a certain range) is found (third candidate), the V data of the neighboring pixels in the V data of the third candidate pixel is found. The pixel corresponding to the peak is found (step S15, position segment center position candidate determining means). As described above, the lightness value V of the position segment changes sinusoidally, and the value of V is the highest at the center of the position segment. Through the processing in step S15, the center position of the position segment is obtained. Incidentally, peak detection fails in an image portion having an area where the value of brightness V is constant and does not change like a polka dot pattern (a peak cannot be detected).

  The border area is represented in black. Therefore, the brightness V is discriminated by a threshold value close to black, and pixels having V data having a value lower than the threshold value are extracted as a black region. If the pixels extracted as the black area are connected (continuous) to each other and form a closed area (closed loop), it is highly likely that the boundary area of the monospectral marker. In this way, the boundary area is extracted (step S16, marker area (boundary area) extracting means).

  In the boundary area extracted in step S16, it is determined whether or not there is a pixel of the center position candidate of the position segment determined in step S15. If there is, the number is counted. In the case of the monospectral marker shown in FIGS. 1 (A) and 2 (A), there are four position segments in the boundary region. Therefore, it is confirmed whether or not the number counted above matches the number of position segments of the monospectral marker, and if they match, the pixel of the center position candidate is set as the center position of the position segment. The center position of this position segment is used to obtain the center position of the ID segment in the processing shown in FIG. Also, the fact that the number of center positions of the position segments existing in the extracted boundary region matches the number of known monospectral marker position segments means that the boundary region and the position segment belong to the monospectral marker. This means that it can be determined (step S17, marker identification means).

  In this way, it is possible to identify (specify) the monospectral marker from the captured image only by extracting the position segment and the boundary region. The marker detection device 10 outputs data representing the position of the identified monospectral marker.

  FIG. 14 shows an example of processing for identifying a monospectral marker using an ID segment in addition to a position segment. The process shown in FIG. 12 identifies the monospectral marker using only the position segment (determines that the monospectral marker is present in the scene image and specifies the position on the image). In this process, the center position (position of the center pixel) of the position segment in the monospectral marker is obtained. Since the monospectral marker is identified by the number of position segments existing in the boundary region of the monospectral marker, two monospectral markers having the same number of position segments cannot be identified. However, if the hue values of the position segments in the two monospectral markers are different, they can be identified by the hue value.

  In the process shown in FIG. 14, based on the result of the process shown in FIG. 12, the ID segment is further used to identify the monospectral marker with higher accuracy, and two or more monospectral markers exist. In this case, these two or more monospectral markers can be distinguished from each other if the position segment number, the number, or the hue value H of the ID segment are the same, even if the position segment number is the same.

  Taking the monospectral marker MK1 shown in FIG. 1 (A) or FIG. 2 (A) as an example, in this monospectral marker MK1, position segments are arranged at positions corresponding to the four corners in a square boundary region. Five ID segments are arranged in the middle of these four position segments. Since the center positions (coordinate data) of the four position segments are obtained by the processing in FIG. 12, the position of the ID segment can be obtained using the relative positional relationship with the position segment based on this (step) S21).

  For example, even if the monospectral marker is tilted on the image as shown in Fig. 15 (the monospectral marker is slightly deformed into a parallelogram, rhombus, trapezoid, etc. depending on the shooting angle) In the coordinate system in which the center positions of the four position segments Sp1, Sp2, Sp3, Sp4 are [0, 1], [1, 1], [0, 0] [1, 0], the ID segment Sd1 , Sd2, Sd3, Sd4, and Sd5, the coordinates of the center positions are represented by [0.5, 1], [0, 0.5], [0.5, 0.5], [1, 0.5], and [0.5, 0], respectively. Based on the center position on the image of the four position segments detected by the processing of FIG. 12, the center position (pixel) on the image of the ID segment can be obtained through the conversion by the coordinate system.

  If the position on the image of the center position of the ID segment is obtained, the value of the hue H at the center position of the ID segment can be obtained from the hue H data stored in the H memory 51 (step S22). If all of the hue H values are within an allowable range centered on a predetermined hue H value (H = 120 for ID segment Sd2 (green), H = 180 for ID segment Sd3 (blue), etc.) (H pattern matching), it is determined to be a predetermined monospectral marker (step S23, marker identifying means). Of course, if even one of the five ID segments does not have a hue H within a known range, it is not determined to be a monospectral marker. Even if the center positions of the position segments are the same, two or more types of monospectral markers having different ID segment positions and different hue patterns can be distinguished from each other. The marker detection device 10 outputs data representing the identified monospectral marker, data representing its position, and the like.

  The monospectral marker can be expressed using a YUV color space, a Lab color space, or the like in addition to the HSV color space. In the case of the YUV color space, luminance Y is used as the first element, and color difference U or V is used as the second element. In the case of the Lab color space, the lightness L is used as the first element, and the red intensity a or the yellow intensity b is used as the second element.

  The monospectral marker and its detection method and apparatus according to the present invention can be used as an augmented reality marker as an example that can be applied in many fields. In this case, the augmented reality especially when an object moves is used. Realize the real value in realization (for example, surgical navigation, plant maintenance, robot position estimation, various games).

10 Marker detector
21 Image conversion processing
22 DC component removal treatment
23, 41, 42, 4n V memory
31, 32, 3n band pass filter (BPF)
51 H memory
52 S memory
60 Marker detection processing
70 Digital camera S11 Filtering processing (means)
S13 Second element determination process (means)
S15 Position segment center position candidate determination process (means)
S16 Marker area extraction process (means)
S17 Marker identification process (means)
S23 Marker identification processing (means)
MK1, MK2, MK3, MK4, MK5, MK6, MK11, MK12 monospectral markers Sp1, Sp2, Sp3, Sp4, Sp7, Sp8, Sp9, Sp11, Sp12, Sp13, Sp14 Position segments Sd1, Sd2, Sd3, Sd4, Sd5, Sd7, Sd8, Sd9, Sd11, Sd12 ID segment

Claims (7)

A marker represented on a medium having a two-dimensional surface using a color space including a first element representing a value related to brightness and a second element representing a value related to hue;
The marker has a marker area having a boundary where the value of the first element is a base value, and at least two independent position segments are arranged at predetermined positions in the marker area;
The position segment has a center position, and in all directions through the center position, the value of the first element shows a peak at the center position and a single value between the value of the peak and the base value. Changes with frequency,
The second element of the position segment has a value representing a particular hue;
Image data obtained by imaging a scene including a monospectral marker is converted into image data of the color space, and at least the value of the first element and the second element are converted for each pixel. And the value of
Filtering with a band pass filter having a pass band including a frequency corresponding to the single frequency with respect to the value of the first element to identify a pixel passing through the band pass filter;
Extracting the marker region for the value of the first element;
Determining whether the value of the second element is within a range representing the specific hue;
Among the pixels in which the value of the first element passes through the band pass filter and the value of the second element is within the range representing the specific hue, the value of the first element is Identify the peak pixel and determine the center position candidate for the position segment.
The number of center position candidates existing in the extracted marker region among the determined center position candidates is identified as a monospectral marker by matching with a predetermined number.
A monospectral marker detection method characterized by the above. An ID segment is further arranged in the marker region of the monospectral marker away from the position segment, and the second element of the ID segment has a value different from the value of the second element of the position segment Is,
When the number of center position candidates existing in the extracted marker region matches a predetermined number, the center position candidate is set as the center position of the position segment, and at least the ID segment is based on the center position. Determine the center position, and determine whether the second element value of the determined center position is within a predetermined range to determine whether it is a specific monospectral marker,
The method of detecting a monospectral marker according to claim 1 . Each of the values of the first element is filtered using a plurality of band pass filters having different pass bands depending on the distance from the imaging device to the monospectral marker, and passes through one of the band pass filters. The monospectral marker detection method according to claim 1 or 2 , wherein a pixel to be identified is specified. A marker represented on a medium having a two-dimensional surface using a color space including a first element representing a value related to brightness and a second element representing a value related to hue;
The marker has a marker area having a boundary where the value of the first element is a base value, and at least two independent position segments are arranged at predetermined positions in the marker area;
The position segment has a center position, and in all directions through the center position, the value of the first element shows a peak at the center position and a single value between the value of the peak and the base value. Changes with frequency,
The second element of the position segment has a value representing a particular hue;
Image data obtained by imaging a scene including a monospectral marker is converted into image data of the color space, and at least the value of the first element and the second element are converted for each pixel. Image conversion means for obtaining the value of
A pixel having a band pass filter having a pass band including a frequency corresponding to the single frequency, and filtering the value of the first element with the band pass filter and passing through the band pass filter Filtering means for identifying
Marker area extraction means for extracting the marker area for the value of the first element;
Second element determination means for determining whether or not the value of the second element is within a range representing the specific hue;
Among the pixels specified by the filtering means and the value of the second element within the range representing the specific hue by the second element determination means, the value of the first element Is extracted by the marker region extraction means from among the center position candidates determined by the center position candidate determination means, and the center position candidate determination means for determining the pixel indicating the peak and determining the center position candidate of the position segment Marker identifying means for identifying a monospectral marker by matching the number of center position candidates existing in the marker area with a predetermined number;
Monospectral marker detection device comprising: An ID segment is further arranged in the marker region of the monospectral marker away from the position segment, and the second element of the ID segment has a value different from the value of the second element of the position segment Is,
When the number of center position candidates existing in the marker area extracted by the marker area extracting means matches a predetermined number, the center position candidate is set as the center position of the position segment based on the center position. Marker identifying means for determining at least the center position of the ID segment and determining whether or not the value of the second element at the determined center position is within a predetermined range;
The monospectral marker detection apparatus according to claim 4 , further comprising: The filtering means includes a plurality of band pass filters having different pass bands depending on the distance from the imaging device to the monospectral marker, and the value of the first element is obtained by the plurality of band pass filters. The monospectral marker detection apparatus according to claim 4 or 5 , wherein each pixel is filtered to specify a pixel that passes through one of the band pass filters. The monospectral marker detection device according to any one of claims 4 to 6 , further comprising an imaging device that images the scene.