JP6508730B2 - Light emission marker device, marker detection device, transmission system, marker light emission method, marker detection method, and program - Google Patents

Description

  The present invention relates to a light emission marker device, a marker detection device, a transmission system, a marker light emission method, a marker detection method, and a program.

  In recent years, development of technology related to augmented reality has become popular. In order to realize augmented reality, it is necessary to associate the position of the user with the position of the target object. In order to do this, GPS (Global Positioning System) is mainly used outdoors, but since GPS can not be used indoors such as museums and art galleries, radio waves, infrared rays, image processing, etc. are used. In this regard, for example, Patent Document 1 discloses a technique for detecting a two-dimensional marker by image processing.

JP 2007-148558 A

  Recently, image processing for detecting a marker is often performed on an image obtained using a camera attached to a mobile device such as a smartphone. However, the implementation of such image processing has a problem of computational load for mobile devices and the like.

  An object of the present invention is, in view of the problems described above, a light emission marker device capable of suppressing a calculation load of image processing for detecting a marker, a marker detection device, a transmission system, a marker light emission method, a marker detection method, and a program To provide.

  A light emission marker device according to an aspect of the present invention is a light emission portion capable of emitting light with a plurality of light emission patterns, and light emission of displaying a frame with a light emission pattern of any of the plurality of light emission patterns at a predetermined frame rate. A light emission control means for performing control so that the unit performs, the light emission control means is a frame used for detecting a position of a marker by a marker detection device for acquiring an image obtained by photographing the light emission unit; Light emission in a first light emission pattern, light emission in a second light emission pattern of the detection frame, light emission in a predetermined light emission pattern of an information transfer frame which is a frame for transmitting information to the marker detection device, the marker The non-light emission as an end frame, which is a frame for notifying the detection device of the completion of the transmission of information, is sequentially repeated to the light emitting unit That.

  Further, a marker detection device according to one aspect of the present invention includes an image acquisition unit that acquires an image of the light emission unit of a light emission marker device having a light emission unit capable of emitting light with a plurality of light emission patterns; A difference image generation unit that generates a difference image of two acquired consecutive images, a search unit that searches for an existing range of the light emitting unit in the image based on the difference image, and two consecutive images Pattern analysis means for performing matching processing with a predetermined pattern image on partial images belonging to the existing range in an image captured later, information for outputting information according to the matching processing result by the pattern analysis means And output means.

  Further, a transmission system according to an aspect of the present invention includes a light emission marker device and a marker detection device, wherein the light emission marker device is a light emitting unit capable of emitting light with a plurality of light emission patterns, and at a predetermined frame rate. And a light emission control unit configured to control the light emitting unit to display a frame according to any one of a plurality of light emission patterns, and the light emission control unit detects the position of the marker by the marker detection device. Light emission in a first light emission pattern of a detection frame which is a frame used for light emission, light emission in a second light emission pattern of the detection frame, and an information transmission frame which is a frame for transmitting information to the marker detection device. Light emission with a predetermined light emission pattern, no emission as an end frame which is a frame for notifying the marker detection device of completion of transmission of information Are sequentially performed by the light emitting unit, and the marker detection device is an image acquiring unit that acquires an image obtained by photographing the light emitting unit, and a difference image of two consecutive images acquired by the image acquiring unit. In the image captured after the two consecutive images, the difference image generation means for generating a second image, the search means for searching for the existence range of the light emitting unit in the image based on the difference image, and It has pattern analysis means for performing a matching process with a predetermined pattern image on a partial image to which the image belongs, and information output means for outputting information according to the matching process result by the pattern analysis means.

  Further, in the marker light emission method according to one aspect of the present invention, continuous display of frames is performed at a predetermined frame rate by using light emission units capable of light emission with a plurality of light emission patterns, and in the continuous display, The marker detection device for acquiring an image obtained by photographing the light emitting unit causes the light emitting unit to emit light in a first light emission pattern, which is a frame used for detecting the position of the marker, and the detection frame is emitted as a second light emission Make the light emitting unit emit light in a pattern, and let the light emitting unit emit light in an information transmission frame, which is a frame for transmitting information to the marker detection device, in a predetermined light emission pattern, and complete transmission of information to the marker detection device A series of displays that cause the light emitting unit to perform no light emission as an end frame which is a frame for notification are sequentially repeated.

Further, in the marker detection method according to one aspect of the present invention, an image obtained by capturing the light emitting unit of a light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emission patterns is acquired. A difference image is generated, and the existence range of the light emitting unit in the image is searched based on the difference image, and a partial image belonging to the existence range in images captured after the two consecutive images, Perform matching processing with a predetermined pattern image,
Output information according to the matching process result.

  Further, a program according to an aspect of the present invention is obtained by an image acquisition step of acquiring an image obtained by imaging the light emitting portion of a light emitting marker device having a light emitting portion capable of emitting light with a plurality of light emission patterns; A difference image generation step of generating a difference image of two consecutive images, a search step of searching for the existence range of the light emitting unit in the image based on the difference image, and photographing after the two consecutive images Analysis step of performing matching processing with a predetermined pattern image on partial images belonging to the existing range in the processed image, and information output outputting information according to the matching processing result in the pattern analysis step Have the computer execute the steps.

  According to the present invention, it is possible to provide a light emission marker device, a marker detection device, a transmission system, a marker light emission method, a marker detection method, and a program capable of suppressing the calculation load of image processing for detecting a marker. .

FIG. 1 is a block diagram showing a schematic configuration of a transmission system according to an embodiment. It is a schematic diagram shown about the transmission system which has several light emission marker apparatus and several marker detection apparatus. It is a block diagram showing an example of the hardware constitutions of the luminescence marker device concerning an embodiment. It is a schematic diagram which shows an example of a light emission part. It is a block diagram which shows an example of the hardware constitutions of the marker detection apparatus concerning embodiment. It is a time chart which shows an example of the time transition of the luminescence pattern of a luminescence part. It is a schematic diagram which shows the example of a display in a display. It is a flowchart which shows an example of operation | movement of an image acquisition part. It is a flowchart which shows an example of operation | movement of a marker recognition part. It is a schematic diagram which shows a mode that a difference image is produced | generated from two sheets of images. It is a flowchart which shows an example of operation | movement of an information output part. It is a schematic diagram which shows the modification of the light emission pattern of an information transmission flame | frame. It is a schematic diagram which shows the other modification of the light emission pattern of an information transmission flame | frame. It is a time chart which shows an example of time transition of the luminescence pattern concerning a modification. It is a schematic diagram which shows the two-dimensional marker in ARToolKit.

  Before describing the embodiment, a marker detection method according to a comparative example will be described first. In the marker detection method according to this comparative example, a technique called ARToolKit is used to detect a marker from a captured image of a camera. In this method, a black square-shaped two-dimensional marker is used in a white square frame as shown in FIG. In this method, in order to detect a two-dimensional marker, the entire image of the two-dimensional marker captured by the camera is binarized. As a result, the black portion in the original image appears white, and the black portion appears white in the original image. Next, a white block is searched from the upper left of the obtained image. Then, when a white mass is found, pattern matching processing is performed on the partial image in the vicinity. A marker is detected by this. However, in this method, the pattern matching process is likely to be performed on a portion that is not a marker but is represented as a white block. As the number of black portions (that is, portions appearing as white blocks in a binarized image) increases in the image captured by the camera, the pattern matching process increases and the calculation load increases. In addition, since the two-dimensional marker is black and white, when the two-dimensional marker is placed in a dark place or the like, there is a problem that detection becomes difficult.

<Overview of Embodiment>
Next, an embodiment will be described. First, prior to the detailed description of the embodiment, an outline of the embodiment according to the present invention will be described. FIG. 1 is a block diagram showing a schematic configuration of a transmission system 1 according to the embodiment. As shown in FIG. 1, the transmission system 1 includes a light emission marker device 10 and a marker detection device 20. The transmission system 1 is a system in which the marker detection device 20 detects a marker displayed by the light emission marker device 10, and the marker detection device 20 outputs information according to the marker.

  The light emission marker device 10 has a light emission unit 11 and a light emission control unit 12. The light emission part 11 can be light-emitted by several light emission pattern, and each display aspect of the light emission part 11 makes a marker, respectively. The light emission control unit 12 controls the light emitting unit 11 to display a frame with a light emission pattern of any of the plurality of light emission patterns at a predetermined frame rate. The light emission pattern includes a non-light emission state in which light is not emitted.

  Here, under the control of the light emission control unit 12, the light emitting unit 11 displays three types of markers, a detection frame, an information transfer frame, and an end frame. The detection frame is a frame used by the marker detection device 20 to detect the position of the marker. The information transfer frame is a frame for transmitting information to the marker detection device 20. In other words, the information transfer frame is a frame for the marker detection device 20 to read information from the detected marker. The end frame is a frame for notifying the marker detection device 20 of the completion of transmission of information.

  The light emission control unit 12 emits light in the first light emission pattern of the detection frame, light emission in the second light emission pattern of the detection frame, light emission in a predetermined light emission pattern of the information transfer frame, and no light emission as an end frame. In order, it controls so that the light emission part 11 is repeatedly performed. Note that the first light emission pattern and the second light emission pattern are different.

  The marker detection device 20 includes an image acquisition unit 21, a marker recognition unit 22, and an information output unit 23, as shown in FIG. The image acquisition unit 21 acquires an image obtained by imaging the light emitting unit 11 of the light emission marker device 10. The marker recognition unit 22 includes a difference image generation unit 221, a search unit 222, and a pattern analysis unit 223.

  The difference image generation unit 221 generates a difference image of two consecutive images acquired by the image acquisition unit 21. The search unit 222 searches for the existing range of the light emitting unit 11 in the image based on the generated difference image. The pattern analysis unit 223 performs a process of matching a predetermined pattern image with a partial image belonging to the above existing range in images captured after two consecutive images used to generate the difference image. . The information output unit 23 outputs information according to the matching processing result by the pattern analysis unit 223.

  According to the transmission system 1, as described above, the light emission marker device 10 continuously displays markers while changing the light emission pattern. And the marker detection apparatus 20 produces | generates the difference image of the image which image | photographed the light emission part 11. FIG. In the difference image, the non-changed background image portion is lost, and the image portion of the light-emitting unit 11 having a change is extracted. Therefore, it becomes possible to easily identify at which position in the image the light emitting unit 11 is present. Therefore, it is possible to easily and accurately specify the partial image to be subjected to the pattern matching process. This can suppress the pattern matching process from being performed on the partial image not including the marker. That is, according to the transmission system 1, the calculation load of the image processing for detecting a marker can be suppressed.

<Details of Embodiment>
Next, details of the embodiment will be described. FIG. 2 is a schematic view showing a transmission system 1 having a plurality of light emission marker devices 10 and a plurality of marker detection devices 20. As shown in FIG. As shown in FIG. 2, the transmission system 1 may include a plurality of light emission marker devices 10 or may include a plurality of marker detection devices 20. Although three light emission marker devices 10 and three marker detection devices 20 are shown in FIG. 2, these numbers are an example, and the transmission system 1 includes one or more light emission marker devices 10 and one or more markers. The detection device 20 may be included.

  In the present embodiment, as shown in FIG. 2, the marker detection device 20 has a camera 201 and a display 202, and is, for example, a mobile terminal such as a smartphone. Therefore, in the present embodiment, it is assumed that the image acquisition unit 21 described above acquires an image captured by the camera 201, and the information output unit 23 described above displays and outputs information on the display 202. In addition, these are an example and the image acquisition part 21 may acquire the image transmitted to the marker detection apparatus 20 from the other apparatus provided with the camera. In this case, the marker detection device 20 may not have the camera 201. Further, the information output unit 23 may perform arbitrary output such as audio output as well as display output.

  FIG. 3 is a block diagram showing an example of the hardware configuration of the light emission marker device 10 according to the embodiment. The light emission marker device 10 includes a light emission unit 11, a memory 101, and a processor 102.

  The light emitting unit 11 is a dot matrix LED (Light Emitting Diode) as shown in FIG. 4, and the light emission pattern thereof can be freely controlled by the light emission control unit 12. In addition, although A-3880EG made from ParaLight is mentioned as an example of such LED, it is not restricted to this. In the present embodiment, as shown in FIG. 4, in order to enable a detection distance comparable to that of a two-dimensional marker in ARToolKit, an LED with 8 × 8 LEDs is used. The number of LEDs is an example, and can be arbitrarily changed according to the size of the LEDs used.

  The memory 101 is configured by a combination of volatile memory and non-volatile memory. Memory 101 may include storage located remotely from processor 102. In this case, the processor 102 may access the memory 101 via an input / output interface not shown. The memory 101 is used to store software (computer program) and the like executed by the processor 102.

  The processor 102 implements the light emission control unit 12 by reading software (computer program) from the memory 101 and executing it. Thus, the light emission marker device 10 has a function as a computer. The processor 102 may be, for example, a microprocessor, an MPU, or a CPU. Processor 102 may include multiple processors.

  The light emission marker device 10 is installed, for example, in the vicinity of an indoor exhibit such as a museum. The light emission marker device 10 repeats the display of the detection frame, the information transfer frame, and the end frame according to the light emission pattern set in advance. A specific example of the display of the light emission marker device 10 will be described later.

  FIG. 5 is a block diagram showing an example of the hardware configuration of the marker detection device 20 according to the embodiment. The marker detection device 20 includes a camera 201, a display 202, a memory 203, and a processor 204.

  The camera 201 is a digital camera provided with an imaging element such as a lens, a charge coupled device (CCD) sensor, or a complementary metal oxide semiconductor (CMOS).

  The display 202 is a display device that displays an arbitrary image, and may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

  The memory 203 is configured by a combination of volatile memory and non-volatile memory. Memory 203 may include storage located remotely from processor 204. In this case, the processor 204 may access the memory 203 via an input / output interface not shown. The memory 203 is used to store software (computer program) executed by the processor 204, an image captured by the camera 201, and data such as information to be displayed and output according to a marker.

  The processor 204 reads out software (computer program) from the memory 203 and executes the software to realize the image acquisition unit 21, the marker recognition unit 22, and the information output unit 23. Thus, the marker detection device 20 has a function as a computer. The processor 204 may be, for example, a microprocessor, an MPU, or a CPU. Processor 204 may include multiple processors.

  Note that the above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include tangible storage media of various types. Examples of non-transitory computer readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), Compact Disc Read Only Memory (CD-ROM), CD- R, CD-R / W, semiconductor memory (for example, mask ROM, Programmable ROM (PROM), Erasable PROM (EPROM), flash ROM, Random Access Memory (RAM)). Also, the programs may be supplied to the computer by various types of transitory computer readable media. Examples of temporary computer readable media include electrical signals, light signals, and electromagnetic waves. The temporary computer readable medium can provide the program to the computer via a wired communication path such as electric wire and optical fiber, or a wireless communication path.

  Next, control of the light emitting unit 11 by the light emission control unit 12 in the light emitting marker device 10 will be described. In the present embodiment, assuming that the shooting frame rate of the camera 201 of the marker detection device 20 is the most versatile 30 FPS (Frames Per Second), the light emission control unit 12 displays frames in the light emitting unit 11 at 30 FPS. Let me do it. As described above, in the present embodiment, the shooting frame rate of the camera 201 and the frame rate of the display of the light emitting unit 11 are the same. FIG. 6 is a time chart showing an example of the temporal transition of the light emission pattern of the light emitting unit 11. As described above, under the control of the light emission control unit 12, the light emitting unit 11 performs display in the order of the detection frame, the information transfer frame, and the end frame, and after the end frame is displayed, the detection frame, information transfer frame, end frame again Display in the order of. That is, the light emitting unit 11 repeats a series of displays. As described above, since the display is performed at 30 FPS, the light emission control unit 12 controls the light emission to perform the display of 33 milliseconds per frame.

  As shown in FIG. 6, the light emission control unit 12 causes the light emitting unit 11 to display the detection frame with the light emission pattern 52 after displaying the detection frame with the light emission pattern 51 on the light emission unit 11. Here, the light emission control unit 12 controls the light emission in the light emission pattern 51 and the light emission in the light emission pattern 52 to be one frame each. By doing this, even when the shooting frame rate of the camera 201 and the frame rate of the display of the light emitting unit 11 are not synchronized, that is, even when the timings of the two are different, the camera 201 The camera 201 can capture two consecutive images of different patterns as detection frames regardless of the shutter speed. That is, in the difference image of the two continuous images, the display of the light emitting unit 11 can be left. Therefore, the marker detection device 20 does not depend on the shutter speed of the camera 201, even when the shooting frame rate of the camera 201 and the frame rate of display of the light emitting unit 11 are not synchronized. It is possible to detect the position inside.

  This will be specifically described. For example, when the camera 201 shoots the light emitting unit 11 in the light emission pattern 51 in the first shooting and shoots the light emitting unit 11 in the light emission pattern 52 in the second shooting (ie, FIG. As described above, the exposure time T of the camera 201 falls within the time during which the state of the light emission pattern 51 is maintained during the first imaging, and within the time during which the state of the light emission pattern 52 is maintained during the second imaging. Of the light emission pattern 51 and the light emission pattern 52 as a detection frame.

  Also, for example, the camera 201 captures the light emitting unit 11 in the state of the light emission pattern 51 and the light emission pattern 52 in the first shooting, and the state of the light emission pattern 52 and the status of the light emission pattern 53 in the second photographing When the light emitting unit 11 is photographed (that is, the exposure time T of the camera 201 straddles the state of the light emission pattern 51 and the state of the light emission pattern 52 at the time of the first photographing, and at the time of the second In the case of straddling the state of and the state of the light emitting pattern 53), two images of the light emitting pattern 51 and the light emitting pattern 52 merged and the light emitting pattern 52 and the light emitting pattern 53 merged are obtained as detection frames.

  Also, for example, the camera 201 captures the light emitting unit 11 in the state of the light emission pattern 55 and the state of the light emission pattern 51 in the first shooting, and the state of the light emission pattern 51 and the state of the light emission pattern 52 in the second shooting When the light emitting unit 11 is photographed (that is, the exposure time T of the camera 201 straddles the state of the light emission pattern 55 and the state of the light emission pattern 51 at the time of the first photographing, and at the time of the second photographing In the case of straddling the state of and the state of the light emitting pattern 52), two images of the light emitting pattern 55 and the light emitting pattern 51 merged and the light emitting pattern 51 and the light emitting pattern 52 merged are obtained as detection frames.

  After the light emission control unit 12 causes the light emission unit 11 to display the detection frame in the light emission pattern 52, the light emission control unit 12 causes the light emission unit 11 to display an information transmission frame of a predetermined light emission pattern. The light emission pattern of the information transfer frame is different from the light emission pattern of the detection frame. In the present embodiment, the light emission control unit 12 performs control so that the same light emission pattern is displayed over a plurality of continuous frames in light emission with this predetermined light emission pattern. Specifically, for example, as shown in FIG. 6, the light emission pattern 53 which is a detection frame is displayed over two frames, and the light emission pattern 54 which is a detection frame is also displayed over two frames. By doing this, an image in which a predetermined light emission pattern is displayed is always obtained for one sheet regardless of when the photographing is performed at any timing. Therefore, the light emission marker device 10 can transmit a predetermined light emission pattern to the marker detection device 20. That is, the light emission marker device 10 can transmit the light emission pattern 53 and the light emission pattern 54, respectively.

  In the example shown in FIG. 6, two types of light emission patterns, light emission pattern 53 which is a light emission pattern for lighting four points, and light emission pattern 54 for lighting three points, are transmitted from the light emission marker device 10 to the marker detection device 20. It will be done. Thus, the transmission system 1 can arbitrarily increase or decrease the amount of information to be transmitted by changing the type of light emission pattern displayed as the information transmission frame.

  As described above, the display time of the information transfer frame is variable according to the amount of information to be transmitted. For this reason, it is necessary to notify completion of transmission of information. Therefore, the light emission marker device 10 displays the end frame. Specifically, the light emission control unit 12 causes the light emitting unit 11 to display a non-light emission state as an end frame. Here, the light emission control unit 12 controls to display the end frame over a plurality of frames. Specifically, for example, as shown in FIG. 6, the light emission pattern 55 which is the end frame is displayed over two frames. By doing this, it is possible to obtain an image in a state where no light is lit for one sheet, regardless of when the photographing is performed at any timing. Therefore, the marker detection device 20 can detect the completion of the transmission of the information from the light emission marker device 10.

  Next, the details of the marker detection device 20 will be described. The marker detection device 20 captures an image of the light emitting unit 11 of the light emission marker device 10 with the camera 201. Then, according to the light emission pattern transmitted from the light emission marker device 10, that is, according to the marker indicated by the light emission marker device 10, information is displayed on the display 202 as shown in FIG. In the example shown in FIG. 7, when the light emitting unit 11 of the light emitting marker device 10 installed near the object 60 is photographed by the camera 201 of the marker detection device 20, information 61 corresponding to the marker is displayed on the display 202. It shows how it is displayed.

  This is done as follows. First, in the marker detection device 20, dedicated application software for using the camera 201 is activated. After activation, the camera 201 is directed such that the light emitting unit 11 of the light emitting marker device 10 appears. When the light emitting unit 11 is photographed by the camera 201, information 61 corresponding to the detected marker is displayed on the display 202. The marker detection device 20 repeats such marker search and display of information.

  As described above, a plurality of light emission marker devices 10 may exist in the transmission system 1. The marker detection device 20 can individually recognize each of the plurality of light emission marker devices 10 even if the light emission units 11 of the plurality of light emission marker devices 10 are simultaneously photographed. Moreover, it is also possible to image | photograph the light emission part 11 of one light emission marker apparatus 10 with the camera 201 of the several marker detection apparatus 20, respectively, and to display the information according to a marker in each marker detection apparatus 20. FIG.

  As shown in FIG. 1, the marker detection device 20 has an image acquisition unit 21, a marker recognition unit 22, and an information output unit 23 as internal functions. The details of these will be described below.

  The image acquisition unit 21 uses the camera 201 and stores an image captured by the camera 201 in a shared memory (memory 203). Photographing by the camera 201 is continuously performed, and storage of the image in the shared memory is also continuously performed.

  FIG. 8 is a flowchart showing an example of the operation of the image acquisition unit 21. In step 10 (S10), the image acquisition unit 21 activates the camera 201. Subsequently, at Step 11 (S11), the image acquisition unit 21 captures an image to acquire image data, and holds the image data in the shared memory. The image acquisition unit 21 repeatedly performs the operation of step 11 to continuously acquire image data.

  The marker recognition unit 22 has a function of determining from the image acquired by the image acquisition unit 21 whether a marker is present or not. The marker recognition unit 22 includes the difference image generation unit 221, the search unit 222, and the pattern analysis unit 223 as described above. FIG. 9 is a flowchart showing an example of the operation of the marker recognition unit 22. The processing of the marker recognition unit 22 will be described below with reference to FIG. As shown in FIG. 9, in the processing of the marker recognition unit 22, first, the processing of the difference image generation unit 221 is performed, then, the processing of the search unit 222 is performed, and finally, the processing of the pattern analysis unit 223 is performed. Processing is performed.

  The difference image generation unit 221 calculates a difference image of two consecutive images acquired by the image acquisition unit 21. Therefore, first, in step 20 (S20), the difference image generation unit 221 acquires two images from the shared memory. Next, in step 21 (S21), as shown in FIG. 10, the difference image generation unit 221 calculates the difference between the pixels of the two images acquired in step 20, and acquires a difference image. Here, FIG. 10 shows that the background object 73 is removed by taking the difference between the image 71 and the image 72, and in the difference image 74, only the light emission pattern of the light emitting unit 11 is extracted as the difference. . In step 21, in order to reduce the processing load, the difference image generation unit 221 compares, for example, only the R value among the RGB values and takes a difference. In step 21, the difference image generation unit 221 converts the obtained difference image into a black and white binarized image. After step 21, the process shifts to step 22 by the search unit 222.

  The search unit 222 searches for a marker based on the difference image. More specifically, the search unit 222 first searches for the existing range of the light emitting unit 11 in the image based on the difference image. Specifically, in step 22 (S22), the search unit 222 searches for a pixel group of a black block having a size equal to or larger than a predetermined size (for example, 10 pixels × 10 pixels) in order to suppress the false detection rate. . If such a pixel group is found, the search unit 222 determines that the existing range of the pixel group is the existing range of the light emitting unit 11. That is, in step 23 (S23), the search unit 222 determines whether or not the pixel group of the black block is present in the difference image. When the pixel group of the black block does not exist in the difference image (No in step 23), the process returns to step 20, two images are acquired from the shared memory, and the same process is performed again. These processes are repeated until the light emitting unit 11 is found in the difference image. When the pixel group of the black block is present in the difference image (Yes in Step 23), the search unit 222 holds the existing range of the light emitting unit 11 in Step 24 (S24). Specifically, the search unit 222 temporarily stores it in the memory 203 or the like.

  If the search unit 222 can detect the existing range of the light emitting unit 11 in the difference image, subsequently, the pattern analysis unit 223 analyzes the marker pattern. That is, after step 24, the process proceeds to step 25 by the pattern analysis unit 223.

  The pattern analysis unit 223 analyzes the pattern of the marker with respect to the images captured after two consecutive images used to generate the difference image. Specifically, first, in step 25 (S25), the pattern analysis unit 223 newly acquires an image from the shared memory in order to acquire the light emission pattern of the information transfer frame. In the present embodiment, the pattern analysis unit 223 acquires, for example, images for two sheets. Thereby, the pattern analysis unit 223 can always acquire the image of the light emission pattern displayed by the light emission marker device 10 as the information transfer frame.

  Subsequently, in step 26 (S26), the pattern analysis unit 223 performs trimming processing on the image acquired in step 25 so as to include the existing range of the light emitting unit 11 held in step 24. Thus, the pattern analysis unit 223 limits the target range of the image to be subjected to the matching process. The pattern analysis unit 223 also converts RGB values into HSV values for the image after trimming. This is because the matching process is performed using the luminance information so as to easily identify the light emitting part.

  After step 26, in step 27 (S27), the pattern analysis unit 223 performs a process of matching the image subjected to the process in step 26 with a pattern image of a predetermined marker. Note that, as this matching process, any known pattern matching process can be used. When a pattern image of a predetermined marker is found in the image (Yes in Step 28), in Step 29 (S29), the pattern analysis unit 223 holds an ID number corresponding to the found pattern. Specifically, the pattern analysis unit 223 temporarily stores it in the memory 203 or the like. Thereafter, the process returns to step 25 and two more images are obtained from the shared memory. That is, the same process is performed to obtain the pattern of the next information transmission frame. On the other hand, when the pattern image of the predetermined marker is not found (No in step 28), in step 30 (S30), the pattern analysis unit 223 emits no light in the image acquired in step 25. It is determined that the image is an end frame that has not been made. Subsequently, in step 31 (S31), the pattern analysis unit 223 notifies the information output unit 23 of the ID number held so far. After step 31, the process shifts to the process by the information output unit 23.

  The information output unit 23 displays and outputs information such as CG (Computer Graphics) according to the matching processing result by the pattern analysis unit 223 on the display 202. Specifically, the information output unit 23 displays information corresponding to the ID number notified from the pattern analysis unit 223.

  FIG. 11 is a flowchart showing an example of the operation of the information output unit 23. In step 40 (S40), the information output unit 23 searches for information corresponding to the ID number notified from the search unit 222. This information is stored, for example, in the memory 203 or the like. Subsequently, at step 41 (S41), the information output unit 23 displays the information searched at step 40 on the display 202. The information output unit 23 updates the information to be displayed on the display 202 each time a new ID number is notified from the search unit 222 by repeatedly performing these processes.

  The embodiment has been described above. In the present embodiment, the existing range of the light emitting unit 11 is extracted from the difference image of the two images. Then, the pattern matching process is performed limited to this existing range. Therefore, it is possible to suppress the number of times the pattern matching process is performed on the image of the portion where the marker is not present, and to suppress the calculation load.

  In the two-dimensional marker in ARToolKit, only a single pattern is displayed, so only single information can be transmitted. In addition, there are many dark places in indoor museums and science museums, etc. In such dark places, two-dimensional markers could not be detected correctly in ARToolKit. On the other hand, in the present embodiment, an arbitrary amount of information can be transmitted from the light emission marker device 10 to the marker detection device 20 by the information transmission frame. Further, since the marker is indicated by light emission, the marker detection device 20 can recognize the marker even in a dark place.

  The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the scope of the present invention. For example, in the above embodiment, the frame rate was 30 FPS, but may be another frame rate such as 60 FPS. In addition, when the LED blinks, it is easy for the human eye to notice and has an influence on the vision. However, the influence on the sight can be suppressed by increasing the frame rate like 60 FPS or the like.

  In the above embodiment, the 8 × 8 dot matrix LED is used, but it is also possible to configure the light emitting unit 11 using more LEDs, such as a 16 × 16 dot matrix LED is there. The light emitting unit 11 can also emit light using a liquid crystal panel instead of the LED.

  In addition, the light emission pattern shown in the embodiment is an example, and the light emission pattern can be variously changed. For example, as shown in FIG. 12, it is also possible to represent information by a light emission pattern like a trapezoid. Further, as shown in FIG. 13, it is also possible to represent information by a light emission pattern such as a circle. The time transition of the whole light emission pattern of the light emission part 11 when the information transmission flame | frame of these two light emission patterns is used is shown in FIG. Thus, the light emission pattern can be set arbitrarily.

Reference Signs List 1 transmission system 10 light emission marker device 11 light emission unit 12 light emission control unit 20 marker detection device 21 image acquisition unit 22 marker recognition unit 23 information output unit 201 camera 202 display 221 difference image generation unit 222 search unit 223 pattern analysis unit

Claims (9)

A light emitting portion capable of emitting light with a plurality of light emitting patterns;
A light emission control unit configured to control the light emitting unit to display a frame by a light emission pattern of any of the plurality of light emission patterns at a predetermined frame rate;
The light emission control means emits light in a first light emission pattern of a detection frame, which is a frame used for detecting a position of a marker by a marker detection device for acquiring an image obtained by photographing the light emission unit, the first of the detection frame 2. Light emission in a light emission pattern of 2, light emission in a predetermined light emission pattern of an information transfer frame which is a frame for transmitting information to the marker detection device, and a frame for notifying the marker detection device of completion of transmission of information A light emission marker device, which causes the light emitting unit to repeatedly perform no light emission as an end frame. The marker detection device acquires an image obtained by photographing the light emitting unit at the predetermined frame rate,
The light emission marker device according to claim 1, wherein the light emission control means controls light emission in the first light emission pattern and light emission in the second light emission pattern to be one frame each. The light emission marker device according to claim 2, wherein the light emission control means controls to display the same light emission pattern over a plurality of continuous frames in light emission with the predetermined light emission pattern. The light emission marker device according to claim 2 or 3, wherein the light emission control means controls to display the end frame over a plurality of frames. An image acquisition unit configured to acquire an image of the light emitting unit of the light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emitting patterns;
A difference image generation unit that generates a difference image of two consecutive images acquired by the image acquisition unit;
Search means for searching for the existing range of the light emitting unit in the image based on the difference image;
Pattern analysis means for performing a matching process with a predetermined pattern image on partial images belonging to the existing range in images taken after the two consecutive images;
A marker detection device comprising: information output means for outputting information according to the matching processing result by the pattern analysis means. A light emission marker device and a marker detection device;
The light emission marker device is
A light emitting portion capable of emitting light with a plurality of light emitting patterns;
A light emission control unit configured to control the light emitting unit to display a frame by a light emission pattern of any of the plurality of light emission patterns at a predetermined frame rate;
The light emission control means emits light in a first light emission pattern of a detection frame which is a frame used for detecting the position of a marker by the marker detection device, light emission in a second light emission pattern of the detection frame, Light emission in a predetermined light emission pattern of an information transfer frame which is a frame for transmitting information to a marker detection device, no light emission as an end frame which is a frame for notifying the marker detection device of completion of transmission of information , In order, let the light emitting unit repeat,
The marker detection device
An image acquisition unit configured to acquire an image obtained by photographing the light emitting unit;
A difference image generation unit that generates a difference image of two consecutive images acquired by the image acquisition unit;
Search means for searching for the existing range of the light emitting unit in the image based on the difference image;
Pattern analysis means for performing a matching process with a predetermined pattern image on partial images belonging to the existing range in images taken after the two consecutive images;
A transmission system comprising: information output means for outputting information according to the matching processing result by the pattern analysis means. Continuous display of frames at a predetermined frame rate is performed using a light emitting unit capable of emitting light with a plurality of light emission patterns,
In the continuous display,
A marker detection apparatus for acquiring an image obtained by photographing the light emitting unit causes the light emitting unit to emit light in a first light emission pattern, which is a frame used for detecting the position of the marker.
Causing the light emitting unit to emit light in the second light emission pattern;
Causing the light emitting unit to emit an information transmission frame, which is a frame for transmitting information to the marker detection device, in a predetermined light emission pattern;
A marker light emission method comprising sequentially repeating a series of displays in which the light emitting unit is caused to emit no light as an end frame which is a frame for notifying the marker detection device of completion of transmission of information. Obtaining an image of the light emitting unit of the light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emitting patterns;
Generate a difference image of two consecutive acquired images,
Searching for the existing range of the light emitting unit in the image based on the difference image;
Performing a matching process with a predetermined pattern image on partial images belonging to the existing range in images captured after the two consecutive images;
A marker detection method that outputs information according to the matching processing result. An image acquisition step of acquiring an image obtained by photographing the light emitting unit of a light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emission patterns;
A difference image generation step of generating a difference image between two consecutive images acquired in the image acquisition step;
A search step of searching for the existing range of the light emitting unit in the image based on the difference image;
A pattern analysis step of performing a matching process with a predetermined pattern image on partial images belonging to the existing range in images captured after the two consecutive images;
A program for causing a computer to execute an information output step of outputting information according to the matching processing result in the pattern analysis step.

Images