JP2018098716A - Light emitting marker device, marker detection device, transmission system, marker light emission method, marker detection method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a calculation load of image processing for detecting a marker.SOLUTION: A light emitting marker device 10 includes a light emission part 11 for enabling light emission with a plurality of light emitting patterns, and a light emission control part 12 for controlling the light emission part 11 so as to display a frame by any light emitting pattern among the plurality of light emitting patterns at a prescribed frame rate. The light emission control part 12 sequentially and repeatedly causes the light emission control part 11 to perform light emission of a first light emitting pattern of a detection frame, light transmission of a second light emitting pattern of the detection frame, light emission of a prescribed light emitting pattern of an information transmission frame, and non-light emission as a final frame.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a light emitting marker device, a marker detecting device, a transmission system, a marker light emitting method, a marker detecting method, and a program.

  In recent years, technology related to augmented reality has been actively developed. In order to realize augmented reality, it is necessary to associate the position of the user with the position of the target object. In order to do this, GPS (Global Positioning System) is mainly used outdoors, but since GPS cannot be used indoors such as museums and art galleries, radio waves, infrared rays, image processing, or the like is used. In this regard, for example, Patent Document 1 discloses a technique for detecting a two-dimensional marker by image processing.

JP 2007-148558 A

  Recently, image processing for detecting a marker is often performed on an image obtained by using a camera attached to a mobile device such as a smartphone. However, for mobile devices and the like, there is a problem of calculation load in performing such image processing.

  In view of the above-described problems, an object of the present invention is to provide a light emitting marker device, a marker detecting device, a transmission system, a marker light emitting method, a marker detecting method, and a program capable of suppressing the calculation load of image processing for detecting a marker. Is to provide.

  A light-emitting marker device according to an aspect of the present invention includes: a light-emitting unit that can emit light with a plurality of light-emitting patterns; and a light-emitting unit that displays a frame display using any one of the plurality of light-emitting patterns at a predetermined frame rate. A light emission control unit that controls the detection unit to perform a detection frame that is a frame used by a marker detection device that acquires an image of the light emission unit to detect the position of the marker. Light emission in a first light emission pattern, light emission in a second light emission pattern of the detection frame, light emission in a predetermined light emission pattern of an information transmission frame which is a frame for transmitting information to the marker detection device, the marker Non-light emission as an end frame, which is a frame for notifying the detection device of the completion of transmission of information, is sequentially repeated on the light emitting unit. That.

  The marker detection device according to one aspect of the present invention includes an image acquisition unit that acquires an image obtained by photographing the light emitting unit of a light emitting marker device having a light emitting unit that can emit light in a plurality of light emission patterns, and the image acquisition unit. A difference image generating means for generating a difference image between the acquired two consecutive images, a search means for searching for an existence range of the light emitting unit in the image based on the difference image, and the two consecutive images. Pattern analysis means for performing a matching process with a predetermined pattern image on a partial image belonging to the existence range in an image captured later, and information for outputting information according to a matching process result by the pattern analysis means Output means.

  The transmission system according to one aspect of the present invention includes a light emitting marker device and a marker detection device, and the light emitting marker device includes a light emitting unit capable of emitting light with a plurality of light emission patterns, and a predetermined frame rate. A light emission control unit that controls the light emission unit to display a frame using any one of a plurality of light emission patterns, and the light emission control unit detects the position of the marker by the marker detection device. Light emission in the first light emission pattern of the detection frame which is a frame used for the light emission, light emission in the second light emission pattern of the detection frame, information transmission frame which is a frame for transmitting information to the marker detection device Light emission in a predetermined light emission pattern, non-emergence as an end frame that is a frame for notifying the marker detection device of completion of information transmission Are sequentially performed by the light emitting unit, and the marker detection device is configured to acquire an image obtained by capturing the light emitting unit, and a difference image between two consecutive images acquired by the image acquiring unit. A difference image generation means for generating the image, a search means for searching for an existence range of the light emitting unit in the image based on the difference image, and the existence range in an image photographed after the two consecutive images. Pattern analysis means for performing a matching process with a predetermined pattern image for the partial image to which the image belongs, and an information output means for outputting information according to the matching process result by the pattern analysis means.

  Further, in the marker light emission method according to one aspect of the present invention, the light emitting unit capable of emitting light with a plurality of light emission patterns is used to perform continuous display of frames at a predetermined frame rate, and in the continuous display, A marker detection device that acquires an image obtained by photographing a light emitting unit causes the light emitting unit to emit light with a first light emission pattern as a detection frame that is a frame used to detect the position of the marker, and the detection frame is subjected to second light emission. The light emitting unit is caused to emit light in a pattern, and an information transmission frame, which is a frame for transmitting information to the marker detecting device, is caused to emit light to the light emitting unit in a predetermined light emitting pattern, and the marker detecting device completes transmission of information. A series of displays for causing the light emitting unit to perform no light emission as an end frame, which is a frame for notification, is sequentially repeated.

Further, in the marker detection method according to one aspect of the present invention, an image obtained by photographing the light emitting unit of the light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emission patterns is acquired, and two consecutive images acquired are acquired. A difference image is generated, and based on the difference image, the existence range of the light emitting unit in the image is searched, and for a partial image belonging to the existence range in an image taken after the two consecutive images, Perform a matching process with a predetermined pattern image,
Information corresponding to the matching processing result is output.

  The program according to one embodiment of the present invention is acquired in an image acquisition step of acquiring an image obtained by photographing the light emitting unit of a light emitting marker device having a light emitting unit capable of emitting light with a plurality of light emission patterns, and the image acquiring step. A difference image generation step for generating a difference image between two consecutive images, a search step for searching for an existence range of the light emitting unit in the image based on the difference image, and shooting after the two consecutive images A pattern analysis step for performing a matching process with a predetermined pattern image on a partial image belonging to the existence range in the generated image, and an information output for outputting information according to a matching process result in the pattern analysis step Causing the computer to execute the steps.

  ADVANTAGE OF THE INVENTION According to this invention, the light emission marker apparatus, marker detection apparatus, transmission system, marker light emission method, marker detection method, and program which can suppress the calculation load of the image process for detecting a marker can be provided. .

1 is a block diagram showing a schematic configuration of a transmission system according to an embodiment. It is a schematic diagram shown about the transmission system which has a some light emission marker apparatus and a some marker detection apparatus. It is a block diagram which shows an example of the hardware constitutions of the light emission marker apparatus concerning embodiment. It is a schematic diagram which shows an example of a light emission part. It is a block diagram which shows an example of the hardware constitutions of the marker detection apparatus concerning embodiment. It is a time chart which shows an example of the time transition of the light emission pattern of a light emission part. It is a schematic diagram which shows the example of a display in a display. It is a flowchart which shows an example of operation | movement of an image acquisition part. It is a flowchart which shows an example of operation | movement of a marker recognition part. It is a schematic diagram which shows a mode that a difference image is produced | generated from two images. It is a flowchart which shows an example of operation | movement of an information output part. It is a schematic diagram which shows the modification of the light emission pattern of an information transmission frame. It is a schematic diagram which shows the other modification of the light emission pattern of an information transmission frame. It is a time chart which shows an example of the time transition of the light emission pattern concerning a modification. It is a schematic diagram which shows the two-dimensional marker in ARTToolKit.

  Before describing an embodiment, first, a marker detection method according to a comparative example will be described. In the marker detection method according to this comparative example, a technique called ARTToolKit is used, and a marker is detected from a captured image of a camera. In this method, a two-dimensional marker having a black square shape is used in a white square frame as shown in FIG. In this method, in order to detect a two-dimensional marker, the entire image obtained by photographing the two-dimensional marker with a camera is binarized. As a result, an image in which the portion that was black in the original image is represented in white and the portion that was white in the original image is represented in black is obtained. Next, a white block is searched from the upper left of the obtained image. When a white block is found, pattern matching processing is performed on a partial image in the vicinity. Thereby, a marker is detected. However, with this method, there is a high possibility that pattern matching processing is performed on a portion that is not a marker but is represented as a white lump. As the number of black portions (that is, the portions that appear as white blocks in the binarized image) in the image photographed by the camera increases, the pattern matching processing increases and the calculation load increases. In addition, since the two-dimensional marker is black and white, there is a problem that detection becomes difficult when the two-dimensional marker is placed in a dark place or the like.

<Outline of the embodiment>
Next, embodiments will be described. First, prior to detailed description of the embodiment, an outline of the embodiment according to the present invention will be described. FIG. 1 is a block diagram illustrating a schematic configuration of a transmission system 1 according to the embodiment. As shown in FIG. 1, the transmission system 1 includes a light emitting marker device 10 and a marker detection device 20. The transmission system 1 is a system in which the marker detection device 20 detects a marker displayed by the light emitting marker device 10 and the marker detection device 20 outputs information corresponding to the marker.

  The light emitting marker device 10 includes a light emitting unit 11 and a light emission control unit 12. The light emitting unit 11 can emit light with a plurality of light emitting patterns, and each display mode of the light emitting unit 11 serves as a marker. The light emission control unit 12 controls the light emission unit 11 to display a frame using any one of the plurality of light emission patterns at a predetermined frame rate. Note that the light emission pattern includes a non-light emission state in which light emission is not performed.

  Here, the light emitting unit 11 displays three types of markers, that is, a detection frame, an information transmission frame, and an end frame under the control of the light emission control unit 12. The detection frame is a frame used by the marker detection device 20 to detect the position of the marker. The information transmission frame is a frame for transmitting information to the marker detection device 20. In other words, the information transmission frame is a frame for the marker detection device 20 to read information from the detected marker. The end frame is a frame for notifying the marker detection device 20 of the completion of information transmission.

  The light emission control unit 12 emits light with the first light emission pattern of the detection frame, light emission with the second light emission pattern of the detection frame, light emission with a predetermined light emission pattern of the information transmission frame, and no light emission as the end frame. In order, control is performed so that the light emitting unit 11 is repeatedly performed. Note that the first light emission pattern and the second light emission pattern are different.

  As shown in FIG. 1, the marker detection device 20 includes an image acquisition unit 21, a marker recognition unit 22, and an information output unit 23. The image acquisition unit 21 acquires an image obtained by photographing the light emitting unit 11 of the light emitting marker device 10. The marker recognition unit 22 includes a difference image generation unit 221, a search unit 222, and a pattern analysis unit 223.

  The difference image generation unit 221 generates a difference image between two consecutive images acquired by the image acquisition unit 21. The search unit 222 searches for the existence range of the light emitting unit 11 in the image based on the generated difference image. The pattern analysis unit 223 performs a matching process with a predetermined pattern image on a partial image belonging to the above-described existence range in an image photographed after two consecutive images used for generating a difference image. . The information output unit 23 outputs information corresponding to the matching processing result by the pattern analysis unit 223.

  According to the transmission system 1, the light emission marker apparatus 10 displays a marker continuously, changing a light emission pattern as above-mentioned. And the marker detection apparatus 20 produces | generates the difference image of the image which image | photographed the light emission part 11. FIG. In the difference image, the background image portion without change is lost, and the image portion of the light emitting unit 11 with change is extracted. For this reason, it is possible to easily specify at which position in the image the light emitting unit 11 exists. Therefore, the partial image to be subjected to the pattern matching process can be easily and accurately specified. This can suppress the pattern matching process being performed on the partial image that does not include the marker. That is, according to the transmission system 1, it is possible to suppress the calculation load of image processing for detecting the marker.

<Details of the embodiment>
Next, details of the embodiment will be described. FIG. 2 is a schematic diagram showing a transmission system 1 having a plurality of light emitting marker devices 10 and a plurality of marker detection devices 20. As illustrated in FIG. 2, the transmission system 1 may include a plurality of light emitting marker devices 10 or a plurality of marker detection devices 20. In FIG. 2, three light emitting marker devices 10 and three marker detection devices 20 are shown. However, these numbers are merely examples, and the transmission system 1 includes one or more light emitting marker devices 10 and one or more markers. The detection device 20 may be included.

  In the present embodiment, as shown in FIG. 2, the marker detection device 20 includes a camera 201 and a display 202, and is, for example, a mobile terminal such as a smartphone. Therefore, in the present embodiment, description will be made assuming that the above-described image acquisition unit 21 acquires an image captured by the camera 201 and the above-described information output unit 23 displays and outputs information on the display 202. Note that these are examples, and the image acquisition unit 21 may acquire an image transmitted to the marker detection device 20 from another device including a camera. In this case, the marker detection device 20 may not include the camera 201. Moreover, the information output part 23 may perform arbitrary outputs, such as not only a display output but an audio | voice output.

  FIG. 3 is a block diagram illustrating an example of a hardware configuration of the light emitting marker device 10 according to the embodiment. The light emitting marker device 10 includes a light emitting unit 11, a memory 101, and a processor 102.

  The light emitting unit 11 is a dot matrix LED (Light Emitting Diode) as shown in FIG. 4, and the light emission pattern can be freely controlled by the light emission control unit 12. An example of such an LED is A-3880EG manufactured by ParaLight, but is not limited thereto. In the present embodiment, as shown in FIG. 4, in order to enable a detection distance comparable to that of a two-dimensional marker in ARTToolKit, an LED having 8 × 8 is used. In addition, about the number of LED, it is an example and can be changed arbitrarily according to the size etc. of LED to be used.

  The memory 101 is configured by a combination of a volatile memory and a nonvolatile memory. Memory 101 may include a storage that is located remotely from processor 102. In this case, the processor 102 may access the memory 101 via an input / output interface (not shown). The memory 101 is used for storing software (computer program) executed by the processor 102.

  The processor 102 reads the software (computer program) from the memory 101 and executes it, thereby realizing the light emission control unit 12. As described above, the light emitting marker device 10 has a function as a computer. The processor 102 may be, for example, a microprocessor, MPU, or CPU. The processor 102 may include a plurality of processors.

  For example, the light emitting marker device 10 is installed in the vicinity of an indoor exhibit such as a museum. The light emitting marker device 10 repeats display of the detection frame, the information transmission frame, and the end frame according to a preset light emission pattern. A specific example of the display of the light emitting marker device 10 will be described later.

  FIG. 5 is a block diagram illustrating an example of a hardware configuration of the marker detection device 20 according to the embodiment. The marker detection device 20 includes a camera 201, a display 202, a memory 203, and a processor 204.

  The camera 201 is a digital camera including an imaging element such as a lens, a CCD (Charge Coupled Device) sensor, or a CMOS (Complementary Metal Oxide Semiconductor).

  The display 202 is a display device that displays an arbitrary image, and may be, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

  The memory 203 is configured by a combination of a volatile memory and a nonvolatile memory. Memory 203 may include storage located remotely from processor 204. In this case, the processor 204 may access the memory 203 via an input / output interface (not shown). The memory 203 is used for storing software (computer program) executed by the processor 204, images taken by the camera 201, data such as information displayed and output in accordance with the marker, and the like.

  The processor 204 reads out and executes software (computer program) from the memory 203, thereby realizing the image acquisition unit 21, the marker recognition unit 22, and the information output unit 23. As described above, the marker detection device 20 has a function as a computer. The processor 204 may be, for example, a microprocessor, MPU, or CPU. The processor 204 may include a plurality of processors.

  The above-described program can be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (eg flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg magneto-optical discs), compact disc read only memory (CD-ROM), CD-ROMs. R, CD-R / W, and semiconductor memory (for example, mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)) are included. The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  Next, control of the light emission part 11 by the light emission control part 12 in the light emission marker apparatus 10 is demonstrated. In the present embodiment, assuming that the shooting frame rate of the camera 201 of the marker detection device 20 is 30 FPS (Frames Per Second), which is the most general purpose, the light emission control unit 12 displays frames on the light emitting unit 11 at 30 FPS. To do. As described above, in this embodiment, the shooting frame rate of the camera 201 and the display frame rate of the light emitting unit 11 are the same. FIG. 6 is a time chart illustrating an example of a time transition of the light emission pattern of the light emitting unit 11. As described above, according to the control of the light emission control unit 12, the light emitting unit 11 performs display in the order of the detection frame, the information transmission frame, and the end frame, and after displaying the end frame, the detection frame, the information transmission frame, and the end frame again. Display in this order. That is, the light emitting unit 11 repeats a series of displays. Since the display is performed at 30 FPS as described above, the light emission control unit 12 controls the light emission so as to display 33 milliseconds per frame.

  As shown in FIG. 6, the light emission control unit 12 causes the light emission pattern 51 to display the detection frame on the light emission unit 11, and then causes the light emission pattern 52 to display the detection frame on the light emission unit 11. Here, the light emission control unit 12 controls the light emission by the light emission pattern 51 and the light emission by the light emission pattern 52 to each be one frame. By doing in this way, even if the shooting frame rate of the camera 201 and the display frame rate of the light emitting unit 11 are not synchronized, that is, even when the timings of the two are different, Regardless of the shutter speed, the camera 201 can capture two consecutive images of different patterns as a detection frame. That is, the display of the light emitting unit 11 can remain in the difference image between the two consecutive images. Therefore, the marker detection device 20 does not depend on the shutter speed of the camera 201 even when the shooting frame rate of the camera 201 and the display frame rate of the light emission unit 11 are not synchronized. The position within can be detected.

  This will be specifically described. For example, when the camera 201 captures the light emitting unit 11 in the state of the light emission pattern 51 in the first image capturing, and captures the light emitting unit 11 in the state of the light emitting pattern 52 in the second image capturing (that is, in FIG. 6). As described above, the exposure time T of the camera 201 falls within the time during which the state of the light emission pattern 51 is maintained during the first shooting, and within the time during which the state of the light emission pattern 52 is maintained during the second shooting. 2), two images of the light emission pattern 51 and the light emission pattern 52 are obtained as detection frames.

  Further, for example, the camera 201 captures the light emitting unit 11 in the state of the light emission pattern 51 and the state of the light emission pattern 52 in the first image capturing, and the state of the light emitting pattern 52 and the state of the light emission pattern 53 in the second image capturing. (That is, the exposure time T of the camera 201 straddles the state of the light emission pattern 51 and the state of the light emission pattern 52 during the first image pickup, and the light emission pattern 52 during the second image pickup). In this case, two images, that is, an image obtained by merging the light emission pattern 51 and the light emission pattern 52 and an image obtained by merging the light emission pattern 52 and the light emission pattern 53 are obtained as detection frames.

  For example, the camera 201 captures the light emitting unit 11 in the state of the light emission pattern 55 and the state of the light emission pattern 51 in the first photographing, and the state of the light emission pattern 51 and the state of the light emission pattern 52 in the second photographing. (That is, the exposure time T of the camera 201 straddles the state of the light emission pattern 55 and the state of the light emission pattern 51 during the first image pickup, and the light emission pattern 51 during the second image pickup). In this case, two images, that is, an image obtained by merging the light emission pattern 55 and the light emission pattern 51 and an image obtained by merging the light emission pattern 51 and the light emission pattern 52 are obtained as detection frames.

  The light emission control unit 12 displays the detection frame on the light emitting unit 11 with the light emission pattern 52 and then displays the information transmission frame of the predetermined light emission pattern on the light emitting unit 11. The light emission pattern of the information transmission frame is different from the light emission pattern of the detection frame. In the present embodiment, the light emission control unit 12 controls to display the same light emission pattern over a plurality of consecutive frames in light emission with this predetermined light emission pattern. Specifically, for example, as shown in FIG. 6, a light emission pattern 53 that is a detection frame is displayed over two frames, and a light emission pattern 54 that is a detection frame is also displayed over two frames. By doing so, an image in which a predetermined light emission pattern is always displayed is always obtained for any one shot at any timing. Therefore, the light emission marker device 10 can transmit a predetermined light emission pattern to the marker detection device 20. That is, the light emitting marker device 10 can transmit the light emitting pattern 53 and the light emitting pattern 54, respectively.

  In the example shown in FIG. 6, two types of light emission patterns, that is, a light emission pattern 53 that lights four points and a light emission pattern 54 that lights three points, are transmitted from the light emitting marker device 10 to the marker detection device 20. Will be. As described above, the transmission system 1 can arbitrarily increase or decrease the amount of information to be transmitted by changing the type of light emission pattern displayed as the information transmission frame.

  As described above, the display time of the information transmission frame is variable according to the amount of information to be transmitted. Therefore, it is necessary to notify the completion of information transmission. Therefore, the light emitting marker device 10 displays an end frame. Specifically, the light emission control unit 12 causes the light emitting unit 11 to display a non-light emitting state as the end frame. Here, the light emission control unit 12 controls to display the end frame over a plurality of frames. Specifically, for example, as shown in FIG. 6, the light emission pattern 55 as the end frame is displayed over two frames. In this way, no matter what timing is taken, an image in which no image is lit is always obtained. Therefore, the marker detection device 20 can detect the completion of transmission of information from the light emitting marker device 10.

  Next, details of the marker detection device 20 will be described. The marker detection device 20 photographs the light emitting unit 11 of the light emitting marker device 10 with the camera 201. And according to the light emission pattern transmitted from the light emission marker apparatus 10, ie, according to the marker which the light emission marker apparatus 10 shows, as shown in FIG. In the example shown in FIG. 7, when the light emitting unit 11 of the light emitting marker device 10 installed in the vicinity of the object 60 is photographed by the camera 201 of the marker detecting device 20, information 61 corresponding to the marker is displayed on the display 202. It shows how it is displayed.

  This is done as follows. First, in the marker detection device 20, dedicated application software for using the camera 201 is activated. After activation, the camera 201 is directed so that the light emitting unit 11 of the light emitting marker device 10 is reflected. When the light emitting unit 11 is photographed by the camera 201, information 61 corresponding to the detected marker is displayed on the display 202. The marker detection device 20 repeats such marker search and information display.

  As described above, a plurality of light emitting marker devices 10 may exist in the transmission system 1. Even if the marker detection device 20 captures the light emitting units 11 of the plurality of light emitting marker devices 10 at the same time, each of them can be individually recognized. It is also possible to photograph the light emitting unit 11 of one light emitting marker device 10 with the cameras 201 of the plurality of marker detecting devices 20 and display information corresponding to the markers in each marker detecting device 20.

  As illustrated in FIG. 1, the marker detection device 20 includes an image acquisition unit 21, a marker recognition unit 22, and an information output unit 23 as internal functions. Details of these will be described below.

  The image acquisition unit 21 uses the camera 201 to store an image captured by the camera 201 in a shared memory (memory 203). Shooting by the camera 201 is continuously performed, and images are also continuously stored in the shared memory.

  FIG. 8 is a flowchart illustrating an example of the operation of the image acquisition unit 21. In step 10 (S10), the image acquisition unit 21 activates the camera 201. Subsequently, as Step 11 (S11), the image acquisition unit 21 acquires the image data by photographing, and stores it in the shared memory. The image acquisition unit 21 repeats the operation of step 11 and acquires image data continuously.

  The marker recognition unit 22 has a function of determining whether or not a marker exists from the image acquired by the image acquisition unit 21. As described above, the marker recognition unit 22 includes the difference image generation unit 221, the search unit 222, and the pattern analysis unit 223. FIG. 9 is a flowchart showing an example of the operation of the marker recognition unit 22. Hereinafter, the process of the marker recognition unit 22 will be described with reference to FIG. As shown in FIG. 9, in the processing of the marker recognition unit 22, first, the processing of the difference image generation unit 221 is performed, then the processing of the search unit 222 is performed, and finally, the pattern analysis unit 223 performs processing. Processing is performed.

  The difference image generation unit 221 calculates a difference image between two consecutive images acquired by the image acquisition unit 21. For this reason, first, in Step 20 (S20), the difference image generation unit 221 acquires two images from the shared memory. Next, in step 21 (S21), as shown in FIG. 10, the difference image generation unit 221 calculates the pixel difference between the two images acquired in step 20, and acquires the difference image. Here, FIG. 10 shows a state in which the background object 73 is removed by taking the difference between the image 71 and the image 72, and only the light emission pattern of the light emitting unit 11 is extracted as the difference in the difference image 74. . In step 21, in order to reduce the processing load, the difference image generation unit 221 compares only the R value, for example, of the RGB values and obtains a difference. In step 21, the difference image generation unit 221 converts the obtained difference image into a black and white binary image. After step 21, the process proceeds to step 22 by the search unit 222.

  The search unit 222 searches for a marker based on the difference image. More specifically, the search unit 222 first searches the existence range of the light emitting unit 11 in the image based on the difference image. Specifically, in step 22 (S22), the search unit 222 searches for a pixel group of a black block having a size larger than a predetermined size (for example, 10 pixels × 10 pixels) in order to suppress the false detection rate. . If such a pixel group is found, the search unit 222 determines that the existence range of the pixel group is the existence range of the light emitting unit 11. That is, in step 23 (S23), the search unit 222 determines whether or not a black block pixel group exists in the difference image. If the black block pixel group does not exist in the difference image (No in step 23), the process returns to step 20, two images are acquired from the shared memory, and the same process is performed again. These processes are repeated until the light emitting unit 11 is found in the difference image. When a pixel group of black blocks is present in the difference image (Yes in Step 23), the search unit 222 holds the existence range of the light emitting unit 11 in Step 24 (S24). Specifically, the search unit 222 temporarily stores this in the memory 203 or the like.

  When the search unit 222 can detect the existence range of the light emitting unit 11 in the difference image, the pattern analysis unit 223 analyzes the marker pattern. That is, after step 24, the process proceeds to step 25 by the pattern analysis unit 223.

  The pattern analysis unit 223 analyzes the marker pattern with respect to an image photographed after two consecutive images used for generating the difference image. Specifically, first, in step 25 (S25), the pattern analysis unit 223 acquires a new image from the shared memory in order to acquire the light emission pattern of the information transmission frame. In the present embodiment, the pattern analysis unit 223 acquires, for example, two images. Thereby, the pattern analysis part 223 can always acquire the image of the light emission pattern which the light emission marker apparatus 10 displayed as an information transmission frame.

  Subsequently, in step 26 (S26), the pattern analysis unit 223 performs a trimming process on the image acquired in step 25 so as to include the existence range of the light emitting unit 11 held in step 24. In this way, the pattern analysis unit 223 limits the target range of the image on which the matching process is performed. Further, the pattern analysis unit 223 converts the RGB value into the HSV value for the trimmed image. This is because the matching process is performed using the luminance information so that it is easy to identify the location where light is emitted.

  After step 26, in step 27 (S27), the pattern analysis unit 223 performs a matching process with a pattern image of a predetermined marker on the image processed in step 26. For this matching process, any known pattern matching process can be used. If a pattern image of a predetermined marker is found in the image (Yes in step 28), in step 29 (S29), the pattern analysis unit 223 holds an ID number corresponding to the found pattern. Specifically, the pattern analysis unit 223 temporarily stores this in the memory 203 or the like. Thereafter, the processing returns to step 25, and two images are acquired from the shared memory. That is, the same processing is performed to obtain the next information transmission frame pattern. On the other hand, when a predetermined pattern image of the marker is not found (No in step 28), in step 30 (S30), the pattern analysis unit 223 emits nothing from the image acquired in step 25. It is determined that the image is an end frame image that has not been made. Subsequently, in step 31 (S31), the pattern analysis unit 223 notifies the information output unit 23 of the ID number held so far. After step 31, the processing shifts to processing by the information output unit 23.

  The information output unit 23 displays and outputs information such as CG (Computer Graphics) according to the matching processing result by the pattern analysis unit 223 on the display 202. Specifically, the information output unit 23 displays information corresponding to the ID number notified from the pattern analysis unit 223.

  FIG. 11 is a flowchart illustrating an example of the operation of the information output unit 23. In step 40 (S40), the information output unit 23 searches for information corresponding to the ID number notified from the search unit 222. This information is stored in the memory 203, for example. Subsequently, in step 41 (S41), the information output unit 23 displays the information searched in step 40 on the display 202. By repeatedly performing these processes, the information output unit 23 updates the information displayed on the display 202 each time a new ID number is notified from the search unit 222.

  The embodiment has been described above. In the present embodiment, the existence range of the light emitting unit 11 is extracted from the difference image between the two images. Then, pattern matching processing is performed limited to this existence range. For this reason, the frequency | count of performing a pattern matching process with respect to the image of the part in which a marker does not exist can be suppressed, and calculation load can be suppressed.

  In the two-dimensional marker in ARTToolKit, only a single pattern can be transmitted because only a single pattern is displayed. In addition, indoor art museums and science museums have many dark places, and the two-dimensional marker in ARTToolKit cannot be detected correctly in such dark places. In contrast, in the present embodiment, an arbitrary amount of information can be transmitted from the light emitting marker device 10 to the marker detection device 20 by the information transmission frame. Further, since the marker is indicated by light emission, the marker detection device 20 can recognize the marker even in a dark place.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention. For example, in the above embodiment, the frame rate is 30 FPS, but other frame rates such as 60 FPS may be used. Note that blinking the LED makes it easier for people to see and affects the vision, but increasing the frame rate, such as 60 FPS, can suppress the effect on the vision.

  In the above embodiment, the 8 × 8 dot matrix LED is used. However, the light emitting unit 11 can be configured by using more LEDs such as a 16 × 16 dot matrix LED. is there. Moreover, the light emission part 11 can also be made to light-emit using a liquid crystal panel instead of LED.

  Moreover, the light emission pattern shown by embodiment is an example, and a light emission pattern can be variously changed. For example, as shown in FIG. 12, information can be expressed by a light emission pattern such as a trapezoid. In addition, as shown in FIG. 13, information can be expressed by a light emission pattern such as a circle. FIG. 14 shows the time transition of the overall light emission pattern of the light emitting unit 11 when the information transmission frame of these two light emission patterns is used. Thus, the light emission pattern can be set arbitrarily.

DESCRIPTION OF SYMBOLS 1 Transmission system 10 Light emission marker apparatus 11 Light emission part 12 Light emission control part 20 Marker detection apparatus 21 Image acquisition part 22 Marker recognition part 23 Information output part 201 Camera 202 Display 221 Difference image generation part 222 Search part 223 Pattern analysis part

Claims (9)

A light emitting section capable of emitting light in a plurality of light emission patterns;
A light emission control means for controlling the light emitting unit to display a frame by any one of the plurality of light emission patterns at a predetermined frame rate;
The light emission control unit emits light in a first light emission pattern of a detection frame, which is a frame used by a marker detection device that acquires an image obtained by photographing the light emitting unit to detect the position of the marker. Light emission in the light emission pattern 2, light emission in a predetermined light emission pattern of an information transmission frame which is a frame for transmitting information to the marker detection device, and a frame for notifying completion of information transmission to the marker detection device A light emitting marker device that causes the light emitting unit to repeatedly perform non-light emission as an end frame in order. The marker detection device acquires an image obtained by photographing the light emitting unit at the predetermined frame rate,
The light emission marker device according to claim 1, wherein the light emission control unit controls light emission by the first light emission pattern and light emission by the second light emission pattern to be one frame each. The light emission marker device according to claim 2, wherein the light emission control unit controls to display the same light emission pattern over a plurality of consecutive frames in light emission with the predetermined light emission pattern. The light emission marker device according to claim 2 or 3, wherein the light emission control means controls to display the end frame over a plurality of frames. An image acquisition means for acquiring an image of the light emitting part of a light emitting marker device having a light emitting part capable of emitting light in a plurality of light emission patterns;
Difference image generation means for generating a difference image between two consecutive images acquired by the image acquisition means;
Search means for searching for an existence range of the light emitting unit in the image based on the difference image;
Pattern analysis means for performing a matching process with a predetermined pattern image on a partial image belonging to the existence range in an image photographed after the two consecutive images;
A marker detection device comprising: information output means for outputting information according to a matching processing result by the pattern analysis means. A light emitting marker device and a marker detection device;
The light emitting marker device is:
A light emitting section capable of emitting light in a plurality of light emission patterns;
A light emission control means for controlling the light emitting unit to display a frame by any one of the plurality of light emission patterns at a predetermined frame rate;
The light emission control means emits light in a first light emission pattern of a detection frame that is a frame used by the marker detection device to detect a position of the marker, emits light in a second light emission pattern of the detection frame, Light emission in a predetermined light emission pattern of an information transmission frame that is a frame for transmitting information to the marker detection device, and no light emission as an end frame that is a frame for notifying the marker detection device of completion of transmission of information , In order, let the light emitting unit repeat,
The marker detection device is
Image acquisition means for acquiring an image obtained by photographing the light emitting unit;
Difference image generation means for generating a difference image between two consecutive images acquired by the image acquisition means;
Search means for searching for an existence range of the light emitting unit in the image based on the difference image;
Pattern analysis means for performing a matching process with a predetermined pattern image on a partial image belonging to the existence range in an image photographed after the two consecutive images;
A transmission system comprising: information output means for outputting information in accordance with a result of the matching processing by the pattern analysis means. Using a light emitting part that can emit light in multiple light emission patterns, perform continuous display of frames at a predetermined frame rate,
In the continuous display,
A marker detection device that acquires an image obtained by photographing the light emitting unit causes the light emitting unit to emit light in a first light emission pattern with a detection frame that is a frame used to detect the position of the marker,
Causing the light emitting unit to emit light with a second light emission pattern;
An information transmission frame that is a frame for transmitting information to the marker detection device is caused to emit light to the light emitting unit with a predetermined light emission pattern,
A marker light emitting method that sequentially repeats a series of displays in which the light emitting unit performs no light emission as an end frame that is a frame for notifying the marker detection device of completion of information transmission. Obtain an image of the light emitting part of a light emitting marker device having a light emitting part capable of emitting light with a plurality of light emission patterns,
Generate a difference image between two acquired images,
Based on the difference image, search for the existence range of the light emitting unit in the image,
A matching process with a predetermined pattern image is performed on a partial image belonging to the existence range in an image photographed after the two consecutive images,
Marker detection method that outputs information according to the matching processing result. An image acquisition step of acquiring an image of the light emitting part of a light emitting marker device having a light emitting part capable of emitting light in a plurality of light emission patterns;
A difference image generation step for generating a difference image between two consecutive images acquired in the image acquisition step;
A search step for searching for an existence range of the light emitting unit in the image based on the difference image;
A pattern analysis step of performing a matching process with a predetermined pattern image on a partial image belonging to the existence range in an image photographed after the two consecutive images;
A program for causing a computer to execute an information output step of outputting information according to a matching processing result in the pattern analysis step.

Images