JP2007127536A - Posture detecting system, and light emitter - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a posture detecting system capable of specifying an object and capable of detecting a posture of the object, by a simple configuration. <P>SOLUTION: A head sensor 4 provided with a plurality of infrared tags 6-1 to 6-4 is attached to a head part of a user, the infrared tags 6-1 to 6-4 are flashed in response to ID numbers allocated uniquely to the infrared tags 6-1 to 6-4, and are flashed in response to three-dimensional position information for expressing a three-dimensional position relationship of the infrared tags 6-1 to 6-4, and a detector 1 photographs a near-infrared image including the user, detects the infrared tags 6-1 to 6-4, using the photographed near-infrared image, detects the ID numbers and the three-dimensional position information, based on flashing conditions of the detected infrared tags 6-1 to 6-4, and detects the posture of the head part of the user, based on the the ID numbers of the detected respective infrared tags 6-1 to 6-4 and the three-dimensional position information thereof. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a posture detection system that detects a posture of an object by detecting a light-emitting device mounted on a predetermined position of the object by a detection device, and a light-emitting device used in the posture detection system.

  In recent years, in order to recognize the movement of a person in a room, a technique for knowing the position of an individual using a wireless or wearable navigation system has been developed. In addition, a technology for automatically recognizing an ID (identification information) of an object viewed by a person using an ID tag that rapidly blinks infrared rays has been developed (see, for example, Non-Patent Documents 1 and 2).

  However, in the method using the ID tag that blinks the infrared light at a high speed, since a normal camera is used, the processing speed cannot be ensured sufficiently, and for example, even a low-speed object such as a person's movement can be captured. I couldn't. In addition, there is a camera that uses a special high-speed camera to improve the processing speed. However, the camera is expensive and large-sized, and cannot be applied to a use worn by a person.

For this reason, the inventors of the present application can identify an object located in the direction of the user's line of sight, and can identify an object that can be reduced in size and cost so that the processing speed is high and a person can wear it. A system is being developed (see Patent Document 1). In this object identification system, an infrared image of a predetermined imaging region including an object is captured, and an optical spot having a predetermined size is identified from the infrared image. The identification information of the object is acquired by detecting it as a tag.
Tsuyoshi Aoki, infrared tag read by camera and its application, interactive system and software VIII (WISS 2000), Japan Society for Software Science, Modern Science, 2000, pp. 131-136 Nobuyuki Matsushita, 4 others, ID Cam: Image sensor capable of simultaneously acquiring scene and ID, Interaction 2002, Information Processing Society of Japan, 2002, pp. 9-16 JP 2004-208229 A

  However, in the above object identification system, since only one ID tag is attached to the user, the user's head posture can be detected even though the user's overall movement (flow line) can be detected. It cannot be detected.

  The objective of this invention is providing the attitude | position detection system and light-emitting device which can identify a target object with a simple structure and can detect the attitude | position of a target object.

  An attitude detection system according to the present invention includes a light emitting device mounted at a predetermined position of an object and a detection device that detects the light emitting device, the light emitting device being fixed at a predetermined position with respect to the object, A plurality of light emitting means for emitting infrared light, and a three-dimensional position that blinks each light emitting means in accordance with identification information uniquely assigned to each light emitting means and represents a three-dimensional positional relationship among the plurality of light emitting means A light emission control unit that blinks at least one light emission unit according to information, and the detection device has an optical axis set in a predetermined direction, and takes an infrared image of a predetermined imaging region including the object An imaging unit that detects the plurality of light emitting units using an infrared image captured by the imaging unit, and detects a blinking state of the plurality of light emitting units detected by the detection unit Information detecting means for detecting identification information and three-dimensional position information of each light emitting means, and posture detecting means for detecting the posture of the object based on the identification information and three-dimensional position information detected by the information detecting means; Is provided.

  In the posture detection system according to the present invention, a plurality of light emitting means for emitting infrared light are fixed at predetermined positions with respect to an object, and each light emitting means is in accordance with identification information uniquely assigned to each light emitting means. Blinks, and at least one light emitting means blinks according to the three-dimensional position information representing the three-dimensional positional relationship of the plurality of light emitting means, and at this time, an infrared image of a predetermined photographing region including the object is taken, A plurality of light emitting means are detected using the captured infrared image, and the blinking state of the detected light emitting means is detected, and the identification information and three-dimensional position information of each light emitting means are detected, and the detected identification information Since the posture of the target is detected based on the three-dimensional position information, the target can be specified and the posture of the target can be detected with a simple configuration.

  The light emission control means preferably causes each light emission means to blink according to the identification information and blinks according to three-dimensional position information representing its own three-dimensional positional relationship with respect to the other light emission means.

  In this case, each light-emitting means blinks according to its own identification information and also blinks according to three-dimensional position information representing its own three-dimensional positional relationship with respect to other light-emitting means, so that a captured infrared image is used. By detecting the blinking state of each detected light emitting means, identification information and three-dimensional position information can be acquired from each light emitting means, and the posture of the object can be reliably detected.

  Preferably, the plurality of light emitting means are fixed to a rigid member, the rigid member is fixed to an elastic member, and the light emitting device is attached to the object via the elastic member.

  In this case, the plurality of light emitting means are fixed to the rigid member, the rigid member is fixed to the elastic member, and the light emitting device is attached to the object via the elastic member, so that the three-dimensional position of the light emitting means relative to the object is determined. The light emitting device can be easily attached to the object while being fixed.

  Preferably, the light emitting device is mounted on a user's head, and the posture detection unit detects the posture of the user's head based on the identification information and three-dimensional position information detected by the information detection unit. .

  In this case, the light emitting device is mounted on the user's head, and the posture of the user's head is detected based on the detected identification information and three-dimensional position information. Can be detected with high accuracy.

  The plurality of light emitting means include three or more light emitting means, and the posture detecting means is based on identification information and three-dimensional position information of the three or more light emitting means detected by the information detecting means. It is preferable to detect the posture of the head.

  In this case, since the posture of the user's head is detected based on the identification information and three-dimensional position information of three or more light emitting means, the posture of the user's head can be detected with higher accuracy.

  The plurality of light emitting means includes a first light emitting means for emitting infrared light in front of the user's head, a second light emitting means for emitting infrared light on the right side of the user's head, and a left side of the user's head. It is preferable to include a third light emitting unit that emits infrared light and a fourth light emitting unit that emits infrared light behind the user's head.

  In this case, since the light emitting means is mounted over the entire circumference of the user's head, the identification information and the three-dimensional position information can be detected from at least two light emitting means no matter which direction the user is facing. Regardless of the direction of the user, the posture of the user's head can be reliably detected.

  A light-emitting device according to the present invention is a light-emitting device mounted at a predetermined position of an object, and is fixed to the object at a predetermined position and emits infrared rays, and each light-emitting means. A light emission control means for causing each light emitting means to blink according to the uniquely assigned identification information, and for causing at least one light emitting means to blink according to three-dimensional position information representing a three-dimensional positional relationship among the plurality of light emitting means; Is provided.

  In the light emitting device according to the present invention, a plurality of light emitting means for emitting infrared light is fixed at a predetermined position with respect to the object, and each light emitting means is in accordance with identification information uniquely assigned to each light emitting means. In addition to blinking, at least one light-emitting means blinks according to the three-dimensional position information representing the three-dimensional positional relationship between the plurality of light-emitting means. Dimensional position information can be detected. As a result, since the posture of the target can be detected based on the detected identification information and three-dimensional position information, the target can be specified and the posture of the target can be detected with a simple configuration. be able to.

  According to the present invention, a plurality of light emitting means for emitting infrared light are fixed at predetermined positions with respect to an object, and each light emitting means blinks according to identification information uniquely assigned to each light emitting means. The at least one light-emitting means blinks according to the three-dimensional position information representing the three-dimensional positional relationship among the plurality of light-emitting means, and at this time, an infrared image of a predetermined imaging region including the object is taken and the captured infrared A plurality of light emitting means are detected using an image, the blinking state of the detected light emitting means is detected, and identification information and three-dimensional position information of each light emitting means are detected, and the detected identification information and three-dimensional position are detected. Since the posture of the object is detected based on the information, the object can be specified and the posture of the object can be detected with a simple configuration.

  Hereinafter, an attitude detection system according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an attitude detection system according to an embodiment of the present invention. In the following description, a case where the posture of the user's head is detected as an example of detecting the posture of the object will be described, but the object to which the present invention is applicable is not particularly limited to the user's head, Various things can be applied as long as the posture changes.

  The posture detection system shown in FIG. 1 includes a detection device 1 and a headset 4. Headset 4 is attached to the user and transmits ID numbers (identification information) uniquely assigned to each of the plurality of infrared tags 6-1 to 6-4 by flashing infrared rays. The three-dimensional position information representing the three-dimensional positional relationship of 6-1 to 6-4 is transmitted by blinking infrared rays.

  The detection device 1 is transmitted from infrared tags 6-1 to 6-4 provided in a headset 4 that is disposed in a space where the user is located, for example, a predetermined location on the ceiling, and is mounted on the user located in the imaging range. The infrared tag ID number and the three-dimensional position information are detected, and a visible light image including the user wearing the headset 4 is taken. Further, the detection device 1 detects the posture of the user based on the detected identification information and the three-dimensional position information, and outputs information regarding the detected posture of the user and a captured visible light image to the server 11. The server 11 performs predetermined processing such as addition of time information on the input information.

  The information collected as described above is used for various purposes in the server 11, for example, used to create an interaction corpus serving as a knowledge base in which interaction data relating to human-to-human interaction is accumulated. The

  FIG. 2 is a schematic perspective view showing an appearance when the user wears the headset 4 shown in FIG. 1, and FIG. 3 is a schematic top view for explaining the configuration of the headset 4 shown in FIG. It is a right side view. As shown in FIGS. 2 and 3, the headset 4 is configured as a glasses-type headset that is worn on the user's head, and includes four infrared tags 6-1 to 6-4 and a microphone unit 5. And a fixing portion 42 in which the nose pad member 7 is disposed at the center thereof. The main body 41 is composed of a rigid member so that the relative positional relationship between the infrared tags 6-1 to 6-4 does not change, and the fixing portion 42 is composed of an elastic member so that the user can easily wear it. Is fixed to the fixing portion 42. The headset 4 is attached and substantially fixed to the user's head when the fixing portion 42 sandwiches the user's head from the front near the forehead.

  A first infrared tag 6-1 having an optical axis in the front direction (0 degree) of the user is disposed on the front surface portion of the main body portion 41 of the headset 4, and is disposed on the right front portion of the main body portion 41 of the headset 4. Is arranged with a second infrared tag 6-2 having an optical axis obliquely forward to the right (60 degrees to the right) of the user. A third infrared tag 6-3 having an optical axis at 60 degrees is disposed. The first to second infrared tags 6-1 to 6-3 are provided with two LEDs in the same direction. This is provided in order to obtain a sufficient amount of light from the first to second infrared tags 6-1 to 6-3, and these two LEDs blink in synchronization.

  Further, the main body 41 of the headset 4 has a rod-like support portion 43 from the right side surface portion toward the rear, and a fourth portion having an optical axis at the tip of the support portion 43 at the rear of the user (180 degrees). An infrared tag 6-4 is arranged, and the fourth infrared tag 6-4 is provided with two LEDs (Light Emitting Diodes) so that the angle between the optical axes is 90 degrees, and is about 180 degrees behind the user. Configured to be detectable in range, the two LEDs flash in sync.

  In addition, the number and position of the infrared tag arrange | positioned at the headset 4 are not specifically limited to said example, A various change is possible, and the number and position of LED used for one infrared tag are also the same. The present invention is not particularly limited to the above example, and various modifications are possible.

  A microphone unit 5 is disposed at the lower center of the front surface of the main body 41 of the headset 4. When the user wears the headset 4, the microphone unit 5 is disposed near the user's eyebrows. As described above, by arranging the microphone unit 5 so as to be positioned in the portion between the eyebrows of the user so as to be fixed to the headset 4, it is possible to reliably record a sound with a certain sound quality. Therefore, the sound quality can be improved as compared with a conventional movable microphone unit disposed near the user's mouth.

  A nose pad member 7 is disposed at the lower center of the front surface of the fixing portion 42 of the headset 4. Similar to general glasses, the nose pad member 7 is configured to substantially fix the headset 4 to the user's head by being placed on the user's nose.

  Referring to FIG. 1 again, as described above, the headset 4 includes the plurality of infrared tags 6-1 to 6-4 and the microphone unit 5. In the following description, two LEDs of the first to fourth infrared tags 6-1 to 6-4 will be described as one LED 61 for easy explanation.

  The infrared tags 6-1 to 6-4 include an LED 61 and a drive circuit 62, respectively. The LED 61 is composed of an infrared LED or the like. For example, a high-power light emitting diode for optical communication (DN311 manufactured by Stanley) or the like can be used, and an infrared LED having a weak directivity and about 800 nm that is close to visible light is preferably used. Can be used.

  The drive circuit 62 is composed of a microcomputer or the like. For example, an Atmel 4 MHz drive microcomputer AT90S2323 can be used, and the ID number uniquely assigned to the infrared tags 6-1 to 6-4 is identified. The LED 61 is controlled to blink as possible. In addition, the drive circuit 62 blinks each LED 61 according to the three-dimensional position information representing the three-dimensional positional relationship on the headset 4 of each infrared tag 6-1 to 6-4. In this embodiment, the drive circuit is provided for each infrared tag, that is, the LED. However, the drive circuit is not particularly limited to this example, and the blinking state of all or a plurality of LEDs may be controlled by one drive circuit. .

  Specifically, the drive circuit 62 uses the Manchester encoding method, with a data interval of 8 ms / 1 bit, a start bit (2 bits), a first (ID1) / continue (ID continuous) code (1 bit), an ID number ( 10 bits) according to the data format of the three-dimensional position information and the parity bit (1 bit), the LED 61 is blinked, and the ID number and the three-dimensional position information are repeatedly transmitted.

  For example, as the three-dimensional position information, in the local coordinate system defined on the headset 4, the infrared tag 6-1 is at (x1, y1, z1), and the infrared tag 6-2 is (x2, y2, z2). ), The infrared tag 6-3 is at (x3, y3, z3), and the infrared tag 6-4 is at (x4, y4, z4), the LED 61 of the infrared tag 6-1 is (x1, y1, The three-dimensional position information indicating z1) is transmitted, the LED 61 of the infrared tag 6-2 transmits three-dimensional position information indicating (x2, y2, z2), and the LED 61 of the infrared tag 6-3 is (x3, y3, The three-dimensional position information representing z3) is transmitted, and the LED 61 of the infrared tag 6-4 transmits the three-dimensional position information representing (x4, y4, z4). In this case, the ID number and three-dimensional position information of the infrared tag can be acquired from each detected infrared tag, and the posture of the user's head can be reliably detected.

  The three-dimensional position information is not particularly limited to the above example, and various changes can be made. For example, the position of the infrared tag 6-1 is set to the origin of the local coordinate system defined on the headset 4, and the infrared tag 6-1 is positioned at its own identification information and origin. The position information may be transmitted, and the other infrared tags 6-2 to 6-4 may transmit their own identification information and three-dimensional position information representing a relative position from the origin. In this case, the information amount of the three-dimensional position information transmitted from the infrared tag 6-1 can be reduced.

  Also, one infrared tag, for example, the infrared tag 6-1 has three-dimensional position information indicating the three-dimensional positions of all the infrared tags 6-1 to 6-4 and the ID numbers of the infrared tags having the respective three-dimensional positions. The other infrared tags 6-2 to 6-4 may transmit only their own identification information and omit transmission of the three-dimensional position information. In this case, the amount of information transmitted from the infrared tags 6-2 to 6-4 can be reduced.

  Further, all the infrared tags 6-1 to 6-4 represent their own ID numbers, the three-dimensional positions of all the infrared tags 6-1 to 6-4, and the ID numbers of the infrared tags having the respective three-dimensional positions. You may make it transmit three-dimensional position information. In this case, identification information and three-dimensional position information of all infrared tags can be acquired from the detected infrared tag, and the posture of the user's head can be detected more reliably.

  The microphone unit 5 includes an audio processing circuit 51 and a microphone 52. The microphone 52 collects the user's utterance or ambient sound and outputs it to the voice processing circuit 51, and the voice processing circuit 51 outputs the recorded voice signal to the portable computer 10. When not recording sound, the microphone unit 5 may be omitted, and a speaker or the like may be integrated with the headset in order to transmit the sound or the like to the user.

  The detection device 1 includes an infrared detection unit 2 and an image capturing unit 3. The image photographing unit 3 includes a lens 31, an optical element 32, and a CCD camera 33. The lens 31 guides visible light and near-infrared light in a predetermined shooting range including the user wearing the headset 4 to the optical element 32. The optical element 32 reflects near-infrared light, guides it to the infrared detection unit 2, passes visible light, and forms a visible light image on the CCD camera 33. The CCD camera 33 captures a visible light image and outputs a video signal to the server 11.

  As described above, the image capturing unit 3 and the infrared detection unit 2 can capture a visible light image and a near infrared image on the same optical axis using the visible light and the near infrared light separated by the optical element 32. Therefore, the positions on the visible light image and near-infrared image obtained by photographing the user (infrared tags 6-1 to 6-4) are completely matched, and not only the user is identified but also the posture of the user's head is highly accurate. Can be detected.

  The infrared detection unit 2 includes a reflective element 21, a CMOS image sensor 22, an image processing device 23, and a posture detection unit 24. The reflective element 21 reflects near-infrared light from the optical element 32 and forms an image on the CMOS image sensor 22. The CMOS image sensor 22 captures a near-infrared image composed of the formed near-infrared light and outputs it to the image processing device 23. As the CMOS image sensor 22, for example, a high-sensitivity monochrome CMOS image sensor LM9630 manufactured by National Semiconductor Co., Ltd. can be used. In this case, the resolution is 128 × 98 pixels and the frame rate is 500 fps.

  The image processing device 23 controls the CMOS image sensor 22 and performs data processing, detects the infrared tags 6-1 to 6-4 from the near-infrared image photographed by the CMOS image sensor 22, and detects the detected infrared tag 6-1. The ID number and the three-dimensional position information are detected from the flashing state of ˜6-4, and data such as the ID number and the three-dimensional position information is output to the posture detection unit 24. As described above, the infrared detection unit 2 can detect the ID numbers and the three-dimensional position information of the infrared tags 6-1 to 6-4 using a near-infrared image created mainly from only the near-infrared rays. The adverse effect of light having a wavelength in the visible light region that causes disturbance can be sufficiently reduced, and the detection process can be speeded up.

  The posture detection unit 24 detects the posture of the user's head based on the ID number of the infrared tag detected by the image processing device 23 and the three-dimensional position information. That is, the posture detection unit 24 uses the known posture calculation method based on the two-dimensional position of each infrared tag on the infrared image and the three-dimensional position information of each infrared tag, that is, the three-dimensional space of the user's head. Detect upper position and angle. The posture detection unit 24 outputs posture information representing the detected posture of the user to the server 11 in accordance with a data transmission standard such as RS232C.

  The server 11 receives the user posture information and the like output from the detection device 1 and executes a process of creating an interaction corpus that is a knowledge base in which interaction data relating to human-human interaction and the like is accumulated.

  Next, the image processing apparatus 23 shown in FIG. 1 will be described in detail. FIG. 4 is a block diagram showing a configuration of the image processing apparatus 23 shown in FIG. As shown in FIG. 4, the image processing device 23 includes a binarization circuit 81, an SRAM (Static Random Access Memory) 82, a brightness change detection circuit 83, and a decoding circuit 84. For example, the binarization circuit 81 and the brightness change detection circuit 83 are configured by a CPLD (Complex Programmable Logic Device) or the like, and the decode circuit 84 is configured by a CPU or the like, and is decoded by executing a predetermined decode program by the CPU. Process. Note that the configuration of the image processing device 23 is not particularly limited to the above example, and various modifications such as configuring all of them from dedicated hardware circuits are possible.

  The binarization circuit 81 receives the brightness data of each pixel constituting the photographed near-infrared image from the CMOS image sensor 22 and binarizes the brightness value of each pixel by comparing with a reference brightness value stored in advance. The binarized pixel data is sequentially stored in the SRAM 82. For example, when lightness data is output from the CMOS image sensor 22 with 256 gradations, the lightness of 128 gradations or more is converted to “1”, and the lightness of less than 128 gradations is converted to “0”. The multi-value processing is not particularly limited to the above example, and the brightness of 128 gradations or more is converted to “2”, less than 128 gradations of 64 gradations or more to “1”, and less than 64 gradations to “0”. , “2” can be changed to “bright”, and “0” can be changed to “blink” to determine the blinking state.

  The SRAM 82 has a storage capacity capable of storing a plurality of, for example, 672 near-infrared images taken by the CMOS image sensor 22, and stores the binarized pixel data output from the binarization circuit 81 for each image. To do.

  The brightness change detection circuit 83 reads out the pixel data of a plurality of, for example, five binarized images taken continuously, from the SRAM 82, and the pixel data at the same position on each binarized image blinks. All blinking pixels that alternately repeat “1” or “0” in time are detected as light spots (infrared tags 6-1 to 6-4), and the detected multiple light spots on the binarized image And the light spot position information for specifying the position of the image and the number of frames taken of the image, for example, the storage address of the blinking pixel data in the SRAM 82 are output to the decoding circuit 84.

  Further, when a plurality of adjacent pixels are blinking, the lightness change detection circuit 83 detects these pixels as one light spot. In this case, since a plurality of adjacent pixels blinking at the same time can be detected as one light spot, regardless of the size of one pixel of the CMOS image sensor 22, the LEDs 61 of various sizes can be used as one light spot. It can be accurately detected as a point.

  The decode circuit 84 sequentially reads and decodes the binarized pixel data of the blinking pixels in the binarized image from the SRAM 82 based on the light spot position information of the blinking pixels, and decodes the plurality of light spots, that is, the plurality of infrared tags 6-1. The ID number of ˜6-4 and the three-dimensional position information are output to the server 11.

  In addition, when the pixel corresponding to the light spot detected by the brightness change detection circuit 83 is not blinking, the decoding circuit 84 reads the binarized pixel data of the pixel located around the pixel from the SRAM 82, and The blinking state of the light spot is decoded again using the binary pixel data of the pixel that changes in time, and the ID numbers and the three-dimensional position information of the infrared tags 6-1 to 6-4 are detected. In this case, since the blinking state of the light spot can be decoded using not only the pixel detected as the light spot but also the binarized values of the pixels located around the pixel, the user moves at high speed. Even in this case, it is possible to simultaneously and continuously identify the plurality of infrared tags 6-1 to 6-4 that move as the user moves.

  In the present embodiment, the headset 4 corresponds to an example of a light emitting device, the detection device 1 corresponds to an example of a detection device, and the infrared tags 6-1 to 6-4 (LEDs 61) correspond to an example of light emitting means. The drive circuit 62 corresponds to an example of a light emission control unit, the reflective element 21 and the CMOS image sensor 22 correspond to an example of a photographing unit, and the binarization circuit 81, the SRAM 82, and the brightness change detection circuit 83 are an example of a light emission detection unit. The decoding circuit 84 corresponds to an example of an information detection unit, and the posture detection unit 24 corresponds to an example of a posture detection unit. The main body portion 41 corresponds to an example of a rigid member, and the fixing portion 42 corresponds to an example of an elastic member. The infrared tag 6-1 corresponds to an example of a first light emitting unit, the infrared tag 6-2 corresponds to an example of a second light emitting unit, and the infrared tag 6-3 is an example of a third light emitting unit. The infrared tag 6-4 corresponds to an example of a fourth light emitting unit.

  Next, an infrared tag detection process performed by the detection apparatus 1 configured as described above will be described. FIG. 5 is a flowchart for explaining posture detection processing by the detection apparatus 1 shown in FIG.

  First, the image processing device 23 initializes the CMOS image sensor 22 and the like (step S11), and sequentially captures and acquires near-infrared images of the full screen (128 × 98 pixels) (step S12). Next, the binarization circuit 81 receives brightness data of each pixel constituting the photographed near-infrared image from the CMOS image sensor 22 and compares the brightness value of each pixel with a predetermined reference brightness value to binarize. ("0" or "1"), and the binarized pixel data is stored in the SRAM 82 for each near-infrared image (step S13).

  Next, the brightness change detection circuit 83 reads the binarized pixel data of each pixel of a plurality of binarized images taken continuously from the SRAM 82, and the binarity at the same position on each binarized image. All blinking pixels whose pixel data is alternately repeating “1” or “0” in a predetermined order are detected as light spots, and each position on the image of the detected light spots and the number of frames taken for the image are detected. Is output to the decoding circuit 84 (step S14).

  Next, the decoding circuit 84 sequentially reads and decodes the binarized pixel data of each light spot of the plurality of binarized images from the SRAM 82 based on the light spot position information of the blinking pixel (step S15), and outputs the decoded result. A parity check is performed on the read data, and read data determination processing is performed (step S16).

  If the parity check is correct (YES in step S16), the decode circuit 84 obtains the decoded infrared tag ID number and three-dimensional position information, and two-dimensional position information representing the position of the infrared tag on the image. It outputs to the attitude | position detection part 24 (step S17).

  On the other hand, if the parity check is not correct (NO in step S16), the decoding circuit 84 sets the pixels located around the pixel based on the light spot position information of the blinking pixel, for example, the upper, lower, left, and right sides of the blinking pixel. The binarized pixel data of 8 pixels located is read from the SRAM 82, the logical sum of the read binarized pixel data and the binarized pixel data of the blinking pixel is decoded (step S18), and the parity is again obtained with respect to the decoded result. A check is performed and read data determination processing is performed (step S19).

  If the parity check is correct (YES in step S19), the decoding circuit 84 obtains the decoded infrared tag ID number and three-dimensional position information, and two-dimensional position information indicating the position of the infrared tag on the image. It outputs to the attitude | position detection part 24 (step S17). If the parity check is not correct, the process returns to step S12 and the above processing is continued.

  Next, the posture detection unit 24 detects the posture of the user's head from the ID number of the infrared tag output by the decoding circuit 84, the three-dimensional position information, and the two-dimensional position information (step S20), and the detected user's head is detected. Posture information is transmitted to the server 11, and the server 11 stores the transmitted posture information. Then, it returns to step S12 and said process is continued. Note that the posture of the head can be detected as to which direction the user's head is facing if at least two infrared tags are detected. If three infrared tags are detected, It can be detected including the direction of rotation.

  As described above, in the present embodiment, according to the ID number uniquely assigned to the four infrared tags 6-1 to 6-4 fixed to the headset 4 attached to the user's head. The infrared tags 6-1 to 6-4 blink, and the infrared tags 6-1 to 6-4 blink according to the three-dimensional position information of the infrared tags 6-1 to 6-4. 6-1 to 6-4, an infrared image including at least two or more infrared tags is taken, at least two or more infrared tags are detected using the taken infrared image, and the detected infrared tag blinks Since the state is detected and the ID number and three-dimensional position information of each infrared tag are detected, and the posture of the user's head can be detected from the detected ID number and three-dimensional position information of each infrared tag. User's infrared tag With a simple structure that is mounted on part, it is possible to detect the posture of the head of the user with high accuracy.

  Also, a binarized image obtained by binarizing the photographed near-infrared image is created, and a flashing light spot is detected based on a temporal change in brightness of each pixel of the created binarized image. Therefore, a large number of light spots included in one image can be detected simultaneously and at high speed, and the postures of the heads of a plurality of users wearing the headset 4 can be detected simultaneously and continuously. . In addition, since a binary multi-valued image is created in comparison with a predetermined reference brightness value, a special memory such as a frame buffer is not required for the binarization process, and a binary image is created at high speed. In addition, since the data capacity can be reduced, the storage capacity of the SRAM 82 can be kept low. Furthermore, since the binarization circuit 81 and the lightness change detection circuit 83 are constituted by CPLD, a plurality of light spots can be detected at higher speed.

  In the present embodiment, ID numbers and three-dimensional position information are transmitted from the infrared tags 6-1 to 6-4 using a continuous light emission pattern. However, the present invention is not particularly limited to this example. 6-4 transmits only the ID number, and among the plurality of infrared tags 6-1 to 6-4, only the infrared tag whose data transmission is permitted by the detection device 1 transmits the ID number and the three-dimensional position information. It may be.

  FIG. 6 is a communication sequence diagram for explaining communication processing between the infrared tags 6-1 to 6-3 and the detection apparatus 1. In FIG. 6, for ease of explanation, communication processing between the infrared tags 6-1 to 6-3 and the detection apparatus 1 will be described. The same applies to the case of communicating with more infrared tags. Can be done.

  First, the portable computer 10 causes the infrared tags 6-1, 6-2, and 6-3 to transmit their ID numbers, and the detection device 1 causes the infrared tags 6-1, 6-2, and 6-6 that are transmitting to transmit. 3 is received (steps S101 to S103).

  Next, the detection apparatus 1 sends a start signal for permitting data transmission to one of the detected infrared tags 6-1 to 6-3 together with the ID number of the infrared tag 6-1. It transmits to the portable computer 10 via the server 11 (step S104). At this time, the other infrared tags 6-2 and 6-3 recognize that the data transmission is not permitted because the start signal is issued to the infrared tags having ID numbers other than the other infrared tags (6-2). Steps S105 and S106).

  In this example, the start signal for permitting data transmission is transmitted from the server 11 to the portable computer 10 by wireless communication. However, the present invention is not particularly limited to this example, and the headset 4 includes an infrared detector, and the detection device 1 When the is equipped with an infrared tag, the start signal may be transmitted by infrared communication. The same applies to a stop signal described later. The data transmission permission order may be the permission of data transmission in order from the infrared tag that has detected the ID number earliest, or a predetermined order is determined in advance, and the data transmission permission is determined based on this order. May be given.

  Next, the portable computer 10 that has received the start signal causes the infrared tag 6-1 to transmit its three-dimensional position information together with the ID number (step S107). The detection apparatus 1 that has completed the reception of the three-dimensional position information from the infrared tag 6-1 sends a stop signal for the infrared tag 6-1 to the portable computer 10 via the server 11 together with the ID number of the infrared tag 6-1. By transmitting, the infrared tag 6-1 stops data transmission (step S108). Then, the detection apparatus 1 performs all of the infrared tags 6-1 to 6-6 by sequentially repeating the above processing on the other infrared tags 6-2 and 6-3 in the captured image of the CMOS image sensor 22. -3 position information from -3 is acquired (steps S109 to S114).

  In the above case, since data (three-dimensional position information) is not simultaneously transmitted from a plurality of infrared tags, signals from the plurality of infrared tags are not received simultaneously, and the blinking pattern of each infrared signal by the plurality of infrared tags is incorrect. Recognition can be prevented.

  Next, another example of the CMOS image sensor 22 and the image processing device 23 shown in FIG. 1 will be described in detail. FIG. 7 is a block diagram showing a configuration of another example of the CMOS image sensor and the image processing apparatus shown in FIG.

  As shown in FIG. 7, the CMOS image sensor 22a captures a near-infrared image composed of near-infrared rays reflected by the reflecting element 21 (see FIG. 1) and outputs it to the image processing device 23a. As the CMOS image sensor 22a, for example, an imaging area CMOS type imaging device MT9M413 manufactured by MICRON can be used. In this case, the resolution is 1280 × 1024 pixels and the frame rate is 400 fps.

  In the case of using the high-resolution CMOS image sensor as described above, if it has sensitivity to near-infrared light and visible light, the CCD camera 33 is omitted and the CMOS image sensor 22a is used for the visible light image. It can also be used for photographing, and the configuration of the detection apparatus 1 can be further simplified.

  The image processing device 23a performs control and data processing of the CMOS image sensor 22a, detects the infrared tags 6-1 to 6-4 from the near-infrared image photographed by the CMOS image sensor 22a, and detects the detected infrared tag 6-1. The ID number and the three-dimensional position information are detected from the flashing state of ˜6-4, and the data of the ID number and the three-dimensional position information is output to the posture detection unit 24.

  The image processing device 23 a includes a quaternization circuit 91, a frame buffer 92, an SRAM 93, a brightness change detection circuit 94, and a decoding circuit 95. For example, the quaternarization circuit 91 and the brightness change detection circuit 94 are configured by an FPGA (Field Programmable Gate Array) or the like, and the decoding circuit 95 is configured by a CPU or the like, and the CPU executes a predetermined decoding program as follows. Perform the decoding process. Note that the configuration of the image processing device 23a is not particularly limited to the above example, and various modifications such as configuring all from a dedicated hardware circuit are possible.

  The quaternarization circuit 91 processes a near infrared image using a plurality of frames, for example, 20 frames as a comparison processing unit, stores the infrared image of the first frame in the frame buffer 92, and configures each infrared image of the second frame. The pixel brightness data is compared with the brightness data of each pixel constituting the infrared image of the first frame stored in the frame buffer 92, and the four-valued pixel data is stored in the SRAM 93 as a comparison result. The infrared image of the second frame is stored in the frame buffer 92.

  Thereafter, in the same manner as described above, the quaternary circuit 91 performs the brightness data of each pixel constituting the infrared image of the current frame and the brightness data of each pixel constituting the infrared image of the immediately previous frame stored in the frame buffer 92. The four-valued pixel data obtained by the four-value comparison are stored in the SRAM 93, and the process of storing the infrared image of the current frame (excluding the last frame of the comparison processing unit) in the frame buffer 92 is repeated. The SRAM 93 stores the quaternary pixel data of the quaternary image output from the quaternary circuit 91 in units of comparison processing, for example, 20 frames.

  For example, if the lightness value of the determination target pixel in the current frame is p, the lightness value of the corresponding pixel in the previous frame is P, and a predetermined threshold value is T, the quaternary circuit 91 stores P ≦ p <P + T in the SRAM 93. “0” is stored, “1” is stored when P + T ≦ p, “2” is stored when P−T <p <P, and “3” is stored when p ≦ P−T. And the brightness is compared with the immediately preceding frame for each pixel and classified into four stages.

  When the final frame of the comparison processing unit is stored in the SRAM 93, the lightness change detection circuit 94 reads out the four-valued pixel data of a plurality of frames, for example, 20 frames as the comparison processing unit from the SRAM 93, and reads out the four-value quantization. The blinking state of each pixel is determined based on the quaternary pixel data of the image, and the positions on the quaternary image of a plurality of light spots determined as light spots (infrared tags 6-1 to 6-4) from the determination result Then, the light spot position specifying information for specifying the number of frames taken of the image, for example, the storage address of the blinking pixel data in the SRAM 93 is output to the decoding circuit 95. Further, when it is determined that a plurality of adjacent pixels are blinking, the brightness change detection circuit 94 detects these pixels as one light spot.

  For example, when the comparison processing unit is 20 frame units, the brightness change detection circuit 94 outputs a total of four or more quaternary pixel data of “1” or “3” in all 19 frames as the blinking state of each pixel. In this case, the pixel is determined to be “0”, and when both the four-valued pixel data “1” and the four-valued pixel data “3” appear within 10 consecutive frames, the pixel is set to “1”. If the total of four-valued pixel data “1” or “3” appears 2 times or more and 3 times or less in all 19 frames, the pixel is determined as “2” and “1” in all 19 frames. Or if the total of four-valued pixel data of “3” appears only once or less in total, the pixel is determined as “3”, and a pixel whose determination result is “0” or “1” is detected as a light spot.

  The decoding circuit 95 sequentially reads the quaternary pixel data of the blinking pixels in the quaternary image from the SRAM 93 based on the light spot position specifying information of the blinking pixels, and sets the case where the quaternary pixel data is “1” to “1”. "(Bright state)" When the quaternary pixel data is "3", it is decoded as "0" (bright state), and the ID numbers and three-dimensional positions of a plurality of light spots, that is, infrared tags 6-1 to 6-4 Information is output to the posture detection unit 24.

  Further, in order to cope with the case where the infrared tags 6-1 to 6-4 move across the pixels in a plurality of frames of the comparison processing unit, the brightness change detection circuit 94 has a determination result of “1” or “2”. Pixel position specifying information may be output to the decoding circuit 95. In this case, the decode circuit 95 reads the brightness determination result data of the pixels located around the pixel from the SRAM 93, and also uses the quaternary pixel data that changes in time around the surrounding pixels to change the blinking state of the light spot. Decoding is performed again to detect the ID numbers and three-dimensional position information of the infrared tags 6-1 to 6-4. Also, the near-infrared image is labeled in shades for each frame, and the barycentric coordinates, area, circularity, fillet diameter, average brightness, etc. of each labeled area are recorded, and the LED spot of the shape is judged to determine the light spot. You may make it detect.

  In this example, the CMOS image sensor 22a corresponds to an example of a photographing unit, the quaternary circuit 91, the frame buffer 92, the SRAM 93, and the brightness change detection circuit 94 correspond to an example of a light emission detection unit, and the decoding circuit 95 detects information. It corresponds to an example of means.

  Next, posture detection processing using the CMOS image sensor 22a and the image processing device 23a shown in FIG. 7 will be described. FIG. 8 is a flowchart for explaining a posture detection process using the CMOS image sensor 22a and the image processing device 23a shown in FIG.

  First, the image processing device 23a initializes the CMOS image sensor 22a and the like (step S21), and sequentially captures and acquires near-infrared images of the full screen (1280 × 1024 pixels) (step S22). Next, the quaternary circuit 91 stores the brightness data of each pixel constituting the near-infrared image of the previous frame in the frame buffer 92, and the brightness data of each pixel constituting the near-infrared image of the current frame and the frame buffer 92. Are compared with the brightness data of each pixel constituting the near-infrared image of the immediately preceding frame stored in, and the comparison result is quaternized, and the quaternary pixel data is stored in the SRAM 93 in comparison processing units (step S23).

  Next, when the final frame of the comparison processing unit is stored in the SRAM 93, the lightness change detection circuit 94 reads out 19 frames of quaternary pixel data as a comparison processing unit from the SRAM 93, and reads out the quaternary pixel data thus read out. Based on the above, the blinking state of each pixel is quaternized to the above “0” to “3” and determined (step S24).

  Here, when the blinking state is “0” or “1” (YES in step S25), the brightness change detection circuit 94 determines the pixel as a light spot (infrared tags 6-1 to 6-4) and determines The light spot position specifying information for specifying each position of the plurality of light spots on the image and the number of frames taken of the image is output to the decoding circuit 95.

  Next, the decode circuit 95 sequentially reads out the quaternary pixel data of each light spot of the plurality of quaternary images from the SRAM 93 based on the light spot position specifying information of the blinking pixel, and the quaternary pixel data is “1”. Is decoded as “1” and the case where the quaternary pixel data is “3” is set to “0” (step S26), a parity check is performed on the decoding result, and a determination process of read data is performed (step S27). ).

  If the parity check is correct (YES in step S27), the decoding circuit 95 obtains the decoded infrared tag ID number and three-dimensional position information, and two-dimensional position information indicating the position of the infrared tag on the image. It outputs to the attitude | position detection part 24 (step S28).

  On the other hand, if the blinking state is not “0” or “1” (NO in step S25), or if the parity check is not correct (NO in step S26), the decoding circuit 95 is based on the position specifying information of the blinking pixel. The brightness change determination result data of pixels located in the periphery of the pixel, for example, 8 pixels located above, below, left, and right of the blinking pixel are read from the SRAM 93, and the read quaternary pixel data and the quaternary pixel of the blinking pixel are read. The data is decoded using the data (step S29), the parity check is performed again on the decoding result, and the read data determination process is performed (step S30).

  If the parity check is correct (YES in step S30), the decoding circuit 95 obtains the decoded infrared tag ID number and three-dimensional position information, and two-dimensional position information representing the position of the infrared tag on the image. It outputs to the attitude | position detection part 24 (step S28). If the parity check is not correct, the process returns to step S22 and the above processing is continued.

  Next, the posture detection unit 24 detects the posture of the user's head from the ID number of the infrared tag output by the decoding circuit 95, the three-dimensional position information, and the two-dimensional position information (step S31), and then step S22. The above processing is continued after returning to step S2.

  As described above, also in this example, a large number of light spots included in one image can be detected simultaneously and at high speed, and the postures of the heads of a plurality of users wearing the headsets 4 can be simultaneously and continuously. Can be detected automatically. Further, since the quaternary image is created in comparison with the brightness of the near-infrared image captured immediately before, a quaternary image that accurately reflects the change in brightness is created even when the shooting environment changes. be able to. Furthermore, since the quaternarization circuit 91 and the brightness change detection circuit 94 are made of FPGA, a plurality of light spots can be detected at higher speed.

  In the present embodiment, the infrared tags 6-1 to 6-4 are detected to detect the posture of the user's head, but the detection target is not particularly limited to this example, and the space in which the user is located An infrared tag may be attached to various objects arranged inside, and the infrared tag attached to the object may be detected by the detection device 1. In this case, since the posture of the user's head is detected and the position of the target object is detected, the target object that the user is interested in is identified based on the posture of the user's head and the position of the target object. More detailed interaction data can be accumulated.

  Next, an application example of the object identification system configured as described above will be described. FIG. 9 is a schematic diagram for explaining an example of collecting interaction corpora in the exhibition hall using the posture detection system shown in FIG.

  As shown in FIG. 9, a visitor P2 or the like wears a portable computer 10 while wearing a headset 4 (infrared tags 6-1 to 6-4) on the head. The infrared tag 6 attached to the object is attached to the chest of the instructor P1, and as an environment-side object, the computer M1 and the robot M2 for explaining the exhibition, the stuffed toy M3 for exhibition, the board B1 for explaining the exhibition, etc. Attach and send each individual ID number. Moreover, the detection apparatus 1 is attached to the ceiling or wall of the exhibition hall.

  By arranging each device as described above, for example, the visitor P2 is identified by the detection device 1, and the posture of the head of the visitor P2 is detected. As an object, the instructor P1, the robot M2, and the like are identified. In this way, it is possible to collect interaction data relating to the interaction between people in the exhibition hall, and it is possible to create an interaction corpus as a knowledge base from the collected data.

  In the above description, interaction corpus collection has been described as an example. However, the application to which the present invention can be applied is not particularly limited to the above example, and can be applied to various applications. Further, although one infrared tag is attached to one object, a plurality of infrared tags that transmit the same ID number to one object may be provided. Further, in the present embodiment, the captured image is binarized or quaternarized, but examples of multilevel binarization are not particularly limited to these examples, and other numbers such as ternarization may be used. .

  Moreover, in this Embodiment, although the some infrared tag 6-1 to 6-4 was provided in the headset 4 with which a user's head is mounted | worn, it is not specifically limited to this example, Other of a user's body An infrared tag may be attached to a part such as a hand or a foot. In the present embodiment, the infrared tag 6-1 to 6-4 is provided in the headset 4, but the configuration of the headset is not particularly limited to this example. The image capturing unit 3 may be attached to the headset 4 to detect an infrared tag of an object in the direction of the user's line of sight.

It is a block diagram which shows the structure of the attitude | position detection system by one embodiment of this invention. It is a typical perspective view which shows an external appearance when a user wears the headset shown in FIG. FIG. 3 is a schematic top view and a right side view for explaining the configuration of the headset shown in FIG. 2. It is a block diagram which shows the structure of the image processing apparatus shown in FIG. It is a flowchart for demonstrating the attitude | position detection process by the detection apparatus shown in FIG. It is a communication sequence diagram for demonstrating the communication process between the infrared tag shown in FIG. 1, and a detection apparatus. It is a block diagram which shows the structure of the other example of the CMOS image sensor and image processing apparatus which are shown in FIG. It is a flowchart for demonstrating the attitude | position detection process using the CMOS image sensor and image processing apparatus which are shown in FIG. It is a schematic diagram for demonstrating the collection example of the interaction corpus in the exhibition hall using the attitude | position detection system shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Detection apparatus 2 Infrared detection part 3 Image photographing part 4 Headset 5 Microphone part 6-1 to 6-4 Infrared tag 10 Portable computer 11 Server 21 Reflective element 22 CMOS image sensor 23 Image processing apparatus 24 Attitude detection part 31 Lens 32 Optical element 33 CCD camera 51 Audio processing circuit 52 Microphone 61 LED
62 drive circuit 81 binarization circuit 82 SRAM
83 Lightness Change Detection Circuit 84 Decode Circuit 91 Quadrature Circuit 92 Frame Buffer 93 SRAM
94 brightness change detection circuit 95 decoding circuit

Claims (7)

A light emitting device mounted at a predetermined position of the object, and a detection device for detecting the light emitting device,
The light emitting device
A plurality of light emitting means that are fixed in place with respect to the object and emit infrared rays;
Each light emitting means blinks according to the identification information uniquely assigned to each light emitting means, and at least one light emitting means is provided according to the three-dimensional position information representing the three-dimensional positional relationship of the plurality of light emitting means. Light emission control means for flashing,
The detection device includes:
Photographing means for photographing an infrared image of a predetermined photographing region including the object;
A light emission detecting means for detecting the plurality of light emitting means using an infrared image photographed by the photographing means;
Information detecting means for detecting blinking states of the plurality of light emitting means detected by the detecting means to detect identification information and three-dimensional position information of each light emitting means;
A posture detection system comprising: posture detection means for detecting the posture of the object based on identification information and three-dimensional position information detected by the information detection means.   The light emission control means causes each light emission means to blink according to the identification information and blinks according to three-dimensional position information representing its own three-dimensional positional relationship with respect to other light emission means. The posture detection system according to 1.   The plurality of light emitting means are fixed to a rigid member, the rigid member is fixed to an elastic member, and the light emitting device is attached to the object via the elastic member. Or the attitude | position detection system of 2 description. The light emitting device is mounted on a user's head,
The said attitude | position detection means detects the attitude | position of the said user's head based on the identification information and the three-dimensional position information which were detected by the said information detection means. Attitude detection system. The plurality of light emitting means includes three or more light emitting means,
5. The posture detecting means detects the posture of the user's head based on identification information and three-dimensional position information of three or more light emitting means detected by the information detecting means. Attitude detection system. The plurality of light emitting means includes
First light emitting means for emitting infrared light in front of the user's head;
Second light emitting means for emitting infrared light to the right side of the user's head;
A third light emitting means for emitting infrared light to the left side of the user's head;
6. The posture detection system according to claim 4, further comprising fourth light emitting means for emitting infrared light behind the user's head. A light emitting device mounted at a predetermined position of an object,
A plurality of light emitting means that are fixed in place with respect to the object and emit infrared rays;
Each light emitting means blinks according to the identification information uniquely assigned to each light emitting means, and at least one light emitting means is provided according to the three-dimensional position information representing the three-dimensional positional relationship of the plurality of light emitting means. A light emitting device comprising: a light emission control means for blinking.

Images