JP7199822B2 - Detection device and detection method - Google Patents

Description

The present invention relates to a detection device and detection method.

As a method for detecting an object that performs some action with a camera, for example, Patent Document 1 discloses optical motion capture using near-infrared light.

Further, as a method for improving the detection accuracy of an object from an image obtained by imaging, Patent Document 2 discloses a method of performing detection focusing on the color of each pixel of the image. Further, Patent Document 3 discloses a method of detecting an object using a background subtraction method.

JP 2005-256232 A JP 2012-14535 A JP-A-2005-50382

In recent years, there is a need to detect rapidly moving objects using cameras mounted on smartphones and the like. However, since the method described in Patent Document 1 uses near-infrared light, it is necessary to add a special mechanism or the like. In addition, both the method described in Patent Document 2 and the method described in Patent Document 3 require more image processing for detecting an object than in a conventional camera. software implementation costs may increase. Moreover, it is conceivable that neither the method described in Patent Document 2 nor the method described in Patent Document 3 can accurately detect the object when the moving speed of the object is high.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a detection apparatus and a detection method capable of detecting a moving object from a captured image with a simpler configuration.

In order to achieve the above object, a detection device according to one aspect of the present invention includes a light source unit that irradiates light onto an object to which a retroreflecting material is attached, and the retroreflection by the light from the light source unit. an imaging unit that receives reflected light from a material and captures an image of the object; and reflector detection that specifies an area where the retroreflective material is captured from image data obtained by imaging by the imaging unit. and a parameter estimation unit for estimating a parameter related to the arrangement of the object based on the detection result of the reflector detection unit.

Further, the detection method according to one aspect of the present invention includes irradiating an object to which a retroreflecting material is attached with light from a light source, and reflecting light from the retroreflecting material from the light source. An imaging step of receiving light by an imaging unit to capture an image of the object, and a reflector detecting unit specifying an area in which the retroreflecting material is imaged from image data obtained by imaging in the imaging step. and a parameter estimation step of estimating a parameter relating to the arrangement of the object based on the detection result in the reflector detection step.

ADVANTAGE OF THE INVENTION According to this invention, the detection apparatus and detection method which can detect a moving object from the imaged image by simpler structure are provided.

It is a figure explaining the usage example of the detection apparatus which concerns on one Embodiment of this invention. It is a block diagram explaining the function of a detection apparatus. It is a figure explaining the outline of a series of processings concerning the detection of the target object by a detection apparatus. FIG. 5 is a diagram illustrating a procedure for setting parameters by a parameter setting unit; It is a figure explaining the procedure concerning the detection of a reflector. It is a figure explaining the procedure concerning estimation of a parameter. FIG. 10 is a diagram for explaining changes in contours of retroreflecting materials due to differences in shutter speed; It is a figure explaining calculation etc. of the position of a target object. It is a figure explaining the case where the number of retroreflective materials is changed. It is a figure explaining the hardware constitutions of a detection apparatus.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

FIG. 1 is a diagram illustrating a usage example of a detection device 1 according to one embodiment of the present invention. FIG. 2 is a block diagram illustrating functions of the detection device 1. As shown in FIG. The detection device 1 is implemented as a mobile terminal such as a smart phone, for example. Further, when the detection device 1 is a smartphone, the function of the detection device 1 is, for example, to provide the user with a view using the smartphone as if the user exists in a virtual space. It can be used in providing an immersive virtual reality (VR) experience. However, the function of the detection device 1 can be applied to applications other than those described above.

FIG. 1 shows an example in which the detection device 1 functions as part of an HMD (Head Mounted Display) 2 worn by a user U. As shown in FIG. A known HMD can be used as the HMD 2 . The detection device 1 has a function of capturing an image of an object O to be detected and detecting the object O. As shown in FIG. A retroreflective material 3 is attached to the object O. As shown in FIG. The object O is detected by irradiating the object O with light from the detection device 1 and detecting the light reflected by the retroreflective material 3 with the detection device 1 . Information related to the detected object O is presented to the user via the HMD 2 by being displayed on the monitor or the like of the detection device 1 .

The object O is not particularly limited, and examples thereof include familiar objects such as plastic bottles, original objects produced by a 3D printer, and human bodies. In addition, as the retroreflecting material 3 to be attached (pasted) to the object O, for example, a known retroreflecting material such as a capsule lens type, an enclosed lens type, and an exposed lens type can be appropriately selected and used. can. In addition, a plurality of retroreflectors 3 can be attached to the object O while being spaced apart from each other. When a plurality of retroreflecting materials 3 are attached to the object O, it is possible to specify the position, orientation, etc. of the object O from the arrangement of the plurality of retroreflecting materials 3 . This point will be described later.

As shown in FIG. 2 , the detection device 1 includes an imaging section 11 , a light source section 12 , a camera parameter setting section 13 , a reflector detection section 14 , a parameter estimation section 15 and a display section 16 .

The imaging unit 11 has a function of imaging the object O. As shown in FIG. For example, an RGB camera can be used as the imaging unit 11 . As the imaging unit 11, an RGB camera that is generally installed in mobile terminals such as smartphones can be used. However, a camera of a different type than the RGB camera may be used. Further, the imaging unit 11 may acquire the image of the object O as a still image or as a moving image.

The light source unit 12 has a function of irradiating the object O with light. As the light source unit 12, for example, a flashlight (flashlight) can be used. Although the wavelength of the light emitted from the light source unit 12 is not particularly limited, the light emitted from the light source unit 12 is required to include light in a wavelength band detectable by the imaging unit 11 . As the light source unit 12, a flashlight that is generally installed in mobile terminals such as smartphones can be used. The imaging unit 11 and the light source unit 12 are provided outside the detection device 1 as shown in FIG.

The camera parameter setting unit 13 has a function of making various settings related to the imaging unit 11 and the light source unit 12 . The camera parameter setting unit 13 controls, for example, changes in shutter speed settings when the imaging unit 11 captures an image of the object O, switching between light emission/stop by the light source unit 12, and the like. Based on the setting of the camera parameter setting unit 13 , the light source unit 12 irradiates the object O with light, and the imaging unit 11 captures the reflected light from the retroreflection material 3 .

The reflector detection unit 14 has a function of specifying an area where the retroreflector 3 is captured from the image captured by the imaging unit 11 . Although the details will be described later, the position of the object O can be specified by specifying the area where the retroreflector 3 is imaged by the reflector detector 14 .

The parameter estimating section 15 has a function of estimating parameters relating to the arrangement of the object O based on information such as the imaging area of the retroreflecting material 3 specified by the reflecting material detecting section 14 . Parameters related to the placement of the object O include, but are not limited to, parameters specifying the position and orientation of the object O, for example. A specific parameter estimation method will be described later.

The display unit 16 is composed of, for example, a monitor or the like, and has a function of displaying information related to the parameters estimated by the parameter estimation unit 15 . Alternatively, the image picked up by the imaging unit 11 may be displayed on the display unit 16 as it is. Further, when the detection device 1 is a device that provides the user U with a VR experience, the display unit 16 displays an image for presenting the user U with the VR experience. At that time, the display unit 16 displays an image based on the parameters estimated by the parameter estimation unit 15 .

Next, a method of detecting the object O by the detection device 1 will be described with reference to FIGS. 3 and 4 to 6. FIG. FIG. 3 is a diagram for explaining an outline of a series of processing related to detection of the object O by the detection device 1. As shown in FIG. 4 to 6 are diagrams for explaining the details of the processing performed in each step.

The detection device 1 first takes an image of the object O to which the retroreflection material 3 is attached (S01: imaging step). Specifically, when the light source unit 12 irradiates the object O with light, the imaging unit 11 images reflected light from the retroreflection material 3 . As a result, image data in which reflected light from the retroreflection material 3 is imaged is obtained. Next, the object O is detected by specifying the region where the retroreflector 3 is imaged from the image data by the imaging unit 11 by the reflector detector 14 (S02: reflector detection step). Next, the parameter estimation unit 15 estimates parameters related to the arrangement of the object O (S03: parameter estimation step). Then, the estimation result is displayed by the display unit 16 (S04: display step). A specific description will be given below.

FIG. 4 shows the flow of processing related to parameter adjustment by the camera parameter setting unit 13 when the imaging unit 11 performs imaging (S01). The camera parameter setting unit 13 sets the shutter speed of the imaging unit 11 to about 0.5 ms to 1 ms (S11). This shutter speed is a value when the imaging unit 11 is an RGB camera. Also, the above shutter speed is faster than a general shutter speed for an RGB camera. By setting the shutter speed to the above value, even if the object O moves quickly, it is possible to reduce blurring or the like that occurs in the image. Also, when the above shutter speed is set, the amount of light received by the imaging unit 11 is reduced, so the resulting image becomes darker than when the normal shutter speed is used. On the other hand, in the camera parameter setting unit 13, the light source unit 12 is configured to constantly irradiate the object O with light while the imaging unit 11 is performing imaging (S12). With such a configuration, the retroreflecting material 3 emits reflected light with respect to the light from the light source unit 12. Therefore, in the imaging unit 11, objects other than the retroreflecting material 3 are imaged darkly. , the retroreflective material 3 can be imaged in a state in which a sufficient amount of light is ensured. Note that the execution order of setting the shutter speed (S11) and controlling the constant illumination of the light source unit 12 (S12) may be changed.

Next, FIG. 5 shows the flow of processing related to the detection of the object O in the image data acquired by the imaging unit 11 (S02). As described above, when the imaging unit 11 takes an image of the object O, the shutter speed is increased to reduce the amount of light received by the imaging unit 11, and the light source unit 12 irradiates the object O with light. , the amount of reflected light from the retroreflecting material 3 is ensured. With such a configuration, it is possible to suitably detect the retroreflection material 3 . However, depending on the amount of light, the retroreflection material 3 may not be properly detected. Therefore, when specifying the area where the retroreflection material 3 is imaged from the image data imaged by the imaging unit 11, the amount of light received by the imaging unit 11 is not appropriate, and the area where the retroreflection material 3 is imaged cannot be specified. If it is difficult, the shutter speed is adjusted and the image is captured again (S21). FIGS. 7(a), 7(b), and 7(c) are examples showing how the shape of the retroreflection material 3 appears in the image data due to the difference in shutter speed. FIG. 7A is a diagram showing an example of an image when the shutter speed of the imaging unit 11 is slower than the appropriate range for imaging the retroreflective material 3 (for example, 10 ms or longer). When the shutter speed is slower than the appropriate range, the image captured by the imaging unit 11 also includes information such as the shape of an object other than the area A where the retroreflection material 3 is captured. FIG. 7(b) is a diagram showing an example of an image when the shutter speed of the imaging unit 11 is within an appropriate range for imaging the retroreflective material 3 (for example, 0.5 ms to 1 ms). . When the shutter speed is within an appropriate range, the image captured by the imaging unit 11 does not include information such as the shape of the object other than the area A' where the retroreflection material 3 is captured. FIG. 7C is a diagram showing an example of an image when the shutter speed of the imaging unit 11 is faster than the appropriate range for imaging the retroreflective material 3 (for example, 0.2 ms or less). . If the shutter speed is faster than the appropriate range, the imaging unit 11 can only receive light in such an amount that the shape of the retroreflection material 3 cannot be recognized. Therefore, the image captured by the imaging unit 11 shows a region A'' in which the shape of the retroreflection material 3 cannot be recognized.

As described above, the shutter speed may change whether the shape of the retroreflection material 3 in the obtained image data can be appropriately detected. Therefore, based on image data captured at a predetermined shutter speed, the shutter speed may be finely adjusted and the image captured again.

The fine adjustment of the shutter speed may be performed by the user U of the detection device 1, and the camera parameter setting unit 13 of the detection device 1 determines whether re-imaging is necessary based on the captured image data. may be

Next, the reflector detection unit 14 acquires image data captured by the imaging unit 11 (S22). At this stage, it may be possible to work with grayscale images to reduce software processing. Next, the grayscale image is binarized to obtain a binary image (S23). A luminance threshold value is set for each pixel constituting a grayscale image, and binarization is performed using this threshold value. For example, when the luminance value ranges from 0 to 255, the threshold is set to 200, pixels with luminance less than 200 are treated as luminance 0, and pixels with luminance 200 or more are treated as luminance 255. By binarizing the image, if the shutter speed and the amount of light are appropriately adjusted, pixels that have received the reflected light from the retroreflective material 3 can be distinguished from other pixels.

Next, the binarized image is used to extract the contour of the captured object (S24). If the area of the contour-extracted region is greater than or equal to the threshold value (for example, 20 pixels in a 480×320 pixel image), the region is the region where the retroreflector 3 is imaged, that is, the retroreflector 3 is It is detected as an imaged area of the mounted object O (S25).

Next, the parameter estimation (S03) regarding the position of the object O by the parameter estimator 15 will be described with reference to FIG.

The position of the retroreflecting material 3 can be identified on the detection device 1 side by specifying the area where the retroreflecting material 3 attached to the object O is imaged by the reflecting material detection unit 14 . Therefore, by using the information about the position of the retroreflection material 3, the detecting device 1 can estimate the parameters about the shape of the object O and the like.

Specifically, first, the parameter estimating unit 15 acquires the positional information on the image of the two retroreflective members 3 attached to the object O, and then calculates the barycentric coordinates (S31). Here, as shown in FIG. 8A, the description will be made assuming that two retroreflectors 31 and 32 are attached to the object O. FIG. When the contours of the retroreflectors 31 and 32 are extracted by the reflector detector 14 (see S24), the center-of-gravity position of the retroreflector 31 can also be calculated. Therefore, for example, the position of the center of gravity of the retroreflecting material 31 can be calculated by approximating each of the retroreflecting materials 31 and 32 to a rectangle. Here, the center-of-gravity position P1 (see FIG. 8A) of the retroreflecting material 31 is (G _x1 , G _y1 ), and the center-of-gravity position P2 of the retroreflecting material 32 (see FIG. 8A) is ( G _x2 , G _y2 ). Note that the procedure for calculating the center-of-gravity position is not particularly limited, and for example, the configuration may be such that the center-of-gravity position is calculated without performing rectangular approximation.

If the center-of-gravity positions of the two retroreflectors 31 and 32 attached to the object O can be calculated, estimation of the parameters related to the object O can proceed. Specifically, the position of the object O can be estimated. The positions at which the retroreflecting materials 31 and 32 are attached to the object O are not particularly limited. Assuming that (the central position of) the object O is provided between the two retroreflectors 31 and 32, the position of the object O can be estimated (S32). If the position of the object O is estimated on this premise, for example, the position P (G _x , G _y ) of the object O (see FIG. 8A) can be calculated based on the following formula (1). can be done.

In addition, assuming that the object O (at the center position) is provided between the two retroreflectors 31 and 32, the orientation (direction) of the object O is estimated from the arrangement of the retroreflectors 31 and 32. (S33). For example, the orientation G _θ (see FIG. 8A) of the object O can be calculated based on the following formula (2).

Note that the method of estimating the position and orientation of the object O is not limited to the above, and can be changed as appropriate. Parameters can be estimated by appropriately changing the parameter estimation method (formula) relating to the position and orientation of the object O based on the positions of the retroreflective materials 31 and 32 attached to the object O. Accuracy can be improved.

The display contents of the display unit 16 can be changed by reflecting the parameters estimated by the parameter estimation unit 15 (S34). When the detection device 1 is a device that provides a VR experience to the user U, the display unit 16 associates the parameters estimated by the parameter estimation unit 15 with, for example, actions of a specific character (object) in the VR environment. Thus, an image reflecting the parameters of the object O (retroreflecting material) can be displayed.

Note that the method of estimating parameters by the parameter estimating unit 15 is appropriately changed depending on the number of retroreflecting materials attached to the object O and the like. For example, when the number of retroreflecting materials attached to the object O is three or more, the position of the object O is It is preferable to estimate the parameters related to the object O such as the direction and direction.

The example shown in FIG. 8(b) differs from the example shown in FIG. 8(a) in that a retroreflector 33 is provided. A movable cover C, which is a component of the object O, is attached to the retroreflection material 33. For example, the cover C can be removed from the retroreflection material 33 under the control of the operator of the object O. It's becoming In such a case, the number of retroreflectors detected by the reflector detector 14 may vary depending on the arrangement of the cover C. FIG. Therefore, the parameter estimator 15 can perform the estimation without considering the information on the retroreflector 33 when estimating the parameters related to the position and orientation of the object O. FIG.

If the detection device 1 is a device that provides a VR experience to the user U, the change in the number of retroreflectors detected in the detection device 1 due to removal of the cover C as shown in FIG. 8(b) may be used as an input interface for the detection device 1 in the VR environment.

Further, the method for realizing a change in the number of retroreflectors detected by the detection device 1 is not limited to the cover C described above. FIGS. 9(a) and 9(b) show an example of the retroreflection material 33 using PDLC 40 (polymer dispersed liquid crystal). PDLC has the property of turning an opaque white state into a transparent state by applying a voltage to the liquid crystal. Therefore, by overlapping the PDLC 40 on the retroreflecting material 33, it is possible to hide the retroreflecting material 33 and change the number of the retroreflecting materials. Specifically, normally, as shown in FIG. 9A, the retroreflection material 33 can be kept in a state in which light does not reach it. On the other hand, by dynamically controlling the application of voltage to the PDLC using a switch or the like, a state in which light reaches the retroreflecting material 33 (a state in which reflected light can be emitted) is achieved, as shown in FIG. ). Such a configuration for dynamically changing the number of retroreflectors detected by the detection device 1 among the retroreflectors attached to the object O can be realized by various methods.

As described above, in the detecting device 1 and the detecting method according to the present embodiment, by irradiating the object O to which the retroreflecting material 3 is attached with the light from the light source unit 12, the retroreflecting material 3 The image pickup unit 11 picks up the reflected light from the . Then, after the reflecting material detection unit 14 identifies the area where the retroreflecting material 3 is imaged, the parameter estimation unit 15 estimates the parameters related to the placement of the object O based on the detection result of the reflecting material detection unit 14. do. In the above-described detection device, it is possible to capture an image of reflected light from the retroreflection material 3 without providing a special configuration or the like for the imaging section 11 . In addition, specifying the region where the retroreflecting material 3 is imaged by the reflecting material detection unit 14 and estimating the parameters based on the result can also be performed without performing complicated calculations. Therefore, according to the detection device 1 and detection method according to this embodiment, it is possible to detect a moving object from a captured image with a simpler configuration.

In addition, a plurality of retroreflecting materials such as retroreflecting materials 31 and 32 are attached to the object, and the imaging unit 11 detects the reflection from the plurality of retroreflecting materials 31 and 32 attached to the object O. The light is received, and the parameter estimator 15 estimates a parameter related to the position or orientation of the object O as a parameter related to the arrangement of the object. By adopting such a configuration, it is possible to estimate a parameter related to the position or orientation of the object O based on information related to the regions in which a plurality of retroreflecting materials are imaged. Therefore, moving objects can be detected in more detail.

Further, the configuration may further include a camera parameter setting section 13 capable of changing the shutter speed of the imaging section 11 based on the intensity of the reflected light. With such a configuration, the shutter speed can be changed according to the material of the retroreflecting material and the light amount of the light source unit 12, so that the retroreflecting material can be imaged more preferably. Detection accuracy of moving objects is also enhanced. However, instead of providing the camera parameter setting unit 13, the shutter speed may be fixed in advance. Even in that case, the imaging unit 11 can image the retroreflective material.

Moreover, the configuration may further include a display unit 16 that reflects the parameters estimated by the parameter estimation unit 15 . With such a configuration, it is possible to display an image or the like according to the estimated parameters. However, instead of the configuration having the display unit 16, for example, a configuration in which the estimated parameters are directly output to an external device or the like may be employed.

Furthermore, the display unit 16 can be configured to reflect the parameters estimated by the parameter estimation unit 15 on a specific object displayed on the display unit 16 . With such a configuration, for example, it is possible to display an object that behaves similarly to the object O by reflecting the parameters of the object O (retroreflection material).

In addition, the reflector detection unit 14 binarizes the image data obtained by imaging with the imaging unit 11 to specify the contour of the retroreflector, thereby detecting the area where the retroreflector is imaged. can be specified. With such a configuration, it is possible to preferably specify the region where the image of the retroreflecting material is captured.

In the above embodiment, the case where the detection device 1 is configured by one device has been described, but the functions related to the detection device 1 may be distributed to a plurality of devices. .

Further, the method of detecting an area in which the retroreflecting material is imaged by the reflector detecting unit 14 and the method of estimating parameters by the parameter estimating unit 15 are not limited to the procedures described in the above embodiments, and can be changed as appropriate. can.
(others)

The block diagrams used in the description of the above embodiments show blocks in functional units. These functional blocks (components) are implemented by any combination of hardware and/or software. Further, means for realizing each functional block is not particularly limited. That is, each functional block may be implemented by one device physically and/or logically coupled, or may be implemented by two or more physically and/or logically separated devices directly and/or indirectly. These multiple devices may be connected together (eg, wired and/or wirelessly).

For example, the detection device 1 according to one embodiment of the present invention may function as a computer that performs the processing of the detection device 1 according to this embodiment. FIG. 10 is a diagram showing an example of the hardware configuration of the detection device 1 according to this embodiment. The detection device 1 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

Note that in the following description, the term "apparatus" can be read as a circuit, device, unit, or the like. The hardware configuration of the detection device 1 may be configured to include one or more of the devices shown in the figure, or may be configured without some devices.

Each function in the detection device 1 is performed by the processor 1001, by loading predetermined software (program) on hardware such as the memory 1002, the processor 1001 performs calculation, communication by the communication device 1004, memory 1002 and storage 1003 It is realized by controlling the reading and/or writing of data in the .

The processor 1001, for example, operates an operating system to control the entire computer. The processor 1001 may be configured with a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, registers, and the like. For example, each function of the detection device 1 may be implemented by the processor 1001 .

The processor 1001 also reads programs (program codes), software modules, and data from the storage 1003 and/or the communication device 1004 to the memory 1002, and executes various processes according to them. As the program, a program that causes a computer to execute at least part of the operations described in the above embodiments is used. For example, each function of the detection device 1 may be stored in the memory 1002 and implemented by a control program running on the processor 1001 . Although it has been described that the above-described various processes are executed by one processor 1001, they may be executed by two or more processors 1001 simultaneously or sequentially. Processor 1001 may be implemented with one or more chips. Note that the program may be transmitted from a network via an electric communication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and RAM (Random Access Memory). may The memory 1002 may also be called a register, cache, main memory (main storage device), or the like. The memory 1002 can store executable programs (program code), software modules, etc. to perform a method according to an embodiment of the invention.

The storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disk), smart card, flash memory (eg, card, stick, key drive), floppy disk, magnetic strip, and/or the like. Storage 1003 may also be called an auxiliary storage device. The storage medium described above may be, for example, a database, server, or other suitable medium including memory 1002 and/or storage 1003 .

The communication device 1004 is hardware (transmitting/receiving device) for communicating between computers via a wired and/or wireless network, and is also called a network device, network controller, network card, communication module, or the like. For example, each function of the detection device 1 may be implemented by the communication device 1004 .

The input device 1005 is an input device (for example, keyboard, mouse, microphone, switch, button, sensor, etc.) that receives input from the outside. The output device 1006 is an output device (eg, display, speaker, LED lamp, etc.) that outputs to the outside. Note that the input device 1005 and the output device 1006 may be integrated (for example, a touch panel).

Devices such as the processor 1001 and the memory 1002 are connected by a bus 1007 for communicating information. The bus 1007 may be composed of a single bus, or may be composed of different buses between devices.

Further, the detection device 1 includes hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). may be configured, and a part or all of each functional block may be realized by the hardware. For example, processor 1001 may be implemented with at least one of these hardware.

Although the present embodiments have been described in detail above, it will be apparent to those skilled in the art that the present embodiments are not limited to the embodiments described herein. This embodiment can be implemented as modifications and changes without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description in this specification is for the purpose of illustration and explanation, and does not have any restrictive meaning with respect to the present embodiment.

Notification of information is not limited to the aspects/embodiments described herein and may be done in other ways. For example, notification of information includes physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), higher layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, It may be implemented by broadcast information (MIB (Master Information Block), SIB (System Information Block)), other signals, or a combination thereof. RRC signaling may also be called an RRC message, and may be, for example, an RRC connection setup message, an RRC connection reconfiguration message, or the like.

The procedures, sequences, flow charts, etc. of each aspect/embodiment described herein may be interchanged so long as there is no inconsistency. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

Input/output information and the like may be stored in a specific location (for example, memory), or may be managed in a management table. Input/output information and the like may be overwritten, updated, or appended. The output information and the like may be deleted. The entered information and the like may be transmitted to another device.

The determination may be made by a value represented by one bit (0 or 1), by a true/false value (Boolean: true or false), or by numerical comparison (for example, a predetermined value).

Each aspect/embodiment described herein may be used alone, in combination, or switched between implementations. In addition, the notification of predetermined information (for example, notification of “being X”) is not limited to being performed explicitly, but may be performed implicitly (for example, not notifying the predetermined information). good too.

Software, whether referred to as software, firmware, middleware, microcode, hardware description language or otherwise, includes instructions, instruction sets, code, code segments, program code, programs, subprograms, and software modules. , applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, and the like.

Software, instructions, etc. may also be sent and received over a transmission medium. For example, the software can be used to access websites, servers, or other When transmitted from a remote source, these wired and/or wireless technologies are included within the definition of transmission media.

Information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may refer to voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. may be represented by a combination of

The terms explained in this specification and/or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

As used herein, the terms "system" and "network" are used interchangeably.

In addition, the information, parameters, etc. described in this specification may be represented by absolute values, may be represented by relative values from a predetermined value, or may be represented by corresponding other information. .

The names used for the parameters described above are not limiting in any way. Further, the formulas, etc. using these parameters may differ from those explicitly disclosed herein.

As used herein, the terms "determining" and "determining" may encompass a wide variety of actions. "Judgement", "determining" are, for example, judging, calculating, computing, processing, deriving, investigating, looking up (e.g., table , searching in a database or other data structure), regarding ascertaining as "determining" or "determining". Also, "judgment" and "determination" are used for receiving (e.g., receiving information), transmitting (e.g., transmitting information), input, output, access (accessing) (for example, accessing data in memory) may include deeming that a "judgment" or "decision" has been made. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" by resolving, selecting, choosing, establishing, comparing, etc. can contain. In other words, "judgment" and "decision" may include considering that some action is "judgment" and "decision".

The terms "connected", "coupled", or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, It can include the presence of one or more intermediate elements between two elements being "connected" or "coupled." Couplings or connections between elements may be physical, logical, or a combination thereof. As used herein, two elements are referred to by the use of one or more wires, cables and/or printed electrical connections and, as some non-limiting and non-exhaustive examples, radio frequency They can be considered to be “connected” or “coupled” to each other through the use of electromagnetic energy, such as electromagnetic energy having wavelengths in the microwave, light (both visible and invisible) regions.

As used herein, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

Where "first," "second," etc. designations are used herein, any reference to such elements does not generally limit the quantity or order of those elements. These designations may be used herein as a convenient method of distinguishing between two or more elements. Thus, references to first and second elements do not imply that only two elements may be employed therein or that the first element must precede the second element in any way.

Wherever "include," "including," and variations thereof are used in the specification or claims, these terms are synonymous with the term "comprising." are intended to be inclusive. Furthermore, the term "or" as used in this specification or the claims is not intended to be an exclusive OR.

In this specification, plural devices are also included unless the context or technicality clearly dictates that there is only one. Throughout this disclosure, the plural shall be included unless the context clearly indicates the singular.

DESCRIPTION OF SYMBOLS 1... Detection apparatus, 3, 31, 32... Retroreflection material, 11... Imaging part, 12... Light source part, 13... Camera parameter setting part, 14... Reflective material detection part, 15... Parameter estimation part, 16... Display part .

Claims (6)

A detection device that functions as part of an HMD worn by a user and presents information about an object to the user via the HMD,
a light source unit that emits light to the object to which the retroreflecting material is attached;
an imaging unit that receives light reflected from the retroreflective material by the light from the light source unit and performs imaging of the object;
a reflecting material detection unit that identifies an area in which the retroreflecting material is imaged from image data obtained by imaging by the imaging unit;
a parameter estimating unit for estimating a parameter related to the placement of the object based on the detection result of the reflector detecting unit;
a camera parameter setting unit capable of changing the shutter speed of the imaging unit based on the intensity of the reflected light;
A detection device, comprising: A plurality of the retroreflective materials are attached to the object,
The imaging unit receives reflected light from the plurality of retroreflective materials attached to the object,
The detection device according to claim 1, wherein the parameter estimator estimates a parameter related to the position or orientation of the object as the parameter related to the arrangement of the object. 3. The detection device according to claim 1 , further comprising a display unit that reflects the parameters estimated in the parameter estimation unit. 4. The detection device according to claim 3 , wherein said display unit reflects said parameter estimated by said parameter estimation unit on a specific object displayed on said display unit. The reflecting material detection unit specifies the contour of the retroreflecting material by binarizing the image data obtained by the imaging by the imaging unit, thereby detecting the area where the retroreflecting material is imaged. A detection device according to any one of claims 1 to 4 , wherein the detection device is specified. A detection method in a detection device that functions as part of an HMD worn by a user and presents information related to an object to the user via the HMD, comprising:
Light is emitted from a light source unit to the object to which the retroreflecting material is attached, and light reflected from the retroreflecting material by the light from the light source unit is received by an imaging unit, and is reflected on the object. an imaging step of performing such imaging, and performing imaging again by changing the shutter speed when the shape of the retroreflecting material cannot be detected in the image obtained by imaging the retroreflecting material;
a reflecting material detection step of identifying an area where the retroreflecting material is imaged from the image data obtained by imaging in the imaging step in the reflecting material detection unit;
a parameter estimating step of estimating a parameter related to the placement of the object based on the detection result of the reflector detecting step;
A detection method comprising:

Images