JP2023068869A - Wireless transmission/reception system - Google Patents

Abstract

To provide a wireless transmission/reception system in which a wireless receiver can surely receive a signal transmitted wirelessly from a wireless transmitter.SOLUTION: A wireless transmission/reception system 1 comprises: a wireless transmitter 10; and a wireless receiver 20. The wireless transmitter 10 comprises an antenna 11, an imaging unit 12, a control unit 19 and the like. The control unit 19 receives input of image data acquired by the imaging unit 12, detects the position of the wireless receiver 20 in the image and orients the transmission beam direction of the antenna 11 toward the wireless receiver 20 on the basis of the position of the wireless receiver 20. The wireless receiver 20 comprises an antenna 21, an imaging unit 22, a control unit 29 and the like. The control unit 29 receives input of image data acquired by the imaging unit 22, detects the position of the wireless transmitter 10 in the image and orients the reception beam direction of the antenna 21 toward the wireless transmitter 10 on the basis of the position of the wireless transmitter 10.SELECTED DRAWING: Figure 1

Description

The present invention relates to a radio transmitting/receiving system.

2. Description of the Related Art A radio transmission/reception system wirelessly transmits and receives signals between a radio transmitter having an antenna for signal transmission and a radio receiver having an antenna for signal reception via each antenna. Devices used as wireless transmitters include, for example, cameras used in outside-in systems, wearable cameras, mobile devices such as notebook computers and smartphones, video playback devices, and the like. Devices used as wireless receivers include, for example, head mounted displays (HMDs), monitor devices, mobile devices such as notebook computers and smartphones, and smart glasses (AR glasses).

Devices used as radio transmitters or radio receivers are not limited to the above examples. A game system including an outside-in camera (hereinafter sometimes simply referred to as a "camera") as a wireless transmitter and an HMD as a wireless receiver will be described below as a wireless transmission/reception system.

Games using HMDs include first-person games and mirror games. First-person games are games that are played from a graphical perspective seen by the player character's own eyes, such as kickboxing, soccer, and baseball games. In the first-person game, the behavior of the person wearing the HMD is reflected in the game. On the other hand, a mirror-type game is a game in which an image of the player character is reflected in the game, and includes, for example, a dance game and a dress-up game. In the mirror-type game, the behavior of the person wearing the HMD is reflected in the game, or an image of the person wearing the HMD is displayed during the game.

Game systems using HMDs include an inside-out system and an outside-in system. The inside-out system displays an image captured by a camera provided on the HMD on the display of the HMD. Compared to the outside-in system, the inside-out system does not require the transmission and reception of wireless signals. It is difficult or impossible to capture blind spot information.

On the other hand, the outside-in system can use motion information of the person's own body in the blind spot from the HMD. The outside-in system, for example, uses leg motion information in kickboxing and the like, and uses back motion information in a dress-up game. In the outside-in system, one or a plurality of cameras provided separately from the HMD take images of a person wearing the HMD or detect their behavior, and the obtained imaging data or motion data signals are sent. Transmit wirelessly to the HMD. The HMD that has received the signal transmitted from the camera displays an image on the display of the HMD based on the signal.

In a radio transmitting/receiving system such as the above example, it is important that a radio receiver reliably receives a signal that is radio-transmitted from a radio transmitter. An invention aimed at this is disclosed in Patent Document 1.

JP 2013-51570 A Japanese Patent No. 4689758

The invention disclosed in Patent Document 1 can achieve the intended purpose in a limited situation or application in the radio transmitting/receiving system as described above. However, for example, if the distance between the wireless transmitter and the wireless receiver is long, or if it is necessary to send a large amount of information from the wireless transmitter to the wireless receiver at high speed, the multi-viewer on the wireless transmitter side When it is necessary to display high-resolution video captured by a camera, etc., on the display of the wireless receiver with low latency so as not to give the player a sense of discomfort, etc., antennas for both the wireless transmitter and wireless receiver It is conceivable that continuous high-speed communication with a high gain and always maintaining a facing state or a state close to it is required. Have difficulty.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and is a wireless transmission/reception system that enables a wireless receiver to reliably receive a signal wirelessly transmitted from a wireless transmitter in various situations or applications. intended to provide

A radio transmission/reception system of the present invention includes a radio transmitter that transmits a signal and a radio receiver that receives the signal transmitted from the radio transmitter. The radio transmitter includes a first antenna capable of controlling a transmission beam direction and transmitting a signal, a first imaging unit capturing an image of a range including the radio receiver, and an image captured by the first imaging unit. and a first controller for directing the transmit beam of the first antenna to the radio receiver based on the position of the radio receiver at. The radio receiver includes: a second antenna capable of controlling the direction of a receiving beam and receiving a signal; a second imaging section for capturing an image of a range including the radio transmitter; and an image captured by the second imaging section. and a second controller for directing the receive beam of the second antenna toward the wireless transmitter based on the location of the wireless transmitter at.

It is preferable that the radio transmitter is attached with a first marker. In this case, the second control unit of the radio receiver controls the position of the first marker in the image captured by the second imaging unit. Preferably, the receive beam direction of the second antenna is directed toward the radio transmitter based on. Also, it is preferable that the radio receiver is attached with a second marker. Preferably, the transmit beam of the first antenna is steered to the radio receiver based on the position.

The wireless transmitter further includes a first ranging unit that measures the distance to the wireless receiver, and the first control unit adjusts the transmission power of the first antenna based on the distance measurement by the first ranging unit. is preferred. The radio receiver further includes a second distance measuring unit that measures the distance to the radio transmitter, and the second control unit adjusts the reception sensitivity of the second antenna based on the distance measurement value obtained by the second distance measuring unit. Adjusting is preferred.

Preferably, the radio transmitter transmits a signal containing video information to the radio receiver. Preferably, the radio transmitter transmits uncompressed signals to the radio receiver. The position or orientation of the radio transmitter and/or radio receiver may be undefined.

According to the present invention, a wireless receiver can reliably receive a signal wirelessly transmitted from a wireless transmitter in various situations or applications.

FIG. 1 is a diagram showing the configuration of a radio transmission/reception system 1. As shown in FIG. FIG. 2 is a diagram explaining a method of detecting the position of the HMD in an image obtained by imaging a range including the HMD. FIG. 3 is a flow chart for explaining the operation of the control section 19 of the radio transmitter 10. As shown in FIG. FIG. 4 is a flow chart for explaining the operation of the control section 29 of the radio receiver 20. As shown in FIG. FIG. 5 is a diagram showing the configuration of the radio transmission/reception system 1A. FIG. 6 is a diagram showing the configuration of the radio transmission/reception system 1B.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and overlapping descriptions are omitted. The present invention is not limited to these exemplifications, but is indicated by the scope of the claims, and is intended to include all modifications within the meaning and scope of equivalents of the scope of the claims.

As a wireless transmission/reception system, an outside-in system including a camera as a wireless transmitter and an HMD as a wireless receiver will be described. In a radio transmitting/receiving system, the directivity of antennas of radio transmitters and radio receivers must be high for the following reasons.

The first reason is to take measures against space attenuation when using high-frequency radio waves. Signals transmitted and received between the camera and the HMD are mainly video signals. The video displayed on the display of the HMD should have a high resolution (e.g., 4K, 8K, etc.) and a high frame rate (e.g., 60 fps, 120 fps, etc.) from the viewpoint of immersive feeling for the person wearing the HMD. is desirable. High-speed communication is necessary to transmit high-resolution, high-frame-rate video. High-speed wireless transmission and reception requires the use of broadband radio waves. At this time, millimeter waves of gigahertz or higher and submillimeter waves such as terahertz are used as the central carrier frequency for wireless communication. It is known that radio waves propagating in space have a larger spatial loss in inverse proportion to the square of the wavelength, that is, a shorter communicable distance (Friis equation). To lengthen the communication distance, it is necessary to increase the effective radiated power (EIRP). For this purpose, it is important to improve the directivity of the antennas on both the transmitting and receiving sides and increase the antenna gains on both the transmitting and receiving sides.

The second reason is to avoid radio wave interference between multiple wireless communications. A plurality of cameras may be provided in an outside-in system. When a plurality of cameras are provided, it is necessary to avoid interference of radio waves of a plurality of wireless communications. Techniques for avoiding this include multiple access such as frequency division multiplexing and time division multiplexing. However, since these multiple accesses put pressure on the communication band, it becomes a factor of increasing latency (communication delay, device response speed reduction). Latency is a factor that hinders the sense of immersion in first-person games and mirror games, so it is not desirable. To solve this problem, it is important to improve the directivity of antennas on both the transmitting and receiving sides.

From the above, when realizing an outside-in system with high directivity, active array antennas are used as antennas for each of the camera and HMD, and by utilizing beamforming in this active array antenna, each antenna of the camera and HMD It is expected that the directivity of the By electrically controlling the direction of transmission or reception by the antenna, the active array antenna can improve the antenna sensitivity in a specific direction and reduce interference with other radio equipment. Quality can be improved.

However, when this antenna system realizes an outside-in system with improved directivity, there are the following problems. Wireless communication between the camera and the HMD is realized by directing a wireless main beam with enhanced directivity toward the other party. That is, it is necessary to accurately direct the main beam of the antenna of the camera toward the HMD, and to direct the main beam of the antenna of the HMD toward the camera. To achieve this, both the sender and the receiver need to estimate the direction in which the other party is. This estimation of the other party's direction is sometimes performed based on the information of the transmitted and received radio signals, but it may be difficult or take a long time when the radio wave power is low.

For example, IEEE802.11ad, which uses millimeter waves (60 GHz) as a carrier frequency, employs a sector level sweep method as adaptive beamforming. In this method, beam direction estimation (opposite direction estimation) is performed by repeatedly widening and narrowing the directivity of the antenna. When the directivity of the antenna is widened, there is an advantage in that the direction of arrival of the radio beam can be widely estimated.

Since the HMD also moves from moment to moment depending on the position and face orientation of the person wearing the HMD, if it takes time to estimate the direction of the other party, content latency and connection loss will occur. This problem becomes more pronounced as the frequency of wireless communication increases. Furthermore, in a closed space such as a room, the radio waves are diffusely reflected by the walls, etc., so the reflected waves may be stronger than the direct waves, and the main beam may be directed in a completely different direction from the other party's antenna. There is Since outside-in systems are often used indoors, it is important to deal with this problem.

The radio transmitting/receiving system of the embodiment described below has been devised based on the knowledge and studies of the inventors as described above.

FIG. 1 is a diagram showing the configuration of a radio transmission/reception system 1. As shown in FIG. A radio transmission/reception system 1 includes a radio transmitter 10 and a radio receiver 20 . Radio transmitter 10 transmits radio signals to radio receiver 20 . The radio receiver 20 receives signals transmitted from the radio transmitter 10 .

The position or orientation of the radio transmitter 10 and/or the radio receiver 20 is uncertain. The wireless transmitter 10 is, for example, a camera, a wearable camera, a mobile device such as a notebook computer or a smart phone, a video playback device, or the like. The wireless receiver 20 is, for example, an HMD, a monitor device, a mobile device such as a notebook computer or a smart phone, smart glasses (AR glasses), and the like.

The wireless transmitter 10 includes an antenna (first antenna) 11, an imaging section (first imaging section) 12, an image processing section 13, a baseband processing section 14, a modulation section 15, an antenna control section 16, and a control section (first control section). part) 19.

The antenna 11 transmits a radio signal, and is capable of controlling the transmission beam direction (main beam direction) of the radio signal. The image capturing unit 12 captures an image of a range including the wireless receiver 20 and outputs image data obtained by the capturing to the image processing unit 13 . The image processing unit 13 receives the image data output from the imaging unit 12, performs image processing on the image data, and outputs the processed image data to the baseband processing unit 14 and the control unit 19. .

The baseband processing unit 14 receives the image data output from the image processing unit 13 , performs baseband processing on the image data, and outputs the processed image data to the modulation unit 15 . The modulation unit 15 receives the image data output from the baseband processing unit 14 , modulates the image data, and outputs the processed image data to the antenna 11 .

Antenna control unit 16 controls the transmission beam direction of the radio signal transmitted by antenna 11 . The control unit 19 controls the overall operation of the radio transmitter 10 . In particular, the control unit 19 receives image data captured by the imaging unit 12, processed by the image processing unit 13 and output, detects the position of the wireless receiver 20 in the image, and detects the position of the wireless receiver 20 in the image. The antenna controller 16 is instructed to direct the transmission beam direction of the antenna 11 toward the radio receiver 20 based on the position of .

The wireless receiver 20 includes an antenna (second antenna) 21, an imaging section (second imaging section) 22, an image processing section 23, a baseband processing section 24, a modulation section 25, an antenna control section 26, a video processing section 27, a display A unit 28 and a control unit (second control unit) 29 are included.

The antenna 21 receives a radio signal (video signal), and can control the receiving beam direction (main beam direction) of the radio signal. The image capturing unit 22 captures an image of a range including the wireless transmitter 10 and outputs image data obtained by the capturing to the image processing unit 23 . The image processing unit 23 receives the image data output from the imaging unit 22 , performs image processing on the image data, and outputs the processed image data to the control unit 29 .

Modulator 25 receives video signal data received by antenna 21 , demodulates the video signal data, and outputs the processed video signal data to baseband processor 24 . The baseband processing unit 24 receives the video signal data output from the modulation unit 25, performs baseband processing on the video signal data, and outputs the processed video signal data to the video processing unit 27. . The video processing unit 27 receives the video signal data output from the baseband processing unit 24, converts the video signal data into data that can be displayed on the display unit 28, and displays the converted video signal data on the display unit. 28. The display unit 28 receives the video signal data output from the video processing unit 27 and displays an image based on the video signal data.

Antenna control unit 26 controls the reception beam direction of the radio signal received by antenna 21 . Control unit 29 controls the overall operation of radio receiver 20 . In particular, the control unit 29 receives image data captured by the imaging unit 22, processed by the image processing unit 23, and output, detects the position of the wireless transmitter 10 in the image, and detects the position of the wireless transmitter 10 in the image. The antenna controller 26 is instructed to direct the reception beam direction of the antenna 21 toward the radio transmitter 10 based on the position of .

It is preferable that each of antenna 11 and antenna 21 has high directivity of the main beam for transmission or reception. As an antenna with high directivity, an array antenna, a lens antenna with enhanced directivity using a lens, a reflect array using a reflecting mirror, and the like are used. Array antennas can control the beam direction using phase shifters. The lens antenna can control the beam direction by changing the position of the lens with a motor or the like.

Each of imaging unit 12 and imaging unit 22 includes a lens system and an imaging element. A CMOS image sensor, a CCD image sensor, or the like is used as an imaging device.

Detection of the position of the wireless transmitter 20 in the image by the controller 19 of the wireless transmitter 10 and detection of the position of the wireless transmitter 10 in the image by the controller 29 of the wireless receiver 20 use various image recognition techniques. This may be done by detecting feature points of the wireless receiver 20 or wireless transmitter 10 in the image. Appropriate image recognition technology can be selected according to the hardware to be implemented and the required positional accuracy. Image recognition techniques used here include, for example, YOLO (You Only Look Once), Deep Neural Network (DNN), the image recognition technique described in Patent Document 2, and the like. Among DNNs, R-CNN (Region Based Convolutional Neural Network) is preferably used.

FIG. 2 is a diagram explaining a method of detecting the position of the HMD in an image obtained by imaging a range including the HMD. An image obtained by imaging a range including the HMD (FIG. 2(a)) is divided into a plurality of regions (FIG. 2(b)) centered on each position in the image, and each of the plurality of regions is divided into A partial image is cut out from (FIG. 2(c)). The R-CNN is trained in advance using teacher image data from a plurality of HMDs. The partial images of each of the plurality of regions are input to the trained R-CNN (Fig. 2(d)), and the determination result of whether each partial image is an HMD is output from the R-CNN (Fig. 2(e) )). This makes it possible to detect the position of the partial image determined to be the HMD by the R-CNN (that is, the position of the HMD in the image (FIG. 2(a))).

FIG. 3 is a flow chart for explaining the operation of the control section 19 of the radio transmitter 10. As shown in FIG. The control unit 19 repeatedly performs the following steps S11 to S13. In step S<b>11 , the control unit 19 acquires from the imaging unit 12 an image obtained by imaging the range including the wireless receiver 20 with the imaging unit 12 . At step S12, the control unit 19 detects the position of the wireless receiver 20 in the acquired image. In step S<b>13 , the control unit 19 directs the transmission beam direction of the antenna 11 toward the radio receiver 20 based on the detected position of the radio receiver 20 .

FIG. 4 is a flow chart for explaining the operation of the control section 29 of the radio receiver 20. As shown in FIG. The control unit 29 repeatedly performs the following steps S21 to S23. In step S<b>21 , the control unit 29 acquires from the imaging unit 22 an image obtained by imaging the range including the wireless transmitter 10 with the imaging unit 22 . At step S22, the control unit 29 detects the position of the wireless transmitter 10 in the acquired image. In step S<b>23 , the control unit 29 directs the reception beam direction of the antenna 21 toward the wireless transmitter 10 based on the detected position of the wireless transmitter 10 .

Even if the position or orientation of either or both of the wireless transmitter 10 and the wireless receiver 20 is indefinite, the control unit 19 of the wireless transmitter 10 repeats each process of steps S11 to S13 in real time, The control unit 29 of the wireless receiver 20 repeats the processes of steps S21 to S23 in real time, so that the wireless receiver 20 can reliably receive the signal wirelessly transmitted from the wireless transmitter 10 .

For example, when the distance between the wireless transmitter 10 and the wireless receiver 20 is long, it is necessary to wirelessly send a large amount of information (for example, a signal including video information) from the wireless transmitter 10 to the wireless receiver 20 at high speed. In some cases, such as when it is necessary to display an image acquired by the imaging unit 12 on the wireless transmitter 10 side on the display unit 28 on the wireless receiver 20 side without latency, The signal can be reliably received by the radio receiver 20 . Also, in order to reduce the latency, the signal transmitted from the radio transmitter 10 to the radio receiver 20 is preferably an uncompressed signal rather than a signal compressed by a compression algorithm such as H.264. be.

Next, a modification of the radio transmission/reception system 1 (FIG. 1) will be described. FIG. 5 is a diagram showing the configuration of the radio transmission/reception system 1A. A radio transmission/reception system 1A includes a radio transmitter 10A and a radio receiver 20A. The radio transmitter 10A transmits radio signals to the radio receiver 20A. The radio receiver 20A receives the signal transmitted from the radio transmitter 10A.

Compared with the configuration of the radio transmitting/receiving system 1 (FIG. 1), the radio transmitting/receiving system 1A (FIG. 5) differs in that the radio transmitter 10A is provided with a marker (first marker) 31, and the radio receiver 20A is different in that a marker (second marker) 41 is attached. The difference is that the control unit 19 of the radio transmitter 10A directs the transmission beam direction of the antenna 11 to the radio receiver 20A based on the position of the marker 41 in the image captured by the imaging unit 12 and acquired. Also, the controller 19 of the radio receiver 20A is different in directing the reception beam direction of the antenna 21 to the radio transmitter 10A based on the position of the marker 31 in the image captured by the imaging unit 22 and acquired.

Each of the markers 31 and 41 may be a characteristic mark, or may emit or reflect light of a specific wavelength (for example, infrared rays). By attaching such markers to the wireless transmitter 10A and the wireless receiver 20A, the control unit 19 of the wireless transmitter 10A can detect the position of the marker 41 in the image captured by the imaging unit 12. Also, the control unit 19 of the radio receiver 20A can easily detect the position of the marker 31 in the image captured by the imaging unit 22 and acquired. This makes it possible to more reliably adjust the transmission beam direction of the antenna 11 in the radio transmitter 10A and the reception beam direction of the antenna 21 in the radio receiver 20A. can be received more reliably.

Next, another modification of the radio transmission/reception system 1 (FIG. 1) will be described. FIG. 6 is a diagram showing the configuration of the radio transmission/reception system 1B. A radio transmission/reception system 1B includes a radio transmitter 10B and a radio receiver 20B. The radio transmitter 10B transmits radio signals to the radio receiver 20B. The radio receiver 20B receives the signal transmitted from the radio transmitter 10B.

Compared to the configuration of the radio transmission/reception system 1 (FIG. 1), in the radio transmission/reception system 1B (FIG. 6), the radio transmitter 10B includes a distance measurement unit (first distance measurement unit) that measures the distance to the radio receiver 20B. 32 is further included, and the difference is that the control unit 19 adjusts the transmission power of the antenna 11 based on the distance measurement value obtained by the distance measuring unit 32 . Further, the radio receiver 20B further includes a distance measuring unit (second distance measuring unit) 42 that measures the distance to the radio transmitter 10B, and the control unit 29 controls the antenna based on the distance measurement value obtained by the distance measuring unit 42. 21 is different in that the reception sensitivity of V.21 is adjusted.

Each of the distance measurement unit 32 and the distance measurement unit 42 may be a TOF (Time-of-Flight) sensor or a stereo camera. By providing such a distance measuring unit, the transmission power of the antenna 11 in the radio transmitter 10B is appropriately set, the reception sensitivity of the antenna 21 in the radio receiver 20B is appropriately set, and the radio transmitter The wireless receiver 20B can more reliably receive the signal wirelessly transmitted from 10B.

DESCRIPTION OF SYMBOLS 1, 1A, 1B... Radio|wireless transmission/reception system, 10, 10A, 10B... Radio transmitter, 11... Antenna, 12... Imaging part, 13... Image processing part, 14... Baseband process part, 15... Modulation part, 16... Antenna Control unit 19 Control unit 20, 20A, 20B Wireless receiver 21 Antenna 22 Imaging unit 23 Image processing unit 24 Baseband processing unit 25 Modulation unit 26 Antenna control unit , 27... Video processing unit, 28... Display unit, 29... Control unit, 31... Marker, 32... Distance measuring unit, 41... Marker, 42... Distance measuring unit.

Claims (8)

A wireless transmitter that transmits a signal, and a wireless receiver that receives the signal transmitted from the wireless transmitter,
The radio transmitter includes a first antenna capable of controlling a transmission beam direction and transmitting a signal, a first imaging unit imaging a range including the radio receiver, and obtained by being imaged by the first imaging unit. a first controller that directs a transmission beam direction of the first antenna toward the wireless receiver based on the position of the wireless receiver in the captured image;
The radio receiver includes a second antenna capable of controlling a reception beam direction and receiving a signal, a second imaging unit imaging a range including the radio transmitter, and an image obtained by being imaged by the second imaging unit. a second controller that directs a receive beam direction of the second antenna toward the wireless transmitter based on the position of the wireless transmitter in the captured image;
Radio transmitting and receiving system. the radio transmitter is labeled with a first marker;
A second control unit of the wireless receiver directs a reception beam direction of the second antenna toward the wireless transmitter based on the position of the first marker in the image captured by the second imaging unit,
The radio transmitting/receiving system according to claim 1. the radio receiver is marked with a second marker;
A first control unit of the wireless transmitter directs a transmission beam direction of the first antenna to the wireless receiver based on the position of the second marker in the image captured by the first imaging unit,
3. The radio transmission/reception system according to claim 1 or 2. The radio transmitter further includes a first distance measuring unit that measures a distance to the radio receiver, and the first control unit controls the distance measurement of the first antenna based on the distance measurement value obtained by the first distance measuring unit. adjust the transmit power,
The radio transmission/reception system according to any one of claims 1 to 3. The radio receiver further includes a second distance measuring section that measures a distance to the radio transmitter, and the second control section controls the distance measurement of the second antenna based on the distance measurement value obtained by the second distance measuring section. adjust the reception sensitivity,
The radio transmission/reception system according to any one of claims 1 to 4. the wireless transmitter transmits a signal containing video information to the wireless receiver;
The radio transmission/reception system according to any one of claims 1 to 5. the wireless transmitter transmits an uncompressed signal to the wireless receiver;
The radio transmission/reception system according to any one of claims 1 to 6. the position or orientation of either or both of the wireless transmitter and/or the wireless receiver is uncertain;
The radio transmission/reception system according to any one of claims 1 to 7.

Images