JP7214611B2 - Apparatus, program, and method for identifying terminal based on degree of correspondence with object in motion state - Google Patents

Description

The present invention relates to technology for acquiring information about a terminal as a communication target, and particularly to technology for identifying the terminal.

The forthcoming 5th generation mobile communication system (5G) will use millimeter-wave radio waves to enable high-performance communications such as high-speed, large-capacity, low-delay, and multi-terminal connections. Here, millimeter waves have high rectilinearity and are less likely to be diffracted, and there is a high possibility that their propagation will be blocked or attenuated by an object existing between the terminal and the base station. Therefore, in 5G, the presence of such an object causes the received power to drop sharply, resulting in a significant deterioration in communication quality, which is a serious problem.

Here, if it is possible to predict a decrease in received power due to such an object, it becomes possible to maintain communication quality by controlling communication paths of terminals, for example. As a technique for predicting the received power, for example, in Non-Patent Document 1, time-series 3D environment information acquired by an RGB-D camera capable of generating not only RGB images but also depth images is used, and machine learning is performed in a communication device. Techniques for predicting received power are disclosed.

Specifically, in this method, the time-series information of the received power measured (at the same time as the image is taken) at the terminal that received the radio waves from the transmitting station is used as the correct data for the time-series environmental information that is the result of imaging the communication environment. A machine learning model is constructed using the linked teacher data, and the predicted value of received power at the receiving terminal is determined from the time-series environmental information acquired by the RGB-D camera using this constructed machine learning model. ing.

Further, for example, Patent Document 1 discloses a technique for improving the estimation accuracy of communication quality in the initial operation stage of the system or in a communication environment with low frequency of occurrence in communication quality prediction processing by machine learning as disclosed in Non-Patent Publication 1. is disclosed. Specifically, in the virtual space formed from the measurement results of the communication environment, information is virtually generated by the RGB-D camera, and by estimating the communication quality of the receiving terminal in the virtual space by propagation simulation, the virtual space It is said that it is possible to generate a large amount of learning data above and build a machine learning model with improved estimation accuracy.

JP 2018-148297 A JP 2016-212675 A JP 2019-144941 A JP 2019-174164 A

Masashi Nishio, "Prediction and Control of Wireless Communication Quality by Machine Learning", IEICE Technical Report, Vol. 118, No. 57, SR2018-1, pp.1-8, 2018

However, in the conventional technology as disclosed in Non-Patent Document 1 described above, for example, in a normal situation in which a plurality of objects are included in time-series environmental information, among these objects, when the correct data is It is not possible to identify the terminal corresponding to the sequence received power information. As a result, it becomes very difficult to construct a machine learning model with high estimation accuracy that can be used in actual operation. That is, it is determined whether or not the object captured by the RGB-D camera (which may include the terminal) and the terminal related to the measured received power are the same object (whether or not there is a correspondence relationship). Therefore, highly reliable learning data cannot be generated in the first place.

Also, even if a machine learning model such as that disclosed in Non-Patent Document 1 is applied, for example, an object (terminal) detected from time-series environmental information generated by an RGB-D camera is detected by this machine learning model. It is completely unknown whether or not it relates to the received power predicted by the transmitting station. In the first place, it becomes impossible to even determine whether or not the predicted received power relates to any terminal.

In this regard, the technology disclosed in Patent Document 1 can determine the correspondence relationship between the terminal imaged by the RGB-D camera and the measured reception power in the virtual space. It is possible to build a machine learning model corresponding to However, even if such a model is applied, in the case of prediction of received power in actual operation, the terminal cannot be identified after all, resulting in the same problem as described in Non-Patent Document 1 described above.

Here, Patent Literature 2 discloses an object recognition system that grasps the behavior of a mobile terminal and associates the mobile terminal with a mobile object detected by a computer. In this system, each mobile terminal sequentially detects various state quantities such as acceleration acting on its own terminal, and transmits the time-series detection results to the server as terminal behavior data. Based on the signal, the behavior of moving objects present in the imaging range is monitored, and data indicating temporal changes in state quantities relating to the behavior of each moving object are stored in the server-side storage unit as estimation result history data. Next, by comparing the terminal behavior data transmitted from the mobile terminal and the estimation result history data for each mobile body, the correspondence relationship between the mobile terminal and the mobile terminal existing in the shooting range is specified.

However, in the system according to Patent Literature 2, the criterion for specifying the correspondence relationship is only the physical state quantity caused by the behavior of the mobile object, which is exactly the pending issue, in which each mobile terminal is connected to a base station. Information related to the communication performed is not utilized at all. In addition, since it is necessary for the mobile terminal to acquire information on a predetermined state that changes due to its own behavior, each mobile terminal must be equipped with equipment such as a sensor capable of detecting a predetermined state. Furthermore, since it is necessary to transmit the detection results one by one, the situation becomes technically troublesome.

Further, in Patent Document 3, in an environment where one or more persons possessing wireless receivers are staying or moving, flow line information of the person acquired using a camera or the like, A technique for associating a machine with (a person who owns the machine) is disclosed. Specifically, this technology identifies the time zone in which the person stays in a predetermined area from the flow line information of the person, and sets a radio transmitter and a receiver possessed by the person in a position corresponding to the predetermined area. A person who is likely to stay in a predetermined area during that time period is determined from the reception history information generated by communication with the person, and the person is associated with the flow line information specifying the time period. It is a thing.

However, in the technique of Patent Document 3, in order to associate a person possessing a wireless receiver with the flow line information of the person acquired by the camera, it is a major premise that the person is staying in a predetermined area. It's becoming Of course, setting a large number of predetermined areas increases the probability that the person will stay in the predetermined area, but then it becomes necessary to install a radio transmitter for each of the large number of predetermined areas.

Further, Patent Literature 4 discloses a technique for estimating the position of a terminal by acquiring received electromagnetic wave information in a terminal that has received an electromagnetic wave from an electromagnetic wave source (beacon). Specifically, this technology generates learning data including this received electromagnetic wave information, object recognition information acquired from an object recognition means capable of recognizing a movable object, and terminal position information relating to the correct position of the terminal. , generate a model for terminal position estimation, and use the generated model to determine the position estimation target based on the feature data including the received electromagnetic wave information and object recognition information at the time of position estimation acquired from the position estimation target terminal. It determines the position of the terminal.

However, in the technique of Patent Document 4, basically, the correspondence between the terminal related to the received electromagnetic wave information and the object (terminal) related to the object recognition information is based on the terminal position estimated from the received electromagnetic wave information and the object recognition information. Since it is based on the geometrical positional relationship (e.g., distance) with the acquired object position, there is a possibility of incorrect association if, for example, there are many objects near the estimated terminal position. becomes higher. Moreover, in practice, it is necessary to arrange a plurality of beacons within an area, and it is not realistic to apply this technique to the entire general communication area.

Therefore, the present invention provides a correspondence relationship between a terminal and an object existing in an environment in which the terminal may exist, without relying on the mutual positional relationship or only on the positional relationship, and information related to communication with the terminal. It is an object to provide an apparatus, program and method that can be used to determine.

According to the present invention, communication radio wave information, which is information related to communication radio waves, is acquired from a communication means that communicates with a terminal, and based on the communication radio wave information, terminal movement, which is information related to changes in the position of the terminal, is obtained. terminal motion state determining means for determining state information;
Acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect a predetermined target from the environmental information, and target that is information related to changes in the detection position of the target a target motion state determining means for determining motion state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. a correspondence relationship determining means for determining whether or not there is a correspondence relationship between the terminal and the target, and determining information related to the correspondence relationship between the terminal and the target ;
The target motion state determining means acquires target motion state information determined from environment information obtained from a sensor different from the sensor in the other terminal identification device from another terminal identification device existing in the surroundings,
The correspondence determination means also calculates the degree of correspondence between the acquired target motion state information and the determined terminal motion state information, and determines the correspondence between the terminal and the target based on the degree of correspondence. determine whether there is
A terminal identification device characterized by the following is provided.

As an embodiment of the terminal identification device according to the present invention, the communication radio wave information includes time-series information of the frequency of the received radio wave, time-series information of the intensity of the received radio wave, and time-series information of the radiation direction of the radio wave related to communication. , and at least one of round trip time chronological information in communication,
The terminal motion state information is determined from at least one of the shift of the frequency, the variation over time of the intensity, the variation over time of the radial orientation, and the variation per hour of the round-trip time. Information related to the speed or angular velocity of the terminal,
The target motion state information is information related to the velocity or angular velocity of the target determined from changes in the detected position of the target per time,
The degree of correspondence is the degree to which the information regarding the speed or angular velocity of the terminal matches the information regarding the speed or angular velocity of the target,
It is also preferable that the correspondence determination means determines whether or not the terminal and the object have a correspondence relationship based on the degree of matching.

Further, in the above embodiment, the terminal motion state information is selected from among the deviation of the frequency, the variation per hour of the intensity, the variation per hour of the radiation direction, and the variation per hour of the round trip time. a plurality of pieces of information relating to the velocity or angular velocity of the terminal determined from each of at least two of
The degree of matching is the degree of matching between each of the plurality of pieces of information relating to the speed or angular velocity of the terminal and the information relating to the velocity or angular velocity of the target, and is calculated multiple times,
The correspondence determination means determines whether the terminal and the object have a correspondence relationship based on the ratio of the plurality of matching degrees that are equal to or greater than a predetermined number, or the sum of the plurality of matching degrees or the sum of the weightings. It is also preferable to determine whether or not

Furthermore, as another embodiment of the terminal identification device according to the present invention, the terminal motion state means also determines terminal location information, which is information relating to the location of the terminal, based on the communication radio wave information,
The target motion state determining means also determines target position information, which is information relating to the detected position of the target detected from the environmental information,
The correspondence determining means calculates a degree of position matching, which is a degree of matching between the determined terminal position information and the determined target position information, and determines whether the terminal and the target are based on the degree of position matching. It is also preferable to determine whether or not there is correspondence.

Further, as still another embodiment of the terminal identification device according to the present invention, the sensors are a plurality of sensors provided at different positions,
The target motion state determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target motion state information from each of the plurality of environmental information, and among the determined target motion state information, Among those whose detection positions of the target related to the target motion state information are closer than a predetermined value , and those whose feature values of the detection regions of the target related to the target motion state information are similar to each other than a predetermined value For one or both , it is also preferable that the target motion state information relates to the same target.

Further, as still another embodiment of the terminal identification device according to the present invention, the terminal includes a plurality of terminals installed in or carried by a mobile body,
This terminal identification device
approaching target identifying means for identifying a plurality of targets that are estimated to approach each other by a predetermined amount or more later, based on the detected positions of the plurality of targets;
Information related to at least one of at least one terminal determined by the correspondence determination means to have a correspondence relationship with one of the plurality of identified targets, and information related to the fact that it is estimated that the terminal will approach the predetermined target or more. to the communication interface provided .

According to the present invention, the learning data including the environmental information and the information related to the target detected from the environmental information, and the terminal identification apparatus as described above can establish a corresponding relationship with the target. It is built by training a neural network algorithm using learning data with information related to the radio waves of the terminal determined to be present as correct data, and environmental information as information related to explanatory variables and detected from the environmental information A learning model is provided that causes a computer to function as a terminal radio wave information estimating unit that, when inputting information related to a target, outputs information related to the radio wave of the target terminal as information related to the objective variable .

According to the present invention, furthermore, data including the environmental information and information related to the target detected from the environmental information is input to the learning model according to the present invention, and the terminal output by the learning model a terminal radio wave information estimating means for estimating information related to the radio wave of the target as a terminal based on the information related to the radio wave of
A communication relay apparatus having communication control means for determining whether or not to switch the communication path of the object as a terminal based on information related to estimated radio waves is provided.

According to the present invention, there is also provided a program for operating a computer capable of determining a correspondence relationship between a terminal and a predetermined target,
Acquiring communication radio wave information, which is information related to communication radio waves, from a communication means that communicates with the terminal, and determines terminal movement state information, which is information related to changes in the position of the terminal, based on the communication radio wave information. terminal motion state determining means;
Acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect the target from the environmental information, and target movement that is information related to changes in the detected position of the target. target motion state determination means for determining state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. causing the computer to function as correspondence determination means for determining whether or not there is a correspondence relationship between the terminal and the target, and determining information related to the correspondence relationship between the terminal and the target ;
The target motion state determination means acquires target motion state information determined from environment information obtained from a sensor different from the sensor in the other terminal identification device from another terminal identification device existing in the surroundings,
The correspondence determination means also calculates the degree of correspondence between the acquired target motion state information and the determined terminal motion state information, and determines the correspondence between the terminal and the target based on the degree of correspondence. determine whether there is
There is provided a terminal identification program characterized by:

According to the present invention, there is further provided a terminal identification method by a computer capable of determining a correspondence relationship between a terminal and a predetermined target, comprising:
Acquires communication radio wave information, which is information related to communication radio waves, from the communication means that communicates with the terminal, and determines terminal motion state information, which is information related to changes in the position of the terminal, based on the communication radio wave information. On the other hand, acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect the predetermined target from the environmental information, and detect changes in the detection position of the target. a first step of determining information of target motion state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. a second step of determining whether or not there is a correspondence relationship between the terminal and the target, and determining information related to the correspondence relationship between the terminal and the target ;
In a first step, from another terminal identification device existing in the vicinity, target motion state information determined from environmental information obtained from a sensor different from the sensor in the other terminal identification device is obtained;
In the second step, the degree of correspondence between the acquired target motion state information and the determined terminal motion state information is also calculated, and based on the degree of correspondence, the terminal and the target are in a corresponding relationship. determine whether there is
A terminal identification method characterized by:

According to the terminal identification device, program and method of the present invention, the correspondence relationship between a terminal and an object existing in an environment in which the terminal may exist can be determined without depending on the mutual positional relationship or only on the positional relationship. It is possible to make a decision using information related to communication with the terminal.

1 is a schematic diagram showing an embodiment of a terminal identification device according to the present invention; FIG. FIG. 4 is a schematic diagram for explaining an embodiment of processing in a terminal motion state determination unit, a target motion state determination unit, and a correspondence relationship determination unit according to the present invention; FIG. 4 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention; FIG. 4 is a schematic diagram for explaining an embodiment of processing in a model construction unit and a terminal radio wave information estimation unit according to the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Terminal identification device]
FIG. 1 is a schematic diagram showing one embodiment of a terminal identification device according to the present invention.

A base station 1 as a terminal identification device according to the present invention shown in FIG. It is a device capable of communicating with the terminal 2 of . Further, in this embodiment, the base station 1 is equipped with a camera 103 which is an RGB-D camera capable of photographing the condition of the communication area with a predetermined angle of view.

The (RGB and depth) image data (video data) generated by this camera 103 includes:
(a) a person who possesses or carries the terminal 2,
(b) Mobile objects such as automobiles, motorcycles, railroad vehicles, robots, drones, etc. equipped with, including or boarded by the terminal 2, and (c) equipment, facilities, buildings, fixed objects where the terminal 2 is installed Such "objects" may be included (captured), and furthermore, people, mobile objects, equipment, facilities, buildings, and fixed (including plants such as trees) that are not related to the terminal 2 "Objects" such as things can also be included.

By the way, such a "target" can determine the correspondence relationship with the terminal 2, which is the target of identification. obtain. In particular, 5G, which is the communication method of the present embodiment, uses radio waves in the millimeter wave band for communication. There is a high possibility that its propagation will be blocked or attenuated by an "object" that exists between

Therefore, in 5G, the intervention of such a "target" causes a rapid drop in the power of received radio waves at the base station 1, resulting in a significant deterioration in communication quality, which is a serious problem. Here, if it is possible to predict a decrease in the received radio wave power at the base station 1 due to such a "target", it becomes possible to maintain the communication quality by performing control such as switching the communication path of the terminal 2. FIG. Therefore, the base station 1 of the present embodiment not only determines the correspondence between the terminal 2 and the "target", but also predicts the received radio wave power and controls the communication path, as will be described later. It is also a device that enables

Similarly in FIG. 1, the base station 1 first determines the correspondence relationship between the terminal 2 and the "target".
(A) "Communication radio wave information", which is information related to radio waves of communication, from communication means (communication interface 101, communication control unit 114, and communication history information accumulation unit 102 provided by itself in this embodiment) for communicating with terminal 2 a terminal motion state determination unit 111 that obtains and determines "terminal motion state information" that is information related to the position change of the terminal 2 based on the "communication radio wave information";
(B) Acquire "environmental information" (camera image data in this embodiment), which is information related to the environment, from a sensor (camera 103 in this embodiment) that senses the environment in which the terminal 2 may exist. a target motion state determination unit 112 that detects a predetermined "target" from the "information" and determines "target motion state information" that is information related to changes in the detection position of the detected "target";
(C) Calculate the degree of correspondence between the determined "terminal exercise state information" and the determined "target exercise state information", and based on this "correspondence degree", calculate the "terminal exercise It is determined whether or not the terminal 2 related to the "state information" and the "target" related to the "target motion state information" have a corresponding relationship, and the information related to the corresponding relationship between the terminal 2 and the "target"" and a correspondence determination unit 113 that determines "terminal target correspondence information".

Here, the "communication radio wave information" in (A) above includes (a) time-series information on the frequency of received radio waves, (b) time-series information on the intensity of received radio waves, and (c) emission of radio waves related to communication. "Terminal motion state information" includes at least one of (d) time-series information of direction and (d) time-series information of round-trip time in communication, and "terminal motion state information" determined therefrom includes (a') received radio waves (b') variation per hour in intensity of received radio waves, (c') variation per hour in radiation direction of radio waves associated with communication, and (d') round-trip time in communication It is also preferably information about the velocity or angular velocity of the terminal 2, determined from at least one of the variations of the hit.

On the other hand, the "object motion state information" in (B) above is also preferably information related to the velocity or angular velocity of the "object" determined from changes in the detected position of the "object" per hour.

Furthermore, in these preferred embodiments, the "correspondence degree" in (C) above is set as a degree of matching between the information regarding the speed or angular velocity of the terminal 2 and the information regarding the speed or angular velocity of the "target", Correspondence determination unit 113 determines whether terminal 2 and the "object" have a correspondence relation based on such "correspondence degree".

Thus, according to the base station 1 as a terminal identification device according to the present invention, the correspondence relationship between the terminal 2 and the "target" existing in the environment in which the terminal 2 can exist (detected from the "environmental information") can be determined by comparing the respective "motion states" derived from the "communication radio wave information" of the terminal 2 and the position change of the detection "target". That is, the "terminal object correspondence information" can be determined from the comparison of the respective "exercise states" without relying on the positional relationship between the terminal 2 and the "object".

Here, as will be described later, in the base station 1, an embodiment in which "terminal target correspondence information" is determined based not only on the mutual "movement state" of the terminal 2 and the "target" but also on the mutual positional relationship is adopted. It is also possible to take However, in any case, the base station 1 can determine the correspondence relationship between the terminal 2 and the "target" without relying only on the mutual positional relationship. As a result, for example, when a large number of "targets" exist (when they are detected) in the vicinity of the terminal 2, which is the identification target, an erroneous correspondence relationship occurs due to the slight difference in position between them. It is also possible to significantly reduce the risk of being determined by taking into account their "motion state".

Although 5G is adopted as the communication method between the base station 1 and the terminal 2 in this embodiment, other communication methods such as LTE may of course be used. Communication between devices and terminals may be based on various other wireless communication standards. For example, even in communication systems where the problem of shielding by objects is not as prominent as in 5G, there are more than a few situations in which it is desirable to accurately identify subordinate terminals using "targets" detected from "environmental information." For example, there are various needs for terminal identification, such as determining the relationship between a page viewed by a user determined to have a corresponding relationship with a certain terminal and the flow line of the user and utilizing it for marketing and management. On the other hand, according to the terminal identification device of the present invention, it is possible to perform such terminal identification processing with high accuracy.

Moreover, the terminal identification device according to the present invention is not limited to a communication relay device such as a base station. For example, as a dedicated device for terminal identification processing, it may be provided in the form of being connected to a communication relay device/equipment such as a base station. Further, as the terminal identification device according to the present invention, it is possible to use a cloud server, a non-cloud server device, a personal computer (PC), a notebook or tablet computer, or the like, which is equipped with the terminal identification program according to the present invention. be.

Furthermore, at least one of the above (A) to (C), which are components of the terminal identification device (base station 1) according to the present invention, is a device different from the other components. It is not impossible to take. For example, it is possible to implement the above functions (A) to (C) by a plurality of servers as a whole. Even in such a case, the entirety of these can be regarded as a terminal identification device or system that implements the terminal identification method according to the present invention.

[Functional Configuration of Terminal Identification Device, Terminal Identification Program]
Similarly, in the functional block diagram of FIG. 1, the base station 1 includes a communication interface 101, a communication history information accumulation unit 102, a camera 103, and a camera image accumulation unit as an embodiment of a terminal identification device and a communication relay device according to the present invention. It has a unit 104 and a processor memory. Here, the processor memory stores an embodiment of the communication relay program including the terminal identification program according to the present invention, and has a computer function, and by executing this communication relay program, Terminal identification processing and communication relay processing are performed.

In addition, the processor memory includes, as functional components, a terminal motion state determination unit 111 including a terminal position determination unit 111a, a target motion state determination unit 112 including a target position determination unit 112a, a correspondence relationship determination unit 113, a communication It has a control unit 114 , an approaching target identification unit 121 , a model construction unit 131 , and a terminal radio wave information estimation unit 132 . These functional components can be regarded as the functions of the terminal identification program and the communication relay program stored in the processor memory. , can be understood as an embodiment of the terminal identification method and communication relay method according to the present invention.

Similarly, in the functional block diagram of FIG. is acquired and recorded, and communication history information is generated by organizing the information in chronological order, and the communication history information storage unit 102 stores and manages the information.

Here, in this embodiment, this communication history information is communication radio wave information that is information related to radio waves used in communication with the terminal 2, specifically,
(a) time-series information of the frequency of radio waves received from the terminal 2;
(b) time-series information of the strength of the radio wave received from the terminal 2 (received signal power, RSSI (Received Signal Strength Indicator));
The information includes at least one of (c) time-series information on the radiation direction of radio waves related to communication with the terminal 2, and (d) time-series information on the round trip time (RTT) in communication with the terminal 2. ing.

The terminal motion state determination unit 111 acquires the above-described communication radio wave information from the communication history information storage unit 102, and determines terminal motion state information, which is information related to changes in the position of the terminal 2, based on this communication radio wave information. Here, in this embodiment, this terminal motion state information is
(a′) frequency deviation of radio waves received from terminal 2;
(b') variation per time in the strength of the radio wave received from the terminal 2 (received signal power, RSSI);
(c') the terminal determined from at least one of the variation per hour of the radiation direction of radio waves in communication with the terminal 2, and (d') the variation per hour of the RTT in communication with the terminal 2. 2 velocity or angular velocity.

First, for the above (a′), the terminal motion state determination unit 111 determines the deviation Δf_t from the frequency f0_t originally set for communication from the time-series information of the frequency f_t of the radio wave in (a) above. By generating time-series information and applying an object velocity measurement method using the Doppler effect, which is well known in the field of radio wave engineering, to this deviation Δf_t, the time-series information of the velocity vector v_e of the terminal 2 is obtained. may be generated and used as the terminal motion state information.

In addition, for the above (b′), the terminal motion state determining unit 111, based on the time-series information of the received signal power (RSSI) E_t of the above (b) and the previously set transmission signal strength from the terminal 2, For example, using Friis' transmission formula, which is well known in the field of radio engineering, time-series information of the distance D_t to the terminal 2 is generated, and time-series information of the velocity vector v_e of the terminal 2 is obtained from the time change of this distance D_t. It may be generated and used as the terminal motion state information.

Furthermore, for the above (c′), the terminal motion state determination unit 111 determines (base station 1) may be generated and used as terminal motion state information.

Furthermore, for the above (d'), the terminal motion state determining unit 111 determines the radio wave propagation speed V and the coefficient 0.5 that makes the RTT one-way for the RTTR_t from the time-series information of the RTTR_t of the above (d). Time-series information of the communication distance (R_t*V/2) is generated by multiplication, and time-series information of the velocity vector v_e of the terminal 2 is generated from the change in the communication distance over time, and this is used as the terminal motion state information may be

Note that the velocity vector v_e described above may be a projection vector of the velocity vector of the terminal 2 in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2) depending on the situation or more accurately.

The terminal motion state information determination processing of the terminal 2 has been described above. Based on this, it is also preferable to determine the terminal position information, which is the information related to the position of the terminal 2 . Specifically, here, not only the received signal power at the base station 1,
(a) received signal power measurement values at a plurality of base stations existing around the base station 1;
(b) Determining the terminal location information of the terminal 2 using a well-known base station positioning technique that acquires and uses, for example, through communication, received signal power measurement values at the terminal 2 that has received radio waves from a plurality of base stations. can be done.

Similarly, in the functional block diagram of FIG. A celestial (omnidirectional) camera can also be used. Of course, instead of the camera 103, it is also possible to adopt a sensor capable of sensing the environment in which the terminal 2 may exist and generating environmental information, such as LiDAR, laser/infrared positioning device, TOF camera, and thermography device. be.

Here, in this embodiment, the camera 103 is an RGB-D camera capable of generating an RGB image and a depth image as environment information, and the generated environment information (RGB image data and depth image data) is the camera image data. It is saved and managed by the storage unit 104 .

Note that the camera 103 is installed inside the base station 1 in the present embodiment, but is installed at a position distant from the base station 1, for example, a surveillance camera in town, and communicates with the base station 1. It may be connected. Moreover, as will be described later, it is possible to adopt an embodiment in which a plurality of cameras 103 are provided at mutually different positions regardless of whether they are inside or outside the base station 1 .

The target motion state determination unit 112 acquires (RGB and depth) image data (environmental information) from the camera image storage unit 104, detects a “predetermined target” from the image data, and determines the velocity or acceleration of the target. (target motion state information), in this embodiment, time-series information of velocity vector v_m or angular velocity vector w_m is determined. By the way, this velocity vector v_m is combined with the velocity vector v_e determined by the terminal motion state determination unit 111 to project the velocity vector of a predetermined target in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2). It may be determined as a vector.

Here, the detected "predetermined target" is, for example,
(a) a person (user) possessing a mobile terminal,
(b) A vehicle equipped with a terminal 2 having a drive recorder function, a CAN information transfer function, an interface function for automatic driving control by a server, etc., and
(c) "Person" who may include the terminal 2 as the communication destination related to the communication history information, such as an autonomous mobile robot or an autonomous flying drone equipped with a terminal 2 having an interface function for autonomous movement control by a server It can be "or" mobile. Further, depending on the case, "equipment", "facility", "building", "fixed object", etc., which may include the terminal 2, may also be "predetermined objects".

In addition, time-series information (information related to velocity or angular velocity) of velocity vector v_m or angular velocity vector w_m in such a predetermined target is, in this embodiment,
(a) The target position determination unit 112a of the target motion state determination unit 112 detects a predetermined target from the acquired time-series image data, and time-series information of the detection position of the detected target within the image space. and then transforming the detected position from the image space coordinate system to the real space coordinate system,
(b) It can be determined by determining the change per time of the detected position in the derived real space coordinate system. Incidentally, the angular velocity vector w_m is calculated with the origin of the vector, that is, the center of rotation as the position of the base station 1 (camera 103).

Here, regarding the detection of the predetermined target from the (RGB and depth) image data in (a) above, for example, a model for detecting the predetermined target is constructed based on a well-known machine learning algorithm for various image recognition, and the using the model to identify a detection area (e.g., bounding box) in the image data where it is assumed that a predetermined target exists, and also calculating a score indicating the likelihood that the detection area is the predetermined target, A detection region having a score high enough to satisfy a predetermined condition can be determined as a region of interest. Also, the detection position of the predetermined target can be set as a representative point (for example, the center of gravity, the middle point of the lower end, etc.) of the determined target region.

Incidentally, when a stereo camera is adopted as the camera 103, for example, non-patent literature: Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang By applying the object detector described in Fu, Alexander C. Berg, “SSD: single shot multibox detector”, European Conference on Computer Vision, Computer Vision-ECCV 2016, pp. 21-37, 2016, the region of interest and the detection position can be determined.

Furthermore, when LiDAR is used as the sensor according to the present invention instead of the camera 103, the point cloud, which is the generated environment information, can be obtained by, for example, Charles R. Qi, Hao Su, Kaichun Mo, Leonidas Using the object detector described in J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation”, Journal of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, 2016 By doing so, the target area and the detection position can be determined.

Incidentally, the target exercise state determination unit 112 determines whether a predetermined target is not detected from the environmental information, that is, when one predetermined target is not measured (sensed), or when it is measured but is not detected due to the influence of noise or the like. terminates the target exercise state determination processing for the environment information.

Further, the target position determination unit 112a of the target motion state determination unit 112 can determine the position of the detected predetermined target (detected position) as described above. For each target, after associating the target with the detected position at each point in time, time-series information (movement history information) of the position of the target is generated. This also makes it possible to track each predetermined target detected individually.

Here, for such object tracking processing, for example, a Kalman filter, which is a well-known state estimation method, is applied to the determined object area (bounding box), and the current object detection area ( bounding box), and the overlapping area of this predicted detection area (bounding box) and the currently detected detection area (bounding box) is used as an evaluation value, and the target area is determined based on the evaluation value. is also preferred. Incidentally, such object tracking processing is described in, for example, non-patent literature: Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft, “Simple Online and Realtime Tracking”, Journal of 2016 IEEE International Conference on Image Processing (ICIP), 3464-3468, 2016.

Further, as another embodiment, when a plurality of sensors other than the camera 103 or the above-described camera are provided at mutually different positions inside or outside the base station 1, the target motion state determination unit 112
(a) acquiring a plurality of environmental information from each of the plurality of cameras 103 (or sensors);
(b) determining target motion state information (e.g., information relating to a given target's velocity or angular velocity) from each of the plurality of environmental information;
(c) Among the determined target motion state information,
and/or
(c2) Objects whose image feature values of the target detection regions (bounding boxes) related to the target motion state information are similar to each other by a predetermined value or more (for example, objects whose mutual distance in the image feature value space is less than a predetermined threshold value). about,
It is also preferable that these pieces of target motion state information relate to the same target. Further, in this case, a representative value (for example, an average value) of the detection position group for the target related to the target motion state information may be treated as the position of the same target.

As a result, it is possible to avoid a situation in which a certain terminal 2 is not included in the acquired environment information, resulting in a failure of identification processing for the terminal 2 . For example, environmental information obtained by one camera 103 (or sensor) is not included in the environmental information (i.e., when a certain target is located behind another target and is shielded by the other target). Not a few) situations will also occur. For example, from the viewpoint of one camera 103, it is quite possible that a person carrying the terminal 2 is blocked by a stopped bus and cannot be seen.

On the other hand, if environmental information obtained by a plurality of cameras 103 (or sensors) installed at different positions is used, objects that are not included in certain environmental information can be included in other environmental information from different sensing (measurement) viewpoints. As a result, it is sufficiently possible to avoid the above-mentioned situation of concern.

By the way, as a typical example of using the target motion state information related to the plurality of cameras 103 (or sensors) as described above, the base station 1 uses the information of one or more other base stations 1 existing around itself. For example, the target motion state information generated by the camera 103 (or sensor) installed in the base station 1 is received and acquired from each of them, and the same target determination process described above is performed. In this case, it is possible for a plurality of base stations 1 to work together to more preferably carry out the terminal identification processing according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the correspondence determination unit 113
(a) terminal motion state information determined by the terminal motion state determining unit 111, in this embodiment, information related to the velocity or angular velocity of the terminal 2;
(b) calculating the degree of correspondence between the target motion state information determined by the target motion state determination unit 112, which in the present embodiment is information related to the detected velocity or angular velocity of the predetermined target, and calculating the degree of correspondence; Based on this "degree of correspondence", it is determined whether or not the terminal 2 related to the terminal motion state information (a) and the predetermined target related to the target motion state information (b) have a corresponding relationship. , determines terminal object correspondence information, which is information relating to the correspondence relationship between the terminal 2 and a predetermined object.

Here, in the present embodiment, the above "correspondence degree" is the above (a) information related to the speed or angular velocity of the terminal 2 and the above (b) information related to the detected predetermined target speed or angular velocity. is a value indicating the degree to which are matched, for example,
(a) About the reciprocal of the difference (1/|v_e−v_m|) between the velocity vector v_e of the terminal 2 at each point in the predetermined time interval and the velocity vector v_m of the predetermined target detected at the same point in time A representative value (for example, an average value) in the predetermined time interval may be used as the degree of correspondence C in the predetermined time interval,
(b) Regarding the reciprocal of the difference (1/|w_e−w_m|) between the angular velocity vector w_e of the terminal 2 at each point in the predetermined time interval and the angular velocity vector w_m of the predetermined object detected at the same point in time A representative value (for example, an average value) in the predetermined time interval may be used as the degree of correspondence C in the predetermined time interval.

Next, when the calculated degree of correspondence C exceeds a predetermined threshold value C_th (when C>C_th), the correspondence determining unit 113 determines that the terminal 2 and the detected predetermined target (in the predetermined time interval) are It is determined that there is a correspondence relationship, and for example, terminal object correspondence information in which the terminal ID of the terminal 2 and the object ID of the predetermined object are associated and recorded is determined. Note that this threshold C_th may be empirically set to a suitable value by an administrator, for example.

Further, as a modification, the correspondence determination unit 113 calculates the degree of correspondence C for all pairs generated between all the terminals 2 and all the detected predetermined targets, It is also preferable to determine the correspondence between the terminal 2 and the predetermined target by a greedy algorithm that selects a set that gives (correspondence) as a solution, and determines the terminal target correspondence information.

FIG. 2 is a schematic diagram for explaining an embodiment of processing in the terminal motion state determination unit 111, the target motion state determination unit 112, and the correspondence determination unit 113. As shown in FIG.

In the embodiment shown in FIG. 2, a terminal 2 mounted in a car and moving emits radio waves of frequency f0_t, and a base station 1 communicating with this terminal 2 receives the frequency ( f0_t + Δf_t) radio waves are received. The terminal motion state determining unit 111 of the base station 1 acquires communication radio wave information (communication history) in which the terminal ID of the terminal 2 is associated with the received radio wave frequency (f0_t+Δf_t), and from the time-series information of the frequency deviation Δf_t , time-series information (terminal motion state information) of the velocity vector v_e of the terminal 2 is generated using a well-known Doppler effect object velocity measurement method.

On the other hand, the vehicle on which the terminal 2 is mounted is photographed by the camera 103, including its moving state, and the target motion state determining unit 112 of the base station 1 stores the image data of the vehicle as environment information including this vehicle. is obtained, and detection processing of "automobile", which has been set in advance as a predetermined target, is performed on the image data, and time-series information of the detected position of the automobile in which the terminal 2 is mounted is generated. Next, time-series information of the displacement (position change) vector Δl_m is generated from the time-series information of the detected position, and time-series information of the velocity vector v_m corresponding to the displacement vector per unit time (target motion status information).

Thereafter, the correspondence determining unit 113 of the base station 1 determines a predetermined The reciprocal of the magnitude of the vector difference (1/|v_e−v_m|) at each point in the time interval is calculated, and the degree of correspondence C(v_e, v_m) is determined from these values. Next, when the determined degree of correspondence C(v_e, v_m) exceeds a predetermined threshold value C_th (when C>C_th), the correspondence determining unit 113 determines that the terminal 2 and the detected automobile (predetermined target) correspond to each other. It determines that there is a relationship, and finally generates a terminal object correspondence table (terminal object correspondence information) in which the terminal ID of the terminal 2 and the detected object ID of the automobile are recorded in association with each other.

As described above, according to the base station 1 of the present embodiment, the correspondence relationship between the terminal 2 and a predetermined object (automobile in FIG. can be determined by comparing the velocities (states of motion) of . That is, without relying on the mutual positional relationship between the terminal 2 and the predetermined object, the terminal object correspondence information can be determined from the comparison of the respective velocities (motion states). Even if it is a terminal, the identification process can be carried out more reliably.

An embodiment of terminal identification processing has been described above using FIG. 2. In this embodiment, speed information is generated from the frequency of the received radio wave. On the other hand, as another preferred embodiment, the terminal motion state information is
(a′) frequency deviation of radio waves received from terminal 2;
(b') variation per time in the strength of the radio wave received from the terminal 2 (received signal power, RSSI);
At least two (for example, all four) of (c′) variation per hour in the radiation direction of radio waves associated with communication with terminal 2 and (d′) variation per hour of RTT in communication with terminal 2 It is also preferable to use information related to the velocities or angular velocities of a plurality of terminals 2 determined from each of the.

In this case, the degree of correspondence C is the degree to which each piece of information relating to the velocity or angular velocity of these terminals 2 matches the information relating to the detected velocity or angular velocity of the predetermined target (for example, the magnitude of the difference between the two vectors). ), and multiple calculations will be made. Here, the correspondence determination unit 113, for example,
(a) when the ratio of those having a predetermined threshold value C_th or more among these plurality of degrees of correspondence C is a predetermined value (for example, 0.5) or more, or
(b) when the sum of these multiple degrees of correspondence C or the sum of the weightings is equal to or greater than a predetermined value,
It is also preferable to determine that the terminal 2 and the detected predetermined target are in correspondence.

In any case, in the embodiment in which the identification processing of the terminal 2 is performed by integrating a plurality of degrees of correspondence C in this way, the information related to velocity or angular velocity is multilaterally estimated by different methods, so that various environments can be used. Therefore, the accuracy of identification processing of the terminal 2 is maintained or improved even under the condition of (1).

Furthermore, as another preferred embodiment, the correspondence determining unit 113 (FIG. 1)
(a) terminal position information determined by the terminal position determination unit 111a (FIG. 1) of the terminal motion state determination unit 111;
(b) Calculate the degree of position correspondence (degree of position matching) LC, which is the degree of matching with the target position information determined by the target position determination unit 112a (FIG. 1) of the target motion state determination unit 112, and calculate the motion state described above. It is also preferable to determine whether or not the terminal 2 and the detected predetermined target have a corresponding relationship based on not only the degree of correspondence C of , but also the degree of location correspondence LC.

Here, the degree of position correspondence LC is, for example, the representative value of the reciprocal of the absolute value (distance) of the difference between the position of the terminal 2 and the position of the predetermined target at each point in the predetermined time interval. (eg average value). Further, the correspondence determination unit 113 determines that, for example, the degree of correspondence C of the exercise state exceeds a predetermined threshold C_th (C>C_th) and the degree of position correspondence LC exceeds a predetermined threshold LC_th (LC>LC_th). ), it is also preferable to determine that the terminal 2 and the detected predetermined target are in correspondence.

In this way, by performing identification processing of the terminal 2 in consideration of the position correspondence degree LC, it is also possible to determine the correspondence relationship (between the terminal 2 and the predetermined target) with higher accuracy.

Furthermore, another preferred embodiment regarding the degree of correspondence C of exercise states will be described. In this embodiment, there are at least one base station 1 around the base station 1, and each base station 1 determines the target motion state information from the environmental information obtained by its own camera 103, and further determines by itself. It is possible to mutually exchange the target motion state information.

Here, the target motion state determination unit 111 (FIG. 1) acquires the target motion state information determined by the surrounding base stations 1, and the correspondence determination unit 113 (FIG. 1) acquires the information thus acquired. Also calculates the degree of correspondence C' between the target motion state information determined and the determined terminal motion state information, and based on not only the degree of correspondence C of the target motion state information determined by itself but also the degree of correspondence C', the terminal 2 and a predetermined target are also preferably determined. In this case, it is possible to perform terminal identification processing similar to the terminal identification processing performed in the embodiment in which the base station 1 is connected to a plurality of cameras 103, as already described. This also makes it possible to avoid a situation in which a certain terminal 2 is not included in the environment information obtained by its own camera 103, resulting in the base station 1 failing in identification processing of the terminal 2. It becomes.

[Application example of terminal identification processing]
An application example of the terminal identification processing described above will be described below.

In the functional block diagram of FIG. 1 , the approaching target identifying unit 121 detects (other than the predetermined target) detected by the target motion state determining unit 112 in a situation where there are a plurality of mobile terminals 2 in the communication area environment. Based on the detected positions of the plurality of targets (including targets), a plurality of targets that are estimated to approach each other by a predetermined amount or more later are specified.

For example, the approaching target identification unit 121 may determine each detected target at a future time after a predetermined time based on the current detected position of each detected target and the velocity vector calculated from the past time change of the detected position. It is also preferable to estimate the positions of the objects and specify the objects whose mutual distance is equal to or less than a predetermined threshold value at the future time point as the approaching object group.

In this case, the communication control unit 114 receives the information of the plurality of targets identified by the approaching target identification unit 121, and the correspondence determination unit 113 determines that there is a correspondence relationship with one of these plurality of targets. Identify (at least one) terminal 2, and send information, such as an approach/collision alarm, encounter/encounter forecast, etc., to at least one of the identified terminals 2 that is estimated to approach more than a predetermined amount. It is transmitted (from the communication interface 101).

FIG. 3 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention.

According to the application example of FIG. 3, the base station 1 remotely monitors an automobile (or a robot, etc.) that is communicatively connected by 5G. Specifically, the base station 1 communicates with a terminal 2_001 whose terminal ID is 001 and a terminal 2_002 whose terminal ID is 002.
(a) The terminal 2_001 and the detected automobile 3_002 whose target ID is 002 are in correspondence (that is, the terminal 2_001 is mounted on the automobile 3_002),
(b) The terminal 2_002 and the detected car 3_003 whose target ID is 003 are in correspondence (that is, the terminal 2_002 is mounted on the car 3_003),
A terminal object correspondence table that records this fact is generated.

On the other hand, the approaching object identification unit 121 identifies the vehicle 3_003 and the driver 4_001 whose object ID is 001 as elements of one approaching object group, and the vehicle 3_003 and the vehicle 4_001 are one approaching object group. To the communication control unit 114, the proximity target group information, which is information to the effect that it is included in the . In this application example, the driver 4_001 cannot be recognized by the sensors mounted on the car 3_003 because the building standing along the road is an obstacle.

In such a situation, the communication control unit 114
(a) Based on this approaching target group information, the terminal 2_002 having a correspondence relationship with the car 3_003 included in the approaching target group is determined as the approach alarm notification destination from the obtained terminal target correspondence table,
(b) Alarm information to the effect that "a rider 4_001 (also included in the approaching target group) is approaching" is transmitted to the terminal 2_002 determined as the notification destination.

In addition, the approaching object identification unit 121 also transmits the detected position information of the rider 4_001 to the terminal 2_002 in accordance with this alarm information, and the current position (and its flow line) of the rider 4_001 is displayed on the screen of the terminal 2. It is also preferred to have it highlighted on the map.

By performing the processing described above, for example, it is possible to promote safe driving by drivers and safe walking/running of people, and to prevent collision accidents caused by moving bodies such as automobiles. In particular, it is also possible to notify the car, which is the party concerned, through the identified terminal 2 of a dangerous situation for a certain car that cannot be detected or predicted by the onboard sensor (camera). It becomes possible.

[Communication relay processing, communication relay program]
Various embodiments related to the identification processing of the terminal 2 have been described above. Hereinafter, using the functional block diagram of FIG. 1 and FIG. One embodiment of a process is described. Note that FIG. 4 is a schematic diagram for explaining an embodiment of processing in the model construction unit 131 and the terminal radio wave information estimation unit 132, which will be described later.

In the functional block diagram of FIG. 1, the model construction unit 131 constructs a learning model for estimating terminal radio wave information. Specifically, in this embodiment, the model construction unit 131, as shown in FIG.
(a) time-series image data (environmental information) generated by the camera 103 (which may include an object related to the terminal 2 or an object that may become a radio wave obstruction);
(b) learning data including information on a predetermined target detected from the time-series image data, for example, information on the detected position of the target (target position information),
(c) received radio wave information (at each time point or representative time point of the environment information in (a) above) related to the terminal 2 determined to have a corresponding relation with this predetermined target in the correspondence determining unit 113, for example, received radio wave Generate learning data with electric power as correct data, and with the learning data, a machine learning algorithm known in the field of image recognition (for example, a neural network algorithm with multiple convolutional layers) and their outputs (Combination with Fully-Connected Layers) to build a terminal radio wave information estimation model.

Here, as already described in detail, in the conventional machine learning model disclosed in Non-Patent Document 1, for example, an object detected from time-series environmental information generated by a camera is detected by this machine learning model. It is not possible to determine whether or not it corresponds to the terminal related to the received radio wave power predicted by.

On the other hand, the terminal radio wave information estimation model constructed by the model construction unit 131 is information related to a target that is exactly associated (determined to have a correspondence relationship) with the received radio wave information as correct data, such as a target position It is constructed by including information in the learning data. Therefore, the received radio wave information predicted by this terminal radio wave information estimation model, for example, the received radio wave power, can be interpreted as a predicted value for a predetermined target determined as the identification destination of the terminal 2 related to the received radio wave information. . In other words, the reception radio wave information estimation process based on the present terminal radio wave information estimation model reliably solves the unresolved problem of reliable identification of the correspondence target in the estimation process based on the conventional machine learning model.

Next, the terminal radio wave information estimation unit 132 uses the terminal radio wave information estimation model constructed by the model construction unit 131 to estimate received radio wave information of the terminal 2, for example, received radio wave power. Specifically, in this embodiment, the terminal radio wave information estimation unit 132, as shown in FIG.
(a) time-series image data (environmental information) generated by the camera 103, which includes the terminal 2 to be estimated and may include an object that may be a radio wave obstruction;
(b) applying target location information of a predetermined target detected from the time-series image data and determined to have a corresponding relationship with the terminal 2, which is an estimation target, to the terminal radio wave information estimation model; As an input, based on the received radio wave power (as estimated information) output from the terminal radio wave information estimation model due to this input, the predicted value of the received radio wave power (at the estimation time point or time interval) of the terminal 2 to be estimated is decide.

Finally, the communication control unit 114 (FIG. 1), which has obtained the predicted value of the received radio wave power of the terminal 2 (having high estimation accuracy as described above), based on this predicted value, the terminal 2 (or the terminal 2 and (predetermined target corresponding to) to determine whether or not to switch the communication path. For example, if the predicted value at a given future point in time or time interval is below a given power threshold, communication with this terminal 2 may be handed over to another neighboring base station 1 .

Here, in other adjacent base stations 1, the predicted value of received radio wave power in the same terminal 2 is also determined, and the predicted value is shared among the base stations 1 to specify the largest predicted value. , it is also preferable to switch the communication path to the base station 1 that has determined the specified maximum predicted value.

As described above, according to the base station 1 of the present embodiment, by using the terminal radio wave information estimation model constructed using the terminal identification processing result, communication failure due to communication path obstructions, which is a major problem in 5G, can be prevented. Therefore, it is possible to perform communication relay processing that can reliably solve the problem of (1), especially the problem of temporary communication failure due to a moving shield.

As described in detail above, according to the present invention, the corresponding relationship between a terminal and an object existing in an environment in which the terminal can exist is determined based on each "movement" derived from the communication radio wave information and the change in the detected position of the object. By comparing the "states" with each other, it becomes possible to determine. That is, the "terminal object correspondence information" can be determined from the comparison of the respective "motion states" without relying on the mutual positional relationship between the terminal and the object or only on the positional relationship.

In addition, such terminal identification processing according to the present invention solves the problem of communication failure due to communication path obstructions in the upcoming 5G, and notifies/warns an automobile equipped with a terminal of the approach of a person or other vehicle. Furthermore, it is applied in various situations and fields, such as determining the relationship between the user's browsing page determined to be in correspondence with a certain terminal and the user's flow line, and utilizing it for marketing and management. It is possible.

In the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical ideas and aspects of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 base station (terminal identification device, communication relay device)
101 communication interface 102 communication history information storage unit 103 camera 104 camera image storage unit 111 terminal motion state determination unit 111a terminal position determination unit 112 target motion state determination unit 112a target position determination unit 113 correspondence determination unit 114 communication control unit 121 approach target Identification unit 131 Model construction unit 132 Terminal radio wave information estimation unit 2, 2_001, 2_002 Terminal 3_002, 3_003 Automobile 4_001 Driver

Claims (10)

A terminal that acquires communication radio wave information, which is information related to communication radio waves, from a communication means that communicates with a terminal, and determines terminal movement state information, which is information related to changes in the position of the terminal, based on the communication radio wave information. motion state determining means;
Acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect a predetermined target from the environmental information, and target that is information related to changes in the detection position of the target a target motion state determining means for determining motion state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. a correspondence relationship determining means for determining whether or not there is a correspondence relationship between the terminal and the target, and determining information related to the correspondence relationship between the terminal and the target ;
The target motion state determining means acquires target motion state information determined from environment information obtained from a sensor different from the sensor in the other terminal identification device from another terminal identification device existing in the vicinity. ,
The correspondence determination means also calculates the degree of correspondence between the acquired target motion state information and the determined terminal motion state information, and determines the correspondence between the terminal and the target based on the degree of correspondence. determine whether it is in
A terminal identification device characterized by: The communication radio wave information includes time-series information on the frequency of received radio waves, time-series information on the intensity of received radio waves, time-series information on the radiation direction of radio waves related to communication, and time-series information on round trip time in communication. including at least one of
The terminal motion state information is determined from at least one of the shift of the frequency, the variation over time of the intensity, the variation over time of the radial orientation, and the variation per hour of the round-trip time. Information related to the speed or angular velocity of the terminal,
The target motion state information is information related to the velocity or angular velocity of the target determined from changes in the detected position of the target per time,
The degree of correspondence is the degree to which the information regarding the speed or angular velocity of the terminal matches the information regarding the speed or angular velocity of the target,
2. The terminal identification apparatus according to claim 1, wherein said correspondence determination means determines whether or not said terminal and said object have a correspondence relationship based on the degree of matching. The terminal motion state information is determined from each of at least two of the frequency shift, the intensity variation over time, the radial orientation variation per hour, and the round trip time variation per hour. a plurality of pieces of information related to the velocity or angular velocity of the terminal,
The degree of matching is the degree of matching between each of the plurality of pieces of information relating to the speed or angular velocity of the terminal and the information relating to the velocity or angular velocity of the target, and is calculated multiple times,
The correspondence determining means determines that the terminal and the object have a correspondence relationship based on the ratio of the plurality of degrees of matching that are equal to or greater than a predetermined value, or the sum of the degrees of matching or the sum of the weightings. 3. The terminal identification device according to claim 2, wherein it is determined whether or not. The terminal motion state means also determines terminal location information, which is information related to the location of the terminal, based on the communication radio wave information,
The target motion state determination means also determines target position information, which is information related to the detection position of the target detected from the environmental information,
The correspondence determining means calculates a degree of position matching, which is a degree of matching between the determined terminal position information and the determined target position information, and based on the degree of position matching, determines the relationship between the terminal and the target. 4. The terminal identification device according to any one of claims 1 to 3, wherein it is determined whether or not there is a correspondence relationship between . The sensor is a plurality of sensors provided at different positions,
The target motion state determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target motion state information from each of the plurality of environmental information, and determines the target motion state information among the determined target motion state information. , the detection positions of the target related to the target motion state information are closer than a predetermined value , and the feature amounts of the detection regions of the target related to the target motion state information are similar to each other by a predetermined value or more. 5. The terminal identification device according to any one of claims 1 to 4, wherein the target motion state information for one or both of the above is related to the same target. The terminal includes a plurality of terminals installed or carried on a mobile body,
The terminal identification device is
approaching target identifying means for identifying a plurality of targets that are estimated to approach each other by a predetermined amount or more later, based on the detected positions of the plurality of targets;
At least one of at least one terminal determined to have a corresponding relationship with one of the plurality of specified targets by the correspondence relationship determination means is estimated to approach at least the predetermined target 6. The terminal identification device according to any one of claims 1 to 5 , further comprising communication control means for transmitting information to the provided communication interface . Learning data including the environmental information and information related to a target detected from the environmental information, and having a corresponding relationship with the target by the terminal identification device according to any one of claims 1 to 6 . It is built by training a neural network algorithm using learning data with information related to the radio waves of the terminal determined as correct data, and environmental information as information related to explanatory variables and A learning model characterized by causing a computer to function as a terminal radio wave information estimating unit that, when information about a target is input, outputs information about the radio wave of the target terminal as information about the target variable . Data including the environmental information and information related to the target detected from the environmental information is inputted to the learning model according to claim 7 , and based on the information related to the radio wave of the terminal output by the learning model terminal radio wave information estimating means for estimating information related to radio waves of the target as a terminal;
and communication control means for determining whether or not to switch the communication path of the target as a terminal, based on the information related to the estimated radio wave. A program that causes a computer capable of determining a correspondence relationship between a terminal and a predetermined object to function,
Acquiring communication radio wave information, which is information related to communication radio waves, from a communication means that communicates with the terminal, and determines terminal movement state information, which is information related to changes in the position of the terminal, based on the communication radio wave information. terminal motion state determining means;
Acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect the target from the environmental information, and target movement that is information related to changes in the detected position of the target. target motion state determination means for determining state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. causing a computer to function as correspondence determination means for determining whether or not there is a correspondence relationship with an object, and determining information related to the correspondence relationship between the terminal and the object ;
The target motion state determining means acquires target motion state information determined from environment information obtained from a sensor different from the sensor in the other terminal identification device from another terminal identification device existing in the vicinity. ,
The correspondence determination means also calculates the degree of correspondence between the acquired target motion state information and the determined terminal motion state information, and determines the correspondence between the terminal and the target based on the degree of correspondence. determine whether it is in
A terminal identification program characterized by : A terminal identification method by a computer capable of determining a correspondence relationship between a terminal and a predetermined target,
Acquires communication radio wave information, which is information related to communication radio waves, from the communication means that communicates with the terminal, and determines terminal motion state information, which is information related to changes in the position of the terminal, based on the communication radio wave information. On the other hand, acquire environmental information that is information related to the environment from a sensor that senses the environment in which the terminal may exist, detect the predetermined target from the environmental information, and detect changes in the detection position of the target. a first step of determining information of target motion state information;
A degree of correspondence between the determined terminal motion state information and the determined target motion state information is calculated, and based on the degree of correspondence, a terminal related to the terminal motion state information and a terminal related to the target motion state information are determined. a second step of determining whether or not there is a correspondence relationship between the terminal and the target, and determining information related to the correspondence relationship between the terminal and the target ;
In a first step, from another terminal identification device existing in the vicinity, target motion state information determined from environmental information obtained from a sensor different from the sensor in the other terminal identification device is obtained;
In the second step, the degree of correspondence between the acquired target motion state information and the determined terminal motion state information is also calculated, and based on the degree of correspondence, the terminal and the target are in a corresponding relationship. determine whether there is
A terminal identification method characterized by:

Images