JP2021078022A - Apparatus for identifying terminal based on correspondence level with target on communication state, program, and method - Google Patents

Abstract

To provide a terminal identification apparatus determining a correspondence relation between a terminal and a target existed under an environment in which the terminal may exist, by using information in accordance with a communication with the terminal without depending on only a mutual position relation or the position relation.SOLUTION: A terminal identification apparatus includes: a terminal communication state determination part that determines terminal communication state information as information of a communication state between terminals on the basis of communication information acquired by communication means; a target communication state determination part that acquires environment information from a sensor for sensing an environment and determines target communication state information as information of the communication state, which is implemented if a predetermined target contains the terminal, from the environmental information; and a correspondence relation determination part that calculates a level of a correspondence between the determined terminal communication state information and the target communication state information, determines whether or not the terminal according to the terminal communication state information has a correspondence relation with the prescribed target according to the target communication state information, and determines the correspondence relation information between the terminal and the predetermined target.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for acquiring information about a terminal as a communication target, particularly a technique for identifying the terminal.

The upcoming 5th generation mobile communication system (5G) will make it possible to realize high-performance communication such as high speed, large capacity, low delay, and multi-terminal connection by using radio waves in the millimeter wave band. Here, millimeter waves have high straightness and are less likely to be diffracted, and there is a high possibility that their propagation will be blocked or attenuated by an object existing between the terminal and the base station. Therefore, in 5G, there is a big problem that the received power is sharply lowered due to the intervention of such an object, and the communication quality is significantly deteriorated.

Here, if it is possible to predict a decrease in the received power due to such an object, it is possible to maintain the communication quality by controlling the communication path of the terminal. As a technique for predicting the received power, for example, Non-Patent Document 1 uses time-series three-dimensional environment information acquired by an RGB-D camera capable of generating a depth image in addition to an RGB image, and is used in a communication device by machine learning. A method for predicting received power is disclosed.

Specifically, in this method, for the time-series environmental information that is the result of imaging the communication environment, the time-series information of the received power measured (at the same time as imaging) at the terminal that received the radio wave from the transmitting station is used as the correct answer data. A machine learning model is constructed using the associated teacher data, and the predicted value of the received power at the receiving terminal is determined from the time-series environmental information acquired by the RGB-D camera using the constructed machine learning model. ing.

Further, for example, Patent Document 1 is a technique for improving the estimation accuracy of communication quality in the initial operation stage of a system or in a communication environment where the frequency of occurrence is low in the communication quality prediction processing by machine learning as disclosed in Non-Patent Gazette 1. Is disclosed. Specifically, in the virtual space formed from the measurement results of the communication environment, the information by the RGB-D camera is virtually generated, and the communication quality of the receiving terminal in the virtual space is estimated by the propagation simulation. It is said that it is possible to generate a large amount of training data above and build a machine learning model with improved estimation accuracy.

JP-A-2018-148297 Japanese Unexamined Patent Publication No. 2016-212675 Japanese Unexamined Patent Publication No. 2019-144941 Japanese Unexamined Patent Publication No. 2019-174164

Rishi Nishio, "Wireless Communication Quality Prediction and Communication Control by Machine Learning", Shingaku Giho, Vol. 118, No. 57, SR2018-1, pp. 1-8, 2018

However, in the prior art as disclosed in Non-Patent Document 1 described above, for example, in a normal situation where a plurality of objects are included in the time-series environmental information, when the correct answer data is obtained among these objects. It is not possible to identify the terminal corresponding to the received power information of the series. As a result, it becomes very difficult to build a machine learning model with high estimation accuracy that can be used in actual operation. That is, it is determined whether or not the object (which may include the terminal) captured by the RGB-D camera and the terminal related to the measured received power are the same target (whether or not they have a corresponding relationship). Since it cannot be done, it is not possible to generate highly reliable training data in the first place.

Even if a machine learning model as disclosed in Non-Patent Document 1 is applied, for example, an object (terminal) detected from time-series environmental information generated by an RGB-D camera is the machine learning model. It is completely unknown whether or not it is related to the received power predicted by, and therefore, even on the transmitting station side, which terminal is connected to the predicted received power, and even In the first place, it becomes impossible to determine even whether or not the predicted received power is related to any terminal.

In this regard, in the technique disclosed in Patent Document 1, the correspondence relationship between the terminal imaged by the RGB-D camera and the measured received power can be determined in the virtual space, so that, for example, a situation in which a plurality of terminals can move. It is possible to build a machine learning model corresponding to. However, even if such a model is applied, the terminal cannot be identified after all in the scene of predicting the received power in the actual operation, and the same problem as that of Non-Patent Gazette 1 as described above occurs.

Here, Patent Document 2 discloses an object recognition system that grasps the behavior of a mobile terminal and associates the mobile terminal with a moving object detected by a computer. In this system, each mobile terminal sequentially detects various state quantities such as acceleration acting on its own terminal, transmits the time-series detection result to the server as terminal behavior data, and the server receives an image input from the camera. The behavior of moving objects existing in the shooting range is monitored based on the signal, and data indicating the time change of the state amount related to the behavior of each moving object is stored in the server-side storage unit as estimation result history data. Next, by comparing the terminal behavior data transmitted from the mobile terminal with the estimation result history data for each mobile body, the correspondence relationship between the mobile body existing in the shooting range and the mobile terminal is specified.

However, in the system according to Patent Document 2, the only criterion for specifying the correspondence is the physical state quantity caused by the behavior of the mobile body, and each mobile terminal is a base station, which is a matter of concern. No information related to the communication to be performed is used. In addition, since it is necessary for the mobile terminal side to acquire information on a predetermined state that changes due to its own behavior, equipment such as a sensor capable of detecting the predetermined state is indispensable for each mobile terminal. Furthermore, since it becomes necessary to sequentially transmit the detection results, it becomes a technically troublesome situation.

Further, Patent Document 3 describes the flow line information of a person acquired by using a camera or the like in an environment in which one or more persons possessing a wireless receiver are staying or moving, and the wireless reception. A technique for associating with a machine (a person who owns it) is disclosed. Specifically, this technology identifies the time zone in which the person is staying in the predetermined area from the flow line information of the person, and the wireless transmitter installed at the position corresponding to the predetermined area and the receiver possessed by the person. From the reception history information generated by communication with and, a person who is likely to be staying in a predetermined area in that time zone is determined, and the person is associated with the flow line information that specifies the time zone. It has become a thing.

However, in the technique of Patent Document 3, in order to associate the flow line information of the person who possesses the wireless receiver with the flow line information of the person acquired by the camera, it is a major premise that the person stays in a predetermined area. It has become. Of course, by setting a large number of predetermined areas, the probability that the person will stay in the predetermined area increases, but in that case, it becomes necessary to install wireless transmitters in each of the large number of predetermined areas.

Further, Patent Document 4 discloses a technique of acquiring received electromagnetic wave information in a terminal that has received an electromagnetic wave from an electromagnetic wave source (beacon) and estimating the position of the terminal. Specifically, this technique generates learning data including this received electromagnetic wave information, object recognition information acquired from an object recognition means capable of recognizing a movable object, and terminal position information related to the correct position of the terminal. , A model for terminal position estimation is generated, and using the generated model, the position estimation target is based on the feature data including the received electromagnetic wave information and the object recognition information at the time of position estimation acquired from the position estimation target terminal. It determines the position of the terminal.

However, in the technique of Patent Document 4, the correspondence between the terminal related to the received electromagnetic wave information and the object (terminal) related to the object recognition information is basically based on the terminal position estimated from the received electromagnetic wave information and the object recognition information. Since it is performed based on the geometrical positional relationship (for example, distance) with the acquired object position, for example, if there are many objects near the estimated terminal position, there is a possibility that incorrect mapping will be performed. Will be high. Further, in reality, it is necessary to arrange a plurality of beacons in the area, and it is not realistic to apply them to the entire general communication area.

Therefore, the present invention provides information related to communication with the terminal without relying on the mutual positional relationship or only the positional relationship for the correspondence relationship between the terminal and the object existing in the environment in which the terminal can exist. It is intended to provide devices, programs and methods that can be utilized and determined.

According to the present invention
Based on the information related to communication acquired from the communication means that communicates with the terminal, the terminal communication state determining means that determines the terminal communication state information, which is the information related to the communication state with the terminal, and the terminal communication state determining means.
Environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist is acquired, and environmental information that can include a predetermined target and / or a failure target that may interfere with communication is acquired, and the environmental information is obtained. Therefore, if the predetermined target includes a terminal, the target communication state determining means for determining the target communication state information, which is the information related to the communication state to be embodied with the predetermined target,
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. Provided is a terminal identification device having a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information relating to the correspondence relationship between the terminal and the predetermined target.

As an embodiment of the terminal identification device according to the present invention, the terminal communication state determining means also determines the terminal position information which is the information related to the position of the terminal based on the information related to the communication.
The target communication state determining means also determines the target position information, which is the information related to the detection position of the predetermined target detected from the environment information.
The correspondence relationship determining means calculates the degree of position correspondence, which is the degree of correspondence between the determined terminal position information and the determined target position information, and based on the degree of position correspondence, the terminal and the predetermined target. It is also preferable to determine whether or not there is a correspondence relationship with.

Further, as an embodiment relating to the terminal and the target communication status information of the present invention, the terminal communication status information is information relating to the presence or absence of a communication connection with the terminal.
The target communication state determining means communicates with the predetermined target based on the detection position of the predetermined target detected from the environment information and the detection position of the failure target when the failure target is detected. The straight line portion connecting the two is determined, and whether or not the obstacle target is located on the straight line portion or within a predetermined area including the straight line portion, or whether or not the straight line portion is blocked. Information on whether or not a communication connection with a predetermined target is possible is calculated, and the information on whether or not the communication connection is possible is determined as the target communication status information.
The correspondence relationship determining means is based on the communication connection correspondence degree, which is the degree of correspondence between the information relating to the presence or absence of communication connection as the terminal communication status information and the information relating to the availability of communication connection as the target communication status information. It is also preferable to determine whether or not the terminal and the predetermined target have a corresponding relationship.

Further, as another embodiment relating to the terminal and the target communication status information of the present invention, the terminal communication status information is information relating to the strength of the radio wave received from the terminal.
The target communication state determining means communicates with the predetermined target based on the detection position of the predetermined target detected from the environment information and the detection position of the failure target when the failure target is detected. The straight line portion connecting the two is determined, the degree of shielding information relating to the degree to which the obstacle target blocks the predetermined area including the straight line portion is calculated, and the predetermined communication state information is used as the target communication state information from the shielding degree information. If the target includes a terminal, determine the information related to the strength of the radio wave that will be received from the predetermined target.
The correspondence relationship determining means is based on the radio wave intensity correspondence degree, which is the degree of correspondence between the information related to the radio wave strength as the terminal communication state information and the information related to the radio wave strength as the target communication state information. It is also preferable to determine whether or not the terminal and the predetermined target have a corresponding relationship.

Further, as another embodiment relating to the terminal and the target communication status information of the present invention, the terminal communication status information is information relating to the intensity of the radio wave received from the terminal.
The target communication state determining means uses a learned target radio wave information estimation model that outputs information related to the strength of the radio wave received by the communication means according to the input environmental information, and uses the predetermined environmental information from the environmental information. If the target includes a terminal, the information related to the strength of the radio wave that will be received from the predetermined target is derived, and the information related to the strength of the radio wave is determined as the target communication state information.
The correspondence relationship determining means is based on the degree of correspondence between the terminal and the terminal, which is the degree of correspondence between the information related to the radio wave strength in the terminal communication state information and the information related to the radio wave strength in the target communication state information. It is also preferable to determine whether or not the predetermined target has a correspondence relationship.

Further, in the embodiment in which the correspondence relationship is determined using the radio field intensity correspondence degree described above, the terminal communication state information includes information relating to the presence / absence of a communication connection with the terminal.
The target communication state determining means communicates with the predetermined target based on the detection position of the predetermined target detected from the environment information and the detection position of the failure target when the failure target is detected. The straight line portion connecting the two is determined, and whether or not the obstacle target is located on the straight line portion or within a predetermined area including the straight line portion, or whether or not the straight line portion is blocked. Information on whether or not a communication connection with a predetermined target is possible is calculated, and information on whether or not the communication connection is possible is also determined as information included in the target communication status information.
The correspondence relationship determining means is the communication connection correspondence degree, which is the degree of correspondence between the information relating to the presence or absence of communication connection in the terminal communication status information and the information relating to the availability of communication connection in the target communication status information, and the radio wave strength. Based on whether or not the correspondence is equal to or higher than a predetermined value, or based on the sum of the communication connection correspondence and the radio field strength correspondence or the weighted sum, the terminal and the predetermined target are subjected to each other. It is also preferable to determine whether or not there is a correspondence relationship.

Further, as another embodiment of the terminal identification device according to the present invention, the above-mentioned sensors are a plurality of sensors provided at different positions from each other.
The target communication status determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target communication status information from each of the plurality of environmental information, and among the determined target communication status information, The feature quantities of the detection areas of the predetermined targets related to the target communication status information are similar to each other and / or the detection positions of the predetermined targets related to the target communication status information are closer to the predetermined values or more. It is also preferable that the target communication status information of the objects relates to the same predetermined object.

Further, as a further other embodiment in the terminal identification device according to the present invention, the target communication state determining means is a sensor different from the above-mentioned sensor in the other terminal identification device from other terminal identification devices existing in the vicinity. Acquires the target communication status information determined from the environment information acquired from
The correspondence relationship determining means also calculates the degree of correspondence between the acquired target communication status information and the determined terminal communication status information, and based on the degree, the terminal and the predetermined target have a correspondence relationship. It is also preferable to determine whether or not there is.

Further, as a further other embodiment of the terminal identification device according to the present invention, the terminal includes a plurality of terminals installed or carried on a mobile body, and the terminal identification device is a terminal identification device.
An approach target identifying means for identifying a plurality of predetermined targets that are estimated to be closer to each other later than a predetermined value based on the detected positions of the plurality of detected targets.
It is presumed that the correspondence relationship determination means approaches at least one of at least one terminal determined to have a correspondence relationship with any of the specified plurality of predetermined objects, and approaches the predetermined value or more. It is also preferable to further have a communication control means for transmitting such information.

According to the present invention, the learning data includes the environmental information and the information related to the predetermined target detected from the environmental information, and the terminal identification device described above corresponds to the predetermined target. Provided is a learning model for estimating terminal radio wave information, which is constructed by learning data in which information related to a communication state with a terminal determined to be related is used as correct answer data.

According to the present invention, further, data including the environmental information and information related to a predetermined target detected from the environmental information is input to the above-mentioned learning model, and the terminal output by the learning model. A terminal communication state estimation means that estimates information related to the communication state of this predetermined target as a terminal based on the information related to the communication state, and
Provided is a communication relay device having a communication control means for determining whether or not to switch the communication path of the predetermined target as a terminal based on the information related to the estimated communication state.

According to the present invention, it is also a program for operating a computer capable of determining a correspondence relationship between a terminal and a predetermined target.
A terminal communication state determining means that determines terminal communication status information, which is information related to a communication status with the terminal, based on information related to communication acquired from a communication means that communicates with the terminal.
Environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist is acquired, and environmental information that can include the predetermined target and / or a failure target that may interfere with communication is acquired, and the environment is obtained. From the information, if the predetermined target includes a terminal, the target communication state determining means for determining the target communication state information, which is the information related to the communication state to be embodied with the predetermined target, and
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. A terminal identification program for functioning a computer as a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information relating to the correspondence relationship between the terminal and the predetermined target is provided.

According to the present invention, a computer-based terminal identification method capable of determining the correspondence between a terminal and a predetermined target is further provided.
Based on the information related to communication acquired from the communication means that communicates with the terminal, the terminal communication status information, which is the information related to the communication status with the terminal, is determined, while the environment in which the terminal can exist is determined. Environmental information, which is information related to the environment, is acquired from the sensing sensor and may include a predetermined target and / or a failure target that may cause a communication failure, and the predetermined target is obtained from the environmental information. A step of determining the target communication state information, which is information related to the communication state to be embodied with the predetermined target if the terminal is included, and
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. Provided is a terminal identification method having a step of determining whether or not a target has a correspondence relationship and determining information relating to a correspondence relationship between the terminal and the predetermined target.

According to the terminal identification device, the program and the method of the present invention, the correspondence between the terminal and the object existing in the environment in which the terminal can exist is the said, without relying on the mutual positional relationship or only the positional relationship. It is possible to make a decision using information related to communication with the terminal.

It is a schematic diagram which shows one Embodiment of the terminal identification apparatus by this invention. It is a schematic diagram for demonstrating one Embodiment of the processing in the terminal communication state determination part, the target communication state determination part, and the correspondence relation determination part which concerns on this invention. It is a schematic diagram for demonstrating one application example of the terminal identification method by this invention. It is a schematic diagram for demonstrating one Embodiment of the processing in the model construction part and the terminal radio wave information estimation part which concerns on this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Terminal identification device]
FIG. 1 is a schematic view showing an embodiment of a terminal identification device according to the present invention.

The base station 1 as the terminal identification device according to the present invention shown in FIG. 1 is a communication relay device compatible with 5G (fifth generation mobile communication system) in the present embodiment, and is also a communication terminal compatible with 5G. It is a device capable of performing communication with a plurality of terminals 2. Further, in the present embodiment, the base station 1 includes a camera 103, which is an RGB-D camera capable of capturing a situation in a communication area with a predetermined angle of view.

The (RGB and depth) image data (video data) generated by the camera 103 includes
(A) A person who possesses or carries a terminal 2
(B) Moving objects such as automobiles, motorcycles, railroad cars, robots, drones, etc. equipped with, including, or boarding the terminal 2, and (c) equipment, facilities, buildings, fixed objects, etc. on which the terminal 2 is installed. There is a possibility that such "objects" are included (imaged), and in addition, people, moving objects, facilities, facilities, buildings, and fixed (including plants such as trees) that are not related to the terminal 2 are fixed. An "object" such as an object can also be included.

By the way, while such an "object" can determine the correspondence with the terminal 2 to be identified, it also becomes an obstacle to communication radio waves because it exists between the terminal 2 and the base station 1. obtain. In particular, 5G, which is the communication method of the present embodiment, uses radio waves in the millimeter wave band for communication, but the millimeter wave has high straightness and is less likely to be diffracted, so that the terminal 2 and the base station 1 are used. There is a high possibility that the propagation will be blocked or attenuated by the "object" that exists between and.

Therefore, in 5G, it is a big problem that the received signal power at the base station 1 is drastically reduced due to the intervention of such an "object", and the communication quality is significantly deteriorated. Here, if it is possible to predict the decrease in the received signal power at the base station 1 due to such an "object", it is possible to maintain the communication quality by controlling the communication path of the terminal 2 or the like. Therefore, the base station 1 of the present embodiment not only determines the correspondence between the terminal 2 and the "target", but also predicts the received signal power and controls the communication path, as will be described later. It is a device that makes it possible.

Similarly, in FIG. 1, the base station 1 first determines the correspondence between the terminal 2 and a predetermined target, as a specific feature thereof.
(A) Based on information (for example, communication history information) related to communication acquired from a communication means (in this embodiment, a communication interface 101, a communication control unit 114, and a communication history information storage unit 102) that communicates with a terminal. , The terminal communication state determination unit 111 that determines the "terminal communication state information" which is the information related to the communication state with the terminal 2.
(B) "Environmental information" that is information related to the environment from the sensor (camera 103 in this embodiment) that senses the environment in which the terminal 2 can exist, and is an obstacle that can be a predetermined target and / or a communication obstacle. "Environmental information" that can include a target is acquired, and from this "environmental information", if the predetermined target includes the terminal 2, it is information related to a communication state that will be realized with the predetermined target. Target communication status determination unit 112 that determines "target communication status information",
(C) Calculate the "correspondence degree" which is the degree of correspondence between the determined "terminal communication state information" and the determined "target communication state information", and based on this "correspondence degree", "terminal communication". It is determined whether or not the terminal 2 related to the "state information" and the predetermined target related to the "target communication state information" have a correspondence relationship, and the "terminal" which is the information related to the correspondence relationship between the terminal 2 and the predetermined target. It has a correspondence relationship determination unit 113 that determines "target correspondence information".

Here, the "predetermined object" detected in the above (B) may be a "person" or a "moving body" carrying, carrying, or having the possibility of including the terminal 2, which will be described later. it can. Further, in some cases, "equipment", "facility", "building", "fixed object", etc., which may include the terminal 2, may also be "predetermined objects". Further, the "failure target" is a target that can cause a communication failure as described above, and can be, for example, a target other than the "predetermined target" among the targets detected from the environmental information.

Further, the above-mentioned "terminal communication state information" of (A) is a suitable form.
(Α) Information related to the presence or absence of a communication connection with the terminal 2, or (β) Information related to the strength of the radio wave received from the terminal 2 (for example, received signal power, RSSI (Received Signal Strength Indicator)).
Can be.

On the other hand, the "target communication status information" of the above (B) will be described in detail later in accordance with the above "terminal communication status information".
(Α') Information on whether or not a communication connection with the predetermined target is possible, which is determined by using the detection position of the fault target when the fault target is detected, or (β') the fault target is detected. In this case, if the predetermined target includes the terminal 2, the information related to the strength of the radio wave determined by using the detection position of the obstacle target, or the information estimated from the "environmental information" using the target radio wave information estimation model, is applicable. It can be information related to the strength of radio waves to be received from a predetermined target.

Further, in these preferred forms, the "correspondence" of the above (C) is
(A) The degree of communication connection matching, which is the degree of matching between the information relating to the presence or absence of communication connection as "terminal communication status information" and the information relating to the availability of communication connection as "target communication status information", or (b). ) As the degree of radio intensity matching, which is the degree of agreement between the amount of radio wave strength (received signal power value, RSSI value) in the "terminal communication status information" and the amount of radio wave intensity in the "target communication status information". The correspondence relationship determination unit 113 determines whether or not the terminal 2 and the predetermined target have a correspondence relationship based on such a “correspondence degree”.

As described above, according to the base station 1 as the terminal identification device according to the present invention, the correspondence between the terminal 2 and a predetermined target (detected from "environmental information") existing in the environment in which the terminal 2 can exist. Can be determined by comparing each "communication state" with each other. That is, the "terminal target correspondence information" can be determined from the comparison of the "communication states" of the terminal 2 and the predetermined target without depending on the mutual positional relationship.

As will be described later, in the base station 1, an embodiment in which "terminal target correspondence information" is determined based not only on the mutual "communication state" of the terminal 2 and a predetermined target but also on the mutual positional relationship. It is also possible to take. However, in any case, the base station 1 can determine the correspondence relationship between the terminal 2 and the predetermined target without relying only on the mutual positional relationship. As a result, for example, when a large number of predetermined objects exist in the vicinity of the terminal 2 to be identified (when detected), an erroneous correspondence relationship occurs because the difference in position between them is small. It is also possible to reduce the risk of being determined by considering those "communication states".

Although 5G is adopted as the communication method between the base station 1 and the terminal 2 in the present embodiment, naturally other communication methods such as LTE may be used, and the terminal identification according to the present invention. The communication between the device and the terminal may be based on various other wireless communication standards. For example, even in a communication method in which the problem of shielding by an object is not as remarkable as 5G, there are not a few situations in which it is desired to accurately identify a subordinate terminal using an object detected from "environmental information". For example, there are various terminal identification needs, such as determining the relationship between the browsing page of a user who is determined to have a corresponding relationship with a certain terminal and the flow line of the user and utilizing it for marketing and management. On the other hand, according to the terminal identification device according to the present invention, it is possible to carry out such a terminal identification process with high accuracy.

Further, the terminal identification device according to the present invention is not limited to a communication relay device such as a base station. For example, as a dedicated device for terminal identification processing, it may be provided in a form of being connected to a communication relay device / equipment such as a base station. Further, as the terminal identification device according to the present invention, it is also possible to use a cloud server, a non-cloud server device, a personal computer (PC), a notebook computer, a tablet computer, or the like equipped with the terminal identification program according to the present invention. is there.

Furthermore, at least one of the above (A) to (C), which is a component of the terminal identification device (base station 1) according to the present invention, is a device different from the other components. It is not impossible to take. For example, it is possible to realize the above-mentioned functions (A) to (C) by the whole of a plurality of servers. Here, even in such a case, all of these can be regarded as a terminal identification device or system that implements the terminal identification method according to the present invention.

[Functional configuration of terminal identification device, terminal identification program]
Similarly, in the functional block diagram of FIG. 1, the base station 1 has a communication interface 101, a communication history information storage unit 102, a camera 103, and a camera image storage as an embodiment of the terminal identification device and the communication relay device according to the present invention. It has a unit 104 and a processor memory. Here, the processor memory stores an embodiment of a communication relay program including the terminal identification program according to the present invention, and also has a computer function, and by executing this communication relay program, Perform terminal identification processing and communication relay processing.

Further, the processor memory communicates with the terminal communication state determination unit 111 including the terminal position determination unit 111a, the target communication state determination unit 112 including the target position determination unit 112a, and the correspondence relationship determination unit 113 as functional components. It has a control unit 114, an approach target identification unit 121, a model construction unit 131, and a terminal radio wave information estimation unit 132. Note that these functional components can be regarded as the functions of the terminal identification program and the communication relay program stored in the processor memory, and arrows are drawn between the functional components of the base station 1 in the functional block diagram of FIG. The process flow shown in connection with is also understood as an embodiment of the terminal identification method and the communication relay method according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the communication control unit 114 functions as a base station by controlling the wireless communication operation between the communication interface 101 and each terminal 2, and further communicates with each terminal 2. Various information related to the above is acquired and recorded to generate communication history information in which the information is arranged in chronological order, and the communication history information storage unit 102 stores and manages the information.

The terminal communication status determination unit 111 acquires communication history information from the communication history information storage unit 102, and determines terminal communication status information which is information related to communication with the terminal 2 based on the communication history information. Here, in the present embodiment, this terminal communication state information is
(Α) Information related to the presence or absence of a communication connection with the terminal 2, and (β) Received signal power (RSSI) of the radio wave signal received from the terminal 2.
Either one or both.

Regarding (α) above, the terminal communication state determination unit 111 provides time-series information as to whether or not a communication connection with the terminal 2 is established at each time point, that is, when communication is established at time point t. 1. Two-value communication connection presence / absence variable that takes a value of -1 when it is not established:
(1) B_t ∈ {−1, 1}
Time series information of may be generated.

Further, regarding the above (β), the terminal communication state determination unit 111 may generate time-series information of the received signal power variable E_t indicating the signal power value at the time point t of the radio wave signal received from the terminal 2. Further, as described above, as a preferred embodiment, the terminal communication state determination unit 111 provides the terminal communication state information including both the time series information of the communication connection presence / absence variable B_t and the time series information of the received signal power variable E_t. It is also preferable to determine.

Although the terminal communication state information determination process of the terminal 2 has been described above, the terminal position determination unit 111a of the terminal communication state determination unit 111 can be used in the acquired communication history information, although in a preferred embodiment described later. It is also preferable to determine the terminal position information, which is the information related to the position of the terminal 2, based on the received signal power included (or generated from the communication history information). Here, specifically, not only the received signal power at the base station 1 but also
(A) Received signal power measurement values at a plurality of base stations existing around the base station 1 and
(B) To determine the terminal position information of the terminal 2 by using, for example, a well-known base station positioning technology that acquires and uses the received signal power measurement value of the terminal 2 that has received radio waves from a plurality of base stations by communication. Can be done.

Similarly, in the functional block diagram of FIG. 1, the camera 103 may be a photographing device compatible with visible light, near infrared rays, or infrared rays including a solid-state image sensor such as a CCD image sensor or a CMOS image sensor, and is a stereo camera. It can also be a spherical (omnidirectional) camera. Of course, instead of the camera 103, it is also possible to adopt a sensor that can sense the environment in which the terminal 2 can exist and generate environmental information, such as a LiDAR, a laser / infrared positioning device, a TOF camera, and a thermography device. is there.

Here, in the present embodiment, the camera 103 is an RGB-D camera capable of generating an RGB image and a depth image as environmental information, and the generated environmental information (RGB image data and depth image data) is a camera image. It is stored and managed by the storage unit 104.

Although the camera 103 is installed in the base station 1 in the present embodiment, for example, it is a camera installed at a position separated from the base station 1, for example, a surveillance camera in the city, and communicates with the base station 1. It may be connected. Further, as will be described later, it is also possible to take an embodiment in which a plurality of cameras 103 are provided at different positions regardless of whether the base station 1 is inside or outside.

Next, the target communication state determination unit 112
(A) Image data (environmental information) from the camera image storage unit 104 (RGB and depth) that may include a "predetermined object" and / or an "obstacle target" that may interfere with communication. To get and
(B) From the acquired image data, if the "predetermined target" includes a terminal, the "target communication status information" which is the information related to the communication status to be embodied with the "predetermined target" is determined. To do.

Here, the "predetermined object" of the above (a) and (b) is, for example,
(A) The person (user) who owns the mobile terminal,
(B) An automobile equipped with a terminal 2 having a drive recorder function, a CAN information transfer function, an interface function for automatic driving control by a server, and the like, and further.
(C) A person who may include a terminal 2 which is a communication destination related to communication history information, such as an autonomous mobile robot or an autonomous flight drone equipped with a terminal 2 having an interface function for autonomous movement control by a server. It can be "or" moving body ". In some cases, "equipment", "facility", "building", "fixed object", etc., which may include the terminal 2, may also be "predetermined objects".

In the present embodiment, when a "predetermined target" is detected from one image data, the other target detected from the image data is treated as a "disorder target" with respect to the "predetermined target". Become.

<When "target communication status information" is communication connection availability information>
In addition, the "target communication status information" determined in (a) above is
(Α') It can be information related to whether or not a communication connection with a "predetermined target" is possible. In this case, the target communication state determination unit 112 sets the detection position of the "predetermined target" detected from the acquired image data and the detection position of the "failure target" when the "failure target" is detected. Based on
(A) Determine the straight line that connects the "predetermined target" and base station 1 (antenna).
(B) Communication connection availability information (shielding presence / absence information) whether or not the "failure target" is located on this straight line or within a predetermined area including this straight line, or whether or not this straight line is blocked. Is calculated,
(C) It is also preferable to determine the calculated communication connection availability information as the target communication status information.

More specifically, as the communication connection availability information in (b) above, the detection position of any "failure target" is the above straight line at a certain point in time t (= k, k + 1, ..., k + M). A binary communication connection enable / disable variable that takes a value of -1 if it is located on a minute or within a rotating body region of a predetermined size that includes this straight line as a rotation axis, and 1 if it is not.
(2) B'_t ∈ {−1, 1}
Time series information of is generated, and this may be used as the target communication state information. Here, the rotating body region may be a cylindrical region, or may be formed by rotating a predetermined curve around a rotation axis such as an ellipsoid.

Further, as a modification mode, the target communication state determination unit 112 sets a predetermined size (cross-sectional area at the detection position) in advance for the "failure target" according to the type (class), and any of the "failure targets". When the "obstacle target" blocks the above straight line portion by its size (cross-sectional area) at the detection position (the above straight line portion penetrates the cross section of the "obstacle target"), -1, If this is not the case, B'_t in (2) above may be defined as a communication connection availability variable that takes a value of 1.

<When the "target communication status information" is the received signal strength information>
Furthermore, the "target communication status information" determined in (a) above is
(Β') If the "predetermined target" includes a terminal, it can be information related to the intensity of radio waves to be received from the "predetermined target". In this case, the target communication state determination unit 112 sets the detection position of the "predetermined target" detected from the acquired image data and the detection position of the "failure target" when the "failure target" is detected. Based on
(A) Determine the straight line that connects the "predetermined target" and base station 1 (antenna).
(B) Calculate the degree of shielding information related to the degree to which the "obstacle target" blocks a predetermined area including this straight line.
(C) From the calculated degree of shielding information, as the target communication state information, if the "predetermined target" includes the terminal 2, the information related to the strength of the radio wave to be received from the "predetermined target" is determined. It is also preferable. Here, more specifically, the shielding degree information in (b) above is calculated as follows.

First, a predetermined size (cross-sectional area at a detection position having a predetermined area) is set in advance for the "obstacle target" according to the type (class), and further, the above straight line portion is used as the rotation axis. After setting a rotating body region of a predetermined size to be included as, at a certain point in time t (= k, k + 1, ..., k + M),
(B1) In any of the "obstacle targets", a part of the cross-sectional region is not included in the above-mentioned rotating body region at the detection position (the cross-sectional region of the obstacle target and the rotating body region have an overlapping portion). If not) then take a value of 1, while
(B2) In any of the "obstacle targets", a part of the cross-sectional region is included in the above-mentioned rotating body region at the detection position (the cross-sectional region of the obstacle target and the rotating body region have an overlapping portion). ) in this case, the following equation _{(3) R = (S a} ∩S b -) / S a
The time series information of the variable E'_t that takes the R value calculated by can be used as the shielding degree information. Incidentally, also in the case of the above (b1), the case where the R value is 1 (although the R value calculation process does not need to be executed) can be included in the above equation (3) after all.

Here, in the above equation (3), S _a is a set representing a cross-sectional area (perpendicular to the rotation axis) at the detection position of the "obstacle target" in the above-mentioned rotating body region (for example, a unit point forming the region). A set of regions), and S _b is a set representing a cross-sectional region at the detection position of the “obstacle target” (for example, a set of unit point regions forming the region). Further, S _b ^- is _{a complement of S b} , and "/" is the area value of the cross-sectional area related to the set of molecules (for example, the number of unit point regions) and the area value of the cross-sectional area related to the denominator (for example, the unit). It is an operator that indicates division by the number of point domains).

Next, the target communication state determination unit 112 displays the time series information of this variable E'_t (or the variable E'_t multiplied by a predetermined constant) if the "predetermined target" includes the terminal 2. Information related to the strength of the radio wave to be received from the "predetermined target" is used, and this is used as the target communication status information. Hereinafter, the variable E'_t will be referred to as a received power equivalent variable in order to distinguish it from the above-mentioned received signal power variable E_t.

When there are two or more "obstacle targets" that take the above R value, S _b is the region formed by the entire cross-sectional region of these "obstacle targets", that is, the union of these cross-sectional regions. It may be a represented area. Further, the rotating body region may be a cylindrical region, or may be formed by rotating a predetermined curve around a rotation axis such as an ellipsoid.

Further, as a modification mode, the target communication state determination unit 112 uses a trained "target radio wave information estimation model" that outputs information related to the strength of the received radio wave according to the input image data, and obtains an image. From the data, if the "predetermined target" includes the terminal 2, the information related to the strength of the radio wave to be received from the "predetermined target" is derived, and the information related to the strength of the radio wave is obtained from the target communication state. You may decide on information.

Here, as the above-mentioned "target radio wave information estimation model", a known machine learning model as disclosed in Non-Patent Document 1 can be adopted. Specifically, the image data at a certain time point t (= k, k + 1, ..., k + M) is input to this model, the estimated value of the received signal power is output, and this estimated value is equivalent to the received power. As the variable E'_t value, the time series information of the received power equivalent variable E'_t is used as the target communication state information.

As mentioned above, "target communication status information",
(Α') Communication connection enable / disable information (time series information of communication connection enable / disable variable B'_t), and (β') Received radio field intensity information (time series information of received power equivalent variable E'_t) The processing of was explained. Here, as the process of detecting the target from the image data, which is carried out in these processes, for example, a model for target detection is constructed based on a machine learning algorithm well known for various image recognition, and the model is used. It is used to identify a detection area (for example, a bounding box) in which an object is presumed to exist in the image data, and further calculate a score indicating the certainty that the detection area is an object to satisfy a predetermined condition. A well-known image recognition process such as determining a detection region having a high score as a target region can be adopted. Further, the detection position of the target can be a representative point (for example, the center of gravity or the midpoint of the lower end) of the determined target region.

By the way, when a stereo camera is adopted as the camera 103, for example, non-patent documents: Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, “SSD: single shot multibox detector”, European Conference on Computer Vision, Computer Vision-ECCV 2016, pp. 21-37, by applying the object detector described in 2016. And it is possible to determine the detection position.

When LiDAR is used as the sensor according to the present invention instead of the camera 103, for example, Charles R. Qi, Hao Su, Kaichun Mo, Leonidas are used for the point cloud (point cloud) which is the generated environmental information. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation”, Journal of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, using the object detector described in 2016 Thereby, the target area and the detection position can be determined.

Further, when the target communication state determination unit 112 does not detect a predetermined target from the environmental information, that is, when one predetermined target is not measured (sensed), or when it is measured but is not detected due to the influence of noise or the like. It is also preferable to end the target communication state determination process for the environment information.

Further, the target position determination unit 112a of the target communication state determination unit 112 can determine the position (detection position) of the detected target as described above, but in the present embodiment, further, for each detected target, After associating the target with the detection position at each time point, time-series information (movement history information) of the position of the target is generated. This also makes it possible to track each detected object individually.

Here, for such a target tracking process, for example, a Kalman filter, which is a well-known state estimation method, is applied to the determined target area (bounding box), and the target detection area (from the state at the past time point to the current time point) ( The bounding box) is predicted, and the target area is determined based on the evaluation value using the overlapped area between the predicted detection area (bounding box) and the detection area (bounding box) detected at the present time as an evaluation value. It is also preferable. By the way, such target tracking processing is described in, for example, non-patent documents: Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft, “Simple Online and Realtime Tracking”, Journal of 2016 IEEE International Conference on Image Processing (ICIP), Pp. 3464-3468, 2016.

Further, as another embodiment, when a plurality of sensors other than the camera 103 or the above-mentioned camera are provided at different positions inside or outside the base station 1, the target communication state determination unit 112 may be used.
(A) Obtaining a plurality of environmental information from each of the plurality of cameras 103 (or sensors),
(B) The target communication status information is determined from each of these multiple environmental information, and the target communication status information is determined.
(C) Of the determined target communication status information
(C1) For objects whose detection positions related to the target communication status information are closer than a predetermined value (for example, those whose distances are less than a predetermined threshold value), and / or
(C2) Image features of the target detection area (bounding box) related to the target communication state information are similar to each other more than a predetermined value (for example, the distance between them in the image feature space is less than a predetermined threshold value). about,
It is also preferable that the target communication status information relates to the same target. Further, in this case, the representative value (for example, the average value) of the detection position group for the target related to the target communication state information may be treated as the position of the same target.

As a result, it is possible to avoid a situation in which a certain terminal 2 is not included in the acquired environmental information, and as a result, the identification process of the terminal 2 fails. For example, with respect to the environmental information obtained by one camera 103 (or sensor), one object is located behind another object and is shielded by the other object, so that it is not included in the environmental information (not imaged). The situation will occur not a little. For example, from one camera 103, it is quite possible that the person holding the terminal 2 is obstructed by the stopped bus and cannot be seen.

On the other hand, if environmental information from a plurality of cameras 103 (or sensors) having different installation positions is used, an object that is not included in one environmental information is included in other environmental information that has a different sensing (measurement) viewpoint. As a result, it is fully possible to avoid the above-mentioned concerns.

By the way, as a typical example of using the target communication state information related to the plurality of cameras 103 (or sensors) as described above, the base station 1 is one or more base stations 1 existing around itself. It is possible to receive and acquire the target communication status information generated by the camera 103 (or sensor) installed in the base station 1 from each of them, and perform the same target determination process as described above. In this case, it is possible for a plurality of base stations 1 to cooperate with each other to more preferably carry out the terminal identification process according to the present invention.

Similarly, in the functional block diagram of FIG. 1, the correspondence relationship determination unit 113
(A) Terminal communication status information determined by the terminal communication status determination unit 111, time-series information of the communication connection presence / absence variable B_t, or time-series information of the received signal power variable E_t in the present embodiment.
(B) The target communication status information determined by the target communication status determination unit 112, the time series information of the communication connection enable / disable variable B'_t, or the time series information of the received power equivalent variable E'_t correspond to each other in the present embodiment. The degree of correspondence C is calculated, and based on this degree of correspondence C, the terminal 2 related to the terminal communication status information of the above (a) and the predetermined target related to the target communication status information of the above (b) correspond to each other. It is determined whether or not there is a relationship, and the terminal target correspondence information which is the information related to the correspondence relationship between the terminal 2 and the predetermined target is determined.

Here, in the present embodiment, the above-mentioned degree of correspondence C is
(Α) As a cross-correlation between the communication connection availability variable B_t and the communication connection availability variable B'_t at each time point t in the predetermined time interval (time interval from t = k to k + M), the following equation (4) C = Σ _{t = k} ^{k + M} B_t * B'_t
May be calculated using, or
(Β) As a cross-correlation between the received signal power variable E_t and the received power equivalent variable E'_t at each time point t in the predetermined time interval (time interval from t = k to k + M), the following equation (5) C = Σ _{t = k} ^{k + M} E_t * E'_t
It can also be calculated using. Here, in the above equations (4) and (5), Σ _{t = k} ^{k + M} is an operator indicating the summation of the time point t (= k, k + 1,…, k + M). ..

Next, when the calculated correspondence degree C exceeds the predetermined threshold value C_th (when C> C_th), the correspondence relationship determination unit 113 causes the terminal 2 and the detected predetermined target (in the predetermined time interval) to move to each other. It is determined that there is a correspondence relationship, and for example, the terminal target correspondence information recorded by associating the terminal ID of the terminal 2 with the target ID of the predetermined target is determined. Note that this threshold C_th may be set to an empirically suitable value by, for example, an administrator.

Further, as a modification mode, the correspondence relationship determination unit 113 calculates the correspondence degree C for all the pairs generated between all the terminals 2 and all the detected predetermined targets, and the optimum result among them. It is also preferable to determine the correspondence relationship between the terminal 2 and the predetermined target by the greedy algorithm that selects the set that gives the (correspondence relationship) as the solution, and determine the terminal target correspondence information.

FIG. 2 is a schematic diagram for explaining an embodiment of processing by the terminal communication state determination unit 111, the target communication state determination unit 112, and the correspondence relationship determination unit 113.

In the embodiment shown in FIG. 2, the moving terminal 2 mounted on the automobile establishes a communication connection with the base station 1, and the base station 1 relates to the communication with the terminal 2. Communication history information is generated. Here, the terminal communication state determination unit 111 of the base station 1 is based on the generated communication history information, and at each time point t, the terminal ID of the terminal 2 (003 in FIG. 2) and the radio wave received from the terminal 2. A received signal power table including a record recorded in association with the received signal power value of the signal is generated, and further, received signal power information for the terminal 2, that is, time series information of the received signal power variable E_t is obtained from this table. Generate.

On the other hand, the automobile equipped with the terminal 2 is photographed by the camera 103 of the base station 1 including its moving state. The target communication state determination unit 112 of the base station 1 acquires image data obtained by imaging the vehicle as environmental information including the vehicle, and presets the image data as predetermined targets and obstacle targets. The "automobile" detection process is performed to generate time-series information of the detection position in each of the automobile equipped with the terminal 2 and other automobiles (as obstacles) traveling around.

Here, in FIG. 2, a vehicle equipped with the terminal 2 is assigned 008 as a target ID, and a bus (vehicle) detected as a failure target is assigned 905 as a target ID. Below, these cars are tracked. Hereinafter, the target communication status information is generated with a vehicle having a target ID of 008 (hereinafter abbreviated as vehicle 008) as a predetermined target (a target that may include the terminal 2), but of course, the target ID is 905. It is also possible to target a certain bus (hereinafter, abbreviated as bus 905) as a predetermined target. In this case, the automobile 008 will be treated as an obstacle target.

Next, the target communication state determination unit 112
(A) A straight line connecting the automobile 008, which is a predetermined target, and the base station 1 (antenna) is determined.
(B) Calculate the R value of the above equation (3) related to the degree to which the bus 905, which is the target of the failure, blocks the "cylindrical region centered on the determined straight line", and use the calculated R value as the received power equivalent variable E. As the'_t value, the time series information of the received power equivalent variable E'_t for the automobile 008 is generated.

After that, the correspondence determination unit 113 of the base station 1 sets the time series information of the received signal power variable E_t generated for the terminal 2 and the time series information of the received power equivalent variable E'_t generated for the automobile 008. The degree of correspondence C (E_t, E'_t) of the above equation (5) corresponding to the cross-correlation is determined. Next, when the determined correspondence degree C (E_t, E'_t) exceeds the predetermined threshold value C_th (when C> C_th), the correspondence relationship determination unit 113 corresponds between the terminal 2 and the automobile 008 (predetermined target). It is determined that there is a relationship, and finally, a terminal target correspondence table (terminal target correspondence information) including a record recorded by associating the terminal ID (003) of the terminal 2 with the target ID (008) of the automobile 008 is generated. To do.

As described above, according to the base station 1 of the present embodiment, the correspondence between the terminal 2 and the predetermined target (automobile 008 in FIG. 2) existing in the environment in which the terminal 2 can exist is determined by the terminal 2 and the predetermined target. It can be determined by comparing the information (communication state) related to the strength of the received radio wave about the target with each other. That is, the terminal target correspondence information can be determined from the comparison of the respective communication states without relying on the mutual positional relationship between the terminal 2 and the predetermined target, and in particular, even if the terminal 2 is a movable terminal, the terminal 2 can be moved. The identification process can be carried out more reliably.

Although one embodiment of the terminal identification process has been described above with reference to FIG. 2, in this embodiment, the degree of correspondence between the terminal 2 and a predetermined target C (E_t, E'_t) (hereinafter, also referred to as the degree of correspondence with radio field intensity). ) Is used to determine the correspondence between the two. On the other hand, of course, the degree of correspondence C (B_t, B'_t) of the above equation (4) calculated as the cross-correlation between the communication connection presence / absence variable B_t and the communication connection possibility / rejection variable B'_t (hereinafter, communication connection correspondence) The correspondence between the two may be determined based on (also referred to as degree).

Furthermore, as another preferred embodiment, for the terminal 2 and the predetermined target, the terminal communication status information includes the time-series information of the communication connection presence / absence variable B_t and the time-series information of the received signal power variable E_t, and further, the target. Communication connection compatibility C (B_t, B'_t) and radio strength compatibility C are included in the communication status information, including the time-series information of the communication connection enable / disable variable B'_t and the time-series information of the reception power equivalent variable E'_t. After calculating (E_t, E'_t), the correspondence relationship determination unit 113 (Fig. 1)
(A) When both or one of the communication connection correspondence degree C (B_t, B'_t) and the radio field strength correspondence degree C (E_t, E'_t) is equal to or higher than the predetermined threshold value C_th. Or,
(B) When the sum or weighted sum of the communication connection correspondence C (B_t, B'_t) and the radio field strength correspondence C (E_t, E'_t) is equal to or more than a predetermined value.
It may be determined that the terminal 2 and the detected predetermined target have a corresponding relationship.

In this way, in the embodiment in which the identification processing of the terminal 2 is performed by integrating the two correspondence degrees C calculated for the communication states different from each other, the correspondence relationship is estimated from various angles by different methods for the communication state. As a result, the accuracy of the identification process of the terminal 2 is maintained or improved even under various communication environments.

Furthermore, as another preferred embodiment, the correspondence relationship determination unit 113 (FIG. 1) is
(A) Terminal position information determined by the terminal position determination unit 111a (FIG. 1) of the terminal communication state determination unit 111, and
(B) The position correspondence degree (position matching degree) LC, which is the degree of matching with the target position information determined by the target position determining unit 112a (FIG. 1) of the target communication state determining unit 112, is calculated, and the above-mentioned communication state is calculated. It is determined whether or not the terminal 2 and the detected predetermined target have a correspondence relationship based on not only the correspondence degree (communication connection correspondence degree and / or radio field strength correspondence degree) C but also the position correspondence degree LC. It is also preferable to do so.

Here, the position correspondence degree LC is, for example, a representative value in the predetermined time interval for the reciprocal of the absolute value (distance) of the difference between the position of the terminal 2 and the position of the predetermined target at each time point in the predetermined time interval. (For example, the average value) can be set. Further, the correspondence relationship determination unit 113 is, for example,
(A) Communication status correspondence C (communication connection correspondence C (B_t, B'_t) and / or radio field strength correspondence C (E_t, E'_t)) exceeds a predetermined threshold value C_th (C> C_th). And (b) the degree of positional correspondence LC exceeds a predetermined threshold value LC_th (LC> LC_th).
It is also preferable to determine that the terminal 2 and the detected predetermined target have a corresponding relationship. In the above (a), when both C (B_t, B'_t) and C (E_t, E'_t) are adopted as the correspondence degree C of the communication state, both of them exceed the predetermined threshold value C_th. Is a condition.

In this way, by carrying out the identification process of the terminal 2 in consideration of the position correspondence degree LC, it is possible to determine a more accurate correspondence relationship (between the terminal 2 and a predetermined target).

Further, another preferred embodiment regarding the degree of correspondence C of the communication state will be described. In this embodiment, at least one base station 1 exists around the base station 1, and each base station 1 uses the target communication status information (communication connection enable / disable variable B'_t) from the environmental information obtained by its own camera 103. And / or the time-series information of the received power equivalent variable E'_t) is determined, and the target communication status information determined by the user can be exchanged with each other.

Here, the target communication status determination unit 111 (FIG. 1) acquires the target communication status information (time-series information of B'_t and / or E'_t) determined by the surrounding base station 1, and further responds. The relationship determination unit 113 (FIG. 1) combines the target communication status information acquired in this way with the determined terminal communication status information (time-series information of the communication connection presence / absence variable B_t and / or the received signal power variable E_t). Correspondence degree (communication connection correspondence degree and / or radio strength correspondence degree) C'is also calculated, and not only the correspondence degree C of the target communication state information decided by oneself but also the correspondence degree C'is determined as the terminal 2. It is also preferable to determine whether or not there is a correspondence with the target of.

In this case, it is possible to carry out the terminal identification process similar to the terminal identification process performed in the embodiment in which the base station 1 is connected to the plurality of cameras 103 as described above. Further, this makes it possible to avoid a situation in which a certain terminal 2 is not included in the environmental information by its own camera 103, and as a result, the base station 1 fails in the identification process of the terminal 2. It becomes.

[Application example of terminal identification processing]
Hereinafter, an application example of the terminal identification process described above will be described.

In the functional block diagram of FIG. 1, the approach target identification unit 121 is detected by the target communication state determination unit 112 in a situation where a plurality of movable terminals 2 exist in the communication area environment (other than a predetermined target). Based on the detection positions of a plurality of objects (including an object, that is, an obstacle object), a plurality of objects that are estimated to approach each other later than a predetermined value are identified.

For example, the approach target identification unit 121 is each at a future time point after a predetermined time elapses, based on the current detection position of each detected target and the velocity vector calculated from the past time change of the detection position. It is also preferable to estimate the position of the target and specify the target whose distance from each other at the future time point is equal to or less than a predetermined threshold as the approach target group.

In this case, the communication control unit 114 receives the information of the plurality of targets specified by the approach target identification unit 121, and the correspondence relationship determination unit 113 determines that the communication control unit 114 has a correspondence relationship with any of these plurality of targets. (At least one) terminal 2 is specified, and at least one of the specified terminals 2 is addressed with information related to the presumed approach to a predetermined value or more, such as an approach / collision alarm, an encounter / encounter forecast, and the like. It is transmitted (from the communication interface 101).

FIG. 3 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention.

According to the application example of FIG. 3, the base station 1 remotely monitors an automobile (or a robot or the like) connected by communication with 5G. Specifically, the base station 1 is in communication connection with the terminal 2_001 having the terminal ID 001 and the terminal 2_002 having the terminal ID 002, and further, the correspondence relationship determination unit 113 of the base station 1 is
(A) The terminal 2_001 and the detected automobile 3_002 having the target ID 002 are in a corresponding relationship (that is, the terminal 2_001 is mounted on the automobile 3_002).
(B) The terminal 2_002 and the detected automobile 3_003 having the target ID of 003 are in a corresponding relationship (that is, the terminal 2_002 is mounted on the automobile 3_003).
It is decided that this is done, and a terminal target correspondence table that records that fact is generated.

On the other hand, the approach target identification unit 121 specifies the automobile 3_003 and the driver 4_001 whose target ID is 001 as one element of the approach target group, and the automobile 3_003 and the driver 4_001 are one approach target group. The approach target group information, which is the information to be included in, is output to the communication control unit 114. In this application example, the driver 4_001 is in a situation where the building standing along the road becomes an obstacle and cannot be recognized by the sensor mounted on the automobile 3_003.

In such a situation, the communication control unit 114
(A) Based on the approach target group information, the terminal 2_002 having a correspondence relationship with the automobile 3_003 included in the approach target group is determined as the approach alarm notification destination from the acquired terminal target correspondence table.
(B) The alarm information that "the vehicle 4_001 (also included in the approach target group) is approaching" is transmitted to the terminal 2_002 determined as the notification destination.

In addition, the approach target identification unit 121 also transmits the detection position information of the pedestrian 4_001 to the terminal 2_002 in accordance with this alarm information, and sets the current position (and its flow line) of the pedestrian 4_001 to the road on the screen of the terminal 2. It is also preferable to highlight it on the map.

By carrying out the processing as described above, for example, it is possible to promote safe driving by the driver and safe walking / running of a person, and to prevent a collision accident caused by a moving object such as an automobile. In particular, a dangerous situation for a certain automobile, however, a dangerous situation that cannot be detected or predicted by the in-vehicle sensor (camera) can be notified to this automobile, which is the very party concerned, via the identified terminal 2. It will be possible.

[Communication relay processing, communication relay program]
Although various embodiments related to the identification process of the terminal 2 have been described above, the communication relay in the base station 1 realized by using the functional block diagram of FIG. 1 and FIG. 4 and the identification process is described below. An embodiment of the process will be described. Note that FIG. 4 is a schematic diagram for explaining an embodiment of processing in the model building unit 131 and the terminal radio wave information estimation unit 132, which will be described later.

In the functional block diagram of FIG. 1, the model building unit 131 builds a learning model for estimating terminal radio wave information. Specifically, in the present embodiment, the model building unit 131 is as shown in FIG. 4 (A).
(A) Time-series image data (environmental information) generated by the camera 103 (which may include an object related to the terminal 2 and an object which may be an electromagnetic interference obstacle) and
(B) Learning data including information related to a predetermined target detected from this time-series image data, for example, information related to a detection position of the target (target position information).
(C) Received radio wave information (at each time point or representative time point of the environmental information in (a) above) related to the terminal 2 determined to have a correspondence relationship with this predetermined target by the correspondence relationship determination unit 113, for example, a received signal. Training data using power as correct data is generated, and the training data is used to generate machine learning algorithms well-known in the field of image recognition (for example, a plurality of convolutional layers in a neural network algorithm and their outputs). The terminal radio wave information estimation model is constructed by using the combination with the fully connected layers (Fully-Connected Layers).

Here, as already described in detail, in the conventional machine learning model as disclosed in Non-Patent Document 1, for example, an object detected from time-series environmental information generated by a camera is the machine learning model. It is not possible to determine whether or not it corresponds to the terminal related to the received signal power predicted by.

On the other hand, the terminal radio wave information estimation model constructed by the model building unit 131 is information related to the target (determined to have a corresponding relationship) that is exactly associated with the received radio wave information as correct answer data, for example, the target position. The information is included in the learning data and constructed. Therefore, the received radio wave information predicted by this terminal radio wave information estimation model, for example, the received signal power, can be interpreted as a predicted value for a predetermined target determined as the identification destination of the terminal 2 related to the received radio wave information. .. That is, the estimation process of the received radio wave information by the terminal radio wave information estimation model surely solves the unsolved problem of surely identifying the corresponding target in the estimation process by the conventional machine learning model.

Next, the terminal radio wave information estimation unit 132 estimates the received radio wave information of the terminal 2, for example, the received signal power, using the terminal radio wave information estimation model constructed by the model construction unit 131. Specifically, in the present embodiment, the terminal radio wave information estimation unit 132 has the terminal radio wave information estimation unit 132 as shown in FIG. 4 (B).
(A) Time-series image data (environmental information) generated by the camera 103, which includes the terminal 2 which is the estimation target and may include an object which may be an electromagnetic interference.
(B) The target position information of the predetermined target detected from the time-series image data and determined to have a corresponding relationship with the terminal 2 which is the estimation target is applied to the terminal radio wave information estimation model. As an input, based on the received signal power (as estimated information) output from the terminal radio wave information estimation model by this input, the predicted value of the received signal power (at the estimated time point or time interval) of the terminal 2 to be estimated is calculated. decide.

Finally, the communication control unit 114 (FIG. 1), which has acquired the predicted value of the received signal power of the terminal 2 (having high estimation accuracy as described above), uses the predicted value as a basis for the terminal 2 (or the terminal 2). It is determined whether or not to switch the communication path of (a predetermined target corresponding to). For example, when the predicted value in a predetermined future time point or time interval is equal to or less than a predetermined power threshold value, communication with the terminal 2 may be handed over to another adjacent base station 1.

Here, also in another adjacent base station 1, the predicted value of the received signal power in the same terminal 2 is determined, and the predicted value is shared among the base stations 1 to specify the largest predicted value. It is also preferable to switch the communication path to the base station 1 that has determined the specified maximum predicted value.

As described above, according to the base station 1 of the present embodiment, by using the terminal radio wave information estimation model constructed by using the terminal identification processing result, the communication failure due to the communication path obstruction which becomes a big problem in 5G. In particular, it is possible to carry out communication relay processing that can surely solve the problem of temporary communication failure due to a moving obstruction.

As described in detail above, according to the present invention, the correspondence between the terminal and the object existing in the environment in which the terminal can exist can be determined by comparing each "communication state" with each other. It will be possible. That is, it is possible to determine the terminal target correspondence information from the comparison of each "communication state" without relying on the mutual positional relationship between the terminal and the target or only the positional relationship.

Further, the terminal identification process according to the present invention solves the problem of communication failure due to the communication path obstruction in the coming 5G, and notifies / warns the vehicle equipped with the terminal of the approach of a person or another vehicle. Furthermore, it is applied in various situations and fields, such as determining the relationship between the browsing page of a user who is determined to have a corresponding relationship with a certain terminal and the flow line of the user and utilizing it for marketing and management. It is possible.

In the various embodiments of the present invention described above, various changes, modifications and omissions in the technical idea and the scope of the present invention can be easily made by those skilled in the art. The above explanation is just an example and does not attempt to restrict anything. The present invention is limited only to the scope of claims and their equivalents.

1 Base station (terminal identification device, communication relay device)
101 Communication interface 102 Communication history information storage unit 103 Camera 104 Camera image storage unit 111 Terminal communication status determination unit 111a Terminal position determination unit 112 Target communication status determination unit 112a Target position determination unit 113 Correspondence relationship determination unit 114 Communication control unit 121 Approach target Specific part 131 Model construction part 132 Terminal radio wave information estimation part 2, 2_001, 2_002 Terminal 3_002, 3_003 Automobile 4_001 Traveler

Claims (13)

Based on the information related to communication acquired from the communication means that communicates with the terminal, the terminal communication state determining means that determines the terminal communication state information, which is the information related to the communication state with the terminal, and the terminal communication state determining means.
Environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist is acquired, and environmental information that can include a predetermined target and / or a failure target that may interfere with communication is acquired, and the environmental information is obtained. Therefore, if the predetermined target includes a terminal, the target communication state determining means for determining the target communication state information, which is the information related to the communication state to be embodied with the predetermined target,
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. A terminal identification device comprising a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information relating to the correspondence between the terminal and the predetermined target. The terminal communication state determining means also determines terminal position information, which is information related to the position of the terminal, based on the information related to the communication.
The target communication state determining means also determines the target position information which is the information related to the detection position of the predetermined target detected from the environment information.
The correspondence relationship determining means calculates a position correspondence degree which is a degree of correspondence between the determined terminal position information and the determined target position information, and based on the position correspondence degree, the terminal and the predetermined The terminal identification device according to claim 1, wherein it determines whether or not there is a correspondence with the target. The terminal communication status information is information relating to the presence or absence of a communication connection with the terminal.
The target communication state determining means determines the predetermined target and the predetermined target based on the detection position of the predetermined target detected from the environmental information and the detection position of the failure target when the failure target is detected. A straight line connecting to the communication means is determined to indicate whether or not the obstacle target is located on the straight line or within a predetermined area including the straight line, or whether or not the straight line is blocked. , Calculate the information related to the possibility of communication connection with the predetermined target, and determine the information related to the possibility of communication connection as the target communication status information.
The correspondence relationship determining means is based on the communication connection correspondence degree, which is the degree of correspondence between the information relating to the presence or absence of communication connection as the terminal communication status information and the information relating to the availability of communication connection as the target communication status information. The terminal identification device according to claim 1 or 2, wherein it determines whether or not the terminal and the predetermined target have a corresponding relationship. The terminal communication status information is information related to the strength of radio waves received from the terminal.
The target communication state determining means determines the predetermined target and the predetermined target based on the detection position of the predetermined target detected from the environmental information and the detection position of the failure target when the failure target is detected. The straight line portion connecting to the communication means is determined, the degree of shielding information relating to the degree to which the obstacle target blocks the predetermined area including the straight line portion is calculated, and the target communication state information is obtained from the shielding degree information. If the predetermined target includes a terminal, determine the information related to the strength of the radio wave that will be received from the predetermined target, and determine the information.
The correspondence relationship determining means is based on the radio wave intensity correspondence degree, which is the degree of correspondence between the information related to the radio wave intensity as the terminal communication state information and the information related to the radio wave intensity as the target communication state information. The terminal identification device according to claim 1 or 2, wherein it determines whether or not the terminal and the predetermined target have a corresponding relationship. The terminal communication status information is information related to the strength of radio waves received from the terminal.
The target communication state determining means uses a learned target radio wave information estimation model that outputs information related to the strength of the radio wave received by the communication means according to the input environmental information, and uses the learned target radio wave information estimation model to obtain the said from the environmental information. If the predetermined target includes a terminal, the information related to the strength of the radio wave to be received from the predetermined target is derived, and the information related to the strength of the radio wave is determined as the target communication state information.
The correspondence relationship determining means is based on the radio wave intensity correspondence degree, which is the degree of correspondence between the information related to the radio wave strength in the terminal communication state information and the information related to the radio wave strength in the target communication state information. The terminal identification device according to claim 1 or 2, wherein it is determined whether or not the predetermined object and the predetermined object have a corresponding relationship with each other. The terminal communication status information includes information relating to the presence or absence of a communication connection with the terminal.
The target communication state determining means determines the predetermined target and the predetermined target based on the detection position of the predetermined target detected from the environmental information and the detection position of the failure target when the failure target is detected. A straight line connecting to the communication means is determined to indicate whether or not the obstacle target is located on the straight line or within a predetermined area including the straight line, or whether or not the straight line is blocked. , The information relating to the propriety of communication connection with the predetermined target is calculated, and the information relating to the propriety of communication connection is also determined as the information included in the target communication status information.
The correspondence relationship determining means has a communication connection correspondence degree, which is a degree of correspondence between the information relating to the presence or absence of a communication connection in the terminal communication status information and the information relating to the availability of a communication connection in the target communication status information, and the radio wave. The terminal and the predetermined target are based on whether or not the strength correspondence is equal to or higher than a predetermined value, or based on the sum of the communication connection correspondence and the radio wave strength correspondence or the weighted sum. The terminal identification device according to claim 4 or 5, wherein the terminal identification device determines whether or not there is a correspondence relationship. The sensors are a plurality of sensors provided at different positions from each other.
The target communication status determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target communication status information from each of the plurality of environmental information, and among the determined target communication status information. , The detection positions of the predetermined targets related to the target communication status information are close to each other, and / or the feature amount of the detection region of the predetermined target related to the target communication status information is similar to the predetermined or higher. The terminal identification device according to any one of claims 1 to 6, wherein the target communication state information thereof relates to the same predetermined target. The target communication state determining means acquires target communication state information determined from environmental information acquired from a sensor other than the sensor in the other terminal identification device from other terminal identification devices existing in the vicinity. ,
The correspondence relationship determining means also calculates the degree of correspondence between the acquired target communication state information and the determined terminal communication state information, and based on the degree, the terminal and the predetermined target have a correspondence relationship. The terminal identification device according to any one of claims 1 to 7, wherein it is determined whether or not the terminal is present in the terminal identification device. The terminal includes a plurality of terminals installed or carried on a mobile body.
The terminal identification device is
An approach target identifying means for identifying a plurality of predetermined targets that are estimated to be closer to each other later than a predetermined value based on the detected positions of the plurality of detected targets.
It is presumed that the correspondence relationship determination means approaches at least one of at least one terminal determined to have a correspondence relationship with any of the specified predetermined objects, and approaches the predetermined value or more. The terminal identification device according to any one of claims 1 to 8, further comprising a communication control means for transmitting the information according to the above. Learning data including the environmental information and information related to a predetermined target detected from the environmental information, and the terminal identification device according to any one of claims 1 to 9 can be used to obtain the predetermined target. A learning model for estimating terminal radio wave information, which is constructed by learning data in which information related to a communication state with a terminal determined to have a correspondence relationship is used as correct answer data. Data including the environmental information and information related to a predetermined target detected from the environmental information is input to the learning model according to claim 10, and the communication state of the terminal output by the learning model is related to the data. A terminal communication state estimation means that estimates information related to the communication state of the predetermined target as a terminal based on the information, and
A communication relay device comprising a communication control means for determining whether or not to switch a communication path of the predetermined target as a terminal based on information related to an estimated communication state. A program that functions a computer that can determine the correspondence between a terminal and a predetermined target.
A terminal communication state determining means that determines terminal communication status information, which is information related to a communication status with the terminal, based on information related to communication acquired from a communication means that communicates with the terminal.
Environmental information that is information related to the environment from a sensor that senses the environment in which the terminal can exist is acquired, and environmental information that can include the predetermined target and / or a failure target that may interfere with communication is acquired, and the environment is obtained. From the information, if the predetermined target includes a terminal, the target communication state determining means for determining the target communication state information, which is the information related to the communication state to be embodied with the predetermined target, and
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. A terminal identification program characterized in that a computer functions as a correspondence relationship determining means for determining whether or not a target has a correspondence relationship and determining information related to the correspondence relationship between the terminal and the predetermined target. A computer-based terminal identification method that can determine the correspondence between a terminal and a predetermined target.
Based on the information related to communication acquired from the communication means that communicates with the terminal, the terminal communication status information, which is the information related to the communication status with the terminal, is determined, while the environment in which the terminal can exist is determined. Environmental information, which is information related to the environment, is acquired from the sensing sensor and may include a predetermined target and / or a failure target that may cause a communication failure, and the predetermined target is obtained from the environmental information. A step of determining the target communication state information, which is information related to the communication state to be embodied with the predetermined target if the terminal is included, and
The degree of correspondence between the determined terminal communication status information and the determined target communication status information is calculated, and based on the degree, the terminal related to the terminal communication status information and the predetermined target communication status information are related. A terminal identification method comprising a step of determining whether or not a target has a correspondence relationship and determining information relating to a correspondence relationship between the terminal and the predetermined target.

Images