JP7258732B2 - Apparatus, program and method for identifying terminal based on correspondence between communication type and target type - Google Patents

Description

The present invention relates to technology for acquiring information about a terminal as a communication target, and particularly to technology for identifying the terminal.

The forthcoming 5th generation mobile communication system (5G) will use millimeter-wave radio waves to enable high-performance communications such as high-speed, large-capacity, low-delay, and multi-terminal connections. Here, millimeter waves have high rectilinearity and are less likely to be diffracted, and there is a high possibility that their propagation will be blocked or attenuated by an object existing between the terminal and the base station. Therefore, in 5G, the presence of such an object causes the received power to drop sharply, resulting in a significant deterioration in communication quality, which is a serious problem.

On the other hand, if it is possible to predict a drop in received power due to such an object, it is possible to maintain communication quality by controlling communication paths of terminals, for example. As a technique for predicting the received power, for example, in Non-Patent Document 1, time-series 3D environmental information acquired by an RGB-D camera capable of generating not only RGB images but also depth images is used, and machine learning is performed in communication devices. Techniques for predicting received power are disclosed.

Specifically, in this method, the time-series information of the received power measured (at the same time as the image is taken) at the terminal that received the radio waves from the transmitting station is used as the correct data for the time-series environmental information that is the result of imaging the communication environment. A machine learning model is constructed using the linked teacher data, and the predicted value of received power at the receiving terminal is determined from the time-series environmental information acquired by the RGB-D camera using this constructed machine learning model. ing.

Further, for example, Patent Document 1 discloses a technique for improving the estimation accuracy of communication quality in the initial operation stage of the system or in a communication environment with low frequency of occurrence in communication quality prediction processing by machine learning as disclosed in Non-Patent Publication 1. is disclosed. Specifically, in the virtual space formed from the measurement results of the communication environment, information is virtually generated by the RGB-D camera, and by estimating the communication quality of the receiving terminal in the virtual space by propagation simulation, the virtual space It is said that it is possible to generate a large amount of learning data above and build a machine learning model with improved estimation accuracy.

JP 2018-148297 A JP 2016-212675 A JP 2019-144941 A JP 2019-082363 A

Masashi Nishio, "Prediction and Control of Wireless Communication Quality by Machine Learning", IEICE Technical Report, Vol. 118, No. 57, SR2018-1, pp.1-8, 2018

However, in the conventional technology as disclosed in Non-Patent Document 1 described above, for example, in a normal situation in which a plurality of objects are included in time-series environmental information, among these objects, when the correct data is It is not possible to identify the terminal corresponding to the sequence received power information. As a result, it becomes very difficult to construct a machine learning model with high estimation accuracy that can be used in actual operation. That is, it is determined whether or not the object captured by the RGB-D camera (which may include the terminal) and the terminal related to the measured received power are the same object (whether or not there is a correspondence relationship). Therefore, highly reliable learning data cannot be generated in the first place.

Also, even if a machine learning model such as that disclosed in Non-Patent Document 1 is applied, for example, an object (terminal) detected from time-series environmental information generated by an RGB-D camera is detected by this machine learning model. It is completely unknown whether or not it relates to the received power predicted by the transmitting station. In the first place, it becomes impossible to even determine whether or not the predicted received power relates to any terminal.

In this regard, the technology disclosed in Patent Document 1 can determine the correspondence relationship between the terminal imaged by the RGB-D camera and the measured reception power in the virtual space. It is possible to build a machine learning model corresponding to However, even if such a model is applied, in the case of prediction of received power in actual operation, the terminal cannot be identified after all, resulting in the same problem as described in Non-Patent Document 1 described above.

Here, Patent Literature 2 discloses an object recognition system that grasps the behavior of a mobile terminal and associates the mobile terminal with a mobile object detected by a computer. In this system, each mobile terminal sequentially detects various state quantities such as acceleration acting on its own terminal, and transmits the time-series detection results to the server as terminal behavior data. Based on the signal, the behavior of moving objects present in the imaging range is monitored, and data indicating temporal changes in state quantities relating to the behavior of each moving object are stored in the server-side storage unit as estimation result history data. Next, by comparing the terminal behavior data transmitted from the mobile terminal and the estimation result history data for each mobile body, the correspondence relationship between the mobile terminal and the mobile terminal existing in the shooting range is specified.

However, in the system according to Patent Literature 2, the criterion for specifying the correspondence relationship is only the physical state quantity caused by the behavior of the mobile object, which is exactly the pending issue, in which each mobile terminal is connected to a base station. Information related to the communication performed is not utilized at all. In addition, since it is necessary for the mobile terminal to acquire information on a predetermined state that changes due to its own behavior, each mobile terminal must be equipped with equipment such as a sensor capable of detecting a predetermined state. Furthermore, since it is necessary to transmit the detection results one by one, the situation becomes technically troublesome.

Further, in Patent Document 3, in an environment where one or more persons possessing wireless receivers are staying or moving, flow line information of the person acquired using a camera or the like, A technique for associating a machine with (a person who owns the machine) is disclosed. Specifically, this technology identifies the time zone in which the person stays in a predetermined area from the flow line information of the person, and sets a radio transmitter and a receiver possessed by the person in a position corresponding to the predetermined area. A person who is likely to stay in a predetermined area during that time period is determined from the reception history information generated by communication with the person, and the person is associated with the flow line information specifying the time period. It is a thing.

However, in the technique of Patent Document 3, in order to associate a person possessing a wireless receiver with the flow line information of the person acquired by the camera, it is a major premise that the person is staying in a predetermined area. It's becoming Of course, setting a large number of predetermined areas increases the probability that the person will stay in the predetermined area, but then it becomes necessary to install a radio transmitter for each of the large number of predetermined areas.

Furthermore, in Patent Document 4, in order to estimate the position of the terminal that received the radio wave from the radio source, the radio wave attenuation of the environment is determined based on the object distribution in the environment estimated from the camera image etc., and the radio wave attenuation is reflected. A technique is disclosed for determining the location of a terminal based on an existence probability density function, using the amount of radio waves received by the terminal as a variable.

However, in the technique of Patent Document 4, basically, the correspondence between the terminal related to the received radio wave information and the objects (terminals) constituting the object distribution is based on the degree of radio wave attenuation at each position in the environment. Since it is performed based on the existence probability at each position, for example, when there are many objects in the vicinity of the determined terminal position, especially when the degree of radio wave attenuation at each object position is similar, This increases the possibility of erroneous association. Moreover, in actual operation, it is necessary to arrange a plurality of beacons within an area, and it is not realistic to apply this method to the entire general communication area.

Therefore, the present invention provides a correspondence relationship between a terminal and an object existing in an environment in which the terminal may exist, without relying on the mutual positional relationship or only on the positional relationship, and information related to communication with the terminal. It is an object to provide an apparatus, program and method that can be used to determine.

According to the invention,
communication type determination means for determining communication type information, which is information related to the type of communication with at least one terminal, based on information related to communication acquired from communication means that communicates with at least one terminal;
A target type that acquires environmental information, which is information related to the environment, from a sensor that senses the environment in which the terminal may exist, and determines target type information, which is information related to the type of at least one target detected from the environmental information. a determining means;
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target A terminal identification apparatus having a correspondence determining means for identifying a target to be identified as a terminal according to the above, and determining terminal target correspondence information in which the terminal and the identified target are associated with each other.

In the terminal identification device according to the present invention, the communication type determining means
(a) Communication method type including at least one of a communication method compatible with eMBB (enhanced Mobile BroadBand), a communication method compatible with URLLC (Ultra-Reliable and Low Latency Communications), and a communication method compatible with mMTC (massive Machine Type Communication) by selection from among a group; or
(b) By selecting from a communication data type group including at least one of voice call data, video distribution data, web page data, e-mail data, posting data, application data, device control data, and sensor data ,
It is also preferable to determine the communication type information.

Further, as an embodiment of the terminal identification device according to the present invention, at least part of the type correspondence information is information related to one-to-many, many-to-one, or many-to-many correspondence,
The correspondence determination means uses a greedy algorithm that sequentially determines correspondences from a set of terminals and objects whose correspondence is uniquely determined, or a combinatorial optimization algorithm that can solve the maximum flow problem of graph theory, to determine the terminal object. It is also preferable to determine correspondence information .

Furthermore, as another embodiment of the terminal identification device according to the present invention, the communication means obtains the target type information from the terminal holding the target type information relating to itself,
The terminal identification device uses the acquired target type information and the communication type information determined for the terminal holding the target type information, and manages the type correspondence to generate at least a part of the type correspondence information. It is also preferable to further have a part.

Further, as still another embodiment of the terminal identification device according to the present invention, the target type determining means determines that the target detected at a certain time is a new target that has not been detected or cannot be detected at the previous time. determine whether or not
For the target type information of the new target, the correspondence determining means determines the communication type information of the terminal newly connected for communication at the certain time or within a predetermined time range including the certain time based on the type correspondence information. It is also preferable to determine the terminal target correspondence information for the new target by judging only the correspondence with.

Furthermore, in the above-described embodiments related to new target determination, the target type determining means determines the position of the target detected at the certain point in time and/or the change in the detected position of the target and/or It is also preferable to determine whether this object is a new object based on the velocity information.

Further, as still another embodiment of the terminal identification device according to the present invention, the target type determining means determines whether or not the target detected at a certain point in time is the same as the target detected at the previous point in time. ,
The correspondence determination means determines that the target detected at the certain point in time is the same as the target detected at the previous point in time, and the target detected at the previous point in time corresponds to the certain terminal. When it is determined that there is a relationship, it is also preferable to determine that the object detected at the certain point in time is the identification destination of the certain terminal.

Furthermore, as still another embodiment of the terminal identification device according to the present invention, the communication type determination means determines the location and/or speed of the terminal based on information related to radio waves of the communication as information related to the communication. The terminal state information, which is related information, is also determined,
The target type determination means also determines target state information, which is information related to the position and/or speed of the target, from the detected position of the target and/or changes in the detected position,
The correspondence determination means calculates a state correspondence degree that is a degree of correspondence between the determined terminal state information and the determined target state information, and based on the state correspondence degree, determines the terminal target correspondence information. is also preferably determined.

Further, as still another embodiment of the terminal identification device according to the present invention, the communication type determination means also obtains terminal communication state information, which is information regarding the state of communication with the terminal, based on the information regarding the communication. decide and
The target type determination means also determines, from the environment information, target communication state information, which is information relating to a communication state to be realized with the target if the target includes a terminal,
The correspondence relation determination means calculates a communication state correspondence degree, which is a degree of correspondence between the determined terminal communication state information and the determined target communication state information, and based on the communication state correspondence degree, determines the terminal target It is also preferable to determine correspondence information .

Furthermore, as still another embodiment of the terminal identification device according to the present invention, the sensors are a plurality of sensors provided at different positions,
The target type determining means acquires a plurality of environmental information from each of the plurality of sensors, determines the target type information from each of the plurality of environmental information, and selects the target type from among the determined target type information. The target type information about the detected positions of the targets related to the information that are closer than a predetermined value and/or the feature amounts of the detection areas of the targets related to the target type information that are similar to each other than a predetermined value. are preferably related to the same subject.

Further, as still another embodiment of the terminal identification device according to the present invention, the target type determination means is from another terminal identification device existing in the surroundings, from a sensor different from the above sensor in the other terminal identification device Acquire the target type information determined from the acquired environment information,
The correspondence determination means may also determine the terminal target correspondence information for the target associated with the acquired target type information and the terminal associated with the determined communication type information based on the type correspondence information. preferable.

Furthermore, as still another embodiment of the terminal identification device according to the present invention, the terminal includes a plurality of terminals installed in or carried by a mobile body, and the terminal identification device comprises:
approaching target identifying means for identifying a plurality of targets that are estimated to approach each other by a predetermined amount or more later, based on the detected positions of the plurality of targets;
Information related to at least one of at least one terminal determined by the correspondence determination means to have a correspondence relationship with one of the plurality of identified targets, and information related to the fact that it is estimated that the terminal will approach the predetermined target or more. It is also preferable to further have a communication control means for transmitting.

According to the present invention, the learning data including the environment information and the information related to the object detected from the environment information, and the above-described terminal identification apparatus determines that the object is the identification destination . It is constructed by training a neural network algorithm using learning data with information related to the state of communication with the determined terminal as correct data , and environmental information as information related to explanatory variables and from the environmental information A computer is caused to function as a terminal communication state information estimating unit that, when inputting information related to a detected target, outputs information related to the communication state of the target terminal as information related to the target variable . A learning model is provided.

According to the present invention, data including the environmental information and information related to the target detected from the environmental information is input to the learning model described above, and the communication state of the terminal output by the learning model is terminal communication state estimating means for estimating information related to the communication state of the target as a terminal based on information related to
A communication relay apparatus having communication control means for determining whether or not to switch a communication path of a target terminal based on information related to an estimated communication state is provided.

According to the present invention, there is further provided a program for operating a computer capable of determining a correspondence relationship between at least one terminal and at least one object,
communication type determination means for determining communication type information, which is information related to the type of communication with the terminal, based on information related to communication acquired from the communication means that communicates with the terminal;
Target type determining means for acquiring environmental information, which is information related to the environment, from a sensor that senses the environment in which the terminal may exist, and determining target type information, which is information related to the type of the target detected from the environmental information. and,
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target A terminal identification program that causes a computer to function as correspondence determination means for identifying a target to be identified as a terminal according to the above and determining terminal target correspondence information that associates the terminal with the specified target.

According to the present invention, there is still further a terminal identification method in a computer capable of determining correspondence between at least one terminal and at least one object, comprising:
Based on the information related to communication acquired from the communication means that communicates with the terminal, communication type information, which is information related to the type of communication with the terminal, is determined, and the environment in which the terminal can exist is determined. a step of acquiring environment information, which is information related to the environment, from a sensing sensor, and determining target type information, which is information related to the type of the target detected from the environmental information;
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target and identifying a target to be identified as a terminal according to the above, and determining terminal target correspondence information that associates the terminal with the specified target.

According to the terminal identification device, program and method of the present invention, the correspondence relationship between a terminal and an object existing in an environment in which the terminal may exist can be determined without depending on the mutual positional relationship or only on the positional relationship. It is possible to make a decision using information related to communication with the terminal.

1 is a schematic diagram showing an embodiment of a terminal identification device according to the present invention; FIG. FIG. 4 is a schematic diagram showing a specific example of type correspondence information according to the present invention; FIG. 4 is a schematic diagram for explaining an embodiment of processing in a communication type determination unit, target type determination unit, and correspondence relationship determination unit according to the present invention; FIG. 10 is a schematic diagram for explaining another embodiment of processing in the communication type determination unit, target type determination unit, and correspondence relationship determination unit according to the present invention; FIG. 11 is a schematic diagram for explaining still another embodiment of processing in the communication type determining unit, target type determining unit, and correspondence relationship determining unit according to the present invention; FIG. 11 is a schematic diagram for explaining still another embodiment of processing in the communication type determining unit, target type determining unit, and correspondence relationship determining unit according to the present invention; FIG. 4 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention; FIG. 4 is a schematic diagram for explaining an embodiment of processing in a model construction unit and a terminal radio wave information estimation unit according to the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[Terminal identification device]
FIG. 1 is a schematic diagram showing one embodiment of a terminal identification device according to the present invention.

A base station 1 as a terminal identification device according to the present invention shown in FIG. 1 is a communication relay device compatible with 5G (fifth generation mobile communication system) in this embodiment, and is also a communication terminal compatible with 5G. It is a device capable of communicating with a plurality of terminals 2 . Further, in this embodiment, the base station 1 is equipped with a camera 103 which is an RGB-D camera capable of photographing the condition of the communication area with a predetermined angle of view.

The (RGB and depth) image data (video data) generated by this camera 103 includes:
(a) a person who possesses or carries the terminal 2,
(b) Mobile objects such as automobiles, motorcycles, railroad vehicles, robots, drones, etc. equipped with, including, or boarded by the terminal 2, and (c) goods, equipment, facilities, and buildings to which the terminal 2 is attached/installed , fixed objects, etc., may be included (captured), and furthermore, people, moving objects, articles, equipment, facilities, buildings, (trees, etc.) that are not related to the terminal 2 may be included (imaged). It can also include "objects" such as fixed objects (including plants of the same species).

By the way, such a "target" can determine the correspondence relationship with the terminal 2, which is the target of identification. obtain. In particular, 5G, which is the communication method of the present embodiment, uses radio waves in the millimeter wave band for communication. There is a high possibility that its propagation will be blocked or attenuated by an "object" that exists between

Therefore, in 5G, the intervention of such a "target" causes a sudden drop in the received signal power at the base station 1, and a significant deterioration in communication quality becomes a serious problem. Here, if a decrease in the received signal power at the base station 1 due to such a "target" can be predicted, it becomes possible to maintain the communication quality by performing control such as switching the communication path of the terminal 2. FIG.

Therefore, the base station 1 of this embodiment not only determines the correspondence between the terminal 2 and the "target", but also predicts the received signal power and controls the communication path as described later. It is a device that makes it possible.

Similarly in FIG. 1, the base station 1 first determines the correspondence relationship between the terminal 2 and the "target".
(A) Information related to communication (for example, communication history information ), a communication type determination unit 111 that determines "communication type information" that is information related to the type of communication with the terminal 2,
(B) Acquire "environmental information", which is information related to the environment, from a sensor (camera 103 in this embodiment) that senses the environment in which the terminal 2 may exist, and detect at least one " A target type determination unit 112 that determines "target type information" that is information related to the type of "target";
(C) the terminal 2 related to the determined "communication type information" based on the "type correspondence information" that is information that associates information related to the type of communication and information related to the type of target; and a correspondence relationship determination unit 113 that determines "terminal target correspondence information" that is information relating to the correspondence relationship with the "target" related to the "target type information".

Here, the "communication type information" in (A) above is, for example, a communication method defined in 5G (possibly defined or set for 5G).
(a) a communication method compatible with eMBB (enhanced Mobile BroadBand, high-speed large capacity);
At least one of (b) URLLC (Ultra-Reliable and Low Latency Communications, ultra-reliable and low-delay) compatible communication method, and (c) mMTC (massive Machine Type Communication, massive terminal) compatible communication method It can be information of a communication method type selected from a group of communication method types included.

Alternatively, the "communication type information" is a communication data type group including at least one of voice call data, video distribution data, web page data, e-mail data, posting data, application data, device control data, and sensor data. Information of the communication data type (or data usage type) selected from the above may be used. Furthermore, as this "communication type information", it is possible to employ information on types related to other various types of communication, for example, type information on wireless communication standards other than 5G.

On the other hand, the "target type information" in (B) above is information that specifies the type of the "target" as an object, such as "person", "automobile", or "cardboard (packed package)". In this embodiment, when detecting a "target" from camera image data as "environmental information" by a known image recognition technique, a "class" set as a detection target can be used as "target type information". can.

Further, the "type correspondence information" in (C) above may be prepared as general-purpose information common to general communication areas, but for example, it is not possible to identify terminals such as downtown areas and factory premises areas. It is also preferable that the communication type and the target type are pre-associated and set by the setter in accordance with the communication and target conditions in a predetermined area. Alternatively, as will be described later, it is also possible for the type correspondence management unit 121 (FIG. 1) to automatically generate at least part of the "type correspondence information".

As described above, according to the base station 1 as the terminal identification device according to the present invention, the terminal 2 and the "object" existing in the environment where the terminal 2 may exist (detected from the "environmental information") The correspondence can be determined based on the correspondence of the respective type information (communication type information and target type information). That is, it is possible to determine the "terminal target correspondence information" by taking into consideration the correspondence relationship of the type information relating to each without relying on the mutual positional relationship between the terminal 2 and the "target".

As will be described later, in the base station 1, it is also possible to adopt an embodiment in which "terminal target correspondence information" is determined based not only on the type information relating to the terminal 2 and the "target" but also on the mutual positional relationship. It is possible. However, in any case, the base station 1 can determine the correspondence relationship between the terminal 2 and the "target" without relying only on the mutual positional relationship.

Also, for this reason, for example, when a large number of "targets" exist (detected) in the vicinity of the terminal 2, which is the target of identification, an erroneous correspondence relationship is determined because the difference in position between them is minute. It is also possible to reduce the risk of being harassed by considering type information that is not related to location.

Although 5G is adopted as the communication method between the base station 1 and the terminal 2 in this embodiment, other communication methods such as LTE may of course be used. Communication between devices and terminals may be based on various other wireless communication standards.

For example, even in communication systems where the problem of shielding by objects is not as prominent as in 5G, there are more than a few situations in which it is desirable to accurately identify subordinate terminals using "targets" detected from "environmental information." For example, there are various needs for terminal identification, such as determining the relationship between a page viewed by a user determined to have a corresponding relationship with a certain terminal and the flow line of the user, and utilizing it for marketing and management. On the other hand, according to the terminal identification device of the present invention, it is possible to perform such terminal identification processing with high accuracy.

Moreover, the terminal identification device according to the present invention is not limited to a communication relay device such as a base station. For example, as a dedicated device for terminal identification processing, it may be provided in the form of being connected to a communication relay device/equipment such as a base station. Further, as the terminal identification device according to the present invention, it is possible to use a cloud server, a non-cloud server device, a personal computer (PC), a notebook or tablet computer, or the like, which is equipped with the terminal identification program according to the present invention. be.

Furthermore, at least one of the above (A) to (C), which are components of the terminal identification device (base station 1) according to the present invention, is included in a device separate from the other components. It is not impossible to take For example, it is possible to implement the above functions (A) to (C) by a plurality of servers as a whole. Even in such a case, the entirety of these can be regarded as a terminal identification device or system that implements the terminal identification method according to the present invention.

[Functional Configuration of Terminal Identification Device, Terminal Identification Program]
Similarly, in the functional block diagram of FIG. 1, the base station 1 includes a communication interface 101, a communication history information accumulation unit 102, a camera 103, and a camera image accumulation unit as an embodiment of a terminal identification device and a communication relay device according to the present invention. It has a unit 104 and a processor memory. Here, the processor memory stores an embodiment of the communication relay program including the terminal identification program according to the present invention, and has a computer function, and by executing this communication relay program, Terminal identification processing and communication relay processing are performed.

In addition, the processor memory includes, as functional components, a communication type determination unit 111 including a terminal state determination unit 111a and a terminal communication state determination unit 111b, and a target type determination unit including a target state determination unit 112a and a target communication state determination unit 112b. It has a unit 112 , a correspondence determination unit 113 , a communication control unit 114 , a type correspondence management unit 121 , an approaching target identification unit 131 , a model construction unit 141 , and a terminal radio wave information estimation unit 142 . These functional components can be regarded as the functions of the terminal identification program and the communication relay program stored in the processor memory. , can be understood as an embodiment of the terminal identification method and communication relay method according to the present invention.

Similarly, in the functional block diagram of FIG. acquires and records various information related to the communication performed in , generates communication history information in which the information is arranged in chronological order, and causes the communication history information storage unit 102 to store and manage the information.

Here, the communication control unit 114 naturally recognizes the communication method of communication with each terminal 2 , and in this embodiment, the communication history information is the terminal ID (identifier) of the terminal 2 for each terminal 2 . ) and the communication method (the corresponding standard) in the 5G communication implemented between the terminal 2, specifically, eMBB, URLLC and mMTC. there is

<Determination of communication type information>
The communication type determination unit 111 acquires communication history information from the communication history information accumulation unit 102, and determines the terminal ID of the terminal 2 to be identified, the communication method (corresponding standard) with this terminal 2, and the specific determines communication type information that is information associated with any one of eMBB, URLLC, and mMTC.

<Determination of terminal state information (terminal position information, terminal speed information)>
In addition, in a preferred embodiment that performs terminal identification processing in consideration of the position and/or speed of the terminal 2, which will be described later, the terminal state determination unit 111a of the communication type determination unit 111 includes a communication history information storage unit Information related to communication radio waves included in the communication history information acquired from 102 (or generated from the communication history information), for example, based on the received signal power (RSSI) of the radio signal received from the terminal 2, the terminal 2 It is also preferable to determine terminal location information, which is information related to location.

Here, not only the received signal power at this base station 1,
(a) received signal power measurement values at a plurality of base stations existing around the base station 1;
(b) The terminal position, which is the time-series information of the location of the terminal 2, is acquired by, for example, communication, the received signal power measurement value at the terminal 2 that has received radio waves from a plurality of base stations, and by a well-known base station positioning technology. Information can be determined.

The terminal state determination unit 111a can also determine terminal speed information, which is time-series information of the speed of the terminal 2, by calculating a time change in the terminal position from this terminal position information. Hereinafter, the determined terminal position information and terminal speed information are collectively referred to as terminal state information.

By the way, as a change mode, for the above terminal speed information,
(a) shift in frequency of radio waves received from terminal 2;
(b) variation per hour of received signal power (RSSI) of radio waves received from terminal 2;
(c) is also preferably determined based on the velocity of the terminal 2 derived from at least one of the time variation of the RTT in communication with the terminal 2;

Among these, for the above (a), time-series information of the deviation Δf_t from the frequency f0_t originally set for communication is generated from the time-series information of the frequency f_t of the received radio wave, and for this deviation Δf_t For example, by applying an object velocity measurement method using the Doppler effect, which is well known in the field of radio engineering, time-series information of the velocity vector v_e of the terminal 2 may be generated and used as the terminal velocity information.

Regarding the above (b), the Friis transmission formula, which is well known in the field of radio engineering, is used from the time series information of the received signal power (variable) E_t and the preset transmission signal strength from the terminal 2. Then, time-series information of the distance D_t to the terminal 2 is generated, time-series information of the velocity vector v_e of the terminal 2 is generated from the time change of this distance D_t, and this may be used as the terminal velocity information.

Furthermore, regarding the above (c), from the time-series information of RTTR_t, the time of communication distance (R_t * V / 2) is obtained by multiplying the RTTR_t by the radio wave propagation speed V and the coefficient 0.5 that makes the RTT one-way. Series information may be generated, time-series information of the velocity vector v_e of the terminal 2 may be generated from this change in communication distance over time, and this may be used as terminal velocity information.

Note that the velocity vector v_e described above may be a projection vector of the velocity vector of the terminal 2 in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2) depending on the situation or more accurately.

<Determination of terminal communication status information>
Furthermore, in a preferred embodiment that performs terminal identification processing in consideration of the communication state with the terminal 2, which will also be described later, the terminal communication state determination unit 111b of the communication type determination unit 111 uses communication history information Communication history information is acquired from the storage unit 102, and terminal communication state information, which is information relating to the state of communication with the terminal 2, is determined based on this communication history information.

Here, in this embodiment, this terminal communication state information is
(α) information regarding the presence or absence of a communication connection with the terminal 2, and (β) received signal power (RSSI) of the radio signal received from the terminal 2
Either one or both of

Regarding (α) above, the terminal communication state determination unit 111b determines whether or not the communication connection with the terminal 2 is established at each point in time. 1, a binary communication connection presence variable that takes the value -1 if not established:
(1) B_t∈{－1, 1}
may be generated.

Regarding (β) above, the terminal communication state determining unit 111b may generate time-series information of the received signal power variable E_t indicating the signal power value of the radio signal received from the terminal 2 at time t. Also, as described above, as a preferred embodiment, the terminal communication state determination unit 111b determines terminal communication state information including both the time-series information of the communication connection presence/absence variable B_t and the time-series information of the received signal power variable E_t. It is also preferred to determine

Similarly in the functional block diagram of FIG. 1, the camera 103 may be a visible light, near-infrared, or infrared photographic device having a solid-state imaging device such as a CCD image sensor or a CMOS image sensor. A camera, a stereo camera, an omnidirectional (omnidirectional) camera, or the like can also be used. Of course, instead of the camera 103, it is also possible to adopt a sensor capable of sensing the environment in which the terminal 2 may exist and generating environmental information, such as LiDAR, laser/infrared positioning device, TOF camera, and thermography device. be.

Here, in this embodiment, the camera 103 is an RGB-D camera capable of generating an RGB image and a depth image as environment information, and the generated environment information (RGB image data and depth image data) is the camera image data. It is saved and managed by the storage unit 104 .

Note that the camera 103 is installed inside the base station 1 in the present embodiment, but is installed at a position distant from the base station 1, for example, a surveillance camera in town, and communicates with the base station 1. It may be connected. Moreover, as will be described later, it is possible to adopt an embodiment in which a plurality of cameras 103 are provided at mutually different positions regardless of whether they are inside or outside the base station 1 .

<Determination of target type information>
The target type determination unit 112
(a) acquire (RGB and depth) image data (environmental information) from the camera image storage unit 104;
(b) Detect a “target” that may include the terminal 2 from the acquired image data,
(c) Determine the "class" of the detected "object", such as "person", "automobile", or "cardboard", as object type information, which is information about the type of the object.

Here, specifically, the target type information of (c) above can be information in which the target ID assigned to the detected "target" and the "class" related to the "target" are associated with each other. .

In addition, the "target" in (b) above is, for example,
(a) a person (user) possessing a mobile terminal;
(b) A vehicle equipped with a terminal 2 having a drive recorder function, a CAN information transfer function, an interface function for automatic driving control by a server, etc.
(c) Autonomous mobile robots and autonomous flying drones equipped with a terminal 2 having an interface function for autonomous mobile control by a server, and
(d) A person, mobile object, or article that may include the terminal 2 that is the communication destination related to the communication history information, such as an article attached/installed to the terminal 2 that has a sensor information transfer function. can. In some cases, equipment, facilities, buildings, fixed objects, etc. that may include the terminal 2 may also be "objects."

Furthermore, as the target detection process from the image data in (b) above, for example, a model for target detection is constructed based on a well-known machine learning algorithm (for example, an algorithm using a neural network) for various image recognition, Using the model, identify the detection area (e.g., bounding box) in the image data that is presumed to contain the "object" corresponding to the set "class", and furthermore, the probability that the detection area is the object. well-known image recognition processing such as calculating a score indicating , and determining a detection area having a score high enough to satisfy a predetermined condition as a target area. Further, in this case, the detection position of the "object" can be the representative point (for example, the center of gravity or the middle point of the lower end) of the determined object area.

Incidentally, when an RGB camera or a stereo camera is adopted as the camera 103, for example, non-patent literature: Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng -By applying the object detector described in Yang Fu, Alexander C. Berg, "SSD: single shot multibox detector", European Conference on Computer Vision, Computer Vision-ECCV 2016, pp. 21-37, 2016, It is possible to determine the region of interest and the detection position.

In addition, when LiDAR is used as a sensor according to the present invention instead of the camera 103, for example, Charles R. Qi, Hao Su, Kaichun Mo, Leonidas Using the object detector described in J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation”, Journal of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 77-85, 2016 By doing so, the target area and the detection position can be determined.

Furthermore, it is also possible to apply the technique disclosed in Japanese Patent Application Laid-Open No. 2019-174164, which estimates the position of an object (which may include a communication terminal) using a machine learning algorithm from an RGB camera image.

In addition, the target type determination unit 112 determines when no "target" is detected from the environmental information, that is, when not even one "target" is measured (sensed), or when it is measured but is not detected due to the influence of noise or the like. , it is also preferable to end the target type determination processing for the environment information.

Furthermore, in the target detection processing in the target type determination unit 112, as described above, the position of the detected target (detection position) can also be determined. Time-series information (movement history information) of the position of the target is generated after being associated with the detection area (detection position) at each point in time. This also allows each detected object to be tracked individually.

Here, for such object tracking processing, for example, a Kalman filter, which is a well-known state estimation method, is applied to the determined object area (bounding box), and the current object detection area ( bounding box), and the overlapping area of this predicted detection area (bounding box) and the currently detected detection area (bounding box) is used as an evaluation value, and the target area is determined based on the evaluation value. is also preferred.

Incidentally, such object tracking processing is described in, for example, non-patent literature: Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft, “Simple Online and Realtime Tracking”, Journal of 2016 IEEE International Conference on Image Processing (ICIP), 3464-3468, 2016.

Furthermore, although in a preferred embodiment of terminal identification processing to be described later, the target type determining unit 112 determines whether the target detected at a certain point in time t is the previous point in time, for example, the previous point in time (t−1 ), and notifies the correspondence determination unit 113 of the determination result.

Here, the target type determining unit 112 creates a list of targets (target IDs) that are detected or tracked at each point in time, and refers to this list to determine the targets detected at a certain point in time t. It can also be determined whether is a new subject. In addition, as a modification, the target is a new target based on information regarding the position and / or speed of the target, which is determined from the detected position of the target detected at a certain time t and / or the change in the detected position. , i.e., whether this object has newly entered the environment of the terminal identification object.

For example, specifically, when the distance from the boundary of the image data range (capture range of the camera 103) of the camera 103 to the position of this target is less than a predetermined threshold, this target is determined to be a new target. may or, in addition to this condition, determining that the object is a new object if the object's velocity has a component directed into the image data range in a direction perpendicular to the nearest boundary. is also preferred.

Furthermore, as another embodiment, when a plurality of sensors other than the camera 103 or the above-described camera are provided at mutually different positions regardless of whether they are inside or outside the base station 1, the target type determination unit 112
(a) acquiring a plurality of environmental information from each of the plurality of cameras 103 (or sensors);
(b) determining target type information from each of the plurality of environmental information;
(c) Of the determined target type information,
and/or
(c2) With respect to objects whose image feature amounts of the detection area (bounding box) of the target related to the target type information are similar to each other more than a predetermined amount (for example, those whose mutual distance in the image feature amount space is less than a predetermined threshold value) ,
It is also preferable that these pieces of target type information relate to the same target. Further, in this case, a representative value (for example, an average value) of the detected position group for the target related to the target type information may be treated as the position of the same target.

As a result, it is possible to avoid a situation in which a certain terminal 2 is not included in the acquired environment information, resulting in a failure of identification processing for the terminal 2 . For example, environmental information obtained by one camera 103 (or sensor) is not included in the environmental information (is not captured) because the target is located behind another target and is shielded by the other target. A few situations do occur. For example, from the viewpoint of one camera 103, it is quite possible that a person carrying the terminal 2 is blocked by a stopped bus and cannot be seen.

On the other hand, if environmental information obtained by a plurality of cameras 103 (or sensors) installed at different positions is used, objects that are not included in certain environmental information can be included in other environmental information from different sensing (measurement) viewpoints. As a result, it is sufficiently possible to avoid the above-mentioned situation of concern.

By the way, as a typical example of using target type information related to a plurality of cameras 103 (or sensors) as described above, the base station 1 detects each of one or more other base stations 1 existing in its surroundings. Therefore, the target type information generated by the camera 103 (or sensor) installed in the base station 1 is received and acquired, and the same target determination process described above is performed. In this case, it is possible for a plurality of base stations 1 to work together to more preferably carry out the terminal identification processing according to the present invention.

<Determination of target state information (target position information, target speed information)>
Furthermore, although in a preferred embodiment that performs terminal identification processing in consideration of the position and/or speed of the terminal 2, which will be described later, the target state determination unit 112a of the target type determination unit 112 From the detected position and / or change in the detected position, information related to the position and / or speed of this target, for example, time-series information of position (target position information) and / or time-series information of velocity vector (target speed information) It is also preferred to determine the target state information that is

By the way, this target velocity vector is combined with the velocity vector determined by the terminal state determination unit 111a, and the projection vector of the target velocity vector in a predetermined direction (for example, the direction connecting the base station 1 and the terminal 2) is may be determined.

<Determination of target communication state information>
Furthermore, in a preferred embodiment that performs terminal identification processing in consideration of the communication state with the terminal 2, which will also be described later, the target communication state determination unit 112b of the target type determination unit 112
(a) The (RGB and depth) image data (environmental information) from the camera image storage unit 104 includes not only a target to be identified as a terminal, but also a target that may interfere with communication (hereinafter also referred to as a target of failure). Acquire image data (environmental information) that may be included,
(b) It is also preferable to determine, from the acquired image data, target communication state information, which is information relating to a communication state that will be realized with a detected target if the target includes a terminal.

Note that in this embodiment, when a certain "object" is detected from one image data, other objects detected from the image data are treated as "obstructed objects" for the certain "object". .

Here, the target communication state information determined in (b) above is
(α′) communication connection availability information (time-series information of communication connection availability variable B′_t described later), and (β′) received radio wave intensity information (time-series information of received power equivalent variable E′_t described later)
can be either one or both of

First, in the case of (α') above, the target communication state determination unit 112b determines the detection position of the "target" detected from the acquired image data, based on the detected position of
(a) determining a straight line segment connecting the "target" and (the antenna of) the base station 1;
(b) Communication connection possibility information (shielding presence/absence information) indicating whether or not the "obstacle target" is located on this straight line segment or within a predetermined area including this straight line segment, or whether or not this straight line segment is blocked. to calculate
(c) The calculated communication connectability information can be determined as the target communication state information.

More specifically, as the communication connection propriety information of (b) above, the detection position of any of the "failure targets" is the above straight line A binary communication connection enable/disable variable that takes a value of -1 if it is located on a segment or within a rotating body region of a predetermined size that includes this straight segment as its axis of rotation, and takes a value of 1 otherwise:
(2) B'_t∈{－1, 1}
may be generated and used as the target communication state information. Here, the body of revolution region may be a cylindrical region, or may be formed by rotating a predetermined curve around a rotation axis, such as an ellipsoid.

As a modification, the target communication state determination unit 112b presets a predetermined size (cross-sectional area at the detection position) according to the type (class) of the "failure target", and selects one of the " -1 if the "obstacle target" blocks the above straight line segment with its size (cross-sectional area) at its detection position (the above straight line segment penetrates the cross section of the "obstacle target"), Otherwise, B'_t in (2) above may be defined as a communication connection enable/disable variable that takes a value of 1.

On the other hand, in the case of (β') above, the target communication state determination unit 112b determines the detection position of the "target" detected from the acquired image data, based on the detected position of
(a) determining a straight line segment connecting the "target" and (the antenna of) the base station 1;
(b) calculating shielding degree information relating to the degree to which the "obstacle target" shields a predetermined area including the straight line segment;
(c) From the calculated degree of shielding information, it is also preferable to determine, as the target communication state information, information relating to the intensity of radio waves to be received from the “target” if the “target” includes the terminal 2 . Specifically, the shielding degree information (b) above is calculated as follows.

First, a predetermined size (cross-sectional area at the detection position having a predetermined area) is set in advance for the "obstacle target" according to its type (class), and the above straight line segment is set as the rotation axis. After setting a rotating body region of a predetermined size including as
(b1) Part of the cross-sectional area of any "obstacle target" at the detection position is not included in the above rotating body area (the cross-sectional area of the obstacle target and the rotating body area have an overlapping portion) not) takes a value of 1, whereas
(b2) In any of the "obstacle targets", a part of its cross-sectional area at the detection position is included in the above rotating body area (the cross-sectional area of the obstacle target and the rotating body area have an overlapping part ), the following formula (3) R = (S _a ∩ S _b ^- )/S _a
The time-series information of the variable E′_t, which takes the R value calculated with , can be used as the degree-of-shielding information. Incidentally, the above case (b1) can also be included in the above equation (3) as a case where the R value is 1 (although it is not necessary to execute the R value calculation process).

Here, in the above equation (3), S _a is a set (for example, unit points A set of areas), and S _b is a set (for example, a set of unit point areas forming the area) representing a cross-sectional area at the detection position of the “obstacle target”. In addition, S _b ⁻ is the complement of S _b , and “/” indicates the area value of the cross-sectional area related to the set of molecules (eg, the number of unit point areas), and the area value of the cross-sectional area related to the denominator (eg, the unit This is an operator indicating division by the number of point areas).

Next, the target communication state determining unit 112b converts the time-series information of this variable E'_t (or the variable E'_t multiplied by a predetermined constant) to the "target" if the "target" includes the terminal 2. , and this information is used as target communication state information. Incidentally, the variable E'_t is hereinafter referred to as a variable equivalent to received power in order to distinguish it from the received signal power variable E_t described above.

If there are two or more "obstacle targets" that take the above R value, S _b is the area formed by the entire cross-sectional area of these "obstacle targets", that is, the union of these cross-sectional areas. It may be a region to be represented. Further, the body of revolution region may be a cylindrical region, or may be formed by rotating a predetermined curve around an axis of rotation, such as an ellipsoid.

Furthermore, as a modification, the target communication state determination unit 112b uses a learned "target radio wave information estimation model" that outputs information related to the strength of the received radio wave according to the input image data, If the "target" includes the terminal 2, the information on the strength of the radio wave to be received from the "target" is derived from the data, and the information on the strength of the radio wave is determined as the target communication state information. may

Here, a known machine learning model as disclosed in Non-Patent Document 1 can be adopted as the above-mentioned "target radio wave information estimation model". Specifically, image data at a certain point in time t (=k, k+1, ..., k+M) is input to this model, and an estimated value of the received signal power is output. As the variable E'_t value, the time-series information of this received power equivalent variable E'_t is used as the target communication state information.

Similarly, in the functional block diagram of FIG. saves and manages type correspondence information, which is information associated with .

FIG. 2 is a schematic diagram showing a specific example of type correspondence information according to the present invention. Here, in this specific example, the type correspondence information is a type correspondence table.

First, in the type correspondence table shown in FIG. 2A, the target type "car" is associated with the communication type "URLLC", and the target type "person" is associated with the communication type "eMBB". Furthermore, the target type “cardboard” is associated with the communication type “mMTC”. That is, in this case, the communication type associated with the target type is the communication method type (in 5G).

On the other hand, in the type correspondence table shown in FIG. 2B, the target type "robot" is associated with the communication type "control information communication", and the target type "person" is associated with the communication type "video distribution". , and furthermore, the target type “corrugated cardboard” is associated with the communication type “sensor information transfer”. That is, in this case, the communication type associated with the target type is the communication data type (or data usage type).

In the type correspondence table described above, one-to-one correspondence is made between target types and communication types. It is also possible to set and use an information type correspondence table.

For example, in the type correspondence table shown in FIG. 2C, the communication type "URLLC" is associated with all of the target types "automobile", "drone", and "robot", and there is a many-to-one correspondence. contains relationships.

Further, in the type correspondence table shown in FIG. 2D, two communication types "URLLC" and "eMBB" are associated with the target type "automobile", and two communication types are associated with the target type "person". Communication types “eMBB” and “mMTC” are associated, while two target types “automobile” and “person” are associated with the communication type “eMBB”, and two target types “mMTC” It can also be understood that the two target types "person" and "cardboard" are associated with each other, so that the type correspondence table shown in FIG. ing.

As described above, with regard to the specific type correspondence relationships shown in FIGS. It can be set according to the situation of the target. Depending on the situation, there may be a one-to-one correspondence between the target type and the communication type, or a one-to-many, many-to-one, or many-to-many correspondence may be made at least partly. It is possible.

For example, in a certain area subject to terminal identification, communication with robots may be implemented by both mMTC for transferring sensor information and URLLC for control information communication. In addition, there may be a situation where both humans and robots perform communication by eMBB for transmitting and receiving moving images.

Also, as a preferred modification, the type correspondence management unit 121 preferably automatically generates at least part of these type correspondence tables. for example,
(a) The terminal 2 holds target type information related to itself (for example, "cardboard" if the terminal 2 is attached to a cardboard box), and this self-held target type information is transmitted at the time of communication connection or at the base send to station 1,
(b) the communication control unit 114 acquires target type information related to the terminal 2 from such a terminal 2 via the communication interface 101;
(c) The type correspondence management unit 121 determines (by the communication type determination unit 111) the acquired target type information and the terminal 2 that holds this target type information (the transmission source of this target type information). It is also preferable to generate at least part of the type correspondence table using the communication type information.

<Determination of correspondence relationship: Use communication type information and target type information>
Returning to the functional block diagram of FIG. Then, the terminal object correspondence information (for example, a terminal object correspondence table in which the correspondence between the terminal ID and the object ID is recorded) with the object related to the decided object type information is determined.

FIG. 3 is a schematic diagram for explaining an embodiment of processing in the communication type determination unit 111, the target type determination unit 112, and the correspondence determination unit 113. As shown in FIG.

In the embodiment shown in FIG. 3, the base station 1:
(a) a terminal 2_001 having a terminal ID of 001 attached to a cardboard;
(b) a terminal 2_002 with a terminal ID of 002 mounted in a car, and (c) a terminal 2_003 with a terminal ID of 003, carried by a person.
are communicating with Incidentally, the terminal ID of each terminal is assigned to the terminal when the base station 1 establishes communication connection with the terminal.

Here, the communication type determination unit 111 acquires information related to the type of communication performed with each terminal, associates the terminal 2_001 (terminal ID=001) with the communication type "mMTC", and A communication type table (communication type information) is generated in which the communication type "URLLC" is associated with the terminal ID=002) and the communication type "eMBB" is associated with the terminal 2_003 (terminal ID=003).

On the other hand, from the image data obtained by the camera 103, the target type determining unit 112 determines the above (b) car, the above (c) person, and the above (a) cardboard box as "automobile", "person", and "cardboard", respectively. Cardboard" is detected, and 001, 002 and 003 are given as target IDs to each.

Next, the target type determination unit 112 puts together these processing results, and associates the target type "automobile" with the automobile (target ID=001) and the target type "person" with the person (target ID=002). Furthermore, a target type table (target type information) is generated in which the target type "cardboard" is associated with the cardboard (target ID=003).

Finally, the correspondence determination unit 113 uses the generated type correspondence table, which is the type correspondence information in which the target type and the communication type are associated one-to-one (as shown in FIG. 3). A terminal target correspondence table is generated from the communication type table and the target type table.

Here, specifically, the correspondence determining unit 113
(a1) read one communication type (for example, "mMTC") associated with the terminal ID (for example, terminal ID=001) of one terminal to be identified (for example, terminal 2_001) in the communication type table;
(a2) read out one target type (e.g., "cardboard") having a correspondence relationship with the one communication type (e.g., "mMTC") read out in (a1) above in the type correspondence table of FIG. 3;
(a3) read one target ID (for example, target ID=003) associated with one target type (for example, "corrugated cardboard") read in (a2) from the target type table;
(a4) The terminal ID (e.g., terminal ID=001) of one terminal (e.g., terminal 2_001) to be identified in (a1) above and one target ID (e.g., target ID=003) read in (a3) above Generate one record that is recorded in association with
(a5) Repeat steps (a1) to (a4) for each terminal to be identified (terminal 2_001, terminal 2_002, and terminal 2_003) to create a terminal target correspondence table (for example, as shown in FIG. 3). It is also preferred to generate

Incidentally, the correspondence determining unit 113 can also generate the terminal object correspondence table by the reverse procedure of the above steps (a1) to (a4). i.e.
(a1′) Read out one target type (for example, “automobile”) associated with the target ID (for example, target ID=001) of one target (for example, automobile) that is a candidate for the terminal identification destination in the target type table. ,
(a2′) read out one communication type (for example, “URLLC”) having a correspondence relationship with the one target type (for example, “automobile”) read in (a1′) in the type correspondence table of FIG. 3;
(a3') read one communication ID (for example, communication ID=002) associated with one communication type (for example, "URLLC") read in (a2') from the communication type table;
(a4′) The object ID (for example, object ID=001) of one object (for example, a car) that is a candidate for the terminal identification destination in (a1′) above, and one communication ID (for example, communication ID = 002) to generate one record that is recorded in association with
(a5') Repeat the above steps (a1') to (a4') for each target (automobile, person, cardboard) that is a candidate for terminal identification destination, and correspond to the terminal target (as shown in FIG. 3) It is also preferable to generate a table.

The processing in the case where the type correspondence table (type correspondence information) is information in which communication types and target types are associated one-to-one has been described above. In contrast to this, the processing in the case where at least part of the type correspondence table (type correspondence information) is information relating to one-to-many, many-to-one, or many-to-many correspondences will be described below.

Even in this case, basically, based on the type correspondence table, the process of investigating the correspondence between the communication type table (communication type information) and the target type table (target type information) is performed. As a suitable method, the unit 113 (FIG. 1) may generate a terminal object correspondence table by a greedy algorithm that successively determines correspondence relationships from a set of terminals 2 and objects for which correspondence is uniquely determined. preferable.

Specifically, the correspondence determination unit 113
(a) In the target type table (or communication type table), the target type (or communication type),
(b) read a communication type (or target type) having a correspondence relationship with the read target type (or communication type) in the type correspondence table;
(c) processes (a) to (c) for identifying the terminal ID associated with the read communication type (or target type) in the communication type table, particularly steps (b) and (c); It is also preferable to sequentially read out (specify) target IDs (or terminal IDs) from which reading (identification) can be performed.

As another suitable method, the correspondence determination unit 113 may generate the terminal object correspondence table by a combinatorial optimization algorithm capable of solving the maximum flow problem of graph theory. in particular,
(a) configuring a bipartite graph with each element (record) of the communication type table and each element (record) of the target type table as nodes;
(b) A maximum flow problem (source to sink Set up a graph for the problem of finding the maximum total flow rate,
(c) correspondence between each element (record) of the communication type table and each element (record) of the target type table using the Ford-Fulkerson algorithm, which is well known in the field of graph theory; Determine relationships (generate terminal target correspondence table)
is also preferred. Of course, it is also possible to generate the terminal target correspondence table using other combinatorial optimization algorithms.

FIG. 4 is a schematic diagram for explaining another embodiment of processing in the communication type determining unit 111, the target type determining unit 112, and the correspondence determining unit 113. As shown in FIG.

In the embodiment shown in FIG. 4, similarly to the embodiment in FIG. It communicates with a terminal 2_002 with an ID of 002 and (c) a terminal 2_003 with a terminal ID of 003 owned by a person.

Here, the communication type determination unit 111 acquires information related to the type of communication performed with each terminal, associates terminal 2_001 (terminal ID=001) with "mMTC", and sets terminal 2_002 (terminal ID=001) to = 002) with "URLLC" and further with terminal 2_003 (terminal ID = 003) with "eMBB" to generate a communication type table.

On the other hand, from the image data obtained by the camera 103, the target type determining unit 112 determines the above (b) car, the above (c) person, and the above (a) cardboard box as "automobile", "person", and "cardboard", respectively. Cardboard" is detected, and 001, 002 and 003 are given as target IDs to each.

Next, the object type determining unit 112 puts together these processing results, associates the automobile (object ID=001) with “automobile”, associates the person (object ID=002) with “person”, and furthermore A target type table is generated in which (target ID=003) is associated with "cardboard".

Finally, the correspondence determination unit 113 uses a type correspondence table in which the correspondence between target types and communication types (as shown in FIG. 4) is not necessarily one-to-one, and the generated communication type table and A terminal object correspondence table is generated from the object type table.

Here, specifically, the correspondence determination unit 113 first
(a1) In the type correspondence table of FIG. 4, recognize the target type "corrugated cardboard" for which the corresponding communication type is uniquely determined, and further specify the communication type "mMTC" corresponding to this, and
(a2) In the target type table, specify the target ID (target ID=003) associated with the recognized target type "cardboard",
(a3) in the communication type table, identify the terminal ID (terminal ID=001) associated with the communication type "mMTC" identified in (a1) above;
(a4) Generate a record in which the specified target ID (target ID=003) and the specified terminal ID (terminal ID=001) are associated and recorded, and make it part of the terminal target correspondence table.

Next, the correspondence determining unit 113
(b1) In the communication type table, recognizing that communication types other than "mMTC" for which correspondence destinations have already been determined are "URLLC" and "eMBB";
(b2) In the type correspondence table of FIG. 4, the target type "person" can have a correspondence relationship only with "eMBB" among "URLLC" and "eMBB" recognized in (b1) above (that is, "mMTC" has a unique correspondence),
(b3) In the target type table, identify the target ID (target ID=002) associated with the target type "person" recognized in (b2) above,
(b4) in the communication type table, identify the terminal ID (terminal ID=003) associated with the communication type "eMBB" recognized in (b2);
(b5) Generate a record in which the specified target ID (target ID=002) and the specified terminal ID (terminal ID=003) are recorded in association with each other, and make it a part of the terminal target correspondence table.

Finally, the correspondence determining unit 113
(c1) The target type "automobile" associated with the target ID (target ID=001) for which the correspondence destination has not yet been determined in the target type table, and the communication ID for which the correspondence destination has not yet been determined ( recognizing that the communication type "URLLC" associated with the communication ID = 002) has a corresponding relationship in the type correspondence table of FIG.
(c2) Generate a record in which the target ID (target ID=001) recognized in (c1) and the terminal ID (terminal ID=002) are associated and recorded to complete the terminal target correspondence table. .

Incidentally, in the above process, initially, the process starts from the step of recognizing the target type "corrugated cardboard" for which the communication type in the correspondence relationship is uniquely determined.
(a1′) In the type correspondence table of FIG. 4, the processing can also be started from the step of recognizing the communication type “URLLC” for which the corresponding target type is uniquely determined. That is, it is possible to generate the terminal object correspondence table by the reverse procedure of the above steps with respect to the object type and the communication type.

In this way, even when using a type correspondence table (type correspondence information) that includes one-to-one, many-to-one, or many-to-many correspondences, which can be sufficiently configured depending on the situation of the terminal and the target, the terminal target It is possible to generate a correspondence table (terminal object correspondence information) and facilitate the completion of terminal identification processing.

An embodiment of the terminal identification process when at least part of the type correspondence table (type correspondence information) is information relating to one-to-many, many-to-one, or many-to-many correspondences has been described above. Another embodiment of terminal identification processing using such a type correspondence table (type correspondence information) will be described below.

As described above, when at least part of it uses the type correspondence table relating to one-to-many, many-to-one, or many-to-many correspondence, a plurality of "communication type , or a plurality of "target types" correspond to the "communication type" of the terminal 2 to be identified, resulting in a possibility that the terminal identification process fails. .

On the other hand, in an environment where terminals should be identified, a plurality of targets of different types may have terminals 2 that perform the same type of communication, or a plurality of targets of the same type may perform different types of communication. A situation in which a plurality of terminals 2 are provided can occur frequently, so failure in terminal identification processing as described above must be avoided as much as possible.

Therefore, the target type determination unit 112 (FIG. 1) determines whether the target detected at a certain time t is the same as the target detected at the previous time, for example, the previous time (t-1). (determine whether or not the target is being tracked with one target ID), notify the determination result to the correspondence determining unit 113 (FIG. 1), and The decision unit 113
(a) the object detected at time t has been determined to be the same as the object detected at the previous time (e.g., time (t-1)) (subject tracked with one object ID ), and (b) when it is determined that the detected object and a certain terminal 2 are in a corresponding relationship at the previous time (for example, time (t-1)),
It is also preferable to determine that the object detected at the certain time t has a corresponding relationship with the certain terminal 2 .

As a result, for example, even if a plurality of objects of the same type are detected at a certain time, those objects that have been successfully associated with the terminal 2 in the past will be marked as being tracked and marked as the same terminal 2 as in the past. (terminal ID of) can be associated. Here, in the target type table and the communication type table, the group (record of) of the target and the terminal 2 that have been successfully associated is excluded from the reference target, and the reference target is narrowed down to facilitate the remaining matching to succeed. can also be done.

However, even with the method described above, there may be cases where the terminal identification process cannot be completed depending on the situation. For example, as described above, when a plurality of objects of the same type are detected at a certain time, none of these objects become identification destinations of the terminal 2 thereafter, and if they continue to remain in this environment, the above It becomes difficult to advance the terminal identification process only by the technique.

Therefore, (a) the target type determination unit 112 (FIG. 1) determines that the target detected at a certain time t is not detected or could be detected at a previous time (t−1), for example. determine whether it is a "new subject" that does not exist,
(b) The correspondence determination unit 113 (FIG. 1) determines the target type information of the "new target" based on the type correspondence table (type correspondence information) at time t or for a predetermined time including time t. It is also possible to adopt an embodiment in which only the correspondence with the communication type information of the terminal 2 newly connected within the range is determined, and the information related to the correspondence relationship with the terminal 2 for the "new target" is determined. preferable.

As a result, for example, in a situation where multiple objects of the same type are detected at a certain time, even if the association with these terminals 2 is not successful, a newly detected As for the target that has entered the environment, it is possible to successfully associate the target with the terminal 2 that is newly connected for communication.

FIG. 5 is a schematic diagram for explaining still another embodiment of processing in the communication type determining unit 111, the target type determining unit 112, and the correspondence determining unit 113. In FIG.

In the embodiment shown in FIG. 5, at time T, the base station 1, like the embodiment in FIG.
(a) a terminal 2_001 having a terminal ID of 001 attached to a cardboard;
(b) a terminal 2_002 with a terminal ID of 002 mounted in a car, and (c) a terminal 2_003 with a terminal ID of 003, carried by a person.
However, at the next time (T+1), in addition to these terminals,
(d) A terminal 2_004 with a terminal ID of 004 mounted in a car
are communicating with Incidentally, the terminal ID of each terminal is assigned to the terminal when the base station 1 establishes communication connection with the terminal.

Here, in this embodiment, the same type correspondence table as in the embodiment of FIG. 3 is used. Therefore, at time T, the communication type determination unit 111 and the target type determination unit 112 respectively generate a communication type table and a target type table similar to those shown in FIG. A terminal object correspondence table similar to that shown in 3 is generated.

Next, at the subsequent time (T+1), the communication type determination unit 111 creates a record in which the terminal ID (terminal ID=004) of the terminal 2_004 newly connected for communication and the communication type "URLLC" are associated with each other. , is added to the already generated communication type table, and a new flag is given to this record. On the other hand, the target type determination unit 112 adds a record that associates the target ID of the newly detected target (target ID=004) with the target type "automobile" to the already generated target type table, Similarly, a new flag is given to this record.

Next, the correspondence determination unit 113
(a1) reading out the communication type "URLLC" associated with the terminal ID (terminal ID=004) of the terminal 2_004 to which the new flag is added in the communication type table;
(a2) read out the target type "automobile" having a corresponding relationship with the communication type "URLLC" read out in (a1) above in the type correspondence table of FIG. 5;
(a3) In the target type table, read the target ID (target ID=004) of the target (record) associated with the target type "automobile" read in (a2) above and to which the new flag is added. ,
(a4) Generate a record in which the terminal ID (terminal ID=004) of the terminal 2_004 to be identified in (a1) above is associated with the target ID (target ID=004) read out in (a3) above. Then, it is added to the already generated terminal object correspondence table (the terminal object correspondence table is updated).

Incidentally, the correspondence determination unit 113 can also generate the terminal object correspondence table by a procedure that is exactly the opposite of the above steps. i.e.
(a1′) In the target type table, it is also possible to start from the step of reading out the target type “automobile” associated with the target ID (target ID=004) of the target terminal identification destination candidate to which the new flag is added. It is possible.

<Determination of correspondence: Terminal state information and target state information are also used>
Various embodiments of terminal identification processing based on communication type information and target type information have been described above with reference to FIGS. An embodiment that preferably implements the identification process will be described.

As already described in detail, the terminal state determination unit 111a (FIG. 1) of the communication type determination unit 111 receives "terminal state information" which is information related to the position and/or speed of the terminal 2, for example, the position of the terminal 2. Time-series information (terminal position information) and time-series information of the velocity vector of the terminal 2 (terminal velocity information) can be determined. On the other hand, the target state determination unit 112a (FIG. 1) of the target type determination unit 112 selects "target state information" which is information related to the position and/or speed of the target, for example, time-series information of the target position (target position information). and time-series information of the target velocity vector (target velocity information) can be determined.

Next, the correspondence determination unit 113 (FIG. 1) calculates a "degree of state correspondence" that is the degree of correspondence between the determined "terminal state information" and "target state information". , it is also preferable to generate the terminal object correspondence information (terminal object correspondence table).

Here, the location correspondence degree LC as the “state correspondence degree” between the terminal location information (time-series information on the location of the terminal 2) and the target location information (time-series information on the location of the target) is, for example, A representative value (for example, an average value) of the reciprocal of the absolute value (distance) of the difference between the position of the terminal 2 and the target position at each point in time can be set as the representative value (for example, the average value) in the predetermined time interval.

Further, the speed correspondence degree VC as the "state correspondence degree" between the terminal speed information (time-series information of the speed vector of the terminal 2) and the target speed information (time-series information of the target speed vector) is, for example, a predetermined time interval can be a representative value (for example, average value) of the reciprocal of the magnitude of the difference between the velocity vector of the terminal 2 and the target velocity vector at each point in time in the predetermined time interval.

Here, the correspondence determining unit 113
(a) (It is not possible to make a unique correspondence only with the terminal object correspondence table), but according to the terminal object correspondence table, there is a possibility that there is a correspondence relationship (in the table, the corresponding communication type and the corresponding target type corresponding) terminal 2 and target set,
(b) "State correspondence" (location correspondence LC and/or speed correspondence VC) exceeds a predetermined threshold (position correspondence threshold LC_th and/or speed correspondence threshold VC_th) ((LC>LC_th) and / or (VC > VC_th)) pairs are in correspondence with each other,
(c) It is also preferable to add the record of the set to the terminal object correspondence table.

FIG. 6 is a schematic diagram for explaining still another embodiment of processing in the communication type determining unit 111, the target type determining unit 112, and the correspondence determining unit 113. In FIG.

In the embodiment shown in FIG. 6, the base station 1, similar to the situation at time (T+1) in the embodiment of FIG.
(a) a terminal 2_001 having a terminal ID of 001 attached to a cardboard;
(b) a terminal 2_002 with a terminal ID of 002, which is mounted in a car;
(c) a terminal 2_003 with a terminal ID of 003 carried by a person; and (d) a terminal 2_004 with a terminal ID of 004 mounted in a car.
are communicating with

Here, the communication type determination unit 111 obtains and determines the communication type of communication being performed with each terminal and the terminal position (terminal state information) of the terminal, and determines terminal 2_001 (terminal ID=001). ) is associated with the communication type “mMTC” and the terminal position (100, 200, 50) (x, y and z coordinate values in the xyz space coordinate system, respectively), and the communication type “URLLC” is associated with the terminal 2_002 (terminal ID = 002). Associate the terminal location (50, 150, 100), associate the terminal 2_003 (terminal ID = 003) with the communication type "eMBB" and the terminal location (100, 230, 130), and further associate the terminal 2_004 (terminal ID = 004) with the communication type "eMBB" and the terminal position (200, 150, 100).

On the other hand, from the image data obtained by the camera 103, the target type determination unit 112 classifies the vehicle (b), the person (c), the cardboard (a), and the vehicle (d) as "vehicle , "person", "cardboard", and "automobile", and assign 001, 002, 003, and 004 as target IDs to each. decide.

Furthermore, the target type determination unit 112 puts together these processing results, associates the target type “automobile” and the target position (50, 150, 100) with the automobile (object ID=001), and associates the person (object ID=002) with the object type “automobile” Associate the target type "person" with the target position (100,230,130), map the cardboard (target ID = 003) with the target type "cardboard" and the target position (100,200,50), and furthermore, the car (target ID = 004) generates a target type table in which the target type "car" is associated with the target position (200, 150, 100).

Here, the correspondence determining unit 113 first
(a1) In the type correspondence table of FIG. 6, recognize the target type "corrugated cardboard" for which the corresponding communication type is uniquely determined, and further specify the communication type "mMTC" corresponding to this, and
(a2) In the target type table, specify the target ID (target ID=003) associated with the recognized target type "cardboard",
(a3) in the communication type table, identify the terminal ID (terminal ID=001) associated with the communication type "mMTC" identified in (a1) above;
(a4) Generate a record in which the specified target ID (target ID=003) and the specified terminal ID (terminal ID=001) are associated and recorded, and make it part of the terminal target correspondence table.

Next, the correspondence determination unit 113 recognizes that the rest of the correspondence cannot be performed depending on the type correspondence table of FIG.
(b1) The reciprocal of the distance (position correspondence degree: in this case, the calculated value is infinite and the maximum value that the position correspondence degree variable can take on the program) exceeds a predetermined threshold,
(b2) Generate a record in which the target ID (target ID = 001) recognized in (b1) and the terminal ID (terminal ID = 002) are associated and recorded, and use it as a part of the terminal target correspondence table. do.

Next, the correspondence determining unit 113, after recognizing that the remaining correspondence cannot be performed depending on the type correspondence table of FIG. 6,
(c1) The reciprocal of the distance (position correspondence degree: also in this case, the calculated value is infinite and the maximum value that the position correspondence degree variable can take on the program) exceeds a predetermined threshold,
(c2) Generate a record in which the target ID (target ID = 004) recognized in (c1) and the terminal ID (terminal ID = 004) are associated and recorded, and use it as a part of the terminal target correspondence table. do.

Finally, the correspondence determining unit 113
(d1) The target type "person" associated with the target ID (target ID=002) whose correspondence destination has not yet been determined in the target type table, and the communication ID (whose correspondence destination has not yet been determined in the communication type table) recognizing that the communication type "eMBB" associated with the communication ID = 003) has a corresponding relationship in the type correspondence table of FIG. 6,
(d2) Generate a record in which the target ID (target ID=002) recognized in (d1) and the terminal ID (terminal ID=003) are associated and recorded to complete the terminal target correspondence table. .

Incidentally, the correspondence determination unit 113, instead of (d1) above,
(d1′) The reciprocal of the distance (position Correspondence: In this case as well, the calculated value is infinite and the maximum possible value of the position correspondence variable in terms of the program) exceeds a predetermined threshold. be.

In the above process, the process initially starts from the step of recognizing the target type "corrugated cardboard" for which the communication type in the correspondence relationship is uniquely determined.
(a1′) In the type correspondence table of FIG. 6, the processing can also be started from the step of recognizing the communication type “mMTC” for which the target type in correspondence is uniquely determined. That is, it is possible to generate the terminal object correspondence table by the reverse procedure of the above steps with respect to the object type and the communication type.

In this way, when using a type correspondence table (type correspondence information) containing one-to-one, many-to-one, or many-to-many correspondences, which can be sufficiently configured depending on the situation of the terminal and the target, the communication type and Even in a situation where the terminal identification process cannot be completed only by investigating the correspondence between the target types, the identification process of as many terminals 2 as possible is performed by using the target state information and the terminal state information, and the terminal identification process is performed. can be made easier to complete.

<Determination of correspondence relationship: Terminal communication state information and target communication state information are also used>
Various embodiments of terminal identification processing using terminal state information and target state information have been described above with reference to FIG. , an embodiment that preferably implements the terminal identification process will be described.

As already explained in detail, the terminal communication state determining unit 111b (FIG. 1) of the communication type determining unit 111 determines "terminal communication state information", for example, time-series information of the communication connection presence/absence variable B_t and/or received signal power variable E_t , can be determined. On the other hand, the target communication state determination unit 112b (FIG. 1) of the target type determination unit 112 determines “target communication state information”, for example, time-series information of the communication connection availability variable B′_t and/or received power equivalent variable E′_t. Time series information, can be determined.

Next, the correspondence determination unit 113 (FIG. 1) calculates a "communication state correspondence degree" that is the degree of correspondence between the determined "terminal communication state information" and "target communication state information", and calculates the "communication state correspondence degree". It is also preferable to generate the terminal object correspondence information (terminal object correspondence table) based also on the "communication state correspondence degree".

Here, communication connection compatibility as "communication status compatibility" between the time-series information (terminal communication state information) of the communication connection presence/absence variable B_t and the time-series information (target communication state information) of the communication connection availability variable B'_t CC is, for example, the cross-correlation between the communication connection presence/absence variable B_t and the communication connection availability variable B'_t at each time t in a predetermined time interval (time interval t = k to k+M), and the following equation (4) CC = Σ _{t = k} ^{k + M} B_t * B'_t
may be calculated using

In addition, the received power correspondence degree PC as the "communication state correspondence degree" between the time-series information (terminal communication state information) of the received signal power variable E_t and the time-series information (target communication state information) of the received power equivalent variable E'_t is, for example, the cross-correlation between the received signal power variable E_t at each point in time t in a predetermined time interval (time interval from t = k to k+M) and the time-series information of the received power equivalent variable E'_t, given by the following equation ( 5) PC = Σ _{t = k} ^{k + M} E_t * E'_t
may be calculated using Here, in the above equations (4) and (5), Σ _{t = k} ^{k + M} is an operator indicating the summation for time t (= k, k + 1, ..., k + M) .

Correspondence determination unit 113 then
(a) (It is not possible to make a unique correspondence only with the terminal object correspondence table), but according to the terminal object correspondence table, there is a possibility that there is a correspondence relationship (in the table, the corresponding communication type and the corresponding target type corresponding) terminal 2 and target set,
(b) "Communication state compatibility" (communication connection compatibility CC and/or received power compatibility PC) exceeds a predetermined threshold (communication connection compatibility threshold CC_th and/or received power compatibility threshold PC_th) (( CC>CC_th) and/or (PC>PC_th)) pairs are in correspondence with each other,
(c) It is also preferable to add the record of the set to the terminal object correspondence table.

In this way, when using a type correspondence table (type correspondence information) containing one-to-one, many-to-one, or many-to-many correspondences, which can be sufficiently configured depending on the situation of the terminal and the target, the communication type and Even in a situation where the terminal identification process cannot be completed only by examining the correspondence between the target types, the target communication state information and the terminal communication state information are used to carry out the identification process for as many terminals 2 as possible. It is possible to make it easier to complete the identification process.

Various embodiments of terminal identification processing have been described above. Here, an embodiment in which a plurality of base stations 1 cooperate to perform terminal identification processing will be described. In this embodiment, at least one base station 1 exists around the base station 1, and each base station 1 determines the target type information from the environment information obtained by its own camera 103, and further determines the target type information by itself. It is possible to mutually exchange the target type information.

Here, the target type determination unit 112 (FIG. 1) acquires the target type information determined by the surrounding base stations 1, and furthermore, the correspondence relationship determination unit 113 (FIG. 1) acquires the targets thus acquired. It is also preferable to determine the correspondence relationship between the type information and the communication type information determined (by its own base station 1), generate and update the terminal target correspondence table, and perform terminal identification processing.

In this case, it is possible to perform terminal identification processing similar to the terminal identification processing performed in the embodiment in which the base station 1 is connected to a plurality of cameras 103, as already described. Further, as a result, a certain terminal 2 is not included in the environment information obtained by its own camera 103, and is therefore not detected, and as a result, the base station 1 fails in identifying the terminal 2. It is possible to avoid the situation.

Furthermore, in the various embodiments described above, the "type correspondence information" is set and used as a type correspondence table, but this "type correspondence information" is set without taking the form of an individual table. Embodiments are also possible.

For example, as already explained, in an embodiment in which the terminal 2 holds its own target type information, and the base station 1 acquires the target type information from the terminal 2,
(a) The communication type determination unit 111 adds the acquired target type information to the communication type information determined for the terminal 2 to be identified (specifically, the terminal ID of this terminal 2 and the communication type , associated with the target type of the acquired target type information),
(b) The correspondence determination unit 113 identifies, in the target type information determined by the target type determination unit 112, a target ID corresponding to the same target type as the target type of the acquired target type information, Terminal target correspondence information may be generated in which the specified target ID and the terminal ID of (a) are associated with each other.

By the way, in this case, it can be understood that (the part functioning as) "type correspondence information" is automatically generated in the communication type information added to the acquired target type information.

[Application example of terminal identification processing]
An application example of the terminal identification processing described above will be described below.

In the functional block diagram of FIG. 1, the approaching target identification unit 131 detects the positions of the plurality of targets detected by the target type determination unit 112 in a situation where there are a plurality of mobile terminals 2 in the communication area environment. Based on this, a plurality of targets that are estimated to approach each other by a predetermined amount or more later are specified.

For example, the approaching target identification unit 131 may determine the position of each detected target at the present point in time and the velocity vector calculated from the past time change of the detected position, and determine the position of each detected target at a future point in time after the lapse of a predetermined time. It is also preferable to estimate the positions of the objects and specify the objects whose mutual distance is equal to or less than a predetermined threshold value at the future time point as the approaching object group.

In this case, the communication control unit 114 receives the information of the plurality of targets identified by the approaching target identification unit 131, and the correspondence determination unit 113 determines that there is a correspondence relationship with one of these plurality of targets. Identify (at least one) terminal 2, and send information, such as an approach/collision alarm, encounter/encounter forecast, etc., to at least one of the identified terminals 2 that is estimated to approach more than a predetermined amount. It is transmitted (from the communication interface 101).

FIG. 7 is a schematic diagram for explaining an application example of the terminal identification method according to the present invention.

According to the application example of FIG. 7 , the base station 1 remotely monitors automobiles (or drones, robots, etc.) that are communicatively connected by 5G. Specifically, the base station 1 communicates with a terminal 2_001 whose terminal ID is 001 and a terminal 2_002 whose terminal ID is 002.
(a) The terminal 2_001 and the detected automobile 3_002 whose target ID is 002 are in correspondence (that is, the terminal 2_001 is mounted on the automobile 3_002),
(b) The terminal 2_002 and the detected car 3_003 whose target ID is 003 are in correspondence (that is, the terminal 2_002 is mounted on the car 3_003),
A terminal object correspondence table that records this fact is generated.

On the other hand, the approaching object identification unit 131 identifies the vehicle 3_003 and the driver 4_001 whose object ID is 001 as elements of one approaching object group, and the vehicle 3_003 and the vehicle 4_001 are included in one approaching object group. to the communication control unit 114. In this application example, the driver 4_001 cannot be recognized by the sensors mounted on the car 3_003 because the building standing along the road is an obstacle.

In such a situation, the communication control unit 114
(a) Based on this approaching target group information, the terminal 2_002 having a correspondence relationship with the car 3_003 included in the approaching target group is determined as the approach alarm notification destination from the obtained terminal target correspondence table,
(b) Alarm information to the effect that "a rider 4_001 (also included in the approaching target group) is approaching" is transmitted to the terminal 2_002 determined as the notification destination.

In addition, the approaching object identification unit 131 also transmits the detected position information of the rider 4_001 to the terminal 2_002 in accordance with this alarm information, and the current position (and its flow line) of the rider 4_001 is displayed on the screen of the terminal 2. It is also preferred to have it highlighted on the map.

By performing the processing described above, for example, it is possible to promote safe driving by drivers and safe walking/running of people, and to prevent collision accidents caused by moving bodies such as automobiles. In particular, it is also possible to notify the car, which is the party concerned, through the identified terminal 2 of a dangerous situation for a certain car that cannot be detected or predicted by the onboard sensor (camera). It becomes possible.

[Communication relay processing, communication relay program]
Various embodiments related to the identification process of the terminal 2 have been described above. Hereinafter, with reference to the functional block diagram of FIG. 1 and FIG. One embodiment of a process is described. Note that FIG. 8 is a schematic diagram for explaining an embodiment of the processing in the model construction unit 141 and the terminal radio wave information estimation unit 142, which will be described later.

In the functional block diagram of FIG. 1, the model construction unit 141 constructs a learning model for estimating terminal radio wave information. Specifically, in this embodiment, the model construction unit 141, as shown in FIG. 8A,
(a) time-series image data (environmental information) generated by the camera 103 (which may include an object related to the terminal 2 or an object that may become a radio wave obstruction);
(b) learning data including information on a predetermined target detected from the time-series image data, for example, information on the detected position of the target (target position information),
(c) received radio wave information (at each point in time or representative time of the environmental information in (a) above), for example, a received signal, related to the terminal 2 determined to have a corresponding relationship with this predetermined target in the correspondence determining unit 113 Generate learning data with electric power as correct data, and with the learning data, a machine learning algorithm known in the field of image recognition (for example, a neural network algorithm with multiple convolutional layers) and their outputs (Combination with Fully-Connected Layers) to build a terminal radio wave information estimation model.

Here, as already described in detail, in the conventional machine learning model disclosed in Non-Patent Document 1, for example, an object detected from time-series environmental information generated by a camera is detected by this machine learning model. It is not possible to determine whether or not this corresponds to the terminal associated with the predicted received signal power.

On the other hand, the terminal radio wave information estimation model constructed by the model construction unit 141 is information related to a target that is exactly associated (determined to have a correspondence relationship) with the received radio wave information as correct data, such as the target position It is constructed by including information in the learning data. Therefore, the received radio wave information predicted by this terminal radio wave information estimation model, for example, the received signal power, can be interpreted as a predicted value for a predetermined target determined as the identification destination of the terminal 2 related to the received radio wave information. . In other words, the reception radio wave information estimation process based on the present terminal radio wave information estimation model reliably solves the unresolved problem of reliable identification of the correspondence target in the estimation process based on the conventional machine learning model.

Next, the terminal radio wave information estimation unit 142 uses the terminal radio wave information estimation model constructed by the model construction unit 141 to estimate received radio wave information of the terminal 2, such as received signal power. Specifically, in this embodiment, the terminal radio wave information estimation unit 142, as shown in FIG. 8B,
(a) time-series image data (environmental information) generated by the camera 103, which includes the terminal 2 to be estimated and may include an object that may be a radio wave obstruction;
(b) applying target location information of a predetermined target detected from the time-series image data and determined to have a corresponding relationship with the terminal 2, which is an estimation target, to the terminal radio wave information estimation model; As an input, based on the received signal power (as estimated information) output from the terminal radio wave information estimation model due to this input, the predicted value of the received signal power (at the estimation time point or time interval) of the terminal 2 to be estimated is decide.

Finally, the communication control unit 114 (FIG. 1), which has obtained the predicted value of the received signal power of the terminal 2 (having high estimation accuracy as described above), based on this predicted value, the terminal 2 (or the terminal 2 and (predetermined target corresponding to) to determine whether or not to switch the communication path. For example, if the predicted value at a given future point in time or time interval is below a given power threshold, communication with this terminal 2 may be passed to another neighboring base station 1 .

Here, in other neighboring base stations 1, the predicted value of the received signal power in the same terminal 2 is also determined, and the predicted value is shared among the base stations 1 to specify the largest predicted value. , it is also preferable to switch the communication path to the base station 1 that has determined the specified maximum predicted value.

As described above, according to the base station 1 of the present embodiment, by using the terminal radio wave information estimation model constructed using the terminal identification processing result, communication failure due to communication path obstructions, which is a major problem in 5G, can be prevented. Therefore, it is possible to perform communication relay processing that can reliably solve the problem of (1), especially the problem of temporary communication failure due to a moving shield.

As described in detail above, according to the present invention, the correspondence between a terminal and an object existing in an environment in which the terminal may exist is determined based on the correspondence of the type information (communication type and target type) associated with each. It becomes possible to determine That is, the terminal object correspondence information can be determined by considering the correspondence relationship of the type information relating to each without depending on the mutual positional relationship between the terminal and the object or only on the positional relationship.

In addition, such terminal identification processing according to the present invention solves the problem of communication failure due to communication path obstructions in the upcoming 5G, and notifies/warns an automobile equipped with a terminal of the approach of a person or other vehicle. Furthermore, it is applied in various situations and fields, such as determining the relationship between the user's browsing page determined to be in correspondence with a certain terminal and the user's flow line, and utilizing it for marketing and management. It is possible.

In the various embodiments of the present invention described above, various changes, modifications and omissions within the scope of the technical ideas and aspects of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 base station (terminal identification device, communication relay device)
101 communication interface 102 communication history information accumulation unit 103 camera 104 camera image accumulation unit 111 communication type determination unit 111a terminal state determination unit 111b terminal communication state determination unit 112 target type determination unit 112a target state determination unit 112b target communication state determination unit 113 correspondence Relationship determination unit 114 Communication control unit 121 Type correspondence relationship management unit 131 Approaching target identification unit 141 Model construction unit 142 Terminal radio wave information estimation unit 2, 2_001, 2_002, 2_003, 2_004 Terminal 3_002, 3_003 Automobile 4_001 Driver

Claims (16)

communication type determination means for determining communication type information, which is information related to the type of communication with at least one terminal, based on information related to communication acquired from communication means that communicates with at least one terminal;
A target type that acquires environmental information, which is information related to the environment, from a sensor that senses the environment in which the terminal may exist, and determines target type information, which is information related to the type of at least one target detected from the environmental information. a determining means;
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target a terminal identification device for determining correspondence relationship determining means for identifying a target to be identified as a terminal according to claim 1, and determining terminal target correspondence information in which the terminal is associated with the specified target. The communication type determining means selects at least one of a communication method compatible with eMBB (enhanced Mobile BroadBand), a communication method compatible with URLLC (Ultra-Reliable and Low Latency Communications), and a communication method compatible with mMTC (massive Machine Type Communication). Or at least one of voice call data, video distribution data, web page data, e-mail data, posting data, application data, device control data, and sensor data 2. The terminal identification device according to claim 1, wherein said communication type information is determined by selection from a group of communication data types included. At least part of the type correspondence information is information related to one-to-many, many-to-one, or many-to-many correspondence,
The correspondence determination means uses a greedy algorithm that sequentially determines correspondences from a set of terminals and objects for which correspondences are uniquely determined, or a combinatorial optimization algorithm that can solve the maximum flow problem of graph theory. 3. The terminal identification device according to claim 1, wherein object correspondence information is determined. The communication means acquires the target type information from the terminal holding the target type information related to itself,
The terminal identification device uses the acquired target type information and the communication type information determined for the terminal holding the target type information to manage type correspondence to generate at least part of the type correspondence information. 4. The terminal identification device according to any one of claims 1 to 3, further comprising a unit. The target type determination means determines whether or not the target detected at a certain time is a new target that has not been detected or cannot be detected at the previous time,
For the target type information of the new target, the correspondence determining means determines, based on the type correspondence information, the communication type of the terminal newly connected for communication at the certain point in time or within a predetermined time range including the certain point in time. 5. The terminal identification device according to any one of claims 1 to 4, wherein said terminal object correspondence information for said new object is determined by judging only correspondence with information. The target type determination means determines whether the target is new based on information related to the position and/or speed of the target, which is determined from the detected position of the target detected at the certain time and/or the change in the detected position. 6. The terminal identification device according to claim 5, wherein it is determined whether or not the terminal is a target of the terminal identification. The target type determination means determines whether or not the target detected at a certain time is the same as the target detected at the previous time,
The correspondence determining means determines that the target detected at the certain point in time is the same as the target detected at the previous point in time, and the target detected at the previous point in time and the certain terminal 7. The apparatus according to any one of claims 1 to 6, wherein when it is determined that there is a corresponding relationship, the object detected at the certain point in time is determined to be the identification destination of the certain terminal. terminal identification device. The communication type determination means also determines terminal state information, which is information related to the position and/or speed of the terminal, based on information related to radio waves of the communication as information related to the communication,
The target type determination means also determines target state information, which is information related to the position and/or speed of the target, from the detected position of the target and/or changes in the detected position,
The correspondence determining means calculates a state correspondence degree, which is a degree of correspondence between the determined terminal state information and the determined target state information, and calculates the terminal target correspondence information based on the state correspondence degree as well. 8. The terminal identification device according to any one of claims 1 to 7, wherein the terminal identification device determines information. The communication type determination means also determines terminal communication state information, which is information related to the state of communication with the terminal, based on the information related to the communication,
The target type determination means also determines, from the environment information, target communication state information, which is information relating to a communication state that will be realized with the target if the target includes a terminal,
The correspondence determination means calculates a communication state correspondence degree, which is a degree of correspondence between the determined terminal communication state information and the determined target communication state information, and based on the communication state correspondence degree, determines whether the terminal 9. The terminal identification device according to any one of claims 1 to 8, wherein the terminal identification device determines object correspondence information . The sensor is a plurality of sensors provided at different positions,
The target type determining means obtains a plurality of environmental information from each of the plurality of sensors, determines the target type information from each of the plurality of environmental information, and determines the target type information among the determined target type information. Regarding the detection positions of the targets related to the type information that are closer than a predetermined distance, and/or the feature amounts of the detection areas of the targets related to the target type information that are similar to each other than a predetermined amount, the target types 10. The terminal identification device according to any one of claims 1 to 9, wherein the information relates to the same object. The target type determination means acquires target type information determined from environment information acquired from a sensor other than the sensor in the other terminal identification device from another terminal identification device existing in the vicinity,
The correspondence determination means also determines the terminal target correspondence information for the target related to the acquired target type information and the terminal related to the determined communication type information based on the type correspondence information. The terminal identification device according to any one of claims 1 to 10, characterized by: The terminal includes a plurality of terminals installed or carried on a mobile body,
The terminal identification device is
approaching target identifying means for identifying a plurality of targets that are estimated to approach each other by a predetermined amount or more later, based on the detected positions of the plurality of targets;
At least one of at least one terminal determined to have a corresponding relationship with one of the plurality of specified targets by the correspondence relationship determination means is estimated to approach at least the predetermined target 12. The terminal identification device according to claim 1, further comprising communication control means for transmitting information. Learning data including the environmental information and information related to a target detected from the environmental information, wherein the target is identified by the terminal identification device according to any one of claims 1 to 12. It is constructed by training a neural network algorithm using learning data with information related to the state of communication between terminals determined as correct data, and environmental information as information related to explanatory variables The computer functions as a terminal communication state information estimating unit that outputs information on the communication state of the target terminal as information on the target variable when information on the detected target is input from the learning model to Data including the environmental information and information about the target detected from the environmental information is input to the learning model according to claim 13, and the information about the communication state of the terminal output by the learning model is used. terminal communication state estimation means for estimating information related to the communication state of the target as a terminal based on
and communication control means for determining whether or not to switch the communication path of the target as a terminal based on information related to the estimated communication state. A program for operating a computer capable of determining a correspondence relationship between at least one terminal and at least one object,
communication type determination means for determining communication type information, which is information related to the type of communication with the terminal, based on information related to communication acquired from the communication means that communicates with the terminal;
Target type determining means for acquiring environmental information, which is information related to the environment, from a sensor that senses the environment in which the terminal may exist, and determining target type information, which is information related to the type of the target detected from the environmental information. and,
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target A terminal identification program , characterized in that it causes a computer to function as correspondence determining means for identifying a target to be identified as a terminal according to the above, and determining terminal target correspondence information that associates the terminal with the identified target. A terminal identification method in a computer capable of determining a correspondence relationship between at least one terminal and at least one object,
Based on the information related to communication acquired from the communication means that communicates with the terminal, communication type information, which is information related to the type of communication with the terminal, is determined, and the environment in which the terminal can exist is determined. a step of acquiring environment information, which is information related to the environment, from a sensing sensor, and determining target type information, which is information related to the type of the target detected from the environmental information;
Determined communication type information that is the target related to the determined target type information based on the type correspondence information that is information that associates the information regarding the type of communication with the information regarding the type of the target identifying a target to be identified for a terminal according to the above, and determining terminal target correspondence information in which the terminal and the identified target are associated with each other.

Images