JP7284881B1 - Mobile communication system and communication method - Google Patents

Abstract

A multi-profile SIM automatically switches profiles according to the communication status of a mobile phone network.
A mobile communication system includes a user terminal equipped with a multi-profile SIM, a plurality of mobile operator networks with which the user terminal can communicate, and a SIM profile switching device connected to the user terminal and the plurality of mobile operator networks. And prepare. The SIM profile switching device comprises a reward determination unit that determines a reward in reinforcement learning based on the communication status of each of multiple mobile operator networks. The user terminal uses a learning model configured to learn to select an optimal SIM profile by reinforcement learning based on the state of the user terminal and a reward; and a SIM profile selection unit that selects the
[Selection drawing] Fig. 7

Description

The present invention relates to mobile communication systems and communication methods. In particular, the present invention relates to profile switching in multi-profile SIMs.

A multi-profile SIM has been developed that allows a user terminal such as a mobile phone terminal or a smart phone to connect to a plurality of mobile phone networks with a single SIM card (see Non-Patent Document 1, for example). A multi-profile SIM holds profiles of multiple carriers (information for connecting to mobile phone networks) in a single SIM card, and the user terminal switches between these multiple profiles for use. be able to. As a result, for example, when a communication failure occurs, the communication can be secured by switching the profile in the SIM card and connecting to a different mobile phone network.

"IIJ develops a multi-profile SIM that can connect to multiple mobile phone networks with one SIM", [online], Internet Initiative Japan Inc., [searched on April 24, 2023], Internet <https:// www.iij.ad.jp/news/pressrelease/2022/1025.html＞

However, in the conventional multi-profile SIM, the user of the user terminal must manually switch profiles, and automatic profile switching according to the communication status of the mobile phone network has not been realized.

According to one aspect of the present invention, a user terminal equipped with a multi-profile SIM, a plurality of mobile operator networks with which said user terminal can communicate, and a SIM connected to said user terminal and said plurality of mobile operator networks. a profile switching device, wherein the SIM profile switching device comprises: a communication status management unit that manages communication status of each of the plurality of mobile operator networks; a reward determination unit that determines a reward in reinforcement learning based on each communication situation of the a transmission unit for transmitting to the user terminal, wherein the user terminal has a learning model configured to learn selection of an optimum SIM profile by reinforcement learning based on the state of the user terminal and the reward; a SIM profile selector that uses the learning model to select a SIM profile based on the state of the user terminal; and communicates with one of the plurality of mobile operator networks using the selected SIM profile. A mobile communication system is provided, comprising: a communication unit;

Further, according to one aspect of the present invention, when the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is good, the remuneration determination unit determines the remuneration value for the SIM profile may be determined so as to maximize the reward.

Further, according to one aspect of the present invention, the remuneration determination unit determines that the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is not good, and the other mobile operator network The remuneration may be determined such that the remuneration value for the SIM profile corresponding to the other mobile operator network is maximized when communication conditions are good.

Further, according to one aspect of the present invention, the learning model receives the state s _t of the user terminal as an input, and the action value _Q It may be a model composed of a neural network based on deep reinforcement learning, which outputs (s _t , a _t ).

Further, according to one aspect of the present invention, the learning model is output from the neural network when communication conditions of a mobile operator network corresponding to the SIM profile currently used by the user terminal are good. Learning may be performed using the maximum action value among the plurality of action values.

In addition, according to one aspect of the present invention, the learning model determines that the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is not good, and that the communication status of another mobile operator network is not good. If the situation is favorable, learning may be performed using the behavior value corresponding to the SIM profile of the other mobile operator network among the plurality of behavior values output from the neural network.

Further, according to one aspect of the present invention, a user terminal equipped with a multi-profile SIM, a plurality of mobile operator networks with which the user terminal can communicate, and a terminal connected to the user terminal and the plurality of mobile operator networks. a SIM profile switching device, wherein said SIM profile switching device acquires communication status for each of said plurality of mobile operator networks; and said SIM profile switching device. determining a reward in reinforcement learning based on the communication status of each of said plurality of mobile operator networks, said reward being related to selection of a SIM profile to be used by said user terminal; a step of the SIM profile switching device transmitting the reward to the user terminal; the user terminal using the learning model to select a SIM profile based on the state of the user terminal; and the user terminal using the selected SIM profile to perform the plurality of communicating with one of the mobile operator networks of .

According to the present invention, a user terminal can perform communication by selecting an optimum SIM profile according to the communication status of a mobile operator's network from among multiple SIM profiles held in a SIM card.

1 is a network configuration diagram of a mobile communication system according to an embodiment; FIG. FIG. 4 is a network configuration diagram of a mobile communication system according to another embodiment; FIG. 4 is a schematic diagram showing transition of states of a user terminal; 1 is a block diagram showing functional configurations of a SIM profile switching device and a user terminal according to an embodiment of the present invention; FIG. It is a figure which shows an example (DQN) of a learning model. FIG. 10 is a diagram showing another example of a learning model (Fixed Target Q-Network); 4 is a flow chart showing operations of a SIM profile switching device and a user terminal according to an embodiment of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a network configuration diagram of a mobile communication system 10 according to one embodiment of the present invention. The mobile communication system 10 comprises a user terminal 100 , a first mobile operator network MN ₁ , a second mobile operator network MN ₂ and a SIM profile switching device 400 . Each mobile operator network MN ₁ , MN ₂ comprises a radio access network 200 and a core network 300 respectively. In the mobile communication system 10, a user terminal 100 is capable of communicating with a network such as the Internet 500 via the radio access networks 200 and core networks 300 of mobile operator networks _MN1 , _MN2 . Note that the number of mobile operator networks is not limited to two and may be arbitrary.

A user terminal 100 is a wireless communication device such as a mobile phone terminal or a smart phone. The user terminal 100 has one SIM card 110, and this SIM card 110 stores a plurality of SIM profiles. A SIM profile is information for connecting the user terminal 100 to a mobile operator network. In the example of FIG. 1, user terminal 100 has two SIM profiles SP ₁ , SP ₂ in SIM card 110 . SIM profile SP ₁ corresponds to the first mobile operator network MN ₁ and SIM profile SP ₂ corresponds to the second mobile operator network MN ₂ . The user terminal 100 can connect to a first mobile operator network MN ₁ using SIM profile SP ₁ and connect to a second mobile operator network MN ₂ using SIM profile SP ₂ . can. Note that the number of SIM profiles held in SIM card 110 may be more than two.

The SIM profile switching device 400 is a device that controls switching of SIM profiles in the user terminal 100 . SIM profile switching device 400 is configured as a computer having a processor and memory, and the processor reads out and executes a program stored in the memory to realize a predetermined function. The function and operation of SIM profile switcher 400 are described in detail below.

In the mobile communication system 10 of FIG. 1, the first mobile operator network MN ₁ and the second mobile operator network MN ₂ both comply with the fifth generation (5G) communication standard. A radio access network (NG-RAN: Next Generation Radio Access Network) 200 includes a plurality of base stations. The core network 300 includes AMF (Access and Mobility Management Function) 302, SMF (Session Management Function) 304, PCF (Policy Control Function) 306, UDM (Unified Data Management) 308, UDR (Unified Data Repository) 310, UPF (User Plane Function) 312 nodes.

The AMF 302 is a node that provides a mobility control function and performs mobility control such as location registration, paging, and handover. SMF 304 is a node that provides session management functions and maintains, establishes, modifies and releases sessions. The PCF 306 is a node that provides policy control functions related to quality such as data transfer rate and delay time. The UDM 308 is a node that manages user contract information and authentication information. The UDR 310 is a node that stores a subscriber database that holds user terminal identification numbers and location information. The UPF 312 is a node that communicates user data with user terminals. Each of these nodes conforms to the 5G communication standard, and detailed description thereof is omitted here.

Either or both of the first mobile operator network MN ₁ and the second mobile operator network MN ₂ may comply with the fourth generation (4G) communication standard. FIG. 2 is a network configuration diagram of a mobile communication system 10 in which a first mobile operator network MN ₁ conforms to 5G communication standards and a second mobile operator network MN ₂ conforms to 4G communication standards. In the mobile communication system 10 of FIG. 2, the radio access network 200 of the second mobile operator network MN ₂ is E-UTRAN (Evolved Universal Terrestrial Radio Access Network). The core network 300 of the second mobile operator network MN ₂ includes MME (Mobility Management Entity) 314, SGW (Serving Gateway) 316, PGW-U (Packet data network Gateway-User Plane) 318, PGW-C (Packet data Network Gateway-Control Plane) 320, PCRF (Policy and Charging Rules Function) 322, HSS (Home Subscriber Server) 324, and UDR 326 nodes. These conform to the 4G communication standard, and detailed description thereof is omitted here.

FIG. 3 is a schematic diagram showing state transitions of the user terminal 100. As shown in FIG. The state of _the user _{terminal 100} transitions from state _{s1 to} state _s2 , from state _s2 to state _s3 , . Between each state, the user terminal 100 can take N actions a ₁ , a ₂ , . . . , a _N . For example, the user terminal 100 performs action a ₂ in state s _t−1 and transitions to a new state s _t , then performs action a ₁ in state s _t and further transitions to a new state s _t+1. be able to. Actions that the user terminal 100 may take include selecting a SIM profile from among multiple SIM profiles. For example, action a ₁ of the user terminal 100 corresponds to selecting SIM profile SP ₁ corresponding to the first mobile operator network MN ₁ , action a ₂ of the user terminal 100 corresponds to the second mobile operator network MN ₂ corresponding to selecting SIM profile SP ₂ (and so on). Note that the number N of actions that can be taken is equal to the number of SIM profiles held in the SIM card 110 of the user terminal 100 . The state _st of the user terminal 100 may include, for example, RRC IDLE, RRC INACTIVE, RRC CONNECTED, etc. defined by 3GPP (registered trademark).

FIG. 4 is a block diagram showing functional configurations of the SIM profile switching device 400 and the user terminal 100 in the mobile communication system 10. As shown in FIG. SIM profile switching device 400 includes user terminal information management section 402 , communication status management section 404 , remuneration determination section 406 , and transmission section 408 . User terminal 100 includes learning model 102 , SIM profile selection section 104 , and communication section 106 .

The user terminal information management unit 402 of the SIM profile switching device 400 receives from the user terminal 100 the state _st of the user terminal 100 and the action a _i (i is any of 1, 2, . . . , N) selected by the user terminal 100. ) to obtain user terminal information. The user terminal information management unit 402 further includes all possible states of the user terminal 100 and information on which state the user terminal 100 will transition to next when each action is taken in each state (for example, state transition diagram as shown).

The communication status management unit 404 of the SIM profile switching device 400 is configured to manage the communication status of each of the multiple mobile operator networks in the mobile communication system 10 . For example, the communication status management unit 404, from the respective NG-RAN 200, AMF 302, UDM 308, UDR 310, and UPF 312 in the mobile operator networks MN ₁ and MN ₂ , the CPU usage rate of each device and the Various types of information including traffic volume and failure occurrence information are collected periodically, and based on the collected information, the communication status of each mobile operator network MN ₁ and MN ₂ (for example, the communication band is tight and congestion is occurring) (whether or not it is likely to occur, etc.) can be grasped.

Remuneration determination unit 406 of SIM profile switching device 400 determines a reward for learning the optimum SIM profile in user terminal 100 based on the communication status of each mobile operator network specified by communication status management unit 404. configured to Specifically, the remuneration determining unit 406 determines a remuneration such that an action of selecting a SIM profile of a mobile operator network with good communication conditions has a high remuneration value (for example, maximizes).

For example, the remuneration determining unit 406 determines that the SIM profile currently used by the user terminal 100 is the SIM profile SP ₁ of the first mobile operator network MN ₁ and the communication status of the first mobile operator network MN ₁ is good. , then it has a reward value of "1" for action _a1 , which is the action of selecting SIM profile _SP1 , and a reward value of "1" for all other actions aj ( _j ≠1). The reward may be determined so as to have 0”. Further, for example, the remuneration determining unit 406 determines that the SIM profile currently used by the user terminal 100 is the SIM profile SP ₁ of the first mobile operator network MN ₁ , but the communication status of the first mobile operator network MN ₁ is If not good and the communication condition of the second mobile operator network MN ₂ is good, reward for action _a2 , which is the action of selecting the SIM profile _SP2 corresponding to the second mobile operator network _MN2 . The reward may be determined to have a value of "1" and to have a reward value of "0" for all other actions a _j (j≠2).

The transmission unit 408 of the SIM profile switching device 400 transmits to the user terminal 100 the reward determined by the reward determination unit 406 and the state s _t+1 of the user terminal 100 after transition due to the action of the user terminal 100. configured as The state s _t+1 of the user terminal 100 after transition can be identified by specifying the current state s _t and the action a _i selected in the current state s _t in the user terminal information management unit 402 .

Communication between SIM profile switching device 400 and user terminal 100 (transmission of user terminal information from user terminal 100 to SIM profile switching device 400, reward from SIM profile switching device 400 to user terminal 100, and post-transition state transmission) is performed via the U-plane (user plane) communication path of the mobile operator network currently used by the user terminal 100 .

The learning model 102 of the user terminal 100 is configured to learn to select the optimum SIM profile through reinforcement learning based on the state of the user terminal 100 and the reward given by the SIM profile switching device 400 . For example, as the learning model 102, as shown _in FIG. 5, the state s _t of the user terminal 100 is input to the input layer, and all actions (multiple SIM profiles Selection of any one SIM profile from) DQN (Deep Q-Network) configured to output the action value Q (s _t , a _t ) for each of a _t from each node of the output layer can apply a neural network based on Also, the learning model 102, as shown in FIG. 6, may be based on a Fixed Target Q-Network using two neural networks (main network and target network) with the same configuration as in FIG. In the DQN of FIG. ₅ and the Fixed Target Q-Network of FIG. ₆ , the neural network parameters ( Learning of the learning model 102 is performed by updating the weights and thresholds).

A SIM profile selector 104 of the user terminal 100 is configured to select a SIM profile based on the output of the learning model 102 . For example, the SIM profile selection unit 104 selects the SIM profile corresponding to the maximum action value among the action values Q(s _t , a _t ) for each action at output from the output layer of the neural _network . It can be selected as the SIM profile to use.

The communication portion 106 of the user terminal 100 is configured to communicate with one of the mobile operator networks using the SIM profile selected by the SIM profile selection portion 104 .

FIG. 7 is a flow chart showing operations of SIM profile switching device 400 and user terminal 100 in mobile communication system 10 . In the following, an example will be described in which the learning model 102 of the user terminal 100 is configured as a neural network based on the Fixed Target Q-Network. Note that the user terminal 100 is currently connecting to and communicating with the mobile operator network MN J using the SIM profile SP _J corresponding to the J-th mobile operator network MN _J among the plurality of mobile operator networks _. It is assumed that there is

At step 702 , the state s _t of the user terminal 100 is input to the input layer of the main network that constitutes the learning model 102 of the user terminal 100 . Upon receiving the input of state s _t , learning model 102, in step 704, receives from each node of the output layer of the main network the user terminal 100 in state s _t SIM profiles SP ₁ to SP _N (where N is 2 or more). is an _integer and represents the total number of SIM profiles held in the SIM card 110 attached to _{the user} terminal 100). , . . . , N). For example, node 1 in the output layer outputs the action value Q _main (s _t , a ₁ ) when the user terminal 100 selects SIM profile SP ₁ , that is, performs action a ₁ , and node 2 outputs the user An action value Q _main (s _t , a ₂ ) is output when the terminal 100 selects SIM profile SP ₂ ie performs action a ₂ , and so on. The number of nodes in the output layer is the same as the number of SIM profiles in SIM card 110 .

Here, the learning model (main network and target network) 102 of the user terminal 100 is trained so as to output the maximum action value for the SIM profile SP _J currently used by the user terminal 100. shall be In subsequent step 706, the user terminal 100 selects the action a _J corresponding to the maximum action value Q _main (s _t , a _J ) among all the action values obtained in step 704 and the state s _t of the user terminal 100 to the SIM profile switching device 400 via the U-plane of the currently used mobile operator network. At step 708, the user terminal information management section 402 of the SIM profile switching device 400 acquires this user terminal information.

Next, at step 710 , the communication status manager 404 of the SIM profile switching device 400 determines the communication status for each of the multiple mobile operator networks in the mobile communication system 10 . In step 712, the remuneration determination unit 406 of the SIM profile switching device 400 acquires user terminal information from the user terminal information management unit 402 and the communication status of each mobile operator network from the communication status management unit 404. determines a reward R _t+1 for updating the learning model 102 of the user terminal 100 according to whether the communication status of the mobile operator network currently used is good or not. For example, if the communication status of the mobile operator network currently used by the user terminal 100 is good, the reward value for the action a _J of selecting the SIM profile of the mobile operator network is maximized, and When the communication conditions of the mobile operator network currently used by the user terminal 100 are not good, the action of selecting the SIM profile of another (eg, K (≠J)) mobile operator network with good communication conditions. A reward R _t+1 is determined such that the reward value for a _K is maximized.

Next, in step 714, transmitting section 408 of SIM profile switching device 400 transmits reward R _t+1 determined in step 712 and user terminal 100 after transition when user terminal 100 takes action a _J in state s _t . state s _t+1 to the user terminal 100 . Furthermore, in step 716 , an instruction to update the learning model 102 of the user terminal 100 (see below) is sent from the SIM profile switching device 400 to the user terminal 100 .

Next, in step 718 , in order to obtain the action value used to update the learning model 102 , the next state of the user terminal 100 is input to the input layer of the target network, which is the other neural network that constitutes the learning model 102 of the user terminal 100 . s _t+1 is entered. Upon receiving this input, in step 720, the target network of the learning model 102, like the main network in step 704, _generates the action value Q _target (s t _{+ 1} , a _i ).

Next, in step 722, the main network in the learning model 102 of the user terminal 100 is updated using the action value output from the target network. The update follows the content of the update instruction sent from SIM profile switching device 400 to user terminal 100 in step 716 . The content of the update instruction is linked with the reward R _t+1 determined by the reward determination unit 406 of the SIM profile switching device 400 . Specifically, (A) when the communication status of the mobile operator network currently used by the user terminal 100 is good, the maximum action value max _ai Q _target (s _t+1 , a _i )=Q _target (s _t+1 , a _J ) is used to update the main network (max _ai represents the maximum value for all possible actions a _i ). On the other hand, (B) when the communication status of the mobile operator network currently used by the user terminal 100 is not good, regardless of which of the action values output from the target network is the maximum, the communication status is The behavioral value Q _target (s _t+1 ,a _K ) corresponding to the SIM profile of the Kth good mobile operator network is used to update the main network. A specific update formula is as follows. The main network is trained so that the loss function E of the following equation is minimized.

where γ is the time discount rate. Thus, in step 722, the main network in the learning model 102 of the user terminal 100 is updated (that is, the parameters of the neural network are adjusted) according to the above equation (1) or (2) depending on the communication status of the mobile operator network. is done. Steps 702 to 722 are then repeated again to proceed with the training of the main network. Also, after step 722, in step 724, the parameters of the target network are synchronized with the parameters of the main network after every predetermined number of iterations.

Using the learning model 102 learned in this way, the user terminal 100 can select a SIM profile to use. Specifically, step 704 can be followed by step 726, where the SIM profile selection unit 104 of the user terminal 100 selects all behavioral values Q _main ( _st ,a The maximum behavioral value is selected from _i ), and the SIM profile corresponding to the maximum behavioral value is determined as the optimum SIM profile. Then, at step 728, the communication unit 106 of the user terminal 100 uses the SIM profile selected by the SIM profile selection unit 104 to communicate with the mobile operator network corresponding to that SIM profile. Thus, the user terminal 100 can perform communication by selecting an optimum SIM profile according to the communication status of the mobile operator's network from among multiple SIM profiles held in the SIM card 110 .

Although the embodiment of the present invention has been described above, the present invention is not limited to this, and various modifications can be made without departing from the scope of the invention.

10 mobile communication system 100 user terminal 102 learning model 104 SIM profile selection unit 106 communication unit 110 SIM card 200 radio access network 300 core network 400 SIM profile switching device 402 user terminal information management unit 404 communication status management unit 406 reward determination unit 408 transmission Part 500 Internet MN ₁ , MN ₂ Mobile Operator Network SP ₁ , SP ₂ SIM Profile

Claims (7)

A mobile communication comprising: a user terminal equipped with a multi-profile SIM; a plurality of mobile operator networks with which the user terminal can communicate; and a SIM profile switching device connected to the user terminal and the plurality of mobile operator networks. a system,
The SIM profile switching device,
a communication status management unit that manages the communication status of each of the plurality of mobile operator networks;
A reward determination unit that determines a reward in reinforcement learning based on the communication status of each of the plurality of mobile operator networks, wherein the reward is related to selection of a SIM profile to be used by the user terminal. a decision unit;
a transmission unit that transmits the reward to the user terminal;
The user terminal is
a learning model configured to learn optimal SIM profile selection by reinforcement learning based on the state of the user terminal and the reward;
a SIM profile selection unit that selects a SIM profile based on the state of the user terminal using the learning model;
a communication unit that communicates with one of the plurality of mobile operator networks using the selected SIM profile;
mobile communication system. The remuneration determination unit determines the remuneration so that the remuneration value for the SIM profile currently used by the user terminal is maximized when the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is good. The mobile communication system according to claim 1, wherein: If the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is not good, and the communication status of another mobile operator network is good, the remuneration determination unit 3. The mobile communication system according to claim 2, wherein said reward is determined so as to maximize the reward value for a SIM profile corresponding to a mobile operator network of . The learning model takes as input the state s _t of the user terminal and outputs an action value Q(s _t , _{at )} when an action at of selecting each SIM profile from among a plurality of SIM profiles is performed. , a model constructed by a neural network based on deep reinforcement learning. The learning model selects the maximum action value among the plurality of action values output from the neural network when the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is good. 5. The mobile communication system according to claim 4, wherein learning is performed using. When the communication status of the mobile operator network corresponding to the SIM profile currently used by the user terminal is not good, and the communication status of other mobile operator networks is good, the learning model uses the neural network 6. The mobile communication system according to claim 5, wherein learning is performed using a behavior value corresponding to the SIM profile of the other mobile operator network among the plurality of behavior values output from. A mobile communication comprising: a user terminal equipped with a multi-profile SIM; a plurality of mobile operator networks with which the user terminal can communicate; and a SIM profile switching device connected to the user terminal and the plurality of mobile operator networks. A method of communication in a system, comprising:
the SIM profile switching device acquiring communication status for each of the plurality of mobile operator networks;
A step in which the SIM profile switching device determines a reward in reinforcement learning based on the communication status of each of the plurality of mobile operator networks, wherein the reward is the selection of a SIM profile to be used by the user terminal. a step associated with
the SIM Profile Switcher transmitting the reward to the user terminal;
the user terminal having a learning model learn to select an optimal SIM profile by reinforcement learning based on the state of the user terminal and the reward;
the user terminal using the learning model to select a SIM profile based on the state of the user terminal;
the user terminal communicating with one of the plurality of mobile operator networks using the selected SIM profile;
communication methods, including

Images