JP2021158458A - Terminal device, program to be executed by computer, and computer readable recording medium with program recorded thereon - Google Patents

Abstract

To provide a terminal device which effectively utilizes a frequency by suppressing a packet loss.SOLUTION: Prediction means 3 predicts a channel idle time and a channel use time on a plurality of channels. A learning apparatus 5 executes Q learning in which a length of a transmission time of a packet is defined as a state, a selection of a channel is defined as an action and a throughput in the case where a communication is made successful is defined as a reward; and outputs output information including an action in the case where a maximum Q value is obtained in each state. Control means 4 detects from the output information the action in the case where the maximum Q value is obtained in the state of the Q learning corresponding to a length of a transmission time of a packet for transmission, and selects a channel which is selected by the detected action, as a channel for transmission. Transmission means 7 transmits the packet for transmission to a base station through the channel for transmission without executing back-off in the case where it is determined by the control means that the packet for transmission is transmissible.SELECTED DRAWING: Figure 2

Description

The present invention relates to a terminal device, a program to be executed by a computer, and a computer-readable recording medium on which the program is recorded.

In the CSMA / CA (Carrier Sense Multiple Access / Collision Avoidance) method represented by wireless LAN (Local Area Network), backoff time (transmission from other stations) occurs when packet collision / loss occurs due to simultaneous transmission. After detecting that the radio wave has stopped, the packet is not transmitted immediately, and the packet collision probability is reduced by lengthening the (intentional waiting time until transmission by the own station) (Patent Document 1).

Japanese Unexamined Patent Publication No. 2006-013894

However, in recent years, increasing the backoff time indiscriminately due to the increase in the number of terminals does not lead to the resolution of packet collisions and causes the communication delay of the terminals themselves.
Therefore, according to the embodiment of the present invention, there is provided a terminal device that suppresses packet loss and effectively utilizes the frequency.

Further, according to the embodiment of the present invention, a program for suppressing packet loss and causing a computer to effectively use a frequency is provided.

Further, according to an embodiment of the present invention, there is provided a computer-readable recording medium on which a program for suppressing packet loss and causing a computer to effectively use frequencies is recorded.

(Structure 1)
According to an embodiment of the present invention, the terminal device includes predictive means, a learner, selection means, and transmission means. The predicting means predicts the channel free time, which is the time when the channel is free, and the channel usage time, which is the time when the channel is used, for all of the plurality of channels. The learner transmits the communication result of the communication for transmitting the packet to the base station, the competing terminal information which is the information of the competing terminal device competing with the terminal device, and from the transmission of one packet to the transmission of the next packet. Based on the input information consisting of the channel standby time, which is the standby time of the channel, or the input information consisting of the non-transmission information indicating that the packet cannot be transmitted to the base station, the length of the packet transmission time is set as the state. The action is to select a channel, Q learning is executed with the reciprocal of the throughput or channel waiting time when communication is successful as a reward, and the Q value is updated repeatedly until the end condition of Q learning is satisfied. Output information consisting of a configuration in which each state when the end condition of Q learning is satisfied, the maximum Q value in each state, and the action when the maximum Q value is obtained in each state are interrelated. Execute the learning process to output. The selection means detects the action when the maximum Q value corresponding to the Q learning state corresponding to the length of the transmission time of the packet to be transmitted is obtained from the output information, and selects the channel selected by the detected action. Select as the transmission channel. The transmitting means does not perform a backoff when the first condition, which consists of the channel free time predicted by the predicting means in the transmitting channel being longer than the length of the transmitting time of the packet to be transmitted, is satisfied. , Send the packet you want to send to the base station on the transmission channel.

(Structure 2)
In the configuration 1, when the competing terminal information consists of the number of competing terminal devices and the communication result indicates success of communication, the learning device increases as the number of competing terminal devices decreases, and the number of competing terminal devices increases. The first calculation process that calculates the reward so that it becomes smaller as the number increases is executed, and the second calculation that calculates the reward so that the reward becomes larger as the channel waiting time becomes shorter and becomes smaller as the channel waiting time becomes longer. Execute the process and execute the learning process.

(Structure 3)
In configuration 2, the learner multiplies the throughput by a first weighting factor consisting of the first value when the number of competing terminal devices is the first number, and the number of competing terminal devices is larger than the first number. When the number of the second number is large, the throughput is multiplied by the first weighting coefficient consisting of the second value smaller than the first value to execute the first arithmetic processing.

(Structure 4)
In configuration 3, the learner executes the first arithmetic processing by multiplying the throughput by the reciprocal of the number of competing terminal devices as the first weighting factor.

(Structure 5)
In any of configurations 2 through 4, when the channel wait time consists of the first time length, the learner multiplies the throughput by a second weighting factor consisting of a third value, and the channel wait time is first. When the second time length is longer than the time length of, the throughput is multiplied by the second weighting coefficient consisting of the fourth value smaller than the third value to execute the second arithmetic processing.

(Structure 6)
In any of the configurations 2 to 5, the learner executes the second arithmetic processing by multiplying the throughput by the reciprocal of the channel standby time as the second weighting factor.

(Structure 7)
In the configuration 1, when the input information consists of non-transmissionable information in the learning process, the learner updates the Q value with the reward as zero.

(Structure 8)
In any one of configurations 1 to 7, the terminal device further comprises receiving means. The receiving means receives the competing terminal information from the base station on the control channel.

(Structure 9)
In any of the configurations 1 to 8, the communication result is obtained from the first index when the transmitting means transmits the packet to be transmitted to the base station and then receives the ACK packet indicating that the packet has been received from the base station. Therefore, when the transmitting means transmits the packet to be transmitted to the base station and then does not receive the ACK packet from the base station, the second index is used.

(Structure 10)
In any one of configurations 1 to 9, in addition to the first condition, the transmission means has a second condition in which, as a result of carrier sense in the transmission channel, another terminal device is not communicating in the transmission channel. When is satisfied, the packet to be transmitted is transmitted to the base station on the transmission channel without executing backoff.

(Structure 11)
Also, according to embodiments of the present invention, the program is:
The first step in which the predictor predicts the channel free time, which is the time the channel is free, and the channel usage time, which is the time the channel is used, for all of the plurality of channels.
From the transmission of one packet to the transmission of the next packet, the communication result of the communication in which the learner transmits the packet to the base station, the competing terminal information which is the information of the competing terminal device competing with the terminal device, and the competing terminal information. Based on the input information consisting of the channel standby time, which is the standby time of the channel, or the input information consisting of the non-transmission information indicating that the packet cannot be transmitted to the base station, the length of the packet transmission time is set as the state of the channel. The action is to select, and Q learning is executed with the reciprocal of the throughput or channel waiting time when communication is successful as a reward, and the Q value is updated repeatedly until the end condition of Q learning is satisfied. Outputs output information consisting of a configuration in which each state when the learning end condition is satisfied, the maximum Q value in each state, and the action when the maximum Q value is obtained in each state are interrelated. The second step of executing the learning process to be
The selection means detects the action when the maximum Q value corresponding to the Q learning state corresponding to the length of the transmission time of the packet to be transmitted is obtained from the output information, and selects the channel selected by the detected action. The third step of selecting as the transmission channel and
When the transmitting means satisfies the first condition that the channel free time predicted by the predicting means in the transmitting channel is longer than the length of the transmitting time of the packet to be transmitted, the backoff is not executed. , A program for causing a computer to execute a fourth step of transmitting a packet to be transmitted to the base station on the transmission channel.

(Structure 12)
In the configuration 11, when the competing terminal information consists of the number of competing terminal devices and the communication result indicates success of communication in the second step, the learning device becomes larger as the number of competing terminal devices decreases. The first arithmetic process for calculating the reward so that the number of competing terminal devices increases is executed, and the reward is increased as the channel standby time becomes shorter and decreases as the channel standby time becomes longer. The learning process is executed by executing the second arithmetic process to be calculated.

(Structure 13)
In configuration 12, in the second step, the learner multiplies the throughput by the first weighting factor consisting of the first value when the number of competing terminal devices is the first number, and the number of competing terminal devices is increased. When the second number is larger than the first number, the throughput is multiplied by the first weighting coefficient consisting of the second value smaller than the first value to execute the first arithmetic processing.

(Structure 14)
In configuration 13, in the second step, the learner executes the first arithmetic processing by multiplying the throughput by the reciprocal of the number of competing terminal devices as the first weighting factor.

(Structure 15)
In any of configurations 12 through 14, the learner multiplies the throughput by a second weighting factor of a third value when the channel latency consists of the first time length in the second step. When the channel standby time consists of a second time length longer than the first time length, the throughput is multiplied by a second weighting factor consisting of a fourth value smaller than the third value to perform the second arithmetic processing. To execute.

(Structure 16)
In any of configurations 12 through 15, the learner executes the second arithmetic processing in the second step by multiplying the throughput by the reciprocal of the channel standby time as the second weighting factor.

(Structure 17)
In configuration 11, when the input information consists of non-transmissionable information in the learning process of the second step, the learner updates the Q value with the reward as zero.

(Structure 18)
In any of configurations 11 through 17, the program causes the receiving means to further cause the computer to perform a fifth step of receiving competing terminal information from the base station on the control channel.

(Structure 19)
In any of the configurations 11 to 18, the communication result is obtained from the first index when the transmitting means transmits the packet to be transmitted to the base station and then receives the ACK packet indicating that the packet has been received from the base station. Therefore, when the transmitting means transmits the packet to be transmitted to the base station and then does not receive the ACK packet from the base station, the second index is used.

(Structure 20)
In any one of configurations 11 to 19, in addition to the first condition, the transmission means has a second condition in which, as a result of carrier sense in the transmission channel, another terminal device is not communicating in the transmission channel. When is satisfied, the packet to be transmitted is transmitted to the base station on the transmission channel without executing backoff.

(Structure 21)
Further, according to an embodiment of the present invention, the recording medium is a computer-readable recording medium on which the program according to any one of configurations 11 to 20 is recorded.

Frequency can be used effectively by suppressing packet loss.

It is the schematic of the communication system in embodiment of this invention. It is the schematic of the terminal apparatus shown in FIG. It is the schematic of the prediction means shown in FIG. It is a conceptual diagram of the received power spectrum. It is a conceptual diagram which shows the example which arranged the busy duration subset and the idle duration subset in time series. It is a schematic diagram of the predictor shown in FIG. It is a figure which shows the correspondence table of a transition pattern and a coefficient b ₁ , b ₂ , ..., b _p. It is a conceptual diagram of a channel waiting time. It is a conceptual diagram of a Q table. It is 1st schematic diagram for demonstrating the method of updating the Q table. It is a 2nd schematic diagram for demonstrating the method of updating the Q table. It is a 3rd schematic diagram for demonstrating the method of updating the Q table. It is a figure for demonstrating the method of determining whether or not a transmission packet can be transmitted. It is a flowchart for demonstrating operation of the terminal apparatus shown in FIG. It is a flowchart for demonstrating the detailed operation of step S5 of FIG. It is a flowchart for demonstrating the detailed operation of step S12 of FIG. It is a conceptual diagram of output information.

Embodiments of the present invention will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are designated by the same reference numerals, and the description thereof will not be repeated.

FIG. 1 is a schematic diagram of a communication system according to an embodiment of the present invention. With reference to FIG. 1, the communication system 100 includes a base station BS and a plurality of terminal devices TM. The base station BS and the plurality of terminal devices TM are arranged in the wireless communication space.

The base station BS has a communication range REG1. The plurality of terminal devices TM are arranged within the communication range REG1.

The base station BS transmits the packet to any one of the plurality of terminal devices TM and receives the packet from any one of the plurality of terminal devices TM.

Then, when the base station BS receives the packet, it transmits an ACK (Acknowledgment) packet indicating that the packet has been received to the terminal device (any of a plurality of terminal device TMs) that is the source of the packet.

Further, the base station BS transmits competing terminal information, which is information about a terminal device other than the source terminal device, to the source terminal device via a control channel.

When transmitting a packet to the base station BS, each of the plurality of terminal devices TM selects a transmission channel CH_T by the method described later, executes carrier sense on the selected transmission channel CH_T, and then transmits the packet. If it is determined that this is possible, the packet is transmitted to the base station BS on the transmission channel without executing the backoff.

Further, each of the plurality of terminal devices TM receives a packet from the base station BS. Then, when each of the plurality of terminal devices TM receives the packet, the ACK packet is transmitted to the base station BS.

Further, each of the plurality of terminal devices TM receives competing terminal information from the base station BS on the control channel.

In the following, each of the plurality of terminal devices TM will be referred to as “terminal device 10”.

FIG. 2 is a schematic view of the terminal device shown in FIG. With reference to FIG. 2, the terminal device 10 includes an antenna 1, a receiving means 2, a predicting means 3, a control means 4, a learning device 5, an application 6, and a transmitting means 7.

According to the control from the control means 4, the receiving means 2 performs carrier sense via the antenna 1 on a plurality of channels having a plurality of frequencies different from each other in the frequency band used for wireless communication in the communication system 100, and receives power. Acquires a received power spectrum showing the time dependence of. Then, the receiving means 2 outputs the received power spectrum to the predicting means 3.

Further, the receiving means 2 receives the competing terminal information from the base station BS on the control channel via the antenna 1, and outputs the received competing terminal information to the control means 4.

Further, the receiving means 2 performs carrier sense via the antenna 1 on the transmission channel CH_T according to the control from the control means 4, and outputs the result of the carrier sense to the control means 4.

Further, the receiving means 2 receives an ACK packet from the base station BS on the transmission channel CH_T via the antenna 1, and outputs the received ACK packet to the control means 4.

The prediction means 3 receives the received power spectrum from the receiving means 2. Then, in response to the control from the control means 4, the predictor means 3 uses the channel free time, which is the length of time the channel is free, and the channel, based on the received power spectrum, by the method described later. It detects the channel usage time, which is the length of time, and predicts the channel free time and channel usage time after transmitting the packet based on the detected channel free time and channel usage time. Then, the prediction means 3 outputs the predicted prediction channel free time and the prediction channel usage time to the control means 4.

The control means 4 controls the receiving means 2 so as to execute carrier sense on a plurality of channels.

Further, the control means 4 controls the prediction means 3 so as to predict the channel free time and the channel use time.

Further, the control means 4 receives the competing terminal information and the ACK packet from the receiving means 2, receives the predicted channel free time and the predicted channel usage time from the predicting means 3, and receives a transmission packet which is a packet to be transmitted from the application 6.

Then, when the length of the transmission time when transmitting the transmission packet is longer than the predicted channel free time, the control means 4 determines that the transmission packet cannot be transmitted, and generates transmission non-transmission information. ..

Further, when the control means 4 receives the ACK packet from the receiving means 2 after outputting the transmitting packet to the transmitting means 7, it generates a first index indicating that the communication for transmitting the packet to the base station BS is successful. Then, when the transmission packet is output to the transmission means 7 and then the ACK packet is not received from the reception means 2, a second index indicating that the communication for transmitting the packet to the base station BS has failed is generated. Then, the control means 4 generates a communication result including the first index and the second index. The first index is composed of, for example, "1", and the second index is composed of, for example, "0".

Further, the control means 4 has a channel waiting time which is the length of the waiting time of the channel from the transmission of the transmission packet to the transmission of the next transmission packet based on the predicted channel free time and the predicted channel usage time. Is calculated.

When the length of the transmission time when transmitting the transmission packet is shorter than the predicted channel free time, the control means 4 generates the input information IF_INPUT1 including the competing terminal information, the communication result, and the channel standby time. Then, the control means 4 outputs the generated input information IF_INPUT1 to the learner 5, sets the length of the packet transmission time as a state, sets the channel as an action, and uses the communication throughput as a reward for Q-learning. The learner 5 is controlled so as to execute.

When the control means 4 generates the non-transmission information, the control means 4 generates the input information IF_INPUT2 composed of the non-transmission information, outputs the generated input information IF_INPUT2 to the learner 5, sets the reward to zero, and updates the Q value. Control the learner 5.

The control means 4 outputs the input information IF_INPUT1 or the input information IF_INPUT2 to the learner 5, and then receives the output information IF_OUT from the learner 5. Then, the control means 4 divides the data amount of the transmission packet by the transmission rate to calculate the transmission time of the transmission packet.

Then, the control means 4 refers to the output information IF_OUT, and among the plurality of Q values in the state corresponding to the length of the transmission time of the transmission packet, the maximum Q value and the action corresponding to the maximum Q value. And detect. Then, the control means 4 selects the channel selected in the detected action as the transmission channel CH_T.

After that, the control means 4 controls the receiving means 2 so as to execute the carrier sense in the transmitting channel CH_T. When the control means 4 receives the carrier sense result from the receiving means 2, it determines whether or not the transmission packet can be transmitted based on the received carrier sense result.

When the control means 4 determines that the transmission packet can be transmitted, the control means 4 outputs the transmission packet and the transmission channel CH_T to the transmission means 7, and transmits the transmission packet on the transmission channel CH_T so as to transmit the transmission packet. 7 is controlled.

On the other hand, when the control means 4 determines that the transmission of the transmission packet is impossible, the control means 4 generates the transmission non-transmission information. After that, the control means 4 receives a new transmission packet from the application 6.

The application 6 generates a transmission packet and outputs the generated transmission packet to the control means 4.

When the transmission means 7 receives the transmission packet and the transmission channel CH_T from the control means 4, the transmission means 7 transmits the transmission packet to the base station BS via the transmission channel CH_T via the antenna 1.

FIG. 3 is a schematic view of the prediction means 3 shown in FIG. With reference to FIG. 3, the prediction means 3 includes a determination unit 31, a measurement unit 32, a classification unit 33, a prediction unit 34, and a predictor 35.

The determination unit 31 receives a received power spectrum from the receiving means 2, and determines whether it is in a busy state (Busy) or an idle state (Idle) based on the received received power spectrum. Then, the determination unit 31 detects the start time and the end time in the busy state (Busy), and also detects the start time and the end time in the idle state (Idle). Then, the determination unit 31 outputs the start time and end time of the busy state (Busy) and the start time and end time of the idle state (Idle) to the measurement unit 32.

The measurement unit 32 receives from the determination unit 31 a start time and an end time in a busy state (Busy) and a start time and an end time in an idle state (Idle). Then, the measuring unit 32 measures the busy duration based on the start time and the end time of the busy state (Busy), and measures the idle duration based on the start time and the end time of the idle state (Idle). .. Then, the measurement unit 32 outputs the busy duration and the idle duration to the classification unit 33.

The classification unit 33 receives the busy duration and the idle duration from the measurement unit 32. Then, the classification unit 33 classifies the busy duration and the idle duration into a subset by a method described later. Then, the classification unit 33 outputs the classified subset of the busy duration and the idle duration to the prediction unit 34.

The prediction unit 34 receives a subset of the busy duration and the idle duration from the classification unit 33. Then, the prediction unit 34 outputs a subset of the busy duration and the idle duration to the predictor 35. After that, the prediction unit 34 receives the predicted busy duration and the predicted idle duration, which are the results of predicting the future busy duration and the idle duration, from the predictor 35, and the received predicted busy duration and the predicted idle duration. Is output to the control means 4 as the predicted channel usage time and the predicted channel free time, respectively.

FIG. 4 is a conceptual diagram of the received power spectrum. In FIG. 4, the vertical axis represents the received power and the horizontal axis represents the time.

With reference to FIG. 4, the received power spectrum SP_RSSI changes in received power over time.

Upon receiving the received power spectrum SP_RSSI from the receiving means 2, the determination unit 31 sets the threshold value RSSI_th. The threshold RSSI_th is, for example, −80 [dBm].

Then, the determination unit 31 detects times t1 to t8 in which the received power of the received power spectrum SP_RSSI coincides with the threshold value RSSI_th. After that, the determination unit 31 determines that the section between times t1 to t2, t3 to t4, t5 to t6, and t7 to t8 is in a busy state based on the times t1 to t8, and determines the time t2 to t3, t4 to t5, and t6. The section from to t7 is determined to be in the idle state.

Then, the determination unit 31 associates the busy state with the section of the time t1 to t2, t3 to t4, t5 to t6, t7 to t8 [busy state: t1 to t2, t3 to t4, t5 to t6, t7. ~ T8] is generated, and [idle state: t2 to t3, t4 to t5, t6 to t7] in which the idle state is associated with the section of time t2 to t3, t4 to t5, t6 to t7 is generated. Then, the determination unit 31 outputs [busy state: t1 to t2, t3 to t4, t5 to t6, t7 to t8] and [idle state: t2 to t3, t4 to t5, t6 to t7] to the measurement unit 32. do.

The measuring unit 32 receives [busy state: t1 to t2, t3 to t4, t5 to t6, t7 to t8] and [idle state: t2 to t3, t4 to t5, t6 to t7] from the determination unit 31. Then, the measuring unit 32, the time length _{T leg1} from time t1 to time t2, the time length _{T leg2} from time t3 to time t4, the time length _T from the time t5 to time t6 _Leg3, and from time t7 to time t8 to measure the time length _{T Leg4} of, respectively, to generate the busy duration _T Busy1 _{~T Busy4.}

Further, the measuring unit 32, the time length _{T Leg5} from time t2 to time t3, the time length _{T Leg6} from time t4 to time t5, and the time length _{T Leg7} from time t6 to time t7 is measured, respectively, Idle duration _TIdl1 to _Tide3 are generated.

Then, the measuring unit 32 associates the busy durations T _{Busy1 to} T _Busy4 with the sections of the times t1 to t2, t3 to t4, t5 to t6, and t7 to t8 [t1 to t2: T _Busy1 / t3 to t4]. : T _Busy2 / t5 to t6: T _Busy3 / t7 to t8: T _Busy4 ] is generated.

Further, the measuring unit 32 associates the idle durations _TIdl1 to _Tide3 with the intervals of the times t2 to t3, t4 to t5, t6 to t7 [t2 to t3: _TIdl1 / t4 to t5: _TIdl2 /. t6 to t7: _TIdl3 ] is generated.

Then, the measuring unit 32 has [t1 to t2: T _Busy1 / t3 to t4: T _Busy2 / t5 to t6: T _Busy3 / t7 to t8: T _Busy4 ] and [t2 to t3: _TIdl1 / t4 to t5: T. _Idl2 / t6 to t7: _TIdl3 ] is output to the classification unit 33.

The classification unit 33 includes [t1 to t2: T _Busy1 / t3 to t4: T _Busy2 / t5 to t6: T _Busy3 / t7 to t8: T _Busy4 ] and [t2 to t3: T _Idl1 / t4 to t5: _TIdl2 / t6 to t7: _TIdl3 ] is received from the measuring unit 32.

In addition, the classification unit 33 holds in advance a time range for classifying the busy duration and the idle duration into a subset.

The time range for classifying busy duration and idle duration into subsets is, for example:
(1) Busy duration subset: less than 1 ms → step1, 1 ms or more → step2
(2) Idle duration subset: less than 0.5 ms → step1, 0.5 ms or more → step2
Classifying unit 33 based on the time range of busy duration subset (1), classifies each of the busy duration _T Busy1 _{~T Busy4} any subset step1, step2, a time range of idle duration subset (2 ), Each of the idle durations _{TIdl1 to TIdl3} is classified into one of the subsets step1 and step2.

FIG. 5 is a conceptual diagram showing an example in which a busy duration subset and an idle duration subset are arranged in chronological order.

With reference to FIG. 5, the classification unit 33 classifies the time t1 to t2, t3 to t4, t5 to t6, t7 to t8, the time t2 to t3, t4 to t5, t6 to t7, and a subset. Busy duration T _{Busy1 to} T _Busy4 , and idle duration T _{Idl1 to} T _Idl3 classified into subsets, for example, busy duration T _Busy1 (step2) / idle duration _TIdl1 (step1) / busy duration T _Busy2 (step1) / Idle duration T _Idl2 (step2) / Busy duration T _Busy3 (step1) / Idle duration T _Idl3 (step1) / Busy duration T _Busy4 (step1) Busy duration T _Busy1 ~ T _Busy4 and arranging the idle duration _T Idl1 _{~T Idl3} in chronological order.

A plurality of busy durations and idle durations arranged in the time series shown in FIG. 5 are generated.

_{Then, the classification unit 33 outputs} the busy durations T Busy 1 to T Busy ₄ and the idle durations T _{Idl 1 to} T _{Idl 3} arranged in time series to the prediction unit 34.

Prediction unit 34, when receives the busy duration arranged in series _T Busy1 _{~T Busy4} and idle duration _T Idl1 _{~T Idl3} from the classification unit 33, the busy duration arranged in time series received _{T BUSY1} through T the _Busy4 and idle duration _T Idl1 _{~T Idl3} outputs to the prediction unit 35.

Predictor 35, when it receives the busy duration arranged in series _T Busy1 _{~T Busy4} and idle duration _T Idl1 _{~T Idl3} from the prediction unit 34, when the busy duration arranged in series _T Busy1 _{~T Busy4} and idle The future busy duration and idle duration are predicted based on the durations T _{Idl1 to TIdl3} , and the predicted busy duration and the predicted idle duration are output to the prediction unit 34.

A method of predicting the future busy duration and idle duration in the predictor 35 will be described.

FIG. 6 is a schematic view of the predictor 35 shown in FIG. With reference to FIG. 6, the predictor 35 has one layer, two layers, ..., K layer. K is an integer of 2 or more.

Each of the 1st layer, 2nd layer, ..., And K layer has a number of analysis units (◯ in FIG. 6) corresponding to the number of _{subsets Si (i is an integer of 2 or more).} FIG. 6 shows a schematic diagram of the predictor 35 when the _{number of subsets Si is two.}

If the number of subsets is _{S i,} total data stream will _{S 1 × S 2 × ··· ×} S K. The case shown in FIG. 6, each of the number of subsets _S 1 to S _K, since it is "2", the total data stream will 2 × 2 × ··· × 2 = 2 K.

Each of the 1st layer, the 2nd layer, ..., And the K layer analyzes the busy / idle correlation. Then, the first layer receives busy / idle data from the prediction unit 34, analyzes the busy / idle correlation based on the received busy / idle data, and transfers the analysis result of the busy / idle correlation to the second layer. Output.

Busy / idle data consists busy duration _T Busy1 _{~T Busy4} and idle duration _T Idl1 _{~T Idl3} arranged in time series as shown in FIG. Based on the busy / idle data, the first layer has busy durations T _{Busy1 to} T _Busy3 and idle durations _{TIdl1 to TIdl3} , busy duration T _Busy1 (step2) → idle duration _TIdl1 (step1) → busy. Duration T _Busy2 (step1) → Idle duration T _Idl2 (step2) → Busy duration T _Busy3 (step1) → Idle duration T _Idl3 (step1) → Busy duration T _Busy4 (step1) A transition pattern is detected, and the correlation between the busy duration and the idle duration is analyzed based on the detected transition pattern.

More specifically, the first layer is _{a transition from busy duration T Busy1} (step2) to idle duration _TIdl1 (step1), idle duration _TIdl1 (step1) → busy duration T _Busy2 (step1). Transition, busy duration T _Busy2 (step1) → idle duration T _Idl2 (step2), idle duration T _Idl2 (step2) → busy duration T _Busy3 (step1), busy duration T _Busy3 ( step1) → evolution of idle duration _T Idl3 to (step1), and idle duration _T Idl3 (step1) → on the basis of the busy continuing transition time _T Busy4 to (step1), after the busy duration of a subset step2 _{T BUSY1} arrives idle duration _{T Idl1} subset step1, busy duration _{T BUSY2} subset step1 is reached after the idle duration _{T Idl1} subset step1, idle duration subset step2 after busy duration of a subset step1 _{T BUSY2} T _IDL2 is reached, busy duration _{T Busy3} subset step1 is reached after the idle duration _{T IDL2} subset step2, idle duration _{T Idl3} subset step1 is reached after the busy duration of a subset step1 _{T Busy3,} By detecting the arrival of the busy duration T _Busy4 of the subset step1 after the idle duration _TIdl3 of the subset step1, the correlation between the adjacent busy duration / idle duration and the adjacent idle duration / busy duration Correlation, that is, the degree of connection between the busy duration subset and the idle duration subset in the adjacent busy duration / idle duration, and the idle duration subset and busy in the adjacent idle duration / busy duration. Analyze the degree of connection with a subset of duration.

That is, the first layer comes after the busy duration T _Busy of the subset step 1 and the degree Dg1 indicating whether the subset of the idle duration T _{Idle is the subset step 1 and the busy duration T Busy} of the subset step 1. degree indicating subset of idle duration _{T idle} is the degree Dg2 indicating which subset step2, a subset of the idle duration _{T idle} arriving the next busy duration of a subset step2 _{T busy} is whether a subset step1 to Dg3 and ,, the degree Dg4 subset busy duration of a subset step2 _{T busy} next idle duration for the arrival of _{T idle} is indicating which subset step2, busy continuing arriving at the next idle duration _{T idle} subset step1 subset of time _{T busy} is the degree Dg5 indicating which subset step1, a subset of the busy duration _{T busy} arriving at the next idle duration subsets step1 _{T idle} is the degree Dg6 indicating which subset step2, subset next subset busy duration _{T busy} arriving idle duration _{T idle} of step2 is the degree Dg7 indicating which subset step1, busy duration _{T busy} arriving at the next idle duration subsets step2 _{T idle} The degree Dg8 indicating whether the subset is the subset step2 is analyzed as the analysis result.

Then, when the data of the first layer is busy data, the idle that arrives next to the subset of the _{busy duration T Busy (either step 1 or 2) based on the analysis result (degrees Dg1 to Dg8).} predicting a subset of duration _{T idle} S_PDC_1_1 (either step1,2), and outputs the prediction subset of the predicted idle duration _{T idle} S_PDC_1_1 (either Step1,2) to two layers.

Further, the first layer, the busy data of one layer if idle data, based on the analysis result (the degree Dg1～Dg8), arrives next subset of idle duration _{T Idle} (either Step1,2) A subset S_PDC_1_2 (one of steps 1 and 2) of the duration T _Busy is predicted, and the predicted subset S_PDC_1_2 (one of the steps 1 and 2) of the predicted _{idle duration T Idle is output to the second layer.}

The second layer receives the predicted subset S_PDC_1_1 (one of steps 1 and 2) and busy / idle data from the first layer, or the predicted subset S_PDC_1_2 (either of steps 1 and 2) and busy / idle data.

Then, the second layer analyzes the above-mentioned degrees Dg1 to Dg8 based on the busy / idle data in the same manner as the first layer.

After that, when the data of the second layer is idle data, _{the prediction of the busy duration T Busy} that comes after the prediction subset S_PDC_1_2 of _{the idle duration T Idle} based on the analysis result (degrees Dg1 to Dg8) is predicted. Generate a subset S_PDC_2_1.

Further, the two-layer, if the data of the two layers is busy data, analysis results based on (degree Dg1～Dg8), to generate the idle duration _T prediction subset S_PDC_2_2 of _Idle arriving at the next prediction subset S_PDC_1_1.

Then, the second layer outputs the predicted subset S_PDC_2 or the predicted subset S_PDC_2 to the third layer.

Hereinafter, in the same manner, the K layer receives the predicted subset S_PDC_K-1_1 and busy / idle data, or the predicted subset S_PDC_K-1_2 and busy / idle data from the K-1 layer.

Then, the K layer analyzes the above-mentioned degrees Dg1 to Dg8 based on the busy / idle data in the same manner as the first layer.

After that, when the data of the K layer is idle data, the K layer generates a predicted subset S_PDC_K_1 of _{the busy duration T Busy} that comes next to the predicted subset S_PDC_K-1_1 based on the analysis results (degrees Dg1 to Dg8). do. Then, the K layer outputs the prediction subset S_PDC_K_1 as the predicted value of the _{busy data (busy duration T Busy).}

Also, K layer, if the data of the K layer are busy data, generates an analysis result based on the (degree Dg1～Dg8), prediction subset S_PDC_K-1_2 prediction subset S_PDC_K_2 idle duration arriving at the next _{T Idle} do. Then, K layer outputs the prediction subset S_PDC_K_1 as the predicted value of the idle data (idle duration _{T Idle).}

When the K layer of the predictor 35 is 5 layers, the prediction unit 34 has _{a subset of busy duration T Busy} (one of steps 1 and 2) (1 layer) / a subset of _{idle duration T Idle (steps 1 and 2).} Any) (2 layers) / Busy duration T _Busy subset (1 or 2 steps) (3 layers) / Idle duration T _Idle subset (1 or 2 steps) (4 layers) / Busy continuation Idle / busy data Data1 consisting of a subset of time T _Busy (one of steps 1 and 2) (5 layers) and _{a subset of idle duration T Idle} (any of steps 1 and 2) (1 layer) / busy duration T _{A subset of Busy} (one of steps 1 and 2) (2 layers) / _{a subset of idle duration T Idle} (one of steps 1 and 2) (3 layers) / _{a subset of busy duration T Busy} (one of steps 1 and 2) ) (4 layers) / idle duration _Idle / busy data Data2 consisting of a subset of TIdle (one of steps 1 and 2) (5 layers) is alternately output to the predictor 35.

When the predictor 35 receives the idle / busy data Data1 from the prediction unit 34, the first layer, the third layer, and the fifth layer become busy data. The 2nd and 4th layers are idle data. As a result, the predictor 45 outputs the predicted idle duration.

Further, when the predictor 35 receives the idle / busy data Data2 from the prediction unit 34, the 1st layer, the 3rd layer and the 5th layer become idle data, and the 2nd layer and the 4th layer become busy data. As a result, the predictor 45 outputs the predicted busy duration.

In this way, the predictor 35 alternately outputs the predicted idle duration and the predicted busy duration to the prediction unit 34.

Then, the prediction unit 34 alternately outputs the predicted idle duration and the predicted busy duration received from the predictor 35 to the control means 4.

Since the predicted idle duration and the predicted busy duration include either of the subset steps 1 and 2, the control means 4 is based on a subset of the predicted idle duration and the predicted busy duration (either of the subset steps 1 and 2). , The length of the predicted idle duration and the length of the predicted busy duration can be known.

The predictor 35 generally predicts a predicted idle duration or a predicted busy duration by the following equation.

In the equation (1), X _i , X _i-1 , ..., X _i-p-1 are the busy / idle measurement times of the latest p (p is an integer of 2 or more). It is a known value. Further, X _{i + 1} is a predicted value. Further, b ₁ , b ₂ , ..., B _p are coefficients.

_{When predicting X i + 1} by the equation (1), it is necessary to determine the coefficients b ₁ , b ₂ , ..., B _p.

In this case, the predictor 35 learns the _{coefficients b 1} , b ₂ , ..., B _{p in the training section.}

Coefficients _{b 1,} b 2 in the training interval, ..., learning of _{b p} will be described. Coefficients _{b 1,} b 2, · · ·, training data for calculating the _{b p,} for example, a 200 sample, the order of the prediction equation (1), for example, the coefficient _{b 1} as a secondary (p = 2) , B ₂ is learned.

The predictor 35 calculates the predicted values of the busy duration and the idle duration by the method described above based on the training data of 200 samples, and calculates the coefficients b ₁ and b ₂ . In this case, the predictor 35 calculates the _{coefficients b 1} and b ₂ by solving the normal equations resulting from the least squares equation using the Levinson-Durbin induction method.

The predictor 35 calculates the _{coefficients b 1} and b ₂ for all the transition patterns of the busy duration T _Busy / idle duration T _{Idle in the training section.}

For example, when the number of subsets S _i is 2 and K = 5, the transition pattern of _{busy duration T Busy} / idle duration T _{Idle is 32.}

Accordingly, the predictor 35, 32 for all the transition pattern, coefficients _b 1, _{b 2} calculates the prediction unit 34 in association with each of the coefficients _b 1, _{b 2} with the calculated 32 transition pattern Output to.

The prediction unit 34 receives the correspondence between the 32 transition patterns and the 32 sets of coefficients b ₁ and b ₂ from the predictor 35, and receives the 32 transition patterns and the 32 sets of coefficients b ₁ and b. _Maintain the correspondence with 2.

_{The predictor 35 similarly calculates the coefficients b 1} , b ₂ , ..., B _p for all 32 transition patterns even when p is other than 2, and the calculated coefficients b ₁ , _b 2, ···, in association with each of the 32 transition patterns _{b p} outputs to the prediction unit 34.

Prediction unit 34, and 32 transition patterns, 32 sets of coefficients _{b 1,} b 2, · · ·, receives the correspondence relationship between _{b p} from the predictor 35, and 32 changes the pattern thereof received, 32 The correspondence with the set coefficients b ₁ , b ₂ , ..., B _p is maintained.

FIG. 7 is a diagram showing a correspondence table between the transition pattern and the coefficients b ₁ , b ₂ , ..., B _p. With reference to FIG. 7, the correspondence table TBL1 includes a transition pattern and a set of coefficients. Transition patterns and coefficient sets are associated with each other.

Transition pattern 1, transition pattern 2, transition pattern 3, ..., Transition pattern m (m is the total number of transition patterns determined based on the number of subsets and the number of K layers) are set of coefficients b, respectively. ₁₁ , b ₁₂ , b ₁₃ , ..., b _1p , coefficient set b ₂₁ , b ₂₂ , b ₂₃ , ..., b _2p , coefficient set b ₃₁ , b ₃₂ , b ₃₃ , ..., b _3p , ..., Coefficient set b _m1 , b _m2 , b _m3 , ..., b _mp .

The prediction unit 34 holds the correspondence table TBL1. Then, when the prediction unit 34 receives the busy duration / idle duration (see FIG. 5) arranged in time series from the classification unit 33, the prediction unit 34 is arranged in the received time series with reference to the correspondence table TBL1. It is determined which of the transition patterns 1 to the transition pattern m stored in the correspondence table TBL1 the transition pattern of the busy duration / idle duration (see FIG. 5) matches, and the busy duration / idle duration is determined. A set of coefficients (b ₁₁ , b ₁₂ , b ₁₃ , ..., b _1p ; b ₂₁ , b ₂₂ , b ₂₃ , ..., b _2p ; ...; b _m1 , b _m2 , b _m3 , ..., b _mp ) is detected. Then, the prediction unit 34 outputs the set of the detected coefficients and the busy duration / idle duration (see FIG. 5) arranged in time series to the predictor 35.

After that, the prediction unit 34 receives the prediction busy duration / prediction idle duration from the predictor 35.

Q-learning in the learner 5 will be described. FIG. 8 is a conceptual diagram of the channel standby time.

With reference to FIG. 8, the time t _CPT is the time when the communication for transmitting the packet on any of the channels # 1 to # 4 is completed, and is set to the present time. Then, the length of the waiting time from the current time t _CPT to the next channel usage time (busy duration T _Busy ) is defined as the channel standby time WT.

In this case, in channel # 1, _{the length of the waiting time from the current time t CPT} to the next channel usage time (busy duration T _Busy1 ) is the channel standby time WT1, and in channel # 2, the waiting time is WT1. The length of the waiting time from the current time t _CPT to the next channel usage time (busy duration T _Busy2 ) is the channel standby time WT2, and in channel # 3, from the current time t _CPT , the next the length of the waiting time to the channel use time (busy duration _{T Busy3)} is a channel waiting time WT 3, in the channel # 4, the current time _{t CPT,} next channel use time (busy duration _{T Busy4} ) Is the channel standby time WT4.

The control means 4 detects the channel standby times WT1 to WT4 in each channel # 1 to channel # 4 based on the predicted channel free time and the predicted channel usage time received from the predicting means 3.

FIG. 9 is a conceptual diagram of the Q table. The length of the transmission time of the packet, for example, "long", since it can take three states s _t "medium" and "short", represented by "short" to "1", the "medium""2" It is represented by, and "length" is represented by "3".

“Long” means, for example, the length of the packet transmission time is 2 ms, “medium” means, for example, the length of the packet transmission time is 1 ms, and “short” means, for example, the length of the packet transmission time. The length is 0.5 ms.

Action a _t to select a channel to be used for communication, for example, selects the channel # 1, selects the channel # 2, and selects the channel # 3, and selects the channel # 4, the four actions a _t Since it is possible, selecting channel # 1 is represented by "1", selecting channel # 2 is represented by "2", selecting channel # 3 is represented by "3", and channel # 4 is represented by "3". The selection is represented by "4".

Therefore, the Q table is represented by a matrix of 3 rows × 4 columns and contains 12 Q values (= Q _{1, 1 to} Q _{3, 4} ).

The initial values of the 12 Q values (= Q _{1,1 to} Q _3,4 ) are all "0". Then, there is the terminal device 10 transmits time packet length state s _{t (=} 1 to 3 either), when acted a _t to select one of the channels # 1 to channel # 4, _{When the packet can be transmitted, the reward rt + 1} in the next communication is calculated based on the communication result (communication success / failure), the competing terminal information, and the channel waiting time WT.

The shorter the channel standby time WT shown in FIG. 8, the higher the channel utilization efficiency. Therefore, the shorter the channel standby time WT, the larger the communication throughput, and the longer the channel standby time WT, the higher the communication throughput. Weighting is performed based on the channel standby time WT so that

Further, the smaller the number of competing terminal devices, the smaller the competition and the larger the communication throughput, and the larger the number of competing terminal devices, the smaller the communication throughput. Weighting is performed based on the number of competing terminal devices so that the smaller the number, the higher the communication throughput, and the larger the number of competing terminal devices, the smaller the communication throughput.

As a result, the Q value in the Q table is updated by the following equation.

In equation (2), rt _{+ 1} is a reward, α is a learning rate, and γ is a discount rate.

In embodiments of the present invention, α is, for example, 0.5 and γ is, for example, 0.

_{The reward rt + 1} of the formula (2) is calculated by the following formula.

F of the formula (3A) is the throughput of communication. W ₁ of the formula (3A) is a weighting coefficient calculated by the formula (3B), and _NCFL of the formula (3B) is the number of competing terminal devices. As a result, the weighting coefficients w ₁ is smaller The more the number N _CFL competing terminal device, increases if less number N _CFL competing terminals.

_{Further, w 2} in the formula (3A) is a weighting coefficient calculated by the formula (3C), and WT in the formula (3C) is a channel standby time in each channel. As a result, the weighting coefficient w ₂ becomes larger as the channel waiting time WT becomes shorter, and becomes smaller as the channel waiting time WT becomes longer.

Further, v of the formula (3A) is composed of "1" when the communication is successful, and is composed of "0" when the communication is unsuccessful (see formula (3D)). Therefore, when the communication fails, the reward rt _{+ 1} becomes “0”.

As shown in equation (3), the learning unit 5 is greater if less number N _CFL competing terminal device calculates the reward r _{t + 1} to be smaller The more the number N _CFL competing terminals. _{Then, when the number N CFL} of the number of competing terminal devices is the first number, the learner 5 multiplies the throughput F by the _{weighting coefficient w 1} consisting of the first value, and _{the number N CFL} of the competing terminal devices is the first. the weighting coefficients w ₁ consisting of small second value than the first value when a larger second number than the number multiplying the throughput F.

Further, the learner 5 calculates the _{reward rt + 1} so that the channel standby time WT becomes larger as the channel standby time WT becomes shorter and the channel standby time WT becomes smaller as the channel standby time WT becomes longer. Then, when the channel standby time WT consists of the first time length, the learner 5 _{multiplies the throughput F by the weighting coefficient w 2} consisting of the third value, and the channel standby time WT is larger than the first time length. When it consists of a long second time length, the throughput is multiplied by a _{weighting factor w 2 consisting of a fourth value smaller than the third value.}

When the packet transmission is "impossible", the reward rt _{+ 1} becomes "0". This is because the throughput F cannot be calculated when the packet transmission is "impossible".

Learning unit 5, Q value corresponding to the state _{s t} and the action _{a t (= Q s, a} ) and, with the reward _{r t + 1,} the learning rate and α, and the discount rate γ are substituted into the formula (2) Update the Q value (= Q _{s, a).}

Then, when determining an action _{a t} of the terminal device 10, epsilon-greedy method is used. The epsilon-greedy method, the number epsilon (e.g., 0.3) small constant in advance to determine, when the generated random number is less than epsilon, the action a _t randomly determined, the generated random number is epsilon If not below, a method for determining an action a _t the action Q value is the maximum.

When the control means 4 controls the learner 5 to output the communication result, the competing terminal information (= the number of competing terminal devices _NCFL ) and the channel standby time WT to the learner 5 to execute Q-learning. , The channel standby times WT1 to WT4 shown in FIG. 8 are associated with channels # 1 to # 4, respectively, and output to the learner 5. Any Thus, the learning unit 5, when calculating a reward r _{t + 1} after determining the state s _t and action a _t in Q-learning, selected by the determined action a _t channels (= the channel # 1 to # 4 _{The weight coefficient w 2} can be calculated by the equation (3C) using the channel standby time WT (= any of the channel standby times WT1 to WT4) corresponding to (?).

10 to 12 are first to third schematic views for explaining a method of updating the Q table, respectively.

With reference to FIG. 10, in the initial state of Q-learning, all Q values (Q _{1, 1 to} Q _{3, 4} ) in the Q table are “0” (see (a) in FIG. 10).

In the embodiment of the present invention, when the length of the transmission time of the packet is "short", and a state s _t "1", when the length of the transmission time of the packet is "medium", the state s _{When t} is “2” and the length of the packet transmission time is “long”, the state _{st is set} to “3”.

Since the learner 5 has a plurality of lengths of "long", "medium", and "short" as the length of the packet transmission time, there are a plurality of states of "1", "2", and "3". You may select a s _t. Therefore. Learner 5 determines one state s _t at random from a plurality of state s _t. For example, the learning unit 5 shall be determined the state s _t "2".

Also, the learning unit 5 determines an action _{a t} in accordance with epsilon-greedy method. In this case, since all the Q values (Q _{1, 1 to} Q _{3, 4} ) in the Q table are "0" (that is, the maximum Q value is not determined to be one), the learner 5 is a random number. There irrespective of whether it is less than epsilon, determines an activity a _t of the terminal apparatus 10 randomly. Then, the learning unit 5, for example, it is assumed that the determined actions a _t "2".

Then, when the terminal device 10 can transmit the packet, the learner 5 determines the packet (t + 1) by the method described above based on the communication result (communication success / failure), the competing terminal information, and the channel waiting time. Calculate the reward rt _{+ 1 for the second transmission.}

After that, the learning unit 5, and the reward _{r t + 1,} which is calculated, learning rate and α, and the discount rate γ, Q value in the t-th transmission of the packet (state _s t (= "2") and the action _a t (= " The Q value in the initial state corresponding to 2 ”) is substituted into the equation ( _{2) to update the Q value to the Q value (q 2, 2} ) (see (b) in FIG. 10).

Here, the state s _t which is determined by the length of the transmission time of the _packet, and it is a behavior a _t selecting a channel and do not depend on the success of failure and communication of past communications. Further, the reward rt _{+ 1} is calculated not depending on the failure / success of the past communication but on the failure / success of the current communication. Therefore, the discount rate γ, which is a parameter for the cumulative reward, should be set to “0”.

As a result, the Q value in the t-th transmission of the packet is “0” (see (a) in FIG. 10), and γ = 0, so that the updated Q value (= q _{2, 2} ) is Substantially equal to _{αrt + 1.}

Subsequently, the learning unit 5, "long" as the length of the transmission time of the packet, based on the "medium" and "short", for example, to determine the "1" state s _t at random, epsilon-greedy It shall be determined to "4" the action a _t by law.

Then, when the learner 5 can transmit the packet, the reward r in the (t + 1) th transmission is performed by the method described above based on the communication result (communication success / failure), the competing terminal information, and the channel waiting time. Calculate _{t + 1.}

After that, the learning unit 5, and the reward _{r t + 1,} which is calculated, learning rate and α, and the discount rate γ (= 0), Q value in the t-th transmission of the packet (state _s t (= "1") and the action a _{Substituting t} (= Q value in the initial state corresponding to “4”) into equation (2 _{) to update the Q value to Q value (= q 1, 4} ) (see (a) in FIG. 11). ).

On the other hand, the learning unit 5, when the generated random number is not less than epsilon, it determines an activity a _t of the terminal device 10 in action Q value is the maximum. At this point, the Q table is in the state shown in FIG. 10B, so the maximum Q value is q ₂ and 2. Thus, the learning unit 5, the action _{a t} of the terminal device 10 "2" is determined to be (action to select a channel # 2).

_{Then, the learning device 5 calculates the reward rt + 1} when the terminal device 10 executes the action “2” (the action of selecting the channel # 2) by the method described above, _{and uses the calculated reward rt + 1} to formulate the formula. The Q value (= q _2, 2) is updated to the Q value (= q ' _{2, 2} ) according to (2) (see (b) in FIG. 11).

When the communication in the t-th transmission of the packet is unsuccessful, v in the equation (3A) is set to "0" (see equation (3D)), so the reward rt _{+ 1} becomes "0". .. Therefore, when the communication in the t-th transmission of the packet is unsuccessful, the learner 5 _{calculates the reward rt + 1} in the (t + 1) th transmission of the packet as “0” and updates the Q value.

After that, the learner 5 repeatedly executes the above-described operation until the end condition is satisfied, and updates the Q value in the Q table. The end condition is, for example, when the above-mentioned Q value update is executed a predetermined number of times. The predetermined number of times is, for example, 10 times.

Learning unit 5, when the end condition is satisfied, with reference to the Q table when the termination condition is satisfied (see FIG. 12), the corresponding q value the state s _t of "1" (= q _{1, 1} , Q _1,3 , q _1,4 ), the maximum Q value (= q _{1_max} ) and the action _{at_max 1} corresponding to the maximum Q value (= q _{1_max} ) are detected, and the state of "2". q value corresponding to _{s t (= q 2,2, q} 2,3) maximum Q value among the _{(= q 2_max),} maximum Q value and the _{(= q 2_max)} corresponding to act _{a T_max2} detected, "3" q values corresponding to the state _{s t} of _{(= q 3,2,} q _3,4) maximum Q value among the _{(= q 3_max),} maximum Q value _{(= q 3_max)} The action _{at_max3} corresponding to is detected. The generated learning device 5, "1" maximum Q value in state _{s t} of _{(= q 1_max)} and associates the action _{a t_max1} [ "1" state _s t _{/ q} 1_max _{/ a t_max1} of] and generates a "2" maximum Q value in state _{s t} of _{(= q 2_max)} and associates the action _{a t_max2} [ "2" state _s t _{/ q} 2_max _{/ a t_max2} of, "3" generating a maximum Q value in state _{s t} of _{(= q 3_max)} and associates the action _{a t_max3} [ "3" state _s t _{/ q} 3_max _{/ a t_max3} of.

Then, the learning unit 5, [ "1" state _s t _/ q _{1_max / a t_max1} of] [ "2" state _s t _/ q _{2_max / a t_max2} of] and [ "3" state _s t _{/ q 3_max} of / _{At_max3} ] outputs the output information IF_OUT to the control means 4.

Control means 4, the output information IF_OUT (= [ "1" state _s t _/ q _{1_max / a t_max1} of] [ "2" state _s t _/ q _{2_max / a t_max2} of] and [ "3" state _{s t} of / Q _{3_max} / _{at_max3} ]) is received from the learner 5.

Then, the control unit 4, when the state _{s t} corresponds to the length of the transmission time of the transmission packet is a state _{s t} "1", the output information from the IF_OUT in the "1" state _s t _{/ q 1_max /} to detect _a t_max1], to detect the action _{a t_max1} from the detected [ "1" state _s t _/ q _{1_max / a t_max1} of]. Then, the control unit 4 selects action _a channel selected by _{T_max1} (channel # 1, channel # 3, and either channel # 4) as a transmission channel CH_T.

Further, the control unit 4, when the state _{s t} corresponds to the length of the transmission time of the transmission packet is a state _{s t} "2", the output information from the IF_OUT in the "2" state _s t _{/ q 2_max /} to detect _a t_max2], to detect the action _{a t_max2} from the detected [ "2" state _s t _/ q _{2_max / a t_max2} of]. Then, the control unit 4 selects a channel selected by the action _{a t_max2} (channel # 2, and one of the channels # 3) as a transmission channel CH_T.

Furthermore, the control unit 4, when the state _{s t} corresponds to the length of the transmission time of the transmission packet is a state _{s t} "3", the output information from the IF_OUT of [ "3" state _s t _{/ q 3_max /} to detect _a t_max3], to detect the action _{a t_max3} from the detected [ "3" state _s t _/ q _{3_max / a t_max3} of]. Then, the control unit 4 selects a channel selected by the action _{a t_max3} (channel # 2, and either channel # 4) as a transmission channel CH_T.

FIG. 13 is a diagram for explaining a method of determining whether or not the transmission packet can be transmitted.

Referring to FIG. 13, in the current time _T time future than _CPT, the channel # 1, channel exists idle time _{T Idle1,} in the channel # 2, there is a channel idle time _{T Idle2,} channel # In 3, the channel free time _TIdle3 exists, and in channel # 4, the channel free time _TIdle4 exists.

The channel free time _TIdle1 is shorter than the length of the transmission time of the transmission packet. The channel free time T _Idle2 , T _{Idle 3} , and T _{Idle 4} are longer than the length of the transmission time of the transmission packet. The channel free time _TIdle2 is the shortest among the channel free time _TIdle2 , _TIdle3 , and _TIdle4.

When the control means 4 selects, for example, the transmission channel CH_T (any of channels # 1 to # 4), the control means 4 sets the carrier sense in the selected transmission channel CH_T (any of channels # 1 to # 4). When the receiving means 2 is controlled to execute and then the carrier sense result is received from the receiving means 2, another terminal device receives the carrier sense result, and another terminal device performs transmission channel CH_T (channels # 1 to channel # 4). It is determined whether or not communication is performed by either).

When the control means 4 determines that the other terminal device is not communicating on the transmission channel CH_T, the predicted channel free time in the transmission channel CH_T (any of channel # 1 to channel # 4) is used for transmission. Determine if the packet is longer than the transmission time.

Then, when the control means 4 determines that the predicted channel free time in the transmission channel CH_T (any of channel # 1 to channel # 4) is longer than the length of the transmission time of the transmission packet, the transmission channel CH_T And the transmission packet is output to the transmission means 7, and the transmission means 7 is controlled so that the transmission packet is transmitted to the base station BS on the transmission channel CH_T.

On the other hand, when the control means 4 determines that the predicted channel free time in the transmission channel CH_T (any of channel # 1 to channel # 4) is equal to or less than the length of the transmission time of the transmission packet, or when the carrier sense determines. When it is determined that another terminal device is communicating on the transmission channel CH_T based on the result, it is determined that the transmission of the transmission packet is impossible, and the communication prohibition information is generated.

In the case shown in FIG. 13, the channel free state _TIdle2 (= predicted channel free time) in channel # 2 is longer than the length of the transmission time of the transmission packet.

Therefore, the control means 4 determines that the channel free time _TIdle2 (= predicted channel free time) is longer than the length of the transmission time of the transmission packet.

Then, the control means 4 outputs the transmission channel CH_T (= channel # 2) and the transmission packet to the transmission means 7, and transmits the transmission packet to the base station BS on the transmission channel CH_T (= channel # 2). The transmission means 7 is controlled so as to.

As described above, the determination as to whether or not the transmission packet can be transmitted consists of the following two.
(Determination 1) Based on the result of carrier sense in the transmission channel CH_T, it is determined whether or not another terminal device is communicating on the transmission channel CH_T.
(Determination 2) It is determined whether or not the predicted channel free time in the transmission channel CH_T is longer than the transmission time of the transmission packet.

Then, when the two determination results of the determinations 1 and 2 are positive determination results, it is determined that the transmission packet can be transmitted, and the determination result is negative in at least one determination of the determinations 1 and 2. When it is the target determination result, it is determined that the transmission packet cannot be transmitted.

In the embodiment of the present invention, when it is determined only by the determination 2 that the predicted channel free time in the transmission channel CH_T is longer than the transmission time of the transmission packet, the packet is transmitted without performing backoff. The credit packet may be transmitted to the base station BS on the transmission channel CH_T.

FIG. 14 is a flowchart for explaining the operation of the terminal device 10 shown in FIG. With reference to FIG. 14, when the operation of the terminal device 10 is started, the control means 4 determines whether or not there is a transmission packet by determining whether or not a transmission packet has been received from the application 6. (Step S1). In this case, when the control means 4 receives the transmission packet from the application 6, it determines that there is a transmission packet, and when it does not receive the transmission packet from the application 6, it determines that there is no transmission packet.

In step S1, if it is determined that there is a transmission packet, the control unit 4 determines based on the output information IF_OUT output from the learning unit 5 act a _{t (action} to select a channel) (step S2) ..

Then, the control unit 4 controls the receiving unit 2 to perform carrier sensing in the selected channel in the determined action a _t, the receiving means 2, the carrier sense carried out in accordance with control of the control unit 4 (step S3 ).

After that, the prediction means 3 predicts the predicted channel free time and the predicted channel usage time in each channel by the method described above (step S4), and outputs the predicted predicted channel free time and the predicted channel usage time to the control means 4. do.

The control means 4 receives the prediction channel free time and the prediction channel usage time from the prediction means 3.

Subsequently, the control means 4 determines whether or not the transmission packet can be transmitted (step S5). In this case, the control means 4 determines whether or not the transmission packet can be transmitted by the determinations 1 and 2 described above.

When it is determined in step S5 that the transmission packet cannot be transmitted, the control means 4 generates the transmission non-transmission information (step S6).

On the other hand, in step S5, when the transmission packet is determined to be transmittable, the control unit 4, the selected channel and a transmitting channel CH_T in action a _t determined in step S2, the transmission channel CH_T The transmission means 7 is controlled so as to output the transmission packet to the transmission means 7 and transmit the transmission packet to the base station BS on the transmission channel CH_T.

The transmission means 7 receives the transmission channel CH_T and the transmission packet from the control means 4, and transmits the transmission packet to the base station BS on the transmission channel CH_T via the antenna 1 according to the control from the control means 4 (the transmission means 7 receives the transmission channel CH_T and the transmission packet from the control means 4. Step S7).

After that, the control means 4 determines whether or not the ACK packet has been received from the base station BS (step S8). In this case, when the control means 4 receives the ACK packet from the receiving means 2, it determines that the ACK packet has been received from the base station BS, and when it does not receive the ACK packet from the receiving means 2, the control means 4 receives the ACK packet from the base station BS. It is determined that the packet was not received from.

When it is determined in step S8 that the ACK packet has been received from the base station BS, the control means 4 sets the communication result to "1" (step S9).

On the other hand, when it is determined in step S8 that the ACK packet has not been received from the base station BS, the control means 4 sets the communication result to "0" (step S10).

Then, after step S9 or step S10, the control means 4 acquires competing terminal information from the base station BS using the control channel (step S11).

Subsequently, after step S6 or step S11, the control means 4 outputs competing terminal information, transmission non-transmission information, communication result, and channel free time as input information to the learner 5, so that the learner 5 performs Q-learning. To control.

The learner 5 receives competing terminal information, non-transmission information, communication result, and channel free time as input information, and executes Q-learning using the received competing terminal information, non-transmission information, communication result, and channel standby time. (Step S12).

Then, the learner 5 outputs the output information IF_OUT, which is the result of Q-learning, to the control means 4. The control means 4 receives the output information IF_OUT from the learner 5.

After that, the control means 4 determines whether or not to terminate the communication (step S13). When it is determined in step S13 that the communication is not terminated, the series of operations shifts to step S1. After that, in step S13, steps S1 to S13 are repeatedly executed until it is determined that the communication is terminated.

Then, in step S13, when it is determined that the communication is terminated, the operation of the terminal device 10 ends.

FIG. 15 is a flowchart for explaining the detailed operation of step S5 of FIG.

With reference to FIG. 15, after step S4 of FIG. 14, the control means 4 determines whether or not another terminal device is communicating on the transmission channel CH_T based on the result of carrier sense (step). S51).

In step S51, when it is determined in the transmission channel CH_T that no other terminal device is communicating, the control means 4 is based on the predicted channel free time and the predicted channel usage time in each channel received from the predicting means 3. Therefore, it is determined whether or not the predicted channel free time in the transmission channel CH_T is longer than the transmission time of the transmission packet (step S52).

Then, in step S51, when it is determined that another terminal device is communicating on the transmission channel CH_T, or in step S52, the predicted channel free time on the transmission channel CH_T is the transmission time of the transmission packet. When it is determined that the length is not longer than the length, the control means 4 determines that the transmission of the transmission packet is impossible (step S53).

On the other hand, in step S52, when it is determined that the predicted channel free time in the transmission channel CH_T is longer than the length of the transmission time of the transmission packet, the control means 4 determines that the transmission packet can be transmitted. (Step S54).

Then, after step S53, the series of operations shifts to step S6 of FIG. 14, and after step S54, the series of operations shifts to step S7 of FIG.

FIG. 16 is a flowchart for explaining the detailed operation of step S12 of FIG.

With reference to FIG. 16, after step S6 or step S11 of FIG. 14, the learner 5 receives input information from the control means 4 (step S121).

Then, the learning unit 5 determines randomly a state _{s t} at transmission of the t-th packet (step S122).

Then, the learning unit 5, based on the epsilon-greedy method, determines an activity _{a t} in the transmission of the t-th packet (step S123).

Subsequently, the learning unit 5, competing terminal information (e.g., conflict terminal number) is calculated weighting coefficients _{w 1} by the formula (3B) based on (step S124).

Then, the learning unit 5 calculates the weighting coefficient _{w 2} by the formula (3C) based on the channel waiting time WT (step S125).

afterwards. The learner 5 determines whether or not the non-transmissible information is included in the input information (step S126).

In step 126, when it is judged that the transmission failure information is not included in the input information, the learning unit 5, a reward r _{t + 1} in the transmission of the (t + 1) th packet when executing an action a _t in state s _t It calculated by equation (3A) by using the weighting coefficients _{w 1} and the weight coefficient _{w 2} (step S127).

On the other hand, in step 126, when it is judged that the transmission prohibition information is included in the input information, the learning unit 5, compensation in the transmission of the (t + 1) th packet when executing an action a _t in state s _t r _{t + 1} Is calculated as zero (step S128).

After step S127 or step S128, the learning unit 5, the learning rate alpha, discount rate gamma (= 0) and with the reward _{r t + 1,} the Q value of the Q table corresponding to the state _{s t} and action _{a t} Update (step S129).

Then, the learner 5 determines whether or not the Q-learning end condition is satisfied (step S130). The end condition is, for example, that the number of updates of the Q value has reached 10 times.

When it is determined in step S130 that the Q-learning end condition is not satisfied, the series of operations shifts to step S122. After that, in step S130, steps S122 to S130 are repeatedly executed until it is determined that the Q-learning end condition is satisfied.

Then, in step S130, the termination condition of the Q-learning is determined to be satisfied, the learning device 5, when the respective states s _t, and the maximum Q value in each state s _t, the maximum Q value is obtained outputs the output information IF_OUT consisting correspondence between the action _{a t} to the control means 4 (step S131). After that, the series of operations shifts to step S13 of FIG.

By repeatedly executing "NO", S127, and S129 in steps S122 to S125 and S126 shown in FIG. 16, the reward rt _{+ 1} becomes large when the number of competing terminals is small and / or when the channel standby time WT is short. Then, the Q value are determined by the substantially reward r _{t + 1,} the reward r _{t + 1} is large, Q values corresponding to the state s _t and action a _t is updated to a larger Q value.

As a result, the small number of competing terminals and the short channel standby time WT increase the Q value through increasing the _{reward rt + 1.}

Therefore, a small number of competing terminals reduces the probability that the terminal device 10 will become a competing terminal with respect to other terminal devices. As a result, the packet collision probability can be reduced.

Further, the short channel waiting time WT increases the probability that a channel having a short channel waiting time WT is selected. As a result, channel utilization efficiency can be improved.

If step S2 of FIG. 14 is executed for the first time, the learning unit 5 determines randomly action a _t (i.e., to select a channel at random).

Then, the learning unit 5, step S12 performs a Q learning in (the flowchart shown in FIG. 16), act upon the respective states s _t, and the maximum Q value in each state s _t, the maximum Q value is obtained outputs the output information IF_OUT consisting correspondence between a _t to the control means 4.

FIG. 17 is a conceptual diagram of the output information IF_OUT. Referring to FIG. 17, the output information IF_OUT includes a state _{s t,} and the maximum Q value among the Q value corresponding to the state _{s t,} the action _{a t} when the maximum of Q value is obtained.

State s _t, a maximum of Q value, and action a _t when the maximum of Q value is obtained from among the Q value corresponding to the state s _t is correlated with one another.

The Q-table at the end of Q-learning in step S12 (flow chart shown in FIG. 16) includes the Q-table TBL_Q shown in FIG.

Learner 5, the maximum of the determines that the termination condition of the Q-learning are satisfied, "1" Q value corresponding to the state _{s t} of _{(= q 1,1, q 1,3,} q 1,4) the Q value (for _{example, q 1, 3)} and the maximum Q value _{(e.g., q 1, 3)} for detecting a behavior when the obtained _{a t (= "3")} .

Then, the learning unit 5 includes a state _{s t} "1", the maximum Q value _{(e.g., q 1, 3)} and, action _a t (= "3") and the mutual association with output information IF_OUT the Store.

Also, the learning unit 5, Q values corresponding to the state _{s t} of _{"2" (= q 2,2,} q 2,3) maximum Q value among the _{(e.g., q 2, 2)} and the maximum Q value _{(e.g., q 2, 2)} act _a t when the obtained (= "2") and detected.

Then, the learning unit 5, "2" and state _{s t} of the maximum Q value _{(e.g., q 2, 2)} and, action _a t (= "2") and the mutually correlated output information IF_OUT the Store.

Furthermore, the learning unit 5, Q values corresponding to the state _{s t} of _{"3" (= q 3,2,} q 3,4) maximum Q value among the _{(e.g., q 3, 4),} the largest Q value _{(e.g., q 3, 4)} actions _a t when the obtained (= "4") and detected.

Then, the learning unit 5 includes a state _{s t} "3", the maximum Q value _{(e.g., q 3, 4)} and, action _a t (= "4") and in association with one another output information IF_OUT the Store.

The control means 4 receives the output information IF_OUT from the learner 5. Then, the control unit 4, in step S2 after step S12 (the flowchart shown in FIG. 16) is executed, as follows to determine the action a _t (i.e., selects a transmission channel CH_T).

State control means 4, the state _s t of "1" s contained in the output information IF_OUT, from the state _{s t} "2" state _{s t} and "3" of which is determined by the length of the transmission time of the transmission packet The _{state st} that matches _s't is detected.

For example, the control unit 4 detects the state _{s t} "1" as the state _{s t} matching state s' _t. Then, the control means 4, "1" maximum Q value associated with state _{s t} of _{(= q 1_max (= q 1,3} )) and action _{a t (= a t_max1 (=} "3")) Is detected.

Then, the control unit 4 selects "3" to channel # 3 selected in action _{a t} as a transmission channel CH_T.

Even when the state s _t match the state s' _t is the state of "2" state or "3", the control unit 4, similarly, select a channel # 2 or channel # 4 as a transmission channel CH_T do.

By transmitting the packet according to the flowchart shown in FIG. 14 (including the flowchart shown in FIGS. 15 and 16), when the transmission signal is not detected by the carrier sense in the initial stage of learning by the learner 5 in step S12, Or because of random access, if the transmission timing happens to be the same, packet collision will occur, but such packet collision is a necessary cost in finally selecting the optimum channel. As learning progresses, packet collisions will decrease, so this is not a problem.

Further, in the flowchart shown in FIG. 14 (including the flowchart shown in FIGS. 15 and 16), the reward rt _{+ 1} becomes smaller as the number of competing terminal devices increases (see equations (3A) and (3B)). The probability that the terminal device 10 becomes a competing terminal with respect to other terminal devices is reduced.

Therefore, it is possible to reduce the probability that a plurality of terminal devices will try to connect to the same free channel as compared with the case where the channel is selected only by predicting the free channel. As a result, packet loss can be suppressed and the frequency can be effectively used.

The operation of the terminal device 10 may be realized by software. In this case, the terminal device 10 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). Then, the ROM stores a program Prog_A including each step of the flowchart shown in FIG. 14 (including the flowchart shown in FIGS. 15 and 16).

The CPU reads the program Prog_A from the ROM, executes the read program Prog_A, selects a channel having a predicted channel free time that matches the length of the transmission time of the transmission packet as the transmission channel CH_T, and bases the packet. Send to station BS. The RAM temporarily stores the calculated weighting factors w ₁ , w ₂ , the calculated reward rt _{+ 1} , and the updated Q value.

Further, the program Prog_A may be recorded on a recording medium such as a CD or DVD and distributed. When the recording medium on which the program Prog_A is recorded is attached to the computer, the computer reads the program Prog_A from the recording medium, executes the program, and sends a channel having a predicted channel free time that matches the length of the transmission time of the transmission packet. The packet is transmitted to the base station BS by selecting it as the credit channel CH_T.

Therefore, the recording medium on which the program Prog_A is recorded is a computer-readable recording medium.

In the above, it has been described that the competing terminal information consists of the number of competing terminals, but in the embodiment of the present invention, the competing terminal information is not limited to this, and the competing terminal information is an expected value of the packet length occupying the channel. It may be the number of terminal devices having a packet length having a tendency similar to the packet length of the packet to be transmitted by the terminal device 10 itself.

The expected value of the packet length occupying the channel is the average value of the packet lengths when the packets transmitted from the terminal devices other than the terminal device 10 occupy the channel. Then, the expected value of the packet length occupying the channel is acquired in the base station BS, and is transmitted from the base station BS to the terminal device 10 via the control channel as competing terminal information.

Further, the number of terminal devices having a packet length having a tendency similar to the packet length of the packet to be transmitted by the terminal device 10 itself is acquired by the base station BS, and is obtained from the base station BS as competing terminal information from the base station BS to the terminal device 10 via a control channel. Will be sent.

When competing terminal information is from the expected value of the packet length occupying a channel, the weighting coefficients w ₁ is calculated as the inverse of the expected value of the packet length occupying a channel. Further, when a conflict terminal information consists of the number of terminal devices having a packet length of trends similar to the packet length of a packet to be transmitted for the terminal apparatus 10 itself, the weighting coefficients w _1, the packet of the packet to be transmitted for the terminal apparatus 10 itself It is calculated as the inverse of the number of terminal devices having a packet length that tends to be similar to the length.

When the weighting coefficient w ₁ is calculated as the reciprocal of the expected value of the packet length occupying the channel, or the reciprocal of the number of terminal devices having a packet length having a tendency similar to the packet length of the packet to be transmitted by the terminal device 10 itself. When calculated as, it is possible to suppress competition with a terminal device having a high probability of selecting a channel similar to that of the terminal device 10.

Further, in the above, the weight coefficient w ₂ was calculated using the channel standby time WT, but the embodiment of the present invention is not limited to this, and the reciprocal of the channel standby time WT is calculated as the _{reward rt + 1.} May be good. In this case, the reward rt _{+ 1 is calculated by the following equation in which w 2} = 0 in the equation (3A) and the throughput F is changed to the reciprocal of the channel standby time WT.

By setting the reciprocal of the channel waiting time WT as the reward rt _{+ 1} , the shorter _{the channel waiting time WT, the larger the reward rt + 1} , and the longer the channel waiting time WT, the smaller the _{reward rt + 1.} As a result, the channel that improves the utilization efficiency of the channel can be used as the transmission channel CH_T.

Further, in the embodiment of the present invention, when the carrier sense is performed on each of the channels # 1 to # 4, the identification information ID (for example, the MAC address of the wireless LAN) of the terminal device of the source is detected and the channel # 1 is detected. The transmission channel CH_T may be selected according to the number of source terminal devices detected for each of ~ # 4 and the transmission packet length of the terminal device 10.

In this case, the control means 4 selects the transmission channel CH_T in consideration of the number of transmission source terminal devices detected for each of channels # 1 to # 4, in addition to the output information IF_OUT from the learner 5 described above.

Further, the transmission channel CH_T may be selected according to the number of transmission source terminal devices detected for each of channels # 1 to # 4 and the required transmission delay.

More specifically, if it is a transmission packet that can be transmitted immediately, the number of source terminal devices detected for each channel is small, and the predicted value of the channel free time is the terminal device 10. The transmission channel CH_T may be selected from the channels having the transmission packet length or more.

In this case, the control means 4 selects the transmission channel CH_T in consideration of the fact that the transmission packet could be transmitted immediately in addition to the output information IF_OUT from the learner 5 described above.

As a result, as the number of terminals communicating on each channel increases, the variance such as packet length and packet transmission interval increases, and the autocorrelation decreases, so that the accuracy of the predicted value by the predicting means 3 deteriorates, and the accuracy of the predicted value by the predicting means 3 deteriorates. It can solve the problem that it is difficult to obtain the effect.

Note that the decrease in autocorrelation means that when packets are transmitted periodically, the packet transmission interval deviates significantly with the passage of time as compared with the transmission interval of its own first packet.

Further, in the above, the weighting coefficients w ₁ has been described as being calculated as the reciprocal of the number of competing terminal device, in the embodiment of the present invention is not limited thereto, the weight coefficient w ₁ is competing terminal Any calculation formula may be used as long as it is calculated so that it increases as the number of devices decreases and decreases as the number of competing terminal devices increases.

Further, in the above, it has been explained that the weighting coefficient w ₂ is calculated as the reciprocal of the channel waiting time, but in the embodiment of the present invention, the weighting coefficient w ₂ is not limited to this, and the weighting coefficient w 2 is the channel waiting time. Any calculation formula may be used as long as it is calculated so that it becomes smaller as the length becomes longer and increases as the channel standby time becomes shorter.

In the embodiment of the present invention, the control means 4 for selecting the transmission channel CH_T based on the length of the transmission time of the transmission packet and the output information IF_OUT from the learner 5 constitutes the "selection means". ..

Further, in embodiments of the present invention, the process of calculating a reward r _{t + 1} by multiplying the weighting coefficients w ₁ throughput F constitutes a "first processing", the weight coefficient w ₂ throughput F The process of multiplying to _{calculate the reward rt + 1} constitutes a "second arithmetic process".

Furthermore, in embodiments of the present invention, the weighting coefficients w ₁ constitutes "a first weighting factor", the weight coefficient w ₂ constitutes "the second weighting coefficient".

Further, in the embodiment of the present invention, the communication result "1" constitutes the "first index", and the communication result "0" constitutes the "second index".

According to the above-described embodiment, the following effects can be obtained.

Since the channel is selected so that a high throughput can be obtained according to the packet length to be transmitted, the frequency utilization efficiency of the channel can be improved. Further, even when the terminal that transmits packets having different transmission packet lengths and the transmission of the next packet compete with each other, they do not become competitors, and as a result, the packet collision probability can be suppressed.

Further, by selecting the channel according to the number of terminal devices observed for each channel, it is possible to suppress the retransmission caused by the packet collision and suppress the delay.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown by the scope of claims rather than the description of the embodiment described above, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

The present invention applies to terminal devices, programs to be executed by a computer, and a computer-readable recording medium on which the programs are recorded.

1 antenna, 2 receiving means, 3 predicting means, 4 controlling means, 5 learner, 6 application, 7 transmitting means, 10 terminal device, 31 judgment unit, 32 measuring unit, 33 classification unit, 34 predicting unit, 35 predictor, 100 communication system.

Claims (21)

A predictive means for predicting the channel free time, which is the time when the channel is free, and the channel use time, which is the time when the channel is used, for all of the plurality of channels.
The communication result of the communication for transmitting the packet to the base station, the competing terminal information which is the information of the competing terminal device competing with the terminal device, and the waiting of the channel from the transmission of one packet to the transmission of the next packet. Based on the input information consisting of the channel standby time, which is the time, or the input information consisting of the non-transmission information indicating that the packet cannot be transmitted to the base station, the length of the packet transmission time is set as the state and the channel is selected. Q learning is executed with the throughput when communication is successful or the inverse of the channel standby time as a reward, and the Q value is updated repeatedly until the end condition of the Q learning is satisfied. Output information consisting of a configuration in which each state when the end condition of Q learning is satisfied, the maximum Q value in each state, and the action when the maximum Q value is obtained in each state are interrelated. A learner that executes learning processing that outputs
The action when the maximum Q value corresponding to the Q learning state corresponding to the length of the transmission time of the packet to be transmitted is obtained is detected from the output information, and the channel selected by the detected action is sent. Choices to choose as a credit channel and
When the first condition of the transmission channel, which consists of the channel free time predicted by the prediction means being longer than the transmission time of the packet to be transmitted, is satisfied, the backoff is not executed. A terminal device including a transmission means for transmitting a packet to be transmitted to the base station on the transmission channel. When the competing terminal information consists of the number of competing terminal devices and the communication result indicates success of communication, the learning device becomes larger as the number of competing terminal devices decreases. The first arithmetic process for calculating the reward is executed so that the number becomes smaller as the number increases, and the reward becomes larger as the channel waiting time becomes shorter and smaller as the channel waiting time becomes longer. The terminal device according to claim 1, wherein the second arithmetic process to be calculated is executed to execute the learning process. When the number of competing terminal devices is the first number, the learner multiplies the throughput by a first weighting coefficient consisting of the first value, and the number of competing terminal devices is calculated from the first number. The second aspect of claim 2, wherein the first arithmetic processing is executed by multiplying the throughput by the first weighting coefficient consisting of a second value smaller than the first value when the number of the second number is large. Terminal device. The terminal device according to claim 3, wherein the learning device executes the first arithmetic processing by multiplying the throughput by the reciprocal of the number of competing terminal devices as the first weighting factor. When the channel waiting time consists of the first time length, the learner multiplies the throughput by a second weighting coefficient consisting of a third value, and the channel waiting time is longer than the first time length. 2. The second arithmetic processing is executed by multiplying the throughput by the second weighting coefficient consisting of a fourth value smaller than the third value when the second time length is long. The terminal device according to any one of claims 4. The second aspect of the present invention, wherein the learner executes the second arithmetic processing by multiplying the throughput by the reciprocal of the channel standby time as the second weighting factor. Terminal device. The terminal device according to claim 1, wherein the learning device updates the Q value with the reward as zero when the input information consists of the non-transmissionable information in the learning process. The terminal device according to any one of claims 1 to 7, further comprising a receiving means for receiving the competing terminal information from the base station on a control channel. The communication result is composed of the first index when the transmitting means transmits the packet to be transmitted to the base station and then receives an ACK packet indicating that the packet has been received from the base station. The terminal according to any one of claims 1 to 8, which comprises a second index when the means does not receive the ACK packet from the base station after transmitting the packet to be transmitted to the base station. Device. When, in addition to the first condition, the transmission means satisfies the second condition, which comprises that, as a result of carrier sense in the transmission channel, another terminal device is not communicating in the transmission channel. The terminal device according to any one of claims 1 to 9, wherein the packet to be transmitted is transmitted to the base station through the transmission channel without executing backoff. The first step in which the predictor predicts the channel free time, which is the time when the channel is free, and the channel use time, which is the time when the channel is used, for all of the plurality of channels.
From the transmission of one packet to the transmission of the next packet, the communication result of the communication in which the learner transmits the packet to the base station, the competing terminal information which is the information of the competing terminal device competing with the terminal device, and the competing terminal information. Based on the input information consisting of the channel standby time, which is the standby time of the channel, or the input information consisting of the non-transmission information indicating that the packet cannot be transmitted to the base station, the length of the packet transmission time is set as the state. , The action is to select a channel, Q learning is executed with the throughput when communication is successful or the inverse of the channel standby time as a reward, and the update of the Q value is repeated until the end condition of the Q learning is satisfied. A configuration in which each state when the end condition of the Q learning is satisfied, the maximum Q value in each state, and the action when the maximum Q value is obtained in each state are associated with each other. The second step of executing the learning process to output the output information consisting of
The selection means detects an action when the maximum Q value corresponding to the Q learning state corresponding to the length of the transmission time of the packet to be transmitted is obtained from the output information, and is selected by the detected action. The third step of selecting the channel for transmission and
When the transmitting means satisfies the first condition that the channel free time predicted by the predicting means in the transmitting channel is longer than the length of the transmitting time of the packet to be transmitted, the backoff is executed. A program for causing a computer to execute a fourth step of transmitting a packet to be transmitted to the base station on the transmission channel without using the packet. In the second step, when the competing terminal information consists of the number of competing terminal devices and the communication result indicates success of communication, the learning device becomes larger as the number of competing terminal devices decreases. Therefore, the first arithmetic process for calculating the reward is executed so that the number of competing terminal devices increases, and the channel standby time increases as the channel standby time increases, and the channel standby time increases as the channel standby time increases. The program for causing the computer according to claim 11 to execute the learning process by executing the second arithmetic process for calculating the reward so as to be small. In the second step, the learner multiplies the throughput by a first weighting coefficient consisting of a first value when the number of competing terminal devices is the first number, and the number of competing terminal devices. Is a second number greater than the first number, the first weighting factor consisting of a second value smaller than the first value is multiplied by the throughput to execute the first arithmetic processing. The program for causing the computer according to claim 12 to execute. The thirteenth aspect of claim 13, wherein the learner executes the first arithmetic processing by multiplying the throughput by the reciprocal of the number of competing terminal devices as the first weighting coefficient in the second step. A program that lets a computer run. In the second step, when the channel standby time consists of the first time length, the learner multiplies the throughput by a second weighting coefficient consisting of a third value, and the channel standby time When the second time length is longer than the first time length, the second weighting coefficient consisting of the fourth value smaller than the third value is multiplied by the throughput to perform the second arithmetic processing. The program for causing the computer according to any one of claims 12 to 14 to execute the above. Claims 12 to 15 wherein the learner executes the second arithmetic processing by multiplying the throughput by the reciprocal of the channel standby time as the second weighting factor in the second step. A program for causing the computer according to any one of the items to be executed. The learning device causes the computer according to claim 11 to update the Q value with the reward as zero when the input information comprises the non-transmissible information in the learning process of the second step. Program for. The computer according to any one of claims 11 to 17, wherein the receiving means further causes a computer to perform a fifth step of receiving the competing terminal information from the base station on a control channel. program. The communication result comprises a first index when the transmitting means transmits a packet to be transmitted to the base station and then receives an ACK packet indicating that the packet has been received from the base station. The computer according to any one of claims 11 to 18, which comprises a second index when the means does not receive the ACK packet from the base station after transmitting the packet to be transmitted to the base station. A program to be executed by. When, in addition to the first condition, the transmission means satisfies the second condition, which comprises that, as a result of carrier sense in the transmission channel, another terminal device is not communicating in the transmission channel. The program for causing the computer according to any one of claims 11 to 19 to transmit the packet to be transmitted to the base station through the transmission channel without executing backoff. A computer-readable recording medium on which the program according to any one of claims 11 to 20 is recorded.

Images