JP2022150063A - Terminal device, program to be executed by computer, and computer-readable recording medium having program recorded therein - Google Patents

Abstract

To provide a terminal device that performs wireless communication in coexistence with a terminal device for performing wireless communication using a different wireless communication system.SOLUTION: A learning unit 4 repeatedly performs learning based on the state of a transmission channel during an observation period which is a period during which a packet communication result, an idle period of wireless communication, and the presence or absence of wireless communication by other terminal devices is observed, selects a channel bringing a maximum average reward as a transmission channel with a given probability, and selects a packet length bringing the maximum average reward with a given probability according to the state of the transmission channel during the observation period. Control means 3 generates a packet containing transmission data, and outputs the generated packet to transmission means 5 when the transmission channel received from the learning unit 4 is empty. The transmission means 5 transmits the packet received from the control means 3 with a packet length received from the learning unit 4.SELECTED DRAWING: Figure 2

Description

There is an application for exception to loss of novelty

The present invention relates to a terminal device, a program to be executed by a computer, and a computer-readable recording medium recording the program.

In the CSMA/CA (Carrier Sense Multiple Access/Collision Avoidance) method represented by wireless LAN (Local Area Network), etc., when packet collision or loss occurs due to simultaneous transmission, the backoff time (transmission from other stations Packet collision probability is reduced by lengthening the intentional waiting time until transmission by the own station instead of immediately transmitting after detecting that radio waves have stopped (Patent Document 1).

JP 2006-013894 A

However, when wireless communications by a plurality of different wireless communication systems coexist, terminal device A that performs wireless communication using each wireless communication system communicates wirelessly with terminal device B that performs wireless communication using another wireless communication system. Unbridled wireless communication results in packet collisions. As a result, there is a problem that it is difficult for the terminal device A to coexist with the terminal device B and perform wireless communication.

Therefore, according to the embodiment of the present invention, a terminal device that performs wireless communication coexisting with a terminal device that performs wireless communication using a different wireless communication system is provided.

Further, according to the embodiment of the present invention, there is provided a program for causing a computer to perform wireless communication coexisting with a terminal device that performs wireless communication using a different wireless communication system.

Furthermore, according to the embodiment of the present invention, there is provided a computer-readable recording medium recording a program for causing a computer to perform wireless communication coexisting with a terminal device that performs wireless communication using a different wireless communication system. do.

(Configuration 1)
According to an embodiment of the present invention, a terminal device comprises communication means, first detection means, second detection means, and a learner. The communication means transmits packets using a transmission channel, which is a channel for transmitting packets, during the first operation period. The first detection means detects a communication result when the packet is transmitted and detects an idle period of wireless communication after the packet is transmitted in the first operation period every time a packet is transmitted by the communication means. do. The second detection means detects the state of the transmission channel in an observation period, which is a period for observing the presence or absence of radio communication by other terminal devices, in the first operation period each time a transmission channel is received. The learning device receives the communication result detected in the first operation period, the idle period, the state of the transmission channel in the observation period, and the candidate channel that is a candidate for the channel used for packet transmission. A first process of calculating an immediate reward, which is a reward obtained when a packet is transmitted in the transmission channel in the first operation period, based on the period, and calculating 1 corresponding to the state of the transmission channel in the observation period. A reward obtained by averaging the cumulative value of immediate rewards in one transmission channel by the number of times one packet length is selected, and a reward in a second operation period that is an operation period after the first operation period. A second process of calculating a reward using the immediate reward calculated in the first process, and a correspondence table that associates the candidate channel, the state of the transmission channel in the observation period, the packet length of the packet, and the average reward. Created or updated, and based on the created or updated correspondence table, the channel that yields the maximum average reward is selected as the transmission channel with a predetermined probability, and the maximum A third process of selecting a packet length when an average reward is obtained with a predetermined probability and outputting the selected transmission channel and packet length; Executes each time it is accepted. Each time the communication means receives the transmission channel and packet length selected in the third processing from the learning device, the communication means learns during the second operation period when the received transmission channel is available. Send a packet with the packet length received from the device.

(Configuration 2)
In configuration 1, in the first process, the learning device calculates the immediate reward as zero when the communication result is a packet transmission failure, and when the communication result is a packet transmission success, a predetermined reward in the idle period The reciprocal of the addition result obtained by adding the period of is calculated as the immediate reward.

(Composition 3)
In configuration 1 or configuration 2, in the second processing, the learner obtains an immediate reward in the first operation period, an average reward in the first operation period, and one packet length corresponding to the state of the transmission channel in the observation period. is selected, and the average reward for the second operation period is calculated based on the selected number of times to update the average reward.

(Composition 4)
In configuration 3, in the second process, the learner sets the immediate reward in the first action period to _Rt , the average reward in the first action period to Vt, and the average _reward in the second action period to V Let _t+1 and let n be the number of times one packet length is selected corresponding to the state of the transmission channel during the observation period (n is an integer equal to or greater than 1). Update the average reward by calculating _t+1 .

V _t+1 =V _t +(R _t −V _t )/n (1)
(Composition 5)
In configuration 3 or configuration 4, in the third process, the learner has a probability of (1−ε) (ε is a real number in the range of 1 to 0) and the average reward in the second operation period is the maximum A certain channel is selected as a transmission channel from the candidate channels, and an arbitrary channel is selected from the candidate channels as a transmission channel with probability ε.

(Composition 6)
In any one of configurations 3 to 6, in the third processing, the learner selects the packet length that maximizes the average reward in the second operation period for the state of the transmission channel in the observation period.

(Composition 7)
In any one of configurations 1 to 6, the terminal device further includes control means. The control means selects a channel of a band different from the band of the candidate channel as a new candidate channel when the transmission success rate, which is the probability of successful packet transmission, is equal to or less than a threshold, and selects the selected new candidate channel. Control the learner to use The learning device executes the first process, the second process, and the third process using a new candidate channel each time it receives the state of the transmission channel, the communication result, and the idle period during the observation period.

(Composition 8)
Also, according to the embodiment of the present invention, the program
a first step in which the communication means transmits packets using a transmission channel, which is a channel for transmitting packets, during a first operation period;
The first detection means detects, in the first operation period, the communication result at the time the packet is transmitted each time the packet is transmitted in the first step, and the idle period of wireless communication after the packet is transmitted. a second step of detecting
A third detection means for detecting the state of the transmission channel in an observation period, which is a period for observing the presence or absence of wireless communication by other terminal devices, in the first operation period each time the second detection means receives the transmission channel. a step of
A learning device receives the communication result detected in the first operation period, the idle period, the state of the transmission channel in the observation period, and candidate channels that are candidates for channels used for packet transmission, and receives the communication result and A first process of calculating an immediate reward, which is a reward obtained when a packet is transmitted in the transmission channel during the first operation period, based on the vacant period, and corresponding to the state of the transmission channel during the observation period. It is a reward obtained by averaging the cumulative value of immediate rewards in one transmission channel according to the number of times one packet length is selected, and is a reward in a second operation period that is an operation period after the first operation period. A second process for calculating the average reward using the immediate reward calculated in the first process, and a correspondence table that associates the candidate channel, the state of the transmission channel in the observation period, the packet length of the packet, and the average reward. is created or updated, and based on the created or updated correspondence table, the channel that yields the maximum average reward is selected as the transmission channel with a predetermined probability, and the maximum and a third process of selecting the packet length when the average reward of is obtained with a predetermined probability, and outputting the selected transmission channel and packet length, and the state of the transmission channel, the communication result, and the idle period during the observation period. cause the computer to execute a fourth step that is executed each time the
In the first step, the communication means further receives the transmission channel and the packet length selected in the third process from the learner, during the second operation period, when the received transmission channel becomes available. A program for causing a computer to transmit a packet having a packet length received from a learner when a packet is received from the learner.

(Composition 9)
In configuration 8, in the first process of the fourth step, the learning device calculates the immediate reward as zero when the communication result is a packet transmission failure, and when the communication result is a packet transmission success , the reciprocal of the result of adding a predetermined period to the vacant period is calculated as an immediate reward.

(Configuration 10)
In Configuration 8 or Configuration 9, in the second processing of the fourth step, the learning device corresponds to the immediate reward in the first operation period, the average reward in the first operation period, and the state of the transmission channel in the observation period. calculating an average reward in the second operation period based on the number of times one packet length is selected and updating the average reward.

(Composition 11)
In configuration 10, in the second process of the fourth step, the learner sets the immediate reward in the first action period to R _t , the average reward in the first action period to V _t , and the second action period , and the number of times one packet length is selected corresponding to the state of the transmission channel during the observation period is n (n is an _integer equal to or greater than 1.), the following equation (1 ) to update the average reward by calculating the average reward V _t+1 .

V _t+1 =V _t +(R _t −V _t )/n (1)
(Composition 12)
In configuration 10 or configuration 11, in the third process of the fourth step, the learner performs A channel with the maximum average reward is selected as a transmission channel from the candidate channels, and an arbitrary channel is selected from the candidate channels as a transmission channel with probability ε.

(Composition 13)
In any one of configuration 10 to configuration 12, in the third processing of the fourth step, the learner determines the packet length with the maximum average reward in the second operation period with respect to the state of the transmission channel in the observation period. to select.

(Composition 14)
In any one of configuration 8 to configuration 13, when the transmission success rate, which is the probability of successful packet transmission, is equal to or less than a threshold, the control means selects a channel in a band different from the band of the candidate channel as the new candidate channel. further causing the computer to perform a fifth step of selecting and controlling the learner to use the selected new candidate channel;
The learning device executes the first process, the second process, and the third process using a new candidate channel each time it receives the state of the transmission channel, the communication result, and the idle period during the observation period.

(Composition 15)
Further, according to the embodiment of the present invention, the recording medium is a computer-readable recording medium recording the program described in any one of Structures 8 to 14.

Wireless communication can be performed while coexisting with a terminal device that performs wireless communication using a different wireless communication system.

1 is a schematic diagram of a communication system in an embodiment of the invention; FIG. 2 is a schematic diagram of a terminal device shown in FIG. 1; FIG. FIG. 4 is a conceptual diagram of a received power spectrum; FIG. 4 is a diagram for explaining observation periods and vacant periods; FIG. 4 is a schematic diagram of a correspondence table in a learning device; 3 is a diagram for explaining the operation of the learning device shown in FIG. 2; FIG. FIG. 3 is a timing chart for explaining the operation of the terminal device shown in FIG. 2; FIG. 3 is a diagram for explaining the operation of the terminal device shown in FIG. 2 during each operation period; FIG. FIG. 4 is a first schematic diagram showing transition of a correspondence table TBL1; FIG. 11 is a second schematic diagram showing the transition of the correspondence table TBL1; FIG. 11 is a third schematic diagram showing changes in the correspondence table TBL1; FIG. 11 is a fourth schematic diagram showing changes in the correspondence table TBL1; FIG. 11 is a fifth schematic diagram showing changes in the correspondence table TBL1; 3 is a flowchart for explaining the operation of the terminal device shown in FIG. 2; FIG. 3 is a first flow chart for explaining the operation of the learning device shown in FIG. 2; FIG. 3 is a second flow chart for explaining the operation of the learning device shown in FIG. 2; FIG. 4 is a diagram for explaining different methods of determining packet length m;

An embodiment of the present invention will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are denoted by the same reference numerals, and the description thereof will not be repeated.

FIG. 1 is a schematic diagram of a communication system according to an embodiment of the invention. Referring to FIG. 1, communication system 100 includes base station BS1 and terminal device TM1. A base station BS1 and a terminal device TM1 are arranged in a wireless communication space.

Base station BS1 has a coverage area REG1. The terminal device TM1 is located within the communication range REG1.

The base station BS1 transmits packets to the terminal device TM1 and receives packets from the terminal device TM1 using the radio communication system RF1.

Base station BS2 has a coverage area REG2. The base station BS2 is arranged such that the communication range REG2 partially overlaps the communication range REG1 of the base station BS1. The base station BS2 transmits packets to the terminal device TM2 and receives packets from the terminal device TM2 using a radio communication system RF2 different from the radio communication system RF1.

The terminal device TM1 selects a transmission channel for transmitting the packet by a method described later, and transmits the packet to the base station BS1 so as to coexist with the radio communication by the terminal device TM2 on the selected transmission channel. After that, when the terminal device TM1 receives an ACK (acknowledgement) packet indicating that the packet has been received from the base station BS1 through the transmission channel, the terminal device TM1 detects that the packet has been successfully transmitted, and transmits an ACK packet from the base station BS1. When not received, it detects that the transmission of the packet has failed.

In FIG. 1, one terminal device TM1 is shown within the communication range REG1 of the base station BS1, but in reality there are a plurality of terminal devices TM1 within the communication range REG1 of the base station BS1. exist.

In the following, the terminal device TM1 is referred to as "terminal device 10".

FIG. 2 is a schematic diagram of the terminal device shown in FIG. Referring to FIG. 2 , terminal device 10 includes antenna 1 , receiving means 2 , control means 3 , learning device 4 , transmitting means 5 and application 6 .

When the receiving means 2 receives the signal S_carrier_L for performing carrier sense and the selection channel CH_Select from the control means 3, the reception means 2 receives the selection channel CH_Select during the observation period L, which is a period for observing the presence or absence of radio communication by other terminal devices. Carrier sensing is performed via the antenna 1 to obtain a received power spectrum PW_carrier_L that indicates the time dependence of received power, and the obtained received power spectrum PW_carrier_L is output to the control means 3 .

Further, when receiving an ACK packet on the selected channel CH_Select from the base station BS1 via the antenna 1, the receiving means 2 outputs the received ACK packet to the control means 3. FIG. After that, the receiving means 2 performs carrier sensing via the antenna 1 on the selected channel CH_Select and acquires the received power spectrum PW_chn on the selected channel CH_Select. The receiving means 2 then outputs the received power spectrum PW_chn to the control means 3 .

The control unit 3 holds in advance candidate channels CH_cdt_1 and CH_cdt_2, which are candidates for channels used for wireless communication. Candidate channel CH_cdt_1 consists of 1ch, 6ch, and 11ch of 2.4 GHz band, for example, and candidate channel CH_cdt_2 consists of 128ch, 132ch, and 136ch of 5 GHz band, for example. Then, control means 3 outputs candidate channel CH_cdt_1 or candidate channel CH_cdt_2 to learning device 4 .

Also, upon receiving the selected channel CH_Select from the learning device 4 , the control means 3 generates a signal S_carrier_L and outputs the selected channel CH_Select and the signal S_carrier_L to the receiving means 2 . After that, the control means 3 receives the received power spectrum PW_carrier_L from the receiving means 2, and detects the state S of the selected channel CH_Select in the observation period L by a method described later based on the received received power spectrum PW_carrier_L. Then, the control means 3 outputs the state S of the selected channel CH_Select during the observation period L to the learning device 4 .

Further, when the control means 3 receives the transmission data D_TR from the application 6 and the packet length m from the learning device 4, the transmission data having the data amount AOD when the packet length L_PKT of the transmission packet PKT becomes the packet length m D_m is detected from transmission data D_TR, and a transmission packet PKT containing the detected transmission data D_m is generated. Then, when the control means 3 receives the carrier sense result for the selected channel CH_Select from the receiving means 2, it determines whether or not the selected channel CH_Select is available based on the carrier sense result for the selected channel CH_Select. When the control means 3 determines that the selection channel CH_Select is available, the control means 3 outputs the selection channel CH_Select and the transmission packet PKT to the transmission means 5 . When the control means 3 determines that the selection channel CH_Select is not available based on the carrier sense result of the selection channel CH_Select, the control means 3 waits for the selection channel CH_Select to become available and transmits the selection channel CH_Select and the transmission packet PKT to the transmission means. Output to 5.

Furthermore, after outputting the selected channel CH_Select and the transmission packet PKT to the transmission means 5, the control means 3, when receiving an ACK packet from the reception means 2 for a certain period of time, indicates that the transmission packet PKT has been successfully transmitted. detect. The control means 3 then generates a signal S_success indicating that the transmission packet PKT has been successfully transmitted, and outputs the generated signal S_success to the learning device 4 . On the other hand, when the control means 3 does not receive an ACK packet from the receiving means 2 for a certain period of time after outputting the transmission packet PKT to the transmission means 5, it detects that the transmission of the transmission packet PKT has failed. Then, the control means 3 generates a signal S_failure indicating that transmission of the transmission packet PKT has failed, and outputs the generated signal S_failure to the learning device 4 . That is, after outputting the transmission packet PKT to the transmission means 5, the control means 3 determines whether the transmission of the transmission packet PKT has succeeded or failed.

Furthermore, when the control means 3 receives the reception power spectrum PW_chn from the reception means 2 after determining whether the transmission of the transmission packet PKT has succeeded or failed, the control means 3, based on the received reception power spectrum PW_chn, will be described later. The method detects an idle period N. Then, the control means 3 outputs the vacant period N to the learning device 4 .

Further, the control means 3 counts the number N _PKT of transmission packets PKT output to the transmission means 5 and the number N _ACK of ACK packets received from the reception means 2 in a certain period of time, and counts the number N _ACK as the number N Divide by the _PKT to calculate the packet transmission success rate _{R_SUCCESS} . Then, the control means 3 changes the candidate channel CH_cdt when the transmission success rate R _SUCCESS is equal to or less than the threshold value R_th. On the other hand, the control means 3 does not change the candidate channel CH_cdt when the transmission success rate R _SUCCESS is greater than the threshold value R_th. Note that the threshold value R_th is set to 50%, for example.

The learning device 4 receives the candidate channel CH_cdt, the state S of the selected channel CH_Select in the observation period L, the signal S_success or the signal S_failure, and the idle period N from the control means 3 . Then, the learning device 4 performs learning by the multi-armed bandit algorithm based on the candidate channel CH_cdt, the state S of the selected channel CH_Select in the observation period L, the signal S_success or the signal S_failure, and the idle period N, and selects from the candidate channel CH_cdt. Select the channel CH_Select and select the packet length m of the transmission packet PKT. The learning device 4 then outputs the selected channel CH_Select and the packet length m to the control means 3 .

Upon receiving the selection channel CH_Select and the transmission packet PKT from the control means 3, the transmission means 5 transmits the transmission packet PKT via the antenna 1 on the selection channel CH_Select.

The application 6 generates transmission data and outputs the generated transmission data to the control means 3 .

FIG. 3 is a conceptual diagram of a received power spectrum. In FIG. 3, the vertical axis represents received power and the horizontal axis represents time.

Referring to FIG. 3, received power spectrum SP_RSSI changes with the passage of time. The control means 3 preliminarily holds, as a threshold value RSSI_th, a received power value in a signalless state in which no wireless communication system is communicating.

Then, the control means 3 squares the amplitude value of the reception power spectrum SP_RSSI received on the selected channel CH_Select to convert it into a reception power value, and when the converted reception power value is larger than the threshold value RSSI_th, selects It is determined that the state of the channel CH_Select is busy, and when the received power value is equal to or less than the threshold RSSI_th, it is determined that the selected channel CH_Select is idle.

[Learning in the learner]
Learning in the learning device 4 will be described. FIG. 4 is a diagram for explaining observation periods and vacant periods. Referring to FIG. 4, in the embodiment of the present invention, a slot SL, which is the minimum time during which the channel state changes, is set. Slot SL has a length of 10 μs, for example.

Receiving means 2 detects received power spectrum PW_carrier_L by performing carrier sense on selected channel CH_Select during observation period L (slots SL1 and SL2), and outputs detected received power spectrum PW_carrier_L to control means 3 .

Upon receiving the received power spectrum PW_carrier_L from the receiving means 2, the control means 3 squares the amplitude value of the received power spectrum PW_carrier_L to convert it into a received power value. Then, the control means 3 compares the received power value with the threshold RSSI_th, and when the received power value is greater than the threshold RSSI_th, determines that the selected channel CH_Select is in the busy state, and the received power value When it is equal to or less than the threshold RSSI_th, determining that the state of the selected channel CH_Select is idle is performed for slots SL1 and SL2.

In the embodiment of the present invention, the busy state is represented by "1" and the idle state by "0".

Since the observation period L consists of two slots SL1 and SL2, the state S of the selected channel CH_Select during the observation period L is represented by two bits "00", "01", "10" and "11".

When the transmission means 5 transmits the packets at slots SL3 and SL4 after the observation period L, the control means 3 determines whether or not the ACK packet has been received at slot SL5 to determine whether the packet has been successfully transmitted. Determine if it failed.

After that, the receiving means 2 detects the received power spectrum PW_chn by performing carrier sense on the selected channel CH_Select in the slots SL6 to SL8, and outputs the detected received power spectrum PW_chn to the control means 3. FIG.

Upon receiving the received power spectrum PW_chn from the receiving means 2, the control means 3 squares the amplitude value of the received power spectrum PW_chn to convert it into a received power value. Then, the control means 3 compares the received power value with the threshold RSSI_th, and when the received power value is greater than the threshold RSSI_th, determines that the selected channel CH_Select is in the busy state, and the received power value Determining that the state of the selected channel CH_Select is idle when it is equal to or less than the threshold RSSI_th is performed for slots SL6-SL8.

Then, the control means 3 detects that the idle period N is "0" when the slot SL6 is in a busy state, and when the slot SL6 is in an idle state and the slot SL7 is in a busy state, the idle period N is detected. When the period N is detected to be "1" and the slots SL6 and SL7 are idle and the slot SL8 is busy, the idle period N is detected to be "2" and the slot SL6 is detected. ˜SL8 is in an idle state, it detects that the vacant period N is "3". That is, the control means 3 detects the period from the slot SL6 in the idle state to the slot SL in which the idle state continues as the idle period N, starting from the fact that the slot SL6 is in the idle state. In other words, the idle period N is a period during which wireless communication is not performed after packet transmission.

When the control means 3 detects the state S of the selected channel CH_Select and the idle period N in the observation period L, it outputs the detected state S of the selected channel CH_Select and the idle period N in the observation period L to the learning device 4 . Further, when the control means 3 receives an ACK packet from the receiving means 2, it generates a signal S_success and outputs it to the learning device 4. When it does not receive an ACK packet from the receiving means 2, it generates a signal S_failure for learning. Output to device 4.

FIG. 5 is a schematic diagram of a correspondence table in the learning device 4. FIG. Referring to FIG. 5, correspondence table TBL1 includes channel number, state S of selected channel CH_Select in observation period L, packet length m, and average reward V. FIG. The channel number, the state S of the selected channel CH_Select in the observation period L, the packet length m and the average reward V are associated with each other.

The channel numbers consist of 1, ..., a, ..., A (A is the total number of channels included in one candidate channel CH_cdt, and a is an integer from 1 to A). The state S of the selected channel CH_Select in the observation period L consists of "00", "01", "10" and "11". The states “00”, “01”, “10” and “11” of the selected channel CH_Select in the observation period L are associated with channels 1, . . . , a, .

The packet length m consists of 1, 2, . M is the total number of packet lengths m and is an integer of 2 or more. Packet length m=1, packet length m=2, . =M represents the longest packet length. Packet length m=M is set to, for example, the length of DIFS (Distributed Inter Frame Space) in a wireless communication system, packet length m=1 is set to the reference packet length, and m is increased by "1". , the packet length m increases by, for example, 10 μs. The standard packet length is set to 10 μs, for example.

Packet lengths m=1 to M are associated with states “00”, “01”, “10” and “11” of selected channel CH_Select in observation period L in one channel number. In FIG. 5, the column of the packet length m corresponding to the states “01”, “10” and “11” of the selected channel CH_Select during the observation period L is blank. The column of packet length m corresponding to the states "01", "10" and "11" of , stores packet lengths m=1 to M. FIG.

The average reward V is M packet lengths m=1 to M associated with each of the states “00”, “01”, “10” and “11” of the selected channel CH_Select in the observation period L for each channel number. can be mapped to Then, the average reward V is calculated by the following equation.

In equation (1), V _t+1 is the average reward in action period T+1, V _t is the average reward obtained in action period T, R _t is the immediate reward obtained in action period T, and n is the number of times one packet length m corresponding to one state S of the selection channel CH_Select in the observation period L is selected.

Equation (1) expresses the average reward V _t in the operating period T, the immediate reward R _t in the operating period T, and the number n of times n that one packet length m corresponding to the state S of the selected channel CH_Select in the observation period L is selected. Represents calculating the average reward V _t+1 obtained at T+1. The average reward V _t+1 is then calculated for each of the M packet lengths m corresponding to the state S of the selected channel CH_Select in the observation period L.

The immediate reward _Rt in formula (1) is represented by the following formula.

In equation (2), when the packet is successfully transmitted (Success), the immediate reward _Rt consists of the multiplication result obtained by multiplying the reciprocal of the addition result N+1 obtained by adding "1" to the idle period N and the packet length m. , when the packet transmission fails (Failure), the immediate reward R _t is zero (=0).

In Equation (2A), the reciprocal of N+1 is calculated so that the immediate reward _Rt can be calculated even when the vacant period N is zero (=0).

According to equation (2), when the packet is successfully transmitted (Success), the immediate reward R _t becomes larger when the vacant period N is short, becomes smaller when the vacant period N is long, and increases when the packet length is long. becomes larger, and becomes smaller as the packet length m becomes shorter.

In equation (1), the initial value of the average reward Vt is set to zero ( ₌ 0). As a result, when the transmission of a packet fails in operation period T, the average reward V _t+1 is zero (=0) because the immediate reward R _t is zero (=0) (see equation (2B)). . On the other hand, when a packet is successfully transmitted in operation period T, since the immediate reward R _t is m/(N+1) (see equation (2A)), the average reward V _t+1 is m/{n·(N+1) }.

Therefore, if the packet transmission failures continue after the learning in the learner 4 is started, the average reward V _t+1 will not increase.

According to equations (1) and (2), when the immediate reward R _t is greater than the average reward V _{t +1, the average reward V t+1} corresponds to the state S(“00”, “01 ”, “10”, or “11”) increases as the number of times n of selecting one packet length m increases. That is, the average reward V _t+1 increases by keeping the same packet length m corresponding to one state S of the selected channel CH_Select in the observation period L being selected. Therefore, M packet lengths 1 to M (see FIG. 5) corresponding to the state S (one of "00", "01", "10", and "11") of the selected channel CH_Select in the observation period L is first greater than zero (=0), then the same packet length m is selected if the state S of the selected channel CH_Select in the observation period L is the same. may continue to be

On the other hand, when the immediate reward R _t is smaller than the average reward V _t , the average reward V _t+1 is the state S (any of “00”, “01”, “10”, “11” of the selected channel CH_Select in the observation period L or one) decreases as the number of times n that one packet length m is selected increases. This can occur if the idle period N becomes longer. Therefore, from the viewpoint of obtaining a larger average reward V _t+1 , the state S (“00”, “01”, “10”, “11 ”), find a longer packet length m with a higher probability of successful packet transmission, and use the found packet length m as the state S (“ 00", "01", "10", and "11"). As a result, the terminal device 10 can perform wireless communication while avoiding collision with wireless communication by other terminal devices (that is, coexisting with other terminal devices).

FIG. 6 is a diagram for explaining the operation of the learning device 4 shown in FIG. Referring to FIG. 6, learning device 4 holds correspondence table TBL1. Upon receiving the candidate channel CH_cdt from the control means 3, the learning device 4 selects a channel from the candidate channel CH_cdt by the ε-greedy method during the operation period T. More specifically, the learning device 4 determines a certain small number ε (for example, 0.3), generates a random number p consisting of real numbers in the range of 0 to 1, and generates the random number When p is less than or equal to ε, a channel is randomly selected as the selection channel CH_Select_T from the candidate channels CH_cdt, and when the generated random number p is not less than or equal to ε, the channel that gives the maximum average reward V _t in the operation period T is selected. Select from the candidate channels CH_cdt as CH_Select_t.

Then, the learning device 4 outputs the selection channel CH_Select_t selected from the candidate channels CH_cdt to the control means 3 .

After that, when the learning device 4 receives the state St of the selected channel _{CH_Select_t} in the observation period L from the control means 3, the packet length m is calculated for the state St of the selected channel _{CH_Select_t} in the observation period L by the ε-greedy method. select. More specifically, the learning device 4 generates a random number p consisting of real numbers in the range of 0 to 1, and when the generated random number p is equal to or less than ε, the packet length m is randomly selected from the packet lengths 1 to M. _t is selected, and when the generated random number p is not equal to or less than ε, the packet length m _t when the maximum average reward V _t is obtained in the operation period T is selected from the packet lengths 1 to M.

Note that the learning device 4 randomly selects the packet length m _t from the packet lengths 1 to M when the generated random number p is not equal to or less than ε and the maximum average reward V _t does not exist.

After selecting the packet length _mt , the learning device 4 outputs the selected packet length _mt to the control means 3 .

After that, learning device 4 receives the communication result (success or failure of packet transmission) when the packet is transmitted and the idle period N from control means 3. Based on the period N, calculate the immediate reward R _t in the action period T. More specifically, when learning device 4 receives successful packet transmission and idle period N, learning device 4 calculates immediate reward R _t by equation (2A). On the other hand, when the learning device 4 receives the packet transmission failure and the idle period N, it calculates the immediate reward _Rt by Equation (2B). After calculating the immediate reward _Rt , the learning device 4 stores the calculated immediate reward _Rt .

After calculating the immediate reward R _t , the learning device 4 selects the same channel as the selected channel CH_Select_t when calculating the immediate reward R _t as the selected channel _{CH_Select_T} +1, and calculates the average reward V _t+ 1 as the selected channel CH_Select_T+1. It is calculated by the formula (1) using the reward _Vt .

Then, in correspondence table _TBL1 , learning device 4 _averages Store the reward V _t+1 .

Then, the learning device 4 selects the selected channel CH_Select_T+1 by the method described above and outputs the selected selected channel CH_Select_T+1 to the control means 3 in the operation period T+1.

After that, when the learning device 4 receives the state St+1 of the selected channel CH_Select_T ₊ 1 in the observation period L from the control means 3, it selects the packet length mt+ ₁ for the state St+1 of the selected channel _{CH_Select_T} +1 by the method described above. The learning device 4 then outputs the selected packet length mt ₊₁ to the control means 3 .

After that, the learning device 4 repeatedly executes the operation described above.

[Operations other than the learning device in the terminal device]
The control means 3 outputs the candidate channel CH_cdt to the learner 4 and then receives the selected channel CH_Select from the learner 4 .

The control means 3 then generates the signal S_carrier_L and outputs the selected channel CH_Select and the signal S_carrier_L to the receiving means 2 .

After that, the control means 3 receives the received power spectrum PW_carrier_L from the receiving means 2, and based on the received received power spectrum PW_carrier_L, selects the state S (="00", " 01”, “10”, or “11”). Then, the control means 3 outputs the state S (=“00”, “01”, “10”, “11”) of the selected channel CH_Select during the observation period L to the learning device 4 .

Subsequently, when the control means 3 receives the transmission data from the application 6 and the packet length m from the learning device 4, it generates a transmission packet PKT having the packet length m by the method described above.

Then, when the control means 3 receives the carrier sense result from the receiving means 2 and determines that the selection channel CH_Select is available based on the received carrier sense result, the control means 3 transmits the selection channel CH_Select and the transmission packet PKT. Output to the transmission means 5 .

The transmission means 5 receives the selected channel CH_Select and the transmission packet PKT from the control means 3 . Then, the transmission means 5 transmits the transmission packet PKT using the selection channel CH_Select. In this case, the transmitting means 5 transmits the transmission packet PKT at a fixed transmission rate.

After that, when receiving an ACK packet on the selection channel CH_Select for a certain period of time, the receiving means 2 outputs the received ACK packet to the control means 3 . Then, the receiving means 2 performs carrier sense on the selected channel CH_Select, detects the received power spectrum PW_chn, and outputs the detected received power spectrum PW_chn to the control means 3 .

When the control means 3 receives an ACK packet from the receiving means 2, it detects that the packet has been successfully transmitted. When the control means 3 detects that the packet transmission has succeeded, it generates a signal S_success and outputs it to the learning device 4. When it detects that the packet transmission has failed, it generates a signal S_failure. Output to learning device 4 .

Then, when receiving the received power spectrum PW_chn from the receiving means 2, the control means 3 detects the idle period N based on the received power spectrum PW_chn by the above-described method, and transfers the detected idle period N to the learning device 4. Output.

In addition, the control means 3 counts the number of transmitted packets N _PKT and the number N _ACK of ACK packets received from the receiving means 2, divides the number N _ACK by the number of transmitted N _PKT , and obtains the transmission success rate R _SUCCESS . calculate.

Then, the control means 3 determines whether or not the transmission success rate _{R_SUCCESS} is equal to or less than the threshold value R_th.

When the control means 3 determines that the transmission success rate _{R_SUCCESS} is equal to or less than the threshold value R_th, the control means 3 selects candidate channels composed of channels of different bands (candidate channels different from the candidate channels already output to the learning device 4) to the learning device. Output to 4.

On the other hand, when the control means 3 determines that the transmission success rate R _SUCCESS is greater than the threshold value R_th, it maintains the candidate channel (candidate channel CH_cdt_1 or candidate channel CH_cdt_2) already output to the learning device 4 . That is, the control means 3 does not change the candidate channel CH_cdt.

Thereafter, the operations described above are repeatedly performed.

FIG. 7 is a timing chart for explaining the operation of the terminal device 10 shown in FIG. In FIG. 7, the operation timing of the terminal device 10 will be described when channel 2 is selected as the selected channel CH_Select among the candidate channels CH_cdt consisting of channel 1, channel 2 and channel 3. FIG. A period from the timing of arrow AR1 to the timing of arrow AR6 is assumed to be an operation period T. FIG.

Referring to FIG. 7, control means 3 outputs candidate channels (channels 1 to 3) to learning device 4 at a timing prior to the timing of arrow AR1. Then, the control means 3 generates a transmission packet PKT including the transmission data D_m at the timing of the arrow AR1. Also, at the timing of arrow AR1, learning device 4 selects channel 2 from candidate channels (channels 1 to 3) as selected channel CH_Select, and outputs the selected selected channel CH_Select (=channel 2) to control means 3. .

Upon receiving the selected channel CH_Select (=channel 2) from the learning device 4, the control means 3 generates a signal S_carrier_L and outputs the selected channel CH_Select (=channel 2) and the signal S_carrier_L to the receiving means 2.

When the receiving means 2 receives the selected channel CH_Select (=channel 2) and the signal S_carrier_L from the control means 3, during the observation period L from the timing of the arrow AR1 to the timing of the arrow AR2, the carrier in the selected channel CH_Select (=channel 2) Sensing is performed to detect the received power spectrum PW_carrier_L, and the detected received power spectrum PW_carrier_L is output to the control means 3 .

Upon receiving the received power spectrum PW_carrier_L from the receiving means 2, the control means 3 squares the amplitude value of the received power spectrum PW_carrier_L to convert it into a received power value. Then, the control means 3 compares the received power value with the threshold RSSI_th, and determines that the selected channel CH_Select (=channel 2) is busy when the received power value is greater than the threshold RSSI_th. , the state of the selected channel CH_Select (=channel 2) is determined to be idle when the received power value is equal to or less than the threshold RSSI_th.

Then, the control means 3 determines that the state S of the selected channel CH_Select (=channel 2) in the observation period L is one of "00", "01", "10" and "11" at the timing of the arrow AR2. to detect

In FIG. 7, a packet is transmitted in the first slot SL of the observation period L, and the second slot SL of the observation period L is vacant. Based on the spectrum PW_carrier_L, it is detected that the state S of the selected channel CH_Select (=channel 2) is "10" in the observation period L by the method described above. Then, the control means 3 outputs the state S (=“10”) of the selected channel CH_Select (=channel 2) during the observation period L to the learning device 4 .

When the learning device 4 receives the state S (=“10”) of the selected channel CH_Select (=channel 2) in the observation period L from the control means 3, at the timing of the arrow AR2, the selection in the observation period L is performed by the method described above. A packet length m corresponding to the state S (="10") of the channel CH_Select (=channel 2) is selected, and the selected packet length m is output to the control means 3.

Further, when the control means 3 receives the transmission data D_TR from the application 6 and the packet length m from the learning device 4 at the timing of the arrow AR2, the above-described method is performed based on the received transmission data D_TR and the packet length m. generates a transmission packet PKT containing transmission data D_m, and controls the receiving means 2 to perform carrier sensing. Then, when the control means 3 receives the carrier sense result from the receiving means 2 and determines that the selected channel CH_Select is available based on the received carrier sense result, the selected channel CH_Select (=channel 2) and It outputs the transmission packet PKT to the transmission means 5 .

When the control means 3 determines that the selection channel CH_Select is not available, the control means 3 waits for the selection channel CH_Select to become available and outputs the selection channel CH_Select (=channel 2) and the transmission packet PKT to the transmission means 5 .

The transmission means 5 receives the selection channel CH_Select (=channel 2) and the transmission packet PKT from the control means 3 at the timing of the arrow AR2, and transmits the transmission packet PKT using the selection channel CH_Select.

Then, the transmission of the packet is completed at the timing of arrow AR3. After that, when receiving an ACK packet at the timing of arrow AR4, receiving means 2 outputs the received ACK packet to control means 3 .

Upon receiving the ACK packet from the receiving means 2, the control means 3 detects successful transmission of the packet. On the other hand, when the control means 3 does not receive an ACK packet from the receiving means 2, it detects that the packet transmission has failed. Therefore, the control means 3 outputs success or failure of packet transmission to the learning device 4 at the timing of the arrow AR4.

After that, the receiving means 2 detects the reception power spectrum PW_chn by performing carrier sense in the selection channel CH_Select during the period from the timing of the arrow AR5 to the timing of the arrow AR6, and outputs the detected reception power spectrum PW_chn to the control means 3. do.

When the control means 3 receives the received power spectrum PW_chn from the receiving means 2, at the timing of the arrow AR6, based on the received power spectrum PW_chn, the control means 3 detects the idle period N by the method described above, and converts the detected idle period N to Output to learning device 4 .

When learning device 4 receives idle period N from control means 3 at the timing of arrow AR6, learner 4 receives the idle period N and the success or failure of packet transmission received from control means 3 at the timing of arrow AR4. Based on this, the immediate reward Rt in the action period _T is calculated, and the calculated immediate reward _Rt is stored. This immediate reward Rt is given when the same channel as the selected channel _{CH_Select} in the action period T is selected in the action period after the action period T (the action period consisting of the period from the timing of the arrow AR1 to the timing of the arrow AR6). is used to calculate the average reward V _t+1 obtained at , by equation (1).

When the immediate reward _Rt is stored, the learning device 4 uses the immediate reward _Rt in the action period T calculated when the same channel as the selected channel CH_Select is selected in the action period after the action period T ( The average reward V _t+1 in the action period from the timing of arrow AR1 to the timing of arrow AR6 is calculated by equation (1). Then, the learning device 4 associates the average reward with the packet length m (the packet length m when the immediate reward _Rt is calculated) associated with the state S of the selected channel CH_Select in the observation period L in the correspondence table TBL1. Store V _t+1 .

After that, the control means 3 calculates the transmission success rate _{R_SUCCESS} described above. Then, when the calculated transmission success rate _{R_SUCCESS} is equal to or less than the threshold value R_th, the control means 3 outputs another candidate channel CH_cdt to the learning device 4 so that the learning device 4 uses another candidate channel CH_cdt. to control. On the other hand, when the transmission success rate R _SUCCESS is greater than the threshold value R_th, the control means 3 maintains the candidate channel CH_cdt already output to the learning device 4 at the timing before the timing of the arrow AR1. 4 does not output the candidate channel CH_cdt.

When the learning device 4 receives from the control means 3 a candidate channel CH_cdt different from the candidate channel CH_cdt already received from the control means 3, the learning device 4 selects the selection channel CH_Select based on the received candidate channel CH_cdt by the method described above.

After that, the terminal device 10 repeatedly performs the above-described operation every operation period T. FIG.

FIG. 8 is a diagram for explaining the operation of the terminal device 10 shown in FIG. 2 during each operation period.

Referring to FIG. 8, each of the T (T is a positive integer)-th operation period, the (T+1)-th operation period and the (T+2)-th operation period corresponds to the arrow AR1 shown in FIG. It consists of a period from the timing to the timing of the arrow AR6.

The control means 3 receives the selection channel CH_Select_t from the learning device 4 at the timing of the arrow AR1 in the Tth operation period. Then, the receiving means 2 and the control means 3 execute the following (I) to (III).
(I) Detect the state St of the selected channel _{CH_Select_t} in the observation period L at the timing of the arrow AR2 in the T-th operation period.
(II) Obtain the communication result CM_rst_t (success or failure of packet transmission) when the packet is transmitted at the timing of the arrow AR4 in the T-th operation period.
(III) In the period from the timing of the arrow AR5 to the timing of the arrow AR6 in the _T -th operation period, an idle period Nt after the completion of packet transmission is detected.

Then, the control means 3 determines the state S of the selected channel CH_Select_t in the observation period L in (I) to (III), the communication result CM_rst_t when the packet is transmitted (success or failure of packet transmission), and the idle period N Output to learning device 4 .

The learning device 4 performs the following (A) to (D) in the Tth operation period.
(A) Select the selection channel CH_Select_t in the Tth operation period.
(B) Select the packet length _mt according to the state St of the selected channel _{CH_Select_t} in the observation period L in the Tth operation period.
(C) An immediate reward _Rt is calculated by Equation (2) based on the communication result _{CM_rst_t} , idle period Nt, and packet length mt in the _T -th operation period.
(D) Using the immediate reward R _t , the average reward V _t+1 in the (T+1)-th action period is calculated by Equation (1).

Next, in the (T+1)th operation period, the receiving means 2 and the control means 3 execute the above (I) to (III). In this case, the receiving means 2 and the control means 3 detect the state St+1 of the selected channel _{CH_Select_t} +1 in the observation period L in (I), acquire the communication result CM_rst_t+1 when the packet is transmitted in (II), In (III), an idle period _Nt+1 after completion of packet transmission is detected.

On the other hand, the learning device 4 executes the above (A) to (D) in the (T+1)th operation period (see (E) in FIG. 8). In this case, the learning device 4 selects the selected channel CH_Select_t+1 in (A), selects the packet length m _{t+1 according} to the state St+1 of the selected channel CH_Select_t+1 in the observation period L in (B), and (C). In (D), the immediate reward R _t+1 is calculated by Equation (2) based on the communication result CM_rst_t+1, the idle period N _t+1 and the packet length m _t+1 , and in (D) the (T+2)th operation period using the immediate reward R _t+1 The average reward V _t+2 in is calculated by equation (1).

Furthermore, in the (T+2)th operation period, the receiving means 2 and the control means 3 execute the above (I) to (III). In this case, the receiving means 2 and the control means 3 detect the state St+2 of the selected channel _{CH_Select_t} +2 in the observation period L in (I), acquire the communication result CM_rst_t+2 when the packet is transmitted in (II), In (III), an idle period _Nt+2 after completion of packet transmission is detected.

On the other hand, the learning device 4 executes the above (A) to (D) in the (T+2)th operation period (see (F) in FIG. 8). In this case, the learning device 4 selects the selected channel CH_Select_t+2 in (A), selects the packet length m _{t+2 according} to the state St+2 of the selected channel CH_Select_t+2 in the observation period L in (B), and (C). In (D), the immediate reward R _t+2 is calculated by formula (2) based on the communication result CM_rst_t+2, the idle period N _t+2 , and the packet length m _t+2 , and in (D), the (T+3)th operation period using the immediate reward R _t+2 The average reward V _t+3 in is calculated by equation (1).

Note that the learning device 4 calculates the immediate reward R _t and the average reward V _t+1 , selects the selected channel CH_Select_T+1 by the timing of the arrow AR1 in the (T+1)th operation period, and calculates the immediate reward R _t and the average reward V t+1. Calculating the reward V _{t+ 1 and selecting the packet length m t+1} may be performed in the T-th operation period until the timing of the arrow AR2 in the (T+1)-th operation period, and the (T+1)-th operation You can do it during the period.

In addition, the learning device 4 calculates the immediate reward R _t+1 and the average reward V _t+2 , selects the selection channel CH_Select_T+2 by the timing of the arrow AR1 in the (T+2)th operation period, and calculates the immediate reward R _t +1 and the average reward V t+2. Calculating the reward V _{t+ 2 and selecting the packet length m t+2} may be performed in the (T+1)th operation period until the timing of the arrow AR2 in the (T+2)th operation period. may be performed during the operation period of

Then, the receiving means 2, the control means 3 and the learning device 4 repeatedly execute the operations described above.

The T-th operating period constitutes a "first operating period", and the (T+1)-th operating period constitutes a "second operating period".

After the above-described operations are completed in the T-th operation period and the (T+1)-th operation period, the above-described operations are repeatedly performed in the (T+1)-th operation period and the (T+2)-th operation period. . In this case, the (T+1)th operation period constitutes the "first operation period", and the (T+2)th operation period constitutes the "second operation period". Thereafter, similarly, the above-described operations are repeatedly performed in two operation periods. In this case, the two operating periods are adjacent when the same selected channel CH_Select is continuously selected in the two operating periods, and separated when the same selected channel CH_Select is not continuously selected in the two operating periods. ing.

In our embodiment, the immediate reward R _t is the reward obtained when a packet is transmitted on the selected channel CH_Select in one operating period, and the average reward V _t+1 is the selected channel CH_Select in the observation period L is the reward obtained by averaging the cumulative value of immediate reward R _t in one selection channel CH_Select by the number n (cumulative value) of selecting one packet length m corresponding to state S, and after one operation period It is a reward obtained during the action period.

9 to 13 are first to fifth schematic diagrams, respectively, showing changes in the correspondence table TBL1.

FIGS. 9 to 13 show changes in the correspondence table TBL1 for cases where candidate channels are 1ch, 6ch, and 11ch, and packet lengths m are 10 μs, 20 μs, and 30 μs.

Referring to FIG. 9, correspondence table TBL1(A) associates channel states “00”, “01”, “10”, and “11” in observation period L with 1ch, 6ch, and 11ch, respectively. Packet lengths of 10 μs, 20 μs and 30 μs are associated with channel states “00”, “01”, “10” and “11” in the observation period L, respectively, and the average reward V is associated with each packet length of 10 μs, 20 μs and 30 μs. consists of a configuration associated with Since the correspondence table TBL1(A) is the correspondence table in the initial state, all the average rewards V are initial values (=0). Then, the learning device 4 receives the candidate channels (=1ch, 6ch, 11ch) from the control means 3 .

Next, referring to FIG. 10, learning device 4 generates a random number p in the first operation period, and since the generated random number p is equal to or less than ε, candidate channels (=1ch, 6ch, 11ch) ) at random (see correspondence table TBL1(B) in FIG. 10), and outputs channel 6ch to control means 3 as selected channel CH_Select.

After that, the learning device 4 receives the state “01” of the channel 6ch during the observation period L from the control means 3 . Then, the learning device 4 selects the packet length m according to the state "01" of the channel 6ch in the observation period L by the ε-greedy method. When the packet length m is selected by the ε-greedy method, when the generated random number p is ε or less, the packet length m is randomly selected, and when the generated random number p is not ε or less, the maximum average reward V is obtained. will choose the packet length m when it is available.

At this point, all the average rewards V corresponding to the state “01” of channel 6ch in observation period L are zero (0) (see correspondence table TBL1(A) in FIG. 9), and the maximum average reward V is Since it does not exist, when the generated random number p is not equal to or less than ε, the packet length m is randomly selected. On the other hand, when the generated random number p is less than or equal to ε, the packet length m is randomly selected according to the ε-greedy method.

Therefore, the learning device 4 randomly selects the packet length m=10 μs (see correspondence table TBL1(B) in FIG. 10) and outputs the packet length m=10 μs to the control means 3 .

Subsequently, the learning device 4 receives from the control means 3 a signal S_success indicating that the packet has been successfully transmitted, and then receives an idle period N (=2) from the control means 3 .

Then, the learning device 4 calculates the immediate reward R ₁ (=10/3) in the first action period by Equation (2A) based on the signal S_success and the idle period N (=2). At this point, all the _average rewards V1 corresponding to the state “01” of channel 6ch in observation period L are zero (=0) (see correspondence table TBL1(A) in FIG. 9), so learning device 4 Substitute the immediate reward R ₁ (=10/3), the average reward V ₁ (=0), and n=1 into the equation (1) to obtain the average reward V ₂ =0+(10/3−0) /1=10/3 is calculated, and the calculated average reward V ₂ (=10/3) is stored in the correspondence table TBL1(B) in association with the packet length m=10 μs.

Referring to FIG. 11, learning device 4 generates random number p in the second operation period. channel 6ch is selected, and channel 6ch is output to the control means 3 as the selected channel CH_Select.

After that, the learning device 4 receives the state “01” of the channel 6ch during the observation period L from the control means 3 . At this point, the average reward V ₂ (=10/3) is stored in the column of the average reward V corresponding to the state “01” of the channel 6ch in the observation period L in association with the packet length m of 10 μs. (See correspondence table TBL1(B) in FIG. 10). The learning device 4 generates a random number p, and since the generated random number p is not equal to or less than ε, selects the packet length m=10 μs when the maximum _average reward V2 is obtained, and the selected packet length m= 10 μs is output to the control means 3 .

Subsequently, the learning device 4 receives from the control means 3 a signal S_success indicating that the packet has been successfully transmitted, and then receives an idle period N (=1) from the control means 3 .

Then, the learning device 4 calculates the immediate reward R ₂ (=10/2) by Equation (2A) based on the signal S_success and the idle period N (=1), and the calculated immediate reward R ₂ (=10/ 2) is stored.

After that, the learning device 4 substitutes the immediate reward R ₂ (=10/2), the average reward V ₂ (=10/3), and n=2 into Equation (1) to obtain the average reward V ₃ =10/ 3+(10/2−10/3)/2=25/6 is calculated, and the calculated average reward V ₃ (=25/6) is associated with the packet length m=10 μs, and is shown in the correspondence table TBL1 (B). Store.

Referring to FIG. 12, learning device 4 generates a random number p in the third operation period, and since the generated random number p is equal to or less than ε, it randomly selects channel 1ch, and selects channel 1ch. 1ch is output to the control means 3 as the selected channel CH_Select.

After that, the learning device 4 receives the state “00” of the channel 1ch in the observation period L from the control means 3 . At this point, all the average rewards V3 corresponding to the state "00" of channel 1ch in observation period L _are zero (0) (see correspondence table TBL1(C) in FIG. 11). The learning device 4 generates a random number p, and since the generated random number p is equal to or less than ε, it randomly selects a packet length m=20 μs and outputs the packet length m=20 μs to the control means 3 .

After that, the learning device 4 receives from the control means 3 a signal S_failure indicating that the transmission of the packet has failed.

Then, the learning device 4 calculates the immediate reward R ₃ (=0) by Equation (2B) based on the signal S_failure, and stores the calculated immediate reward R ₃ (=0). Then, the learning device 4 calculates the average reward V ₄ (=0) based on the immediate reward R ₃ (=0) and the average reward V ₃ (=0), and the calculated average reward V ₄ (=0 ) is associated with the packet length m of 20 μs corresponding to the state “00” of channel 1ch in the correspondence table TBL1(D) and stored in the average reward V column.

Referring to FIG. 13, learning device 4 generates a random number p in the fourth operation period, and since the generated random number p is greater than ε, the channel when the maximum average reward V ₃ is obtained is Select 6ch. In the correspondence table TBL1(D), the average reward V ₃ =25/6 and the average reward V ₄ =0, so the average reward V ₃ is the maximum.

When the learning device 4 selects the channel 6ch, it outputs the selected channel 6ch to the control means 3 as the selected channel CH_Select.

After that, the learning device 4 receives the state “01” of the channel 6ch during the observation period L from the control means 3 . At this point, V ₃ (=25/6) is the maximum average reward V corresponding to state "01" of channel 6ch in observation period L (see correspondence table TBL1(D) in FIG. 12). The learning device 4 generates a random number p, and since the generated random number p is not equal to or less than ε, the learning device 4 selects a packet length m=10 μs when the maximum average reward V ₃ (=25/6) is obtained, and a packet Output the length m=10 μs to the control means 3 .

Subsequently, the learning device 4 receives from the control means 3 a signal S_success indicating successful transmission of the packet and an idle period N (=3).

Then, the learning device 4 calculates the immediate reward R ₄ (=10/4) by Equation (2A) based on the signal S_success and the idle period N (=3), and the calculated immediate reward R ₄ (=10/ 4) is stored.

Then, the learning device 4 substitutes the immediate reward R ₄ (=10/4), the average reward V ₃ (=25/6), and n=3 into the equation (1) to obtain the average reward V ₅ =25 /6+(10/4-25/6)/3=65/10. Here, calculating the _average reward V5 by substituting the average reward V3 ₍ =25/6) into the equation (1) corresponds to the state S (="01") of the channel 6ch in the observation period L. This is because the average reward V _t calculated for the packet length m=10 μs is V ₃ (=25/6) stored in the correspondence table TBL1(C) of FIG. Therefore, when calculating the average reward V _t+1 for the packet length m=10 μs corresponding to the state S (="01") of the channel 6ch in the observation period L in the operation period T (=4) by the formula (1), The average reward V ₃ (=25/6) calculated in the action period T (=2) before the action period T (=4) is used as the average reward V _t .

As described with reference to FIGS. 9 to 13, among the three columns of the average reward associated with the state "01" of the channel 6ch in the observation period L, one column contains the average reward V ₃ (=25/6). ) is stored (see the correspondence table TBL1(C) in FIG. 11), during the fourth operation period, when the random number p is not equal to or less than ε, the channel When choosing the length m, we can choose the channel length (=10 μs) for the maximum average reward V ₃ (=25/6) (see correspondence table TBL1(C) in FIG. 11 and correspondence table TBL1(E) in FIG. 13) ).

Also, in the first operation period shown in FIG. 10, the packet length m (=10 μs) is selected for the state “01” of channel 6ch in the observation period L, the packet is successfully transmitted, and the immediate reward R ₁ ( = 10/3) is obtained. Further, in the second operation period shown in FIG. 11, packet length m (=10 μs) is selected for state “01” of channel 6ch in observation period L, and immediate reward R ₂ (=10/2) is obtained. and an average reward V ₃ (=25/6) is obtained. Furthermore, in the fourth operation period shown in FIG. 13, packet length m (=10 μs) is selected for state “01” of channel 6ch in observation period L, and average reward V ₅ (=65/10) is obtained. (See correspondence table TBL1(E) in FIG. 13).

As a result, when the state S of the channel 6ch during the observation period L is "01", the probability of successful packet transmission is high if the packet is transmitted after the observation period L has elapsed. is "01", it can be estimated that there is a high probability that a slot is available after the observation period L has elapsed. This also applies to the states "00", "01", "10", and "11" of each channel in the observation period L.

Therefore, the learner 4 repeatedly performs the learning described with reference to FIGS. It is possible to select the packet length m according to (either "11").

Furthermore, the learner 4 repeatedly performs the learning described in FIGS. 9 to 13. For example, when the packet length m=30 μs is selected for the state “00” of the channel 11ch in the observation period L, the packet , and may learn that choosing a packet length of m=10 μs increases the probability of successful packet transmission.

As a result, the learning device 4 selects the packet length m=10 μs for the state “00” of the channel 11ch in the observation period L when the random number p is not equal to or less than ε. Selecting a packet length of m=10 μs and transmitting a packet means transmitting a packet using a short idle period, and each terminal device may selfishly promote only its own packet transmission. Instead, packets can be transmitted using periods when wireless communication by other terminal devices is idle (in other words, taking wireless communication by other terminal devices into consideration). Therefore, each terminal device can perform wireless communication while coexisting with other terminal devices without acquiring information about the presence or absence of wireless communication by other terminal devices from other terminal devices.

Furthermore, since the learning device 4 selects the packet length m by the ε-greedy method, when the random number p is ε or less, the packet length m is randomly selected. It is possible to search for a packet length m with which a larger average reward V is obtained by suppressing continuous selection of the packet length m with which a larger average reward V is obtained.

When the selected channel CH_Select is selected by the ε-greedy method, when the random number p is not equal to or smaller than ε, the learning device 4 selects all of the states “00”, “01”, “10”, and “11” of each channel. By referring to the associated average reward V column (12 average reward V columns in the correspondence tables TBL1(A) to TBL1(E) shown in FIGS. 9 to 13), the maximum average reward V is obtained as the selection channel CH_Select.

When the packet length m is selected by the ε-greedy method, the learning device 4 selects the state S (“00”, “01”, “10” , “11”) (in correspondence tables TBL1(A) to TBL1(E) shown in FIGS. 9 to 13, three average reward V columns ), and select the packet length m that gives the maximum average reward V as the packet length m.

FIG. 14 is a flow chart for explaining the operation of the terminal device 10 shown in FIG. Referring to FIG. 14, when the operation of terminal device 10 is started, control means 3 determines whether there is transmission data D_TR (step S1). In this case, when the control means 3 receives the transmission data D_TR from the application 6, it determines that there is transmission data, and when it does not receive the transmission data D_TR from the application 6, it determines that there is no transmission data.

In step S1, when it is determined that there is transmission data D_TR, the control means 3 generates a signal S_req_m requesting the packet length m and outputs it to the learning device 4. FIG.

In parallel with the operations of the terminal device 10 other than the learner 4, the learner 4 acquires the state S of the selected channel during the observation period L, the communication result when the packet is transmitted, and the idle period N after the completion of the packet transmission. and selects the channel for which the maximum average reward V _t+1 is obtained with a predetermined probability as the selected channel, and according to the state S of the selected channel during the observation period L, is obtained as the packet length _m (step S2). where the average reward V _t+1 is the average reward in the action period T+1.

Upon receiving the signal S_req_m from the control means 3 , the learning device 4 outputs the selected packet length m to the control means 3 .

When receiving the packet length m from the learning device 4, the control means 3 detects the transmission data D_m having the data amount AOD when the packet length L_PKT of the transmission packet PKT becomes the packet length m from the transmission data D_TR, and detects the transmission data D_m. A transmission packet PKT containing the transmitted data D_m is generated (step S3). Then, the control means 3 generates a signal S_req_CH requesting the selected channel and outputs it to the learning device 4 .

Upon receiving a signal S_req_CH from the control means 3, the learning device 4 outputs the selected selected channel to the control means 3. Upon receiving the state S of the selected channel during the observation period L from the control means 3, the learning device 4 sets the packet length m to Output to the control means 3 .

Upon receiving the selected channel from the learner 4 after step S3, the control means 3 detects the state S of the selected channel during the observation period L by the method described above based on the received power spectrum PW_carrier_L received from the receiving means 2. (step S4), and the detected state S of the selected channel in the observation period L is output to the learning device 4. FIG.

After step S4, the control means 3 determines whether or not the selected channel is available based on the result of carrier sense received from the receiving means 2 (the result of carrier sense in the selected channel) (step S5).

In step S5, when it is determined that the selection channel is available, the control means 3 outputs the selection channel CH_Select and the packet (packet for transmission PKT) to the transmission means 5, and the transmission means 5 uses the selection channel CH_Select. Then, the packet (packet for transmission PKT) received from the control means 3 is transmitted with the packet length m (step S6).

After that, the control means 3 detects the communication result when the packet is transmitted based on the presence or absence of the ACK packet (step S7), and outputs the detected communication result to the learning device 4. FIG. Then, based on the received power spectrum PW_chn received from the receiving means 2, the control means 3 detects the idle period N after the completion of packet transmission (step S8), and detects the detected idle period N by the method described above. Output to learning device 4 .

Then, the control means 3 determines whether or not the packet transmission success rate is equal to or less than the threshold value (step S9).

When it is determined in step S9 that the packet transmission success rate is equal to or less than the threshold, the control means 3 changes the candidate channel to another candidate channel (step S10), and changes the changed candidate channel to Output to learning device 4 .

Then, when it is determined in step S9 that the packet transmission success rate is not equal to or lower than the threshold value, or after step S10, the series of operations proceeds to step S1.

In the flowchart shown in FIG. 14, steps S1 to S10 are repeatedly executed as long as the terminal device is driven.

Also, in the flowchart shown in FIG. 14, when the process moves from step S10 to step S1, the selection channel used for packet transmission is selected from other candidate channels (see step S2).

15 and 16 are first and second flow charts, respectively, for explaining the operation of the learning device 4 shown in FIG. Referring to FIG. 15, when the operation of learning device 4 is started, learning device 4 receives candidate channels from control means 3 (step S21).

Then, the learning device 4 initializes the average reward by setting all the average rewards in the correspondence table TBL1 to zero (=0) (step S22).

After that, the learning device 4 generates a random number p between 0 and 1 (step S23). Then, the learning device 4 determines whether or not the random number p is equal to or less than ε (step S24).

When it is determined in step S24 that the random number p is not equal to or less than ε, the learning device 4 determines whether or not the maximum average reward V _t+1 exists in the correspondence table TBL1 (step S25).

When it is determined in step S25 that the maximum average reward V _t+1 exists in the correspondence table TBL1, the learning device 4 selects the channel from which the maximum average reward V _t+1 is obtained from the candidate channels (step S26). Note that when there are a plurality of maximum average rewards V _{t+1 ,} the learning device 4 selects a channel from which any one maximum average reward V t+1 of the plurality of maximum average rewards V _t+1 is obtained from the candidate channels. select.

On the other hand, when it is determined in step S24 that the random number p is equal to or less than ε, or when it is determined in step S25 that the maximum average reward V _t+1 does not exist in the correspondence table TBL1, the learning device 4 selects the candidate channel A channel is randomly selected from (step S27).

After step S26 or step S27, learning device 4 outputs the selected channel to control means 3 as a selected channel (step S28).

After that, the learning device 4 receives the state S of the selected channel during the observation period L from the control means 3 (step S29).

Then, the learning device 4 generates a random number p between 0 and 1 (step S30), and determines whether or not the generated random number p is equal to or less than ε (step S31).

When it is determined in step S31 that the random number p is not equal to or less than ε, the learning device 4 determines whether or not the maximum average reward V _t+1 exists in the correspondence table TBL1 (step S32).

In step S32, when it is determined that the maximum average reward V _{t +1} exists in the correspondence table TBL1, the learning device 4 performs is selected (step S33). Note that when there are multiple maximum average rewards V _t+1 , the learning device 4 selects the packet length m when any one maximum average reward V _t+1 is obtained from the multiple maximum average rewards V _t+1 . do.

On the other hand, when it is determined in step S31 that the random number p is equal to or less than ε, or when it is determined in step S32 that the maximum average reward V _t+1 does not exist in the correspondence table TBL1, the learner 4 randomly selects A packet length m is selected (step S34).

After step S33 or step S34, the learning device 4 outputs the selected packet length m to the control means 3 (step S35). After that, the series of operations proceeds to step S36 in FIG.

Referring to FIG. 16, after step S35 in FIG. 15, learning device 4 receives a packet transmission result from control means 3 (step S36). Subsequently, the learning device 4 receives the vacant period N after completion of packet transmission from the control means 3 (step S37).

Then, learning device 4 calculates an immediate reward _Rt by Equation (2) using the packet transmission result, idle period N, and packet length m (step S38), and stores the calculated immediate reward _Rt . The immediate reward _Rt is the immediate reward in the action period T.

After that, the learning device 4 calculates the average reward V _t+1 according to the equation (1) using the immediate reward R _t (step S39), and assigns the average reward V _t+1 to the state S of the selected channel in the observation period L as the correspondence table Store in TBL1 (step S40).

Then, the series of operations proceeds to step S41 in FIG. 15, and learning device 4 determines whether or not another candidate channel has been received from control means 3 (step S41).

When it is determined in step S41 that another candidate channel has not been received from the control means 3, the series of operations proceeds to step S23.

On the other hand, when it is determined in step S41 that another candidate channel has been received from the control means 3, the series of operations proceeds to step S22.

In the flowchart shown in FIG. 14, the learning device 4 repeatedly executes steps S21 to S41 of the flowcharts shown in FIGS. 15 and 16 in parallel with the operations of the terminal device 10 other than the learning device 4. FIG.

When learning device 4 receives signal S_req_m from control means 3 after step S1 shown in FIG. 14, learning device 4 outputs packet length m to control means 3 (see step S35) in step S2 shown in FIG. When the signal S_req_CH is received from the control means 3 after step S3 shown in FIG. 14, the selected channel is output to the control means 3 in step S2 shown in FIG. 14 (see step S28).

According to the flow charts shown in FIGS. 15 and 16, the learning device 4 selects from the candidate channels the channel when the maximum average reward V _t+1 is obtained with a probability of 1−ε (see step S26), and the probability of ε randomly select a channel from the candidate channels (see step S27). Then, it is determined by the generated random number p whether to select the channel when the maximum average reward Vt ₊₁ is obtained or to select the channel at random (see steps S23 and S24).

Therefore, a packet can be transmitted by selecting a channel when the maximum average reward V _t+1 is obtained with a probability of 1-ε, and a channel can be randomly selected and transmitted with a probability of ε. It is possible to avoid collisions with wireless communication by other terminal devices and increase the probability of successful packet transmission, compared to the case where packets are transmitted by continuous use. As a result, the terminal device 10 can coexist with other terminal devices and perform wireless communication.

Further, according to the flowcharts shown in FIGS. 15 and 16, the learning device 4 adds the average reward If V _t+1 is stored, select the packet length m when the maximum average reward V _t+1 is obtained with a probability of 1−ε (“YES” in step S35, see step S36).

Since the maximum average reward V _t+1 is obtained, it means that the packet has been successfully transmitted with the selected packet length m. 01", "10", and "11", the packet length m for successful packet transmission is determined. Therefore, by changing the packet length m according to the states "00", "01", "10" and "11" of the selected channel during the observation period L, the probability of successful packet transmission can be increased.

Then, for the state S of the selected channel in the observation period L, selecting the packet length m when the maximum average reward V _t+1 is obtained means that the packet length m is equivalent to selecting

In this case, for example, the packet length m1 of the _first length is selected for the state S (="00") of the selected channel in the observation period L, and the state S (="00") of the selected channel in the observation period L 01"), a second packet length _m2 longer than the first length is selected, and for the selected channel state S (="10") in the observation period L, A short _third packet length m3 is selected, and for the selected channel state S (="11") in the observation period L, a _fourth packet length m4 longer than the second length is selected ( m ₃ < m ₁ < m ₂ < m ₄ ).

Then, since the maximum average reward V _t+1 is obtained, when the packet is transmitted with any one of the packet lengths m ₁ to m ₄ , the packet is successfully transmitted, and the packet is proportional to the packet length m , resulting in an immediate reward R that is inversely proportional to the idle period N, resulting in a larger average reward V _t+1 .

In this case, if the vacant period N after the completion of packet transmission becomes shorter, it means that wireless communication is being performed by another terminal device during the period during which the vacant period N is observed. Wireless communication can be performed while coexisting with the terminal device.

Further, by repeating learning according to the flowcharts shown in FIGS. It can be seen that there is a certain trend between the length of idle state.

In the above description, the observation period L is two slots SL, and the channel state S in the observation period L is represented by "00", "01", "10", and "11". In the embodiment, not limited to this, the observation period L is assumed to be three slots SL, and the channel state S in the observation period L is represented by "000", "001", "010", "011", It may be represented by "100", "101", "110", "111", and assuming that the observation period L is four slots SL or more, the channel state S in the observation period L is represented by 4 bits or more. may

The longer the observation period L, the easier it is to obtain the correlation between the channel state S in the observation period L and the vacant slots SL. can be easily selected.

Also, in the above description, packets are transmitted in units of slot SL, but in the embodiment of the present invention, it is not limited to this, and packets may not be transmitted in units of slot SL.

FIG. 17 is a diagram for explaining different methods of determining the packet length m. Referring to FIG. 17, control means 3 receives transmission data from application 6 . Then, for example, the length of the observation period L is 200 μs, and the state S of each channel in the observation period L is represented by 20 bits by determining whether it is in a busy state or an idle state every 10 μs by the method described above. do. Also, as the selectable packet length m, for example, 10 μs, 20 μs, 30 μs, . . . , 100 μs are set. Then, the candidate channels, the state S of each channel in the observation period L, and the selectable packet lengths 1 to M are associated with each other to create a correspondence table having the same structure as the correspondence table TBL1.

When selecting a packet length of 10 μs for the state S of the selected channel during the observation period L, the control means 3 detects transmission data D1 having a length of 10 μs from the transmission data and generates a packet. Further, when transmitting a packet having a packet length of 20 μs at the next timing, the control means 3 detects transmission data D2 having a length of 20 μs from a portion following transmission data D1 and generates a packet. Furthermore, when transmitting a packet having a packet length of 30 μs at the next timing, the control means 3 detects transmission data D3 having a length of 30 μs from the portion following transmission data D2 and generates a packet. Thereafter, the control means 3 similarly detects transmission data having a length matching the selected packet length m and generates a packet.

When the slot SL unit is not used, the idle period N when calculating the immediate reward _Rt is detected by determining whether it is in a busy state or an idle state in units of 10 μs. Then, in order to be able to calculate the immediate reward _Rt even when the idle period N is zero (=0), a predetermined time length (for example, a time length of 10 μs) is added, and the reciprocal of the addition result is is multiplied by the packet length m to calculate the immediate reward.

Note that the operation of the terminal device 10 may be realized by software. In this case, the terminal device 10 includes a CPU (Central Processing Unit), a ROM (Read Only Memory) and a RAM (Random Access Memory). The ROM stores a program Prog_A consisting of steps of the flowchart shown in FIG. 14 (including the flowcharts shown in FIGS. 15 and 16).

The CPU reads the program Prog_A from the ROM, executes the read program Prog_A, selects a packet length m suitable for the state S of the selected channel during the observation period L, and transmits the packet. The RAM temporarily stores the calculated immediate reward R and the like.

Also, the program Prog_A may be recorded on a recording medium such as a CD or DVD and distributed. When the recording medium recording the program Prog_A is loaded into the computer, the computer reads the program Prog_A from the recording medium, executes it, selects the packet length m suitable for the state S of the selected channel during the observation period L, and transmits the packet. Send.

Therefore, the recording medium recording the program Prog_A is a computer-readable recording medium.

According to the above-described embodiments, the terminal device according to the embodiments of the present invention is
communication means for transmitting packets using a transmission channel, which is a channel for transmitting packets, during the first operation period;
a first detection means for detecting, in a first operation period, a communication result when the packet is transmitted and an idle period of wireless communication after the packet is transmitted, each time the packet is transmitted by the communication means; ,
a second detection means for detecting the state of the transmission channel in an observation period, which is a period for observing the presence or absence of wireless communication by other terminal devices, each time the transmission channel is received;
Accepting the communication result detected in the first operation period, the idle period, the state of the transmission channel in the observation period, and the candidate channel that is a candidate for the channel used for packet transmission, and based on the communication result and the idle period , a first process of calculating an immediate reward, which is a reward obtained when a packet is transmitted in the transmission channel during the first operation period, and one packet length corresponding to the state of the transmission channel during the observation period. The average reward that is the reward obtained by averaging the cumulative value of the immediate reward in one transmission channel by the selected number of times and the reward in the second operation period that is the operation period after the first operation period is the first Creates or updates a correspondence table that associates the second process calculated using the immediate reward calculated in the process of 1, the candidate channel, the state of the transmission channel in the observation period, the packet length of the packet, and the average reward. , based on the created or updated correspondence table, selects a channel for transmission with a predetermined probability as a channel for obtaining the maximum average reward, and obtains the maximum average reward according to the state of the transmission channel during the observation period. A third process of selecting the packet length when the packet is available with a predetermined probability and outputting the selected transmission channel and packet length is executed each time the state of the transmission channel, the communication result, and the idle period in the observation period are received. and a learner that
Further, each time the communication means receives from the learning device the transmission channel and the packet length selected in the third processing, during the second operation period, when the received transmission channel is free, from the learning device Any device that transmits a packet having the received packet length may be used.

If the terminal device has such a configuration, it selects a packet length suitable for the state of the transmission channel during the observation period, and after the observation period has passed, transmits a packet suitable for the state of the transmission channel during the observation period. This is because it is possible to transmit packets in a long packet, and wireless communication by other terminal devices is also possible, so that wireless communication can be performed while coexisting with other terminal devices.

Also, the program according to the embodiment of the present invention is
a first step in which the communication means transmits packets using a transmission channel, which is a channel for transmitting packets, during a first operation period;
The first detection means detects, in the first operation period, the communication result at the time the packet is transmitted each time the packet is transmitted in the first step, and the idle period of wireless communication after the packet is transmitted. a second step of detecting
A third detection means for detecting the state of the transmission channel in an observation period, which is a period for observing the presence or absence of wireless communication by other terminal devices, in the first operation period each time the second detection means receives the transmission channel. a step of
A learning device receives the communication result detected in the first operation period, the idle period, the state of the transmission channel in the observation period, and candidate channels that are candidates for channels used for packet transmission, and receives the communication result and A first process of calculating an immediate reward, which is a reward obtained when a packet is transmitted in the transmission channel during the first operation period, based on the vacant period, and corresponding to the state of the transmission channel during the observation period. It is a reward obtained by averaging the cumulative value of immediate rewards in one transmission channel according to the number of times one packet length is selected, and is a reward in a second operation period that is an operation period after the first operation period. A second process for calculating the average reward using the immediate reward calculated in the first process, and a correspondence table that associates the candidate channel, the state of the transmission channel in the observation period, the packet length of the packet, and the average reward. is created or updated, and based on the created or updated correspondence table, the channel that yields the maximum average reward is selected as the transmission channel with a predetermined probability, and the maximum and a third process of selecting the packet length when the average reward of is obtained with a predetermined probability, and outputting the selected transmission channel and packet length, and the state of the transmission channel, the communication result, and the idle period during the observation period. cause the computer to execute a fourth step that is executed each time the
In the first step, the communication means further receives the transmission channel and the packet length selected in the third process from the learner, during the second operation period, when the received transmission channel becomes available. It is sufficient if the packet having the packet length received from the learner is transmitted when the learning device is on.

When the program causes the computer to execute the first to fourth steps, a packet length suitable for the state of the transmission channel during the observation period is selected, and after the observation period has passed, the state of the transmission channel during the observation period is selected. This is because a packet can be transmitted with a packet length suitable for , wireless communication by another terminal device is also possible, and wireless communication can be performed while coexisting with the other terminal device.

In this embodiment of the invention, the selected channel CH_Select selected from the candidate channels constitutes a "channel for transmission".

Further, in the embodiment of the present invention, the receiving means 2 that detects the received power spectrum PW_carrier_L and receives an ACK packet, and the control means 3 that detects the idle period N based on the received power spectrum PW_chn 1 detection means”.

Furthermore, in the embodiment of the present invention, the receiving means 2 that detects the received power spectrum PW_carrier_L and the control means 3 that detects the state S of the selected channel CH_Select in the observation period L based on the received power spectrum PW_carrier_L are: constitute a second detecting means.

Furthermore, in the embodiment of the present invention, the control means 3 for outputting packets to the transmission means 5 and the transmission means 5 for transmitting packets constitute "communication means".

Furthermore, in the embodiment of the present invention, when packets are transmitted in slot SL units, "1" added to the idle period N when calculating the immediate reward _Rt means one slot SL. Therefore, "N+1" in equation (2A) substantially means adding the time length of one slot SL to the time length of N slots SL. Also, when packets are not transmitted in slot SL units, when calculating the immediate reward _Rt , a predetermined length of time (for example, a length of time of 10 μs) is added to the idle period N (consisting of the total idle state of 10 μs). be done. As a result, when calculating the immediate reward _Rt , a predetermined length of time is added to the vacant period N both when packets are transmitted in slot SL units and when packets are not transmitted in slot SL units. become. Therefore, when packets are transmitted in slot SL units, "1" is added to the idle period N when calculating the immediate reward _Rt , and when packets are not transmitted in slot SL units, the immediate reward _Rt A predetermined length of time (for example, a length of 10 μs) added to the vacant period N when calculating constitutes a “predetermined period” added to the vacant period N.

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the description of the above-described embodiments, and is intended to include all modifications within the scope and meaning equivalent to the scope of the claims.

The present invention is applied to a terminal device, a program to be executed by a computer, and a computer-readable recording medium recording the program.

REFERENCE SIGNS LIST 1 antenna, 2 receiving means, 3 control means, 4 learning device, 5 transmitting means, 6 application, 10 terminal device, 100 communication system.

Claims (15)

communication means for transmitting the packet using a transmission channel, which is a channel for transmitting the packet, during a first operation period;
each time the packet is transmitted by the communication means, in the first operation period, a communication result when the packet is transmitted is detected, and an idle period of wireless communication after transmission of the packet is detected; 1 detection means;
a second detection means for detecting the state of the transmission channel in an observation period, which is a period for observing the presence or absence of wireless communication by other terminal devices, each time the transmission channel is received, in the first operation period; ,
receiving the communication result detected in the first operation period, the state of the transmission channel in the idle period and the observation period, and a candidate channel that is a candidate for the channel to be used in the packet transmission, and performing the communication; a first process of calculating an immediate reward, which is a reward obtained when the packet is transmitted over the transmission channel in the first operation period, based on the result and the idle period; A reward obtained by averaging the cumulative value of the immediate reward in one transmission channel according to the number of times one packet length corresponding to the state of the transmission channel is selected, and an operation period after the first operation period. using the immediate reward calculated in the first process, and the state of the candidate channel and the transmission channel in the observation period and the packet length of the packet and the average reward are created or updated, and based on the created or updated correspondence table, the channel when the maximum average reward is obtained is selected with a predetermined probability. Selecting the transmission channel and selecting, with the predetermined probability, the packet length when the maximum average reward is obtained according to the state of the transmission channel during the observation period, and selecting the selected transmission channel and packet a third process for outputting a length, and a learning device that executes each time the state of the transmission channel in the observation period, the communication result, and the idle period are received,
Each time the communication means receives the transmission channel and packet length selected in the third process from the learning device, during the second operation period, when the received transmission channel is available , a terminal device that transmits a packet having a packet length received from the learner. In the first processing, the learning device calculates the immediate reward as zero when the communication result is a failure of transmission of the packet, and when the communication result is a success of transmission of the packet, the 2. The terminal device according to claim 1, wherein a reciprocal of an addition result obtained by adding a predetermined period to an idle period is calculated as the immediate reward. In the second processing, the learning device calculates one packet length corresponding to the immediate reward in the first operation period, the average reward in the first operation period, and the state of the transmission channel in the observation period. 3. The terminal device according to claim 1, wherein an average reward in said second operation period is calculated based on the number of times selected and said average reward is updated. In the second processing, the learning device sets the immediate reward in the first action period to _Rt , the average reward in the first action period to _Vt , and the average reward in the second action period to Let V _t+1 and let n be the number of times one packet length is selected corresponding to the state of the transmission channel during the observation period (n is an integer equal to or greater than 1). 4. A terminal according to claim 3, wherein said average reward is updated by calculating an average reward Vt ₊₁ .
V _t+1 =V _t +(R _t −V _t )/n (1) In the third processing, the learning device selects the channel having the maximum average reward in the second operation period with probability (1−ε) (ε is a real number in the range of 1 to 0). 5. The terminal apparatus according to claim 3, wherein the terminal apparatus selects from candidate channels as said transmission channel, and selects an arbitrary channel from said candidate channels as said transmission channel with probability ?. 3 to 4, wherein in the third processing, the learning device selects a packet length that maximizes the average reward in the second operation period with respect to the state of the transmission channel in the observation period. 6. The terminal device according to any one of 5. when the transmission success rate, which is the probability of successful transmission of the packet, is equal to or less than a threshold, selecting a channel in a band different from the band of the candidate channel as a new candidate channel, and using the selected new candidate channel. further comprising a control means for controlling the learner such that
The learning device performs the first process, the second process, and the third process using the new candidate channel, and obtains the state of the transmission channel, the communication result, and the idle period during the observation period. 7. The terminal device according to any one of claims 1 to 6, which is executed each time it is received. a first step in which the communication means transmits the packet using a transmission channel, which is a channel for transmitting the packet, during a first operation period;
A first detection means detects, in the first operation period, a communication result when the packet is transmitted each time the packet is transmitted in the first step, and after transmission of the packet. a second step of detecting an idle period of wireless communication;
A second detection means detects the state of the transmission channel during an observation period, which is a period for observing the presence or absence of wireless communication by other terminal devices, in the first operation period, each time the transmission channel is received. a third step of
A learning device learns the communication result detected in the first operation period, the state of the transmission channel in the idle period, and the observation period, and a candidate channel that is a candidate for the channel used to transmit the packet. a first process of calculating an immediate reward, which is a reward obtained when the packet is transmitted through the transmission channel in the first operation period, based on the acceptance, the communication result, and the idle period; A reward obtained by averaging the cumulative value of the immediate rewards in one transmission channel by the number of times one packet length corresponding to the state of the transmission channel is selected in the observation period, and in the first operation period a second process of calculating an average reward, which is a reward in a second operation period, which is a later operation period, using the immediate reward calculated in the first process; creating or updating a correspondence table that associates the state of the credit channel, the packet length of the packet, and the average reward, and determining the channel when the maximum average reward is obtained based on the created or updated correspondence table and selecting the packet length when the maximum average reward is obtained according to the state of the transmission channel in the observation period with the predetermined probability, and selecting the selected transmission channel with the probability of causing a computer to execute a third process of outputting a trusted channel and a packet length, and a fourth step of executing each time the state of the transmission channel, the communication result, and the idle period during the observation period are received;
In the first step, the communication means further receives the transmission channel and the packet length selected in the third process from the learner, during the second operation period, the received transmission. A program for executing a computer that transmits a packet having a packet length received from said learner when a trusted channel is free. In the first processing of the fourth step, the learning device calculates the immediate reward as zero when the communication result is a failure to transmit the packet, and the communication result is failure to transmit the packet. 9. The program to be executed by a computer according to claim 8, wherein, when successful, a reciprocal of an addition result obtained by adding a predetermined period to said idle period is calculated as said immediate reward. In the second processing of the fourth step, the learning device corresponds to the immediate reward in the first operation period, the average reward in the first operation period, and the state of the transmission channel in the observation period. 10. The program to be executed by a computer according to claim 8 or 9, wherein the average reward in the second operation period is calculated based on the number of times one packet length is selected and the average reward is updated. . In the second processing of the fourth step, the learner sets the immediate reward in the first action period to R _t , the average reward in the first action period to V _t , and the second Let V _t+1 be the average reward during the operation period, and let n be the number of times one packet length is selected corresponding to the state of the transmission channel during the observation period (n is an integer of 1 or more). 11. The computer-implemented program according to claim 10, wherein said average reward is updated by calculating the average reward Vt ₊₁ according to formula (1) of .
V _t+1 =V _t +(R _t −V _t )/n (1) In the third processing of the fourth step, the learning device has a probability of (1−ε) (ε is a real number in the range of 1 to 0) that the average reward in the second action period is 12. The computer according to claim 10, wherein the maximum channel is selected as the transmission channel from the candidate channels, and an arbitrary channel is selected from the candidate channels as the transmission channel with probability ε. program for. wherein, in the third processing of the fourth step, the learning device selects a packet length that maximizes the average reward in the second operation period for the state of the transmission channel in the observation period; A program to be executed by the computer according to any one of claims 10 to 12. The control means selects a channel having a band different from that of the candidate channel as a new candidate channel when the transmission success rate, which is the probability of successful transmission of the packet, is equal to or less than a threshold value, and further causing the computer to perform a fifth step of controlling the learner to use a candidate channel;
The learning device performs the first process, the second process, and the third process using the new candidate channel, and obtains the state of the transmission channel, the communication result, and the idle period during the observation period. 14. The program to be executed by the computer according to any one of claims 8 to 13, which is executed each time it is received. A computer-readable recording medium recording the program according to any one of claims 8 to 14.

Images