KR102183429B1 - Communication method and system through machine learning based comtention window control - Google Patents

Abstract

Provided is a communication method using WAVE PHY, WAVE lower MAC, and WAVE upper MAC defined in IEEE802.11p and IEEE1609.4. According to the present invention, the communication method comprises the steps of: determining whether a first STA receiving beacon and a safety message in a control channel (CCH) session through broadcast received an ACK message from a second STA; and changing, by the first STA, a contention window (CW) based on whether the ACK message is received.

Description

Communication method and system through machine learning based competition window control {COMMUNICATION METHOD AND SYSTEM THROUGH MACHINE LEARNING BASED COMTENTION WINDOW CONTROL}

The present invention relates to a communication method and system through machine learning-based contention window control.

Vehicle-to-Vehicle (V2V) is a technology that provides connectivity between vehicles that have been driving with the existing Stand-Alone, and can improve the safety of vehicle operation and provide various vehicle convenience functions.

For example, using V2V technology, the vehicle self-checks the vehicle distance and speed through sensors attached to the vehicle, and services such as safe driving, traffic congestion prevention, and traffic accident prevention are provided through two-way communication between vehicles. Can be provided.

Therefore, V2V is emerging as a core technology of various intelligent vehicle services that have recently attracted attention such as Intelligent Transport Systems (ITS) and Connected Car.

On the other hand, IEEE802.11p WAVE communication technology, a high-speed mobile communication standard for vehicles for V2V implementation, is a high-speed mobile communication using OFDM. In addition to the communication performance in a high-speed mobile environment such as multi-pass fading according to the vehicle environment, safety as a Basic Safety Message Congestion channel management is required to support smooth communication of packets that must be guaranteed to be transmitted. To this end, the WAVE communication system operates 7 channels of a 10MHz band in the 5.855~5925GHz band, and the service utilization is improved through the IEEE1609.0~IEEE1609.4 standards.

In addition, for efficient channel management in the WAVE communication system, the IEEE 1609.4 standard channel coordination service is supported, which synchronizes the reference time such as using a GPS PPS signal or Timing Advertisement (TA) packet to set a certain time interval. It supports to use the Control Channel (CCH) for communication system operation and safety message transmission and the Service Channel (SCH) for service provision interchangeably. According to the IEEE802.11 standard, the WAVE communication system provides an Enhanced Distributed Channel Access (EDCA) method in the MAC layer to avoid packet collisions, and each of the four priority queues has a contention window (CW) for backoff. By randomizing the time, the focus is on improving the randomness in packet transmission in the entire system.

Furthermore, when performing broadcast transmission in IEEE802.11p, a backoff algorithm is used to reduce collisions that may occur when performing broadcast transmission in IEEE802.11p. However, in the case of broadcast transmission, since it is not possible to know whether there is a collision, a fixed minimum contention window (CW _min ) is used regardless of whether the transmission is successful. In other words, after selecting a random value from [0,CW _min ], it waits for a corresponding time (Backoff delay) and tries to transmit broadcast.

At this time, when multiple nodes transmit and use data at the same time, frequent transmission failures may occur due to collision. Therefore, there is a need to reduce the number of collisions by adjusting the size of the contention window to increase the reception success rate. .

Registered Patent Publication No. 10-1919442, 2018.11.12

The problem to be solved by the present invention is to provide a communication method and system through machine learning-based contention window control.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

The communication method using WAVE PHY, WAVE lower MAC and WAVE upper MAC defined in IEEE802.11p and IEEE1609.4 according to an aspect of the present invention for solving the above-described problem, broadcasts a beacon and a safety message in a CCH section. Determining, by the first STA received through, whether or not the ACK message has been received from the second STA; And changing, by the first STA, a contention window (CW) based on whether or not the ACK message is received. Includes.

In this case, the CCH interval includes a CW Suitability Verification Interval (CSV) for performing unicast communication in the CCH, and the CSV interval may be a time interval of T _csv .

In this case, the first STA and the second STA may transmit and receive a packet to check the current contention window.

In this case, when the contention window is the minimum contention window, setting the Q table as an initial Q table by the minimum contention window; And updating the contention window based on the following Q-Learning model. Including more,

는 학습 속도,

는 디스카운트 팩터,

는 t에서의 리워드 값이고,At this time, s _t is the state at t (the size of CW), a _t is the possibility of action at t,

Is the learning rate,

Is the discount factor,

Is the reward value at t,

The Q table includes first to seventh rows for the size of the contention window and first to third columns for the BE parameter for determining an action for changing the size of the contention window. The first row is the size of the contention window 3, the second row of the Q table is the size of the contention window, the third row of the Q table is the size of the contention window is 15, and the fourth row of the Q table is a contention window. The size of is 31, the fifth row of the Q table has a size of a contention window 63, the sixth row of the Q table has a size of a contention window 127, and the seventh row of the Q table has a size of a contention window 255. ,

The first column of the Q table is a BE parameter BE-1, the second column of the Q table is a BE parameter BE, and the third column of the Q table is a BE parameter BE+1,

The initial Q table,

The value corresponding to the first row, the first column is -100, the value corresponding to the first row, the second column is 1/3, the value corresponding to the first row, the third column is 1/7, and the value corresponding to the first column of the second row This 1/3, the value corresponding to the second row, the second column is 1/7, the value corresponding to the second row, the third column is 1/15, the value corresponding to the third row, the first column is 1/7, and the third row The value corresponding to the second column is 1/15, the value corresponding to the third row and the third column is 1/31, the value corresponding to the fourth row, the first column is 1/15, and the value corresponding to the second column of the fourth row is 1 /31, the value corresponding to row 4, column 3 is 1/63, the value corresponding to row 5, column 1 is 1/31, the value corresponding to row 5, column 2 is 1/63, row 5 is 3 The value corresponding to the column is 1/127, the value corresponding to the first column of the 6th row is 1/63, the value corresponding to the second column of the 6th row is 1/127, the value corresponding to the third column of the 6th row is 1/255 , A value corresponding to the first column of the seventh row may be 1/127, a value corresponding to the second column of the seventh row may be 1/255, and a value corresponding to the third column of the seventh row may be -100.

At this time, in the updating step, the column corresponding to the largest value among the values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table updated with a probability of 0.9 at each update time. Selecting by action; And selecting, as an action, a column corresponding to an arbitrary value among values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table updated with a probability of 0.1. .

In this case, the step of selecting the largest value as an action corresponds to a value in column BE-1 when the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table are all the same. Selecting a row to be an action; Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE-1 and column BE are the same, and the values in column BE+1 are greater than the same two values. If small, selecting a column corresponding to the value of column BE-1 as an action; Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE-1 and column BE+1 are the same, and the values in column BE are greater than the same two values. If small, selecting a column corresponding to the value of column BE-1 as an action; And the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE and column BE+1 are the same, and the values in column BE-1 are the same. If smaller than that, selecting a column corresponding to the value of the BE column as an action; may include.

In this case, in the step of selecting the largest value as an action, the first of the values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table is the value of column BE-1. , When the second largest value is the value in column BE+1, and the difference between the value in column BE-1 and the value in column BE+1 is within a preset range, the column corresponding to the value in column BE-1 is selected as an action. Step; may include.

In this case, the CSV interval may be determined based on the following equation.

Other specific details of the present invention are included in the detailed description and drawings.

According to various embodiments of the present invention described above, there is an effect of increasing a reception success rate by reducing transmission failures in V2V-based broadcast transmission.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 to 5 are exemplary diagrams for explaining a conventional WAVE communication method according to IEEE802.11p and IEEE1609.4.
6 is an exemplary diagram for explaining the structure of a system according to an embodiment of the present invention.
7 is an exemplary diagram illustrating a CCH frame structure according to an embodiment of the present invention.
8 is a flowchart illustrating a communication method according to an embodiment of the present invention.
9 is a flowchart illustrating a Q-Learning method according to an embodiment of the present invention.
10 is a flowchart illustrating an action selection method according to an embodiment of the present invention.
11 is a flowchart illustrating an action selection method according to another embodiment of the present invention.
12 is a block diagram of an apparatus according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, only the present embodiments are intended to complete the disclosure of the present invention, It is provided to fully inform the technician of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terms used in the present specification are for describing exemplary embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used in the specification, “comprises” and/or “comprising” do not exclude the presence or addition of one or more other elements other than the mentioned elements. Throughout the specification, the same reference numerals refer to the same elements, and “and/or” includes each and all combinations of one or more of the mentioned elements. Although “first”, “second”, and the like are used to describe various elements, it goes without saying that these elements are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical idea of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings that can be commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not interpreted ideally or excessively unless explicitly defined specifically.

The term "unit" or "module" used in the specification refers to a hardware component such as software, FPGA or ASIC, and the "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium, or may be configured to reproduce one or more processors. Thus, as an example, "sub" or "module" refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, It includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided within "sub" or "module" may be combined into a smaller number of components and "sub" or "modules" or into additional components and "sub" or "modules". Can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc., as shown in the figure It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, if a component shown in a drawing is turned over, a component described as "below" or "beneath" of another component will be placed "above" the other component. I can. Accordingly, the exemplary term “below” may include both directions below and above. Components may be oriented in other directions, and thus spatially relative terms may be interpreted according to orientation.

In the present specification, a computer refers to all kinds of hardware devices including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to embodiments. For example, the computer may be understood as including all of a smartphone, a tablet PC, a desktop, a laptop, and a user client and an application running on each device, but is not limited thereto.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each of the steps described herein is described as being performed by a computer, but the subject of each step is not limited thereto, and at least some of the steps may be performed by different devices according to embodiments.

1 to 5 are exemplary diagrams for explaining a conventional WAVE communication method according to IEEE802.11p and IEEE1609.4.

As shown in FIG. 1, the WAVE structure is defined by IEEE802.11p and IEEE1609.x (IEEE1609.0 to IEEE1609.4). Specifically, IEEE802.11p defines WAVE PHY and WAVE lower MAC, and IEEE1609.11 defines WAVE upper MAC.

In the present invention, unless otherwise specified, techniques for V2V communication may be performed by IEEE802.11p and IEEE1609.x. Further, in the present invention, although the description is based on V2V, the same technical idea may be applied to V2I (communication between vehicle and infrastructure) and V2X (communication including V2V and V2I).

Compared with the existing IEEE 802.11a/g, the WAVE PHY layer defined in IEEE802.11p has the effect of reducing the effect of fading that occurs frequently in high-speed movement and outdoor environments by reducing the channel bandwidth. Its main characteristics are communication frequency (5.850~5.925GHz), transmission speed (3~27Mbps), number of channels (7), transmission range (300m~1Km), and channel bandwidth (10MHz).

Meanwhile, as shown in FIG. 2, the WAVE lower MAC defined in IEEE802.11p operates by the Distributed Coordination Function (DCF). Specifically, DCF performs channel access through contention through Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA). At this time, the WAVE lower MAC may determine an arbitrary waiting time using a Binary Exponential Backoff (BEB) algorithm. Specifically, a backoff time may be determined by selecting an arbitrary value from values from 0 to the contention window (CW), and as shown in FIG. 3, when a collision occurs, the value of the contention window Can be increased by 2 times. In this case, the backoff time may be determined based on an equation such as Backoff Time = Random()*SlotTime. In this case, according to an embodiment, a slot time may be 9 microseconds, SIFS may be 16 microseconds, and DIFS may be 64 microseconds.

On the other hand, as shown in Figure 4, the WAVE lower MAC defined in IEEE802.11p can support Enhanced Distributed Channel Access (EDCA), which can differentiate transmission priority by setting a contention window and AIFS for each access category differently. Means you can. Furthermore, by not performing Active Search, Authentication, and Association, which were essential in the existing IEEE 802.11a/g standard, but not in IEEE 802.11p, quick communication setup between vehicles or between vehicles and infrastructure can be possible. have.

Meanwhile, as shown in FIG. 5, in the WAVE upper MAC defined in IEEE1609.4, a channel switching mechanism for 7 multi-channel operations is defined, and the time can be synchronized based on a common time reference.

Further, the synchronization interval (SI, 100 ms) is divided into two sections of CCH and SSH, and this is repeatedly performed. Specifically, a beacon and a safety message may be transmitted (Common CH) in a Control channel (CCH) interval (50 ms), and a user message may be transmitted in a Service Channel (SCH) interval (50 ms). In this case, the beacon frame may be broadcast once for every CCH, and the CCH and SCH intervals may be periodically switched.

That is, the communication method according to the previously defined IEEE802.11p and IEEE1609.4 transmits a beacon and safety message notifying its status in the vehicle environment through broadcast in the CCH section, and It uses a backoff algorithm to reduce collisions accordingly.

At this time, in the case of broadcast, since there is no way to check whether or not there is a collision, there is no choice but to use a fixed minimum contention window (CW _min ) regardless of whether the transmission is successful, and a random value in [0,CW _min ] After selecting, wait for the relevant time (Backoff delay) and then perform broadcast communication.

Therefore, in a broadcast communication environment, when multiple nodes transmit and use data at the same time, frequent transmission failures may occur due to collisions. Therefore, it is necessary to reduce the number of collisions by adjusting the size of the contention window to increase the reception success rate. Will exist.

Accordingly, when implementing a method capable of adjusting the size of a contention window in a CCH using a broadcast communication method, a better effect than the conventional method may occur.

To this end, the present invention proposes a method of receiving an ACK message by combining a unicast communication method in CCH, and adjusting the size of a contention window through a Q-learning algorithm.

6 is an exemplary diagram for explaining the structure of a system according to an embodiment of the present invention.

As shown in FIG. 6, STA A (or the first STA) may include a Q-learning module for adjusting the size of the contention window. When performing broadcast communication, the first STA may receive an ACK message from neighboring STA B (or the second STA). The Q-learning module of the first STA may change the size of the contention window based on whether or not the ACK message is successfully received.

7 is an exemplary diagram illustrating a CCH frame structure according to an embodiment of the present invention.

As described above, the CCH performs broadcast communication, and does not receive an ACK message in case of broadcast communication. Accordingly, in an embodiment of the present invention, the frame structure of the WAVE standard can be changed to include a message exchange section (a section for receiving ACK).

Specifically, as shown in Fig.7, at the end of the CCH interval, the frame structure is changed to include an interval (CW Suitability Verification Interval, CSV interval) for performing unicast communication in the CCH during the T _CSV time. I can.

At this time, each STA (eg, a first STA) may transmit and receive data (Current CW check packet) by selecting one of neighboring STAs (eg, a second STA). STA may mean a device for performing communication.

Meanwhile, according to an embodiment of the present invention, the initial contention window may be set to a minimum contention window (CW _min ), and when the first STA does not receive an ACK, the first STA requests retransmission of the ACK. , It is determined that the transmission has failed. When the first STA receives the ACK, the first STA determines that the transmission is successful in the current contention window.

Meanwhile, the CSV interval may be determined based on Equation 1 below.

[Equation 1]

At this time, T _DIFS is the time of the DCF interframe space, CW _max is the maximum CW value, and the slottime is the time of a slot defined by IEEE 802.11p (for example, the slottime of the OFDM PHY defined by the IEEE 802.11p standard may be 9 μs. ), E[T _Data ] is the expected value of the time it takes to transmit the data (ie Current CW check packet) used in the CSV section, T _SIFS is the short interframe space time, and E[T _ACK ] is the ACK received. It can be the expected value of the time it takes.

According to an embodiment, it may be E[T _Data ] = (Size of data) / (Data rate), and E[T _ACK ] = (Size of ACK) / (Data rate).

8 is a flowchart illustrating a communication method according to an embodiment of the present invention.

In step S110, it may be determined whether the first STA receiving the beacon and the safety message through broadcast in the CCH section has received the ACK message from the second STA.

As described above, the first STA may determine whether to receive the ACK message transmitted from the second STA by adding the SIFS interval and the ACK reception structure to the frame structure of the CCH.

In step S120, the first STA may change the contention window (CW) based on whether the ACK message is received.

9 is a flowchart illustrating a Q-Learning method according to an embodiment of the present invention.

In step S210, the contention window may be updated based on the Q-Learning model.

In this case, the Q-Learning model used in the present invention may be designed as shown in Equation 2 below.

[Equation 2]

는 학습 속도,

는 디스카운트 팩터,

는 t에서의 리워드 값을 의미할 수 있다.At this time, s _t is the state at t (the size of CW), a _t is the possibility of action at t,

Is the learning rate,

Is the discount factor,

May mean a reward value at t.

Specifically, s _t is a state at t (the size of CW), and may be one of 3, 7, 15, 31, 63, 127, and 255. In this case, the minimum contention window CW _min may be 3.

In addition, a _t is the possibility of an action at t, and the action at this time may be any one of an action for reducing the contention window, an action for maintaining the contention window, and an action for increasing the contention window.

는 학습 속도로, 1에 가까울수록 새로운 정보만을 기초로 학습할 수 있다.Meanwhile,

Is the learning speed, and the closer to 1, the more new information can be learned.

는 디스카운트 팩터로, 1에 가까울수록 높은 보상을 획득하기 위해 학습하고, 0에 가까울수록 현재 보상에 가깝도록 학습할 수 있다.Meanwhile,

Is a discount factor, and the closer it is to 1, the higher the reward can be obtained, and the closer to 0, the closer to the current reward.

는 t에서의 리워드 값으로, ACK 메시지를 수신한 경우 1의 값으로, ACK 메시지를 수신하지 못한 경우 -1의 값으로 설정될 수 있다.Meanwhile,

May be set as a reward value at t, a value of 1 when an ACK message is received, and a value of -1 when an ACK message is not received.

In step S220, when the contention window is the minimum contention window, the minimum contention window may set the Q table as the initial Q table.

Specifically, the first STA may set the contention window in the initial broadcast communication process to the minimum contention window (CW=3).

At this time, the Q table includes first to seventh rows for the size of the contention window and first to third columns for the BE parameter for determining an action for changing the size of the contention window, and The first row has a contention window size of 3, the second row of the Q table has a contention window size of 7, and the third row of the Q table has a contention window size of 15, and the fourth row of the Q table competes. The size of the window is 31, the fifth row of the Q table has a size of a contention window 63, the sixth row of the Q table has a size of a contention window 127, and the seventh row of the Q table has a size of a contention window 255 In the first column of the Q table, the BE parameter is BE-1, the second column of the Q table is the BE parameter BE, and the third column of the Q table is the BE parameter BE+1, and the initial Q In the table, the value corresponding to the first row, the first column is -100, the value corresponding to the first row, the second column is 1/3, the value corresponding to the first row, the third column is 1/7, and the second row is the first column. The corresponding value is 1/3, the value corresponding to the second row, the second column is 1/7, the value corresponding to the second row, the third column is 1/15, the value corresponding to the first column of the third row is 1/7, The value corresponding to the third row, the second column is 1/15, the value corresponding to the third row, the third column is 1/31, the value corresponding to the fourth row, the first column is 1/15, and the fourth row corresponds to the second column. The value is 1/31, the value corresponding to row 4, column 3 is 1/63, the value corresponding to row 5, column 1 is 1/31, the value corresponding to row 5, column 2 is 1/63, 5th The value corresponding to the third column of the row is 1/127, the value corresponding to the first column of the sixth row is 1/63, the value corresponding to the second column of the sixth row is 1/127, and the value corresponding to the third column of the sixth row is 1/127. 1/255, a value corresponding to the first column of the 7th row may be 1/127, a value corresponding to the second column of the 7th row may be 1/255, and a value corresponding to the third column of the 7th row may be -100. .

Specifically, the initial Q table (= initial Q table) indicating the Q value for the contention window in the above-described initial communication process may be as shown in Table 1 below.

BE-1 BE BE+1 3 -100 1/3 1/7 7 1/3 1/7 1/15 15 1/7 1/15 1/31 31 1/15 1/31 1/63 63 1/31 1/63 1/127 127 1/63 1/127 1/255 255 1/127 1/255 -100

At this time, a backoff exponent (BE) is a parameter for determining an action for changing the size of the CW, and may be determined as shown in Equation 3 below.

[Equation 3]

That is, when the action determined through the Q table is BE-1, the first STA reduces the size of the contention window, and in the case of BE, the first STA maintains the size of the contention window, and when BE+1, the first STA 1 STA can increase the size of the contention window.

Meanwhile, the first STA may update the Q table shown in Table 1 as the contention window through the Q-Learning model is changed. A specific example of the update will be described later.

10 is a flowchart illustrating an action selection method according to an embodiment of the present invention.

In step S310, the column corresponding to the largest value among the values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table updated with a probability of 0.9 at each update may be selected as an action. have.

In step S320, a column corresponding to a random value among values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table updated with a probability of 0.1 may be selected as an action.

That is, the first STA may change the contention window based on the Q table updated through the Q-Learning model with a probability of 0.9, but may change the contention window separately from the updated Q table with a probability of 0.1.

Through the above-described method, by giving randomness to the contention window change, there is an effect of flexible contention window change.

11 is a flowchart illustrating an action selection method according to another embodiment of the present invention.

Specifically, there may be a case where the values of the rows corresponding to the contention windows in the updated Q table are the same. In this case, by designing the Q-learning model to have a direction of reducing the size of the contention window, it is possible to prevent the size of the contention window from being expanded.

Specifically, in step S410, when a value of column BE-1, a value of column BE, and a value of column BE+1 corresponding to the current state of the Q table are all the same, a column corresponding to the value of column BE-1 may be selected as an action. .

Meanwhile, a case in which two values of the row values corresponding to the contention window in the updated Q table are the same may occur. In this case, if the other value is greater than the same two values, you can select the column corresponding to the other value as an action, but if the other value is less than the same two values, which action will be selected It can be a problem.

Specifically, in step S420, the values in column BE-1 and column BE among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table are the same, and the values in column BE+1 are the same. If less than the same two values, the column corresponding to the value of column BE-1 can be selected as an action.

In step S430, among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE-1 and column BE+1 are the same, and the values in column BE are the same. If it is smaller than the value, the column corresponding to the value of column BE-1 can be selected as an action.

In step S440, among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE and column BE+1 are the same, and the values in column BE-1 are the same. If it is smaller than the value, the column corresponding to the value of the BE column can be selected as an action.

That is, in the case of steps S420 to S440, among the same two actions, an action may be selected such that the size of the contention window is reduced.

Meanwhile, according to various embodiments of the present invention, in step S450, the first value of the column BE-1, the value of column BE, and the value of column BE+1 corresponding to the current state of the Q table is the value of column BE-1. , When the second largest value is the value in column BE+1, and the difference between the value in column BE-1 and the value in column BE+1 is within a preset range, the column corresponding to the value in column BE-1 can be selected as an action.

That is, even if the value corresponding to the action is different, if the difference between the action having the largest value and the action having the second largest value (or the third largest value) is insignificant, the first STA decreases the size of the contention window. You can set the action in the direction.

가 0.6

이 0.9인 경우, 에 대한 경쟁 윈도우 변경 방법을 가정한다.Hereinafter, the operation of the Q-Learning model according to an embodiment of the present invention will be described in detail. For convenience of explanation, the initial state s _t0 is 3, the initial action a _t0 is BE,

A 0.6

When is 0.9, a method of changing the contention window for is assumed.

(Step 1)

The Q table in the initial state is the same as in Table 1 above, and in this case, the value corresponding to the current state is 1/3 (the value corresponding to the bold text in Table 2 below).

BE-1 BE BE+1 3 -100 1/3 1/7 7 1/3 1/7 1/15 15 1/7 1/15 1/31 31 1/15 1/31 1/63 63 1/31 1/63 1/127 127 1/63 1/127 1/255 255 1/127 1/255 -100

In this case, it may be assumed that the first STA fails to receive the ACK message from the second STA, and a probability of 0.9 is selected. That is, since s _t0 is 3 and a _t0 is BE (maintaining the current state), s _t1 may have a contention window size of 3 (maintaining the current state).

In this case, the Q-Learning model can operate according to Equation 4 below, and updates the Q table.

[Equation 4]

이다.At this time, since the first STA has failed to receive the ACK message,

to be.

는 경쟁 윈도우가 3이고 액션이 BE열인 경우이므로 1/3의 값을 가진다.In addition,

Is a value of 1/3 because the contention window is 3 and the action is the BE column.

는 s_t0 상태에서의 최대의 a값이므로, 경쟁 윈도우가 3이고 액션이 BE열인 경우에 대한 1/3의 값을 가진다.In addition,

Is the maximum a value in the state of s _t0 , so it has a value of 1/3 for the case where the contention window is 3 and the action is the BE column.

= [0.4(1/3)+0.6[-1+(0.9)*(1/3)] = -0.28667의 값을 가지게 된다. 이를 바탕으로 업데이트된 Q테이블은 하기 표 3과 같다.In conclusion,

= [0.4(1/3)+0.6[-1+(0.9)*(1/3)] = -0.28667. The updated Q table based on this is shown in Table 3 below.

BE-1 BE BE+1 3 -100 -0.28667 1/7 7 1/3 1/7 1/15 15 1/7 1/15 1/31 31 1/15 1/31 1/63 63 1/31 1/63 1/127 127 1/63 1/127 1/255 255 1/127 1/255 -100

In conclusion, since the largest value for a row with a contention window of 3 is 1/7 (column BE+1), a _t1 can be selected as an action corresponding to BE+1 (that is, an action to increase the CW size). .(Step 2)

The Q table in the updated state is shown in Table 4 below, and in this case, the value corresponding to the current state is 1/7 (value corresponding to the bold text in Table 4 below).

BE-1 BE BE+1 3 -100 -0.28667 1/7 7 1/3 1/7 1/15 15 1/7 1/15 1/31 31 1/15 1/31 1/63 63 1/31 1/63 1/127 127 1/63 1/127 1/255 255 1/127 1/255 -100

In this case, it may be assumed that the first STA fails to receive the ACK message from the second STA and a probability of 0.9 is selected. That is, since s _t1 is 3 and a _t1 is BE+1, s _t2 is The size of the competing window is changed to 7.

In this case, the Q-Learning model can operate according to Equation 5 below, and the Q table of Table 4 is updated.

[Equation 5]

이다.At this time, since the first STA has failed to receive the ACK message,

to be.

는 경쟁 윈도우가 3이고 액션이 BE+1열인 경우이므로 1/7의 값을 가진다.In addition,

Has a value of 1/7 because the contention window is 3 and the action is in BE+1 column.

는 s_t1 상태에서의 최대의 a값이므로, 경쟁 윈도우가 3이고 액션이 BE+1열인 경우에 대한 1/7의 값을 가진다.In addition,

Since is the maximum value of a in the state of s _t1 , it has a value of 1/7 for the case where the contention window is 3 and the action is BE+1 column.

= [0.4(1/7)+0.6[-1+(0.9)*(1/7)] = -0.46571의 값을 가지게 된다. 이를 바탕으로 업데이트된 Q테이블은 하기 표 5와 같다.In conclusion,

= [0.4(1/7)+0.6[-1+(0.9)*(1/7)] = -0.46571. The updated Q table based on this is shown in Table 5 below.

BE-1 BE BE+1 3 -100 -0.28667 -0.46571 7 1/3 1/7 1/15 15 1/7 1/15 1/31 31 1/15 1/31 1/63 63 1/31 1/63 1/127 127 1/63 1/127 1/255 255 1/127 1/255 -100

In conclusion, since the largest value for a row with a contention window of 3 is -0.28667 (column BE), a _t2 may be selected as an action corresponding to BE (ie, an action that maintains the CW size). Through the above process, the Q table is updated, and the size of the contention window may be changed according to the updated Q table.

On the other hand, in the above (step 2), when a probability of 0.1 is selected, a _t1 is of course that an arbitrary action may be selected. However, according to an embodiment of the present invention, when the contention window is 3 (minimum contention window), an arbitrary action is determined as one of BE or BE+1, and the contention window is 255 (maximum contention window). In this case, of course, the arbitrary action may be determined as one of BE-1 or BE.

12 is a block diagram of an apparatus according to an embodiment of the present invention.

The processor 102 may include one or more cores (not shown) and a graphic processing unit (not shown) and/or a connection path (eg, a bus) for transmitting and receiving signals with other components. .

The processor 102 according to an embodiment performs the method described with reference to FIGS. 1 to 11 by executing one or more instructions stored in the memory 104.

For example, the processor 102 acquires new training data by executing one or more instructions stored in the memory, performs a test on the acquired new training data using the learned model, and labels the test result. Extracting first learning data in which the extracted information is obtained with an accuracy of a predetermined first reference value or more, deleting the extracted first learning data from the new learning data, and removing the new learning data from which the extracted learning data is deleted The learned model may be retrained by using.

Meanwhile, the processor 102 temporarily and/or permanently stores a signal (or data) processed inside the processor 102, a RAM (Random Access Memory, not shown) and a ROM (Read-Only Memory). , Not shown) may further include. In addition, the processor 102 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processing unit, RAM, and ROM.

The memory 104 may store programs (one or more instructions) for processing and controlling the processor 102. Programs stored in the memory 104 may be divided into a plurality of modules according to functions.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. Software modules include Random Access Memory (RAM), Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Components of the present invention may be implemented as a program (or application) and stored in a medium in order to be combined with a computer as hardware to be executed. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments include various algorithms implemented with a combination of data structures, processes, routines or other programming elements, including C, C++ , Java, assembler, or the like may be implemented in a programming or scripting language. Functional aspects can be implemented with an algorithm running on one or more processors.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features. You can understand. Therefore, the embodiments described above are illustrative in all respects, and should be understood as non-limiting.

100: device
102: processor
104: memory

Claims (10)

In the communication method using WAVE PHY, WAVE lower MAC and WAVE upper MAC defined in IEEE802.11p and IEEE1609.4,
Determining whether the first STA receiving the beacon and the safety message through broadcast in the CCH section has received an ACK message from the second STA; And
Changing, by the first STA, a contention window (CW) based on whether or not the ACK message is received; Including,
The CCH interval includes a CSV interval (CW Suitability Verification Interval) for performing unicast communication in the CCH, and the CSV interval is a time interval of Tcsv,
The CSV interval is determined based on the following equation.

At this time, T _DIFS is the time of DCF Interframe Space, CW _max is the maximum CW value, Slottime is the time of the slot defined by IEEE 802.11p, and E[T _Data ] is the expectation of the time it takes to transmit the data used in the CSV section. Value, T _SIFS is the short interframe space time, E[T _ACK ] is the expected value of the time it takes to receive ACK delete The method of claim 1,
The first STA and the second STA transmit and receive a packet for checking a current contention window. The method of claim 1,
The step of changing the contention window,
If the contention window is the minimum contention window, setting the Q table as an initial Q table by the minimum contention window; And
Updating the contention window based on the following Q-Learning model; Including more,

At this time, s _t is the state at t (the size of CW), a _t is the possibility of action at t,

Is the learning rate,

Is the discount factor,

Is the reward value at t,
The Q table includes first to seventh rows for the size of the contention window and first to third columns for the BE parameter for determining an action for changing the size of the contention window. The first row is the size of the contention window 3, the second row of the Q table is the size of the contention window, the third row of the Q table is the size of the contention window is 15, and the fourth row of the Q table is a contention window. The size of is 31, the fifth row of the Q table has a size of a contention window 63, the sixth row of the Q table has a size of a contention window 127, and the seventh row of the Q table has a size of a contention window 255. ,
The first column of the Q table is a BE parameter BE-1, the second column of the Q table is a BE parameter BE, and the third column of the Q table is a BE parameter BE+1,
The initial Q table,
The value corresponding to the first row, the first column is -100, the value corresponding to the first row, the second column is 1/3, the value corresponding to the first row, the third column is 1/7, and the value corresponding to the first column of the second row This 1/3, the value corresponding to the second row, the second column is 1/7, the value corresponding to the second row, the third column is 1/15, the value corresponding to the third row, the first column is 1/7, and the third row The value corresponding to the second column is 1/15, the value corresponding to the third row and the third column is 1/31, the value corresponding to the fourth row, the first column is 1/15, and the value corresponding to the second column of the fourth row is 1 /31, the value corresponding to row 4, column 3 is 1/63, the value corresponding to row 5, column 1 is 1/31, the value corresponding to row 5, column 2 is 1/63, row 5 is 3 The value corresponding to the column is 1/127, the value corresponding to the first column of the 6th row is 1/63, the value corresponding to the second column of the 6th row is 1/127, the value corresponding to the third column of the 6th row is 1/255 And a value corresponding to the first column of the seventh row is 1/127, a value corresponding to the second column of the seventh row is 1/255, and a value corresponding to the third column of the seventh row is -100. The method of claim 4,
The updating step,
Selecting a column corresponding to the largest value among the values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table updated with a probability of 0.9 upon each update as an action; And
And selecting, as an action, a column corresponding to an arbitrary value among values of column BE-1, column BE, and values of column BE+1 corresponding to the current state of the Q table updated with a probability of 0.1. The method of claim 5,
The step of selecting the largest value as an action,
Selecting a column corresponding to the value of column BE-1 as an action when the values of column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table are the same;
Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE-1 and column BE are the same, and the values in column BE+1 are greater than the same two values. If small, selecting a column corresponding to the value of column BE-1 as an action;
Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE-1 and column BE+1 are the same, and the values in column BE are greater than the same two values. If small, selecting a column corresponding to the value of column BE-1 as an action; And
Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the values in column BE and column BE+1 are the same, and the values in column BE-1 are greater than the same two values. If small, selecting a column corresponding to the value of the BE column as an action. The method of claim 5,
The step of selecting the largest value as an action,
Among the values in column BE-1, column BE, and column BE+1 corresponding to the current state of the Q table, the first largest value is the value in column BE-1, the second largest value is the value in column BE+1, and When the difference between the value of the BE-1 column and the value of the BE+1 column is within a preset range, selecting a column corresponding to the value of the BE-1 column as an action. delete A memory for storing one or more instructions; And
And a processor that executes the one or more instructions stored in the memory,
The processor executes the one or more instructions,
An apparatus for performing the method of claim 1. A computer program combined with a computer as hardware and stored in a recording medium readable by a computer to perform the method of claim 1.

Images