KR102176020B1 - A Q-Learning-Based Clustering Method Considering the Scalability and Stability in Cognitive Radio Ad Hoc Networks - Google Patents

Abstract

The present invention proposes a Q-learning-based clustering method considering scalability and stability in a cognitive radio ad-hoc network and an apparatus thereof. According to the present invention, the proposed Q-learning-based clustering method considering scalability and stability in a cognitive radio ad-hoc network comprises the steps of: sensing a frequency spectrum of a region through a cognitive radio terminal in the cognitive radio network, and dynamically evaluating the quality of a band and channel using Q-learning; constructing a node channel cluster information message to use the result of the evaluation of the quality of the band and channel and information on a neighboring node and a cluster; and selecting a cluster head, a cluster member node, a gateway node, and an optimal channel within the cluster.

Description

A Q-Learning-Based Clustering Method Considering the Scalability and Stability in Cognitive Radio Ad Hoc Networks}

The present invention relates to a method and system for dynamically configuring a cluster through information exchange between secondary user nodes in a cognitive radio network using multiple channels. More specifically, secondary user nodes The present invention relates to a method and apparatus for dynamically selecting a channel and band quality evaluation, a cluster head, a cluster member node, a gateway node, a common active data channel, and a gateway node based on a sensing result and information obtained through exchange between nodes.

A cognitive wireless ad-hoc network that selectively uses a number of channels is a technology that searches for channels that are not already allocated and configures a wireless network without a base station based on a wired backbone network.

The cognitive radio ad-hoc network utilizes an idle band of primary users using an authenticated frequency band, so that cognitive radio users can communicate within a limit that does not interfere with communication of primary users.

The cognitive radio ad-hoc network is required to detect an idle band, consider different parameters for each channel due to main users, and efficiently select a frequency resource that can be used to individually satisfy the requirements of each member node. .

The cognitive wireless ad-hoc network must configure a cluster network to perform data communication without a wired backbone network in an environment in which the main user exists, and in order to perform data communication stably in response to changes in the operation characteristics and surrounding environment of the main user. A technique for dynamically configuring the topology is required.

The technical problem to be achieved by the present invention is that cognitive wireless users determine the quality of bands and channels through spectrum sensing in a cognitive wireless ad-hoc network environment, and exchange information between nodes to facilitate seamless communication without interference between member nodes in a cluster and between clusters. It is to provide a method and apparatus for dynamically determining a network topology and a used band/channel for the purpose. In addition, it is possible to extend the network life in relation to the factors considered for dynamically determining the topology and the use band/channel above, improve connectivity not only to member nodes, but also to other cluster networks, and use the selected data channel. Thus, it is intended to provide a method and apparatus capable of providing a stable and reliable service and avoiding interference between neighboring ad hoc clusters.

In one aspect, the Q-learning-based clustering method in consideration of scalability and stability in the cognitive radio ad-hoc network proposed in the present invention senses the frequency spectrum of a corresponding region through a cognitive radio terminal in the cognitive radio network, and detects the band and channel quality. Dynamically evaluating using Q-learning, constructing a node channel cluster information message to use the evaluation results for the quality of bands and channels, and neighboring nodes and cluster information, and cluster head, cluster member node, and gateway And selecting an optimal channel within the node and cluster.

The step of sensing the frequency spectrum of the region through the cognitive radio terminal in the cognitive radio network and dynamically evaluating the quality of the band and channel using Q-learning includes calculating the normalized average idle time and the estimated idle time probability, Computing a Q-learning function for calculating the Q-value using the normalized mean idle time and the estimated idle-time probability, dynamically calculating the Q-value for the channel selection behavior of the Q table using the Q-learning function, and And dynamically calculating a Q value for the band selection operation using the Q value for the channel selection operation.

The step of constructing the node channel cluster information message to use the evaluation results of the band and channel quality and the neighboring node and cluster information is the step of constructing a node characteristic field using the node identifier and residual energy, and the available channel list and Constructing the channel quality field using the Q values of the available channels, constructing the neighboring node connectivity field using the neighboring node list and the neighboring node available channel list, and the common activation of the set of connectable neighboring clusters and the connectable neighboring clusters. And configuring a peripheral cluster connectability field using the data channels.

Selecting the cluster head, cluster member node, gateway node, and the optimal channel in the cluster is the step of calculating a channel fitness function for channels accessible by each node to select the cluster head, residual energy of each node, and channel The steps of calculating the cluster head fitness function using the fitness, the number of connectable neighboring clusters, and the number of neighboring nodes, and selecting the cluster head by transmitting and receiving a cluster head request message between nodes using the cluster head fitness function result. Include.

The step of selecting the cluster head, cluster member node, gateway node, and the optimal channel in the cluster is the step of broadcasting the cluster head selection result message by the cluster head selected to select the cluster member node, and receiving the cluster head selection result message. And comparing the received node with its available channel and transmitting a subscription request message, and selecting a cluster head when the node receiving the cluster head selection result message receives a duplicate message.

The step of selecting the cluster head, cluster member node, gateway node, and the optimal channel within the cluster is the step of selecting a gateway node by the cluster head using the received node channel cluster information, and the member nodes that can be connected to any cluster are duplicated. And if present, selecting a gateway node.

The step of selecting the cluster head, the cluster member node, the gateway node, and the optimal channel in the cluster includes the steps of selecting a gateway node by the cluster head using the received node channel cluster information to select the optimal channel in the cluster. And selecting a gateway node when there are duplicate member nodes connectable to the cluster.

In another cognitive wireless ad-hoc network, the Q-learning-based clustering device, which considers scalability and stability, senses the frequency spectrum of the region through the cognitive wireless terminal in the cognitive wireless network, and dynamically determines the quality of the band and channel using Q-learning. Band and channel quality evaluation unit to be evaluated, node channel cluster information message generation unit and cluster head, cluster member node that constitutes node channel cluster information message to use the evaluation result of band and channel quality and neighbor node and cluster information , A gateway node and a selector for selecting an optimal channel in the cluster.

According to embodiments of the present invention, the quality of a band/channel was dynamically measured in a temporally changing environment using Q-learning, and the Q value obtained through this is spectrum sensing, cluster head selection, and common active/backup data channel selection. And determining a band group for selecting a gateway node. In addition, according to embodiments of the present invention, the lifespan of the cluster, the node degree, and the multipurpose function taking into account the residual energy of the node, the connectivity of the member nodes in the cluster, the connectivity between the clusters, the list of available channels and the channel quality are used. Network connectivity, the number of common channels, and quality of common data channels can be improved. In addition, according to embodiments of the present invention, it is possible to derive an optimal set of gateway nodes that can minimize redundancy and maximize connectivity between clusters. Further, according to embodiments of the present invention, in selecting a gateway node, a reliable inter-cluster communication link may be provided by using the Q value of a channel available to the corresponding node.

1 is a schematic diagram of a cognitive wireless ad-hoc network in an environment in which a plurality of main users exist according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating an overall structure of clustering a cognitive wireless ad-hoc network according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a Q-learning-based clustering method in consideration of scalability and stability in a cognitive wireless ad-hoc network according to an embodiment of the present invention.
4 is a diagram illustrating a channel operation process used in a clustering process of a cognitive wireless ad-hoc network according to an embodiment of the present invention.
5 is a diagram illustrating a band group structure for using a frequency channel according to an embodiment of the present invention.
6 is a diagram illustrating a Q table used for Q-learning for evaluating bands and channels in clustering formation according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating a spectrum sensing module periodically performing broadband sensing on channels within a corresponding band group after a Q-learning module selects a band group according to an embodiment of the present invention.
FIG. 8 is a diagram showing the format of a node channel cluster information message used by each node to exchange information with each other to find neighboring nodes in an initial cluster configuration according to an embodiment of the present invention.
9 is a diagram illustrating a process of selecting a cluster head in a situation where there is no neighboring cluster head according to an embodiment of the present invention.
10 is a diagram illustrating a configuration of a Q-learning-based clustering device in consideration of scalability and stability in a cognitive wireless ad-hoc network according to an embodiment of the present invention.
11 is a diagram showing cluster formation in a situation in which multiple clusters are simultaneously created according to an embodiment of the present invention.

The present invention provides a method for dynamically selecting a cluster head, a cluster member, a gateway node, and a common active data channel by member nodes using information on a channel, a neighbor node, and a neighboring cluster in a cognitive wireless ad-hoc network using multiple channels. do.

In the present invention, terms mainly used hereinafter and their meanings are defined as follows.

The cluster head (CH) is defined as a terminal that configures a set of available channels based on the sensing result report from member nodes in the cluster and selects the member node, gateway node, and common active data channel in the cluster. .

A cluster is defined as a set of manageable nodes within one hop distance of the cluster head (CH), which is its communication available radius.

A member node (MN) is defined as a terminal that transmits and receives data within a cluster and performs spectrum sensing to report the presence of a main user for each channel and the quality status of each channel between mutual nodes within one hop distance.

Embodiments of the present invention select a cluster head based on information obtained from nodes within a one-hop distance after cognitive wireless sub-user nodes observe the surrounding environment, and the cluster head exchanges information from the nodes to create a cluster member node, It relates to a method for dynamically selecting a gateway node, a common active data channel. More specifically, the embodiments include a Q-learning operation method for calculating channel and band quality by sensing a channel by each sub-user nodes, a method of configuring a Q table for dynamic channel and band quality estimation, and a method for exchanging information between nodes. Message composition method, channel suitability setting method for channel quality calculation based on Q value obtained through Q-learning, residual energy of node, channel suitability, cluster head suitability setting method based on connectivity information for neighboring clusters and nodes, cluster The present invention relates to a method for selecting a cluster head using head fitness, a method for selecting a common active data channel and a backup channel for a channel used in a cluster, and a method for selecting a member node and a gateway node in a cluster. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a cognitive wireless ad-hoc network in an environment in which a plurality of main users exist according to an embodiment of the present invention.

)과 S 개의 부 사용자인 멤버 노드들 (

)이 존재하는 상황에서 고려된다. A cognitive wireless ad-hoc network using multiple channels according to an embodiment of the present invention includes P primary users (

) And S sub-users (

) Is considered.

Member nodes use a single transceiver and use spectrum sensing to find and use an empty channel that occurs when a nearby main user temporarily does not use the channel, so that the use of the main user channel by the member nodes does not cause any interference to the main user. do. In the embodiments of the present invention, member nodes use a predetermined common control channel (CCC) to exchange control messages required for the initial cluster configuration of the cognitive wireless ad-hoc network or re-adjustment of the cluster configuration.

In the cognitive wireless ad-hoc network using multiple channels according to an embodiment of the present invention, a plurality of member nodes 130 communicate with each other in an environment in which main users 110 may exist, as shown in FIG. 1. Accordingly, the gateway node 140 and the cluster head 150 are selected, and an ad-hoc cluster network is constructed based on these.

Referring to FIG. 1, primary users randomly exist in a geographic network area, and each primary user uses one channel in a corresponding area. The main users operate the channel alternately between operation (ON, busy) mode and idle mode (OFF, idle). The time statistics of the operating mode and the idle mode appearing in the main user channels have an independent identically distributed (i.i.d.) characteristic. The member nodes within the transmission radius of the main user sense the main users and do not use the corresponding channel when the main user operates in the operation mode.

Referring to FIG. 1, member nodes within the transmission radius 120 of the main users 110 and pu ₃ using channel 3 ( ch ₃ ) cannot use the corresponding channel. The transmission radius of the main user is determined by the minimum required main user power sensed by the member nodes and varies depending on the main user transmission power, the sensing requirements for protection of the main user, and wireless channel statistics, but in the present invention, the transmission of the main user The radius is calculated using a predefined transmission radius.

Referring to FIG. 1, a cluster head according to an embodiment of the present invention is selected by Q-learning-based multi-purpose functions in a distributed manner, and a pre-selected cluster head is a common active data channel (CADC). And the Common Backup Data Channel (CBDC). Gateway nodes connect two or more cluster ad-hoc networks and perform inter-cluster data relay through inter-cluster communication. Member nodes and gateway nodes of a cluster are composed of peripheral nodes capable of 1-hop communication with the cluster head.

FIG. 2 is a diagram illustrating an overall structure of clustering a cognitive wireless ad-hoc network according to an embodiment of the present invention.

With reference to FIG. 2, the broadband (ie, band group) spectrum sensing module 210 periodically observes the broadband spectrum and calculates a channel quality level 250 for all channels. The Q value of the channel and band group is updated in the Q learning module 220 based on the past channel quality level and the expected compensation value. The neighboring node detection module 230 controls a node channel cluster information (NCCI) message 260 including the state of the neighboring nodes, the Q value for each channel, and connectivity information for other nodes and clusters. Acquire through a channel or broadcast its own NCCI message. Based on the information of itself and neighboring nodes, each node determines a node eligible to become a cluster head in the cluster head candidate evaluation module 240 and transmits a cluster head request (CH_REQ) message to the node.

A node of a cognitive wireless ad-hoc network according to an embodiment of the present invention can become a cluster head upon receiving a sufficient CH_REQ message 270 and determines an optimal CADC and CBDC. CADC is used for data transfer within the cluster, and CBDC is a data communication channel that can be used as an alternative to the main users when they appear in CADC. The cluster head broadcasts a Cluster Head Announcement (CH_ANM) message 280 including a CH identification number, CADC, and CBDC. The member node receiving the CH_ANM message transmits a JOIN_REQ (Join Request) message 290 to the cluster head that broadcasts CH_ANM when there is a channel to which it can access the CADC. The cluster head selects the optimal gateway nodes for inter-cluster communication among the member nodes using the connectivity of the member nodes to other clusters and the Q value of the channel. Thereafter, the cluster head broadcasts a MN&GN_ANM (Member Node and Gateway Node Announcement) message 295 to neighboring nodes.

FIG. 3 is a flowchart illustrating a Q-learning-based clustering method in consideration of scalability and stability in a cognitive wireless ad-hoc network according to an embodiment of the present invention.

The Q-learning-based clustering method that considers scalability and stability in the proposed cognitive radio ad-hoc network senses the frequency spectrum of the region through the cognitive radio terminal in the cognitive radio network, and dynamically determines the band and channel quality using Q-learning. The step of evaluating 310, the step 320 of constructing a node channel cluster information message to use the evaluation result for the quality of the band and the channel and the neighboring node and cluster information, and the cluster head, the cluster member node, the gateway node, and Including step 330 of selecting an optimal channel in the cluster.

In step 310, the frequency spectrum of the corresponding region is sensed through the cognitive radio terminal in the cognitive radio network, and the quality of the band and channel is dynamically evaluated using Q-learning.

Step 310 is a step of calculating a normalized average idle time and an estimated idle time probability, calculating a Q-learning function for calculating a Q value using the normalized average idle time and the estimated idle time probability, and a Q-learning function. And dynamically calculating a Q value for the channel selection operation of the Q table using and dynamically calculating a Q value for the band selection operation using the Q value for the channel selection operation.

In step 320, a node channel cluster information message is constructed to use the result of the evaluation of the band and channel quality and the neighboring node and cluster information.

Step 320 is a step of configuring a node characteristic field using a node identification factor and residual energy, a step of configuring a channel quality field using the available channel list and Q values of the available channels, the neighboring node list and the neighboring node available channels And configuring a peripheral node connectivity field using the list, and configuring a peripheral cluster connectivity field using a set of connectable peripheral clusters and common active data channels of the connectable peripheral clusters.

In step 330, a cluster head, a cluster member node, a gateway node, and an optimal channel in the cluster are selected.

Step 330 according to an embodiment of the present invention is the step of calculating a channel fitness function for channels accessible by each node to select a cluster head, residual energy of each node, channel fitness, and connectable neighboring clusters. And calculating a cluster head fitness function using the number and the number of neighboring nodes, and selecting a cluster head by transmitting and receiving a cluster head request message between nodes using a result of the cluster head fitness function.

Step 330 according to another embodiment of the present invention is a step in which a cluster head selected to select a cluster member node broadcasts a cluster head selection result message, and the node receiving the cluster head selection result message has its own available channel And transmitting a subscription request message, and when the node receiving the cluster head selection result message receives a duplicate message, the cluster head is selected.

In step 330 according to another embodiment of the present invention, the cluster head selects a gateway node using the received node channel cluster information, and when a member node connectable to an arbitrary cluster exists in duplicate, the gateway And selecting a node.

In step 330 according to another embodiment of the present invention, the cluster head selects a gateway node using the received node channel cluster information in order to select an optimal channel in the cluster, and a member node connectable to an arbitrary cluster. And selecting a gateway node if there is overlapping.

4 is a diagram illustrating a channel operation process used in a clustering process of a cognitive wireless ad-hoc network according to an embodiment of the present invention.

Each node periodically senses 410 the broadband spectrum to obtain all available channels and statistical characteristics thereof. The nodes exchange messages for initial clustering 420 or cluster reconfiguration 470 over a common control channel. After the cluster is formed, data communication within the cluster or between the clusters is performed using CADC (430). The nodes periodically exchange NCCI messages using CADC to report on the current channel and the state of the neighboring nodes (440). After the cluster is formed, the cluster header periodically broadcasts a CH_ANM message to the common control channel so that new nodes are added to the cluster or nodes of other clusters can recognize the cluster head or their CADC (450). When the primary user is detected, the nodes of the cluster switch the data communication channel to CBDC and use it (460).

5 is a diagram illustrating a band group structure for using a frequency channel according to an embodiment of the present invention.

개의 대역그룹들

(510)로 분할하고 각 대역 그룹은 M 개의 채널들로 구성된다(520). When member nodes perform spectrum sensing, it is not practical to sense the entire frequency spectrum ranging from several kHz to several GHz within a given sensing time, so the entire band is divided and sensed. The cognitive wireless ad-hoc network clustering system according to an embodiment of the present invention covers the entire operating band.

Bandgroups

Divided into 510 and each band group is composed of M channels (520).

6 is a diagram illustrating a Q table used for Q-learning for evaluating a band and a channel in clustering formation according to an embodiment of the present invention.

Broadband spectrum sensing according to an embodiment of the present invention is performed by selecting one of the band groups using a Q-learning algorithm. Referring to FIG. 6, the states 610 of the Q table represent each member node, and thus, each node corresponds to a single state. With reference to FIG. 6, operations 620 of the Q table represent band groups for spectrum sensing.

의 대역그룹

을 평가하기 위한 Q값은 [수학식 1]에 의해 산정될 수 있다.Member nodes in Q table with reference to FIG. 6

Band group

The Q value for evaluating can be calculated by [Equation 1].

[Equation 1]

는 멤버노드

의 채널

에 대한 Q값이고 따라서 [수학식 1]은 멤버노드

의 대역 그룹

에 속하는 채널들의 Q값의 평균을 나타낸다.6 and [Equation 1]

Is a member node

Channel of

Is the Q value for [Equation 1]

Band group

Represents the average of the Q values of channels belonging to.

의 Q러닝 모듈은 스펙트럼 센싱을 위해 대역 그룹 중 가장 높은 Q값을 갖는 대역그룹

를 선택하며 [수학식 2]에 의해 산정될 수 있다.Member node

The Q-learning module of is a band group with the highest Q value among band groups for spectrum sensing.

And can be calculated by [Equation 2].

[Equation 2]

FIG. 7 is a diagram illustrating a spectrum sensing module periodically performing broadband sensing on channels within a corresponding band group after a Q-learning module selects a band group according to an embodiment of the present invention.

을 선택한 뒤 스펙트럼 센싱 모듈이 해당 대역 그룹 내 채널들에 대해 주기적으로

번의 광대역 센싱을 수행하는 것을 도시한 도면이다. 도 7를 참조하여

번의 센싱 이후 측정된 채널 품질 수준에 따라 채널과 각 대역 그룹의 Q값들이 갱신된다. 이후에는 다음 대역 그룹에 대한 광대역 센싱이 수행된다. 도 7을 참조하여 센싱간격(sensing interval)은

(710)이고 따라서 대역 그룹별 동작 결정 간격은

이다. 다음 번 대역 그룹의 센싱에서 기 선택된 대역 그룹이 다시 선택될 수 있으며 이 경우 선택된 대역 그룹 (

)의 대역 그룹 센싱시간(

)은

의 정수배가 된다.7 is a Q-learning module band group according to an embodiment of the present invention

After selecting, the spectrum sensing module periodically

It is a diagram showing that the wideband sensing is performed. With reference to FIG. 7

The Q values of the channel and each band group are updated according to the measured channel quality level after sensing once. After that, wideband sensing is performed for the next band group. Referring to Figure 7, the sensing interval is

(710), so the interval for determining the operation of each band group is

to be. In the next band group sensing, the previously selected band group may be selected again. In this case, the selected band group (

) Of the band group sensing time (

)silver

Becomes an integer multiple of

6 and 7, when sensing of a corresponding band group is terminated and another band group is newly selected for broadband spectrum sensing, a Q value related to a channel may be calculated by [Equation 3].

[Equation 3]

를 선택하여 얻는

는 채널관측시간(

)에 대해 정규화된 평균 유휴 시간(

)과 추정 유휴시간확률(

)의 가중 합으로써 [수학식 4]에 의해 산정될 수 있다.Operation with reference to [Equation 3]

Get by choosing

Is the channel observation time (

) Normalized average idle time (

) And the estimated idle time probability (

It can be calculated by [Equation 4] as a weighted sum of ).

[Equation 4]

)은 [수학식 5]에 의해 산정될 수 있다.Referring to [Equation 4], the normalized average idle time (

) Can be calculated by [Equation 5].

[Equation 5]

)은 [수학식 6]에 의해 산정될 수 있다.With reference to [Equation 4], the normalized estimated idle time probability (

) Can be calculated by [Equation 6].

[Equation 6]

In calculating the Q value according to an embodiment of the present invention, the determination of the operation mode and the idle mode of a specific channel may be performed through arbitrary sensing. In the case of energy detection-based spectrum sensing, the energy of the received signal is greater than or equal to a specific detection value. If so, the channel is defined as occupied by the primary user.

에 대해 대역 그룹

이 스펙트럼 센싱을 위해 선택되었으며

번(5 번)의 센싱 이후 대역 그룹

가 선택된다. 대역그룹

의 채널

과

에 대해 다섯 번의 센싱 횟수에 대해 세 번의 유휴 시간 횟수로써 추정 유휴시간확률이 동일(730)하다. 그러나 채널

는

보다 더 긴 정규화된 평균 유휴 시간을 갖는다(720).Secondary user member node with reference to FIG. 7

About band group

Was chosen for this spectrum sensing

Band group after sensing of No. 5 (No. 5)

Is selected. Band group

Channel of

and

The estimated idle time probability is the same (730) as the number of idle times of three to five sensing times of. But the channel

Is

Have a longer normalized average idle time (720).

According to an embodiment of the present invention, channel quality is evaluated by referring to [Equation 1], [Equation 2], [Equation 3], [Equation 4], [Equation 5], and [Equation 6] The Q values calculated for this are used for the selection of the cluster head, the selection of the CDBA, and the selection of the gateway.

FIG. 8 is a diagram showing the format of a node channel cluster information message used by each node to exchange information with each other to find neighboring nodes in an initial cluster configuration according to an embodiment of the present invention.

Each node composes an NCCI message and broadcasts it to neighboring nodes one hop distance through a common control channel, and referring to FIG. 8, the NCCI message includes node characteristics, channel quality Q value, neighbor node information, and connectable cluster information. do.

)(812)을 포함한다. 채널 품질 필드(820)는 가용채널 리스트(821)와 스펙트럼 센싱에 의해 측정된 모든 가용 채널들에 대해 갱신된 Q값의 집합(822)을 포함한다. 그리고, 주변 노드 연결성 필드(830)는 주변 노드 리스트(831)와 1 홉 거리 내의 주변 노드들로부터 브로드캐스팅된 NCCI 메세지를 수신 받아 얻은 각각의 주변 노드들의 가용채널 리스트(832)를 포함한다. 마지막으로 주변 클러스터 연결 가능성 필드(840)는 연결 가증한 주변 클러스터들의 집합(841)과 해당 클러스터들에 대한 현재 CADC들(842)을 포함한다. 주변 클러스터 정보는 주변 클러스터 헤더들이 공통제어채널을 통해 주기적으로 브로드캐스팅하는 CH_ANM 메세지를 수신 받아 얻을 수 있다. Referring to FIG. 8, a node characteristic field 810 includes an identification factor 811 of a node and a current residual energy level (

) (812). The channel quality field 820 includes an available channel list 821 and a set of Q values 822 updated for all available channels measured by spectrum sensing. In addition, the neighboring node connectivity field 830 includes a neighboring node list 831 and an available channel list 832 of each neighboring node obtained by receiving an NCCI message broadcast from neighboring nodes within one hop distance. Finally, the neighbor cluster connection possibility field 840 includes a set 841 of neighboring clusters for which connection is augmented and current CADCs 842 for the clusters. The neighboring cluster information can be obtained by receiving a CH_ANM message periodically broadcast by neighboring cluster headers through a common control channel.

에 의해 브로드캐스팅되며 네 개의 가용 채널을 갖는다. 노드

는 네 개의 주변 노드들

를 갖고 두 개의 주변 클러스터들

를 갖는다. 노드

는 주변 클러스터들의 멤버노드일 수도 있거나 클러스터의 CADC를 사용하여 다른 클러스터에 참여할 수 있다. 주변 클러스터

와

의 CADC는 각각 채널 3과 4이고 주변 클러스터들과 발생할 수 있는 간섭을 피하기 위해 새롭게 구성되는 클러스터는 주변 클러스터들의 CADC들을 사용하지 않아야 한다. 따라서 노드

를 포함하여 새롭게 형성되는 클러스터는 주변 클러스터들

에 의해 사용되고 있는 채널들

를 사용하지 않는다. 클러스터가 형성된 후에 모든 멤버노드들은 CADC를 사용하여 NCCI를 주기적으로 갱신하여 브로드캐스팅한다.8 and 1, the NCCI message is a node

Broadcast by and has four available channels. Node

Is the four neighboring nodes

With two surrounding clusters

Has. Node

May be a member node of neighboring clusters, or may participate in other clusters using the cluster's CADC. Peripheral cluster

Wow

The CADCs of are channels 3 and 4, respectively, and in order to avoid possible interference with the surrounding clusters, the newly formed cluster should not use the CADCs of the surrounding clusters. So node

The newly formed cluster including the surrounding clusters

Channels being used by

Do not use. After the cluster is formed, all member nodes periodically update and broadcast NCCI using CADC.

Referring to FIG. 2, after exchanging NCCI messages during an operation to configure clustering according to an embodiment of the present invention, each node obtains necessary neighboring node information and uses it to calculate the fitness of the neighboring nodes and the cluster head including itself. do. A node's cluster head suitability value indicates how suitable the node is to become a cluster head, and is a multipurpose weighted sum of residual energy, channel quality, number of available channels, number of connectable neighboring nodes, and number of connectable clusters. It is calculated from the fitness function.

에 대해 [수학식 7]에 의해 산정될 수 있다. According to an embodiment of the present invention, the channel fitness function used to calculate the cluster head fitness for a specific node is a node

Can be calculated by [Equation 7].

[Equation 7]

의 채널 적합도 함수는 노드

에 대해 유효가용채널들(Effective Available Channels; EACs)의 집합에 속하는 채널들에 대한 Q값과 해당 채널들을 사용하여 노드

와 연결이 가능한 주변 노드들의 집합을 곱한 값이다.Node with reference to [Equation 7]

The channel fitness function of the node

Q values for channels belonging to the set of Effective Available Channels (EACs) and the corresponding channels are used for the node

It is the product of the set of neighboring nodes that can be connected to.

와 연결 가능한 주변 노드들의 집합은 [수학식 8]과 같이 산정될 수 있다.Node with reference to [Equation 7]

The set of neighboring nodes that can be connected to may be calculated as in [Equation 8].

[Equation 8]

의 유효가용채널의 집합은 노드

의 가용채널의 집합에서 주변 클러스터들의 CADC들을 제거한 것으로써 [수학식 9]와 같이 산정될 수 있다.Node with reference to [Equation 7]

The set of available channels of the node

It can be calculated as [Equation 9] by removing the CADCs of neighboring clusters from the set of available channels of.

[Equation 9]

Referring to [Equation 7], the value of the channel suitability function increases as the number of available available channels is large, the Q value of available available channels is high, and the number of connectable neighboring nodes using available available channels increases.

의 클러스터 헤드 적합도 함수의 값은 노드

의 잔여에너지(

), 채널적합도 함수(

), 노드

와 노드

의 주변노드들을 통해 연결가능한 주변 클러스터들의 수(

), 노드

의 주변노드들의 개수(

)를 사용하여 [수학식 10]과 같이 산정될 수 있다.Node

The value of the cluster head fitness function of the node

Residual energy (

), channel suitability function (

), node

And node

The number of neighboring clusters connectable through neighboring nodes of (

), node

The number of neighboring nodes of (

) Can be calculated as in [Equation 10].

[Equation 10]

,

,

와 노드

의 주변노드들을 통해 연결 가능한 주변 클러스터들의 개수는 [수학식 11]을 사용하여 산정될 수 있다.Node with reference to [Equation 10]

And node

The number of neighboring clusters connectable through neighboring nodes of can be calculated using [Equation 11].

[Equation 11]

가 [수학식 10]을 참조하여 자기 자신과 주변의 1 홉 거리의 노드들에 대한 클러스터 헤드 적합도 함수의 값들을 계산하고 난 뒤에, 노드

는 클러스터 헤드 적합도 함수값이 가장 높은 노드를 선택하여 노드

의 클러스터 헤드 후보로 고려할 수 있는데 이는 [수학식 12]와 같이 산정될 수 있다.Node

After calculating the values of the cluster head goodness-of-fit function for nodes with a one-hop distance between themselves and neighboring nodes with reference to Equation 10, the node

Selects the node with the highest cluster head fitness function

It can be considered as a cluster head candidate of, which can be calculated as in [Equation 12].

[Equation 12]

는 자기 자신을 포함하여 사전에 정의된 비율 이상으로 CH_REQ 메세지를 받았을 때 클러스터 헤드가 되며 이는 [수학식 13]을 사용하여 산정될 수 있다.After selecting a cluster head candidate node according to an embodiment of the present invention, each node transmits a CH_REQ message to the cluster head candidate node using a common control channel. Which node received the CH_REQ message

Is a cluster head when a CH_REQ message is received in excess of a predefined ratio including itself, and this can be calculated using [Equation 13].

[Equation 13]

는 자기 자신에게 요구하는 것을 포함하여 수신된 총 CH_REQ이고

는 백분위로 나타낸 임계값이다.Referring to [Equation 13]

Is the total CH_REQ received, including requests from itself

Is the threshold expressed in percentiles.

9 is a diagram illustrating a process of selecting a cluster head in a situation where there is no neighboring cluster head according to an embodiment of the present invention.

와 자신을 포함하여 주변 노드들의 클러스터 헤드 적합도 함수

를 산정한다. 노드 C의

는 가장 높은 값을 갖기 때문에 모든 노드들은 노드 C에게 CH_REQ 메세지를 송신하고, 노드 C는 노드 A, B, D에 대한 클러스터 헤드가 된다. 도 9를 참조하여

,

,

이다.Referring to FIG. 9, after a plurality of NCCI messages are exchanged, each node acquires neighboring node information. Referring to [Equation 7] and [Equation 10], each node is a channel fitness function

And the cluster head fitness function of neighboring nodes including itself

Calculate Node c

Since is the highest value, all nodes send CH_REQ messages to node C, and node C becomes the cluster head for nodes A, B, and D. With reference to FIG. 9

,

,

to be.

10 is a diagram illustrating a configuration of a Q-learning-based clustering device in consideration of scalability and stability in a cognitive wireless ad-hoc network according to an embodiment of the present invention.

The Q-learning-based clustering device 1000 considering scalability and stability in the proposed cognitive wireless ad-hoc network includes a band and channel quality evaluation unit 1010, a node channel cluster information message generation unit 1020, and a selection unit 1030. do.

The band and channel quality evaluation unit 1010 senses the frequency spectrum of a corresponding region through the cognitive radio terminal in the cognitive radio network, and dynamically evaluates the quality of the band and channel using Q-learning.

The band and channel quality evaluation unit 1010 calculates a normalized average idle time and an estimated idle time probability, and calculates a Q-learning function for calculating a Q value using the normalized average idle time and the estimated idle time probability. Thereafter, the Q value for the channel selection operation of the Q table is dynamically calculated using the Q learning function, and the Q value for the band selection operation is dynamically calculated using the Q value for the channel selection operation.

The node channel cluster information message generation unit 1020 constructs a node channel cluster information message to use the evaluation result of the band and channel quality and neighboring node and cluster information.

The node channel cluster information message generator 1020 configures a node characteristic field using a node identification factor and residual energy, and configures a channel quality field using the available channel list and Q values of the available channels. Thereafter, the neighboring node connectivity field is constructed using the neighboring node list and the neighboring node available channel list, and the neighboring cluster connectability field is constructed using a set of connectable neighboring clusters and common active data channels of connectable neighboring clusters.

The selector 1030 selects a cluster head, a cluster member node, a gateway node, and an optimal channel in the cluster.

The selection unit 1030 according to an embodiment of the present invention calculates a channel fitness function for channels accessible by each node to select a cluster head, and the residual energy of each node, channel fitness, and connectable neighbor clusters. The cluster head fitness function is calculated using the number and the number of neighboring nodes. Thereafter, a cluster head request message is transmitted and received between nodes using the result of the cluster head fitness function to select a cluster head.

In the selection unit 1030 according to another embodiment of the present invention, a cluster head elected to select a cluster member node broadcasts a cluster head selection result message, and the node receiving the cluster head selection result message has its own available channel. And send a subscription request message. Thereafter, when the node receiving the message as a result of the cluster head selection receives a duplicate message, the cluster head is selected.

The selection unit 1030 according to another embodiment of the present invention selects a gateway node by using the node channel cluster information received by the cluster head, and when a member node connectable to an arbitrary cluster exists in duplicate, the gateway Select a node.

The selection unit 1030 according to another embodiment of the present invention selects a gateway node using the node channel cluster information received by the cluster head in order to select an optimal channel in the cluster, and is a member node that can be connected to an arbitrary cluster. If there are duplicates, the gateway node is selected.

11 is a diagram showing cluster formation in a situation in which multiple clusters are simultaneously created according to an embodiment of the present invention.

이기 때문에 노드 A(1110)는 [수학식 13]을 참조하여 클러스터 헤드가 될 조건을 만족하지 못 한다. 따라서 노드 B(1120), D(1130), K(1140)는 클러스터 헤드가 되고, 각 클러스터 헤드는 자신의 클러스터를 구성한다. 노드 E(1150)의 가용 채널들이 노드 D 클러스터의 CADC와 노드 K 클러스터의 CADC를 포함하는 경우, 노드 E(1150)는 클러스터간 통신을 위한 게이트웨이 노드가 된다.Similarly with reference to FIG. 11, each node calculates the cluster head fitness function of neighboring nodes including itself, and transmits a CH_REQ message to a member node having the highest cluster head fitness function. Nodes A (1110), B (1120), D (1130), and K (1140) receive at least one CH_REQ message. With reference to FIG. 11

For this reason, node A 1110 does not satisfy the condition of becoming a cluster head with reference to [Equation 13]. Therefore, nodes B 1120, D 1130, and K 1140 become cluster heads, and each cluster head constitutes its own cluster. When the available channels of node E 1150 include the CADC of the node D cluster and the CADC of the node K cluster, the node E 1150 becomes a gateway node for inter-cluster communication.

가 클러스터 헤드로 선택된 후에 클러스터를 위한 공통 활성 데이터 채널(common active data channel)을 결정한다. 클러스터간 간섭을 피하기 위해 클러스터 헤드는 주변 클러스터들의 CADC들을 사용하지 않아야 한다. 본 발명의 일 실시예에 따른 CADC 선택에서는 채널의 Q값과 채널을 사용하여 연결 가능한 주변 노드들의 개수를 사용한다. CADC 선택은 [수학식 14]를 사용하여 산정될 수 있다.Node with reference to Figure 2

After is selected as the cluster head, a common active data channel for the cluster is determined. To avoid inter-cluster interference, the cluster head should not use the CADCs of the surrounding clusters. In CADC selection according to an embodiment of the present invention, the Q value of the channel and the number of neighboring nodes that can be connected using the channel are used. CADC selection can be calculated using [Equation 14].

[Equation 14]

중에 두 번째로 큰

를 갖는 채널이 있는 경우 해당 채널은 CBDC로 지정되어 주 사용자 신호가 CADC에서 검출되는 경우 CADC를 대체할 수 있다. 이후 클러스터 헤드

는 공통제어채널을 사용하여 1홉 거리 내의 주변 노드들에게 CH_ANM 메세지를 브로드캐스팅하여 주변 노드들이 클러스터 헤드 노드 식별인자, CADC, CBDC의 정보를 획득할 수 있도록 한다.Referring to [Equation 14]

The second largest among

If there is a channel with a, the corresponding channel is designated as CBDC, and can replace CADC if the main user signal is detected by CADC. After cluster head

Broadcasts a CH_ANM message to neighboring nodes within one hop distance using a common control channel so that neighboring nodes can obtain information on the cluster head node identification factor, CADC, and CBDC.

가 다양한 클러스터 헤드들로부터 CH_ANM 메세지를 중복 수신하는 경우 해당 노드는 가장 큰 클러스터 헤드 적합도 함수 값을 갖는 최적 클러스터 헤드

를 선택하며 이러한 방법은 [수학식 15]를 사용하여 산정될 수 있다.2, in order to select a member node of the cluster, when a specific node receives a CH_ANM message and the CADC in the message is the same as one of the available channels of the corresponding node, the node uses the common control channel to use the existing channel. It can be merged into the cluster by sending a JOIN_REQ message to the cluster head. Node

When the node receives duplicate CH_ANM messages from various cluster heads, the node is the optimal cluster head with the largest cluster head fitness function value.

And this method can be calculated using [Equation 15].

[Equation 15]

는 노드

가 수신한 CH_ANM 메세지를 보낸 주변 노드들의 집합이다.Referring to [Equation 15]

Is the node

It is a set of neighboring nodes that sent the CH_ANM message received by

는 주변 노드들로부터 JOIN_REQ 메세지를 수신 받고 기 메세지를 송신한 노드들은 클러스터의 멤버 노드가 된다. 클러스터 헤드는 이미 모든 이웃 노드에 대해 이웃 클러스터 도달성 정보를 가지고 있기 때문에, 어떤 이웃 클러스터가 존재하는지, 어떤 CADC가 이웃 클러스터에 의해 사용되고 어떤 가입 요청 노드가 이웃 클러스터에 도달할 수 있는지를 알고 있다. 이러한 과정은 도 2를 참조하여 주변 노드 탐지 모듈이 동작하는 동안 수신된 NCCI 메세지를 통해 수행될 수 있다. 특정 멤버 노드가 주변 클러스터

과 상호연결될 수 있는 단 하나의 노드인 경우 해당 노드는 주변 클러스터

을 위한 게이트웨이노드

로 결정된다. 클러스터

과 연결 가능한 멤버 노드가 여러 개 존재하는 경우 클러스터 헤드

는

와

의 평균 Q값이 가장 큰 멤버노드를 게이트웨이 노드(

)로 선택하며 이는 [수학식 16]을 통해 산정될 수 있다.Referring to Fig. 2, the cluster head after the CH_ANM message is broadcast

The nodes that received the JOIN_REQ message from neighboring nodes and sent the message become member nodes of the cluster. Since the cluster head already has neighbor cluster reachability information for all neighbor nodes, it knows which neighbor cluster exists, which CADC is used by the neighbor cluster and which join request node can reach the neighbor cluster. This process may be performed through an NCCI message received while the neighboring node detection module is operating with reference to FIG. 2. Cluster around a specific member node

If there is only one node that can be interconnected with the neighboring cluster

Gateway node for

Is determined by cluster

Cluster head when there are multiple member nodes that can be connected to

Is

Wow

The member node with the largest average Q value of the gateway node (

), which can be calculated through [Equation 16].

[Equation 16]

After the gateway nodes are selected with reference to FIG. 2, the cluster head broadcasts an MN&GN_ANM message including a member node identification factor, a gateway node identification factor, and a neighbor cluster identification factor.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It can be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (11)

Sensing a frequency spectrum of a corresponding region through a cognitive radio terminal in the cognitive radio network, and dynamically evaluating the band and channel quality using Q-learning;
Constructing a node channel cluster information message to use a result of the evaluation of the band and channel quality and information on neighboring nodes and clusters; And
Selecting the cluster head, cluster member node, gateway node, and channel within the cluster
Q-learning-based clustering method comprising a. The method of claim 1,
The step of sensing the frequency spectrum of the region through the cognitive radio terminal in the cognitive radio network, and dynamically evaluating the band and channel quality using Q-learning,
Calculating a normalized average idle time and an estimated idle time probability;
Calculating a Q-learning function for calculating a Q value using the normalized average idle time and the estimated idle time probability;
Dynamically calculating a Q value for a channel selection operation of a Q table using a Q learning function; And
Dynamically calculating the Q value for the band selection operation using the Q value for the channel selection operation.
Q-learning-based clustering method comprising a. In claim 1,
The step of constructing a node channel cluster information message to use the evaluation result of the band and channel quality and the neighboring node and cluster information,
Constructing a node characteristic field using the node identification factor and the residual energy;
Constructing a channel quality field using the available channel list and Q values of the available channels;
Configuring a neighboring node connectivity field using the neighboring node list and the neighboring node available channel list; And
The step of constructing a peripheral cluster connectability field using a set of connectable peripheral clusters and common active data channels of connectable peripheral clusters.
Q-learning-based clustering method comprising a. In claim 1,
Selecting a cluster head, a cluster member node, a gateway node, and a channel within the cluster,
Calculating a channel fitness function for channels accessible by each node to select a cluster head;
Calculating a cluster head fitness function using the residual energy of each node, channel fitness, the number of connectable neighboring clusters, and the number of neighboring nodes; And
The step of selecting a cluster head by sending and receiving a cluster head request message between nodes using the result of the cluster head fitness function.
Q-learning-based clustering method comprising a. In claim 1,
Selecting a cluster head, a cluster member node, a gateway node, and a channel within the cluster,
A cluster head selected to select a cluster member node, broadcasting a cluster head selection result message;
Comparing, by the node receiving the cluster head selection result message, with its available channel and transmitting a subscription request message; And
Selecting a cluster head when the node receiving the message as a result of selecting the cluster head receives a duplicate message
Q-learning-based clustering method comprising a. In claim 1,
Selecting a cluster head, a cluster member node, a gateway node, and a channel within the cluster,
Selecting, by the cluster head, a gateway node using the received node channel cluster information; And
Step of selecting a gateway node when there are overlapping member nodes that can be connected to any cluster
Q-learning-based clustering method comprising a. In claim 1,
Selecting a cluster head, a cluster member node, a gateway node, and a channel within the cluster,
Selecting, by the cluster head, a gateway node using the received node channel cluster information to select a channel in the cluster; And
Step of selecting a gateway node when there are overlapping member nodes that can be connected to any cluster
Q-learning-based clustering method comprising a. A band and channel quality evaluation unit that senses a frequency spectrum of a corresponding region through a cognitive radio terminal in the cognitive radio network and dynamically evaluates the quality of the band and channel using Q-learning;
A node channel cluster information message generator for configuring a node channel cluster information message to use a result of evaluation of band and channel quality and neighboring node and cluster information; And
Selector to select cluster head, cluster member node, gateway node, and channel within the cluster
Q learning-based clustering device comprising a. The method of claim 8,
Band and channel quality evaluation unit,
Calculate the Q-learning function to calculate the Q value by calculating the normalized average idle time and the estimated idle time probability, and dynamically calculate the Q-value for the channel selection behavior of the Q table using the Q-learning function, and channel selection. Dynamically calculating the Q value for the band selection operation using the Q value for the operation.
Q-learning-based clustering device. The method of claim 8,
The node channel cluster information message generation unit,
The node characteristic field is constructed using the node identification factor and the residual energy, the channel quality field is constructed using the available channel list and the Q values of the available channel, and the neighboring node connectivity using the neighboring node list and the neighboring node available channel list. The field is composed of a set of connectable peripheral clusters and the common active data channels of connectable peripheral clusters are used to configure the peripheral cluster connectability field.
Q-learning-based clustering device. The method of claim 8,
The choice is,
To select a cluster head, the channel fitness function is calculated for channels accessible by each node, and the cluster head fitness function is calculated using the residual energy of each node, channel fitness, the number of connectable surrounding clusters, and the number of surrounding nodes. And, using the result of the cluster head fitness function, the cluster head request message is sent and received between nodes to select the cluster head,
The cluster head elected to select a cluster member node broadcasts the cluster head selection result message, the node receiving the cluster head selection result message compares it with its available channel and transmits a subscription request message, and the cluster head selection If the node receiving the result message receives a duplicate message, select the cluster head,
The cluster head selects a gateway node using the received node channel cluster information, and if there are overlapping member nodes that can be connected to any cluster, selects the gateway node,
In order to select a channel within the cluster, the cluster head selects a gateway node using the received node channel cluster information, and selects a gateway node when there are duplicate member nodes that can be connected to any cluster.
Q-learning-based clustering device.

Images