WO2022113169A1

WO2022113169A1 - Communication quality prediction apparatus, communication quality prediction system, communication quality prediction method, and communication quality prediction program

Info

Publication number: WO2022113169A1
Application number: PCT/JP2020/043666
Authority: WO
Inventors: 馨子高橋; 理一工藤; 智明小川
Original assignee: 日本電信電話株式会社
Priority date: 2020-11-24
Filing date: 2020-11-24
Publication date: 2022-06-02
Also published as: JP7453587B2; JPWO2022113169A1

Abstract

This communication quality prediction apparatus is provided with: an object information acquisition unit (11) for acquiring information on a peripheral object (2) which is present on the periphery of a communication terminal (TA) and the communication terminal; an accompaniment classification unit (12) for extracting accompaniments accompanying the peripheral object (2), classifying the extracted accompaniments into a plurality of classes, and associating the accompaniment of each class with the peripheral object; and a database (16) which stores information of the plurality of classes, and information of the accompaniment of each class. This communication quality prediction apparatus is further provided with: a communication condition measurement unit (14) for acquiring communication information of the communication terminal; an estimation model generation unit (15) for generating, on the basis of information on the accompaniment associated with the peripheral object (2) and the communication information acquired by the communication condition measurement unit, an estimation model from machine learning on how a positional relationship between the communication terminal and the accompaniment is related to the communication quality of the communication terminal; and a communication condition estimation unit (17) which, in performing wireless communication with the communication terminal, refers to the estimation model and estimates the communication quality of the communication terminal on the basis of the positional relationship between the communication terminal acquired by the object information acquisition unit (11) and the accompaniment associated with the peripheral object.

Description

Communication quality prediction device, communication quality prediction system, communication quality prediction method, and communication quality prediction program

The present invention relates to a communication quality prediction device, a communication quality prediction system, a communication quality prediction method, and a communication quality prediction program.

In a wireless communication system that adopts a communication standard such as 5G, the communication quality may change due to the movement of the communication terminal or the movement of objects around the communication terminal. In particular, in communication using high-frequency radio waves, the straightness of the radio waves is enhanced, so that the influence of surrounding objects becomes more remarkable.

For example, when an autonomous traveling robot equipped with a communication terminal is moved from a predetermined starting point to a target point by wireless communication control, the orientation of the robot, fixed objects existing around the robot, moving objects, pedestrians, etc. Communication quality such as received signal power, signal-to-noise power ratio, RSSI (Received Signal Strength Indication), RSRQ (Received Signal Reference Quality), etc. of the communication terminal changes depending on the surrounding objects.

If the communication quality deteriorates, it may affect the control of the robot and make it impossible to move the robot from the starting point to the target point. For this reason, changes in communication quality when performing wireless communication are predicted in advance, and if deterioration in communication quality is predicted, switching to a communication channel with higher communication quality or moving with higher communication quality is expected. It is necessary to take measures such as selecting a route and running the robot. Therefore, it is desired to predict the communication quality when performing wireless communication control with high accuracy.

Non-Patent Document 1 proposes that the surrounding environment of a communication terminal is photographed by a camera, and the deterioration of communication quality when a pedestrian shields a wireless communication path by using the image of the surrounding environment is predicted by machine learning. ing.

Further, Non-Patent Document 2 proposes to recognize an object from an image of the surrounding environment of a communication terminal and output bounding box information of the recognized object at high speed, which is combined with the technique of Non-Patent Document 1. Therefore, it is considered possible to improve the prediction accuracy of communication quality.

In the above-mentioned non-patent document 1, it is possible to detect an object existing around the moving device, but it is difficult to acquire information on the direction, posture, and position of the detected object. Further, in Non-Patent Document 2, although a surrounding object can be detected by a bounding box, it is still difficult to acquire information on the direction, posture, and position of the detected object. For this reason, there is a problem that it is not possible to measure the influence of an object existing around the communication terminal on communication with high accuracy, and it is difficult to predict the communication quality with high accuracy.

The present invention has been made in view of the above circumstances, and an object thereof is a communication quality prediction device, a communication quality prediction system, and a communication quality prediction capable of predicting the communication quality of a communication terminal with high accuracy. The purpose is to provide a method and a communication quality prediction program.

The communication quality prediction device of one aspect of the present invention is from a communication terminal, an object information acquisition unit that acquires information on surrounding objects existing around the communication terminal, and the surrounding object acquired by the object information acquisition unit. , The incidental items attached to the surrounding object are extracted, the extracted incidental items are classified into a plurality of predefined classes, and the incidental items of each class are associated with the surrounding object. A database that stores class information and information of ancillary objects of each class, a communication state measuring unit that acquires communication information of the communication terminal, information of ancillary objects associated with the surrounding object, and the communication state. Based on the communication information acquired by the measuring unit, it is estimated that the relationship between the positional relationship between the communication terminal and the accessory and the communication quality of the communication terminal is machine-learned to generate an estimation model. In the positional relationship between the communication terminal acquired by the object information acquisition unit and an accessory associated with the surrounding object when wireless communication is performed between the model generation unit and the communication terminal. Based on this, it includes a communication state estimation unit that estimates the communication quality of the communication terminal with reference to the estimation model.

The communication quality prediction system of one aspect of the present invention is provided in a predetermined area set around the communication terminal and a marker attached to at least one of the communication terminal and surrounding objects existing around the communication terminal. It is provided with an object detection sensor for detecting the communication terminal and the surrounding object, and a communication quality prediction device for estimating the communication quality of the communication terminal based on the positional relationship between the communication terminal and the surrounding object. The communication quality prediction device is from an object information acquisition unit that acquires information on the communication terminal and surrounding objects detected by the object detection sensor, and the communication terminal and surrounding objects acquired by the object information acquisition unit. , The accessory that accompanies the communication terminal and the surrounding object is extracted, the extracted accessory is classified into a plurality of predefined classes, and the accessory of each class is associated with the communication terminal and the surrounding object. The object classification unit, the database for storing the information of the plurality of classes and the information of the incidental objects of each class, the communication state measuring unit for acquiring the communication information of the communication terminal, and the communication terminal and surrounding objects are associated with each other. The relationship between the positional relationship between the communication terminal and the accessory and the communication quality of the communication terminal based on the obtained information of the accessory and the communication information acquired by the communication state measuring unit. The communication terminal acquired by the object information acquisition unit and the surrounding object when wireless communication is performed between the estimation model generation unit that generates an estimation model by machine learning of sex and the communication terminal. It is provided with a communication state estimation unit that estimates the communication quality of the communication terminal with reference to the estimation model based on the positional relationship with the accessory associated with the above.

The communication quality prediction method of one aspect of the present invention includes a step of acquiring information on a communication terminal and surrounding objects existing around the communication terminal, and incidental incidental to the surrounding object from the acquired peripheral object. A step of extracting an object, classifying the extracted ancillary objects into a plurality of predefined classes, and associating the ancillary objects of each class with the surrounding object, information of the plurality of classes, and an accessory of each class. Based on the step of storing the information of the above, the step of acquiring the communication information of the communication terminal, the information of the incidental objects associated with the surrounding object, and the acquired communication information, the communication terminal and the said. The step of machine learning the relationship between the positional relationship with an accessory and the communication quality of the communication terminal to generate an estimation model, and when performing wireless communication with the communication terminal, the communication terminal It is provided with a step of estimating the communication quality of the communication terminal with reference to the estimation model based on the positional relationship between the device and the incidental object associated with the surrounding object.

One aspect of the present invention is a communication quality prediction program for operating a computer as the communication quality prediction device.

According to the present invention, it is possible to predict the communication quality of a communication terminal with high accuracy.

FIG. 1 is a block diagram showing a configuration of a communication quality prediction device according to the first embodiment. FIG. 2 is an explanatory diagram showing an area where a mobile robot equipped with a communication terminal moves and a position of a camera installed in this area. FIG. 3 is an explanatory diagram showing a bounding box of a mobile robot detected by the communication quality prediction device according to the first embodiment. FIG. 4A is an explanatory diagram showing a method of classifying a peripheral object into a plurality of incidental objects, and shows an example of classifying the peripheral object and the incidental objects of the peripheral object into different classes. FIG. 4B shows an example of classifying a pedestrian, which is a surrounding object, into a pedestrian and its ancillary “head” and left and right “hands”. FIG. 5A is an explanatory diagram showing a method of classifying surrounding objects into a plurality of incidental objects, and shows an example of classifying the incidental objects included in the surrounding objects into the same class. FIG. 5B shows an example of classifying a truck, which is a peripheral object, into a driver's seat and a loading platform, which are ancillary objects thereof. FIG. 6A is an explanatory diagram showing a method of classifying surrounding objects into a plurality of ancillary objects, and shows an example of classifying the ancillary objects into the same class as the surrounding objects. FIG. 6B shows an example of classifying a pedestrian, which is a surrounding object, into the pedestrian and the mobile phone possessed by the pedestrian. FIG. 7 is a flowchart showing a processing procedure of the communication quality prediction device according to the first embodiment. FIG. 8 is a block diagram showing a configuration of the communication quality prediction device according to the second embodiment. FIG. 9A is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object D1. FIG. 9B is an explanatory diagram showing an example in which the marker 21b is attached to the surrounding object D2. FIG. 9C is an explanatory diagram showing an example in which the marker 21c is attached to the surrounding object E1. FIG. 10A is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object D1. FIG. 10B is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object D2. FIG. 10C is an explanatory diagram showing an example in which the marker 21c is attached to the surrounding object E1. FIG. 11A is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object D1. FIG. 11B is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object D2. FIG. 11C is an explanatory diagram showing an example in which the marker 21a is attached to the surrounding object E1. FIG. 12 is an explanatory diagram showing a bounding box of a mobile robot and a marker detected by the communication quality prediction device according to the second embodiment. FIG. 13 is a flowchart showing a processing procedure of the communication quality prediction device according to the second embodiment. FIG. 14 is a graph showing the change of the ^R2 value with respect to the predicted destination time when the marker is attached and when the marker is not attached. FIG. 15 is a block diagram showing a hardware configuration of the communication quality prediction device.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Structure of the first embodiment]
The communication quality prediction device according to the present embodiment controls a mobile robot (hereinafter abbreviated as "robot") equipped with a communication terminal using a communication standard such as 5G when controlling the movement within a predetermined area. An example of estimating the quality of wireless communication information (hereinafter referred to as "communication quality") between a base station that transmits a command and a communication terminal will be described.

Wireless communication information includes radio wave propagation information of wireless communication signal, channel information related to radio wave propagation information, feedback information of channel information, received signal power, signal to noise power ratio, signal to interference noise power ratio, RSSI (Received Signal). Indicators related to QoE (Quality of experience) such as Strength Indication), RSRQ (Received Signal Reference Quality), packet error rate, number of reached bits, bit error rate, number of reached bits per unit time, and these numerical values. It refers to differential information, indicators calculated from these numerical values using calculation formulas, and setting items of the communication system that affect these indicators.

The communication quality of a robot equipped with a communication terminal changes due to the influence of the position in the area and the objects existing around it. If the communication quality deteriorates, it may not be possible to move the robot to the destination by wireless communication control. In the present embodiment, the communication quality prediction device estimates the communication quality when the communication terminal communicates by using machine learning. Based on the estimated communication quality, the communication quality predictor selects a channel for wireless communication between the base station and the communication terminal, and selects the travel path of the robot in the area to select the base station. It suppresses the deterioration of the communication quality between the communication terminal and the communication terminal.

FIG. 1 is a block diagram showing a configuration of a communication quality prediction device 1 and its peripheral devices according to the first embodiment of the present invention. As shown in FIG. 1, the communication quality prediction device 1 according to the present embodiment includes an object information acquisition unit 11, an accessory classification unit 12, an accessory data creation unit 13, a communication state measurement unit 14, and an estimation model. It includes a generation unit 15, a database 16, and a communication state estimation unit 17.

The object information acquisition unit 11 acquires an image taken by at least one camera 4 (object detection sensor) installed in a predetermined area. The predetermined area is an area in which the robot 3 equipped with the communication terminal TA can move. As shown in FIG. 2, for example, the object information acquisition unit 11 acquires images taken by cameras 4 (4a, 4b) installed at two locations in the area RE. As the camera 4, for example, a depth camera or an RGB camera can be used. In this embodiment, an example in which the camera 4 is used as an example of the object detection sensor will be described, but it is also possible to use a rider, a laser radar, or the like as another object detection sensor.

The object information acquisition unit 11 also acquires at least one of the information of the position, orientation, size, movement speed, and orientation change speed of the object included in the image from the image taken by the camera 4. .. The objects included in the image include a fixed object installed in the area RE, a moving object moving in the area RE, an object 2 such as a pedestrian (hereinafter referred to as "surrounding object"), and a communication terminal TA. Includes the robot 3

The object information acquisition unit 11 also extracts an image of the robot 3 from the image taken by the camera 4 and sets a bounding box for the robot 3. Specifically, when the robot 3 is detected from the image, a rectangular bounding box B1 is set around the robot 3 as shown in FIG. 3, and the width W and height H of the bounding box B1 are measured. do. Further, the center coordinates (X, Y) of the bounding box B1 are set based on the width W and the height H. By setting the bounding box B1 and its center coordinates, the accuracy of recognizing the position of the robot 3 can be improved.

The accessory classification unit 12 classifies the accessory of the surrounding object 2 detected by the object information acquisition unit 11 into a plurality of classes (classification items). The "class" is an index for classifying incidental objects of the surrounding object 2. The classified ancillary objects are associated with each other and are defined as one peripheral object 2. The accessory classification unit 12 also classifies the accessory of the robot 3 into a plurality of classes as needed.

Hereinafter, incidental items and classes will be specifically described with reference to FIGS. 4A to 6B. In FIG. 4A, the peripheral object 2 is defined as the first classification “class C1”, and the incidental objects of the peripheral object 2 are defined as the second classification “class C1a, C1b” which is a subdivision of the class C1. An example to define is shown. Specifically, as shown in FIG. 4B, when a pedestrian is detected as a surrounding object 2, this pedestrian is defined as "class C1". Further, the "head" that is an accessory of the pedestrian is defined as "class C1a", and the left and right "hands" are defined as "class C1b".

In addition, in FIGS. 4A and 4B, an example of classifying the first classification and the subdivided second classification is shown, but the second classification is further subdivided and the third and subsequent classifications are defined. Is also possible.

FIG. 5A shows an example in which the surrounding object 2 is classified into a plurality of incidental objects, and each incidental object is defined as the second classification "class C1a, C1b". Specifically, as shown in FIG. 5B, when a truck is detected as a peripheral object 2, the truck itself is not defined, the driver's seat, which is an accessory of the truck, is "class C1a", and the loading platform is "class C1b". Is defined as.

FIG. 6A shows an example in which the peripheral object 2 and the incidental object 22 attached to the peripheral object 2 are defined as "classes C1 and C2", which are the first classifications, respectively. Specifically, as shown in FIG. 6B, when a pedestrian is detected as a peripheral object 2, and the pedestrian has a mobile phone (incidental 22), the pedestrian and the mobile phone, respectively. Is defined as the first classification, "classes C1 and C2".

The accessory classification unit 12 classifies surrounding objects into a plurality of classes by using at least one of the above three classification methods. Further, as will be described later, the information of each class is associated with each other in one surrounding object 2 and is set as one class group.

Returning to FIG. 1, the incidental data creation unit 13 creates incidental data based on the incidental data of each class classified by the incidental classification unit 12. Specifically, according to the setting input from the outside, the classified data is created by adopting one of the above-mentioned three methods and stored in the database 16.

For example, when a pedestrian walking in the area RE shown in FIG. 2 is photographed by the camera 4a and the pedestrian is detected as a surrounding object, the pedestrian is classified into class C1 as shown in FIG. 3B, for example. The pedestrian "head" is classified into class C1a, the left and right "hands" are classified into class C1b, and the pedestrian "head" is stored in the database 16 as an associated class group.

The database 16 stores the peripheral object 2 created by the incidental data creation unit 13 and the data related to the incidental objects attached to the peripheral object 2. The database 16 also stores the data of the class group created by the accessory data creation unit 13.

The communication state measuring unit 14 measures the communication quality of wireless communication when transmitting a control wireless signal from a base station (not shown) to a communication terminal TA mounted on the robot 3. Specifically, when the robot 3 equipped with the communication terminal TA is moved and controlled in the area RE shown in FIG. 2, the wireless communication information between the base station that transmits the control radio signal and the communication terminal TA Measure quality. As described above, the wireless communication information is RSSI (Received Signal Strength Indication) or the like.

The estimation model generation unit 15 acquires information on the class group of the surrounding object 2 and information such as the position, orientation, moving speed, and rotation speed of each accessory classified by the accessory classification unit 12 from the database 16. The estimation model generation unit 15 also acquires the communication quality measured by the communication state measurement unit 14, and based on the relationship between the position information of the robot 3, the position information of the surrounding object 2 and its ancillary objects, and the communication quality. For example, machine learning using a neural network is performed to generate an estimation model of communication quality. The details of machine learning will be omitted.

The communication state estimation unit 17 transmits the position information of the robot 3 when the communication terminal TA performs the movement control of the robot 3 by transmitting the control radio signal to the communication terminal TA by the base station (not shown). , The communication quality when the robot 3 moves in the area RE is estimated based on the position information of the surrounding object 2 and its ancillary objects, and the communication quality estimation model generated by the estimation model generation unit 15. For example, as shown in FIG. 2, when the robot 3 moves from the starting point P1 to the target point P2, the communication quality by the communication terminal TA at the position of the robot 3 is estimated. The communication state estimation unit 17 also outputs the estimated communication quality information to an external device.

[Operation of the first embodiment]
Next, the operation of the communication quality prediction device 1 according to the first embodiment described above will be described with reference to the flowchart shown in FIG. 7.

First, in step S11, the object information acquisition unit 11 has a robot 3 moving in the region RE shown in FIG. 2 based on an image taken by the camera 4, and a surrounding object 2 existing around the robot 3. Is detected. As described above, the surrounding object 2 refers to a fixed object such as a support, a moving body such as a dolly, a pedestrian, or the like existing in the area RE.

As shown in FIG. 3, the object information acquisition unit 11 also sets the bounding box B1 including the image of the robot 3, and further calculates the center coordinates (X, Y) of the bounding box B1.

In step S12, the accessory classification unit 12 detects the accessory contained in the surrounding object 2 acquired by the object information acquisition unit 11. For example, as shown in FIG. 4B, when a pedestrian is detected as a peripheral object 2, the pedestrian's "head" and left and right "hands" are detected as incidental objects.

In step S13, the accessory data creation unit 13 accepts the input of the class group setting by the user. For example, as shown in FIGS. 4A and 4B described above, a method of classifying into "class C1" indicating the first classification and "classes C1a and C1b" indicating the second subdivided classification is performed by the user. Set.

In step S14, the accessory data creation unit 13 classifies the accessory of the surrounding object 2 acquired by the object information acquisition unit 11 into the class set in the process of step S13 described above. For example, when the methods shown in FIGS. 4A and 4B described above are set, the pedestrian detected as the surrounding object 2 is classified as class C1, the "head" of the pedestrian is classified as class C1a, and the left and right of the pedestrian are left and right. "Hand" is classified into class C1b.

In step S15, the accessory data creation unit 13 sets the information about the accessory and the class classified in the process of step S14 as one class group in association with each other, and stores the information in the database 16.

In step S16, the communication state measuring unit 14 measures the communication quality in wireless communication between the communication terminal TA mounted on the robot 3 and the base station (not shown).

In step S17, the estimation model generation unit 15 performs machine learning based on the information on the communication quality measured by the communication state measurement unit 14 and the information on the surrounding object 2 stored in the database 16, and the communication state is performed. Generate an estimation model for. The information about the surrounding object 2 stored in the database 16 classifies the surrounding object 2 into a plurality of incidental objects and classifies them into a plurality of classes, so that the situation of the surrounding object 2 can be recognized in more detail. Can be done.

In step S18, the communication state estimation unit 17 acquires the information of the surrounding object 2 detected by the object information acquisition unit 11 and the information of each accessory classified by the accessory classification unit 12, and further generates an estimation model. Based on the estimation model generated by the unit 15, the communication state of the communication terminal TA mounted on the robot 3 is estimated.

In step S19, the communication state estimation unit 17 outputs the estimated communication quality information to the external device. In this way, it is possible to obtain the estimation result of the communication quality by the communication terminal TA mounted on the robot 3.

When it is predicted that the communication quality between the communication terminal TA mounted on the robot moving in the area RE and the base station will deteriorate, the channel used for transmitting and receiving the control signal may be switched, or the robot 3 may be used. It is possible to take measures such as changing the movement route of. As a result, it becomes possible to stably carry out the movement control of the robot.

[Effect of the first embodiment]
As described above, the communication quality prediction device 1 according to the first embodiment is the object information acquisition unit 11 for acquiring information on the communication terminal and surrounding objects existing around the communication terminal, and the object information acquisition unit 11. From the acquired peripheral object, the incidental objects attached to the peripheral object are extracted, the extracted incidental objects are classified into a plurality of predefined classes, and the incidental objects of each class are associated with the peripheral object. The classification unit 12, the database 16 that stores the information of the plurality of classes and the information of the accessories of each class, the communication state measurement unit 14 that measures the communication state of the communication terminal, and the surrounding objects are associated with each other. The relationship between the positional relationship between the communication terminal and the accessory and the communication quality of the communication terminal based on the information of the accessory and the communication information acquired by the communication state measuring unit 14. The communication terminal acquired by the object information acquisition unit 11 when performing wireless communication between the estimation model generation unit 15 that generates an estimation model by machine learning and the communication terminal, and the surroundings. It includes a communication state estimation unit 17 that estimates the communication quality of the communication terminal with reference to the estimation model based on the positional relationship with the incidental object associated with the object.

In the present embodiment, a plurality of incidental objects are extracted from the surrounding objects existing around the communication terminal, and each incidental object is further classified into a plurality of classes. Therefore, more detailed information on the surrounding object 2 can be obtained.

For example, when the image of the pedestrian's "hand" is shown in the image taken by the camera 4 and the image of the entire pedestrian is not shown, the method of detecting the surrounding object 2 as one object is walking. I can't recognize the person. However, as shown in FIGS. 3A and 3B, the pedestrian's "hand" is set as an accessory, and the pedestrian is defined as class C1 and the "hand" is defined as class C1b. The presence of pedestrians can be estimated even when the image is not shown.

In addition, it is possible to predict the position, orientation, movement speed, and rotation speed of a pedestrian with high accuracy based on information such as the position, orientation, movement speed, and rotation speed of the "hand", which in turn is highly accurate. It becomes possible to carry out machine learning.

As a result, the information of the surrounding object 2 existing around the communication terminal TA can be recognized in more detail, and the communication quality of the communication terminal can be estimated with high accuracy.

In the present embodiment, an example of classifying the surrounding object 2 into a plurality of classes has been described, but it is also possible to classify the robot 3 equipped with the communication terminal TA into a plurality of classes. By classifying the robot 3 into a plurality of classes, it becomes possible to acquire more detailed information of the robot, and by extension, it is possible to improve the estimation accuracy of the communication quality.

Further, since the information of a plurality of ancillary objects and classes contained in the same surrounding object are associated with each other and stored in the database 16 as one class group, when one ancillary object is detected, this ancillary substance is stored. The surrounding object 2 related to the object can be easily and accurately identified, and the estimation accuracy by the estimation model generation unit 15 can be improved.

[Structure of the second embodiment]
Next, a second embodiment of the present invention will be described. FIG. 8 is a block diagram showing a configuration of a communication quality estimation system 100 including a communication quality prediction device 1a according to a second embodiment.

As shown in FIG. 8, the communication quality prediction device 1a according to the second embodiment is different from the communication quality prediction device 1 shown in FIG. 1 described above in that it includes a marker definition unit 18. Further, it is different in that the peripheral object 2 is provided with the marker 21 and the robot 3 is provided with the marker 31. In the following, the components overlapping with FIG. 1 are designated by the same reference numerals, and the configuration description will be omitted.

As shown in FIG. 8, in the second embodiment, the marker 21 (object marker) is attached to the surrounding object 2. The marker 21 is an object having a predetermined shape or an object having a predetermined color, and is classified as an accessory of the surrounding object 2. The marker 21 is attached in advance to a peripheral object 2 such as a fixed object or a moving body existing in the region RE shown in FIG.

Similarly, the robot 3 is attached with a marker (marker 31 for communication terminals). Similar to the marker 21 described above, the marker 31 is an object having a predetermined shape or an object having a predetermined color, and is classified as an accessory of the robot 3.

In the second embodiment, the detection accuracy of the surrounding objects 2 and the robot 3 is further improved by recognizing the

markers

21 and 31 as incidental objects.

Hereinafter, specific examples of markers will be described with reference to FIGS. 9A to 11C. 9A to 11C are explanatory views showing an example of a marker attached to an object D1 and an object D2 classified in the same class, and an object E1 classified into a subdivided class of the objects D1 and D2.

FIGS. 9A to 9C are explanatory views showing an example in which

different markers

21a, 21b, and 21c are attached to the objects D1, D2, and E1, respectively. Specifically, as shown in FIG. 9A, the marker 21a is attached to the object D1, the marker 21b is attached to the object D2 as shown in FIG. 9B, and the marker 21c is attached to the object E1 as shown in FIG. 9C. In this way, since different markers are attached to each of the objects D1, D2, and E1, it is possible to recognize each object with high accuracy.

10A to 10C are explanatory views showing an example in which the same marker is attached to an object of the same class among the objects D1, D2, and E1 and different markers are attached to an object of a different class.

Specifically, as shown in FIG. 10A, the marker 21a is attached to the object D1, and as shown in FIG. 10B, the marker 21a is similarly attached to the object D2. Further, as shown in FIG. 10C, the marker 21c is attached to the object E1. In this way, by attaching different markers to each class of objects, it is possible to recognize objects of different classes with high accuracy.

11A to 11C are explanatory views showing an example in which the same marker is attached to each of the objects D1, D2, and E1.

Specifically, as shown in FIG. 11A, the marker 21a is attached to the object D1, and as shown in FIG. 11B, the marker 21a is similarly attached to the object D2. Further, as shown in FIG. 11C, the marker 21a is similarly attached to the object E1. In this way, by attaching the same marker to all the objects included in one class group, the recognition accuracy of each object can be improved. In addition, since the number of types of markers can be reduced, it is possible to reduce the load of arithmetic processing.

Each of the

markers

21a, 21b, 21c shown in FIGS. 9A to 11C is, for example, a pole having the same shape installed on the surrounding object 2, and can have a different color for each of the markers 21a to 21c. For example, the marker 21a can be set to red, the marker 21b can be set to blue, the marker 21c can be set to yellow, and the like.

Further, a marker 31 (marker for a communication terminal) is attached to the robot 3. As for the marker 31 for the communication terminal, similarly to the marker 21 described above, an object having a predetermined shape set in advance or an object having a predetermined color is attached to the robot 3 moving in the area RE in advance. For example, a columnar pole is installed as a marker 31 on the robot 3 moving in the area RE (see FIG. 12).

Returning to FIG. 8, the object information acquisition unit 11 extracts the image of the robot 3 from the image taken by the camera 4, and in addition to the bounding box B1 of the robot 3, the bounding box B2 of the marker 31 attached thereto To set. Specifically, when the robot 3 is detected from the image, the object information acquisition unit 11 sets a rectangular bounding box B1 around the robot 3 as shown in FIG. 12, and the width W of the bounding box B1. Measure the height H. The object information acquisition unit 11 sets the center coordinates (X, Y) of the bounding box B1 based on the width W and the height H. The object information acquisition unit 11 also sets the bounding box B2 of the marker 31.

The incidental object classification unit 12 extracts the

markers

21 and 31 from the surrounding objects 2 and the robot 3 acquired by the object information acquisition unit 11 and classifies them as incidental objects. The accessory classification unit 12 also classifies the accessory of the surrounding object 2 (including the marker 21) and the accessory of the robot 3 (including the marker 31) into a plurality of classes.

The marker definition unit 18 defines at least one of the shapes and colors of the object marker 21 and the communication terminal marker 31 and outputs the information to the object information acquisition unit 11. Specifically, as shown in FIGS. 9A to 9C, FIGS. 10A to 10B, and FIGS. 11A to 11C, a marker definition method is set.

[Operation of the second embodiment]
Next, the operation of the communication quality prediction device 1a according to the second embodiment described above will be described with reference to the flowchart shown in FIG.

First, in step S31, the object information acquisition unit 11 moves in the area RE shown in FIG. 2 based on the image taken by the camera 4, and the surrounding object 2 existing around the robot 3. Is detected.

As shown in FIG. 12, the object information acquisition unit 11 also sets a bounding box B1 including an image of the robot 3, a bounding box B2 including an image of the marker 31, and further, center coordinates (X,) of the bounding box B1. Y) is calculated.

In step S32, the accessory classification unit 12 detects the accessory and the marker 21 included in the surrounding object 2 detected by the object information acquisition unit 11, and the accessory and the marker 31 included in the robot.

In step S33, the accessory data creation unit 13 accepts the input of the class group setting by the user.

In step S34, the accessory data creation unit 13 classifies the surrounding objects 2 and the robot 3 detected by the object information acquisition unit 11 into the accessory. Ancillary items include

markers

21 and 31.

In step S35, the accessory data creation unit 13 saves the information about the created accessory in the database 16.

In step S36, the communication state measuring unit 14 measures the communication quality in wireless communication between the communication terminal TA mounted on the robot 3 and the base station (not shown).

In step S37, the estimation model generation unit 15 performs machine learning based on the information on the communication quality measured by the communication state measurement unit 14 and the information on the incidental items stored in the database 16, and the communication state is changed. Generate an estimation model.

In step S38, the communication state estimation unit 17 acquires the information of the surrounding object 2 detected by the object information acquisition unit 11 and the information of the incidental objects classified by the incidental object classification unit 12, and further, the estimation model generation unit. Based on the estimation model generated in 15, the communication state of the communication terminal TA mounted on the robot 3 is estimated.

In step S39, the communication state estimation unit 17 outputs the estimated communication quality information to the external device.

The inventors conducted an experiment to measure the estimation accuracy of communication quality when the marker 31 was installed on the robot 3 and when it was not installed. Specifically, in the area RE shown in FIG. 2, control was performed to move the mobile robot 3 from the starting point P1 to the target point P2 by wireless communication.

The communication standard of the communication terminal TA mounted on the robot 3 uses a wireless LAN (IEEE802.11ac), and the median value of RSSI for 0.1 seconds was measured. In this experiment, a random forest is used as a communication state estimation model, and the relationship between the bounding box information for the past 1 second obtained from the video and the RSSI information after t seconds is learned.

In this experiment, the cases where the predicted destination time t was 0.1 seconds, 0.2 seconds, 1.0 seconds, 2.0 seconds, 5.0 seconds, and 10.0 seconds were evaluated. The number of decision trees in the random forest was set to "500".

A model learned using only the information of the bounding box B1 of the robot 3 shown in FIG. 3 and a model learned using the information of both the bounding box B1 of the robot 3 and the bounding box B2 of the marker 31 shown in FIG. Performance was compared. As a result, the result as shown in FIG. 14 was obtained. For the evaluation, the "R ² score" calculated by the following equation (1) was used.

In equation (1), "Yi" is the measured value of RSSI in the i-th sample, "Yi (^)" is the predicted value of RSSI predicted in the random forest, and "Yave" is the average value of the measured value of RSSI. Is.

In equation (1), the closer the numerical value of "R ² score" is to "1", the higher the prediction accuracy. FIG. 14 is a graph showing the relationship between the predicted destination time and R ^2. The curve s1 shows the case where the marker 31 is not used, and the curve s2 shows the case where the marker 31 is used. As shown in FIG. 14, it is understood that the prediction accuracy of RSSI is higher when the marker 31 is used at any prediction destination time.

[Effect of the second embodiment]
As described above, in the communication quality prediction device 1a according to the second embodiment, the marker 21 is attached to the surrounding object 2 existing in the region RE, and the marker 31 is attached to the robot 3. By attaching the

markers

21 and 31, the accuracy when detecting the surrounding object 2 and the robot 3 is improved. Therefore, the position information of the surrounding objects 2 and the robot 3 existing around the communication terminal TA can be acquired with higher accuracy, and the estimation accuracy of the communication quality can be improved.

Further, in the second embodiment, an example of classifying both the surrounding object 2 and the robot 3 equipped with the communication terminal TA into classes has been described, but the present invention also has a configuration in which only one of them is classified into classes. good.

Further, in the first and second embodiments described above, an example of predicting the communication quality of the communication terminal TA mounted on the mobile robot 3 has been described, but the present invention is not limited to this, and the user is not limited to this. It can also be applied to predict the communication quality of mobile phones that you carry.

As shown in FIG. 15, the communication quality prediction device 1 of the present embodiment described above includes, for example, a CPU (Central Processing Unit, processor) 901, a memory 902, and a storage 903 (HDD: Hard Disk Drive, SSD: Solid). A general-purpose computer system including a State Drive), a communication device 904, an input device 905, and an output device 906 can be used. The memory 902 and the storage 903 are storage devices. In this computer system, each function of the communication quality prediction device 1 is realized by the CPU 901 executing a predetermined program loaded on the memory 902.

Further, although FIG. 15 shows an example in which the CPU 901 is used as the processor, it is also possible to use a GPU (Graphics Processing Unit). By using the GPU, the processing time can be shortened and the real-time performance can be improved.

The communication quality prediction device 1 may be mounted on one computer or may be mounted on a plurality of computers. Further, the communication quality prediction device 1 may be a virtual machine mounted on a computer.

The program for the communication quality prediction device 1 can also be stored in a computer-readable recording medium such as an HDD, SSD, USB (Universal Serial Bus) memory, CD (Compact Disc), or DVD (Digital Versatile Disc). It can also be delivered over the network.

The present invention is not limited to the above embodiment, and many modifications can be made within the scope of the gist thereof.

1, 1a Communication quality prediction device 2 Surrounding objects 3 Robot (mobile robot)
4 (4a, 4b) Camera (object detection sensor)
11 Object information acquisition unit 12 Ancillary object classification unit 13 Ancillary data creation unit 14 Communication state measurement unit 15 Estimated model generation unit 16 Database 17 Communication state estimation unit 18 Marker definition unit 21 (21a, 21b, 21c) Marker (marker for object) )
31 Marker (Marker for communication terminal)
100 Communication quality estimation system RE area TA communication terminal

Claims

A communication terminal and an object information acquisition unit that acquires information on surrounding objects existing around the communication terminal.
From the surrounding object acquired by the object information acquisition unit, the incidental objects attached to the surrounding object are extracted, the extracted incidental objects are classified into a plurality of predefined classes, and the incidental objects of each class are classified into the above-mentioned. Ancillary object classification unit associated with surrounding objects,
A database that stores information on the plurality of classes and information on incidental items in each class,
A communication state measuring unit that acquires communication information of the communication terminal, and
Based on the information of the accessory associated with the surrounding object and the communication information acquired by the communication state measuring unit, the positional relationship between the communication terminal and the accessory and the communication quality of the communication terminal. And the estimation model generator that machine-learns the relationship between and generates an estimation model,
The estimation model based on the positional relationship between the communication terminal acquired by the object information acquisition unit and an accessory associated with the surrounding object when performing wireless communication with the communication terminal. A communication state estimation unit that estimates the communication quality of the communication terminal with reference to
A communication quality prediction device characterized by being equipped with.
The accessory classification unit extracts the accessory that accompanies the communication terminal, classifies the extracted accessory into the plurality of classes, and associates the accessory of each class with the communication terminal.
The estimation model generation unit generates the estimation model based on the information of the communication terminal and the accessories associated with the communication terminal and the communication information acquired by the communication state measurement unit.
The communication quality prediction device according to claim 1.
The object information acquisition unit acquires information on a marker attached to at least one of the communication terminal and the surrounding object.
The communication quality prediction device according to claim 2, wherein the accessory classification unit extracts the marker as an accessory of the communication terminal or the surrounding object.
The communication quality prediction device according to claim 3, wherein the marker has a preset shape or a preset color.
A marker attached to the communication terminal and at least one of the surrounding objects existing around the communication terminal,
An object detection sensor provided in a predetermined area set around the communication terminal and detecting the communication terminal and surrounding objects, and an object detection sensor.
A communication quality prediction device for estimating the communication quality of the communication terminal based on the positional relationship between the communication terminal and the surrounding object is provided.
The communication quality prediction device is
An object information acquisition unit that acquires information on the communication terminal and surrounding objects detected by the object detection sensor, and an object information acquisition unit.
From the communication terminal and surrounding objects acquired by the object information acquisition unit, incidental objects attached to the communication terminal and surrounding objects are extracted, and the extracted incidental objects are classified into a plurality of predefined classes. An accessory classification unit that associates an accessory of each class with the communication terminal and the surrounding object,
A database that stores information on the plurality of classes and information on incidental items in each class,
A communication state measuring unit that acquires communication information of the communication terminal, and
Based on the information of the accessory associated with the communication terminal and surrounding objects and the communication information acquired by the communication state measuring unit, the positional relationship between the communication terminal and the accessory and the communication. An estimation model generator that machine-learns the relationship between the communication quality of the terminal and generates an estimation model,
The estimation is based on the positional relationship between the communication terminal acquired by the object information acquisition unit and the accessory associated with the surrounding object when performing wireless communication with the communication terminal. A communication quality prediction system including a communication state estimation unit that estimates the communication quality of the communication terminal with reference to a model.
A step of acquiring information on a communication terminal and surrounding objects existing around the communication terminal, and
A step of extracting ancillary objects attached to the surrounding object from the acquired peripheral object, classifying the extracted ancillary objects into a plurality of predefined classes, and associating the ancillary substances of each class with the surrounding object. When,
The step of storing the information of the plurality of classes and the information of the incidental items of each class, and
The step of acquiring the communication information of the communication terminal and
Based on the information of the accessory associated with the surrounding object and the acquired communication information, the relationship between the positional relationship between the communication terminal and the accessory and the communication quality of the communication terminal is determined. The steps of machine learning to generate an estimation model,
When performing wireless communication with the communication terminal, the communication quality of the communication terminal is estimated with reference to the estimation model based on the positional relationship between the communication terminal and an accessory associated with the surrounding object. Steps to do and
A communication quality prediction method characterized by being equipped with.
A communication quality prediction program for operating a computer as the communication quality prediction device according to any one of claims 1 to 4.