CN111106969B

CN111106969B - Automatic evaluation method and system of Internet of things platform and storage medium

Info

Publication number: CN111106969B
Application number: CN201911419874.XA
Authority: CN
Inventors: 李阅苗
Original assignee: Qingdao Haier Technology Co Ltd
Current assignee: Qingdao Haier Technology Co Ltd
Priority date: 2019-12-31
Filing date: 2019-12-31
Publication date: 2022-08-05
Anticipated expiration: 2039-12-31
Also published as: CN111106969A

Abstract

The invention provides an automatic assessment method and system of an Internet of things platform and a storage medium, wherein the assessment method comprises the following steps: the method comprises the steps of obtaining a dotting log which is collected by a terminal and aims at an Internet of things platform, wherein the dotting log comprises at least one of the following logs: original operation logs, network environment logs and user operation logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log; the Internet of things platform is evaluated according to the interconnection index, the technical scheme is adopted, the problems that the Internet of things platform cannot be accurately evaluated in the related technology and the like are solved, and an accurate automatic evaluation mode of the Internet of things platform is provided.

Description

Automatic evaluation method and system of Internet of things platform and storage medium

Technical Field

The invention relates to the field of communication, in particular to an automatic assessment method and system for an Internet of things platform and a storage medium.

Background

Interconnection and interworking index: the comprehensive evaluation indexes of the ability of contact and communication among people, network devices, environment and service platforms and the mutual interaction ability directly and quantitatively reflect the user experience of the interconnection and intercommunication whole process, and comprise the following steps: reliability and ease of use. Currently, data can only be collected and counted manually. The data is few, the accuracy is low, the efficiency is low, and the feedback is not timely.

Aiming at the problems that the Internet of things platform cannot be accurately evaluated and the like in the related technology, an effective technical scheme is not provided yet.

Disclosure of Invention

The embodiment of the invention provides an automatic assessment method and system for an Internet of things platform, a storage medium and an electronic device, and at least solves the problems that the Internet of things platform cannot be accurately assessed in the related art and the like.

According to one embodiment of the invention, an automatic evaluation method for an internet of things platform is provided, which comprises the following steps: the method comprises the steps of obtaining a dotting log which is collected by a terminal and aims at an Internet of things platform, wherein the dotting log comprises at least one of the following logs: original operation logs, network environment logs and user operation logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log, wherein the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality level.

Optionally, obtaining a dotting log for the internet of things platform collected by the terminal includes: acquiring the network environment logs in the dotting logs according to the following network environment access rules: continuously ping the routers N times per minute, and recording the average time delay and the overtime times of the routers N times ping to determine local area network data; after the target module is connected with the access gateway, recording time delay and overtime times of the ping M-time main gateway by using heartbeat acquisition parameters or the operation of the ping M-time main gateway so as to determine local area network data; a ping parameter; and a ping result, wherein the ping result only comprises the ping result without timeout, M and N are both positive integers, and N is greater than 1.

Optionally, converting the dotting log into an interconnection log according to a target index rule, including: acquiring network quality levels corresponding to the following parameters: receiving signal strength, local area network connectivity, local area network round trip delay, wide area network connectivity, wide area network round trip delay, and interconnection log standards; converting the dotting log into an interconnection log according to the network quality grade corresponding to the dotting log and an interconnection log standard, wherein the interconnection log at least comprises one of the following parameters: a log global sequence number; log recording time; interconnecting and intercommunicating corresponding services; a response code of the service request; processing duration, network quality level.

Optionally, determining an interconnection index of the internet of things platform according to the interconnection log includes: the following parameter information is obtained: the method comprises the steps that indexes set by an Internet of things platform, standard values of the indexes in different network environments, and index scores determined according to the standard values of the indexes and actual values measured by the indexes are determined, and the total score of the indexes is determined through all the indexes; and determining the interconnection index of the Internet of things platform according to the parameter information and the interconnection log.

Optionally, determining an interconnection index of the internet of things platform according to the parameter information and the interconnection log includes: acquiring an interconnection log and a product model library; performing aggregation statistics on the interconnection and intercommunication log and the product model library according to a target rule to obtain at least one result including: total number of samples, successful number of samples, success rate, average treatment time, longest treatment time, shortest treatment time; splitting the result into a target index and an actual value of the target index according to the parameter information; determining an interconnection index according to a mode of determining total mark of the index in the parameter information and actual values of the target index and the target index, wherein the interconnection index comprises one of the following: equipment classification, product model, index score.

Optionally, the total index score of all the indexes is determined by the following method: and acquiring each index and the weight corresponding to each index, and determining the total index score according to each index and the weight corresponding to each index.

Optionally, the index score includes: the score corresponding to each index and the total index score of all the indexes are evaluated on the Internet of things platform according to the interconnection index, and the evaluation method comprises the following steps: and evaluating the Internet of things platform according to the scores and the total index scores.

Optionally, after the internet of things platform is evaluated according to the scores and the total index scores, the method further includes: and under the condition that the score of any index is lower than a preset threshold value, sending indication information to the Internet of things platform to indicate the Internet of things platform to improve the corresponding score of the index.

Optionally, the index at least includes one of the following: the success rate of local area network control operation, the success rate of wide area network control operation and the disconnection rate of equipment.

According to another embodiment of the present invention, an automated evaluation system for an internet of things platform is provided, including: the system comprises an interconnection and interworking index evaluation platform and a terminal, wherein the interconnection and interworking index evaluation platform is used for acquiring a dotting log which is acquired by the terminal and aims at an Internet of things platform, and the dotting log comprises at least one of the following: original operation logs, network environment logs and user operation logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log, wherein the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality level.

Optionally, the interconnection and interworking index evaluation platform is further configured to obtain the network environment log in the dotting log according to the following network environment access rule: continuously ping the routers N times per minute, and recording the average time delay and the overtime times of the routers N times ping to determine local area network data; after the target module is connected with the access gateway, recording time delay and overtime times of the ping M-time main gateway by using heartbeat acquisition parameters or the operation of the ping M-time main gateway so as to determine local area network data; a ping parameter; and a ping result, wherein the ping result only comprises the ping result without timeout, M and N are both positive integers, and N is greater than 1.

Optionally, the interconnection and interworking index evaluation platform is further configured to obtain a network quality level corresponding to the following parameters: receiving signal strength, local area network connectivity, local area network round trip delay, wide area network connectivity, wide area network round trip delay, and interconnection log standards; converting the dotting log into an interconnection log according to the network quality grade corresponding to the dotting log and the interconnection log standard, wherein the interconnection log at least comprises one of the following parameters: a log global sequence number; log recording time; interconnecting and intercommunicating corresponding services; a response code of the service request; processing duration, the network quality level.

Optionally, the interconnection and interworking index evaluation platform is further configured to obtain the following parameter information: the method comprises the steps that indexes set by an Internet of things platform, standard values of the indexes in different network environments, and index scores determined according to the standard values of the indexes and actual values measured by the indexes are determined, and the total score of the indexes is determined through all the indexes; and determining the interconnection index of the Internet of things platform according to the parameter information and the interconnection log.

Optionally, the interconnection and interworking index evaluation platform is further configured to obtain an interconnection and interworking log and a product model library; performing aggregation statistics on the interconnection and intercommunication log and the product model library according to a target rule to obtain at least one result including: total number of samples, successful number of samples, success rate, average treatment time, longest treatment time, shortest treatment time; splitting the result into a target index and an actual value of the target index according to the parameter information; determining an interconnection index according to a mode of determining total mark of the index in the parameter information and actual values of the target index and the target index, wherein the interconnection index comprises one of the following: equipment classification, product model, index score.

Optionally, the interconnection and interworking index evaluation platform is further configured to obtain each index and a weight corresponding to each index, determine an index total score according to each index and the weight corresponding to each index, and evaluate the internet of things platform according to the score and the index total score.

Optionally, the interconnection and interworking index evaluation platform is further configured to send indication information to the internet of things platform to indicate the internet of things platform to improve the corresponding score of the index when the score of any index is lower than a preset threshold, where the index at least includes one of the following: the success rate of local area network control operation, the success rate of wide area network control operation and the disconnection rate of equipment.

According to a further embodiment of the present invention, there is also provided a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.

According to yet another embodiment of the present invention, there is also provided an electronic device, including a memory in which a computer program is stored and a processor configured to execute the computer program to perform the steps in any of the above method embodiments.

According to the method and the device, the dotting log which is acquired by the terminal and aims at the platform of the Internet of things is acquired; converting the dotting logs into interconnection logs according to a target index rule, and determining interconnection indexes of the Internet of things platform according to the interconnection logs; the Internet of things platform is evaluated according to the interconnection index, the technical scheme is adopted, the problems that the Internet of things platform cannot be accurately evaluated in the related technology and the like are solved, and an accurate automatic evaluation mode of the Internet of things platform is provided.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

fig. 1 is a block diagram of a hardware structure of a computer terminal of an automated evaluation method for an internet of things platform according to an embodiment of the present invention;

fig. 2 is a flowchart of an automated assessment method of an internet of things platform according to an embodiment of the present invention;

fig. 3 is a block diagram of an interconnection index evaluation system according to an embodiment of the present invention;

fig. 4 is a schematic architecture diagram of an automated evaluation method of an internet of things platform according to an embodiment of the invention;

FIG. 5 is a flowchart of processing original service dotting and weblogs into standard interworking logs, according to an alternative embodiment of the present invention;

FIG. 6 is a flowchart of processing a device drop log into a standard interworking log according to an alternative embodiment of the present invention;

FIG. 7 is a flow chart of a general index calculation according to an alternative embodiment of the present invention;

FIG. 8 is a flow diagram of a device drop rate calculation in accordance with an alternative embodiment of the present invention;

FIG. 9 is a block diagram of an interworking portal in accordance with an alternative embodiment of the present invention;

fig. 10 is an architecture diagram of another method for automated assessment of an internet of things platform according to an alternative embodiment of the invention;

fig. 11 is a block diagram of an automated evaluation system of an internet of things platform according to an embodiment of the present invention.

Detailed Description

The invention will be described in detail hereinafter with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.

It should be noted that, in the embodiment of the present invention, the interconnection index: the comprehensive evaluation indexes of the contact and communication capacity among people, the network device, the environment and the service platform and the mutual interaction capacity directly and quantitatively reflect the user experience of the interconnection and intercommunication whole process, and specifically can comprise the following steps: reliability and ease of use.

The method provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar operation system. Taking an example of the computer terminal running on the computer terminal, fig. 1 is a hardware structure block diagram of the computer terminal of the automatic evaluation method of the internet of things platform according to the embodiment of the invention. As shown in fig. 1, the computer terminal may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing system such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally, a transmission device 106 for communication functions and an input-output device 108. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the computer terminal. For example, the computer terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration with equivalent functionality to that shown in FIG. 1 or with more functionality than that shown in FIG. 1.

The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to the method for automatically evaluating an internet of things platform in the embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to execute various functional applications and data processing, i.e., to implement the method described above. The memory 104 may include high speed random access memory and may also include non-volatile memory, such as one or more magnetic storage systems, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission system 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission system 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission system 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

In this embodiment, an automated evaluation method for an internet of things platform is provided, and fig. 2 is a flowchart of an automated evaluation method for an internet of things platform according to an embodiment of the present invention, and as shown in fig. 2, the flowchart includes the following steps:

step 202, obtaining a dotting log for an internet of things platform, wherein the dotting log is acquired by a terminal and comprises at least one of the following: original operation logs, network environment logs and user operation logs;

step 204, converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the internet of things platform according to the interconnection log, wherein the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade;

and step 206, evaluating the Internet of things platform according to the interconnection index.

Through the steps, a dotting log which is acquired by a terminal and aims at the platform of the Internet of things is obtained; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log; the Internet of things platform is evaluated according to the interconnection index, the technical scheme is adopted, the problems that the Internet of things platform cannot be accurately evaluated in the related technology and the like are solved, and an accurate automatic evaluation mode of the Internet of things platform is provided.

There may be various implementation manners for the step S202, and in an optional embodiment, the network environment log in the dotting log is obtained by the following network environment access rules: continuously ping the routers N times per minute, and recording the average time delay and the overtime times of the routers N times ping to determine local area network data; after the target module is connected with the access gateway, recording time delay and overtime times of the ping M-time main gateway by using heartbeat acquisition parameters or the operation of the ping M-time main gateway so as to determine local area network data; a ping parameter; and a ping result, wherein the ping result only comprises the ping result without timeout, M and N are both positive integers, and N is greater than 1.

In the embodiment of the present invention, step S204 may be implemented by the following technical solutions: acquiring network quality levels corresponding to the following parameters: receiving signal strength, local area network connectivity, local area network round trip delay, wide area network connectivity, wide area network round trip delay, and interconnection log standards; converting the dotting log into an interconnection log according to the network quality grade corresponding to the dotting log and an interconnection log standard, wherein the interconnection log at least comprises one of the following parameters: a log global sequence number; log recording time; interconnecting and intercommunicating corresponding services; a response code of the service request; processing duration, network quality level.

The implementation scheme for determining the interconnection index of the Internet of things platform according to the interconnection log can be implemented in the following modes: the following parameter information is obtained: the method comprises the steps that indexes set by an Internet of things platform, standard values of the indexes in different network environments, and index scores determined according to the standard values of the indexes and actual values measured by the indexes are determined, and the total score of the indexes is determined through all the indexes; determining an interconnection index of the platform of the internet of things according to the parameter information and the interconnection log, and further determining the interconnection index of the platform of the internet of things according to the parameter information and the interconnection log, which may include the following technical solutions: acquiring an interconnection log and a product model library; performing aggregation statistics on the interconnection and intercommunication log and the product model library according to a target rule to obtain at least one result including: total number of samples, successful number of samples, success rate, average treatment time, longest treatment time, shortest treatment time; splitting the result into a target index and an actual value of the target index according to the parameter information; determining an interconnection index according to a mode of determining total mark of the index in the parameter information and actual values of the target index and the target index, wherein the interconnection index comprises one of the following: equipment classification, product model, index score.

It should be noted that after the internet of things platform is evaluated, in the case that the score of any index is lower than the preset threshold value, the indication information is sent to the internet of things platform to indicate the internet of things platform to improve the corresponding score of the index.

In the technical solution disclosed above in the embodiment of the present invention, the index at least includes one of the following: the success rate of local area network control operation, the success rate of wide area network control operation and the disconnection rate of equipment.

In order to better understand the technical solution of the present invention, the following approximately explains the interconnection index evaluation system related in the embodiment of the present invention, and it can be understood that:

interconnection and interworking index: the comprehensive evaluation indexes of the ability of contact and communication among people, network devices, environment and service platforms and the mutual interaction ability directly and quantitatively reflect the user experience of the interconnection and intercommunication whole process, and comprise the following steps: reliability and ease of use.

At present, data of an internet of things platform can only be collected and counted manually, and the problems of few data, low accuracy, low efficiency and untimely feedback are solved in the mode.

The technical scheme of the embodiment of the invention can be understood as constructing an interconnection index evaluation system as shown in fig. 3, collecting and evaluating the indexes of the basic platform of the internet of things in all aspects of reliability and usability, and feeding back the evaluation result to the basic platform of the internet of things in time as an important basis for platform improvement and upgrading to form a sustainable virtuous circle mechanism from collection-evaluation-feedback-improvement-collection.

The following explains the above technical solution with reference to an alternative embodiment, but is not intended to limit the technical solution of the embodiment of the present invention, and the alternative embodiment of the present invention may be understood as mainly including the following five steps:

alternative embodiment 1

The method comprises the steps that firstly, terminals such as a module, an SDK (software development kit), an App (application profile) and the like operate and click according to a standard format respectively, and an obtained click log is reported according to requirements.

And step two, the server side collects the dotting logs and stores the dotting logs to the big data platform.

And step three, the big data platform calculates according to an interconnection index calculation rule established by a predetermined standard to obtain an interconnection index.

And step four, feeding back the result of the interconnection index to the IOT platform through an interconnection Portal system (Portal).

And step five, the IOT platform optimizes and improves according to the feedback interconnection index condition.

Alternative embodiment 2

As shown in fig. 4, an architecture of an automated evaluation method for an internet of things platform is provided, and as shown in fig. 4, based on the architecture diagram shown in fig. 4, an alternative embodiment of the present invention includes the following steps:

step 1, the terminal log dotting module at least collects user operation logs and network environment logs and reports the user operation logs and the network environment logs to a log collection module according to requirements.

In this step, the log dotting process may be understood as that the U + APP, the ussdk, and the module perform log dotting and reporting on the operation and the network according to the acquisition requirement formulated by the chain tracking system, specifically,

and the U + APP acquires a dotting log accessing the UWS interface of the cloud platform through a dotting interface provided by the uSDK and reports the dotting log to a cloud platform log gateway.

The uSDK mainly collects local area network control operation, wide area network control operation, configuration binding operation dotting log and network environment log, and APP dotting log.

The module mainly collects and configures binding dotting logs, control command dotting logs, network environment access dotting logs and connection platform stability dotting logs.

It should be noted that the dotting log of the terminal dotting mainly includes: at least one of: original operation log, network environment log and user operation log.

The network environment index may specifically include at least: the network environment access rule comprises indexes such as signal strength, local area network connectivity (ping router success rate), local area network signal round-trip delay (ping router delay), wide area network connectivity (ping gateway success rate), wide area network signal round-trip delay (ping gateway delay) and the like, and further can be realized in the following ways:

1) local area network data acquisition: continuously ping 4 times router per minute, recording average time delay and overtime times

2) The specific method of the wide area network data acquisition index is as follows: after the module establishes connection with the access gateway, the heartbeat acquisition index is used; otherwise, ping 1 is performed by the gateway for a second time, and after the ping is completed, the time delay of the ping and the overtime times (0-1) of the ping are recorded.

3) ping parameters: the interval is 1000 milliseconds, the timeout time is 5000 milliseconds, and the payload length of the ping message is 64 bytes.

4) And ping result: calculating the average delay only uses the result of pings that did not timeout, if all pings timed out, the average delay should be recorded as-1.

And 2, collecting and processing the dotting logs from the terminal by a log collecting and processing module, and converting the dotting logs into interconnection standard logs.

In this step, the collection of the log may be collection of an IOT log and a traffic log, wherein the IOT log includes the following aspects:

1. and (4) inputting, namely collecting the original operation log and the network environment log (realized by IOT) reported from the terminal.

2. And (4) processing, namely preprocessing the collected logs. Extracting operation logs related to success rate and responsiveness operations from a database (OSS); filtering out invalid logs; synchronizing to big data for processing.

3. And outputting the original service log and the network quality log related to the interconnection and interworking index.

Data acquisition: the collected data items are shown in reference to table 1 below:

TABLE 1

As shown in the above table 1, in which,

signal intensity: taking the average value (several or even several) of the signal intensity of the control and APP access cloud platform or binding time points in 2 minutes; if no value exists, the network quality is qualified;

average ping AP delay: taking the average value of average time delay of 2 minutes before and after the control and APP access to the cloud platform or binding the time point (if the value is-1, the network is overtime, and the data does not participate in statistics); if no value exists, the result is qualified; if the values are all-1, the statistics are not participated in;

average time delay of the ping gateway: taking the average value of average time delay of 2 minutes before and after the control and APP access to the cloud platform or binding the time point (if the value is-1, the network is overtime, and the data does not participate in statistics); if no value exists, the result is qualified; if the values are all-1, the statistics are not participated in;

ping AP success rate: selecting ping AP records within 2 minutes before and after a control and APP access cloud platform or a binding time point, and calculating:

ping AP success rate is 1- (sum of args. alost values in 2 minutes records before and after)/(4 x total number of records)

The success rate of the ping gateway is as follows: selecting ping gateway records within 2 minutes before and after a control and APP access cloud platform or a binding time point, and calculating:

the success rate of the ping gateway is 1- (sum of args. ilost values in the front and back 2 minutes records)/total number of records

For the service log, the following aspects are included:

1. inputting: collecting operation logs reported from the U + APP ecocircles;

2. and (4) processing, namely preprocessing the collected logs. Records and attributes associated with the interconnection and interworking indicator are extracted from the existing logs.

3. And outputting the original service log related to the interconnection index.

And 3, converting the operation log into an interconnection index by an interconnection index core calculation module, and calculating into a final interconnection index according to the scoring and calculation rules.

And 4, providing a network classification standard for network quality log processing by the interconnection standard configuration management module, and providing a scoring and calculation rule for the interconnection index core calculation module.

In the embodiment of the present invention, a management interface may be further provided for the interconnection and interworking Portal standard configuration management module, and basic data and rules are provided for the log processing and index calculation processes related in the embodiment of the present invention, where the provided management interface includes: a base network hierarchical standard management interface, an index management interface, etc., wherein,

1) a base network classification standard management interface for maintaining a base network quality classification standard, as shown in table 2 below:

TABLE 2

Further, acquiring module network environment data through time and mac association, acquiring mobile phone network environment data through time and tk association, comparing the 5 index data acquired by the module end and the mobile phone end with the standards in the table, and taking the worst network quality as the final evaluation; the final basic network quality grade is a group of poor network quality data obtained by the control terminal and network quality data obtained by the intelligent household equipment; if one of the module end and the mobile phone end has no index data or only one end has data, the network quality is qualified; when the index data is out of the standard range (i.e., worse than "bad"), the binding, manipulation, etc. records that occur at these times do not participate in the calculations.

2) The index management interface is used for maintaining the following tables 3-5, wherein the table 3 is a table corresponding to service management, the table 4 is a table corresponding to index management, and the table 5 is a standard value of each index in different network basic environments:

TABLE 3

TABLE 4

TABLE 5

The score scoring rule of each index may be described in the form of a function y ═ f (N, X).

Wherein X is a standard value, N is a measured actual value, and y is a return value.

That is, the score y of an index is calculated from the measured actual value N and the standard value X of the index.

For y ═ f (N, X) can be determined as follows, as shown in table 6 below:

TABLE 6

3) An index management interface for maintaining the table shown in Table 7 below

TABLE 7

In addition, the technical scheme for converting the acquired log into the interconnection index log in the embodiment of the invention can be realized by the following modes:

as shown in fig. 5, the original service dotting log and the weblog are processed into a standard interconnection log.

The interconnection standard log comprises sn, time, service, response code, processing market, network environment grade, api, devid and uid.

Wherein Sn is a log global sequence number; time is log recording time; service related to interconnection and interworking, (the service needs to be configurable);

if BIND is binding; LAN _ OP is local area network operation; WAN _ OP for wide area network operation, ACC _ UWS for APP access

Response code-the response code of the service request.

Response time-request time;

the network environment level.

The parameter configurability needs to meet the following requirements: the interconnection and interworking related service can be configured; the base network environment level is configurable.

For the device drop rate processing procedure, as shown in fig. 6, the device drop log is processed into a standard interconnection log.

Sn is a log global sequence number; time, recording the time of the disconnection; service OFFLINE is disconnected (the service needs to be configurable); response code is response code of the service request; reference < interconnection index evaluation criteria >; the processing time is from the disconnection of the equipment to the recovery of networking; network environment level, reference < interconnection and interworking index evaluation specification > (the level needs to be configurable).

Fig. 7 shows a flowchart of the general index calculation, which is implemented as follows:

and inputting the converted interconnection standard log, the product model library, the interaction standard degree and the user satisfaction degree index.

And outputting the interconnection index (date, equipment classification, product model, index and index score).

The treatment process is divided into 5 stages.

1. And carrying out aggregate statistics according to date, product, service and network quality, and counting the total sample number, successful sample number, success rate, average processing time, the longest processing time and the shortest processing time.

2. And splitting the indexes, referring to the index definition, and splitting the statistical result into the indexes and the actual values N of the indexes.

3. Calculating the score of the index according to the date, and calculating each index and the score of the index by referring to the standard value of the index and the index scoring rule.

4. The network quality scores are summed up, and the scores at the network qualities are summed up to a total score.

5. And summarizing the indexes according to the total scores and the weights, and calculating the scores of the indexes into a final index by referring to an index calculation rule.

The parameterable configuration requirements include: the interconnection index is configurable, the standard values of the indexes in different network environments are configurable, the calculation rule of the indexes is configurable, the total score of the interconnection index is configurable, and the weight occupied by the selected indexes and each index in the calculation of the interconnection final index is configurable.

As shown in fig. 8, a flowchart for calculating the offline rate of the device is provided, and the specific implementation manner is as follows:

inputting the converted interconnection standard log, the product model library and the total number of platform devices. The service of the standard log is 'equipment is disconnected', and the processing time is 'equipment recovery network time';

output interconnection index ═ date, equipment classification, product model, index score)

The treatment process is divided into 5 stages.

1. And carrying out aggregation statistics according to dates, products, services and network quality, counting the number of times of disconnection and the longest recovery time of the equipment in different network environments according to the dates, screening abnormal equipment according to the number of times of disconnection of the equipment disconnection standard value and the single network recovery time, and counting the total sample number of the abnormal equipment according to the dates.

3. And calculating the index score according to the date, and calculating the offline rate index score of each device according to the index scoring rule.

4. The network quality scores are summed up, and the total score is summed up.

And the online scoring management interface provides interface maintenance interaction standard degree and user satisfaction index scores.

The available formats are shown in table 8:

TABLE 8

Fig. 9 is a block diagram of an interworking portal. The interconnection portal is expanded based on the existing system, and the main system provides security management including login, registration, authentication, authorization and authentication (such as the sea pole network). The interworking portal employs a BS development architecture based on the H5 framework + Restful interface. The portal provides the following functions for different types of users:

1) and providing interface maintenance interconnection standard parameter configuration for a standard team.

2) And providing an interface for registering the interaction standard degree and the user satisfaction index score of each period for a UED team and a user operation team.

3) And providing an interface for the index assessment personnel to check the interconnection and intercommunication indexes of each period.

4) And providing an interface for the IOT, the U + APP and the user center to check the relevant indexes of each period.

With respect to the dependency basic data, a list of all device classifications such as table 9 may be provided according to the device model, or all product models belonging to a certain device classification such as table 10 may be provided. The dependency basic data may also provide a device class and a product model corresponding to the device mac according to the device model, as in table 11.

Table 9:

device classification	Classification id
		Air conditioner	…
Refrigerator with a door	…
		…	…

Table 10:

table 11:

device id	Device classification	Model of the device
			A6273833873B	Air conditioner	Type	1
A6273833873C	Refrigerator with a door	Type 2
			…	…	…

The number of device models, the number of all devices of all product models per day may be counted as shown in table 12 below.

TABLE 12

And 5, the interconnection and intercommunication Portal is responsible for displaying the calculation result of the interconnection and intercommunication index in a report form and providing the functions of standard metadata maintenance, interaction standard degree and user satisfaction degree on-line scoring.

It should be noted that the interconnection and interworking index core calculation module depends on the data corresponding to the device id and the product model of the big data; the interconnection Portal report interface depends on the corresponding relation data of the equipment classification and the product model of the open platform.

Fig. 10 is a schematic diagram of an architecture of another automated evaluation method for an internet of things platform.

1) Terminal dotting

And the APP, the SDK and the module perform log dotting on the operation and the network according to requirements and report. sdk collects local area network control operation, wide area network control operation, configuration binding operation dotting log and network environment log. The module mainly collects and configures binding dotting logs, control command dotting logs, network environment access dotting logs and connection platform stability dotting logs.

2) Log collection

And collecting an original operation log and a network environment log reported from the terminal, and collecting a user operation log reported from the APP. And preprocessing the collected logs. Records and attributes associated with the interconnection and interworking indicator are extracted from the existing log.

3) Interconnection and interworking standard configuration management

And a management interface is provided for the interconnection and intercommunication Portal standard configuration management module, and basic data and rules are provided for log processing and index calculation. The method mainly comprises the following steps: basic network classification standard management interface, index management interface and the like

4) Interconnection and interworking index calculation

Processing original service dotting logs and weblogs into standard interconnection logs; and outputting the interconnection standard log as a final interconnection index through a plurality of columns of statistics and calculation.

The interconnection standard log comprises sn, time, service, response code, processing market, network environment grade, api, devid and uid. Sn is a log global sequence number; time is log recording time; service, i.e. service related to interconnection and interworking (the service needs to be configurable, such as BIND, LAN _ OP, WAN _ OP, and ACC _ UWS, etc., for local area network operation, wide area network operation, and APP access); response code-response code of service request (refer to < interworking index evaluation specification >). Response time-request time; network environment level (the level needs to be configurable, refer to < interconnection index evaluation specification >); interconnection index (date, equipment classification, product type, index score)

5) Interconnection portal

The portal provides the following functions for different types of users: and providing interface maintenance interconnection standard parameter configuration for an index manager. And providing an interface for a UED team and a user operation team to register the interaction standard degree and the user satisfaction index score of each period. And providing an interface for the index assessment personnel to check the interconnection and intercommunication indexes of each period. And providing an interface for the IOT, the APP and the user center to view the relevant indexes of each period.

6) Basic data

Basic data related to the device model is provided for the computing module and the portal. A sustainable virtuous cycle assessment mechanism from acquisition-assessment-feedback-improvement-acquisition. The overall system flow. And dividing each module in the system architecture diagram, and carrying out data flow and input and output.

Through the embodiment, the invention aims to construct an interconnection index evaluation system, which can acquire and evaluate indexes of the Internet of things basic platform in all aspects of reliability and usability, and can feed back evaluation results to the basic platform in time as an important basis for platform improvement and upgrading. A sustainable, automated, comprehensive virtuous cycle assessment mechanism from acquisition-assessment-feedback-improvement-acquisition is formed.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

In this embodiment, an automated evaluation system of an internet of things platform is further provided, and fig. 11 is a block diagram of a structure of an automated evaluation system of an internet of things platform according to an embodiment of the present invention, as shown in fig. 11, the system includes:

the interconnection and interworking index evaluation platform 32, the terminal 34,

the interconnection and interworking index evaluation platform 32 is configured to obtain a dotting log for the internet of things platform, where the dotting log includes at least one of the following: original operation logs, network environment logs and user operation logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log, wherein the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade; and evaluating the Internet of things platform according to the interconnection index.

According to the method and the device, the dotting log which is acquired by the terminal and aims at the platform of the Internet of things is acquired, wherein the dotting log comprises at least one of the following logs: original operation logs, network environment logs and user operation logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log; the Internet of things platform is evaluated according to the interconnection index, the technical scheme is adopted, the problems that the Internet of things platform cannot be accurately evaluated in the related technology and the like are solved, and an accurate automatic evaluation mode of the Internet of things platform is provided.

Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.

Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:

s1, obtaining a dotting log which is acquired by a terminal and aims at the platform of the Internet of things, wherein the dotting log comprises at least one of the following logs: original operation logs, network environment logs and user operation logs;

s2, converting the dotting logs into interconnection and intercommunication logs according to a target index rule, and determining an interconnection and intercommunication index of the Internet of things platform according to the interconnection and intercommunication logs, wherein the interconnection and intercommunication logs at least comprise one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade;

and S3, evaluating the Internet of things platform according to the interconnection and interworking index.

Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.

Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.

Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:

s1, obtaining a dotting log for the Internet of things platform collected by the terminal, wherein the dotting log comprises at least one of the following: original operation logs, network environment logs and user operation logs;

s2, converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log, wherein the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade;

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.

It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An automatic assessment method for an Internet of things platform is characterized by comprising the following steps:

the method comprises the steps of obtaining a dotting log which is collected by a terminal and aims at an Internet of things platform, wherein the dotting log comprises at least one of the following logs: original operation logs, network environment logs;

converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log; wherein, the interconnection log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade; the interworking index includes one of: equipment classification, product model, index and index score; wherein the exponential score comprises: the score corresponding to each index and the total index score of all the indexes; the indicator includes at least one of: the success rate of local area network control operation, the success rate of wide area network control operation and the disconnection rate of equipment; determining the total index score of all the indexes by: acquiring each index and the weight corresponding to each index, and determining the total index score according to each index and the weight corresponding to each index;

evaluating the Internet of things platform according to the interconnection index;

wherein, the converting the dotting log into an interconnection log according to the target index rule includes: converting the dotting log into an interconnection log according to the network quality grade corresponding to the dotting log and the interconnection log standard;

wherein, the determining the interconnection index of the internet of things platform according to the interconnection log comprises:

the following parameter information is obtained: the method comprises the steps that indexes set by the Internet of things platform, standard values of the indexes in different network environments, and index scores determined according to the standard values of the indexes and actual values measured by the indexes are determined, and the total score of the indexes is determined through all the indexes;

determining an interconnection index of the Internet of things platform according to the parameter information and the interconnection log;

wherein, the determining the interconnection index of the platform of the internet of things according to the parameter information and the interconnection log comprises:

acquiring the interconnection and intercommunication log and a product model library;

performing aggregation statistics on the interconnection and intercommunication logs and the product model library according to a target rule to obtain a processing result;

splitting the processing result into a target index and an actual value of the target index according to the parameter information;

and determining the interconnection index according to the mode of determining the total mark of the index in the parameter information and the actual values of the target index and the target index.

2. The method of claim 1, wherein obtaining a dotting log for an internet of things platform collected by a terminal comprises:

acquiring the network environment logs in the dotting logs according to the following network environment access rules: continuously ping the routers N times per minute, and recording the average time delay and the overtime times of the routers N times ping to determine local area network data; after the target module is connected with the access gateway, recording time delay and overtime times of the ping M-time main gateway by using heartbeat acquisition parameters or the operation of the ping M-time main gateway so as to determine local area network data; a ping parameter; and a ping result, wherein the ping result only comprises the ping result without timeout, M and N are both positive integers, and N is greater than 1.

3. The method of claim 1, wherein converting the dotting log into an interconnection log according to a target index rule, further comprising:

acquiring network quality levels corresponding to the following parameters: received signal strength, local area network connectivity, local area network round trip delay, wide area network connectivity, wide area network round trip delay, and interconnection and interworking log criteria.

4. The method of claim 1, wherein after evaluating the internet of things platform based on the score and the indicator total score, the method further comprises:

and under the condition that the score of any index is lower than a preset threshold value, sending indication information to the Internet of things platform so as to indicate the Internet of things platform to improve the corresponding score of the index.

5. An automated assessment system for an internet of things platform, comprising:

an interconnection index evaluation platform and a terminal, wherein,

the interconnection and interworking index evaluation platform is used for acquiring a dotting log which is acquired by a terminal and aims at an Internet of things platform, wherein the dotting log comprises at least one of the following: original operation logs, network environment logs; converting the dotting log into an interconnection log according to a target index rule, and determining an interconnection index of the Internet of things platform according to the interconnection log; evaluating the Internet of things platform according to the interconnection and interworking index, wherein the interconnection and interworking log at least comprises one of the following parameters: the log global sequence number, the log recording time, the service corresponding to interconnection and intercommunication, the response code of the service request, the processing time length and the network quality grade; the interworking index includes one of: equipment classification, product model, index and index score; wherein the exponential score comprises: the score corresponding to each index and the total index score of all the indexes; the indicator includes at least one of: the success rate of local area network control operation, the success rate of wide area network control operation and the disconnection rate of equipment; determining the total index score of all the indexes by: acquiring each index and the weight corresponding to each index, and determining the total index score according to each index and the weight corresponding to each index;

acquiring the interconnection and interworking log and a product model library;

6. The system of claim 5, wherein the interconnection and interworking index evaluation platform is further configured to obtain the cyber environment log in the dotting log according to the following cyber environment access rules: continuously ping the routers N times per minute, and recording the average time delay and the overtime times of the routers N times ping to determine local area network data; after the target module is connected with the access gateway, recording time delay and overtime times of the ping M-time main gateway by using heartbeat acquisition parameters or the operation of the ping M-time main gateway so as to determine local area network data; a ping parameter; and a ping result, wherein the ping result only comprises the ping result without timeout, M and N are both positive integers, and N is greater than 1.

7. The system of claim 6, wherein the interworking index evaluation platform is further configured to obtain network quality levels corresponding to the following parameters: receiving signal strength, local area network connectivity, local area network round trip delay, wide area network connectivity, wide area network round trip delay, and interconnection log standards; wherein, the interconnection log at least comprises one of the following parameters: a log global sequence number; log recording time; interconnecting and intercommunicating corresponding services; a response code of the service request; processing duration, the network quality level.

8. A computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to carry out the method of any one of claims 1 to 4 when executed.

9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is arranged to execute the computer program to perform the method of any of claims 1 to 4.