CN100531066C - Method and device for determining business parameter grade quantizing range of business service - Google Patents
Method and device for determining business parameter grade quantizing range of business service Download PDFInfo
- Publication number
- CN100531066C CN100531066C CNB2007100650743A CN200710065074A CN100531066C CN 100531066 C CN100531066 C CN 100531066C CN B2007100650743 A CNB2007100650743 A CN B2007100650743A CN 200710065074 A CN200710065074 A CN 200710065074A CN 100531066 C CN100531066 C CN 100531066C
- Authority
- CN
- China
- Prior art keywords
- service
- parameter
- business
- service parameter
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Telephonic Communication Services (AREA)
Abstract
The method includes steps: obtaining information of experience and evaluation of ratable business parameters (BP) for business service (BS) from users; according to information of experience and evaluation from users to determine weight of related BP, in order to determine adjusting step size for BP; using the adjusting step size for BP to carry out multiple times of adjustment for each BP value of BS on each service grade; recording adjusting step size for BP, adjusting number, as well as users' evaluated values on adjusted BS in order to determine graded quantizing ranges of BP. The invention also discloses device for determining graded quantizing ranges of BP for BS. Through determining users' sensitive degrees of each BP of BS, the invention sets up corresponding adjusting step size so as to adjust BP accurately, and further to determine quantizing ranges of each BP accurately.
Description
Technical field
The present invention relates to technical field of the computer network, specifically, relate to telecommunication service and carry out the method and apparatus of determining the business parameter grade quantizing range of business service in the process.
Background technology
In conventional telecommunications operation process, mainly adopt with the technology in the communication products design is that the management thought that leads is carried out the construction of basic telecom service, to satisfy the common requirements of people to communication, yet, fierceness day by day along with telecommunication market competition, huge variation is also taking place in the operation theory of telecom operators, and the day by day saturated trend in basic telecommunication market impels operator to restart to think deeply relation between user, market and the operation.In order to enlarge market share, the telecommunication product of user oriented different qualities is provided, carry out differential competition and will become the important channel that promotes self competitiveness, whether can have more user resources will become the key factor that operator obtains the market advantage, and following telecom operation must will with user be tissue and the operation that guiding is carried out telecommunication service.
In order to satisfy the demand of user to greatest extent to service quality, service level agreement SLA (Service-Level Agreement) is suggested, be used to build the biological environment that network operation develops in a healthy way, what allow the user enjoys is not only a kind of service of oral commitment, but being subjected to regulations, rights and interests obtain the effectively service of guarantee.Ripe telecom operators can set up loyal stable large user by SLA; Emerging telecom operators then can attract the user whereby, win competitive advantage.
In order to realize the smooth implementation of the end-to-end operation of SLA, satisfy the requirement of accurate network monitoring simultaneously, each normal structure begins SLA is carried out deep research one after another, especially the emphasis studied especially of method for expressing among the SLA and management method, the demand and the importance of SLA method for expressing have been proposed in the ietf draft, studied the partial content service grade standard (SLS of SLA, service levelspecification) method for expressing, but select and measurement about the SLA parameter, contents such as key technologies such as each class parameter quantizing range of SLA method for expressing and SLA is definite and system applies do not have concrete regulation; TMF is organized among the standard GB917, SLA life cycle and two kinds of sla management methods of SLA parameter framework have been adopted, wherein the SLA life cycle is used to plan a complete SLA service management flow, SLA parameter framework is used to describe the sorting technique of expression parameter, but this standard does not propose the formulation flow process of concrete service parameter; GB923 has proposed the notion of Key Performance Indicator KPI (Key PerformanceIndicator) and Key Quality Indicator KQI (Key Quality Indicator) SLA has been carried out quantization means, but this standard has only rested on the parameter-definition stage to the research of SLA parameter, does not reach the level of quantitative management.
When each normal structure begins that one after another SLA carried out deep research, also be subjected to the attention of each research organization gradually based on the relevant issues of the telecommunication service proficiency assessment of user experience, each International Standards Organization has all proposed the standards and norms of oneself, wherein, P.800, the ITU-T standard defines and adopts a kind of (MOS) rule of on average appraising to measure voice quality, program by utilization MOS assessment is used as the standard that subjective assessment is measured, based on this subjectivity evaluation and test, the mankind answer and are investigated with the behavior of perceptual speech quality and quantize, answer the voice of which kind of rank quality, obtain how many average subjective value MOS, the mankind will play main reflection effect.The corresponding relation of this voice quality and average subjective value provides the standard foundation for configuration, benchmark and the supervision of network.In addition, the subjective evaluating method of video traffic also comes into one's own gradually, ITU-R BT.500 and ITU-T P.910 in the video subjective estimate method, under controlled test environment, image sequence to be evaluated is played to the estimator to be watched, and the marking of writing down them, then all estimators' marking is added up, use (MOS) rule of on average appraising to come measuring video quality.
Comprehensive above the analysis, though the theoretical research of SLA is carried out, but realize the effective means that the SLA parameter quantification formulates-based on the application of the real-time experiential method of user, still adopt qualitative analysis based on subjective experience, can not embody the ambiguity of user in the QoS cognition, can't carry out quantization means to the evaluation information of QoS to the user, therefore can't realize the coupling of Internet resources adjustment distribution, QoS and user's request.
Summary of the invention
In view of this, the object of the present invention is to provide a kind of method and apparatus of business parameter grade quantizing range of definite business service, according to the ambiguity of user in the QoS cognition, the user is accurately quantized the evaluation information of QoS.
The invention provides a kind of method of business parameter grade quantizing range of definite business service, comprise step:
But obtain the experience evaluation information of user to the evaluation assignment parameter of business service;
Determine the weight of related service parameter according to described information of experience and evaluation from users, adjust step-length to determine service parameter;
Utilizing described service parameter to adjust step-length repeatedly adjusts on each grade of service every service parameter value of business service;
Record traffic parameter adjustment step-length, adjustment number of times and user determine the grade quantizing range of described service parameter to the evaluation of estimate of each adjusted business service.
Preferably, determine the weight of service parameter according to following steps:
Determine the relative Link Importance information of every service parameter of described business service according to described experience evaluation information:
Described relative Link Importance information scale is become Triangular Fuzzy Number;
The fuzzy matrix that structure is made of described Triangular Fuzzy Number;
Calculate described fuzzy matrix, determine the weight of every service parameter according to result of calculation.
Preferably, determine the adjustment step-length of service parameter according to following steps:
Determine the priority of described service parameter according to the size of described weight;
Determine and described priority corresponding business parameter adjustment step-length.
Preferably, described business service is the business service of service parameter numerical value in default service parameter scope, the adjusting range of described default service parameter scope for determining according to network index standard and actual tests result.
Preferably, described experience evaluation information is the information of the relative Link Importance of any two parameters in the described business service of indication.
The device of the business parameter grade quantizing range of a kind of definite business service of the present invention comprises:
Service info element is used to preserve business service information, adjusts the service parameter of described business service, and business service is presented to the user, and obtains the experience evaluation information of user to business service;
Service parameter weight determining unit is used for determining according to described experience evaluation information the weight of the service parameter of described business service;
Adjust the step-length determining unit, be used for determining the adjustment step-length of each service parameter according to described weight;
The grade quantizing range determining unit, be used for the service parameter of described service info element being adjusted according to described adjustment step-length, and record adjusts number of times, adjusts step-length and each back evaluation of user value of adjusting, and determines the grade quantizing range of described business service.
Preferably, described service info element comprises:
The information on services memory cell is used to store the miscellaneous service service;
The information interaction unit is used for the business service of information on services memory cell is presented to the user, and obtains user's experience evaluation information;
First adjustment unit is used in default adjusting range the service parameter of the business service of described information on services memory cell is adjusted;
Second adjustment unit is used for the adjustment step-length of described adjustment step-length determining unit the service parameter of the business service in the described information on services memory cell being adjusted.
Preferably, described service parameter weight determining unit comprises:
The importance degree determining unit is used for determining according to described experience evaluation information the relative Link Importance information of every service parameter of described business service;
The scale unit becomes Triangular Fuzzy Number with described relative Link Importance information scale;
Matrix unit, the fuzzy matrix that structure is made of described Triangular Fuzzy Number;
Computing unit utilizes described fuzzy matrix to calculate the weight of each service parameter.
Preferably, described grade quantizing range determining unit comprises:
Record cell is used to write down the adjustment step-length of described second adjustment unit and adjusts number of times, and adjusts back evaluation of user value;
The distribution situation determining unit is used for determining the distribution situation of adjusted service parameter value in each grade of service according to evaluation of user value after described adjustment step-length and adjustment number of times and the adjustment;
The quantizing range determining unit according to the indication of described distribution situation, is determined the grade quantizing range of described service parameter.
Preferably, described business service is the business service of service parameter numerical value in default service parameter scope, the adjusting range of described default service parameter scope for determining according to network index standard and actual tests result.
By technique scheme as can be known, compared with prior art, the present invention has following characteristics and advantage:
1, the user is quantized the subjective perception of QoS cognition, determined the quantizing range of QoS grade, thereby provide foundation for the coupling that realizes Resource Allocation in Networks, QoS and user's request;
2, determine the sensitivity of user according to evaluation of user information,, thereby realize accurate adjustment, and then accurately determine the quantizing range of each service parameter service parameter so that corresponding adjustment step-length is set to every service parameter of business service.
Description of drawings
Fig. 1 determines the realization flow figure of the method for business service grade parameter quantification scope for the present invention;
Fig. 2 determines to determine to adjust in the realization flow of method of business service grade parameter quantification scope the flow chart of step-length for the present invention;
Fig. 3 determines the structured flowchart of the device of business service grade parameter quantification scope for the present invention;
Fig. 4 is the structural representation of the service info element of the embodiment of the invention;
Fig. 5 is the structural representation of the service parameter weight determining unit of the embodiment of the invention;
Fig. 6 is the structural representation of the grade quantizing range determining unit of the embodiment of the invention.
Embodiment
In the objective world, exist a large amount of bloomings.Each subject, mathematicization, the quantification trend of especially humane, social studies and other " soft science " are pushed the Mathematical treatment problem of ambiguity to middle cardiac status.The more important thing is,, make and calculate the functional image human brain and like that complexity is had recognition capability along with developing rapidly of electronic computer, cybernetics, systematic science, just must research and handle ambiguity.We study human system's behavior, perhaps handle can be comparable with human system's behavior complication system, as aerospace system, people's brain system, social system etc., parameter and variable are a lot of, various factors is interlaced, and system very complex, its ambiguity are also clearly.From the understanding aspect, ambiguity is meant the uncertainty of notion extension, thereby causes the uncertainty of judgement.In daily life, often run into many fuzzy things, do not have clearly demarcated quantitative limits, use some fuzzy words and phrases to describe, describe.Such as, compare youth, tall person, big fatty, good, beautiful, kind, hot, far away ....In the people's work experience, many fuzzy things are often arranged also.For example, determine whether one heat steel water refines, except the precise informations such as temperature, component ratio and the duration of heat that will know molten steel, also need with reference to fuzzy informations such as molten steel color, boiling situations.Therefore, except the computational mathematics that relates to error is arranged very early, also need fuzzy mathematics.The people compares with computer, and in general, human brain has the ability of handling fuzzy information, is a good judge of and handles blooming.But computer is relatively poor to the blooming recognition capability, in order to improve the ability of computer Recognition blooming, just the fuzzy language that people use always need be designed to receptible instruction of machine and program, so that machine can be as human brain the succinct corresponding judgment of making flexibly, thereby improve the efficient of identification automatically and control blooming.Like this, just need to seek a kind of mathematical tool of describing and processing fuzzy information, this just promotes the mathematician and furthers investigate fuzzy mathematics.
Analytic hierarchy process (AHP) (AHP) is that the U.S. scholar Thomas L.Satty that plans strategies for teaches the multi-scheme or the multiobject decision-making technique of a kind of practicality that proposes twentieth century seventies.Its principal character is, it reasonably combines the decision-making of qualitative and quantitative, according to the rule of thinking, psychology decision process stratification, quantification.Analytic hierarchy process (AHP) is a kind of comprehensive system thinking in back that decomposes earlier, by arrangement and comprehensive people's subjective judgement, qualitative analysis and quantitative analysis is organically combined, and realizes the quantification decision-making.At first with the problem stratification that will analyze, according to the character of problem and the general objective that will reach, PROBLEM DECOMPOSITION is become different compositing factors, according to correlation between factor and membership, factor is assembled combination by different levels, form a multi-layer analysis structural model, finally be summed up as lowermost layer (scheme, measure, index etc.) with respect to the weights of top (general objective) relative significance level or the problem of good and bad relatively order.For example, someone prepares to choose a television set, after he understands 7 kinds of dissimilar television sets on the market, which product decision buys, the process of decision-making often is not directly to compare, because there are many incomparable factors, but index is investigated in the middle of choosing some.8 standards such as the size of television set, weight, function, price, outward appearance, ease for operation, prestige, after-sale service for example.And then consider that the quality of various model television sets under above-mentioned each intermediate standard sorts.By this ordering, finally make and choose decision-making.When decision-making, because these several television sets generally are inconsistent for the quality ordering of each intermediate standard, therefore, the policymaker at first will do an estimation to the importance degree of these 8 standards, provide a kind of ordering, then 7 kinds of television sets are found out the ordering weight of each standard respectively, comprehensive these information datas at last, obtain promptly buying the ordering weight of television set at general objective.This weight vectors has been arranged, and decision-making just has been easy to.
This method also provides strong mathematics support for the more rules decision problem in the various engineering practical application, need use fuzzy level parser (Fuzzy-AHP) in the implementation process of the present invention, be on the basis of the Comprehensive Evaluation in fuzzy mathematics, on the structure of weight sets, use the fuzzy consensus matrix introduced based on the thought of analytic hierarchy process (AHP) to business service in a kind of method of estimating of the weight degree of miscellaneous service parameter.It is a kind of the non-quantitation incident to be made the short-cut method of quantitative analysis, also is a kind of means of people's subjective judgement being done objective description.The flexible strategy of utilizing Fuzzy AHP to calculate evaluation index are distributed, and compare with Comprehensive Evaluation and can reduce subjective factor effectively.This is that a kind of qualitative and quantitative combines, the evaluation method that the synthesization degree is higher.Fuzzy AHP (Fuzzy-AHP) is the ambiguity of judgment matrix with the difference of common analytic hierarchy process (AHP), it has simplified the complexity that people judge the target relative importance, realize making a strategic decision by qualitative to quantitatively conversion easily and efficiently by fuzzy judgment matrix, directly, the consistency problem of judgement is resolved by fuzzy judgment matrix structure fuzzy consensus judgment matrix.
Because the user has ambiguity widely to the experience of business service, this class is not affirmed and coarse phenomenon, usually adopt the rule that is subordinate in the fuzzy theory, indeterminacy phenomenon is carried out comparatively objectively quantity description.Employing AHP method is estimated the weight degree of miscellaneous service parameter relevant with user experience in the business service and is ranked according to priority orders, in the calculating of carrying out the service parameter weighted value, the dimension numerical value that the utilization fuzzy mathematics method carries out index transforms.
Basic thought of the present invention is: obtain the user to adjusting the experience evaluation information of service parameter business service afterwards, determine the adjustment step-length of service parameter according to described experience evaluation information, utilize this adjustment step-length that service parameter is adjusted, and determine the grade quantizing range of described service parameter according to the evaluation of user value of obtaining.In order to enable those skilled in the art to fully understand technical scheme of the present invention, further describe in detail below in conjunction with specific embodiments and the drawings.
Please refer to Fig. 1, determine the realization flow figure of the method for business service grade parameter quantification scope for the present invention.
Step S101: but the experience evaluation information of user obtained to described evaluation assignment parameter.
Step S102: determine the weight of related service parameter according to described information of experience and evaluation from users, adjust step-length to determine service parameter.
Step S103: utilize described service parameter to adjust step-length every service parameter value of business service is repeatedly adjusted on each grade of service.
Step S104: record traffic parameter adjustment step-length, adjustment number of times and user determine the grade quantizing range of described service parameter to the evaluation of estimate of adjusted business service.
Record traffic parameter adjustment step-length, adjustment number of times and user determine the distribution situation of service parameter value in each grade of service, and then determine the grade quantizing range of described service parameter according to described distribution situation the evaluation of estimate of adjusted business service.
Described experience evaluation information can be the user to the experience relative importance evaluation between the every service parameter in the described business service, also can be only the service quality of the business service experienced be made evaluation.
With the streaming media service is example, Streaming Media is meant on the broadband and transmits, the communication service with integration, interactivity and synchronism of image, sound and literal can be provided, and this business can be applicable to aspects such as comprising video conference, long-distance education, video request program, tele-medicine and visual telephone service.
The service parameter of described streaming media service service comprises time delay, shake, link utilization, packet loss, throughput, the quality of the business service that is provided will be provided the specific targets of these service parameters, the user is according to subjective feeling, described 5 service parameters are compared in twos, determined relative Link Importance; In order to make evaluation more accord with human thinking state, adopt natural language to be described, described evaluation comprises: of equal importance, important a little, obviously important, strong important and extremely important.
Of equal importance, two elements are compared, and are of equal importance;
Important a little, two elements are compared, and an element is more important a little than another element;
Obviously important, two elements are compared, and an element is obviously more important than another element;
Important strongly, two elements are compared, and an element is much more important than another element;
Extremely important, two elements are compared, and an element is extremely more important than another element;
The user also can only make evaluation to the service quality of described business service, as estimating with the form of marking, adopts the service quality marking of ten point system to the business service experienced.
In above-mentioned realization flow, adjust definite accuracy that is concerning each business service parameter grade quantizing range of described business service of step-length.
Please refer to Fig. 2, determine to determine to adjust in the realization flow of method of business service grade parameter quantification scope the flow chart of step-length for the present invention.
Step S201: the relative Link Importance information of determining service parameter.
Determine the relative Link Importance information of service parameter according to user's experience evaluation information, of equal importance important in a little descriptions such as packet loss as time delay in throughput, time delay.
Step S202: scale Triangular Fuzzy Number.
The relative Link Importance information scale of described every service parameter is become Triangular Fuzzy Number.
If when the service parameter of described business service is relatively judged in twos, because the complexity people of things can't provide its importance scale with an accurate values, but with a Triangular Fuzzy Number a
Ij[l
Ij, m
Ij, u
Ij] represent its judgement.
Described a
IjBe one with m
IjBe the closed interval of intermediate value, described m
IjValue adopt 1,3,5,7,9 these several scales that evaluation information is carried out scale, 1,3,5,7,9 respectively the implications of representative be of equal importance, important a little, obviously important, strong important and extremely important.
Owing to be used for the QoS judge itself is existed ambiguity and subjectivity,, pass through l at this
IjAnd u
IjThe uncertainty of user's evaluation information is scientifically represented in the interval of forming, its interval scope is big more, the ambiguity of service parameter relative Link Importance information that the business service that the user is given is described is just big more, confidence level is just more little, otherwise, the interval more little fog-level of scope is just more little, and confidence level is then big more.
In described interval, choose m
IjThe possibility maximum; And l
IjOr u
IjThe value span be 1～9, and satisfy l
Ij＜m
Ij＜u
Ijl
Ij, u
IjValue provide by the user equally, the fog-level that the user evaluates and tests information has determined l
Ij, u
IjSpan; In the ordinary course of things, l
Ij, u
IjThe value step-length is 1, i.e. l
Ij=m
Ij-1＜m
Ij＜m
Ij+ 1=u
Ij
Step S203: at the experience evaluation information of a plurality of service parameters, the structure fuzzy matrix.
The relative Link Importance information scale of described every service parameter is become after the Triangular Fuzzy Number, become the relative Link Importance information of the service parameter of Triangular Fuzzy Number according to scale, for the experience evaluation information of unique user at a plurality of service parameters, the structure fuzzy matrix is as follows:
a
11 a
12 a
13...a
1n
a
21 a
22 a
23...a
2n
a
31 a
32 a
33...a
2n
a
n1 a
n2 a
n3...a
nn.
a
IjExpression refers to that i element of this level has for example degree of membership of " ... compare ... of equal importance " of fuzzy relation to j element, and described matrix A is just reciprocal matrix, i.e. a
Ij=a
Ij -1=[1/l
Ij, 1/m
Ij, 1/u
Ij].
Step S204: the weighted value that calculates each service parameter.
This step has quoted the method for the degree analyzing integrated decision-making of Chinese scholar Chang Dayong proposition and has carried out weighted value calculating in calculating, and relevant mathematical derivation is based on the fundamental theorem of fuzzy mathematics.Relevant content can be with reference to author's correlation theory that chapter 6 joint (degree analyzing and integrated decision-making) is described in detail in the book of " fuzzy mathematics method in the economic management " (publishing house of Beijing Economics College, 1995).
With a
Ij t=[l
Ij t, m
Ij t, u
Ij t], I, j=1,2,3......n represents the Triangular Fuzzy Number according to t position evaluation of user information scale, according to formula:
Supply the comprehensive judgment matrix of all relevant parameters of business service to user's evaluation information.
Pass through formula again:
Calculate the weight of each service parameter of described business service, according to fuzzy foundation of mathematics theory:
V＝(S
i≥S
j)＝min[V(S
i≥S
1)，...，V(S
i≥S
j)，...，V(S
i≥S
n)]，j＝1，2，...，n，i≠j
D (A1)=V (S
i〉=S
j), then weight vectors ω=(d (A1), d (A2) ..., d (An)), be the weight of each service parameter of business service: ω=(d (A1), d (A2) ..., d (An)).
Step S205: the priority of determining each service parameter.
Determine the priority of each service parameter according to the size of described each service parameter weighted value, the size of service parameter weighted value has been represented the sensitivity of this service parameter of user, and weighted value is big more, and the expression user is high more to its sensitivity, weighted value is more little, represents that then the user is low more to its sensitivity; So weighted value is big more, priority is high more.
Step S206: determine to adjust step-length.
Height according to priority, corresponding adjustment step-length is set, numerical value for the higher service parameter of priority ratio, owing to be responsive more data, adopt less adjustment step-length to adjust, when adjusting, can adopt bigger adjustment step-length to adjust for the numerical value of the lower service parameter of priority.
Among the step S202 of the invention described above embodiment, utilize to convert fuzzy natural language to mathematical linguistics, reflected the ambiguity of user in the business service perception objectively, provide the foundation for quantizing QoS.
Being example with the streaming media service service below is explained in further detail the method for the business parameter grade quantizing range of a kind of definite business service of the present invention.
Streaming Media is meant on the broadband and transmits, the communication service with integration, interactivity and synchronism of image, sound and literal can be provided, and this business can be applicable to aspects such as comprising video conference, long-distance education, video request program, tele-medicine and visual telephone service.
The service parameter of described streaming media service service comprises time delay, throughput, shake, link utilization, packet loss, the quality of the business service that is provided will be provided the specific targets of these service parameters, experience evaluation information according to the user, described 5 service parameters are compared in twos, determined relative Link Importance.
Described relative Link Importance scale is become Triangular Fuzzy Number, 1,3,5,7,9 implications of representing respectively are of equal importance, important a little, obviously important, strong important and extremely important, when the user estimates the streaming media service quality, for example, it is as follows that user one provides the relation of importance degree between time delay and other parameters, time delay: throughput=(3,5,6), time delay: shake=(4,5,7), time delay: link utilization=(6,7,9), time delay: packet loss=(6,7,8), it is obviously important in throughput that the meaning of its representative is respectively time delay, and time delay is obviously important in shake, time delay is strong important in link utilization, and time delay is strong important in packet loss; Because the ambiguity of user's evaluation information has adopted the mode of fuzzy interval to come this ambiguity is further described in the formula; In like manner, the importance degree between other parameters represents also to have followed such rule, according to evaluation of user information, below will describe the fuzzy matrix of being constructed in detail.
According to the evaluation of the relative Link Importance between a pair of above service parameter of user, the fuzzy matrix of structure is as follows:
According to the evaluation of user's relative Link Importance between the service parameter more than two pairs, the fuzzy matrix of structure is as follows:
Pass through formula
Try to achieve comprehensive Triangular Fuzzy Number, all related service parameters of supported business service are to the comprehensive judgment matrix of information of experience and evaluation from users thus.
Pass through formula again:
Calculate the weight of each service parameter of described business service, according to fuzzy foundation of mathematics theory:
V＝(S
i≥S
j)＝min[V(S
i≥S
1)，...，V(S
i≥S
j)，...，V(S
i≥S
n)]，j＝1，2，...，n，i≠j
D (A1)=V (S
i〉=S
j), then weight vectors ω=(d (A1), d (A2) ..., d (An)), be the weight of each service parameter of business service (being followed successively by time delay, throughput, shake, link utilization): ω=(0.349,0.2305,0.2141,0.1692,0.0372).
According to the weighted value of each service parameter, with determining the priority of each service parameter, arrange from high to low according to priority, as follows:
Time delay, throughput, shake, link utilization, packet loss.
Promptly in described service parameter, the user is the most responsive to time delay, and is least responsive to packet loss.
With foregoing Streaming Media is example, every service parameter is according to being arranged as from high to low of priority: time delay, throughput, shake, link utilization, packet loss, then time delay is adjusted, service parameter values such as packet loss or link utilization are adjusted with bigger step-length with less adjustment step-length.
For example use the experience of streaming media service according to some users, it is as shown in table 1 to set the adjustment step-length that each service parameter is adopted:
Table 1: service parameter is adjusted step-length
| Service parameter | Time delay | Throughput | Shake | Link utilization | Packet loss |
| Adjust step-length | 10ms | 20kbps | 20ms | 20％ | 20％ |
The initial value of every service parameter of described streaming media service equals preset initial value, and described preset initial value is the minimum value or the greatest measure of the adjusting range determined according to network index standard and actual tests result, and wherein adjusting range is as shown in table 2:
Table 2: service parameter adjusting range
| Service parameter | Time delay | Throughput | Shake | Link utilization | Packet loss |
| Adjusting range | 50ms～2s | 20kbps～500kbps | 5ms～2s | 0％～100％ | 0％～50％ |
Maximum in the adjusting range that employing is preset or minimum value are initial value, service parameter to business service is configured, and is that 500kbps, link utilization are that initial configuration is carried out in the streaming media service service as described in 100% pair as adopting time delay=50ms, shake=5ms, packet loss=0%, throughput.
Utilize the adjustment step-length shown in the table 1 in the adjusting range shown in the table 2, described service parameter repeatedly to be adjusted, every adjustment once, obtain the user to adjusting the scoring that parameter quality of service is afterwards made, be evaluation of estimate, determine the grade quantizing range of described service parameter according to adjusted service parameter value in the distribution of each grade of service.
The user gives a mark to described business service with ten point system, be set in 9～10 fens scopes for outstanding, in 7～9 fens scopes for good, in 6～7 fens scopes is general, be poor below 6 minutes, recording user is to the scoring of the business service of each service parameter adjustment, utilize the adjustment step-length of record to determine the distribution situation of described service parameter in each grade with the adjustment number of times, and then determine the grade quantizing range of each service parameter according to described distribution situation, with the time delay is example, and following experimental result has write down the adjustment number of times and the user marks as shown in table 3:
Table 3
| Adjust number of times | The 1st time | The 2nd time | The 3rd time | ...... | The 11st time | ...... | The n time |
| User's scoring | 10 | 10 | 9.5 | ...... | 8.5 | ...... | 6 |
Be divided into outstanding grade according to setting 9～10, when scoring is lower than 9 timesharing, adjusts when time counting to the 11st time and stop to adjust, time delay is 50～(50ms+10 in the distribution situation of outstanding grade so
*10ms), promptly the number range of the time delay of outstanding grade is 50ms～150ms, determines the number range of the time delay of other grades with identical method.
Using such method is determined the experimental result record of the quantizing range of shake, packet loss, throughput and link utilization, and is as shown in table 4.
Table 4
Above-mentioned example is that the greatest measure from adjusting range constantly successively decreases and adjusts, and also can increase progressively from the minimum value of adjusting range and adjust.
Please refer to Fig. 3, be the structured flowchart of the device of a kind of definite business service grade parameter quantification scope of the present invention.This device comprises: service info element 101, service parameter weight determining unit 102, adjustment step-length determining unit 103 and grade quantizing range determining unit 104.
The course of work of this device is:
Described evaluation information can be the relative importance evaluation of user to the every service parameter in the described business service, also can only make evaluation to the service quality of described business service.
The adjusting range of described default adjusting range for determining according to network index standard and actual tests result.
Described service info element 101 can comprise: information on services memory cell 1011, information interaction unit 1012, first adjustment unit 1013 and second adjustment unit 1014, as shown in Figure 4.
Described information on services memory cell 1011 storage miscellaneous service services; The user is presented to the business service in the information on services memory cell 1011 in information interaction unit 1012, and obtains evaluation of user information; First adjustment unit 1013 is adjusted the service parameter of the business service in the described information on services memory cell in default adjusting range; Under the control of grade quantizing range determining unit 104, second adjustment unit 1014 is adjusted the service parameter of the business service in the described information on services memory cell 1011 with the adjustment step-length in the described adjustment step-length determining unit 103.
Be example with the streaming media service below, the course of work of this device is further described.
Streaming Media is meant on the broadband and transmits, the communication service with integration, interactivity and synchronism of image, sound and literal can be provided, and this business can be applicable to aspects such as comprising video conference, long-distance education, video request program, tele-medicine and visual telephone service.
The service parameter of described streaming media service service comprises time delay, shake, link utilization, packet loss, throughput, the quality of the business service that is provided will be provided the specific targets of these service parameters, the user is according to subjective feeling, described 5 service parameters are compared in twos, determined the relative Link Importance between the described service parameter; In order to make evaluation more accord with human thinking state, adopt natural language to be described, described evaluation comprises: of equal importance, important a little, obviously important, strong important and extremely important.
The initial value of every service parameter of described streaming media service equals preset initial value, described preset initial value is the minimum value or the greatest measure of default adjusting range, the adjusting range of described default adjusting range for determining according to network index standard and actual tests result.
Wherein adjusting range is as shown in table 5:
Table 5: adjusting range
| Service parameter | Time delay | Throughput | Shake | Link utilization | Packet loss |
| Adjusting range | 50ms～2s | 20kbps～ 500kbps | 5ms～2s | 0％～100％ | 0％～50％ |
To be initial value be configured the service parameter of business service for maximum in the adjusting range that employing is preset or minimum value, is that 500kbps, link utilization are that initial configuration is carried out in the streaming media service service as described in 100% pair as adopting time delay=50ms, shake=5ms, packet loss=0%, throughput.
The user also can only make evaluation to the service quality of described business service, as estimating with the form of marking, adopts the service quality marking of ten point system to the business service experienced.
The user will experience and present quality of service in each parameter value variation process, service parameter weight determining unit 102 recording users are to the experience evaluation information of each parameter value importance ranking, and determine the relative important information of described service parameter according to described experience evaluation information.If when the service parameter of described streaming media service is relatively judged in twos, because the complexity people of things can't provide its importance scale with an accurate values, the general natural language that adopts when certain business service is estimated is as important on an equal basis, important a little, obviously important, strong important and extremely important etc.
Service parameter weight determining unit 102 can comprise: importance degree determining unit 1021, scale unit 1022, matrix unit 1023, computing unit 1024.As shown in Figure 5, determine the structural representation of service parameter weight determining unit of embodiment of the device of business parameter grade quantizing range for the present invention.
Importance degree determining unit 1021 is determined the relative Link Importance information of every service parameter of described business service according to described experience evaluation information, and is important in a little throughput as time delay, and time delay is obviously important in shake etc.
Scale unit 1022 is with Triangular Fuzzy Number a
Ij[l
Ij, m
Ij, u
Ij] this relative Link Importance information is carried out scale, described a
IjBe one with m
IjBe the closed interval of intermediate value, described m
IjValue adopt 1,3,5,7,9 these several scales that evaluation information is carried out scale.The user is estimated: of equal importance, important a little, obviously important, strong important and extremely important scale becomes 1,3,5,7,9, owing to be used for the QoS judge itself is existed ambiguity and subjectivity.
At this, pass through l
IjAnd u
IjThe uncertainty of user's evaluation information is scientifically represented in the interval of forming, its interval scope is big more, the ambiguity of service parameter relative Link Importance information that the business service that the user is given is described is just big more, confidence level is just more little, otherwise the interval more little fog-level of scope is just more little, and confidence level is then big more, in described interval, choose m
IjThe possibility maximum; And l
IjOr u
IjValue can be between 1～9 any one, satisfy l
Ij＜m
Ij＜u
Ij
Scale unit 1022 becomes Triangular Fuzzy Number with described relative Link Importance scale, as time delay: throughput=(3,5,6), time delay: shake=(4,5,7), time delay: link utilization=(6,7,9), time delay: packet loss=(6,7,8) or the like.
Matrix unit 1023 is according to the evaluation of the relative Link Importance between a pair of above service parameter of user, and the fuzzy matrix of structure is as follows:
Matrix unit 1023 is according to the evaluation of user's relative Link Importance between the service parameter more than two pairs, and the fuzzy matrix of structure is as follows:
Matrix unit 1023 is passed through formula
Try to achieve comprehensive Triangular Fuzzy Number, obtain the comprehensive judgment matrix of all parameters of business service thus user's evaluation information.
Computing unit 1024 passes through formula:
Calculate the weight that each service parameter of business service is counted by institute, according to fuzzy foundation of mathematics theory:
V＝(S
i≥S
j)＝min[V(S
i≥S
1)，...，V(S
i≥S
j)，...，V(S
i≥S
n)]，j＝1，2，..，n，i≠j
D (A1)=V (S
i〉=S
j), then weight vectors ω=(d (A1), d (A2) ..., d (An)), be the weight of each service parameter of business service: ω=(0.349,0.2305,0.2141,0.1692,0.0372).
Described service parameter is adjusted step-length determining unit 103 and determined the adjustment step-length of each service parameter according to the size of described weight, determines the priority of each service parameter according to the size of weighted value, arranges from high to low according to priority, and is as follows:
Time delay, throughput, shake, link utilization, packet loss.
Promptly in described service parameter, the user is the most responsive to time delay, and is least responsive to packet loss.
Adjust step-length determining unit 103 and determine the adjustment step-length of each service parameter according to described evaluation information, as shown in table 6:
Table 6: adjust step-length
| Service parameter | Time delay | Throughput | Shake | Link utilization | Packet loss |
| Adjust step-length | 20ms | 20kbps | 20ms | 20％ | 20％ |
Under the control of grade quantizing range determining unit 104, second adjustment unit 1014 in the described service info element 101 is adjusted the corresponding business parameter according to described adjustment step-length, the adjustment number of times of grade quantizing range determining unit 104 record traffic parameter adjustment step-lengths, service parameter, determine the distribution situation of adjusted service parameter value according to described adjustment step-length and adjustment number of times in each grade of service, and, determine the grade quantizing range of described service parameter according to the indication of described distribution situation.
The initial value of streaming media service parameter is that time delay=50ms, shake=5ms, packet loss=0%, throughput are that 500kbps, link utilization are 100%.
Table 7
| Adjust number of times | The 1st time | The 2nd time | The 3rd time | The 4th | The 5th | ... | The n time |
| User's scoring | 10 | 9.5 | 9.0 | 8.5 | 8 | .... | 7 |
As known from Table 7, delay parameter is distributed as 50ms～(50ms+3 in outstanding grade
*20ms), promptly 50ms～110ms is the outstanding grade of " time delay ", and wherein 50ms is an initial value, the 3rd, and the adjustment number of times of outstanding grade, 20ms are to adjust step-length.Draw the number range of the time delay of other grades with identical method.
Using such method is determined the quantizing range of shake, packet loss, throughput and link utilization, and is as shown in table 8.
Table 8
In addition, also the initial value of the service parameter of described streaming media service can be made as time delay=2s, shake=2s, packet loss=50%, throughput is that 20kbps, link utilization are 0%, second adjustment unit 104 carries out the decrement adjustment according to the adjustment step-length of each service parameter shown in the table 6 to each service parameter, the step-length number of times adjusted in grade quantizing range determining unit 104 records, adjust number of times and user's score information, as shown in table 9:
Table 9
| Adjust number of times | The 1st time | The 2nd time | The 3rd time | ...... | The 51st time | ...... | The n time |
| User's marking | 2 | 2 | 3 | ...... | 6 | ...... | 10 |
As known from Table 9, delay parameter is distributed as 2s～(2s-50 in the grade of difference
*20ms), promptly the grade of the difference of " time delay " can be quantified as 2s～(2s-50
*20ms); Using such method is determined the quantizing range of shake, packet loss, throughput and link utilization.
In the foregoing description, described grade quantizing range determining unit 104 can comprise: record cell 1041, distribution situation determining unit 1042 and quantizing range determining unit 1043, as shown in Figure 6.
User's evaluation of estimate that the adjustment step-length of the described service info element 101 of record cell 1041 records and adjustment number of times and service info element 101 provide; Distribution situation determining unit 1042 is determined the distribution situation of adjusted service parameter value in each grade of service according to the indication of described adjustment step-length, adjustment number of times and user's evaluation of estimate; Quantizing range determining unit 1043 is determined the grade quantizing range of described service parameter according to the indication of described distribution situation.
Develop from following Communications Market, the purpose that promotes the customer service quality is client's attentiveness will be transferred to the experience of client to service quality from aspects such as product, prices, by high-quality customer service client and enterprise are closely connected together, thereby set up unique, long-term competitive advantage.Yet the research user adjusts technological means and the realization of optimizing QoS by the experience evaluation to QoS, goes back shorter mention at present in the operation support system framework of operator.The development of the Communications Market that with user is guiding has been complied with in the SLA parametric representation method research that user oriented is experienced, actual satisfy the customer service demand the basis on can deep enough excavation all kinds of crowds' business demand characteristics and consumption feature, thereby can realize " distribution according to need " that quality of service provides.The quantization means of carrying out the SLA parameter by this method can reflect really that the user uses professional feature, meet the user to the service personalization demand, make the SLA parameter of being formulated be close to the market more, improved the success rate of Service Market operation, reduced business risk, for marketing provides reliable foundation, remedy the blank of operator simultaneously in this field.
Utilize the present invention the user can be quantized the subjective perception of QoS cognition, determined the quantizing range of QoS grade, realize the coupling of Resource Allocation in Networks, QoS and user's request.
More than disclosed only be preferred implementation of the present invention; but the present invention is not limited thereto; any those skilled in the art can think do not have a creationary variation, and, all should drop in protection scope of the present invention not breaking away from some improvements and modifications of being done under the principle of the invention prerequisite.
Claims (10)
1, a kind of method of business parameter grade quantizing range of definite business service is characterized in that, comprises step:
But obtain the experience evaluation information of user to the evaluation assignment parameter of business service;
Determine the weight of related service parameter according to described information of experience and evaluation from users, adjust step-length to determine service parameter;
Utilizing described service parameter to adjust step-length repeatedly adjusts on each grade of service every service parameter value of business service;
Record traffic parameter adjustment step-length, adjustment number of times and user determine the grade quantizing range of described service parameter to the evaluation of estimate of each adjusted business service.
2, the method for claim 1 is characterized in that, determines the weight of service parameter according to following steps:
Determine the relative Link Importance information of every service parameter of described business service according to described experience evaluation information:
Described relative Link Importance information scale is become Triangular Fuzzy Number;
The fuzzy matrix that structure is made of described Triangular Fuzzy Number;
Calculate described fuzzy matrix, determine the weight of every service parameter according to result of calculation.
3, method as claimed in claim 2 is characterized in that, determines the adjustment step-length of service parameter according to following steps:
Determine the priority of described service parameter according to the size of described weight;
Determine and described priority corresponding business parameter adjustment step-length.
4, as claim 1 or 2 or 3 described methods, it is characterized in that, described business service is the business service of service parameter numerical value in default service parameter scope, the adjusting range of described default service parameter scope for determining according to network index standard and actual tests result.
5, method as claimed in claim 4 is characterized in that, described experience evaluation information is the information of the relative Link Importance of any two parameters in the described business service of indication.
6, a kind of device of business parameter grade quantizing range of definite business service is characterized in that, comprising:
Service info element is used to preserve business service information, adjusts the service parameter of described business service, and business service is presented to the user, and obtains the experience evaluation information of user to the business service parameter;
Service parameter weight determining unit is used for determining according to described experience evaluation information the weight of the service parameter of described business service;
Adjust the step-length determining unit, be used for determining the adjustment step-length of each service parameter according to described weight;
The grade quantizing range determining unit, be used for the service parameter of described service info element being adjusted according to described adjustment step-length, and record adjusts number of times, adjusts step-length and each back evaluation of user value of adjusting, and determines the grade quantizing range of described business service.
7, device as claimed in claim 6 is characterized in that, described service info element comprises:
The information on services memory cell is used to store the miscellaneous service service;
The information interaction unit is used for the business service of information on services memory cell is presented to the user, and obtains user's experience evaluation information;
First adjustment unit is used in default adjusting range the service parameter of the business service of described information on services memory cell is adjusted;
Second adjustment unit is used for the adjustment step-length of described adjustment step-length determining unit the service parameter of the business service in the described information on services memory cell being adjusted.
8, device as claimed in claim 6 is characterized in that, described service parameter weight determining unit comprises:
The importance degree determining unit is used for determining according to described experience evaluation information the relative Link Importance information of every service parameter of described business service;
The scale unit becomes Triangular Fuzzy Number with described relative Link Importance information scale;
Matrix unit, the fuzzy matrix that structure is made of described Triangular Fuzzy Number;
Computing unit utilizes described fuzzy matrix to calculate the weight of each service parameter.
9, device as claimed in claim 7 is characterized in that, described grade quantizing range determining unit comprises:
Record cell is used to write down the adjustment step-length of described second adjustment unit and adjusts number of times, and each back evaluation of user value of adjusting;
The distribution situation determining unit is used for determining the distribution situation of adjusted service parameter value in each grade of service according to evaluation of user value after described adjustment step-length and adjustment number of times and the adjustment;
The quantizing range determining unit according to the indication of described distribution situation, is determined the grade quantizing range of described service parameter.
10, the device of the business parameter grade quantizing range of definite business service as claimed in claim 9, it is characterized in that, described business service is the business service of service parameter numerical value in default service parameter scope, the adjusting range of described default service parameter scope for determining according to network index standard and actual tests result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007100650743A CN100531066C (en) | 2007-04-02 | 2007-04-02 | Method and device for determining business parameter grade quantizing range of business service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007100650743A CN100531066C (en) | 2007-04-02 | 2007-04-02 | Method and device for determining business parameter grade quantizing range of business service |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101051942A CN101051942A (en) | 2007-10-10 |
| CN100531066C true CN100531066C (en) | 2009-08-19 |
Family
ID=38783152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2007100650743A Active CN100531066C (en) | 2007-04-02 | 2007-04-02 | Method and device for determining business parameter grade quantizing range of business service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100531066C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200364726A1 (en) * | 2014-10-31 | 2020-11-19 | Aeris Communications, Inc. | Automatic connected vehicle enrollment |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102629296A (en) * | 2012-02-29 | 2012-08-08 | 浙江工商大学 | Enterprise credit evaluation method based on gray fuzzy |
| CN103455510B (en) * | 2012-05-31 | 2017-04-12 | 飞拓无限信息技术（北京）有限公司 | Method and device for evaluating user |
| CN103906145B (en) * | 2012-12-25 | 2018-01-30 | 中国移动通信集团四川有限公司 | A kind of SLA threshold generation method and devices of speech business |
| CN103259784A (en) * | 2013-04-11 | 2013-08-21 | 苏州阔地网络科技有限公司 | Method and system for processing educational information |
| CN104517613A (en) * | 2013-09-30 | 2015-04-15 | 华为技术有限公司 | Method and device for evaluating speech quality |
| CN104331332B (en) * | 2014-11-04 | 2017-07-21 | 浪潮电子信息产业股份有限公司 | Virtual resource pre-allocation method based on SLA |
| CN107358346B (en) * | 2017-07-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Evaluation information processing method and device for communication quality |
| CN112990425A (en) * | 2019-12-18 | 2021-06-18 | 中国移动通信集团浙江有限公司 | Automatic classification method of 5G network slices, device thereof, electronic equipment and computer storage medium |
| CN113595775A (en) * | 2021-07-19 | 2021-11-02 | 中国联合网络通信集团有限公司 | Service level agreement mapping method, electronic device, and storage medium |
| CN113643810A (en) * | 2021-08-10 | 2021-11-12 | 西北工业大学 | Cloud edge-end cooperative man-machine object QoS parameter reduction method |
| CN114019449B (en) * | 2022-01-10 | 2022-04-19 | 南京理工大学 | Signal source direction-of-arrival estimation method, signal source direction-of-arrival estimation device, electronic device, and storage medium |
| CN115022660B (en) * | 2022-06-01 | 2024-03-19 | 上海哔哩哔哩科技有限公司 | Parameter configuration method and system for content distribution network |
-
2007
- 2007-04-02 CN CNB2007100650743A patent/CN100531066C/en active Active
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200364726A1 (en) * | 2014-10-31 | 2020-11-19 | Aeris Communications, Inc. | Automatic connected vehicle enrollment |
| US11687947B2 (en) * | 2014-10-31 | 2023-06-27 | Aeris Communications, Inc. | Automatic connected vehicle enrollment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101051942A (en) | 2007-10-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100531066C (en) | Method and device for determining business parameter grade quantizing range of business service | |
| Geisler | An integrated cost-performance model of research and development evaluation | |
| CN105207821B (en) | A kind of network synthesis performance estimating method of service-oriented | |
| US8082168B1 (en) | Method for statistical comparison of occupations by skill sets and other relevant attributes | |
| CN104618924B (en) | User experience quality index system and measurement method based on wireless ubiquitous network | |
| CN105574685A (en) | Subjective and objective combination-based cloud service evaluation method | |
| CN103179592A (en) | QoE (Quality of Experience) comprehensive evaluation method based on hierarchical tree structure | |
| CN109636467A (en) | A kind of comprehensive estimation method and system of the internet digital asset of brand | |
| CN106127613A (en) | Power consumer demand response potential evaluation method under a kind of internet environment | |
| Huang | Evaluating intelligent residential communities using multi-strategic weighting method in China | |
| CN109636184B (en) | Method and system for evaluating account assets of brands | |
| CN117592873A (en) | Data asset value assessment method based on data characteristics | |
| Maricic et al. | Measuring the ict development: the fusion of biased and objective approach | |
| CN107886223A (en) | Civil case complexity index evaluation method and system | |
| Abu-Assab | Integration of preference analysis methods into QFD for elderly people | |
| CN109615183A (en) | A kind of online education industry user experiences health degree assessment system and its application method | |
| Gao et al. | Bidding strategy with forecast technology based on support vector machine in the electricity market | |
| Dhurkari | MCGL: a new reference dependent MCDM method | |
| CN109377110B (en) | Evaluation method and system for brand content assets | |
| Vidiasova et al. | Social efficiency of E-participation portals in Russia: assessment methodology | |
| Mokallaf Sarband et al. | Adaptation of a compromise programming approach for evaluating the localized impacts of water allocation | |
| CN112348565B (en) | Information processing method and device based on block chain | |
| CN110414824B (en) | Service management system based on community system | |
| Dura et al. | The use of ranking sampling method within marketing research | |
| Pandari | A New Framework for Sustainable Development Policymaking Based on Importance–Performance Analysis |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C56 | Change in the name or address of the patentee |
Owner name: BEIJING BRIGHT OCEAN XINTONG TECHNOLOGY CO., LTD. Free format text: FORMER NAME: BEIJING YIYANG XINTONG SOFTWARE INST. CO., LTD. |
|
| CP01 | Change in the name or title of a patent holder |
Address after: 100036 Beijing city Haidian District liangjiadian Huayu No. 130 building 11-12 Patentee after: Beijing BOCO Inter-Telecom Technology Co., Ltd. Address before: 100036 Beijing city Haidian District liangjiadian Huayu No. 130 building 11-12 Patentee before: Beijing Yiyang Xintong Software Inst. Co., Ltd. |