CN105512316A - Knowledge service system combining mobile terminal - Google Patents

Abstract

The invention discloses a knowledge service system combining a mobile terminal. The knowledge service system comprises a cloud service engine and the mobile terminal, wherein the cloud service engine comprises a knowledge acquisition module, a knowledge searching module, a knowledge recommendation module, a knowledge inference module, a visualization processing module, an image recognition module, a knowledge database and an image feature library, wherein the modules, the database and the library are arranged in one server or distributively arranged in multiple servers which can perform mutual access through a high-speed network; the mobile terminal comprises an image acquisition module, an image feature extraction module and a user interface module. The invention further provides a knowledge service method based on the knowledge service system.

Description

A kind of Knowledge Service System in conjunction with mobile terminal

Technical field

The present invention relates to Computer Applied Technology and moving communicating field, relate in particular to a kind of Knowledge Service System in conjunction with mobile terminal.

Background technology

Knowledge services refines knowledge pointedly according to needing of people from various dominant and implicit knowledge resource, and with solving the information service process of advanced stage of customer problem.Its feature of this service is just, it is a kind of service towards knowledge content and solution.Knowledge services is different from traditional information service, knowledge services is not only the service of user-target driven, or towards the service of value-added service, knowledge services is paid close attention to and is emphasized the knowledge and the ability that utilize oneself uniqueness, the new information products with unique value are processed to form to ready-made information, for user solves the problem that other knowledge and ability cann't be solved.

1973, mobile phone invented by Martin storehouse handkerchief, then panel computer till now, and various mobile terminal becomes an indispensable part in our life.Modern mobile terminal possesses the parts such as central processing unit, storer, I/O on hardware; On software, mobile terminal has various special operating system; In communication capacity, mobile terminal is equipped with high-bandwidth communication performance and access way flexibly, makes mobile terminal according to selected business and residing environment, automatically can adjust communication mode.Meanwhile, mobile terminal focuses on the experience of multifunction, personalization and hommization more.Exactly because the development of mobile terminal is with universal, the various application program based on mobile terminal has become indispensable instrument in people's living and studying.Although the Knowledge Service System at present in conjunction with mobile terminal is fast-developing; But, these Knowledge Service Systems not unified standard in knowledge organization, existing defects, and mainly non-semantic based on keyword on knowledge calculates, what provide is only the knowledge services of shallow hierarchy.

Along with the development of computer technology, the performance of computing machine obtains large increase, but still there is the gully exchanged between the mankind with computing machine.The service better utilizing computing machine to allow user and provide, what we will do should not be allow user better adapt to computing machine, and should be allow computing machine adapt to the mankind, allow computing machine can understand the intention of user, for user provides more natural, clog-free service.And natural human-computer interaction technology is computing machine provides possibility with accessible interchange of the mankind.Except using word and computing machine to carry out alternately, we also should such as, by means of the intention of other interactive means by the computer understanding mankind, the natural interactive style of view-based access control model, sound, action, expression.

The present invention is by conjunction with concrete knowledge services demand, and the relation of analysis knowledge inside, becomes this build knowledge completely by knowledge organization; In knowledge reasoning, obtain user behavior, user's input by mobile terminal, and extract the information in user behavior, user's input, thus more significant the reasoning results; Knowledge based obtains the knowledge data base built, and Knowledge Service System provides the services such as knowledge search, knowledge recommendation, knowledge reasoning to user; Knowledge Service System compares the exhibition of knowledge mode traditional with traditional word, picture, video, hyperlink etc., system adopts the mode of information visualization, utilize the visual presentation modes such as basic statistical graph, label-cloud, mind map, the knowledge of containing in more clear to user, clear and definite presenting information; Meanwhile, by the combination of mobile terminal and various service, visualization technique, make Knowledge Service System more characteristic.

Mobile terminal, in conjunction with knowledge services, is not only user and provides more profound, personalized knowledge services, also allow user can more convenient, comprehensively obtain knowledge services, improve user and obtain and the experience of learning knowledge.

Summary of the invention

In conjunction with a Knowledge Service System for mobile terminal, comprise cloud service engine and mobile terminal, wherein:

Cloud service engine comprises:

Knowledge acquisition module, obtains knowledge by the mode of man-machine interaction, and organizes knowledge, manages and store in knowledge data base;

Knowledge search module, the recognition result of knowledge based database and picture recognition module, provides knowledge search service to user;

Knowledge recommendation module, knowledge based database and user's historical behavior, or the knowledge node that searches of knowledge based search module and user's historical behavior, knowledge recommendation service is provided to user, knowledge node is the elementary cell representing human knowledge, by the knowledge of the mankind is carried out cutting, to the data that the representation of knowledge segmented becomes computing machine to store and to identify, be knowledge node;

Knowledge reasoning module, the knowledge node that knowledge based search module searches and user's historical behavior, carry out reasoning with reference to regular collection, provide the reasoning results to user;

Visualization processing module, by visual for the output of at least one in knowledge search module, knowledge recommendation module and knowledge reasoning module, and sends visualization result to mobile terminal;

Picture recognition module, mates the characteristics of image received from mobile terminal with characteristics of image storehouse, to obtain recognition result;

Knowledge data base, stored knowledge; And

Characteristics of image storehouse, stores the characteristics of image of multiple image,

Wherein, above-mentioned each module and storehouse are deployed in a station server, or distribution is deployed in the multiple servers mutually can accessed by express network, and mobile terminal comprises:

Image collection module, obtains image;

Image characteristics extraction module, from image zooming-out feature; And

Subscriber interface module, realizes the man-machine interaction with user.

Present invention also offers a kind of knowledge services method based on this Knowledge Service System.

Accompanying drawing explanation

Fig. 1 is the system construction drawing of the present invention in conjunction with the Knowledge Service System of mobile terminal;

Fig. 2 is the knowledge services method flow diagram of the present invention in conjunction with the Knowledge Service System of mobile terminal;

Fig. 3 is the system flowchart of the present invention in conjunction with the Knowledge Service System of mobile terminal;

Fig. 4 is knowledge acquisition module structural drawing of the present invention;

Fig. 5 is knowledge search function structure chart of the present invention;

Fig. 6 is knowledge recommendation function structure chart of the present invention;

Fig. 7 is knowledge reasoning function structure chart of the present invention;

Fig. 8 is image collection module structural drawing of the present invention;

Fig. 9 is image characteristics extraction function structure chart of the present invention;

Figure 10 is picture recognition module structural drawing of the present invention;

Figure 11 a is the knowledge classification result figure in knowledge acquisition module of the present invention;

Figure 11 b is the knowledge cutting result figure in knowledge acquisition module of the present invention;

Figure 11 c is the partial knowledge node diagram inserted in the civil construction in knowledge acquisition module of the present invention;

Figure 11 d is the knowledge node attributes edit interface in knowledge acquisition module of the present invention;

Figure 11 e is the knowledge node attribute modification interface in knowledge acquisition module of the present invention;

Figure 11 f is the content of " Qin Shihuang Tomb " knowledge node in knowledge acquisition module of the present invention;

Figure 12 a is the result that in knowledge search module of the present invention, " Xuanzang " inquires about;

Figure 12 b is the result of " permanently happy " fuzzy query in knowledge search module of the present invention;

Figure 12 c is the result after the knowledge node imparting weight that in knowledge search module of the present invention, " Xuanzang " and " permanently happy " searches;

Figure 12 d is the net result that in knowledge search module of the present invention, " Xuanzang " and " permanently happy " searches for;

Figure 12 e is that in knowledge search module of the present invention, " Xuanzang " and " permanently happy " Search Results on mobile terminals visual presents result;

Figure 13 a is the historical behavior data of the user 1603 in knowledge recommendation module of the present invention;

Figure 13 b is the recommendation results to user 1603 in knowledge recommendation module of the present invention;

Figure 13 c is the knowledge node similar to knowledge node " Xuanzang " in knowledge recommendation module of the present invention;

Figure 13 d is the result after the knowledge node similar to knowledge node " Xuanzang " in knowledge recommendation module of the present invention sorts;

Figure 13 e is the visualization result of " Xuanzang " recommendation results in knowledge recommendation module of the present invention;

Figure 14 is the picture got in image collection module of the present invention;

Figure 15 is the picture feature the extracted point in image characteristics extraction module of the present invention;

Figure 16 is the example of the sample in characteristics of image storehouse in picture recognition module of the present invention;

Embodiment

How fully the present invention is the profound information of Extracting Knowledge for the technical matters solved, the degree of depth how utilizing Knowledge Visualization technology to realize knowledge is shown, and how the knowledge services based on semanteme is applied with mobile terminal and combine, finally be embodied as that users colony provides outstanding, knowledge services easily, the invention provides a kind of Knowledge Service System in conjunction with mobile terminal for this reason.

Technical scheme provided by the invention is as follows:

Knowledge Service System is made up of cloud service engine and mobile terminal, and each module comprised in system two-tier system is as follows:

One, knowledge acquisition module, by the mode of man-machine interaction obtain knowledge (knowledge comprise following at least one: text, picture, video and audio frequency), and adopt the institutional framework of knowledge tree, knowledge and the resource be associated are organized, manage and stored, concrete execution operates below, comprising:

Step S101, in the present system the knowledge of the mankind is classified, and sorted knowledge is cut into knowledge node again, knowledge by the mankind uses knowledge node to represent, knowledge node is the elementary cell for representing human knowledge, namely by the knowledge of the mankind is carried out cutting, to the data that the representation of knowledge segmented becomes computing machine to store and to identify, knowledge node is; Wherein, knowledge node is represented by knowledge node title, Property Name, property value; Therefore, need the knowledge of knowledge engineer's typing as required to analyze in this step, judge whether the database of current design can represent the content that knowledge node comprises, and the relation between knowledge node; If current database does not support the demand of knowledge typing, then turn to step S102; Otherwise, turn to step S103;

Step S102, the demand according to knowledge typing redesigns knowledge data base;

Step S103, knowledge is carried out cutting according to the demand of Knowledge Service System by knowledge engineer, obtains the knowledge node of certain particle size, and obtains the title of each knowledge node;

Step S104, knowledge node is represented by attribute, property value, carries out reductive analysis by knowledge engineer to knowledge node, obtain the attribute that knowledge node comprises, and determine Property Name, namely when representing a certain attribute of knowledge node, the title of this attribute is unified; In addition, determine the field comprised in attribute, such as, resource (video, picture, audio frequency) corresponding to attribute, latitude and longitude information;

Step S105, knowledge node is inserted knowledge base by knowledge engineer, and by knowledge node by tree structure tissue, namely has set membership, brotherhood between knowledge node;

Step S106, knowledge engineer improves the content of knowledge node, i.e. the attribute of typing knowledge node, property value.

Two, knowledge search module provides knowledge search service to user, specifically comprises:

Step S201, search content comes from the result of image recognition, and search content comprises Chinese character, English character, phrase, phrase, sentence etc.;

Step S202, uses the jieba-0.35 word-dividing mode of Python to carry out participle to the search content that step S201 obtains, then extracts search keyword according to word segmentation result;

Step S203, accurately inquires about the keyword extracted in step S202, the knowledge node that namely search and this keyword mate completely in knowledge base, and gives weights " 9999 " to the knowledge node found;

Step S204, the knowledge node whether finding search keyword to mate completely in determining step S203, if find the knowledge node of mating completely, then proceeds to step S207; Otherwise, proceed to step S205;

Step S205, judges whether to open fuzzy query function, if open fuzzy query function, then turns to step S206; Otherwise, turn to step S207;

Step S206, fuzzy query is carried out by not finding the search keyword of the knowledge node of mating completely in step S203, namely in knowledge base, search for the knowledge node that knowledge node title comprises search keyword, and give weights " 999 " to the knowledge node found;

Step S207, obtaining step S203 and step S206 inquires about the knowledge node obtained, other knowledge nodes that search is relevant to these knowledge node in knowledge base, and give weights to the knowledge node found, the similarity of the weights of knowledge node and knowledge node and search keyword, similarity similarity (node, keyword) computing formula is as follows:

Similarity (node, keyword)=∑ _{i ∈ attribute (node)}match (i, keyword) formula 1

$match(i,keyword)=\left\{\begin{array}{cc}1,& \begin{array}{cc}if& keywor\∈i.value\end{array}\\ 0,& others\end{array}\right.$ Formula 2

Wherein, node represents knowledge node, keyword representative search keyword, i represents one of them attribute of knowledge node node, i.value represents the value (property value can be character string or numeral) of attribute i, all properties of attribute (node) representation node node; Namely knowledge node calculates the number of searching for keyword and mating with knowledge node property value with the similarity of search keyword, and coupling number is higher, then knowledge node is larger with the similarity of search keyword; If containing search keyword in property value, then this search keyword mates with this property value of knowledge node, and namely the value of formula (2) is 1, otherwise the value of formula (2) is 0;

Step S208, searches for step S203, step S206 and step S207 the knowledge node obtained and classifies by the knowledge tree belonging to knowledge node, and removes the knowledge node repeated;

Step S209, by the knowledge node after classification, duplicate removal in S208, sort by the weights of knowledge node, the knowledge node rank that namely weights are higher is more forward;

Step S210, the knowledge node after being arranged by S209 carries out visual presenting, and is about to the input as step S501 of the Search Results that finally obtains.

Three, the mode that knowledge recommendation module adopts mixing to recommend provides knowledge recommendation service to user, except the knowledge node that can obtain according to knowledge search module is recommended, can also recommend, specifically comprise based on the historical behavior of user:

Step S301, judges whether that the historical behavior based on user is recommended, if recommended based on user's historical behavior, turns to step S302; Otherwise, turn to step S305;

Step S302, obtains the historical behavior data of user, i.e. user's knowledge node of once browsing or consulting;

Step S303, based on user's historical behavior data that step S302 obtains, adopt the collaborative filtering (User-basedCollaborativeFiltering) based on user to predict the interested knowledge node of user's possibility, and user is to the interest-degree of these knowledge node; The computing formula of user to the interest-degree of knowledge node is as follows:

P (u, i)=∑ _{v ∈ U (u, K) ∩ N (i)}w _uvr _viformula 3

$w_{uv}=\frac{{\Σ}_{i\∈I_u\∩I_v}\frac{1}{\log (1+|N\left(i\right)\left|\right)}}{\sqrt{\left|I_u\right|\left|I_v\right|}}$ Formula 4

Wherein, u represents user, i represents knowledge node, U (u, K) represent K the user the most similar to the behavior of user u, if namely two users browse or inquired about identical knowledge node, then the behavior of two users is similar, and the identical knowledge node that two users browse or inquired about is more, then two users are more similar; r _virepresent that user v is to the scoring of knowledge node i, namely when user v browses or inquired about knowledge node i then r _viequal 1, otherwise, r _viequal 0; w _uvrepresent the similarity of user u and user v, computing formula as shown in (4), wherein I _uand I _vrepresent the knowledge node that user u and user v browses or inquired about respectively, N (i) represents the user's set browsing or inquired about knowledge node i, and namely the similarity calculated between user calculates the ratio overlapped in the knowledge node that user u and user v browses or inquired about; User to the computing formula of knowledge node interest-degree as shown in (3), user u is to the interest-degree of knowledge node i, namely inquire about in K user before similar to user u or the user of browsed knowledge node i, to the scoring of knowledge node i and the sum of products of user u, v similarity.

Step S304, recommends the knowledge node obtained by step S303, according to the user interest degree sequence of prediction, namely sort to the interest-degree of knowledge node by user to the recommendation knowledge node list of each user, the knowledge point rank that interest-degree is higher is higher;

Step S305, certain knowledge node that knowledge search module obtains is as the input recommended;

Step S306, the similarity of the knowledge node that calculation procedure S305 obtains and other knowledge node, similarity between knowledge node is that calculated off-line is good and stored in database, only need search the similarity of knowledge node and other knowledge node herein in a database, the calculating formula of similarity between knowledge node is as follows:

$\begin{array}{c}sim(A,B)=\mathrm{\α}\×\underset{i,j\∈attribute\left(A\right)\∩attribute\left(B\right)}{\Σ}|value{\left(A\right)}_i\∩value{\left(B\right)}_j|\\ +\mathrm{\β}\×\left(\underset{i\∈attribute\left(B\right)}{\Σ}\right|value{\left(B\right)}_i\∩NodeName\left(A\right)|+\underset{i\∈attribute\left(A\right)}{\Σ}|value{\left(A\right)}_i\∩NodeName\left(B\right)\left|\right)\end{array}$

+ γ × treetype (A, B) formula 5

Alpha+beta+γ=1 formula 6

$treetype(A,B)=\left\{\begin{array}{cc}1,& \begin{array}{cc}if& A.TreeId==B.TreeId\end{array}\\ 0,& others\end{array}\right.$ Formula 7

Wherein, A, B represent knowledge node, and attribute (A), attribute (B) represent the attribute that knowledge node A, B comprise respectively, value (A) _i, value (B) _irepresent the property value (property value can be character string or numeral) of the attribute i of knowledge node A, B respectively, NodeName (A), NodeName (B) represent the title of knowledge node A, B respectively, and A.TreeId, B.TreeId represent the id of knowledge tree belonging to knowledge node A, B respectively; Similarity between knowledge node is divided into three parts, ratio shared by each several part is α, β, γ, Part I, the i.e. corresponding part be multiplied of α, calculate the similarity of the attribute that two knowledge node have, owing to there are character string, sentence, paragraph etc. in property value, therefore when the similarity relatively between attribute, first property value corresponding for attribute carried out word segmentation processing and extract keyword, the Similarity value between the number of matching keywords and attribute; Part II, namely the corresponding part be multiplied of β, calculates the number of times that a knowledge node occurs in the property value of another knowledge node, namely calculates the number of times that a knowledge node title occurs in another knowledge node property value; Part III, the i.e. corresponding part formula (7) be multiplied of γ, whether calculation knowledge node belongs to same knowledge tree, if the knowledge tree id belonging to two knowledge node is identical, then result of calculation is 1, otherwise result of calculation is 0;

Step S307, according to the knowledge node similarity calculated in step S306, searches the knowledge node that similarity is greater than 0 in knowledge base;

Step S308, by the knowledge node found out in step S307, sorts according to the knowledge node similarity obtained in step S306, and the knowledge node rank that namely similarity is larger is more forward;

Step S309, the recommendation results obtained by step S304 or step S308 carries out visual presenting, and puts into step S501 by recommendation results.

Four, knowledge reasoning module adopts the method for forward reasoning to carry out reasoning, specifically comprises:

Step S401, obtains user behavior by mobile terminal, and obtains the result of knowledge node that knowledge search module obtains and image recognition, obtains the factbase of reasoning;

Step S402, scanning rule set, the reasoning factbase obtained with step S401 carrys out the former piece of matched rule, obtains available regular collection; Wherein the Formal Representation of rule is as described below:

Rule Rule

IFPreconditionThenConclusion

Precondition: precondition, the LogicExpression (logical expression) be made up of Proposition (proposition);

Conclusion: conclusion, namely true, the logical expression be also made up of Proposition;

Logical expression LogicExpression: by logical word And (with), Or (or), Not (non-) etc. forms logical proposition;

Proposition Proposition:(PredicateVariableList)

Predicate: predicate, it is the program worked out to realize certain function, generally be divided into public with special, the former as logic compare type > ,=, <, >=,≤etc. with conventionally calculation type+,-, ×, ÷ etc.; The latter is as the FFT (Fourier transform) in remote sensing image processing, FeatherAbstract (feature extraction), TargetRecognize (target identification) etc., special predicate is generally designed according to system requirements together with expert by knowledge engineer, and in the predicate of medical science and factory, the predicate of accident diagnosis is made a world of difference certainly.Predicate can be divided into different groups stored in predicate base;

VariableList: argument table is the parameter required for predicate runs.Variable has three types, and one is constant, as numeral, symbol, time etc.; Two is concepts, and as " building height ", when calculating proposition, it can obtain real data from the attribute of concept; Three is another propositions, forms proposition so nested;

In the applicable rule set that step S403, step S402 obtain, if available rule only has one, then turn to step S404; Otherwise, if available rule has multiple, then carry out conflict resolution; Wherein, conflict resolution uses LEX strategy (lexicographicsort):

(1) the one group of rule performed is removed from conflict set;

(2) one group of rule with more new data is selected;

(3) the more detailed one group of rule of the condition of selective rule;

(4) any one group of rule is selected;

Carry out conflict resolution according to the number order of above-mentioned strategy in conflict resolution, until can select one regular time, conflict resolution process terminates, and turns to step S404;

Step S404, perform the regular R selected in step S403, by the conclusion of regular R right part, the new fact that namely reasoning draws joins factbase;

Step S405, whether the conclusion that determining step S404 obtains is reasoning target, and whether the namely new fact is the target that reasoning goes for, if the new fact is reasoning target, then reasoning success, terminates reasoning; Otherwise, if the new fact is not reasoning target, then turn to step S406;

Step S406, scans current factbase, judges whether that the new fact does not use in reasoning in addition, if also have the new fact not use, then turns to step S402, proceed reasoning; Otherwise reasoning failure, terminates reasoning;

Step S407, the reasoning results obtained by step S405 or step S406 carries out visual presenting, and puts into step S501 by the reasoning results.

Five, image collection module, obtains the image of mobile terminal, specifically comprises:

Step S601, checks whether mobile terminal has network, if mobile terminal has network, then turns to step S602; Otherwise, turn to step S605;

Step S602, opens the camera of mobile terminal, and opens camera preview;

Whether step S603, to user's display reminding information, allow user select to take pictures, if take pictures, then turn to step S604; Otherwise, EOP (end of program);

Step S604, user takes pictures to object, stores the picture obtained of taking pictures;

Step S605, whether prompting user arranges network, if user arranges network, then turns to step S601; Otherwise, EOP (end of program).

Six, image characteristics extraction module, feature extraction is carried out to the picture that mobile terminal gets, specifically comprises:

Step S701, obtains mobile terminal and to take pictures the picture obtained;

Step S702, adopts ORB algorithm to extract picture feature;

Step S703, the picture feature extracted by step S702 uploads on Cloud Server.

Seven, picture recognition module is identified image by the characteristics of image obtained, and specifically comprises:

Step S801, obtaining step S703 uploads to the characteristics of image of Cloud Server;

Whether step S802, check the characteristics of image got correct, if input data are correct, then turn to step S803; Otherwise, turn to step S807;

Step S803, scan image feature database, mates the characteristics of image obtained with the image pattern in characteristics of image storehouse;

Step S804, judges in characteristics of image storehouse, whether have sample and this Image Feature Matching, if the match is successful, then turn to step S805; Otherwise, turn to step S806;

Step S805, the sample mated by step S804, identifies the object in image, and the image recognition result obtained is sent to mobile terminal;

Step S806, images match failure, does not obtain the result of picture recognition, sends the information of recognition failures to mobile terminal;

Step S807, sends the information of input error to mobile terminal.

Beneficial outcomes of the present invention: Knowledge Service System disclosed by the invention, stores this build knowledge data, has the powerful feature of semanteme, good knowledge reasoning ability.Build the specialized knowledge base of domain-oriented beyond the clouds, the relation between combing knowledge, carry out the knowledge information management of architecture, intelligentized knowledge retrieval and recommendation are provided; Utilize visualization technique, the profound level, the variation that realize knowledge present; In conjunction with the application of mobile terminal, user is allowed to obtain knowledge services whenever and wherever possible.Use method provided by the invention, can not only provide convenience for users colony, quick, colourful knowledge services, man-machine interaction can also be combined with knowledge system, working knowledge service system, in the self-service guide to visitors in museum, tourist attractions and museum, promotes the propagation of cultural knowledge.

Below in conjunction with accompanying drawing, with " Silk Road " relevant knowledge content for background, knowledge services mobile phone A PP is example, and introduce the process that user obtains diversification knowledge, the present invention is described in more detail.

The method that the present invention uses both can be installed in the form of software and perform on personal computer, industrial computer and server, also method can be made embedded chip and embody in the form of hardware.

As shown in Figure 1, the invention discloses a kind of Knowledge Service System in conjunction with mobile terminal, this system sets up the knowledge data base of this build by knowledge acquisition module, and knowledge based database knowledge search module, knowledge recommendation module, knowledge reasoning module, visualization model provide corresponding service; Meanwhile, by cloud service engine, user utilizes mobile terminal to visit the knowledge data in high in the clouds, thus can use the various services that knowledge services mobile phone A PP provides anywhere or anytime.

As shown in Figure 3, describe in the two-tier system of mobile terminal and cloud service engine, the relation of data stream between each module:

(1) user uses mobile terminal to take the photo of item of interest;

(2) image collection module (S600) on mobile terminal obtains the picture of user's shooting, and photo is sent to image characteristics extraction module (S700);

(3) image characteristics extraction module (S700) carries out feature extraction to the picture got, and the characteristic extracted is sent to cloud service engine;

(4) picture recognition module (S900) in cloud service engine is according to the characteristic obtained, and characteristics of image storehouse (D2) carries out image recognition, and recognition result is sent to knowledge services functional module;

(5) knowledge services functional module calls different functional modules according to the knowledge services that user goes for, wherein this module comprises knowledge search module (S200), knowledge recommendation module (S300), knowledge reasoning module (S400), and function and the relation of each module are as follows:

Knowledge search module (S200) knowledge based database (D1) is searched for recognition result, search match to this recognition result, relevant knowledge node, and send to knowledge recommendation module (S300) and knowledge reasoning module (S400) by searching for the knowledge node that obtains;

Knowledge recommendation module (S300) knowledge based database (D1) carries out the recommendation of two types, and one is that the knowledge node obtained according to knowledge search module (S200) is recommended; Another kind recommends based on user's historical behavior, and user's historical behavior knowledge recommendation module (S300) can be obtained by mobile terminal;

The factbase of knowledge reasoning module (S400) knowledge based database (D1) and reasoning carries out knowledge reasoning; Wherein reasoning factbase comprises the result from the user behavior of acquisition for mobile terminal, knowledge node that knowledge search module obtains and image recognition;

Knowledge services functional module is integrated the knowledge that knowledge search module (S200), knowledge recommendation module (S300), knowledge reasoning module (S400) obtain, and the data integrated are sent to visualization model (S500);

(6) visualization model (S500) carries out visual presenting to the knowledge obtained, and knowledge is presented result and be sent to subscriber interface module (S800);

(7) be shown on mobile terminal after the subscriber interface module (S800) on mobile terminal adjusts according to the data that cloud service engine sends, the result of knowledge services is presented to user.

As shown in Figure 4, knowledge acquisition module comprises the following steps:

Knowledge acquisition module obtains knowledge by the mode of man-machine interaction, and knowledge organization is become knowledge tree, and such as, the relevant knowledge for " Silk Road " carries out knowledge acquisition:

Step S101, classifies the knowledge in " Silk Road ", and sorted knowledge is cut into knowledge node again, and use knowledge node to represent by knowledge, now knowledge is divided into 10 classes, form 10 knowledge trees, knowledge classification result as shown in fig. lla;

Wherein, knowledge node is represented by knowledge node title, Property Name, property value; Therefore, need the knowledge of knowledge engineer's typing as required to analyze in this step, judge whether the database of current design can represent the content that knowledge node comprises, and the relation between knowledge node; Current database supports the demand of knowledge typing, turns to step S103; Now for historic site tree, further illustrate the process of knowledge acquisition;

Step S103, knowledge is carried out cutting according to the demand of Knowledge Service System by knowledge engineer, obtains the knowledge node of certain particle size, and obtains the title of each knowledge node; Now for historic site tree, the relevant knowledge of historic site is carried out further cutting, cutting result as shown in figure lib;

Step S104, knowledge engineer analyzes the knowledge node comprised in historic site tree, obtains the attribute that all kinds of knowledge node comprises, and determines Property Name, such as, the knowledge node in folk building comprises place, character introduction, picture example, these four attributes of building type; Knowledge node in mausoleum building comprises attributes such as building age, character introduction, site, mausoleum owner, representative picture, picture example, burial time, excavation age; In addition, determine the field comprised in attribute, mainly comprise Property Name, property value, longitude, latitude, resource;

Step S105, knowledge node is inserted knowledge base by knowledge engineer, and by knowledge node by tree structure tissue, namely has set membership, brotherhood between knowledge node; Such as, the partial knowledge node inserted in folk building as shown in fig. live;

Step S106, knowledge engineer improves the content of knowledge node, i.e. the attribute of typing knowledge node, property value, and the attributes edit interface of knowledge node is as illustrated in fig. 1 ld;

The attribute modification interface of knowledge node as illustrated in fig. 1 le;

Improve " Qin Shihuang Tomb " knowledge node of content as shown in figure hf;

As shown in Figure 5, knowledge search module comprises the following steps:

Step S201, user is inputted search content in the search box, such as " Xuanzang is permanently happy ";

Step S202, use the jieba-0.35 word-dividing mode of Python to carry out participle to the search content that step S201 obtains, then extract search keyword according to word segmentation result, the search keyword of extraction is " Xuanzang ", " permanently happy ";

Step S203, the keyword extracted in step S202 is accurately inquired about, the knowledge node of coupling is not found in search keyword " permanently happy " accurately inquiry, " Xuanzang " accurately inquiry finds the knowledge node of coupling, Query Result as figure 12 a shows, wherein the first table shows the knowledge node inquired, the attribute of the knowledge node that the second table display is corresponding;

Step S204, searches for keyword " Xuanzang " and finds the knowledge node of mating completely, proceed to step S207 in determining step S203; The knowledge node of mating completely is not found in another search keyword " permanently happy ", proceeds to step S207;

Step S205, opens fuzzy query function and turns to step S206;

Step S206, fuzzy query is carried out by not finding the search keyword " permanently happy " of the knowledge node of mating completely in step S203, namely in knowledge base, search for the knowledge node " permanently happy door " that knowledge node title comprises search keyword, and give weights " 999 " to the knowledge node found, Query Result is as shown in Figure 12b;

Step S207, obtaining step S203 and step S206 inquires about the knowledge node " Xuanzang " and " permanently happy door " that obtain, other knowledge nodes that search is relevant with " permanently happy door " to " Xuanzang " in knowledge base, and give weights to the knowledge node found, the similarity of the weights of knowledge node and knowledge node and search keyword, the interdependent node obtained is as shown in Figure 12 c second and the 4th table:

Step S208, searches for step S203, step S206 and step S207 the knowledge node obtained and classifies by the knowledge tree belonging to knowledge node, and removes the knowledge node repeated;

Step S209, by the knowledge node after classification, duplicate removal in S208, sort by the weights of knowledge node, the knowledge node rank that namely weights are higher is more forward, and last Query Result is as shown in figure 12d;

Step S210, the knowledge node after being arranged by S209 carries out visual presenting, and be about to the input as step S501 of the Search Results that finally obtains, visual result is as shown in Figure 12 e;

As shown in Figure 6, knowledge recommendation module comprises the following steps:

Step S301, if recommended for user's historical behavior, such as now to recommend the user that user id is 1603, turns to step S302; Recommend for knowledge node " Xuanzang ", turn to step S305;

Step S302, obtains the historical behavior data of user, i.e. user's knowledge node of once browsing or consulting, and wherein the historical behavior data of user 1603 as depicted in fig. 13 a;

Step S303, based on user's historical behavior data that step S302 obtains, adopt the collaborative filtering (User-basedCollaborativeFiltering) based on user to predict the interested knowledge node of user's possibility, and user is to the interest-degree of these knowledge node;

Step S304, step S303 is recommended the knowledge node obtained, according to the user interest degree sequence of prediction, namely the recommendation knowledge node list of each user is sorted to the interest-degree of knowledge node by user, the knowledge point rank that interest-degree is higher is higher, and the recommendation list after sequence as illustrated in fig. 13b;

Step S305, knowledge node " Xuanzang " is as the input recommended;

Step S306, the similarity of the knowledge node " Xuanzang " that calculation procedure S305 obtains and other knowledge node, similarity between knowledge node is that calculated off-line is good and stored in database, only need search the similarity of " Xuanzang " and other knowledge node herein in a database;

Step S307, according to the knowledge node similarity calculated in step S306, in knowledge base, search the knowledge node that similarity is greater than 0, the partial results of searching is as shown in figure 13 c;

Step S308, by the knowledge node found out in step S307, sorts according to the knowledge node similarity obtained in step S306, and the knowledge node rank that namely similarity is larger is more forward, and the result after sequence as shown in figure 13d;

Step S309, the recommendation results obtained by step S304 or step S308 carries out visual presenting, step S501 is put into by recommendation results, such as, the mode of label-cloud recommendation results about knowledge node " Xuanzang " is used to carry out presenting (first 40 of display recommendation results), as shown in figure 13e;

As shown in Figure 7, knowledge reasoning module comprises the following steps:

Step S401, obtains user behavior by mobile terminal, and obtains the result of knowledge node that knowledge search module obtains and image recognition, and obtain the factbase of reasoning, such as, the factbase obtained at present is:

A. the Tang Dynasty

B. flouring period

C. Li Long-ji (patron of actors)

In example, the target of reasoning wishes to find out the concrete period of history, and the period of history uses global variable period to represent;

Step S402, scanning rule set, the reasoning factbase obtained with step S401 carrys out the former piece of matched rule, obtains available regular collection; Wherein, time is that global variable represents the time, and name is also that global variable represents posthumous title of an emperor, and regular collection is as follows:

Or R1:If Tang Dynasty the Tang Dynasty, Thentime >=618andtime≤907

R2:If Li Shih-min, Thenname=Tang second emperor since founding of a country

R3:If Li Long-ji (patron of actors), Thenname=Hsuan Tsung

R4:If (time >=618andtime≤907) and grows prosperity period, the period=view of chastity control orperiod=Katyuan flourishing age

R5:If (time >=618andtime≤907) and grows prosperity andname=Hsuan Tsung in period, period=Katyuan flourishing age

Use a matching rule base in factbase, obtaining available regular collection is { R1};

In the applicable rule set that step S403, step S402 obtain, available rule only has one, then turn to step S404;

Step S404, perform the regular R1 selected in step S403, by the conclusion of regular R1 right part, namely " time >=618andtime≤907 " add factbase, obtain the following fact:

d.time≥618andtime≤907

Step S405, the conclusion that determining step S404 obtains is not reasoning target, turns to step S406;

Step S406, scans current factbase, finds to also have the new fact not use, then turn to step S402, continue reasoning;

Follow-up reasoning process is as follows:

Second time reasoning:

Step S402, true b mates with regular collection, finds the rule not having to mate; True c mates with regular collection, and obtaining available regular collection is { R3};

In the applicable rule set that step S403, step S402 obtain, available rule only has one, then turn to step S404;

Step S404, perform the regular R3 selected in step S403, by the conclusion of regular R3 right part, namely " name=Hsuan Tsung " adds factbase, obtains the following fact:

E.name=Hsuan Tsung

Step S405, the conclusion that determining step S404 obtains is not reasoning target, turns to step S406;

Step S406, scans current factbase, finds to also have the new fact not use, then turn to step S402, continue reasoning;

Third time reasoning:

Step S402, true b, d mate with regular R4; True b, d, e mate with regular R5, and obtaining available regular collection is { R4, R5};

There are many rules in the applicable rule set that step S403, step S402 obtain, carry out conflict resolution; The LEX strategy (lexicographicsort) that application collision is cleared up finds (1), (2) can not clear up conflict, until when performing (3), find more detailed than R4 of the condition of regular R5, therefore choice for use rule R5, turns to step S404;

Step S404, performs in step S403 the regular R5 selected, by the conclusion of regular R5 right part, namely " period=Katyuan flourishing age " add factbase, obtain the following fact:

F.period=Katyuan flourishing age

Step S405, the conclusion that determining step S404 obtains is reasoning target, and the reasoning results namely obtained is period=Katyuan flourishing age, reasoning success;

Step S407, carries out visual presenting to the reasoning results that step S405 obtains, puts into step S501 by the reasoning results;

The semantic description of whole reasoning is as follows:

When the flouring period that the residing epoch are the Tang Dynasty, and be in Li Long-ji's period in place, the so corresponding period of history is " Katyuan flourishing age ".

As shown in Figure 8, image collection module comprises the following steps:

Be now that mobile terminal is to do example explanation with Samsung GALAXYNoteII (N7100/16GB) mobile phone;

Step S601, checks whether mobile phone has network, and mobile phone connecting wireless network, turns to step S602;

Step S602, opens the camera of mobile terminal, and opens camera preview;

Whether step S603, to user's display reminding information, allow user select to take pictures, and user selects to take pictures, and turns to step S604;

Step S604, user takes pictures to object, and store the picture obtained of taking pictures, the picture obtained is as shown in figure 14;

As shown in Figure 9, image characteristics extraction module comprises the following steps:

Step S701, obtain the picture that mobile phone photograph obtains, namely obtaining step S604 is stored in the picture on mobile phone;

Step S702, adopt ORB algorithm to extract picture feature, the unique point extracted is as shown in the red circle marked in Figure 15;

Step S703, the picture feature extracted by step S702 uploads on Cloud Server.

As shown in Figure 10, picture recognition module comprises the following steps:

Such as, the feature (as shown in figure 16) of four width pictures below characteristics of image library storage, carries out image recognition based on this characteristics of image storehouse;

Step S801, obtaining step S703 uploads to the characteristics of image of Cloud Server;

Step S802, checks the characteristics of image got to be correct, turns to step S803;

Step S803, scan image feature database, mates the characteristics of image obtained with the image pattern in characteristics of image storehouse;

Step S804, judges in characteristics of image storehouse, whether have sample and this Image Feature Matching, finds, with the characteristic matching of " sets of brackets on top of the columns " this pictures in Sample Storehouse, to turn to step S805;

Step S805, the sample mated by step S804, the object identified in image is " sets of brackets on top of the columns ", the image recognition result obtained is sent on mobile phone.

Above-described specific embodiment; object of the present invention, technical scheme and beneficial effect are further described; be understood that; the foregoing is only specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any amendment made, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.

Claims (8)

1., in conjunction with a Knowledge Service System for mobile terminal, comprise cloud service engine and mobile terminal, wherein:

Cloud service engine comprises:

Knowledge acquisition module (S100), obtains knowledge by the mode of man-machine interaction, and organizes knowledge, manages and store in knowledge data base (D1);

Knowledge search module (S200), the recognition result of knowledge based database (D1) and picture recognition module (S900), provides knowledge search service to user;

Knowledge recommendation module (S300), knowledge based database (D1) and user's historical behavior, or the knowledge node that searches of knowledge based search module (S200) and user's historical behavior, knowledge recommendation service is provided to user, knowledge node is the elementary cell representing human knowledge, by the knowledge of the mankind is carried out cutting, to the data that the representation of knowledge segmented becomes computing machine to store and to identify, be knowledge node;

Knowledge reasoning module (S400), the knowledge node that knowledge based search module (S200) searches and user's historical behavior, carry out reasoning with reference to regular collection, provide the reasoning results to user;

Visualization processing module (S500), by visual for the output of at least one in knowledge search module (S200), knowledge recommendation module (S300) and knowledge reasoning module (S400), and send visualization result to mobile terminal;

Picture recognition module (S900), mates the characteristics of image received from mobile terminal with characteristics of image storehouse (D2), to obtain recognition result;

Knowledge data base (D1), stored knowledge; And

Characteristics of image storehouse (D2), stores the characteristics of image of multiple image,

Wherein, above-mentioned each module and storehouse are deployed in a station server, or distribution is deployed in the multiple servers mutually can accessed by express network,

Mobile terminal comprises:

Image collection module (S600), obtains image;

Image characteristics extraction module (S700), from image zooming-out feature; And

Subscriber interface module (S800), realizes the man-machine interaction with user.

2. Knowledge Service System as claimed in claim 1, is characterized in that:

After mobile terminal gets image, in image characteristics extraction module (S700) on mobile terminals, carry out the extraction of characteristics of image, the image feature data extracted is sent to cloud service engine.

3. Knowledge Service System as claimed in claim 1, is characterized in that:

In the characteristics of image storehouse (D2) of cloud service engine, store the image feature data arranging in advance and extract, these image feature datas come from the different angles of the corresponding article in knowledge point and the image sampling of distance size, and the algorithm that the image characteristics extraction algorithm adopted and mobile terminal use is consistent.

4. Knowledge Service System as claimed in claim 1, it is characterized in that, knowledge data base (D1) also comprises:

Adopt the institutional framework of knowledge tree, knowledge organized, manage and stores, described knowledge comprise following at least one: text, picture, video and audio frequency.

5., based on a knowledge services method for Knowledge Service System as claimed in claim 1, comprising:

Step S001, user uses mobile terminal to take the photo of item of interest;

Step S002, the image collection module on mobile terminal obtains the picture of user's shooting;

Step S003, image characteristics extraction module carries out feature extraction to the picture got, and the characteristic extracted is sent to cloud service engine;

Step S004, the picture recognition module in cloud service engine is according to the characteristic obtained, and image recognition is carried out in characteristics of image storehouse;

Step S005, the result of image recognition is sent to following at least one: knowledge recommendation module, knowledge search module or knowledge reasoning module;

Step S006, to user provide following at least one: knowledge search service, knowledge recommendation service or knowledge reasoning service;

Step S007, the knowledge obtained by step S006 sends to visualization model;

Step S008, visualization model is carried out visual to the knowledge received, and visual result is sent on mobile terminal, and presents to user by subscriber interface module.

6. method according to claim 5, wherein knowledge data base adopts following organizational form:

The knowledge of the mankind classified, and sorted knowledge is cut into knowledge node again, described knowledge node has certain particle size;

Knowledge node is represented by knowledge node title, Property Name, property value;

By knowledge node by tree structure tissue;

The multimedia resource relevant to knowledge node is carried out arrangement and puts into resources bank, and these resources are classified centered by knowledge node, use the attribute of knowledge node to associate with corresponding knowledge node to sorted resource.

7. method according to claim 5, wherein knowledge search comprises:

Step S201, search content comes from the result of image recognition, search content comprise following at least one: Chinese character, English character, phrase, phrase, sentence etc.;

Step S202, carries out participle to the search content that step S201 obtains, and then extracts search keyword according to word segmentation result;

Step S203, accurately inquires about the keyword extracted in step S202, the knowledge node that namely search and this keyword mate completely in knowledge base, and gives weights " 9999 " to the knowledge node found;

Step S204, the knowledge node whether finding search keyword to mate completely in determining step S203, if find the knowledge node of mating completely, then proceeds to step S207; Otherwise, proceed to step S205;

Step S205, judges whether to open fuzzy query function, if open fuzzy query function, then turns to step S206; Otherwise, turn to step S207;

Step S206, fuzzy query is carried out by not finding the search keyword of the knowledge node of mating completely in step S203, namely in knowledge base, search for the knowledge node that knowledge node title comprises search keyword, and give weights " 999 " to the knowledge node found;

Step S207, obtaining step S203 and step S206 inquires about the knowledge node obtained, other knowledge nodes that search is relevant to these knowledge node in knowledge base, and give weights to the knowledge node found, the similarity of the weights of knowledge node and knowledge node and search keyword, similarity similarity (node, keyword) computing formula is as follows:

Similarity (node, keyword)=∑ _{i ∈ attribute (node)}match (i, keyword) formula 1

formula 2

Wherein, node represents knowledge node, keyword representative search keyword, and i represents one of them attribute of knowledge node node, i.value represents the value of attribute i and can be character string or numeral, all properties of attribute (node) representation node node; Namely knowledge node calculates the number of searching for keyword and mating with knowledge node property value with the similarity of search keyword, and coupling number is higher, then knowledge node is larger with the similarity of search keyword; If containing search keyword in property value, then this search keyword mates with this property value of knowledge node, and namely the value of formula 2 is 1, otherwise the value of formula 2 is 0;

Step S208, searches for step S203, step S206 and step S207 the knowledge node obtained and classifies by the knowledge tree belonging to knowledge node, and removes the knowledge node repeated;

Step S209, by the knowledge node after classification, duplicate removal in S208, sort by the weights of knowledge node, the knowledge node rank that namely weights are higher is more forward;

Step S210, the knowledge node after being arranged by S209 carries out visual presenting, and is about to the input as step S501 of the Search Results that finally obtains.

8. method according to claim 5, wherein knowledge recommendation comprises:

Step S305, certain knowledge node is as the input recommended;

Step S306, the similarity of the knowledge node that calculation procedure S305 obtains and other knowledge node, similarity between knowledge node is that calculated off-line is good and stored in database, only need search the similarity of knowledge node and other knowledge node herein in a database, similarity Slm (A, B) computing formula between knowledge node is as follows:

formula 5

Alpha+beta+γ=1 formula 6

formula 7

Wherein, A, B represent knowledge node, and attribute (A), attribute (B) represent the attribute that knowledge node A, B comprise respectively, value (A) _i, value (B) _irespectively represent knowledge node A, B attribute i property value and can be character string or numeral, value (B) _jrepresent the property value of the attribute j of knowledge node B, NodeName (A), NodeName (B) represent the title of knowledge node A, B respectively, and A.TreeId, B.TreeId represent the id of knowledge tree belonging to knowledge node A, B respectively; Similarity between knowledge node is divided into three parts, ratio shared by each several part is α, β, γ, Part I, the i.e. corresponding part be multiplied of α, calculate the similarity of the attribute that two knowledge node have, owing to there are character string, sentence, paragraph etc. in property value, therefore when the similarity relatively between attribute, first property value corresponding for attribute carried out word segmentation processing and extract keyword, the Similarity value between the number of matching keywords and attribute; Part II, namely the corresponding part be multiplied of β, calculates the number of times that a knowledge node occurs in the property value of another knowledge node, namely calculates the number of times that a knowledge node title occurs in another knowledge node property value; Part III, the i.e. corresponding part formula 7 be multiplied of γ, whether calculation knowledge node belongs to same knowledge tree, if the knowledge tree id belonging to two knowledge node is identical, then result of calculation is 1, otherwise result of calculation is 0;

Step S307, according to the knowledge node similarity calculated in step S306, searches the knowledge node that similarity is greater than 0 in knowledge base;

Step S308, by the knowledge node found out in step S307, sorts according to the knowledge node similarity obtained in step S306, and the knowledge node rank that namely similarity is larger is more forward;

Step S309, the recommendation results obtained by step S304 or step S308 carries out visual presenting, and puts into step S501 by recommendation results.