CN112287104A - Natural language processing method and device - Google Patents
Natural language processing method and device Download PDFInfo
- Publication number
- CN112287104A CN112287104A CN202011038685.0A CN202011038685A CN112287104A CN 112287104 A CN112287104 A CN 112287104A CN 202011038685 A CN202011038685 A CN 202011038685A CN 112287104 A CN112287104 A CN 112287104A
- Authority
- CN
- China
- Prior art keywords
- natural language
- language processing
- data
- service
- processing service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
- G06F40/295—Named entity recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Abstract
The embodiment of the invention provides a natural language processing method and a natural language processing device, which are applied to a natural language processing platform, wherein the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in an urban portal system, and the method comprises the following steps: receiving natural language processing service requests provided by users of a plurality of service platforms; acquiring data to be processed appointed by a user from a service platform to which the user belongs; responding to the natural language processing service request, and calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result; and sending the processing result to a user. In the urban portal system, the natural language processing service requests sent by other service platforms are received by the natural language processing platform deployed in the developer portal system, and the natural language processing service interface is called for processing, so that users of other service platforms of the urban portal system can obtain processing results obtained after natural language processing.
Description
Technical Field
The present invention relates to the field of network technologies, and in particular, to a natural language processing method and a natural language processing apparatus.
Background
With the development of technologies such as computers, networks, communication and the like, the artificial intelligence deep application is taken as a way to construct urban construction conforming to a cross-domain multi-dimensional concept, technical fusion, business fusion, data fusion and application fusion are promoted, cross-level, cross-region, cross-system, cross-department and cross-business cooperative management and service are realized, the urban construction is accelerated by cooperating with various industries, enterprises and teams, and the method is a development direction of smart cities.
At present, most urban developments have no multi-dimensional fusion planning scheme, and the problems that information platforms of various industries, enterprises and teams are not related and mutual-aided in function, information is not shared and exchanged, and information, service flow and application are mutually disjointed cause that the information in the information platforms cannot be fully utilized.
The urban portal system for realizing multi-dimensional fusion aiming at the city and how to make the urban portal system more beneficial to the use of users are the solutions urgently needed at present.
Disclosure of Invention
In view of the above problems, embodiments of the present invention are proposed to provide a natural language processing method and a corresponding natural language processing apparatus that overcome or at least partially solve the above problems.
In order to solve the above problems, an embodiment of the present invention discloses a natural language processing method, which is applied to a natural language processing platform, wherein the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in a city portal system, and a natural language processing service interface provided by the natural language processing platform is registered in advance in a unified service gateway of the city portal system and is released; the city portal system further comprises a plurality of service platforms, and the method comprises the following steps:
receiving natural language processing service requests provided by users of the plurality of business platforms;
acquiring the data to be processed appointed by the user from a service platform to which the user belongs;
responding to the natural language processing service request, and calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result;
and sending the processing result to the user.
Optionally, the natural language processing service interface includes a comment opinion extraction interface, the data to be processed includes comment data, and the natural language processing service request includes a comment opinion extraction request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the comment viewpoint extraction request, and calling the comment viewpoint extraction interface to perform viewpoint extraction processing on the comment data to obtain comment viewpoints.
Optionally, the natural language processing service interface includes an emotional tendency analysis interface, the data to be processed is text data, and the natural language processing service request includes an emotional tendency analysis request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
responding the emotional tendency analysis request, calling the emotional tendency analysis interface to perform emotional tendency analysis processing on the text data, and generating an emotional tendency analysis result.
Optionally, the natural language processing service interface includes an article tag analysis interface, the data to be processed is article data, and the natural language processing service request includes an article tag analysis request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding the article label analysis request, and calling the article label analysis interface to perform label analysis processing on the article data to obtain a label analysis result.
Optionally, the natural language processing service interface includes an article classification interface, the data to be processed is article data, and the natural language processing service request includes an article classification request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the article classification request, and calling the article classification interface to classify the article data to obtain an article classification result.
Optionally, the natural language processing service interface includes a text error correction interface, the data to be processed is text data, and the natural language processing service request includes a text error correction request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the text error correction request, and calling the text error correction interface to perform text error correction processing on the text data to obtain error-corrected text data.
Optionally, the natural language processing service interface includes a text audit interface, the data to be processed is text data, and the natural language processing service request includes a text audit request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the text auditing request, and calling the text auditing interface to perform text auditing processing on the text data to obtain non-compliance information.
The embodiment of the invention also discloses a natural language processing device which is applied to a natural language processing platform, wherein the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in an urban portal system, and a natural language processing service interface provided by the natural language processing platform is registered in advance in a unified service gateway of the urban portal system and is issued; the city portal system further comprises a plurality of service platforms, and the device comprises:
the service request receiving module is used for receiving natural language processing service requests provided by users of the plurality of service platforms;
a to-be-processed data acquisition module, configured to acquire to-be-processed data specified by the user from a service platform to which the user belongs;
the processing service interface calling module is used for responding to the natural language processing service request and calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result;
and the processing result sending module is used for sending the processing result to the user.
Optionally, the natural language processing service interface includes a comment opinion extraction interface, the data to be processed includes comment data, and the natural language processing service request includes a comment opinion extraction request; the processing service interface calling module comprises:
and the first processing service submodule is used for responding to the comment viewpoint extraction request, calling the comment viewpoint extraction interface to perform viewpoint extraction processing on the comment data, and obtaining comment viewpoints.
Optionally, the natural language processing service interface includes an emotional tendency analysis interface, the data to be processed is text data, and the natural language processing service request includes an emotional tendency analysis request; the processing service interface calling module comprises:
and the second processing service submodule is used for responding to the emotional tendency analysis request, calling the emotional tendency analysis interface to perform emotional tendency analysis processing on the text data, and generating an emotional tendency analysis result.
Optionally, the natural language processing service interface includes an article tag analysis interface, the data to be processed is article data, and the natural language processing service request includes an article tag analysis request; the processing service interface calling module comprises:
and the third processing service sub-module is used for responding to the article label analysis request, calling the article label analysis interface to perform label analysis processing on the article data, and obtaining a label analysis result.
Optionally, the natural language processing service interface includes an article classification interface, the data to be processed is article data, and the natural language processing service request includes an article classification request; the processing service interface calling module comprises:
and the fourth processing service sub-module is used for responding to the article classification request, calling the article classification interface to classify the article data and obtaining an article classification result.
Optionally, the natural language processing service interface includes a text error correction interface, the data to be processed is text data, and the natural language processing service request includes a text error correction request; the processing service interface calling module comprises:
and the fifth processing service sub-module is used for responding to the text error correction request and calling the text error correction interface to perform text error correction processing on the text data to obtain error-corrected text data.
Optionally, the natural language processing service interface includes a text audit interface, the data to be processed is text data, and the natural language processing service request includes a text audit request; the processing service interface calling module comprises:
and the sixth processing service sub-module is used for responding to the text auditing request, calling the text auditing interface to perform text auditing processing on the text data, and obtaining non-compliance information.
The embodiment of the invention also discloses a device, which comprises: a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program when executed by the processor implementing the steps of any of the natural language processing methods.
The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of any natural language processing method are realized.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, the natural language processing platform deployed in the developer portal system can receive natural language processing service requests provided by users of a plurality of service platforms in the urban portal system, acquire to-be-processed data specified by the users who provide the requests, execute natural language processing service on the acquired to-be-processed data by calling corresponding natural language processing service interfaces, and return processed processing results to the users who provide the requests. In the urban portal system, the natural language processing service requests sent by other service platforms are received by the natural language processing platform deployed in the developer portal system, and the natural language processing service interface is called for processing, so that users of other service platforms of the urban portal system can obtain processing results obtained after natural language processing.
Drawings
FIG. 1 is a block diagram of a city portal system of an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first embodiment of a natural language processing method according to the present invention;
FIG. 3 is a flowchart illustrating steps of a second embodiment of a natural language processing method according to the present invention;
FIG. 4 is a schematic structural diagram of a natural language processing platform according to an embodiment of the present invention;
FIG. 5 is a block diagram of an embodiment of a natural language processing apparatus according to the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
The city portal system is a public platform portal which integrates comprehensive internet information aiming at city planning and provides comprehensive application services. The service objects of the city portal system include: government, individual, enterprise, developer, can provide a plurality of comprehensive services such as government affairs service, individual service, enterprise service, etc.
Referring to fig. 1, a block diagram of a city portal system according to an embodiment of the present invention is shown, which may specifically include: a front-end system 10, a back-end system 11, an API open platform 12, a developer portal system 13, an operation center 14, and the like.
A front-end system 10 that implements a plurality of functions and provides a plurality of pages; the plurality of pages include a plurality of UI elements corresponding to the plurality of functions; the plurality of functions includes a function supported by a backend system to provide a service.
The front-end system is a client facing the user, and is used as a tool for the user to use various contents in the urban portal system, and the user can use information, applications, API services, system functions and the like in the urban portal system through the front-end system. The front-end system displays a universal front-end development framework such as Vue, React and the like to realize unified presentation of a single page of the multi-service system.
The front-end system supports multi-dimensional user use, including tourists, natural people, corporate legal people, enterprise employees and government personnel. The front-end system supports multi-dimensional business city services, including government affair services, public services, characteristic services and the like. The user may use a variety of city services through the head-end system.
The front-end system can comprise an APP client, a Web client and a Web management end, wherein the Web client faces tourists, natural people, enterprises and government users and is used for city portal system official networks, API open platforms and developer portals. The APP client faces tourists, natural people, enterprises and government users and is used for moving the APP; the Web management end faces to operators and system managers and is used for operation centers and other back-end management systems.
And the back-end system 11 is configured to provide service support for the front-end system, respond to a service request of the front-end system, and execute a corresponding service operation.
And the back-end system takes the universal service component or the technical service as a bridge to get through the bottom data. The front-end system is decoupled from the back-end system, and the back-end system provides service support for the front-end system. The front-end system and the back-end system are separately deployed, and the back-end system serves dynamic capacity expansion to achieve the maximum performance of the system.
And the API open platform 12 is used for providing management services aiming at the API, including API publishing services, purchasing services and using services.
The API open platform provides a unified standard data and system development environment, can be applied to various industries and systems, is an open comprehensive service platform with unified solution capability service, and aims to realize the management and control of the whole flow life cycle of unified capability opening including service capability access, open management, capability application and the like. By publishing the API services to the API open platform, developers can apply for or purchase use on the platform by other persons.
A developer portal system 13 for providing an environment for API, application, data development and deployment, and common procedural components.
The developer portal system can provide the developer with an environment for application, algorithm, and data development and deployment, as well as generic AI building blocks, technical building blocks, and business building blocks. The method comprises the functions of application development, algorithm development, application release, service release and the like. Developers can quickly develop and publish applications and APIs based on the environment, components, templates, etc. provided by the developer portal system. The developer portal system may include three platforms, an algorithm development platform, an application development platform, and a data development platform.
And the operation center 14 is used for managing the content, the users, the applications and the API of the city portal system.
The operation center is a business center station which provides unified daily operation management for managers and operators to the urban portal system. The management personnel can carry out unified management on the content, the users, the applications, the API and the like of the city portal system through the operation center.
The embodiment of the invention provides an urban portal system which can integrate a front-end system, a back-end system, an API open platform, a developer portal system, an operation center and other platforms. The front-end system serves as a client and faces various users, and the users can obtain contents provided by various platforms integrated in the urban portal system by performing operations on the front-end system. The back-end system provides service support for the front-end system, responds to the service request of the front-end system and executes corresponding service operation. The API open platform provides management services aiming at the API, including API publishing services, purchasing services and using services; the method can be used for developers to call API uniformly and establish a standard and uniform information platform. The developer portal system can provide an environment for API, application, data development and deployment, and general purpose procedural components; and the development of various services can be realized by developers. The operation center can manage the content, users, applications and API of the city portal system. The embodiment of the invention provides a comprehensive city portal system for a city, which is oriented to various users in the city, and the users can quickly and conveniently realize various digital services through the city portal system; and standardized service development is realized through the city portal system.
Referring to fig. 2, a flowchart of a first embodiment of a natural language processing method according to the present invention is shown, and is applied to a natural language processing platform, where the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in a city portal system, and a natural language processing service interface provided by the natural language processing platform is registered in advance in a unified service gateway of the city portal system and is published; the city portal system further comprises a plurality of service platforms, and specifically comprises the following steps:
in an embodiment of the present invention, the city portal system may include a developer portal system and other service platforms, the developer portal system and the other service platforms are in communication with each other, and the natural language processing platform deployed in the developer portal system may receive natural language processing service requests provided by users of the other service platforms, so as to respond to the service requests provided by the users and perform corresponding processing.
When users of other multiple service platforms provide natural language processing service requests for a natural language processing platform deployed in a developer portal system, the service communication requests between the multiple service platforms and the developer portal system in the urban portal system are mainly expressed, and when the urban portal system detects the service communication requests provided by the users of the multiple service platforms, the urban portal system can perform identity authentication on the users providing the service requests, namely under the condition that the identity authentication platform judges that a session (time domain) session corresponding to the user and the session corresponding to the user are valid, the service requests provided by the users of the multiple service platforms are sent to the natural language processing platform deployed in the developer portal system.
in practical application, after receiving natural language processing service requests provided by users of other multiple platforms, to-be-processed data to be subjected to natural language processing by the users needs to be acquired, where the to-be-processed data may be user-specified to-be-processed data, and the user-specified to-be-processed data may be data carried in the received natural language processing service requests or data from other sources, which is not limited in the embodiment of the present invention.
In an embodiment of the present invention, after receiving a natural language processing service request provided by a user of another service platform and acquiring data to be processed specified by the user, the natural language processing platform may provide a corresponding natural language processing service for the user, and specifically, may respond to the natural language processing service request provided by the user, and invoke a natural language processing service interface corresponding to the natural language processing service provided by the user to perform natural language processing on the data to be processed, so as to obtain a processed processing result, and send the processing result to the user who provided the service request.
In the embodiment of the invention, the natural language processing platform deployed in the developer portal system can receive natural language processing service requests provided by users of a plurality of service platforms in the urban portal system, acquire to-be-processed data specified by the users who provide the requests, execute natural language processing service on the acquired to-be-processed data by calling corresponding natural language processing service interfaces, and return processed processing results to the users who provide the requests. In the urban portal system, the natural language processing service requests sent by other service platforms are received by the natural language processing platform deployed in the developer portal system, and the natural language processing service interface is called for processing, so that users of other service platforms of the urban portal system can obtain processing results obtained after natural language processing.
Referring to fig. 3, a flowchart illustrating steps of a second embodiment of the natural language processing method according to the present invention is shown, and is applied to a natural language processing platform, where the method specifically includes the following steps:
in one embodiment of the invention, a natural language processing platform deployed in a developer portal system can receive a natural language processing service request made by a user and acquire to-be-processed data specified by the user, so as to respond to the service request made by the user and perform corresponding processing on the acquired to-be-processed data.
The user who proposes the natural language processing service request can be a login user of a developer portal system integrated in the city portal system, and can also be a user of other business platforms except the developer portal system in the city portal system; for the way of proposing the natural language processing service request, after a user logs in a developer portal system or other business platforms, the user may appear on a main page of the system or a main page of the platform, the main page may include an operation bar for the natural language processing service request or a link entry for the natural language processing service request, and at this time, the proposing of the natural language processing service request may be realized through a touch operation of the logged-in user acting on the operation bar or the link entry.
Specifically, referring to fig. 4, which shows a schematic structural diagram of a natural language processing platform in an embodiment of the present invention, the entire natural language processing platform may be built on a k8s cluster (kubernets, container cluster management system), and may include a routing layer 401, an algorithm service layer 402, and a system layer 403.
The routing layer 401 may expose the service to the outside, all the services may use the same domain name, but different algorithmic services need to use different URIs (Uniform Resource identifiers, which are used to locate each Resource available on the Web), for example, for a participle algorithmic service, the corresponding URI may be/v 1/api/token, and for an NER (Named Entity Recognition ), the corresponding URI may be/v 1/api/NER.
The algorithm service layer 402 may be a carrier of algorithm services in a cluster, where the algorithm services may include chinese word segmentation, Part-of-Speech tagging or POS tagging, named entity recognition, dependency syntax analysis, word vectors, word sense similarity, short text similarity, comment view extraction, emotion analysis, article labels, article classification, text error correction, text review, and the like, each algorithm service may be deployed in a multi-instance deployment manner, and is generally deployed in a dual-pod manner, so as to ensure stability of the service, and a live detection mechanism (which means that a user may specify a live detection manner for implementing health inspection of a container) of the cluster may ensure high availability of the service, and when the service requires a higher query rate per second QPS, the implementation may be implemented by adjusting the number of pod instances of each service algorithm. It should be noted that the algorithm type is not limited to the above type, and dynamic expansion can be performed, and when the algorithm service is upgraded, only the algorithm service image upgrade needs to be performed, so that the operation and maintenance cost is low.
The system layer 403 may provide basic components for running the whole natural language processing platform, including a log collection tool fluent for log collection, a distributed search engine elastic search for log storage and indexing, a visualization platform kibana for visualization pages for log query, a distributed file system glusterFS for providing persistent storage capability for k8s, and a management record file lograte for log clipping and cleaning, so as to prevent the stability of service operation from being affected by the backlog of service logs.
in a specific implementation, the received natural language processing service request is responded, a natural language processing service interface is called through a gateway, and natural language processing is performed on data to be processed, wherein the natural language processing service interface is a service interface which is registered and issued in advance to a unified service gateway of the urban portal system and is used for providing natural language processing service.
The natural language processing refers to an artificial intelligence technology based on deep learning and large-scale sample training, and can perform multi-aspect processing on text contents and extract required information for analysis so as to realize functions of accurately understanding languages and semantics.
The received natural language processing service request may include different service requests, the different service requests correspond to different natural language processing service interfaces, and at this time, the natural language processing service interface matched with the type of the service request may be called to process the data to be processed.
In one embodiment of the present invention, step 302 may include the following sub-steps:
a substep S11, responding to the comment opinion extraction request, calling the comment opinion extraction interface to perform opinion extraction processing on the comment data to obtain comment opinions;
specifically, the natural language processing service request may include a comment viewpoint extraction request, the natural language processing service interface invoked in response to the comment viewpoint extraction request may be a comment viewpoint extraction interface, the service provided by the comment viewpoint extraction interface may be a comment viewpoint extraction service for natural language, and at this time, the comment viewpoint extraction service may be adopted to perform viewpoint extraction processing on the comment data to obtain a comment viewpoint in the comment data.
In a preferred embodiment, the opinion extraction process refers to the automatic extraction and analysis of opinion opinions, the analysis of public opinions and the understanding of published opinions, and the support of product optimization and marketing decisions. The method mainly comprises the steps of automatically analyzing comment attention points and comment viewpoints, and outputting comment viewpoint labels and comment viewpoint polarities.
In an embodiment of the present invention, in one case, for the extraction of a single comment viewpoint, a user is required to provide a single comment, for example, a single user comment "taste super good, service is good", at this time, emotional viewpoints therein can be analyzed, and a comment viewpoint label is extracted, and for the above user comment, the extracted comment viewpoint label may be "taste good" and "service good", wherein the polarity of the comment viewpoint label may include two polarities of good comment and bad comment, and the labels with different polarities may be appropriately distinguished by color or other forms; in another case, for mining the product review opinions, a user is required to provide several reviews of a single entity, for example, there may be an entity review set input by the user, and the review set may include several reviews "taste super good, service is good", "waiter service enthusiasm, taste is good, next time comes," his environment is good, service is good "," is a very good restaurant in the nearest place of my family ", etc., at this time, each review in the entity review set may be extracted from the reviews, and finally the overall review label of the entity is obtained through summarization, for the above entity review set, the summarized label may be" taste (10) "" service is good (8) "" environment is still running (4) ", and the polarity of the review inexpensive review label may include two polarities of good review and bad review, and the different labels may be distinguished by color or other means.
A substep S12, responding to the emotional tendency analysis request, calling the emotional tendency analysis interface to perform emotional tendency analysis processing on the text data, and generating an emotional tendency analysis result;
in another case, the natural language processing service request may include an emotional tendency analysis request, the natural language processing service interface invoked in response to the emotional tendency analysis request may be an emotional tendency analysis interface, the service provided by the emotional tendency analysis interface may be an emotional tendency analysis service for natural language, and at this time, the emotional tendency analysis service may be used to perform emotional tendency analysis processing on the text data to generate an emotional tendency analysis result according to the text data.
The text data subjected to the emotional tendency analysis processing can be a text containing subjective information, and at this time, the emotional polarity judgment can be performed on the text containing the subjective information to obtain an emotional tendency analysis result, so that basic technical support can be provided for applications such as public praise analysis, topic monitoring, public opinion analysis and the like.
Substep S13, responding to the article label analysis request, calling the article label analysis interface to perform label analysis processing on the article data to obtain a label analysis result;
in another case, the natural language processing service request may include an article tag analysis request, the natural language processing service interface invoked in response to the emotional tendency analysis request may be an article tag analysis interface, and the service provided by the article tag analysis interface may be an article tag analysis service for natural language, at which time, the article tag analysis service may be used to perform tag analysis processing on article data to generate a tag analysis result according to the article data.
Specifically, the tag analysis processing of the article data can be mainly expressed as the analysis of core keywords of the article to obtain a tag analysis result, and can provide technical support for news personalized recommendation, similar article aggregation, text content analysis and the like.
Substep S14, responding to the article classification request, calling the article classification interface to classify the article data to obtain an article classification result;
in another case, the natural language processing service request may include an article classification request, the natural language processing service interface invoked in response to the article classification request may be an article classification interface, a service provided by the article classification interface may be an article classification service for natural language, and at this time, the article classification service may be used to classify the article data, so as to generate an article classification result according to the article data.
The classification processing of the article data can be mainly expressed as automatic classification of the articles according to content types to obtain article classification results, and can provide basic technical support for applications such as article clustering and text content analysis.
Substep S15, responding to the text error correction request, calling the text error correction interface to perform text error correction processing on the text data to obtain error-corrected text data;
in another case, the natural language processing service request may include a text error correction request, the natural language processing service interface invoked in response to the text error correction request may be a text error correction interface, the service provided by the text error correction interface may be a text error correction service for natural language, and at this time, the text error correction service may be adopted to perform text error correction processing on the text data to obtain error-corrected text data in the text data.
The text error correction processing on the text data can be mainly expressed as identifying the fragments with errors in the text, and can give out correct suggested text contents after error prompt, namely the corrected text data.
And a substep S16, responding to the text auditing request, calling the text auditing interface to perform text auditing processing on the text data, and obtaining non-compliance information.
In another case, the natural language processing service request may include a text audit request, the natural language processing service interface invoked in response to the text audit request may be a text audit interface, a service provided by the text audit interface may be a text audit service for natural language, and at this time, the text audit service may be adopted to perform text audit processing on the text data to obtain non-compliance information in the text data.
Specifically, the text auditing processing of the text data can be realized mainly through an artificial intelligent text auditing technology based on deep learning and large-scale sample training, and the non-compliance information in the text, such as products with yellow and administrative functions for identifying sensitive and non-compliance information, such as pornography, obscenities, terrorism, politics and the like, can be accurately detected to obtain the non-compliance information, so that more reliable content safety guarantee can be provided for application.
In a preferred embodiment, when the natural language processing platform implements the above-mentioned viewpoint extraction processing on the comment data, the emotional tendency analysis processing on the text data, the label analysis processing on the article data, the classification processing on the article data, the text error correction processing on the text data, and the processing operation of the text audit processing on the text data, the natural language processing platform can be implemented together by means of the algorithm functions of the chinese participle, the part of speech tagging, the named entity recognition, the dependency syntax analysis, the word vector, the word meaning similarity, and the short text similarity of the algorithm service layer.
The Chinese word segmentation refers to a process of segmenting continuous natural language texts into word sequences with semantic rationality and integrity; part-of-speech tagging may refer to the process of assigning a part-of-speech to each vocabulary in a natural language text; named entity recognition, namely named name recognition, can refer to recognition of entities with specific meanings in natural language texts, and mainly comprises names of people, names of places, names of organizations, time and date and the like; the dependency syntax analysis can refer to inputting a Chinese sentence, namely obtaining dependency syntax structure information of the sentence, representing the syntax structure information of words (such as structural relations of a predicate, a dynamic guest, a fixed center and the like) by utilizing the dependency relation between words in the sentence, and representing the structure of the whole sentence (such as a predicate guest, a fixed shape complement and the like) by using a tree structure; the word vector refers to that a single Chinese word is input, so that the vector representation of the word can be obtained, the calculation of the word vector can be realized by a training method, and words in a language word list can be mapped into a vector with fixed length mainly by relying on massive high-quality data and a deep neural network technology; the word meaning similarity can be calculated by relying on massive high-quality data and a deep neural network technology and through word vectorization; the short text similarity refers to the semantic similarity between two input Chinese short texts and can be output, and the short text similarity can help to quickly realize applications such as recommendation, retrieval, sequencing and the like.
As an example, when text error correction processing is performed on text data, firstly, continuous text data can be segmented into vocabulary sequences with semantic rationality and integrity through Chinese word segmentation, and the vocabulary sequences which do not meet the semantic rationality and integrity are labeled; then, through dependency syntax analysis, obtaining dependency syntax structure information aiming at a pair of vocabulary sequences with semantic rationality and completeness, such as dependency relationships between words in a sentence (such as structural relationships of a predicate, a dynamic guest, a fixed form and the like) and structures of the whole sentence (such as a predicate guest, a fixed form and the like); after the dependency grammar structure information of the vocabulary sequence is obtained, the vocabulary subsequences which do not accord with the grammar structure in the vocabulary sequence with semantic rationality and integrity can be labeled through the grammar structure information; at the moment, similar word replacement and semantic replacement among texts can be carried out on the previously marked vocabulary sequences which do not accord with semantic rationality and integrity and the vocabulary subsequences which do not accord with a grammatical structure through the word sense similarity and the short text similarity; after the replacement, the processing operations such as Chinese word segmentation and dependency syntactic analysis can be carried out on the replaced vocabulary sequence again until the vocabulary sequence with semantic rationality and integrity and conforming to the syntactic structure is obtained, and the obtained vocabulary sequence is used as the corrected text data and is sent to the user.
It should be noted that, in implementing natural language processing, the embodiments of the present invention are not limited to invoking different natural language processing service interfaces, adopting different natural language processing service algorithms, and invoking combinations between different service interfaces and using combinations between different natural language processing service algorithms.
In an embodiment of the present invention, after receiving a natural language processing service request provided by a user of another service platform and acquiring data to be processed specified by the user, and calling a natural language processing service interface corresponding to the natural language processing service, to perform natural language processing on the data to be processed, the natural language processing platform may send a processed result to the user, so as to implement natural language processing service provided by the natural language processing platform deployed in the developer portal system for the user of the other service platform in the city portal system.
In the embodiment of the invention, the natural language processing platform deployed in the developer portal system can receive natural language processing service requests provided by users of a plurality of service platforms in the urban portal system, acquire to-be-processed data specified by the users who provide the requests, execute natural language processing service on the acquired to-be-processed data by calling corresponding natural language processing service interfaces, and return processed processing results to the users who provide the requests. In the urban portal system, the natural language processing service requests sent by other service platforms are received by the natural language processing platform deployed in the developer portal system, and the natural language processing service interface is called for processing, so that users of other service platforms of the urban portal system can obtain processing results obtained after natural language processing.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 5, a block diagram of a natural language processing apparatus according to an embodiment of the present invention is shown, and is applied to a natural language processing platform, where the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in a city portal system, and a natural language processing service interface provided by the natural language processing platform is registered in advance in a unified service gateway of the city portal system and is published; the city portal system further comprises a plurality of service platforms, and specifically comprises the following modules:
a service request receiving module 501, configured to receive natural language processing service requests provided by users of the multiple service platforms;
a to-be-processed data obtaining module 502, configured to obtain to-be-processed data specified by the user from a service platform to which the user belongs;
a processing service interface calling module 503, configured to respond to the natural language processing service request, and call a corresponding natural language processing service interface to execute a natural language processing service on the data to be processed, so as to obtain a processing result;
a processing result sending module 504, configured to send the processing result to the user.
In an embodiment of the present invention, the natural language processing service interface includes a comment opinion extraction interface, the data to be processed includes comment data, and the natural language processing service request includes a comment opinion extraction request;
the processing service interface calling module 503 may include the following sub-modules:
and the first processing service submodule is used for responding to the comment viewpoint extraction request, calling the comment viewpoint extraction interface to perform viewpoint extraction processing on the comment data, and obtaining comment viewpoints.
In an embodiment of the present invention, the natural language processing service interface includes an emotional tendency analysis interface, the data to be processed is text data, and the natural language processing service request includes an emotional tendency analysis request;
the processing service interface calling module 503 may include the following sub-modules:
and the second processing service submodule is used for responding to the emotional tendency analysis request, calling the emotional tendency analysis interface to perform emotional tendency analysis processing on the text data, and generating an emotional tendency analysis result.
In an embodiment of the present invention, the natural language processing service interface includes an article tag analysis interface, the data to be processed is article data, and the natural language processing service request includes an article tag analysis request;
the processing service interface calling module 503 may include the following sub-modules:
and the third processing service sub-module is used for responding to the article label analysis request, calling the article label analysis interface to perform label analysis processing on the article data, and obtaining a label analysis result.
In an embodiment of the present invention, the natural language processing service interface includes an article classification interface, the data to be processed is article data, and the natural language processing service request includes an article classification request;
the processing service interface calling module 503 may include the following sub-modules:
and the fourth processing service sub-module is used for responding to the article classification request, calling the article classification interface to classify the article data and obtaining an article classification result.
In an embodiment of the present invention, the natural language processing service interface includes a text error correction interface, the data to be processed is text data, and the natural language processing service request includes a text error correction request;
the processing service interface calling module 503 may include the following sub-modules:
and the fifth processing service sub-module is used for responding to the text error correction request and calling the text error correction interface to perform text error correction processing on the text data to obtain error-corrected text data.
In an embodiment of the present invention, the natural language processing service interface includes a text audit interface, the data to be processed is text data, and the natural language processing service request includes a text audit request;
the processing service interface calling module 503 may include the following sub-modules:
and the sixth processing service sub-module is used for responding to the text auditing request, calling the text auditing interface to perform text auditing processing on the text data, and obtaining non-compliance information.
For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.
An embodiment of the present invention further provides an apparatus, including:
the natural language processing method comprises a processor, a memory and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, each process of the natural language processing method embodiment is realized, the same technical effect can be achieved, and in order to avoid repetition, the description is omitted here.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the natural language processing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, the description is not repeated here.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.
The natural language processing method and the natural language processing device provided by the invention are described in detail, and the principle and the implementation mode of the invention are explained by applying specific examples, and the description of the examples is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A natural language processing method is characterized in that the method is applied to a natural language processing platform, the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in a city portal system, and a natural language processing service interface provided by the natural language processing platform is registered in advance in a unified service gateway of the city portal system and is issued; the city portal system further comprises a plurality of service platforms, and the method comprises the following steps:
receiving natural language processing service requests provided by users of the plurality of business platforms;
acquiring the data to be processed appointed by the user from a service platform to which the user belongs;
responding to the natural language processing service request, and calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result;
and sending the processing result to the user.
2. The method of claim 1, wherein the natural language processing service interface comprises a comment opinion extraction interface, wherein the data to be processed comprises comment data, and wherein the natural language processing service request comprises a comment opinion extraction request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the comment viewpoint extraction request, and calling the comment viewpoint extraction interface to perform viewpoint extraction processing on the comment data to obtain comment viewpoints.
3. The method of claim 1, wherein the natural language processing service interface comprises an emotional tendency analysis interface, the data to be processed is text data, and the natural language processing service request comprises an emotional tendency analysis request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
responding the emotional tendency analysis request, calling the emotional tendency analysis interface to perform emotional tendency analysis processing on the text data, and generating an emotional tendency analysis result.
4. The method of claim 1, wherein the natural language processing service interface comprises an article tag analysis interface, the data to be processed is article data, and the natural language processing service request comprises an article tag analysis request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding the article label analysis request, and calling the article label analysis interface to perform label analysis processing on the article data to obtain a label analysis result.
5. The method of claim 1, wherein the natural language processing service interface comprises an article classification interface, the data to be processed is article data, and the natural language processing service request comprises an article classification request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the article classification request, and calling the article classification interface to classify the article data to obtain an article classification result.
6. The method of claim 1, wherein the natural language processing service interface comprises a text error correction interface, the data to be processed is text data, and the natural language processing service request comprises a text error correction request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the text error correction request, and calling the text error correction interface to perform text error correction processing on the text data to obtain error-corrected text data.
7. The method of claim 1, wherein the natural language processing service interface comprises a text audit interface, the data to be processed is text data, and the natural language processing service request comprises a text audit request;
the responding the natural language processing service request, calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result, and the processing result comprises the following steps:
and responding to the text auditing request, and calling the text auditing interface to perform text auditing processing on the text data to obtain non-compliance information.
8. A natural language processing device is applied to a natural language processing platform, the natural language processing platform is deployed in a developer portal system, the developer portal system is integrated in a city portal system, and a natural language processing service interface provided by the natural language processing platform is registered in a unified service gateway of the city portal system in advance and issued; the city portal system further comprises a plurality of service platforms, and the device comprises:
the service request receiving module is used for receiving natural language processing service requests provided by users of the plurality of service platforms;
a to-be-processed data acquisition module, configured to acquire to-be-processed data specified by the user from a service platform to which the user belongs;
the processing service interface calling module is used for responding to the natural language processing service request and calling a corresponding natural language processing service interface to execute natural language processing service on the data to be processed to obtain a processing result;
and the processing result sending module is used for sending the processing result to the user.
9. An apparatus, comprising: processor, memory and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the natural language processing method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the natural language processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011038685.0A CN112287104A (en) | 2020-09-28 | 2020-09-28 | Natural language processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011038685.0A CN112287104A (en) | 2020-09-28 | 2020-09-28 | Natural language processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112287104A true CN112287104A (en) | 2021-01-29 |
Family
ID=74422618
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011038685.0A Pending CN112287104A (en) | 2020-09-28 | 2020-09-28 | Natural language processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112287104A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117290429A (en) * | 2023-11-24 | 2023-12-26 | 山东焦易网数字科技股份有限公司 | Method for calling data system interface through natural language |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1909685A (en) * | 2006-07-18 | 2007-02-07 | 华为技术有限公司 | Method and system for realizing separation of service platform and gate |
| CN101124540A (en) * | 2005-01-04 | 2008-02-13 | 瓦卡亚技术私人有限公司 | System and method for application development and deployment |
| US9015730B1 (en) * | 2013-12-17 | 2015-04-21 | International Business Machines Corporation | Natural language access to application programming interfaces |
| CN110705296A (en) * | 2019-09-12 | 2020-01-17 | 华中科技大学 | Chinese natural language processing tool system based on machine learning and deep learning |
| CN111090730A (en) * | 2019-12-05 | 2020-05-01 | 中科数智(北京)科技有限公司 | Intelligent voice scheduling system and method |
-
2020
- 2020-09-28 CN CN202011038685.0A patent/CN112287104A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101124540A (en) * | 2005-01-04 | 2008-02-13 | 瓦卡亚技术私人有限公司 | System and method for application development and deployment |
| CN1909685A (en) * | 2006-07-18 | 2007-02-07 | 华为技术有限公司 | Method and system for realizing separation of service platform and gate |
| US9015730B1 (en) * | 2013-12-17 | 2015-04-21 | International Business Machines Corporation | Natural language access to application programming interfaces |
| CN110705296A (en) * | 2019-09-12 | 2020-01-17 | 华中科技大学 | Chinese natural language processing tool system based on machine learning and deep learning |
| CN111090730A (en) * | 2019-12-05 | 2020-05-01 | 中科数智(北京)科技有限公司 | Intelligent voice scheduling system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117290429A (en) * | 2023-11-24 | 2023-12-26 | 山东焦易网数字科技股份有限公司 | Method for calling data system interface through natural language |
| CN117290429B (en) * | 2023-11-24 | 2024-02-20 | 山东焦易网数字科技股份有限公司 | Method for calling data system interface through natural language |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Braun et al. | Evaluating natural language understanding services for conversational question answering systems | |
| US10146878B2 (en) | Method and system for creating filters for social data topic creation | |
| Kumar et al. | Analyzing Twitter sentiments through big data | |
| CN102880649B (en) | A kind of customized information disposal route and system | |
| Wijeratne et al. | Emojinet: Building a machine readable sense inventory for emoji | |
| WO2018036239A1 (en) | Method, apparatus and system for monitoring internet media events based on industry knowledge mapping database | |
| CN112749284B (en) | Knowledge graph construction method, device, equipment and storage medium | |
| US20180293302A1 (en) | Natural question generation from query data using natural language processing system | |
| CN108305180B (en) | Friend recommendation method and device | |
| CN112287270A (en) | Content auditing method and device | |
| CN107436916B (en) | Intelligent answer prompting method and device | |
| CN110692042B (en) | Platform selection to perform requested actions in an audio-based computing environment | |
| Qundus et al. | AI supported topic modeling using KNIME-workflows | |
| Garcia et al. | Data Migrations: Exploring the Use of Social Media Data as Evidence for Human Rights Advocacy. | |
| CN112287104A (en) | Natural language processing method and device | |
| KR102243275B1 (en) | Method, device and computer readable storage medium for automatically generating content regarding offline object | |
| CN116701604A (en) | Question and answer corpus construction method and device, question and answer method, equipment and medium | |
| Kanakaraddi et al. | Sentiment Analysis of Covid-19 Tweets Using Machine Learning and Natural Language Processing | |
| CN114328947A (en) | Knowledge graph-based question and answer method and device | |
| KR20230059364A (en) | Public opinion poll system using language model and method thereof | |
| Xavier et al. | A basic approach for extracting and analyzing data from Twitter | |
| KR20170136679A (en) | Method for recommending konwledge contents based on big data and system at the same | |
| Purohit et al. | An information filtering and management model for twitter traffic to assist crises response coordination | |
| Xu et al. | The study of content security for mobile internet | |
| CN113032586B (en) | Method and device for extracting time information in text and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210129 |
|
| RJ01 | Rejection of invention patent application after publication |