CN114547350A - Information processing method, device, equipment and storage medium - Google Patents

Abstract

The embodiment of the application discloses an information processing method, an information processing device, information processing equipment and a storage medium, wherein the method comprises the following steps: acquiring an application to be accessed; in response to an access operation for the application to be accessed, determining multimodal access information associated with the application to be accessed; and classifying the multi-modal access information to obtain target access information.

Description

Information processing method, device, equipment and storage medium

Technical Field

The embodiments of the present application relate to the field of information technologies, and in particular, to an information processing method, apparatus, device, and storage medium.

Background

In the related art, access information generated by access is determined based on text information or image information appearing in the process of accessing an application, and the accuracy of the obtained access information is not high enough.

Disclosure of Invention

The embodiment of the application provides an information processing technical scheme.

The technical scheme of the embodiment of the application is realized as follows:

an embodiment of the present application provides an information processing method, including:

acquiring an application to be accessed;

in response to an access operation for the application to be accessed, determining multimodal access information associated with the application to be accessed;

and classifying the multi-mode access information to obtain target access information.

In some embodiments, the determining, in response to the access operation for the application to be accessed, multi-modal access information associated with the application to be accessed includes: responding to the access operation aiming at the application to be accessed, and acquiring a page to be processed generated in the process that the application to be accessed is accessed; and extracting information of the page information in the page to be processed to obtain the multi-modal access information. Thus, richer and more accurate multi-mode access information can be obtained.

In some embodiments, the multimodal access information includes at least one of: text information, image information, video information, audio information, access behavior information, access events. Therefore, by acquiring information of various different forms, richer access information, namely multi-mode access information, is obtained.

In some embodiments, before the classifying the multi-modal access information to obtain the target access information, the method further comprises: determining an application scene of the application to be accessed; constructing a knowledge graph matched with the application scene; the classifying the multi-modal access information to obtain target access information comprises: and classifying the multi-modal information based on the knowledge graph to obtain the target access information. Therefore, the obtained multi-modal access information is classified based on the constructed knowledge graph to obtain more accurate target access information so as to be convenient for subsequent management and retrieval of relevant access information.

In some embodiments, the classifying the multimodal information based on the knowledge-graph to obtain the target access information comprises: performing information processing on the multi-modal access information to obtain a processing result; and classifying the processing result based on the knowledge graph to obtain the target access information. Therefore, the framework system in the determined target access information can be more perfect, so that the target access information can be managed and retrieved in the following process.

In some embodiments, in a case that the access operation includes an access sub-operation set, before performing information processing on the multi-modal access information to obtain a processing result, the method further includes: determining a target access sub-operation from the access sub-operation set based on a preset access rule; determining intermediate information associated with a target access sub-operation in the multi-modal information; the information processing of the multi-modal access information to obtain a processing result comprises: and carrying out information processing on the intermediate information to obtain the processing result. Therefore, information screening and processing can be performed on the obtained multi-modal information, so that more refined target access information can be obtained in the following.

In some embodiments, the information processing includes at least one of: text parsing, image recognition, video processing, audio recognition, behavior classification, and event analysis. Therefore, the information processing process can be more accurate.

In some embodiments, after classifying the multi-modal access information to obtain target access information, the method further comprises: and in response to receiving a display instruction, displaying at least one of the target access information and recommendation information associated with the target access information. Therefore, the relevant users can timely acquire the relevant information generated in the access process.

In some embodiments, the method further comprises: generating a preset virtual assistant matched with the application to be accessed based on the application scene of the application to be accessed; and adjusting the image parameters of the preset virtual assistant based on at least one of the access operation and the multi-modal access information to obtain and display the target virtual assistant. Therefore, the user experience in the application access process can be improved.

An embodiment of the present application provides an information processing apparatus, the apparatus including:

the acquisition module is used for acquiring the application to be accessed;

the determining module is used for responding to the access operation aiming at the application to be accessed, and acquiring multi-modal access information associated with the application to be accessed;

and the classification module is used for classifying the multi-mode access information to obtain target access information.

The embodiment of the application provides computer equipment, which comprises a memory and a processor, wherein computer executable instructions are stored on the memory, and the processor can realize the information processing method when running the computer executable instructions on the memory.

The embodiment of the application provides a computer storage medium, wherein computer-executable instructions are stored on the computer storage medium, and after being executed, the computer-executable instructions can realize the information processing method.

The embodiment of the application provides an information processing method, an information processing device, information processing equipment and a storage medium, wherein an application to be accessed is obtained firstly; then responding to the access operation aiming at the application to be accessed, and determining multi-modal access information associated with the application to be accessed; and finally classifying the determined multi-modal access information to obtain target access information. In this way, the accuracy and precision of determining access records and access information generated during access to an application can be improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without inventive efforts, wherein:

fig. 1 is a schematic flowchart of a first information processing method according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a second information processing method according to an embodiment of the present application;

fig. 3 is a schematic flowchart of a third information processing method according to an embodiment of the present application;

fig. 4 is a schematic flowchart illustrating an implementation of access information determination by applying the information processing method according to the embodiment of the present application;

fig. 5 is a schematic flowchart illustrating another process of determining access information by applying the information processing method according to the embodiment of the present application;

fig. 6 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, specific technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the examples of the present application, but are not intended to limit the scope of the examples of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, so as to enable the embodiments of the application described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application belong. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of embodiments of the present application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

Knowledge graph: the Knowledge domain visualization or Knowledge domain mapping map is a series of different graphs displaying the relationship between the Knowledge development process and the structure, describes Knowledge resources and carriers thereof by using a visualization technology, and excavates, analyzes, constructs, draws and displays Knowledge and the mutual relation between the Knowledge resources and the carriers.

An exemplary application of the information processing device provided in the embodiment of the present application is described below, and the information processing method provided in the embodiment of the present application can be applied to various types of user terminals such as a notebook computer, a tablet computer, a desktop computer, a camera, a mobile device (e.g., a personal digital assistant, a dedicated messaging device, a portable game device) and the like having a data processing function, and can also be implemented as a server.

The method can be applied to a computer device, and the functions realized by the method can be realized by calling a program code by a processor in the computer device, although the program code can be stored in a computer storage medium, which at least comprises the processor and the storage medium.

An information processing method is provided in an embodiment of the present application, and as shown in fig. 1, is a schematic flow chart of a first information processing method provided in the embodiment of the present application; the description is made with reference to the steps shown in fig. 1:

and step S101, acquiring the application to be accessed.

In some embodiments, the application to be accessed may be any application capable of running on the electronic device; the applications to be accessed can be divided according to applications, such as network communication applications, word processing applications, video/audio playing applications, spreadsheet applications and the like, and can also be divided according to application scenes, such as teaching applications, life applications, work applications, entertainment applications and the like.

In some embodiments, the application to be accessed is obtained, which may be selected from the electronic device according to a preset rule, or an application associated with an operation of an operation user on the electronic device, that is, the application to be accessed, is determined in response to the operation of the operation user on the electronic device; in some embodiments, the user may start a video playing application, that is, an application to be accessed, in the electronic device when the user wants to watch a video, or the user may start an online teaching application on the electronic device to learn, so as to obtain the application to be accessed, that is, any teaching application.

In some embodiments, the number of applications to be accessed may be one, or may be two or more, and in the case that the number of applications to be accessed is two or more, different types of applications in the two or more applications to be accessed are different, for example, the types of applications may include: instant messaging applications, and teaching applications.

Step S102, responding to the access operation aiming at the application to be accessed, and determining multi-modal access information associated with the application to be accessed.

In some embodiments, in response to an access operation for the application to be accessed, multi-modal access information associated with the application to be accessed is perceived and obtained; the multi-modal access information may be information obtained and output from an associated server, or generated, when the application to be accessed is accessed.

In some embodiments, multimodal access information, including but not limited to: text information, image information, video information, audio information accessed based on the access operation; meanwhile, some access behaviors or access events for access information carried inside the access operation can be included, such as: long continuous clicks on access information, repeated views periodically, collection of access information, saving or downloading of access information, and the like.

In some embodiments, the application to be accessed is a web browsing application, and the content of the page is analyzed and recorded through adaptive analysis of the content of the page, so as to record the record based on the accessed browsing page and the related access data, thereby obtaining the multi-modal access information.

In some embodiments, the access operation for the application to be accessed may be issued by any operation object, such as: any user operating the electronic device, or any device that interacts information with the electronic device.

In some embodiments, in a case that the application to be accessed includes a plurality of different applications on the electronic device, when the operation object performs related access operations for the plurality of different applications, multi-modal access information is determined, including but not limited to: personal instant messaging information, personal web browsing information, personal document browsing record information, personal voice or short message information, and the like.

And step S103, classifying the multi-modal access information to obtain target access information.

In some embodiments, the multi-modal information is categorized according to the information type of the multi-modal information, or according to the information content of the multi-modal information, and the like, so as to obtain the target access information.

In some embodiments, a preset knowledge graph can be adopted to classify the multi-modal access information so as to obtain target access information; wherein the knowledge-graph may be an application scenario determination for an application to be accessed.

In some embodiments, the multi-modal information may be analyzed first to obtain an analysis result, and then the analysis result is filled into a preset knowledge graph according to a relevant type to obtain the target access information. Wherein the target access information can also be presented in multi-modal information, such as: video, audio, text, etc.

In some embodiments, the information output or displayed during the accessing process by the application to be accessed, i.e., the multimodal information presented during the accessing process, includes but is not limited to: the method comprises the following steps of classifying text information, audio information, video information, image information, access behaviors and access events generated in the access process, and obtaining the access, namely specific and detailed target access information generated between an access object and an application to be accessed in the access process, namely an interaction process; meanwhile, based on the target access information, the relevant information of the current access can be managed and retrieved in the following process.

In some embodiments, when different applications are accessed, all types of information, namely multi-modal information, generated in the accessing process of the different applications can be obtained in real time, and the multi-modal information is classified and processed, so that access intention information corresponding to the access of the different applications is obtained; that is to say, the information can be collected across applications and across modalities, and further, the collected information of multiple modalities of different applications can be displayed or fed back. In this way, the accuracy and precision of determining access records and access information generated during access to an application can be improved.

The embodiment of the application provides an information processing method, which comprises the steps of firstly, obtaining an application to be accessed; then responding to the access operation aiming at the application to be accessed, and determining multi-modal access information associated with the application to be accessed; and finally classifying the determined multi-modal access information to obtain target access information. In this way, the accuracy and precision of determining access records and access information generated during access to an application can be improved.

In some embodiments, based on the access operation, information extraction is performed from the to-be-processed page generated in the process that the to-be-accessed application is accessed, so that multi-modal access information is obtained. Thus, richer and more accurate multi-mode access information can be obtained. That is, step S102 provided in the foregoing embodiment can be implemented by the following steps S201 to S202, as shown in fig. 2, and the following description is made for a flowchart of a second information processing method provided in the embodiment of the present application, with reference to the steps shown in fig. 1 and fig. 2:

step S201, in response to the access operation for the application to be accessed, acquiring a to-be-processed page generated in the process of accessing the application to be accessed.

In some embodiments, in response to an access operation for an application to be accessed, a to-be-processed page generated or displayed in the process of accessing the application to be accessed is acquired; wherein the to-be-processed page may be related information presented on a display device of the electronic device during the to-be-accessed application is accessed. The page to be processed can be changed in real time along with the access operation. And the number of the pages to be processed can also be one, and can also be two or more.

In some embodiments, when the number of the applications to be accessed is two or more, the pages to be processed generated in the process of accessing by different applications to be accessed are different, and the number of the corresponding pages to be processed is also different.

Step S202, extracting information of the page information in the page to be processed to obtain the multi-modal access information.

In some embodiments, the multi-modal access information is obtained by performing information extraction on page content and page background records in the page to be processed and related access operations on the page to be processed. The page information in the page to be processed can be sensed and stored.

In some possible implementations, the multi-modal access information includes a variety of accessed data, including but not limited to the following, i.e., the multi-modal access information includes at least one of:

text information, image information, video information, audio information, access behavior information, access events.

In some embodiments, the text information, the image information, the video information, and the audio information may be information generated by the application to be accessed in the process of being accessed, where the access behavior information may be access behaviors generated in the process of accessing the application to be accessed, such as: clicking for a long time, repeatedly watching the same video, and the like; the access event may be an event generated during the access of the application to be accessed, such as: storing, collecting, commenting, forwarding and the like.

In some embodiments, by acquiring information of a plurality of different forms, richer access information, namely multi-modal access information, is obtained.

In some embodiments, based on the application scenario of the application to be accessed, a corresponding knowledge graph is constructed; and classifying the obtained multi-modal access information based on the constructed knowledge graph to obtain more accurate and accurate target access information so as to manage and retrieve the relevant access information subsequently. That is, before step S103 provided in the above embodiment, the following step S301 and step S302 may be further performed, as shown in fig. 3, for a flowchart of a third information processing method provided in the embodiment of the present application, the following description is made with reference to the steps shown in fig. 1 to fig. 3:

step S301, determining an application scenario of the application to be accessed.

In some embodiments, the corresponding application scenario may be determined based on the type of the application to be accessed, or the corresponding application scenario may be determined based on the function of the application to be accessed.

In some embodiments, when the application to be accessed is a teaching application, determining that the corresponding application scene is a learning scene; and determining that the corresponding application scene is a conference scene under the condition that the application to be accessed is an instant messaging application for conference communication.

And step S302, constructing a knowledge graph matched with the application scene.

In some embodiments, constructing a knowledge graph matching the application scenario may be generating a knowledge graph corresponding to each of a learning scenario, a meeting scenario, a shopping scenario, an entertainment scenario, and the like.

In some embodiments, where the application to be accessed is an instructional type application, the determined knowledge-graph is an instructional architecture knowledge framework including, but not limited to: different types of knowledge system frames, knowledge system frames with different user dimensions, and a general knowledge system frame. In the case that the application to be accessed is a conference-class application, the determined knowledge-graph is a conference-class framework, including but not limited to: different participant system frames, system frames of different posts and the like.

In some embodiments, the knowledge-graph corresponding to different applications to be accessed may be different.

Here, the classification of the multi-modal access information to obtain the target access information can be implemented by the following step S303:

step S303, classifying the multi-modal information based on the knowledge graph to obtain the target access information.

In some embodiments, the multi-modal information can be classified into the corresponding position of the knowledge graph according to different information contents, so as to obtain the target access information.

In some embodiments, the multi-modal access information may be first processed, and then the obtained processing result may be classified based on the knowledge graph to obtain the target access information. Therefore, the framework system in the determined target access information can be more perfect, so that the target access information can be managed and retrieved in the following process. That is, the above step S303 can be realized by the following steps S3031 and S3032 (not shown in the figure):

step S3031, performing information processing on the multi-modal access information to obtain a processing result.

In some embodiments, the obtained multi-modal access information is processed differently, so as to obtain corresponding processing results based on different modal information in the multi-modal access information.

In some possible implementation manners, in the case that the access operation includes an access sub-operation set, a target access sub-operation may be selected from the access sub-operation set, and then corresponding intermediate information is screened from the obtained multi-modal information, and the intermediate information is subjected to information processing to obtain a final processing result. Therefore, information screening and processing can be performed on the obtained multi-modal information, so that more refined target access information can be obtained in the following. That is, before the information processing is performed on the multi-modal access information in the step S3031 to obtain the processing result, the following process may be further performed:

the method comprises the first step of determining a target access sub-operation from the access sub-operation set based on a preset access rule.

In some embodiments, the preset access rule may be set in advance, or may be determined according to attribute information of the application to be accessed or an application scenario. The target access sub-operation is determined from the access sub-operation set based on a preset access rule, and operations such as saving, forwarding, collecting and the like in the access sub-operation set can be determined as the target access sub-operation.

In a second step, in the multimodal information, intermediate information associated with a target access sub-operation is determined.

In some embodiments, from the obtained multi-modal information, information associated with the target access sub-operation is determined as intermediate information, such as a saved picture is determined as intermediate information and a commented video is determined as intermediate information. The intermediate information may also be multi-modal information, that is, may include but is not limited to: image information, video information, text information, voice information, behavior information, event information, and the like.

In some embodiments, when the number of the applications to be accessed is two or more and the attribute information of the applications to be accessed is different, the corresponding intermediate information may be obtained from different applications to be accessed, where the target access sub-operations corresponding to different applications to be accessed may be the same or different.

Here, the step S3031 of performing information processing on the multi-modal access information to obtain a processing result may be implemented by:

and carrying out information processing on the intermediate information to obtain the processing result.

In some embodiments, the screened intermediate information is subjected to information processing to obtain a corresponding processing result; in this way, multimodal information more matched to the access operation can be obtained.

In some possible implementations, information processing includes, but is not limited to, processing information of various modalities in different ways. Therefore, the information processing process can be more accurate. Here, the information processing described above includes at least one of:

text parsing, image recognition, video processing, audio recognition, behavior classification, and event analysis.

Step S3032, classifying the processing result based on the knowledge graph to obtain the target access information.

In some embodiments, the obtained processing results may be classified and archived based on the determined knowledge-graph, thereby obtaining target access information.

In some possible implementation manners, after the multi-modal access information is classified to obtain the target access information, the target access information can be displayed, so that a related user can timely obtain related information generated in the access process. The information processing method provided by the embodiment of the application can also realize the following steps:

in response to receiving a presentation instruction, presenting at least one of the target access information and recommendation information associated with the target access information.

In some embodiments, in response to the received presentation instruction, the presentation instruction may be issued when the operation object operates the electronic device, or may be generated in a case where the target access information is generated, or may be generated based on a related presentation command in the target access information.

In some embodiments, the recommendation information may be recommended advertising data associated with the targeted access information; the target access information and the recommendation information can be displayed according to a preset display rule while being displayed, and the target access information or the recommendation information can be displayed along with the preset display rule when being independently displayed.

Here, in the information processing method provided in the embodiment of the present application, a virtual assistant matching with an application to be accessed may be further generated, and the virtual assistant is adjusted based on an access operation in the access process and determined multimodal access information, so as to improve a user experience in the access process of the application, that is, the method may be implemented by:

firstly, generating a preset virtual assistant matched with the application to be accessed based on the application scene of the application to be accessed.

In some embodiments, the application context of the application to be accessed may be based on, for example: learning a scene, and generating a corresponding preset virtual assistant, such as: a virtual student image; the application scenario may be based on the application to be accessed, such as: and (3) generating a corresponding preset virtual assistant in the shopping scene, wherein the preset virtual assistant comprises the following steps: a virtual shopping cart image.

And secondly, adjusting the image parameters of the preset virtual assistant based on at least one of the access operation and the multi-modal access information to obtain and display the target virtual assistant.

In some embodiments, the preset virtual assistant image parameters, such as the expression, the action, the display color, the display brightness, and the display duration of the preset virtual assistant, may be adjusted based on the access operation and/or the multimodal access information of the access process, so as to obtain a target virtual assistant dynamically matching the access operation and/or the multimodal access information, and similarly, the target virtual assistant may also be synchronously displayed in the access process, that is, in the presence of the access operation, the target virtual assistant is dynamically displayed.

The information processing method is described below with reference to a specific embodiment, but it should be noted that the specific embodiment is only for better describing the embodiments of the present application and should not be construed as an inappropriate limitation to the embodiments of the present application.

In the related art, the following service contexts, for example: methods of how to help teachers/listeners manage respective information and various states in real time during online retrieval of material scenes, retrieval of information between different applications in electronic devices, or online or offline interactive learning, and in mobile devices, such as: the mobile phone is used as an intelligent terminal to interact with the generated content, behavior, event and schedule management and cooperation; it is common to understand and parse monomodal content while primarily utilizing natural language processing, text processing, user intent recognition, and the like to achieve user intent and text interaction. This makes the resulting user intent and text interaction less accurate.

Based on this, the embodiment of the application provides an information processing method, which can dynamically acquire multi-modal information associated with a relevant application in real time in the process of accessing the relevant application, and fuse and analyze the multi-modal information, so as to obtain richer and more accurate access information and interaction information in the process of accessing the application, and further more accurately identify an access intention. The scheme is mainly realized by the following steps:

firstly, constructing an application-driven content or action knowledge graph according to an application scene corresponding to an accessed application.

And secondly, in the process of accessing the application, collecting the multi-mode information generated by the interaction of the two parties in the accessing process in real time, wherein the multi-mode information comprises but is not limited to: relevant information generated by the accessed application, such as: text information, voice information, image information, video information, etc., and a behavior or event corresponding to the access operation.

Thirdly, processing the multi-modal information, which mainly comprises the following steps: processing information; information identification; analyzing information; information classification, which may specifically involve processing different information in different ways, includes: speech recognition, text understanding or parsing, image recognition or video processing, behavior classification, event analysis; and further obtain the corresponding processing result.

And fourthly, classifying and archiving processing results obtained after the multi-mode information is processed according to the knowledge graph determined in the first step, and sequentially establishing a final knowledge graph generated in the access process, namely the target access information. And simultaneously, based on the final knowledge graph, the corresponding access information can be conveniently and rapidly retrieved and managed subsequently.

As shown in fig. 4, a schematic flowchart of a process for implementing access information determination by applying the information processing method provided in the embodiment of the present application is shown; here, the information processing method provided by the embodiment of the present application may be implemented by default as an agent service (assistant service) inside the electronic device; first, starting an assistant service 401, that is, implementing the following operations based on the assistant service; 402, setting the service range, service duration, service application authority and the like of the assistant service after the assistant service is started; secondly, after the service is started, the assistant service is adopted to receive multimodal information in real time, that is, 403, for example, data generated in the process of accessing a plurality of different applications inside the electronic device can be acquired, including but not limited to: instant messaging information, browsed web page information, browsed document information, and voice/short message information. And after obtaining the multimodal information, continue processing 404 the multimodal information with the assistant service, including but not limited to: natural language processing, web page information crawling, personal document information parsing, voice recognition/understanding, image information recognition/understanding, behavior interaction recognition/understanding, and the like. Finally, the assistant service can be employed to extract 405 the data after multimodal information processing based on preset rules or preset key parameters, extract the information summary, and generate 406 the final access information from the information summary. Here, the information extraction through 405 may make the obtained data of the service more accurate for subsequent efficient access, management and retrieval.

Similarly, as shown in fig. 5, a schematic flow chart for implementing another access information determination by applying the information processing method provided in the embodiment of the present application is shown; the information processing method provided by the embodiment of the application can be used in the online teaching process, for example, in the process of starting the online teaching application, the assistant service is used for performing multi-mode information acquisition, processing, classification, display and the like on the teaching process, so that the teaching information with richer information amount and more specific information system in the teaching process is obtained.

First, an online tutoring process is initiated 501, while the current tutoring content may be synchronously acquired 502, including but not limited to: in the teaching process, a teacher main body identity image, a student identity image and teaching course contents are displayed; secondly, based on the obtained current teaching content, a corresponding knowledge graph is established, that is, a knowledge framework 503 is established: different types of knowledge systems, different users or different dimensions of knowledge structures, common information framework structures, etc.

Thirdly, in the teaching process, the interactive both-party information 504 is obtained in real time, including but not limited to: interactive question and answer information, student/teacher status information and teacher course content information; and the obtained interactive two-party information, namely the multi-modal information, is synchronously processed 505, and the obtained multi-modal information is classified based on the established knowledge framework to obtain a corresponding teaching and auxiliary information set 506. Here, the classification into two broad categories can be based on different users (teachers and students): 5061 to the student, one can get: personalized key prominent learning notes, automatic summarization of classroom key abstracts and personal wrong question recording; for the teacher 5062, we can get: the teaching aid comprises a teaching record of different students, a classroom interaction information record abstract, a teaching negligence point summary, a post-class teaching tutor content abstract and the like.

Finally, information feedback can be performed based on the angle dimension information, i.e., 507, information feedback of different dimensions can be performed based on teachers and students, and simultaneously, intelligent data multi-dimensional analysis 508 and generation of teaching or learning assistant reports 509 can also be performed.

Based on this, the information processing method provided by the embodiment of the application can record the record, the related behavior and the data of the browsing page by performing adaptive analysis on the page content corresponding to the application in the process of accessing or serving the application of the access or service; multi-mode information in the cross-page and cross-application perception access process can be realized; simultaneously, multi-modal information can be synchronously processed, including but not limited to: document identification, text understanding and analysis of the web page content, identification and classification of access behaviors in the access process, information collection and arrangement in the access process and the like are carried out on the document content. Therefore, more accurate and richer access data with more information quantity can be obtained.

In addition, virtual assistants corresponding to browsing relevant pages or accessing relevant users can be synchronously generated, and parameters of the virtual assistants can be dynamically adjusted according to access behaviors and information generated in the access process. And multi-mode information acquired in the access application and advertisement information related to the multi-mode information can be dynamically displayed according to preset related rules and/or parameters.

Fig. 6 is a schematic structural diagram of an information processing apparatus according to an embodiment of the present application, and as shown in fig. 6, the information processing apparatus 600 includes: an acquisition module 601, a determination module 602 and a classification module 603; wherein:

an obtaining module 601, configured to obtain an application to be accessed;

a determining module 602, configured to, in response to an access operation for the application to be accessed, obtain multimodal access information associated with the application to be accessed;

and the classifying module 603 is configured to classify the multi-modal access information to obtain target access information.

In some embodiments, the determining module 602 is further configured to, in response to an access operation for the application to be accessed, obtain a to-be-processed page generated in a process in which the application to be accessed is accessed; and extracting information of the page information in the page to be processed to obtain the multi-modal access information.

In some embodiments, the multimodal access information includes at least one of: text information, image information, video information, audio information, access behavior information, access events.

In some embodiments, the information processing apparatus 600 further includes: the building module is used for determining the application scene of the application to be accessed; constructing a knowledge graph matched with the application scene; the classifying module 603 is further configured to classify the multimodal information based on the knowledge graph to obtain the target access information.

In some embodiments, the classifying module 603 is further configured to perform information processing on the multi-modal access information to obtain a processing result; and classifying the processing result based on the knowledge graph to obtain the target access information.

In some embodiments, in a case that the access operation includes an access sub-operation set, the determining module 602 is further configured to determine a target access sub-operation from the access sub-operation set based on a preset access rule; determining intermediate information associated with a target access sub-operation in the multi-modal information; the classifying module 603 is further configured to perform information processing on the intermediate information to obtain the processing result.

In some embodiments, the information processing includes at least one of: text parsing, image recognition, video processing, audio recognition, behavior classification, and event analysis.

In some embodiments, the information processing apparatus 600 further includes: and the display module is used for responding to the received display instruction and displaying at least one of the target access information and the recommendation information related to the target access information.

In some embodiments, the information processing apparatus 600, the adjusting module, is configured to generate a preset virtual assistant matching the application to be accessed based on an application scenario of the application to be accessed; and adjusting the image parameters of the preset virtual assistant based on at least one of the access operation and the multi-modal access information to obtain and display the target virtual assistant.

It should be noted that the above description of the apparatus-side embodiment is similar to the description of the method embodiment, and has similar beneficial effects as the method embodiment. For technical details which are not disclosed in the device-side embodiments of the present application, reference is made to the description of the method embodiments of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the data display method is implemented in the form of a software functional module and sold or used as a standalone product, the data display method may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a terminal, a server, etc.) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a hard disk drive, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present application further provides a computer program product, where the computer program product includes computer-executable instructions, and after the computer-executable instructions are executed, the information processing method provided by the embodiment of the present application can be implemented.

Accordingly, an embodiment of the present application provides a computer device, fig. 7 is a schematic structural diagram of the computer device provided in the embodiment of the present application, and as shown in fig. 7, the computer device 700 includes: a processor 701, at least one communication bus 704, a communication interface 702, at least one external communication interface, and a memory 703. Wherein communication interface 702 is configured to enable connected communication between these components. The communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface, among others. The processor 701 is configured to execute a program in a memory to implement the information processing method provided by the above embodiments.

Accordingly, an embodiment of the present application further provides a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and when executed by a processor, the computer-executable instructions implement the information processing method provided by the foregoing embodiment.

The above descriptions of the embodiments of the information processing apparatus and the storage medium are similar to the above descriptions of the embodiments of the method, have similar technical descriptions and advantages to the corresponding embodiments of the method, and are limited by space. For technical details not disclosed in the embodiments of the information processing apparatus and storage medium of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the embodiments of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not imply an order of execution, and the order of execution of the processes should be determined by their functions and inherent logic, and should not limit the implementation processes of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element identified by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit. Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

Alternatively, the integrated unit in the embodiment of the present application may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a stand-alone product. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code. The above description is only a specific implementation of the embodiments of the present application, but the scope of the embodiments of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the embodiments of the present application, and all the changes or substitutions should be covered by the scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. An information processing method, characterized in that the method comprises:

acquiring an application to be accessed;

in response to an access operation for the application to be accessed, determining multimodal access information associated with the application to be accessed;

and classifying the multi-mode access information to obtain target access information.

2. The method of claim 1, wherein the determining multimodal access information associated with the application to be accessed in response to the access operation for the application to be accessed comprises:

responding to the access operation aiming at the application to be accessed, and acquiring a page to be processed generated in the process that the application to be accessed is accessed;

and extracting information of the page information in the page to be processed to obtain the multi-mode access information.

3. The method of claim 1 or 2, wherein the multimodal access information comprises at least one of: text information, image information, video information, audio information, access behavior information, access events.

4. The method of any of claims 1 to 3, wherein before the classifying the multi-modal visit information to obtain the target visit information, the method further comprises:

determining an application scene of the application to be accessed;

constructing a knowledge graph matched with the application scene;

the classifying the multi-modal access information to obtain target access information comprises the following steps:

and classifying the multi-modal information based on the knowledge graph to obtain the target access information.

5. The method of claim 4, wherein the classifying the multimodal information based on the knowledge-graph to obtain the target access information comprises:

performing information processing on the multi-modal access information to obtain a processing result;

and classifying the processing result based on the knowledge graph to obtain the target access information.

6. The method of claim 5, wherein in the case that the access operation comprises an access sub-operation set, before performing information processing on the multi-modal access information to obtain a processing result, the method further comprises:

determining a target access sub-operation from the access sub-operation set based on a preset access rule;

determining intermediate information associated with a target access sub-operation in the multi-modal information;

the information processing of the multi-modal access information to obtain a processing result comprises:

and carrying out information processing on the intermediate information to obtain the processing result.

7. The method of claim 6, wherein the information processing comprises at least one of: text parsing, image recognition, video processing, audio recognition, behavior classification, and event analysis.

8. The method of any of claims 1 to 7, wherein after classifying the multi-modal visit information to obtain the target visit information, the method further comprises:

and in response to receiving a display instruction, displaying at least one of the target access information and recommendation information associated with the target access information.

9. The method according to any one of claims 1 to 8, further comprising:

generating a preset virtual assistant matched with the application to be accessed based on the application scene of the application to be accessed;

and adjusting the image parameters of the preset virtual assistant based on at least one of the access operation and the multi-modal access information to obtain and display the target virtual assistant.

10. An information processing apparatus characterized in that the apparatus comprises:

the acquisition module is used for acquiring the application to be accessed;

the determining module is used for responding to the access operation aiming at the application to be accessed, and acquiring multi-modal access information associated with the application to be accessed;

and the classification module is used for classifying the multi-mode access information to obtain target access information.

11. A computer device comprising a memory and a processor, the memory having stored thereon computer-executable instructions, the processor being capable of implementing the information processing method of any one of claims 1 to 9 when executing the computer-executable instructions on the memory.

12. A computer storage medium having stored thereon computer-executable instructions that, when executed, are capable of implementing the information processing method of any one of claims 1 to 9.

Images