CN117875430A

CN117875430A - Performance determination method and device for role playing model, electronic equipment and medium

Info

Publication number: CN117875430A
Application number: CN202410050831.3A
Authority: CN
Inventors: 唐义鸿
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2024-01-12
Filing date: 2024-01-12
Publication date: 2024-04-12

Abstract

The disclosure provides a performance determining method, device, electronic equipment and medium of a role playing model, and belongs to the technical field of computers. The method comprises the following steps: acquiring role information and verification targets of any role corresponding to the role playing model; generating a trap type question based on the role information and the verification target; based on the trap type question, carrying out dialogue with the role playing model to obtain dialogue content; based on the dialog content and the role information, a performance of the role-playing model is determined. According to the technical scheme, the trap is arranged to induce the role playing model to generate the reply inconsistent with the role setting, so that the limit of the role playing model on the consistency of the roles is tested more accurately, and the accuracy of performance determination is improved.

Description

Performance determination method and device for role playing model, electronic equipment and medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for determining performance of a role playing model.

Background

With the development of computer technology, role-playing models have evolved. The role-playing model is capable of simulating a persona to interact with a person in real life to provide a companion demand for the person. However, there is currently a lack of an effective way to determine how well a role playing model performs.

In the related art, a manner is generally adopted in which a technician sets a plurality of questions for the role played by the role playing model; then, according to the problems, dialogue is conducted with the role playing model, and dialogue content fed back to technicians by the role playing model is collected; performance of the role-playing model is then determined based on the dialog content. For example, a technician asks "do you have children" for the role played by the role playing model.

However, in the above-mentioned technical solutions, the existing performance determining method of the role playing model often depends on a manually guided session, and the manually guided session depends on the knowledge of the role played by the human on the role playing model, which is easily affected by subjective factors, so that the performance of the role playing model cannot be accurately determined.

Disclosure of Invention

The present disclosure provides a performance determining method, apparatus, electronic device, and medium for a role playing model, which more accurately tests the limits of the role playing model on the consistency of the roles by setting traps to induce the role playing model to generate replies inconsistent with the setting of roles, thereby improving the accuracy of performance determination. The technical scheme of the present disclosure is as follows:

According to an aspect of the disclosed embodiments, there is provided a performance determining method of a role playing model, including:

acquiring role information of any role corresponding to a role playing model and a verification target, wherein the verification target is used for representing part of the role information to be verified;

generating a trap type question based on the role information and the verification target, wherein the trap type question contains information inconsistent with the role information indicated by the verification target;

based on the trap type question, carrying out dialogue with the role playing model to obtain dialogue content, wherein the dialogue content is used for verifying whether the role playing model keeps role consistency in the dialogue process, and the role consistency refers to that the role information reflected by the role playing model in the dialogue process is consistent with the role information corresponding to the role playing model;

based on the dialog content and the role information, a performance of the role-playing model is determined, the performance being related to a situation in which the role-playing model maintains role consistency.

According to another aspect of the embodiments of the present disclosure, there is provided a performance determining apparatus of a role playing model, including:

An acquisition unit configured to perform acquisition of character information of any one character corresponding to a character playing model and a verification target for representing a part of the character information to be verified;

a generation unit configured to execute generation of a trap-type question containing information inconsistent with the character information indicated by the verification target, based on the character information and the verification target;

a dialogue unit configured to perform a dialogue with the role playing model based on the trap-type question to obtain dialogue content, wherein the dialogue content is used for verifying whether the role playing model maintains role consistency in a dialogue process, and the role consistency refers to that role information reflected by the role playing model in the dialogue process is consistent with role information corresponding to the role playing model;

and a determining unit configured to perform determination of performance of the role playing model based on the dialogue content and the role information, the performance being related to a case where the role playing model maintains role consistency.

In some embodiments, the acquisition unit comprises:

a dialogue subunit configured to perform a multi-round dialogue with the role playing model based on the initial role information of any role corresponding to the role playing model, so as to obtain multi-round dialogue content;

And an acquisition subunit configured to perform acquisition of character information of the character based on the multi-round dialogue content.

In some embodiments, the session subunit is configured to perform, for a first-round session, performing a session with the role-playing model based on initial role information of any role corresponding to the role-playing model, to obtain first session content;

the acquisition subunit is configured to perform acquisition of second dialogue content conforming to the initial role information from the first dialogue content; the role information is determined based on the second dialog content and the initial role information.

In some embodiments, the dialog subunit is configured to execute, for the first-round dialog, create a first dialog topic based on initial role information of any role corresponding to the role playing model, where the first dialog topic is used to represent a direction of information to be communicated in the first-round dialog; generating a start white text of the first-round dialogue based on the first dialogue topic and the initial role information; and based on the open text, performing dialogue with the role playing model to obtain the first dialogue content.

In some embodiments, the apparatus further comprises:

a replacement unit configured to perform replacement of the first dialog topic with a second dialog topic in a case where the first dialog content acquired based on the first dialog topic satisfies a preset condition;

the dialog subunit is further configured to perform continuing the dialog based on the second dialog topic.

In some embodiments, the dialog subunit is configured to perform, for a non-first-round dialog, a dialog with the role-playing model based on the role information determined after the previous-round dialog, resulting in third dialog content;

the obtaining subunit is configured to obtain fourth dialogue content conforming to the role information determined after the previous dialogue from the third dialogue content; and updating the role information based on the fourth dialogue content and the role information determined after the previous dialogue.

In some embodiments, the generating unit is configured to perform acquiring the anti-information of the verification target, the anti-information being information inconsistent with the verification target; acquiring associated information related to the verification target from the role information, wherein the associated information is accurate role information; and taking the association information as interference information of the anti-information, and generating the trap type question based on the association information and the anti-information.

In some embodiments, the determining unit is configured to perform extracting information to be verified from the dialogue content, where the information to be verified is character information in the dialogue content; analyzing the difference between the information to be verified and the character information from multiple angles to obtain multiple gaps, wherein the multiple angles are at least two of facts, individuality, value view, background and self consciousness of the character; based on the plurality of gaps, performance of the role playing model is determined.

According to another aspect of the embodiments of the present disclosure, there is provided an electronic device including:

one or more processors;

a memory for storing the processor-executable program code;

wherein the processor is configured to execute the program code to implement the performance determination method of the role playing model described above.

According to another aspect of the disclosed embodiments, a computer-readable storage medium is provided, which when executed by a processor of an electronic device, causes the electronic device to perform the above-described performance determination method of a role playing model.

According to another aspect of the disclosed embodiments, there is provided a computer program product comprising a computer program/instruction which, when executed by a processor, implements the performance determination method of a role playing model as described above.

The embodiment of the disclosure provides a performance determining method of a role playing model, which generates a trap type question according to role information of any role corresponding to the role playing model and a verification target, wherein the trap type question has information inconsistent with the role information indicated by the verification target, so that the role playing model can be induced to acknowledge the information inconsistent with the role information in the process of carrying out dialogue with the role playing model through the trap type question; that is, by setting the trap, the role playing model is induced to generate a reply inconsistent with the role setting, and then the performance of the role playing model is determined by verifying whether the role information reflected by the dialogue content is consistent with the original role information, so that the limit of the role playing model on the consistency of the roles is tested more accurately.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure and do not constitute an undue limitation on the disclosure.

FIG. 1 is a schematic diagram illustrating an implementation environment for a performance determination method for a role playing model, in accordance with an example embodiment.

Fig. 2 is a flowchart illustrating a method of performance determination of a role playing model, in accordance with an exemplary embodiment.

FIG. 3 is a flowchart illustrating another method of performance determination of a role playing model, in accordance with an exemplary embodiment.

Fig. 4 is a frame diagram of a dialog management system provided in an embodiment of the present disclosure.

Fig. 5 is a block diagram of a performance determining apparatus of a role playing model, according to an exemplary embodiment.

Fig. 6 is a block diagram of an apparatus according to an example embodiment.

Fig. 7 is a block diagram of a terminal according to an exemplary embodiment.

Fig. 8 is a block diagram of a server, according to an example embodiment.

Detailed Description

In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the foregoing figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the disclosure described herein may be capable of operation in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the accompanying claims.

It should be noted that, the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, presented data, etc.), and signals related to the present disclosure are authorized by the target object or sufficiently authorized by each party, and the collection, use, and processing of the related data is required to comply with the relevant laws and regulations and standards of the relevant country and region. For example, the role information and dialog content referred to in this disclosure are acquired with sufficient authorization.

The electronic device may be provided as a terminal or a server, and when the electronic device is provided as a terminal, operations performed by the performance determining method of the role playing model may be implemented by the terminal; when provided as a server, operations performed by the performance determination method of a role playing model may be implemented by the server, the server may receive role playing model and role information authorized by a terminal, and the server determines performance of the role playing model based on the role information; operations performed by the performance determination method of the role playing model may also be implemented by the server and terminal interactions, which are not limited by the disclosed embodiments.

FIG. 1 is a schematic diagram illustrating an implementation environment for a performance determination method for a role playing model, in accordance with an example embodiment. Taking an example in which the electronic device is provided as a server, referring to fig. 1, the implementation environment specifically includes: a terminal 101 and a server 102.

The terminal 101 is at least one of a smart phone, a smart watch, a desktop computer, a laptop computer, an MP3 player, an MP4 player, and a laptop portable computer. An application supporting role playing is installed and run on the terminal 101. The application may be a game-type application, a multimedia-type application, a chat-type application, or a shopping-type application, etc., which is not limited in this regard. The user can log in the application through the terminal 101 to acquire a service provided by the application. For example, the user can log in the application program through the terminal 101 and perform a dialogue with a virtual character in the application program. The virtual role of talking to a user is essentially a role playing model. The role-playing model simulates a role in a conversation with a user. The terminal 101 can be connected to the server 102 through a wireless network or a wired network, and can further transmit character information corresponding to the character playing model to the server 102.

The terminal 101 refers broadly to one of a plurality of terminals, and this embodiment is illustrated with the terminal 101. Those skilled in the art will recognize that the number of terminals may be greater or lesser. For example, the number of the terminals may be several, or the number of the terminals may be tens or hundreds, or more, and the number and the device type of the terminals are not limited in the embodiments of the present disclosure.

Server 102 is at least one of a server, a plurality of servers, a cloud computing platform, and a virtualization center. The server 102 can be connected to the terminal 101 and other terminals through a wireless network or a wired network, and the server 102 can receive the role information transmitted from the terminal 101 and determine the performance of the role playing model according to the role information. In some embodiments, the number of servers described above may be greater or lesser, and embodiments of the present disclosure are not limited in this regard. Of course, the server 102 also includes other functional servers to provide more comprehensive and diverse services.

Fig. 2 is a flowchart illustrating a performance determining method of a role playing model, referring to fig. 2, for example, the performance determining method of a role playing model is applied to a server, and the performance determining method of a role playing model includes the steps of:

In step 201, the server acquires role information of any role corresponding to the role playing model and a verification target, where the verification target is used to represent part of the role information to be verified.

In embodiments of the present disclosure, the role-playing model is capable of playing at least one role to engage in conversations with the user. The role information is information of any role played by the role playing model. The role information may be referred to as a setting or representation of the role played by the role playing model for indicating the role. The role information may include information such as occupation, interests, experiences, personalities, and context of the role played by the role playing model, which is not limited by the disclosed embodiments. The server obtains the role information of any role corresponding to the role playing model. For any role, the server can verify whether the role-playing model can adhere to the role information for that role in the application process (e.g., dialog process). In other words, the server can verify whether the role-playing model can conform to the settings of the role played during the application process. The authentication target refers to part of role information of authentication required in the current authentication process. That is, a certain item of information among character information to be verified in any one verification process. The server determines the verification target according to the role information, and the verification target is not limited in the embodiment of the disclosure.

In step 202, the server generates a trap-type question based on the persona information and the verification target, the trap-type question including information inconsistent with the persona information indicated by the verification target.

In an embodiment of the present disclosure, the server determines information inconsistent with the character information indicated by the authentication target based on the character information and the authentication target. The server may modify character information indicated by the verification target to information that does not exist in the character information; alternatively, the character information indicated by the verification target is modified to information contradictory to the character information, or the like, which is not limited by the embodiment of the present disclosure. Then, the server generates a trap-type question based on information inconsistent with the character information indicated by the verification target. Wherein information inconsistent with the role information indicated by the verification target may be considered as "traps" issued in a trap-type question. Contradictory between two messages means that the two messages are inconsistent or that the two messages cannot appear on a character at the same time.

For example, one piece of information is "no child is in the role played by the role-playing model"; another information is "the role played by the role playing model has children". Both of these information are unlikely to appear on one character at a time. One of which is certainly false information that does not conform to the character information. The server may place this spurious information as a "trap" in a trap-type question.

In step 203, the server performs a dialogue with the role playing model based on the trap type question to obtain dialogue content, where the dialogue content is used to verify whether the role playing model maintains role consistency in the dialogue process, and the role consistency refers to that the role information reflected by the role playing model in the dialogue process is consistent with the role information corresponding to the role playing model.

In the disclosed embodiments, the server may speak with the role playing model with the trap-type question as an open-time. The role-playing model outputs replies to the trap-type questions, thereby obtaining dialogue contents. Trap-type questioning can induce a role-playing model to acknowledge information inconsistent with the role information. That is, there is a high probability that information that does not correspond to the role information will exist in the reply of the role-playing model within the dialog content. After obtaining the session content, the server may verify whether the role information reflected in the session content is consistent with reference to the role information corresponding to the role-playing model to determine whether the role-playing model maintains the role consistency during the session, thereby determining the performance of the role-playing model. That is, the server continues to step 204.

In step 204, the server determines performance of the role-playing model based on the dialog content and the role information, the performance being related to the case where the role-playing model maintains the consistency of the role.

In the embodiment of the disclosure, the server can acquire the role information reflected by the dialogue content from the dialogue content with the role playing model. The server may then compare the role information reflected by the dialog content to the native role information to determine whether the role-playing model complies with the native role information. That is, the server can detect whether the role-playing model maintains role consistency. If the role information reflected by the dialogue content accords with the original role information, the server determines that the performance of the role playing model is better; if the character information reflected by the dialogue content contradicts the original character information, the server determines that the performance of the role playing model is poor. The embodiments of the present disclosure are not limited to the specific manner in which the performance is determined.

For example, it is verified whether a child is in a role played by the role-playing model. The role information indicates that the role played by the role playing model has no children. The trap-type question may be "how your child is doing". The trap-type question default role-playing model plays roles that have children that are inconsistent with the role information. If the role playing model replies to the content of the achievement, such as 'good child achievement', the role playing model indicates that the role playing model acknowledges that the played role has children and is inconsistent with the role information. This illustrates that the role-playing model does not maintain role consistency and is less performing. If no child is explicitly indicated in the reply of the role playing model, the reply is consistent with the role information, thereby indicating that the role playing model keeps the consistency of the role and has better performance.

The embodiment of the disclosure provides a performance determining method of a role playing model, which generates a trap type question according to any role information and a verification target corresponding to the role playing model, wherein the trap type question contains information inconsistent with the role information indicated by the verification target, so that the role playing model can be induced to acknowledge the information inconsistent with the role information in the process of carrying out dialogue with the role playing model through the trap type question; that is, by setting the trap, the role playing model is induced to generate a reply inconsistent with the role setting, and then the performance of the role playing model is determined by verifying whether the role information reflected by the dialogue content is consistent with the original role information, so that the limit of the role playing model on the consistency of the roles is tested more accurately.

In some embodiments, the process of obtaining the role information of any role corresponding to the role playing model includes:

based on the initial role information of any role corresponding to the role playing model, carrying out multi-round dialogue with the role playing model to obtain multi-round dialogue content;

based on the multi-round dialogue content, the role information of the roles is obtained.

The scheme provided by the embodiment of the disclosure is based on the initial role information of any role played by the role playing model, and a plurality of rounds of conversations are developed with the role playing model, so that more information about the role played by the role playing model can be collected, and more abundant and finer-granularity role information can be obtained, namely, the information relevance of the role playing model on the role consistency can be fully mined, and the guarantee is provided for the follow-up determination of the performance of the role playing model.

In some embodiments, performing multiple rounds of conversations with the role-playing model based on initial role information of any role corresponding to the role-playing model to obtain multiple rounds of conversational content, including:

for the first-round dialogue, based on the initial role information of any role corresponding to the role playing model, dialogue is carried out with the role playing model, and first dialogue content is obtained;

Based on the multi-round dialogue content, acquiring the role information of the role, including:

acquiring second dialogue content conforming to the initial role information from the first dialogue content;

character information is determined based on the second dialog content and the initial character information.

According to the scheme provided by the embodiment of the disclosure, for the first-round dialogue, the dialogue is unfolded based on the initial role information of the role played by the role playing model, dialogue content conforming to the initial role information is filtered, the initial role information of the role is enriched according to the dialogue content, so that the role information is obtained, contradictory information in the role information is avoided, consistency of the role information and the initial role information is ensured, guarantee is provided for follow-up verification of whether the role playing model can maintain consistency of the role, and more accurate determination of performance of the role playing model is facilitated.

In some embodiments, for a first-round dialog, performing a dialog with a role-playing model based on initial role information of any role corresponding to the role-playing model to obtain first dialog content, including:

for a first-round dialogue, creating a first dialogue topic based on initial role information of any role corresponding to the role playing model, wherein the first dialogue topic is used for representing the information direction to be communicated in the first-round dialogue;

Generating a start white text of the first-round dialogue based on the first dialogue theme and the initial role information;

and based on the open text, performing dialogue with the role playing model to obtain first dialogue content.

According to the scheme provided by the embodiment of the disclosure, for the first-round dialogue, the first dialogue topic is created according to the initial role information of any role corresponding to the role playing model, and then the open white text of the first-round dialogue is generated according to the first dialogue topic, so that the first dialogue topic is always surrounded in the dialogue process based on the open white text, dialogue content related to the first dialogue topic can be acquired, and subsequent extraction of the role information related to the first dialogue topic is facilitated, namely, the role information of a certain aspect can be acquired in a targeted manner.

In some embodiments, the method further comprises:

under the condition that the first dialogue content acquired based on the first dialogue topic meets the preset condition, replacing the first dialogue topic with a second dialogue topic;

based on the second dialog topic, the dialog is continued.

According to the scheme provided by the embodiment of the disclosure, if the first dialogue content acquired based on the first dialogue topic meets the preset condition, the first dialogue topic can be replaced by the second dialogue topic, so that the dialogue is continued based on the second dialogue topic, the dialogue content is enriched, and the subsequent extraction of richer role information is facilitated; that is, by exchanging the dialog topics to conduct the dialog, the role played by the role playing model is facilitated to be known from multiple aspects, and a guarantee is provided for the follow-up verification of whether the role playing model can maintain consistency of the role, so that the performance of the role playing model can be determined more accurately.

for the non-first-round dialogue, based on the role information determined after the previous-round dialogue, performing dialogue with the role playing model to obtain third dialogue content;

acquiring fourth dialogue content which accords with the role information determined after the previous dialogue from the third dialogue content;

the character information is updated based on the fourth dialog content and the character information determined after the previous dialog.

According to the scheme provided by the embodiment of the disclosure, after each round of dialogue, role information can be determined according to dialogue content; then, in the process of carrying out a new dialogue, carrying out dialogue with the role playing model based on the role information determined after the previous dialogue; then, the role information determined after the previous dialogue is used as a reference, and the non-conforming information in the new dialogue content is filtered, so that the role information is updated; equivalently, the role information is iterated through multiple rounds of conversations, so that the role information reflected in the multiple rounds of conversations is fused together, the role information is enriched, the consistency of the role information and the initial role information is guaranteed, the guarantee is provided for verifying whether the role playing model can maintain the consistency of the role or not later, and the performance of the role playing model can be determined more accurately.

In some embodiments, generating a trap-type question based on the role information and the verification target includes:

acquiring the anti-information of the verification target, wherein the anti-information is inconsistent with the verification target;

acquiring associated information related to a verification target from the role information, wherein the associated information is accurate role information;

and generating a trap type question based on the association information and the anti-information by taking the association information as the interference information of the anti-information.

According to the scheme provided by the embodiment of the disclosure, the anti-information of the verification target is wrong role information, the associated information is accurate role information, the associated information is taken as interference information of the anti-information and is combined with the anti-information to generate the trap type question, namely, the trap type question is generated in a mode that the role information is true and false, so that the anti-information in the trap type question is more difficult to be recognized by the role playing model, the follow-up role playing model is favorably induced to generate a reply inconsistent with the role setting, the limit of the model on the angular consistency is more accurately tested, and further the performance of the role playing model is more accurately determined.

In some embodiments, determining performance of the role-playing model based on the dialog content and the role information includes:

Extracting information to be verified from the dialogue content, wherein the information to be verified is character information in the dialogue content;

analyzing the difference between the information to be verified and the character information from multiple angles to obtain multiple gaps, wherein the multiple angles are at least two of facts, individuality, value view, background and self-consciousness of the character;

based on the plurality of gaps, performance of the role-playing model is determined.

According to the scheme provided by the embodiment of the disclosure, the role information to be verified is extracted from the dialogue content, and then the difference between the role information to be verified and the original role information is analyzed from at least two angles of facts, individuality, value view, the background and self consciousness by taking the original role information as a reference, so that the condition that the role playing model maintains the role consistency is determined, and the performance of the role playing model is accurately determined.

The foregoing fig. 2 illustrates only a basic flow of the present disclosure, and the scheme provided by the present disclosure is further described below based on a specific implementation, and fig. 3 is a flowchart illustrating another method for determining performance of a role playing model according to an exemplary embodiment. Taking an example in which an electronic device is provided as a server, referring to fig. 3, the performance determining method of the role playing model includes:

In step 301, the server obtains role information for any role corresponding to the role playing model.

In the embodiment of the disclosure, the server may obtain, from the historical dialog content of the role playing model, role information of any role corresponding to the role playing model; alternatively, the server may also obtain the role information for the role by directly talking to the role playing model; the embodiments of the present disclosure are not limited in this regard. Wherein historical dialog content may be generated during a dialog with a user during application of the role playing model, embodiments of the present disclosure are not limited in this regard.

In some embodiments, the server may directly talk to the role playing model to obtain the role information for any of the roles corresponding to the role playing model. Accordingly, the process of obtaining the role information of any role corresponding to the role playing model includes: the server performs multiple rounds of conversations with the role playing model based on the initial role information of any role corresponding to the role playing model, and obtains multiple rounds of conversations. Then, the server acquires the role information corresponding to the role playing model based on the multi-round dialogue content. The scheme provided by the embodiment of the disclosure is based on the initial role information of any role played by the role playing model, and a plurality of rounds of conversations are developed with the role playing model, so that more information about the role played by the role playing model can be collected, and more abundant and finer-granularity role information can be obtained, namely, the information relevance of the role playing model on the role consistency can be fully mined, and the guarantee is provided for the follow-up determination of the performance of the role playing model.

Wherein a session management system may be deployed on the server. The dialog management system includes a Chat Agent (Chat Agent). The server may conduct a conversation with the role playing model through a chat agent in the conversation management system. The chat proxy may be built based on a large language model, which is not limited by the disclosed embodiments. According to the scheme provided by the embodiment of the disclosure, the large language model and the role playing model are adopted to conduct dialogue, namely, the joint processing of role playing tasks is realized by simulating a real dialogue scene and guiding dialogue, the information relevance of the role playing model on the role consistency is fully mined, and the guarantee is provided for the follow-up determination of the performance of the role playing model.

In some embodiments, for a first-round dialog, the server dialogs with the role-playing model based on initial role information for any of the roles corresponding to the role-playing model to obtain first dialog content. Then, the server acquires second dialogue content conforming to the initial character information from the first dialogue content. Then, the server determines character information based on the second dialog content and the initial character information. The initial role information may be occupation, preference, and background of the role corresponding to the role playing model, which is not limited by the embodiments of the present disclosure. According to the scheme provided by the embodiment of the disclosure, for the first-round dialogue, the dialogue is unfolded based on the initial role information of the role played by the role playing model, dialogue content conforming to the initial role information is filtered, the initial role information of the role is enriched according to the dialogue content, so that the role information is obtained, contradictory information in the role information is avoided, consistency of the role information and initial setting is ensured, guarantee is provided for follow-up verification of whether the role playing model can maintain consistency of the role, and more accurate determination of performance of the role playing model is facilitated.

Alternatively, the server may also extract intermediate role information from the first dialog content. Then, the server takes the initial role information as a reference, and acquires information conforming to the initial role information from the intermediate role information; and then combining the initial role information to obtain the role information. That is, the server may extract the character information from the session content, filter out the information that does not conform to the initial character information, and then combine the initial character information to obtain the character information.

Before any dialog is conducted, the server can determine the dialog theme so as to expand the dialog around the dialog theme, and more useless information is prevented from being acquired after the dialog deviates from the theme. The following description will take the first-round dialogue as an example. Accordingly, the process of obtaining the first dialogue content by the server includes: for the first round of dialog, the server creates a first dialog topic based on initial role information for any of the roles corresponding to the role-playing model. The first dialog theme is used for indicating the direction of information to be communicated in the first dialog. The server then generates open text for the first round of dialog based on the first dialog topic and the initial role information. The server then dialogues with the role-playing model based on the open text to obtain first dialog content. According to the scheme provided by the embodiment of the disclosure, for the first-round dialogue, the first dialogue topic is created according to the initial role information of any role corresponding to the role playing model, and then the open white text of the first-round dialogue is generated according to the first dialogue topic, so that the first dialogue topic is always surrounded in the dialogue process based on the open white text, dialogue content related to the first dialogue topic can be acquired, and subsequent extraction of the role information related to the first dialogue topic is facilitated, namely, the role information of a certain aspect can be acquired in a targeted manner.

For example, open white text is "in the emergency room of a hospital, you are a doctor". Alternatively, open white text is "how the patient has dizziness and how you can handle".

Wherein the first dialog topic may be determined based on any of the initial role information, which is not limited by the disclosed embodiments. That is, the dialogue topic is related to the role information corresponding to the role playing model, so that the role information corresponding to the role playing model is continuously enriched and expanded in the process of conducting dialogue around the dialogue topic.

For example, the occupation in the initial role information that indicates the role played by the role playing model is a doctor. The server may determine that the first dialog topic is "patient diagnosis" based on the initial role information. The role playing model is configured to determine a disease of the patient based on the symptoms of the patient by providing the symptoms of the patient and the first dialog topic to the role playing model, such that the role playing model is directed by the first dialog topic. The dialog process can mine the knowledge depth of the role-playing model, thereby enriching the role information. Knowledge depth refers to the degree of knowledge of the relevant knowledge on the premise of the character information (e.g., initial character information). In the above example, the server explores the degree of knowledge of medical knowledge of the role-playing model on the premise of the doctor playing, thereby enriching the role information. For example, the role-playing model is more knowledgeable about pediatrics than other departments, and thus can determine that the role played by the role-playing model is more likely to be a pediatrician.

A Topic Manager (Topic Manager) is included in a dialog management system deployed on a server. The server may maintain the current dialog topic or change the current dialog topic through the topic manager.

In some embodiments, the server changes the dialog topic. Accordingly, the server changes the first dialog topic to the second dialog topic in case that the first dialog content acquired based on the first dialog topic satisfies the preset condition. The server then proceeds with the conversation based on the second conversation topic. According to the scheme provided by the embodiment of the disclosure, if the first dialogue content acquired based on the first dialogue topic meets the preset condition, the first dialogue topic can be replaced by the second dialogue topic, so that the dialogue is continued based on the second dialogue topic, the dialogue content is enriched, and the subsequent extraction of richer role information is facilitated; that is, by exchanging the dialog topics to conduct the dialog, the role played by the role playing model is facilitated to be known from multiple aspects, and a guarantee is provided for the follow-up verification of whether the role playing model can maintain consistency of the role, so that the performance of the role playing model can be determined more accurately.

The preset condition may be that an information amount about the first dialog topic in the first dialog content reaches a preset value; alternatively, the preset condition may be a preset result that the first dialog topic has appeared in the first dialog content, or the like, which is not limited by the embodiment of the present disclosure. For example, the preset result is a disease of the patient. The role playing model has replied to the disease of the patient. The second dialog topic may or may not be related to the first dialog topic, which is not limited by the disclosed embodiments. For example, the first dialog topic is "patient diagnosis". The second dialog topic is "treatment regimen". In the event that the role-playing model has replied to a disease in the patient, the server may replace the first session topic with a second session topic to continue communicating with the role-playing model for the treatment regimen.

In some embodiments, for non-first-round conversations, the server performs a conversation with the role-playing model based on the role information determined after the previous-round conversation, resulting in third conversation content. Then, the server acquires fourth dialogue content conforming to the character information determined after the previous dialogue from the third dialogue content. Then, the server updates the character information based on the fourth dialogue content and the character information determined after the previous dialogue. According to the scheme provided by the embodiment of the disclosure, after each round of dialogue, role information can be determined according to dialogue content; then, in the process of carrying out a new dialogue, carrying out dialogue with the role playing model based on the role information determined after the previous dialogue; then, the role information determined after the previous dialogue is used as a reference, and the non-conforming information in the new dialogue content is filtered, so that the role information is updated; equivalently, the role information is iterated through multiple rounds of conversations, so that the role information reflected in the multiple rounds of conversations is fused together, the role information is enriched, the consistency of the role information and the initial setting is ensured, the guarantee is provided for verifying whether the role playing model can maintain the consistency of the role or not later, and the performance of the role playing model can be determined more accurately.

An information Manager (Info Manager) is included in a session management system deployed on a server. The information manager is used for collecting and storing information. The information includes role information corresponding to the role playing model, dialog topics, and current operation. The current operation is divided into two types, session initiation and information verification. Session initiation refers to directing a role-playing model to conduct a session. The operation in step 301 is session initiation, and the role playing model is guided to perform a session through the role information and the session theme, so as to enrich the role information. Information verification refers to verifying whether a role playing model maintains role consistency, thereby determining performance of the role playing model. The operations from step 302 to step 305 are information verification.

The server gathers the role information by conducting multiple rounds of conversations with the role playing model prior to information verification. In the case where the richness or granularity of the character information satisfies the condition, the server starts information verification to determine the performance of the role playing model. Accordingly, the server continues to perform steps 302 through 305.

In step 302, the server acquires an authentication target for representing part of character information to be authenticated based on the character information.

In the embodiment of the present disclosure, the role information corresponding to the role playing model may include information of multiple dimensions such as facts, personality, value, background where the role playing model is located, and self-awareness, which is not limited by the embodiment of the present disclosure. Facts refer to objective statements about roles presented in terms of the first person or third person. Facts may include the persona's occupation, hobbies, interests, past experiences, and other personas or events related to the persona, etc., to which embodiments of the disclosure are not limited. Personality determines the instinctive response of a character to external stimuli. Personality may include speaking style, viewing angle, and emotion in dialogue interaction, such as irritability, mildness, introversion, silence, or happiness, etc. The value view represents the role's specific perspective to the outsides and individuals, as well as the propensity for various abstractions, such as sense, environmentality, or utility. The background covers the world of character life, mainly in terms of time and space. The context is highly relevant to other characters and events encountered by the character. The self-awareness test determines whether the role-playing model can clearly understand that it is functioning. Analog self-awareness requires that the model optionally forget its identity, and it should be understood that "i am an assisted artificial intelligence" or "i am just a computer program" and like concepts. For example, "i am einstein. I propose that the statements such as relativistic indicate that the model has good self-awareness.

And the server acquires the verification target according to the role information. The partial character information indicated by the verification target may be any item of character information. Accordingly, the server randomly selects one item of information from the character information as a verification target. Alternatively, the part of the character information indicated by the authentication target is unverified information in the character information. Accordingly, the server may mark the verified character information. Then, the server selects character information that does not carry a mark as a verification target. Alternatively, the verification target may further include multiple pieces of character information in the character information, so that the server verifies the multiple pieces of character information at a time, which is not limited by the embodiments of the present disclosure.

In step 303, the server generates a trap-type question based on the persona information and the verification target, the trap-type question including information inconsistent with the persona information indicated by the verification target.

In an embodiment of the present disclosure, the server determines information inconsistent with the character information indicated by the authentication target based on the character information and the authentication target. Then, the server generates a trap-type question based on information inconsistent with the character information indicated by the verification target. Information that is inconsistent with the persona information indicated by the verification target may be considered a "trap" that is placed in a trap-type question. The trap type question may contain only "traps"; the method can also comprise interference information capable of covering the trap in the case of comprising the trap so as to improve the difficulty of identifying the trap by the role playing model, thereby testing the limit of the role playing model on the consistency of the angles. The embodiments of the present disclosure are not limited to trap-type questions.

In some embodiments, the trap-type question contains a "trap" and disturbance information that can obscure the "trap". Accordingly, the server generates a trap type question based on the role information and the verification target, and the trap type question generation method comprises the following steps: the server obtains the anti-information of the verification target. The anti-information is information inconsistent with the verification target. Then, the server acquires association information about the authentication target from the character information. The associated information is accurate character information. Then, the server generates a trap type question based on the related information and the anti-information by using the related information as the interference information of the anti-information. According to the scheme provided by the embodiment of the disclosure, the anti-information of the verification target is wrong role information, the associated information is accurate role information, the associated information is taken as interference information of the anti-information and is combined with the anti-information to generate the trap type question, namely, the trap type question is generated in a mode that the role information is true and false, so that the anti-information in the trap type question is more difficult to be recognized by the role playing model, the follow-up role playing model is facilitated to generate a reply inconsistent with the role setting, the limit of the role playing model on the angular consistency is tested more accurately, and further the performance of the role playing model is more accurately determined. Or the server can also obtain the opposite information of the related information, take the opposite information of the related information as interference information, and combine the opposite information of the verification target to generate the trap type question. The embodiments of the present disclosure are not limited in this regard.

In step 304, the server performs a dialogue with the role playing model based on the trap type question to obtain dialogue content, where the dialogue content is used to verify whether the role playing model maintains role consistency in the dialogue process, and the role consistency refers to that the role information reflected by the role playing model in the dialogue process is consistent with the role information corresponding to the role playing model.

In the disclosed embodiments, the server may speak with the role playing model with the trap-type question as an open-time. The role-playing model outputs replies to the trap-type questions, thereby obtaining dialogue contents. Trap-type questioning can induce a role-playing model to acknowledge information inconsistent with the role information. That is, there is a high probability that information that does not correspond to the role information will exist in the reply of the role-playing model within the dialog content. The dialog process is similar to the principle of the dialog in step 301, and the dialog may also be performed by the dialog agent, which will not be described in detail here.

The character information reflected by the dialogue content matches (conforms to) the original character information, and the character information reflected by the dialogue content may be the same as the original character information, or may be the character information reflected by the dialogue content that appears on the basis of the original character information.

For example, the original role information is taken by a doctor. If the role playing model is recognized as doctor content visually appearing in the dialogue content, the server determines that the role information reflected by the dialogue content matches the original role information. Alternatively, if relevant content for diagnosing or treating the disease appears in the dialogue content. Since diagnosing and treating diseases is what the doctor would do. In this case, the server may determine that the character information reflected by the dialogue content matches the original character information.

In step 305, the server determines performance of the role-playing model based on the dialog content and the role information, the performance being related to the case where the role-playing model maintains the consistency of the role.

In the embodiment of the disclosure, the server can acquire the role information reflected by the dialogue content from the dialogue content with the role playing model. The server may then compare the role information reflected by the dialog content with the native role information to determine whether the role playing model complies with the native role information. Accordingly, the process by which the server determines the performance of the role-playing model based on the dialog content and the role information includes: the server extracts information to be verified from the dialogue content, wherein the information to be verified is character information in the dialogue content. Then, the server analyzes the differences between the information to be verified and the role information from multiple angles to obtain multiple gaps. The plurality of angles are at least two of facts, personalities, value views, backgrounds, and self-awareness of the character. The server then determines performance of the role-playing model based on the plurality of gaps. The performance of a role-playing model can reflect whether the role-playing model maintains role consistency in validating targets. According to the scheme provided by the embodiment of the disclosure, the role information to be verified is extracted from the dialogue content, and then the difference between the role information to be verified and the original role information is analyzed from at least two angles of facts, individuality, value view, the background and self consciousness by taking the original role information as a reference, so that the condition that the role playing model maintains the role consistency is determined, and the performance of the role playing model is accurately determined.

The gap may be a similarity between the information to be verified and the character information. For any angle, the server may determine that the role-playing model performs better in maintaining consistency of roles over that angle if the gap corresponding to that angle does not exceed the gap threshold. In the event that the gap corresponding to the angle exceeds a gap threshold, the server may determine that the role-playing model is less performing in maintaining role consistency over the angle. The server may aggregate performance analysis cases from multiple perspectives, output in text form, as performance of the role-playing model. The gap may also be an information proportion of information contradictory to the role information in the information to be verified, which is not limited in the embodiment of the present disclosure.

In some embodiments, the server may also determine whether the role-playing model is good or bad by comparing conversations of at least two role-playing models. The role information corresponding to the at least two role playing models is the same, and model architectures are different. In determining the performance of the two role-playing models, the server uses the same validation targets for validation. The server generates a trap type question based on the character information and the verification target. Then, the server respectively performs a dialogue with at least two role playing models based on the trap type questioning to obtain at least two dialogue contents. Then, for any dialogue content, the server extracts information to be verified from the dialogue content, wherein the information to be verified is character information in the dialogue content. Then, the server analyzes the difference between the information to be verified and the role information to obtain a gap. Among the at least two role-playing models, the corresponding less widely separated role-playing model wins and the other role-playing models defeat. The server may repeat the above process, i.e. conduct multiple rounds of conversations. In this multiple session, the role information verified by the server may be different. After multiple rounds of conversations, the server may calculate a win rate for each role-playing model, with the win rate representing the performance of the role-playing model. Higher win yields indicate better performance of the role-playing model; lower win yields indicate poorer performance of the role playing model. The multiple rounds of conversations may be 20 rounds of conversations, with the number of conversations not being limited by the disclosed embodiments.

In order to more clearly describe the performance determination method of the role playing model provided by the embodiments of the present disclosure, the method is further described below with reference to the accompanying drawings. Fig. 4 is a frame diagram of a dialog management system provided in an embodiment of the present disclosure. Referring to fig. 4, the dialog management information includes a Guidance Manager (guide Manager), a Topic Manager (Topic Manager), an information Manager (Info Manager), and a Chat Agent (Chat Agent). The boot manager is used to determine whether the current operation of the dialog management system is dialog initiation or information verification. The topic manager is used to hold or transform dialog topics. The Information manager is used for extracting Role Information (Role Information) from the dialogue content and storing the Role Information into the database. The database may also store the actions performed (session initiation or information verification) and the session topics that the role-playing system has replied to. Chat agents are used to talk to the role playing model. The server initializes the dialogue management system to empty the irrelevant data and sets the number of dialogue rounds. Then, the server determines the current operation from the character information, the History operation (guide History), and the History dialog Topic (Topic History) through the boot manager. The history operation is the executed operation. The historical dialog topics are the dialog topics replied to by the role playing system. Then, the server determines the current dialog theme according to the character information and the current operation through the theme manager. The server extracts character information from the dialogue content through the information manager. The server then communicates the current operation, the current conversation topic, and the role information to the chat proxy. The server then generates dialogue sentences (open text or trap questions, etc.) from the current operation, the current dialogue topic, and the character information through the chat proxy. The server then sends the dialogue statement to the Role playing model for the purpose of dialogue with the Role playing model (Role-play System). According to the scheme provided by the embodiment of the disclosure, the task of determining the performance of the model is divided into a plurality of subtasks such as determining the current operation, determining the dialogue theme, extracting the role information, and the dialogue, and the subtasks are distributed to the plurality of managers to be responsible for execution. The plurality of managers and chat agents described above may be considered agents or agents. The dialog management system may be regarded as a DOMIS (Dialogue Orchestrator with Trap-settings Multi-agent Interaction System), dialog orchestrator Multi-agent interaction system with trap settings.

The performance determining method of the role playing model provided by the embodiment of the disclosure can be applied to a plurality of models with different architectures for testing, can reflect the real role playing capability of the model, and provides a brand-new evaluation standard for research and development of the role model. Moreover, the performance determining method of the role playing model provided by the embodiment of the disclosure can be widely applied to the fields of artificial intelligence and natural language processing, provides an effective means for developing a role playing system with more deep interaction and accompaniment, and promotes the further development of a large-scale language model on various dialogue tasks.

The embodiment of the disclosure provides a performance determining method of a role playing model, which generates a trap type question according to role information of any role corresponding to the role playing model and a verification target, wherein the trap type question has information inconsistent with the role information indicated by the verification target, so that the role playing model can be induced to acknowledge the information inconsistent with the role information in the process of carrying out dialogue with the role playing model through the trap type question; that is, by setting the trap, the role playing model is induced to generate a reply inconsistent with the role setting, and then by verifying whether the role information reflected by the dialogue content is consistent with the original role information, the performance of the role playing model is determined, the limit of the role playing model on the consistency of the role is tested more accurately, and compared with the mode of asking the role playing model for the role information in the prior art, various information in the role information can be verified in a finer granularity, the defects of the prior art in granularity and dialogue guidance are overcome, so that the role playing model can skillfully avoid the trap under the condition of induction, and the accuracy of performance determination is improved.

Any combination of the above-mentioned optional solutions may be adopted to form an optional embodiment of the present disclosure, which is not described herein in detail.

Fig. 5 is a block diagram of a performance determining apparatus of a role playing model, according to an exemplary embodiment. Referring to fig. 5, the performance determining apparatus of the role playing model includes: an acquisition unit 501, a generation unit 502, a dialogue unit 503, and a determination unit 504.

An acquisition unit 501 configured to perform acquisition of character information of any one character corresponding to the role-playing model and an authentication target for representing part of the character information to be authenticated;

a generating unit 502 configured to execute generation of a trap-type question containing information inconsistent with the character information indicated by the verification target, based on the character information and the verification target;

a dialogue unit 503 configured to perform a trap-based question, perform a dialogue with the role playing model, and obtain dialogue content for verifying whether the role playing model maintains role consistency during the dialogue, where role consistency refers to consistency of role information reflected by the role playing model during the dialogue with role information corresponding to the role playing model;

A determining unit 504 configured to perform determining performance of the role playing model based on the dialogue content and the role information, the performance being related to a case where the role playing model maintains the role consistency.

In some embodiments, FIG. 6 is a block diagram of another performance determining apparatus of a role playing model, shown in accordance with an example embodiment. Referring to fig. 6, the acquisition unit 501 includes:

a dialogue subunit 5011 configured to perform a multi-round dialogue with the role playing model based on the initial role information of any one of the roles corresponding to the role playing model, to obtain multi-round dialogue content;

an acquisition subunit 5012 configured to perform acquisition of character information of a character based on the multi-round dialogue content.

In some embodiments, with continued reference to fig. 6, a dialog subunit 5011 is configured to perform a dialog with the role-playing model based on initial role information for any of the roles corresponding to the role-playing model for a first round of dialog, resulting in first dialog content;

an acquisition subunit 5012 configured to perform acquisition of second dialogue content conforming to the initial character information from the first dialogue content; character information is determined based on the second dialog content and the initial character information.

In some embodiments, with continued reference to fig. 6, the dialog subunit 5011 is configured to perform, for a first-round dialog, creating a first dialog topic based on initial role information for any role corresponding to the role-playing model, the first dialog topic being used to represent a direction of information to be communicated in the first-round dialog; generating a start white text of the first-round dialogue based on the first dialogue theme and the initial role information; and based on the open text, performing dialogue with the role playing model to obtain first dialogue content.

In some embodiments, with continued reference to fig. 6, the apparatus further comprises:

a replacing unit 505 configured to perform replacement of the first dialogue topic with the second dialogue topic in a case where the first dialogue content acquired based on the first dialogue topic satisfies a preset condition;

the dialog subunit 5011 is further configured to perform a continuation of the dialog based on the second dialog topic.

In some embodiments, with continued reference to fig. 6, the dialog subunit 5011 is configured to perform a dialog with the role playing model for a non-first round of dialog based on the role information determined after the previous round of dialog to obtain third dialog content;

an acquisition subunit 5012 configured to perform acquisition of fourth dialogue content conforming to the character information determined after the previous round of dialogue from the third dialogue content; the character information is updated based on the fourth dialog content and the character information determined after the previous dialog.

In some embodiments, with continued reference to fig. 6, the generating unit 502 is configured to perform obtaining of anti-information of the verification target, the anti-information being information inconsistent with the verification target; acquiring associated information related to a verification target from the role information, wherein the associated information is accurate role information; and generating a trap type question based on the association information and the anti-information by taking the association information as the interference information of the anti-information.

In some embodiments, the determining unit 504 is configured to perform extracting information to be verified from the dialogue content, where the information to be verified is character information in the dialogue content; analyzing the difference between the information to be verified and the character information from multiple angles to obtain multiple gaps, wherein the multiple angles are at least two of facts, individuality, value view, background and self-consciousness of the character; based on the plurality of gaps, performance of the role-playing model is determined.

The embodiment of the disclosure provides a performance determining device of a role playing model, which generates a trap type question according to role information of any role corresponding to the role playing model and a verification target, and can induce the role playing model to acknowledge information inconsistent with the role information in the process of carrying out dialogue with the role playing model through the trap type question because the trap type question has information inconsistent with the role information indicated by the verification target; that is, by setting the trap, the role playing model is induced to generate a reply inconsistent with the role setting, and then by verifying whether the role information reflected by the dialogue content is consistent with the original role information, the performance of the role playing model is determined, the limit of the role playing model on the consistency of the role is tested more accurately, and compared with the mode of asking the role playing model for the role information in the prior art, various information in the role information can be verified in a finer granularity, the defects of the prior art in granularity and dialogue guidance are overcome, so that the role playing model can skillfully avoid the trap under the condition of induction, and the accuracy of performance determination is improved.

It should be noted that, when determining the performance of the role playing model, the performance determining apparatus for a role playing model provided in the foregoing embodiment is only exemplified by the division of the functional units, and in practical application, the functional allocation may be performed by different functional units according to needs, that is, the internal structure of the electronic device is divided into different functional units, so as to complete all or part of the functions described above. In addition, the performance determining device of the role playing model provided in the above embodiment and the performance determining method embodiment of the role playing model belong to the same concept, and detailed implementation processes of the performance determining device and the performance determining method embodiment of the role playing model are detailed in the method embodiment, and are not described herein again.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

Fig. 7 is a block diagram of a terminal 700, shown in accordance with an exemplary embodiment, when the electronic device is provided as a terminal. The terminal fig. 7 shows a block diagram of a terminal 700 provided in an exemplary embodiment of the present disclosure. The terminal 700 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion picture expert compression standard audio plane 3), an MP4 (Moving Picture Experts Group Audio Layer IV, motion picture expert compression standard audio plane 4) player, a notebook computer, or a desktop computer. Terminal 700 may also be referred to by other names of user devices, portable terminals, laptop terminals, desktop terminals, etc.

In general, the terminal 700 includes: a processor 701 and a memory 702.

Processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 701 may also include a main processor, which is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 701 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 702 is used to store at least one program code for execution by processor 701 to implement the performance determination method of the role playing model provided by the method embodiments in the present disclosure.

In some embodiments, the terminal 700 may further optionally include: a peripheral interface 703 and at least one peripheral. The processor 701, the memory 702, and the peripheral interface 703 may be connected by a bus or signal lines. The individual peripheral devices may be connected to the peripheral device interface 703 via buses, signal lines or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 704, a display 705, a camera assembly 706, audio circuitry 707, a positioning assembly 708, and a power supply 709.

A peripheral interface 703 may be used to connect I/O (Input/Output) related at least one peripheral device to the processor 701 and memory 702. In some embodiments, the processor 701, memory 702, and peripheral interface 703 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 701, the memory 702, and the peripheral interface 703 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 704 is configured to receive and transmit RF (Radio Frequency) signals, also referred to as electromagnetic signals. The radio frequency circuitry 704 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 704 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 704 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 704 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 704 may also include NFC (Near Field Communication ) related circuitry, which is not limited by the present disclosure.

The display screen 705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display 705 is a touch display, the display 705 also has the ability to collect touch signals at or above the surface of the display 705. The touch signal may be input to the processor 701 as a control signal for processing. At this time, the display 705 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, the display 705 may be one, providing a front panel of the terminal 700; in other embodiments, the display 705 may be at least two, respectively disposed on different surfaces of the terminal 700 or in a folded design; in still other embodiments, the display 705 may be a flexible display disposed on a curved surface or a folded surface of the terminal 700. Even more, the display 705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The display 705 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 706 is used to capture images or video. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 706 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 707 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 701 for processing, or inputting the electric signals to the radio frequency circuit 704 for voice communication. For the purpose of stereo acquisition or noise reduction, a plurality of microphones may be respectively disposed at different portions of the terminal 700. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, the audio circuit 707 may also include a headphone jack.

The power supply 708 is used to power the various components in the terminal 700. The power source 708 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power source 708 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the terminal 700 further includes one or more sensors 709. The one or more sensors 709 include, but are not limited to: acceleration sensor 710, gyroscope sensor 711, pressure sensor 712, fingerprint sensor 714, optical sensor 713, and proximity sensor 714.

The acceleration sensor 710 may detect the magnitudes of accelerations on three coordinate axes of a coordinate system established with the terminal 700. For example, the acceleration sensor 710 may be used to detect components of gravitational acceleration in three coordinate axes. The processor 701 may control the display screen 705 to display a user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 710. Acceleration sensor 710 may also be used for the acquisition of motion data of a game or user.

The gyro sensor 711 may detect a body direction and a rotation angle of the terminal 700, and the gyro sensor 711 may collect a 3D motion of the user on the terminal 700 in cooperation with the acceleration sensor 710. The processor 701 may implement the following functions according to the data collected by the gyro sensor 711: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 712 may be disposed at a side frame of the terminal 700 and/or at a lower layer of the display screen 705. When the pressure sensor 712 is disposed at a side frame of the terminal 700, a grip signal of the user to the terminal 700 may be detected, and the processor 701 performs a left-right hand recognition or a shortcut operation according to the grip signal collected by the pressure sensor 712. When the pressure sensor 712 is disposed at the lower layer of the display screen 705, the processor 701 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The optical sensor 713 is used to collect the intensity of ambient light. In one embodiment, the processor 701 may control the display brightness of the display screen 705 based on the ambient light intensity collected by the optical sensor 713. Specifically, when the intensity of the ambient light is high, the display brightness of the display screen 705 is turned up; when the ambient light intensity is low, the display brightness of the display screen 705 is turned down. In another embodiment, the processor 701 may also dynamically adjust the shooting parameters of the camera assembly 706 based on the ambient light intensity collected by the optical sensor 713.

A proximity sensor 714, also known as a distance sensor, is typically provided on the front panel of the terminal 700. The proximity sensor 714 is used to collect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 714 detects that the distance between the user and the front of the terminal 700 gradually decreases, the processor 701 controls the display 705 to switch from the bright screen state to the off screen state; when the proximity sensor 714 detects that the distance between the user and the front surface of the terminal 700 gradually increases, the processor 701 controls the display screen 705 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 7 is not limiting of the terminal 700 and may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

When the electronic device is provided as a server, fig. 8 is a block diagram illustrating a server 800 according to an exemplary embodiment, where the server 800 may be relatively different due to configuration or performance, and may include one or more processors (Central Processing Units, CPU) 801 and one or more memories 802, where the memory 802 stores at least one program code that is loaded and executed by the processor 801 to implement the performance determining method of the role playing model provided in the above-described respective method embodiments. Of course, the server may also have a wired or wireless network interface, a keyboard, an input/output interface, etc. to perform input/output, and the server 800 may also include other components for implementing the functions of the device, which are not described herein.

In an exemplary embodiment, a computer readable storage medium is also provided, such as a memory 702 or a memory 802, comprising instructions executable by the processor 701 of the terminal 700 or the processor 801 of the server 800 to perform the performance determination method of the role playing model described above. Alternatively, the computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

A computer program product comprising computer programs/instructions which when executed by a processor implement the performance determination method of a role playing model as described above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of determining performance of a role playing model, the method comprising:

2. The method of performance determination of a role playing model according to claim 1, wherein the process of acquiring the role information of any one of the roles corresponding to the role playing model comprises:

and acquiring the role information of the roles based on the multi-round dialogue content.

3. The method for determining performance of a role playing model according to claim 2, wherein the performing a multi-turn dialogue with the role playing model based on the initial role information of any role corresponding to the role playing model to obtain multi-turn dialogue content includes:

for a first-round dialogue, based on initial role information of any role corresponding to the role playing model, dialogue is carried out with the role playing model, and first dialogue content is obtained;

the acquiring the role information of the role based on the multi-round dialogue content comprises the following steps:

the role information is determined based on the second dialog content and the initial role information.

4. The method for determining performance of a role playing model according to claim 3, wherein for the first-round dialogue, based on the initial role information of any role corresponding to the role playing model, dialogue with the role playing model is performed to obtain the first dialogue content, including:

for the first-round dialogue, creating a first dialogue topic based on initial role information of any role corresponding to the role playing model, wherein the first dialogue topic is used for representing the information direction to be communicated in the first-round dialogue;

generating a start white text of the first-round dialogue based on the first dialogue topic and the initial role information;

and based on the open text, performing dialogue with the role playing model to obtain the first dialogue content.

5. The method of performance determination of a role playing model of claim 4, further comprising:

and continuing the dialogue based on the second dialogue topic.

6. The method for determining performance of a role playing model according to any one of claims 2-5, wherein the performing multiple rounds of conversations with the role playing model based on the initial role information of any role corresponding to the role playing model to obtain multiple rounds of conversational content includes:

and updating the role information based on the fourth dialogue content and the role information determined after the previous dialogue.

7. The method of performance determination of a role playing model according to claim 1, wherein the generating a trap-type question based on the role information and the verification target comprises:

acquiring the anti-information of the verification target, wherein the anti-information is inconsistent information with the verification target;

acquiring associated information related to the verification target from the role information, wherein the associated information is accurate role information;

and taking the association information as interference information of the anti-information, and generating the trap type question based on the association information and the anti-information.

8. The method of performance determination of a role playing model according to claim 1, wherein the determining performance of the role playing model based on the dialogue content and the role information comprises:

analyzing the difference between the information to be verified and the character information from multiple angles to obtain multiple gaps, wherein the multiple angles are at least two of facts, individuality, value view, background and self consciousness of the character;

based on the plurality of gaps, performance of the role playing model is determined.

9. A performance determining apparatus for a role playing model, the apparatus comprising:

10. An electronic device, the electronic device comprising:

one or more processors;

a memory for storing the processor-executable program code;

wherein the processor is configured to execute the program code to implement a performance determination method of a role playing model as claimed in any of claims 1 to 8.

11. A computer readable storage medium, wherein instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the performance determination method of a role playing model according to any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the performance determination method of a role playing model according to any of claims 1 to 8.