CN117221503A - Holographic projection system of digital personal mobile terminal - Google Patents
Holographic projection system of digital personal mobile terminal Download PDFInfo
- Publication number
- CN117221503A CN117221503A CN202311475636.7A CN202311475636A CN117221503A CN 117221503 A CN117221503 A CN 117221503A CN 202311475636 A CN202311475636 A CN 202311475636A CN 117221503 A CN117221503 A CN 117221503A
- Authority
- CN
- China
- Prior art keywords
- digital
- mobile terminal
- digital person
- module
- person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 claims abstract description 97
- 230000002452 interceptive effect Effects 0.000 claims description 35
- 238000012545 processing Methods 0.000 claims description 31
- 238000004891 communication Methods 0.000 claims description 19
- 238000004519 manufacturing process Methods 0.000 claims description 12
- 230000009471 action Effects 0.000 claims description 11
- 238000003384 imaging method Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 5
- 238000000034 method Methods 0.000 description 22
- 238000003058 natural language processing Methods 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000001953 sensory effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000009877 rendering Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application relates to the technical field of artificial intelligence, in particular to a holographic projection system of a digital personal mobile terminal. The digital man mobile terminal holographic projection system at least comprises: a mobile terminal and a digital person service platform; the mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction and controlling projection of a digital person in space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user; the digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content.
Description
Technical Field
The application belongs to the technical field of artificial intelligence, and particularly relates to a holographic projection system of a digital personal mobile terminal.
Background
In the current information communication field, as digital people are gradually perceived by people, the development of digital people technology is also gradually changed, but the technical development of digital people is still in a starting stage. The technology of digital people mainly focuses on digital people generation and digital people playing in mobile phones or computers, mobile terminals develop rapidly due to the advantages of the mobile terminals, and the projection of digital people is mainly realized by combining large projection equipment with computers, so that the mobile terminals are poor in convenience.
Therefore, there is a need for a digital human projection system with high convenience.
The above information disclosed in the background section is only for enhancement of understanding of the background of the application and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
In order to solve the problems, the application adopts the following technical scheme:
a digital personal mobile terminal holographic projection system comprising at least: a mobile terminal and a digital person service platform; wherein,
the mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction and controlling projection of a digital person in space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user;
the digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content.
In an alternative embodiment of the present application, the holographic projection module comprises at least:
The multimedia processing chip is arranged on the main board of the mobile terminal and is used for providing at least one of image, video and voice calculation and processing of the digital person;
the light source module is arranged on the lens module of the mobile terminal and used for providing a light source for imaging of the digital person;
and the lens module is arranged on the lens module of the mobile terminal and used for controlling and adjusting the optics emitted by the light source module so as to realize projection imaging of the digital person.
In an alternative embodiment of the present application, the lens module includes at least: a grating and a lens.
In an alternative embodiment of the present application, the auxiliary information module includes at least: microphone and microphone processing module, speaker and speaker processing module.
In an alternative embodiment of the present application, the digital person application program provides an interactive interface of the digital person on a graphical user interface at the time of running, wherein the interactive interface at least comprises: and the digital person control touch button is used for responding to the user control operation and controlling the digital person to be put in or turned off based on the user control.
In an optional embodiment of the present application, the interactive interface includes a plurality of selection controls, one selection control corresponds to one type of digital person, and the selection control is used for responding to a selection operation of a user, and determining a target digital person to be projected from a digital person library according to the selection operation; the application industries of all the digital persons in the digital person library are not identical.
In an alternative embodiment of the present application, the digital personal service platform includes at least:
the digital person production service system is in communication connection with the holographic projection module of the mobile terminal, and is used for constructing or storing a digital person system and determining the interactive content for responding to the interactive information according to the interactive information sent by the mobile terminal through a digital person application program; wherein the interactive content includes, but is not limited to: action content, voice content, and image content.
In an alternative embodiment of the present application, the digital personal service platform further includes:
the artificial intelligent module is in communication connection with the digital person production service system, and is used for analyzing the interactive information sent by the digital person production service system and generating voice content and/or action content corresponding to the interactive information according to the analyzed content.
In an alternative embodiment of the present application, the digital personal service platform further includes:
and the user management module is used for realizing the access and the management and control of the users corresponding to the mobile terminals.
In an optional embodiment of the present application, the above-mentioned digital personal mobile terminal holographic projection system further includes:
And the third party platform is in communication connection with the digital person service platform and is used for accessing the corresponding digital person based on the digital person service platform.
The holographic projection system of the digital personal mobile terminal provided by the embodiment of the application at least comprises: a mobile terminal and a digital person service platform; the mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction and controlling projection of a digital person in space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user; the digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content. According to the first aspect, through interaction combination of the mobile terminal and the digital person service platform, projection, display and interaction of digital persons can be achieved, and convenience is better compared with that of large projection equipment in a traditional mode.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a schematic structural diagram of a holographic projection system of a digital personal mobile terminal according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a holographic projection module in a holographic projection system of a digital personal mobile terminal according to an embodiment of the present application;
FIG. 3 is an interaction schematic diagram of a holographic projection system of a digital personal mobile terminal provided by an embodiment of the application;
fig. 4 is a schematic structural diagram of a digital personal service platform in the holographic projection system of the digital personal mobile terminal according to the embodiment of the present application;
FIG. 5 is a flowchart of a digital human voice interaction method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of interaction in a digital personal service platform in a digital personal voice interaction method according to an embodiment of the present application;
FIG. 7 is a flowchart of determining a first semantic meaning in a digital human voice interaction method according to an embodiment of the present application;
FIG. 8 is a flow chart illustrating the analysis of determining the first semantic meaning in the digital human voice interaction method according to an embodiment of the present application;
FIG. 9 is a flowchart of determining and playing second voice information in a digital human voice interaction method according to an embodiment of the present application;
fig. 10 is an interaction schematic diagram of a digital human voice interaction method in a mobile terminal according to an embodiment of the present application.
Detailed Description
In order to make the technical solutions and advantages of the embodiments of the present application more apparent, the following detailed description of exemplary embodiments of the present application is provided in conjunction with the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application and not exhaustive of all embodiments. It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other.
In the current information communication field, as digital people are gradually perceived by people, the development of digital people technology is also gradually changed, but the technical development of digital people is still in a starting stage. The technology of digital people mainly focuses on digital people generation and digital people playing in mobile phones or computers, mobile terminals develop rapidly due to the advantages of the mobile terminals, and the projection of digital people is mainly realized by combining large projection equipment with computers, so that the mobile terminals are poor in convenience.
Therefore, there is a need for a digital human projection system with high convenience.
Based on this, the embodiment of the application provides a holographic projection system of a digital personal mobile terminal, please refer to the figure, the holographic projection system of the digital personal mobile terminal at least comprises: mobile terminal and digital man service platform, wherein:
the mobile terminal refers to a user operation terminal, and comprises hardware equipment, a software system and the like. The improvement of the hardware part comprises: a holographic projection module, such as a high-performance image/video processing chip, a high-brightness and high-resolution light source, a precise projection grating, a lens and the like, is added to a main board part of the mobile terminal; of course, the corresponding shell, main board circuit layout, etc. of the mobile terminal will also be adjusted accordingly, which will not be described herein in detail, and can be adjusted flexibly according to the actual situation. The improvement of the software module comprises: the improvement of the mobile phone operation system is that a communication interface with a digital personal service platform and a communication interface with other auxiliary information modules are added, so that the communication between the holographic projection module and the auxiliary information modules is realized. The mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction, for example, turning on or off holographic projection of a digital person, and controlling the projection of the digital person in a space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user. The mobile terminal may be a mobile phone terminal as in fig. 1, or may be other wearable devices with optical projection functions, such as PAD, and the embodiment of the present application is not specifically limited, and may be flexibly set according to practical situations.
The digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content. The interactive information refers to action information, voice information, image or video information and the like generated by a digital person, and action information, voice information, image or video information and the like generated by a user, wherein the information is collected through a mobile terminal and sent to a digital person service platform through the digital person application program.
The holographic projection system of the digital personal mobile terminal provided by the embodiment of the application at least comprises: a mobile terminal and a digital person service platform; the mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction and controlling projection of a digital person in space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user; the digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content. According to the first aspect, through interaction combination of the mobile terminal and the digital person service platform, projection, display and interaction of digital persons can be achieved, and convenience is better compared with that of large projection equipment in a traditional mode.
In the embodiment of the application, the holographic projection module and the auxiliary information module are arranged on the mobile terminal to open a communication interface between the mobile terminal and the digital man service platform, holographic projection support is provided on the mobile terminal, and the novel communication protocol and the novel open interface are used for realizing important operations such as opening, throwing, closing and the like of the holographic projection module through an upper digital man application program. Meanwhile, compared with the traditional large-sized projection equipment which needs an external equipment shell, a power supply, a communication line and equipment, the holographic projection module only extracts the holographic projection part at the most core, realizes the driving of the mobile terminal mainboard and invokes the holographic projection module in a mode of projecting lower pixels with lower power, is suitable for the small-sized mobile terminal, and saves more energy on the premise of guaranteeing the projection effect.
The embodiment of the application changes the capabilities of opening projection, throwing hologram, closing projection and other important projection operations in a manner of driving related hardware such as a projector main board into a novel soft and hard integrated manner by combining application software (namely digital personal application program) at the upper layer of the mobile terminal with the mobile terminal.
In an optional embodiment of the present application, in the mobile terminal of the embodiment of the present application, the interactive communication protocol between the operating system and the related modules such as the motherboard, the holographic module, the speaker, etc. may be as follows:
1) Between the digital personal application program and the operating system: the operating system is modified to have the capabilities of decoding and playing the digital human video and calling the holographic module and the loudspeaker according to the actual condition of the digital human video, and the protocol format between the digital human application program and the operating system is as follows:
wherein:
the session ID is the identification ID of a certain digital person, or the ID which is specially allocated and corresponds to the certain digital person one by one for the digital person application program to identify the digital person which is specifically played;
when a plurality of messages are required to transmit digital human videos under the condition that the message sequence is the same session ID, the sequence IDs of different messages are identified;
the command type is a command type related to digital person projection, for example, the following three commands can be used: opening a holographic Projection module (Open), projecting, closing the holographic module (Close); video data and audio data are 0 when the commands are Open holographic projection module (Open) and Close holographic projection module (Close);
Message tags are mainly two types: when 1 is taken, the subsequent information is indicated, when 0 is taken, the information is indicated to be the last information, and if a closing command is not received, the holographic projection module can be closed automatically;
the message body is digital person specific multimedia data, including video stream data and audio data streams.
When the operating system receives the command in the above protocol format, executing the corresponding command:
and when receiving an Open holographic module (Open) command, the method passes through a main board in the mobile terminal. The control system and the like drive the holographic projection module to be opened;
when a Projection command is received, video data is sent to a holographic Projection module through a main board, projection is carried out through an auxiliary information module, audio data is sent to the auxiliary information module through the main board, and playing is carried out through a loudspeaker;
when a Close holographic module (Close) command is received, the holographic projection module is closed through the main board drive.
The protocol format between the operating system and the motherboard/holographic projection module may be the protocol format between the digital personal application program and the operating system, which is not described herein.
The improved operating system has the capability of synchronizing audio and video, on one hand, video data is sent to the holographic projection module for projection through the main board, and on the other hand, audio data is sent to the loudspeaker for playing through the main board, so that both parties cannot lose synchronization.
The protocol of the loudspeaker called by the operating system can be the existing general protocol of the mobile terminal, and only the aim of information interaction can be achieved.
Referring to fig. 2, in an alternative embodiment of the present application, the holographic projection module at least includes: the system comprises a multimedia processing chip, a light source module and a lens module, wherein:
the multimedia processing chip is arranged on a main board of the mobile terminal and is used for providing at least one of image, video and voice calculation and processing of a digital person; complex calculation and processing of the holographic image/video is realized, and the digital person in the digital person service platform can be presented in a holographic image/video mode. The multimedia processing chip may include: the voice processing module, the image processing module, the video processing module, the action capturing and processing module and the like are not exhaustive and can be flexibly set according to actual situations. When the holographic projection module receives an opening instruction of an operating system and establishes a digital person putting channel, the high-performance image/video processing chip processes digital person video information transmitted from the digital person service platform and calls the light source module to generate a high-brightness and high-resolution light source.
The light source module is arranged on the lens module of the mobile terminal, and can generate a light source with high brightness, wide color gamut and high resolution, namely, the light source module is used for providing a light source for imaging of the digital person.
The lens module is arranged on the lens module of the mobile terminal and is used for controlling and adjusting the optics emitted by the light source module, for example, the light can be finely controlled and adjusted, and the differentiation and redirection of the light can be realized, so that the projection imaging of a digital person can be realized. The lens module puts digital people into corresponding projection space through light generated by the light source. The projection space may be an area of space or an area where a holographic projection screen is located.
According to the embodiment of the application, the multimedia processing chip, the light source module, the lens module and the like are arranged in the holographic projection module, so that holographic imaging of a digital person is supported on hardware, the richness of the interaction information of the digital person is increased, and meanwhile, the processing comprehensiveness of the interaction information of the digital person is improved, so that a more real sensory experience effect is provided for a user.
In an alternative embodiment of the present application, the lens module includes at least: a grating and a lens. The parameters of the grating may be, for example: gratings of small dimensions (e.g. 25.4 mm outer diameter, 3 mm thick) (spectral range: 350 nm-2400 nm, wavefront distortion: typically 1/5 wave, more precisely customizable, wavefront uniformity: <1/40 wave RMS, spatial frequency: 125 lp/mm— -3600 lp/mm), parameters of the lens may be, for example: resolution 19201080. Viewing angle 168/178 °, brightness 400 cd/square meter, and color 167.7m. The higher brightness and higher resolution of the digital person can be presented through the grating and the lens of the parameter, the digital person imaging is more real, and the sensory experience of the user is improved.
In an alternative embodiment of the present application, the auxiliary information module includes at least: microphone and microphone processing module, speaker and speaker processing module. For example, the voice of the user is captured through a microphone, then the voice information is sent to a digital person service platform through a digital person application program, the digital person service platform responds and generates response content corresponding to the voice information, the speaker processing module analyzes and converts the response content into a format and sends the format through a speaker, and the voice interaction between the digital person and the user can be realized after the user hears the voice. According to the embodiment of the application, the interactive effect on the voice of the user and the digital person is improved from three dimensions of voice input and output and voice information processing, so that a more real sensory experience effect is provided for the user.
In an alternative embodiment of the present application, the digital person application program provides an interactive interface of the digital person on a graphical user interface at the time of running, wherein the interactive interface at least comprises: and the digital person control touch button is used for responding to the user control operation and controlling the digital person to be put in or turned off based on the user control. The digital person application program is an upper-layer digital person service application, and when a user needs to put a certain digital person, the user can click a corresponding 'putting' button through the digital person service application (which can be APP, web, or any type of application capable of interacting with the user), so that the putting and interaction of the digital person are realized. When the digital person needs to be closed, a closing button in the application can be clicked to realize the closing of the digital person; the digital person service application can also manage various types of digital persons, can display various types of digital persons in a list form, and is convenient for a user to select a certain digital person for putting and interaction.
In an optional embodiment of the present application, the interactive interface includes a plurality of selection controls, one selection control corresponds to one type of digital person, and the selection control is used for responding to a selection operation of a user, and determining a target digital person to be projected from a digital person library according to the selection operation; the application industries of the digital persons in the digital person library are not identical, such as film industry, media industry, game industry, financial industry, travel industry and the like. Through the industry application module, digital people aiming at different industries are provided, the digital people in different industries can be supported to be put in through the channel, the application field and the application range of the digital people are further improved, and the fineness is higher.
In an alternative embodiment of the present application, the digital personal service platform includes at least: the system comprises a digital person production service system, an artificial intelligence module and a user management module, wherein:
the digital person production service system is in communication connection with the holographic projection module of the mobile terminal, and is used for constructing or storing a digital person system and determining the interactive content for responding to the interactive information according to the interactive information sent by the mobile terminal through a digital person application program; wherein the interactive content includes, but is not limited to: action content, voice content, and image content. The digital person production service system comprises: including modeling systems, motion capture systems, rendering platforms, solution platforms, etc., for producing digital persons meeting industry needs. The modeling system is used for constructing digital persons matched with the requirements in real time according to the digital person requirements of the mobile terminal or the third party platform, such as digital persons aiming at the business industry or providing different skins in different scenes, and the like, is not particularly limited, and can be flexibly adjusted according to actual conditions; the motion capture system can be used for capturing the motion of a user through the external arrangement of a sensor and the like, and can also be used for capturing the motion information of a digital person according to the digital person position information collected by the mobile terminal; the rendering platform is used for rendering the light and shadow, color and the like of the digital person so as to improve the imaging effect of the digital person. The solution platform can be configured with a plurality of sets of early warning schemes, corresponding early warning schemes are configured according to preset accidents, and once the accidents are triggered, the matched early warning schemes are directly started, so that automatic processing of closed-loop schemes is realized, and the stability is higher; meanwhile, the solution platform can be set to be open-source, and once the accident is triggered, an alarm signal can be sent out so that a worker can determine the corresponding accident according to the alarm signal, and therefore, the investigation can be timely and efficiently carried out, and the problem can be quickly solved.
The artificial intelligent module is in communication connection with the digital person production service system, and is used for analyzing the interaction information sent by the digital person production service system and generating voice content and/or action content corresponding to the interaction information according to the analysis content. The artificial intelligence module is an AI capability platform and can include: computer vision, intelligent voice, natural language processing and the like, voice and action of a digital person are generated, voice and language replied by a user are recognized, voice and action (audio and video) and the like, which are replied to the user by the digital person, are generated according to recognized content, the interaction intelligent degree of the digital person and the user is improved, and the sensory experience of the user is improved.
The user management module is used for realizing the access and the management and control of the users corresponding to the mobile terminals, managing the business clients (merchants), facilitating the access of the business client system (third party platform) to the digital person service platform, and facilitating the overall management and control of the users and the digital persons.
In an alternative embodiment of the present application, the digital personal mobile terminal holographic projection system further comprises: a third party platform, wherein:
The third party platform is in communication connection with the digital person service platform and is used for accessing corresponding digital persons based on the digital person service platform. The third party platform is a third party platform (merchant platform) with digital person requirements, such as an operation platform of a certain travel company, and the digital person service platform opens holographic digital person releasing service to the third party platform through binding with a certain travel digital person of the digital person service platform, and the digital person service platform opens a corresponding interface to the third party platform through an industry client management module, so that digital person releasing of a third party can be supported through the channel. And the user starts the digital person projection service in the digital person service application program of the mobile terminal and interacts with the digital person of the travel, so that the services of explanation, display, interaction and the like of the travel are obtained.
Referring to fig. 3, the following is a brief description of the interaction procedure of the holographic projection system of the digital personal mobile terminal according to the embodiment of the present application in combination with all the embodiments described above:
step 301, a user clicks a "project digital person" button on a digital person application program of the mobile terminal (before this, the digital person application program may have a digital person list, so that the user can conveniently select a certain digital person to project);
Step 302, a mobile terminal holographic projection module acquires corresponding digital persons from a digital person service platform through a mobile terminal;
step 303, the digital person service platform constructs a corresponding digital person or constructs a corresponding digital person from the prior;
step 304, the digital person service platform returns the corresponding digital person;
step 305, the mobile terminal opens the holographic projection module through an operating system and a main board;
step 306, a digital person projection channel is established among the holographic projection module, the digital person service module and the digital person service platform;
step 307, the holographic projection module projects the digital person to the space opposite to the projection port, the corresponding digital person is displayed in the space, and the digital person service module calls the auxiliary information module in the mobile phone according to the playing scene of the digital person to realize interaction between the user and the digital person; for example, when a digital person speaks, the digital person service module calls a loudspeaker to make a sound, and the sound made by the user is transmitted to the digital person service module through a microphone and then transmitted to the digital person service platform, so that interaction between the user and the digital person is realized;
step 308, after the digital person interaction is finished, the user clicks a close button of a graphical user interface in the digital person application program, so that the digital person can be closed;
Step 309, the digital person application program of the mobile terminal sends a closing request to the digital person service platform;
step 310, the digital person service platform closes the digital person;
step 311, the digital person service platform returns a corresponding response;
step 312, the digital personal service module of the mobile terminal closes the channel through the operating system and the motherboard, and closes the holographic projection module.
In describing the present application and its embodiments, it should be understood that the orientation or positional relationship indicated by the terms "top", "bottom", "height", etc. are based on the orientation or positional relationship shown in the drawings, are merely for convenience of description and simplification of the description, and are not indicative or implying that the apparatus or element in question must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present application.
In the present application and its embodiments, unless explicitly specified and limited otherwise, the terms "disposed," "mounted," "connected," "secured," and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrated; the device can be mechanically connected, electrically connected and communicated; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present application and its embodiments, unless explicitly specified and limited otherwise, a first feature "above" or "below" a second feature may include both the first and second features being in direct contact, and may also include both the first and second features not being in direct contact but being in contact with each other by way of additional features therebetween. Moreover, a first feature being "above," "over" and "on" a second feature includes the first feature being directly above and obliquely above the second feature, or simply indicating that the first feature is higher in level than the second feature. The first feature being "under", "below" and "beneath" the second feature includes the first feature being directly above and obliquely above the second feature, or simply indicating that the first feature is less level than the second feature.
The above disclosure provides many different embodiments, or examples, for implementing different structures of the application. The foregoing description of specific example components and arrangements has been presented to simplify the present disclosure. They are, of course, merely examples and are not intended to limit the application. Furthermore, the present application may repeat reference numerals and/or letters in the various examples, which are for the purpose of brevity and clarity, and which do not themselves indicate the relationship between the various embodiments and/or arrangements discussed. In addition, the present application provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.
In addition, the mobile terminal is at least provided with a holographic projection module for throwing digital people in the mobile terminal and an auxiliary information module for receiving and transmitting voice information, and the digital people service platform is used for processing and replying the voice information. As shown in fig. 4, the digital person service platform further adds a corpus knowledge base in the above structure, where the corpus knowledge base includes reply corpora corresponding to different semantic contents.
The following describes the digital human voice interaction method provided by the embodiment of the application in detail:
referring to fig. 5, the digital human voice interaction method provided by the embodiment of the present application includes the following steps 501 to 503:
step 501, an auxiliary information module is called to receive the first voice information, and the first voice information is sent to a digital personal service platform.
The first voice information refers to intelligent voice information sent by a digital person put in the holographic projection module and/or natural voice information sent by a user. The voice information in the embodiment of the present application may be voice information, or may refer to digital information of voice subjected to format conversion, etc., which is not particularly limited herein, and may be configured as arbitrary voice information according to actual situations.
Step 502, analyzing the first voice information in the digital person service platform to obtain a first semantic of the first voice information.
The first semantic meaning is the content to be expressed by the first voice information, and the first semantic meaning may be represented by text, voice or any other form, which is not specifically limited herein. The parsing process may employ, for example, hidden Markov Models (HMMs), gaussian Mixture Models (GMMs), etc., which are not intended to be exhaustive.
Step 503, determining second voice information matched with the first semantic in the digital personal service platform, and sending the second voice information to the auxiliary information module for playing.
That is, the digital person service platform may reply the first semantic meaning, so as to respond and reply the first voice information sent by the user or the digital person, and send the first voice information to the auxiliary information module of the mobile terminal, so as to play the first voice information through a speaker and the like in the auxiliary information module, thereby realizing voice interaction between the mobile terminal and the digital person.
The digital human voice interaction method provided by the embodiment of the application is applied to a digital human mobile terminal holographic projection system, and the digital human mobile terminal holographic projection system at least comprises: the digital human voice interaction method comprises the following steps of: the method comprises the steps of calling an auxiliary information module to receive first voice information, sending the first voice information to a digital person service platform, analyzing the first voice information at the digital person service platform to obtain first semantics of the first voice information, determining second voice information matched with the first semantics at the digital person service platform, and sending the second voice information to the auxiliary information module for playing.
The method is characterized in that a mobile terminal receives first voice information sent by a user or a digital person, a digital person service platform is configured, the digital person service platform responds to the first voice information through second voice information and sends the first voice information to an auxiliary information module of the mobile terminal for playing, so that voice interaction between the mobile terminal and the digital person is realized. Meanwhile, unidirectional digital person throwing is converted into bidirectional interaction between a user and the digital person, and more realistic and vivid human-computer interaction experience is provided.
Referring to fig. 6, in an alternative embodiment of the present application, the digital personal service platform at least includes: the system comprises an interaction logic module, an intelligent voice module, a natural language processing module and a corpus knowledge base. The interactive logic module is used for realizing communication and interaction between the mobile terminal and the digital person service platform, such as information receiving and transmitting (e.g. sending voice information of a digital person to the mobile terminal, receiving and processing reply voice information from a human, etc.), protocol configuration of an interface, etc. The intelligent voice module is used for converting intelligent voice and natural voice, wherein the intelligent voice refers to voice sent by a user, and the natural voice refers to voice sent by the user. The natural language processing module is used for semantic analysis, and a language analysis model, such as a WACSP semantic analysis model, an LR-ASPP model and the like, is configured in the natural language processing module, and is not exhaustive, and only the purpose of semantic analysis can be achieved. The corpus knowledge base comprises reply corpora corresponding to different semantic contents, the corpus knowledge base is a corpus knowledge base for speaking and replying for digital persons with voice interaction, the corpus knowledge base is organized according to an artificial intelligence technology and is used for being called by an intelligent voice and natural language processing module so as to realize natural interaction between the digital persons and the natural persons, and the more abundant the data of the corpus knowledge base, the smoother the interaction; the data of the library supports interaction between natural people and digital people, and the self-enrichment of corpus content can be realized through continuous updating.
Referring to fig. 7, if the first voice information is intelligent voice information, the step 502 of analyzing the first voice information in the digital personal service platform to obtain a first semantic of the first voice information includes the following steps 701-703:
referring to fig. 6, step 701, invoking an interaction logic module at the digital personal service platform to receive first voice information, and sending the first voice information to the intelligent voice module;
step 702, converting the format of the first voice information into natural voice information in the intelligent voice module;
step 703, sending the first voice information after format conversion to a natural language processing module, and performing semantic analysis on the first voice information in the natural language processing module to obtain a first semantic of the first voice information.
The embodiment of the application at least configures the digital personal service platform: the interactive logic module and the intelligent voice module realize the receiving and transmitting of voice information based on the interactive logic module, namely realize the information interaction with the mobile terminal, then realize the conversion between natural language and intelligent language through the intelligent voice module, finally carry out semantic analysis on the first voice through the natural language processing module and respond and reply based on the corpus content in the corpus knowledge base, so that the efficiency is higher, and the voice interaction between the natural language of the user and the intelligent language of the digital person is realized; and each module works independently, so that the interference is small and the reliability is higher.
In an optional embodiment of the present application, the step 503 of determining, at the digital personal service platform, second voice information matched with the first semantic meaning, and sending the second voice information to the auxiliary information module for playing, includes the following steps:
and determining second voice information matched with the first semantics from a corpus knowledge base in the digital personal service platform, and sending the second voice information to an auxiliary information module for playing.
The embodiment of the application carries out the determination of the first semantic matching content based on the corpus knowledge base pre-configured in the digital personal service platform, has high reliability, lower network requirements and higher efficiency; meanwhile, the corpus knowledge base can be continuously updated, so that the richness of voice interaction is improved.
Referring to fig. 8, in an alternative embodiment of the present application, the step 703 of performing semantic analysis on the first voice information in the natural language processing module to obtain the first semantic of the first voice information includes the following steps 801 to 802:
step 801, extracting keywords from the first voice information by the natural language processing module based on a pre-trained language processing model to obtain semantic keywords;
step 802, determining a first semantic of the first voice information based on the semantic keyword.
The language processing model refers to a neural network model capable of extracting keywords, for example, a TF-IDF model, a TextRank model, etc., which is used for extracting keywords from text, or may be a neural network model capable of directly extracting keywords from speech, etc., which will not be described herein.
According to the embodiment of the application, the first voice information is extracted to obtain the semantic keywords based on the pre-trained language processing model, and then the first semantics of the first voice information are determined based on the semantic keywords, so that the data processing amount during the first semantics determination can be reduced, the efficiency of generating the second voice information is further improved, and the interaction efficiency of the digital human voice interaction method is further improved.
Referring to fig. 9, in an alternative embodiment of the present application, the steps of determining, in the digital personal service platform, second voice information matched with the first semantics from the corpus knowledge base, and sending the second voice information to the auxiliary information module for playing, include the following steps 901-902:
step 901, determining second semantic information matched with the first semantic from a corpus knowledge base in a digital person service platform, and sending the second semantic information to an intelligent voice module;
Step 902, the intelligent voice module converts the second semantic information into second voice information, and sends the second voice information to the auxiliary information module for playing.
With continued reference to fig. 6, first, second semantic information corresponding to the first semantic is determined based on a corpus knowledge base in the digital personal service platform, and then the second semantic information is converted into second voice information, that is, the corpus knowledge base contains corresponding relations between different semantics, conversion of the corresponding relations between the semantics and the voice is not needed, and the capacity of the corpus knowledge base is reduced, so that the storage pressure of the digital personal service platform is reduced, and the structural volume of the holographic projection system of the digital personal mobile terminal in the embodiment of the application is reduced.
In an optional embodiment of the present application, the step 902 of converting the second semantic information into the second voice information at the intelligent voice module and sending the second voice information to the auxiliary information module for playing includes the following steps:
and converting the second semantic information into second voice information at the intelligent voice module, sending the second voice information to the interactive logic module, and sending the second voice information to the auxiliary information module for playing based on the interactive logic module.
In the digital personal service platform, the second semantic information is converted into the second voice information based on the intelligent voice module, and then the second voice information is sent to the mobile terminal through the interactive logic module, and the modules work independently, so that the interference is small, and the reliability of the second voice information transmission can be improved.
In an optional embodiment of the present application, the mobile terminal is configured with a digital personal application program, and the structure and layout of the digital personal application program are described in detail in the above embodiment, which is not described herein again. Correspondingly, the step 503 of sending the second voice information to the auxiliary information module for playing includes the following steps:
and sending the second voice information to a digital personal application program of the mobile terminal, and calling a player to play the second voice information based on the digital personal application program.
The player (such as a loudspeaker and the like) is called by the digital person application program of the mobile terminal to play the second voice information, and meanwhile, the user can control the digital person through the digital person application program, so that interaction is richer, and control convenience is higher.
The above is an interaction process of voice information in a digital personal service platform, and the following is an interaction process of the digital personal voice interaction method provided by the embodiment of the application in a mobile terminal:
Referring to fig. 10, after a user projects a corresponding digital person on a mobile terminal through the above-mentioned digital person mobile terminal holographic projection system, the mobile terminal holographically projects the digital person in the digital person service platform to a corresponding space;
the interactive logic module of the digital person production service module of the digital person service platform organizes the second voice information of the corresponding voice operation of the digital person and returns the second voice information to the digital person application program of the mobile terminal;
the digital personal service application of the mobile terminal plays the second voice information through the holographic projection module and the auxiliary information module;
the user replies the corresponding first voice information of the digital person through a microphone and the like in the auxiliary information module; the first voice information is sent to a digital person service platform through a digital person application program, and the digital person service platform organizes the content to be replied according to the content replied by the first voice information of the user;
the digital person service platform returns corresponding second voice information to the digital person application program of the mobile terminal through the interaction logic module;
the digital personal application program of the mobile terminal plays the voice information through the auxiliary information module operating system, the main board and the loudspeaker;
Thus, the user and the digital person realize corresponding voice interaction through the reciprocating interaction.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A digital personal mobile terminal holographic projection system, comprising at least: a mobile terminal and a digital person service platform; wherein,
the mobile terminal at least comprises a holographic projection module and an auxiliary information module, wherein the holographic projection module is used for receiving a user operation instruction and controlling projection of a digital person in space according to the operation instruction; the auxiliary information module is used for providing an information interaction channel between the digital person and the user;
The digital person service platform is used for receiving the interaction information of the digital person and the user, converting the interaction information into interaction content in a preset format and sending the interaction content to the digital person application program of the mobile terminal so that the digital person application program can control the digital person interaction based on the interaction content.
2. The digital personal mobile terminal holographic projection system of claim 1, in which the holographic projection module comprises at least:
the multimedia processing chip is arranged on the main board of the mobile terminal and is used for providing at least one of image, video and voice calculation and processing of the digital person;
the light source module is arranged on the lens module of the mobile terminal and used for providing a light source for imaging of the digital person;
and the lens module is arranged on the lens module of the mobile terminal and used for controlling and adjusting the optics emitted by the light source module so as to realize projection imaging of the digital person.
3. The digital personal mobile terminal holographic projection system of claim 2, in which the lens module comprises at least: a grating and a lens.
4. The digital personal mobile terminal holographic projection system of claim 1, in which the auxiliary information module comprises at least: microphone and microphone processing module, speaker and speaker processing module.
5. The digital personal mobile terminal holographic projection system of claim 1, in which the digital personal application program provides a digital personal interactive interface at a graphical user interface at runtime, wherein the interactive interface comprises at least: and the digital person control touch button is used for responding to the user control operation and controlling the digital person to be put in or turned off based on the user control.
6. The holographic projection system of claim 5, wherein the interactive interface comprises a plurality of selection controls, one selection control corresponding to each type of digital person, the selection controls being configured to respond to a selection operation by a user and determine a target digital person to be projected from a digital person library according to the selection operation; the application industries of all the digital persons in the digital person library are not identical.
7. The digital personal mobile terminal holographic projection system of claim 1, in which the digital personal services platform comprises at least:
the digital person production service system is in communication connection with the holographic projection module of the mobile terminal, and is used for constructing and/or storing a digital person system and determining the interactive content for responding to the interactive information according to the interactive information sent by the mobile terminal through a digital person application program; wherein the interactive content includes, but is not limited to: action content, voice content, and image content.
8. The digital personal mobile terminal holographic projection system of claim 7, in which said digital personal services platform further comprises:
the artificial intelligent module is in communication connection with the digital person production service system, and is used for analyzing the interactive information sent by the digital person production service system and generating voice content and/or action content corresponding to the interactive information according to the analyzed content.
9. The digital personal mobile terminal holographic projection system of claim 7, in which said digital personal services platform further comprises:
and the user management module is used for realizing the access and the management and control of the users corresponding to the mobile terminals.
10. The digital personal mobile terminal holographic projection system of claim 1, further comprising:
and the third party platform is in communication connection with the digital person service platform and is used for accessing the corresponding digital person based on the digital person service platform.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311475636.7A CN117221503A (en) | 2023-11-08 | 2023-11-08 | Holographic projection system of digital personal mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311475636.7A CN117221503A (en) | 2023-11-08 | 2023-11-08 | Holographic projection system of digital personal mobile terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117221503A true CN117221503A (en) | 2023-12-12 |
Family
ID=89035638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311475636.7A Pending CN117221503A (en) | 2023-11-08 | 2023-11-08 | Holographic projection system of digital personal mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117221503A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519663A (en) * | 2024-01-08 | 2024-02-06 | 广州趣丸网络科技有限公司 | Intelligent production platform for digital people |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140267598A1 (en) * | 2013-03-14 | 2014-09-18 | 360Brandvision, Inc. | Apparatus and method for holographic poster display |
| CN106502377A (en) * | 2015-09-07 | 2017-03-15 | Lg电子株式会社 | Mobile terminal and its control method |
| US20200218198A1 (en) * | 2019-01-04 | 2020-07-09 | International Business Machines Corporation | Movement control of holographic objects with crown movement of a watch device |
| CN112669422A (en) * | 2021-01-07 | 2021-04-16 | 深圳追一科技有限公司 | Simulated 3D digital human generation method and device, electronic equipment and storage medium |
| CN116107463A (en) * | 2022-12-23 | 2023-05-12 | 南京聚贤信息科技有限公司 | 3D virtual digital person interaction system and method |
-
2023
- 2023-11-08 CN CN202311475636.7A patent/CN117221503A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140267598A1 (en) * | 2013-03-14 | 2014-09-18 | 360Brandvision, Inc. | Apparatus and method for holographic poster display |
| CN106502377A (en) * | 2015-09-07 | 2017-03-15 | Lg电子株式会社 | Mobile terminal and its control method |
| US20200218198A1 (en) * | 2019-01-04 | 2020-07-09 | International Business Machines Corporation | Movement control of holographic objects with crown movement of a watch device |
| CN112669422A (en) * | 2021-01-07 | 2021-04-16 | 深圳追一科技有限公司 | Simulated 3D digital human generation method and device, electronic equipment and storage medium |
| CN116107463A (en) * | 2022-12-23 | 2023-05-12 | 南京聚贤信息科技有限公司 | 3D virtual digital person interaction system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117519663A (en) * | 2024-01-08 | 2024-02-06 | 广州趣丸网络科技有限公司 | Intelligent production platform for digital people |
| CN117519663B (en) * | 2024-01-08 | 2024-04-26 | 广州趣丸网络科技有限公司 | Intelligent production platform for digital people |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10650816B2 (en) | Performing tasks and returning audio and visual feedbacks based on voice command | |
| CN115145529B (en) | Voice control device method and electronic device | |
| CN112394895B (en) | Picture cross-device display method and device and electronic device | |
| US7746986B2 (en) | Methods and systems for a sign language graphical interpreter | |
| US8825468B2 (en) | Mobile wireless display providing speech to speech translation and avatar simulating human attributes | |
| CN100433828C (en) | Information-processing apparatus, information-processing methods, recording mediums, and programs | |
| EP3435373B1 (en) | Mobile wireless display providing speech to speech translation and avatar simulating human attributes | |
| CN110503959B (en) | Voice recognition data distribution method and device, computer equipment and storage medium | |
| CN117221503A (en) | Holographic projection system of digital personal mobile terminal | |
| WO2022089224A1 (en) | Video communication method and apparatus, electronic device, computer readable storage medium, and computer program product | |
| CN101809651A (en) | The mobile wireless display of the incarnation of speech to speech translation and simulating human attribute is provided | |
| CN114697742A (en) | Video recording method and electronic equipment | |
| KR20220109373A (en) | Method for providing speech video | |
| WO2021031862A1 (en) | Data processing method and apparatus thereof | |
| CN117198293B (en) | Digital human voice interaction method, device, computer equipment and storage medium | |
| CN116229311B (en) | Video processing method, device and storage medium | |
| CN115086888B (en) | Message notification method and device and electronic equipment | |
| CN109300472A (en) | A kind of audio recognition method, device, equipment and medium | |
| Ding et al. | Interactive multimedia mirror system design | |
| CN109640164A (en) | A kind of playback method and device between multiple virtual reality devices | |
| WO2021202605A1 (en) | A universal client api for ai services | |
| CN113157241A (en) | Interaction equipment, interaction device and interaction system | |
| WO2011000749A1 (en) | Multimodal interaction on digital television applications | |
| CN117215416B (en) | Holographic communication method and device for mobile terminal, computer equipment and storage medium | |
| KR102546532B1 (en) | Method for providing speech video and computing device for executing the method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |