CN113765903A

CN113765903A - Screen projection method and related device, electronic equipment and storage medium

Info

Publication number: CN113765903A
Application number: CN202110990984.2A
Authority: CN
Inventors: 杨学理; 纪世玮; 孙立刚; 王鹏飞; 史德伟; 何亚涛
Original assignee: iFlytek Co Ltd
Current assignee: iFlytek Co Ltd
Priority date: 2021-08-26
Filing date: 2021-08-26
Publication date: 2021-12-07
Anticipated expiration: 2041-08-26
Also published as: CN113765903B

Abstract

The application discloses a screen projection method, a related device, electronic equipment and a storage medium, wherein the screen projection method comprises the following steps: starting a temporary hotspot based on preset distribution network information; the first screen projection equipment is pre-stored with preset distribution network information and is connected to the temporary hotspot based on the preset distribution network information; acquiring an equipment identifier of first screen projection equipment, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection equipment; closing the temporary hotspot, and starting a screen projection hotspot special for the first screen projection equipment based on the target distribution network information; the first screen projection device is connected to the screen projection hotspot based on the target distribution network information. By the scheme, wireless screen projection connection between the display equipment and the screen projection equipment can be realized.

Description

Screen projection method and related device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of screen projection technologies, and in particular, to a screen projection method, a related apparatus, an electronic device, and a storage medium.

Background

With the steady increase of the economy of China in a new normal state, the traditional industry in China is accelerated to upgrade, and the emerging industry is thrive. The business screen projection equipment has a wide market development space as a necessary link in enterprises and in cooperative discussion of the enterprises. The help of the business screen throwing equipment cannot be left in the scenes of cooperation negotiation, teaching, course giving, project discussion, training and popularization and the like.

Research finds that in a business meeting, for a mode of often using a wired mode to connect projection, the problem that a notebook and a connecting wire are frequently replaced exists in multi-person demonstration, and meeting efficiency is greatly reduced.

Disclosure of Invention

The technical problem mainly solved by the application is to provide a screen projection method, a related device, an electronic device and a storage medium, which can realize wireless screen projection connection between a display device and a screen projection device.

In order to solve the above technical problem, a first aspect of the present application provides a screen projection method, including: starting a temporary hotspot based on preset distribution network information; the first screen projection equipment is pre-stored with preset distribution network information and is connected to the temporary hotspot based on the preset distribution network information; acquiring an equipment identifier of first screen projection equipment, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection equipment; closing the temporary hotspot, and starting a screen projection hotspot special for the first screen projection equipment based on the target distribution network information; the first screen projection device is connected to the screen projection hotspot based on the target distribution network information.

The first screen projection device is connected with the display device in a screen projection mode through a screen projection hotspot, the display device is connected with a second screen projection device, and the second screen projection device is in communication connection with the first screen projection device to achieve screen projection connection; or the first screen projection device is connected with the display device in a screen projection mode through the screen projection hotspot, the display device runs a distribution network program, the distribution network program is used for configuring a network module of the display device, and the network module is used for being in communication connection with the first screen projection device to achieve screen projection connection.

The display equipment is connected with second screen projection equipment, and the second screen projection equipment prestores preset distribution network information; based on preset distribution network information, opening a temporary hotspot, comprising: driving a second screen projection device to start a temporary hotspot based on preset distribution network information; opening a screen projection hotspot based on the target distribution network information, comprising: and driving a second screen projection device to start a screen projection hotspot based on the target distribution network information.

The display equipment runs a distribution network program, and the distribution network program prestores preset distribution network information; based on preset distribution network information, opening a temporary hotspot, comprising: driving a network module of the display device to open a temporary hotspot based on preset distribution network information; opening a screen projection hotspot based on the target distribution network information, comprising: and driving a network module to start a screen projection hotspot based on the target distribution network information.

The screen projection method further comprises the following steps: saving the generated target distribution network information as candidate distribution network information; and opening a screen projection hotspot corresponding to the at least one candidate distribution network information based on the at least one candidate distribution network information selected by the user.

The first screen projection equipment comprises current screen projection equipment and screen projection equipment waiting to be projected, and the current screen projection equipment is connected with the display equipment in a screen projection mode; the screen projection method further comprises the following steps: responding to a screen projection request sent by a screen projection device, keeping screen projection connection with the current screen projection device, and detecting the screen projection state of the current screen projection device; wherein, the screen projection state comprises: keeping any one of the screen projection state and the screen abdicating projection state; and determining whether to switch to screen projection connection with the screen projection equipment waiting for screen projection based on the screen projection state.

Wherein, detect the screen projecting state of throwing the screen equipment at present, include: detecting a yielding request from current screen projection equipment, and determining that a screen projection state is a yielding screen projection state; or, the screen projection state is determined to be the screen projection maintaining state without detecting a yielding request from the current screen projection equipment.

Wherein, on the basis of throwing the screen state, confirm whether switch to throw the screen with waiting to throw the screen equipment and be connected, include: outputting a message to be confirmed based on the screen throwing state as a transition screen throwing state; the message to be confirmed is used for prompting whether to confirm the screen of giving way; and responding to a confirmation yielding message sent by the current screen projection equipment, and determining to switch to screen projection connection with the screen projection equipment waiting for screen projection.

The screen projection method further comprises the following steps: screen projection data from a first screen projection device is received and output; the screen projection data comprise the identification result of the audio data and image data, the image data are collected by the first screen projection device from a source device connected with the first screen projection device, and the audio data are collected by the first screen projection device.

The recognition result comprises a recognition text of the audio data, the recognition text comprises at least one of a first text and a second text, the first text is obtained by directly recognizing the audio data, the first text is represented by a source language of the audio data, the second text is represented by a first target language different from the source language, and the second text and the first text have the same semantic meaning; and/or the recognition result comprises transliterated audio of the audio data, the audio data is expressed in a source language, the transliterated audio is expressed in a second target language different from the source language, and the transliterated audio and the audio data have the same semantic meaning.

The screen projection method further comprises at least one of the following steps: storing the data file to the cloud device; downloading a data file from the cloud equipment; and editing the data file in the cloud equipment.

In order to solve the above technical problem, a second aspect of the present application provides a screen projection apparatus, including: the temporary hotspot module is used for starting a temporary hotspot based on preset distribution network information; the first screen projection equipment is pre-stored with preset distribution network information and is connected to the temporary hotspot based on the preset distribution network information; the distribution network information module is used for acquiring the equipment identifier of the first screen projection equipment, generating target distribution network information based on the equipment identifier and sending the target distribution network information to the first screen projection equipment; the screen projection hotspot module is used for closing the temporary hotspot and starting a screen projection hotspot special for the first screen projection equipment based on the target distribution network information; the first screen projection device is connected to the screen projection hotspot based on the target distribution network information.

In order to solve the above technical problem, a third aspect of the present application provides an electronic device, which includes a memory and a processor coupled to each other, where the memory stores program instructions, and the processor is configured to execute the program instructions to implement the screen projection method in the first aspect.

In order to solve the above technical problem, a fourth aspect of the present application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being configured to implement the screen projecting method in the first aspect.

In the scheme, the display device and the first screen projection device can realize wireless screen projection connection based on preset distribution network information, so that the connection mode between the display device and the first screen projection device is simplified, and the mobility and the portability between the devices are improved. In addition, the unique and unchangeable target distribution network information corresponding to the first screen projection equipment is generated based on the equipment identification of the first screen projection equipment, so that the first screen projection equipment is connected with the display equipment in a screen projection mode based on the respective proprietary target distribution network information, and the safety of screen projection connection based on the target distribution network information is improved.

Drawings

FIG. 1 is a schematic flowchart of an embodiment of an audio processing method provided in the present application;

FIG. 2 is a schematic diagram illustrating an embodiment of an audio data transmission protocol provided herein;

FIG. 3 is a schematic flow chart diagram illustrating another embodiment of an audio processing method provided herein;

FIG. 4 is a schematic flowchart of an embodiment of an upgrade screen projection device provided by the present application;

FIG. 5 is a flowchart illustrating an embodiment of step S35 shown in FIG. 3;

FIG. 6 is a schematic illustration of the translation audio based on audio data identification provided herein;

fig. 7 is a schematic diagram of an embodiment of a cloud device provided herein;

FIG. 8 is a schematic flowchart of an embodiment of a screen projection method provided by the present application;

FIG. 9 is a schematic view of an embodiment of a first screen projecting device and a screen projecting connection of a display device provided herein;

FIG. 10 is a schematic flow chart diagram illustrating another embodiment of a screen projection method provided herein;

FIG. 11 is a schematic flow chart diagram illustrating a screen projection method according to another embodiment of the present application;

FIG. 12 is a block diagram of an embodiment of an audio processing apparatus provided herein;

FIG. 13 is a schematic diagram of a frame of an embodiment of a screen projection apparatus provided herein;

FIG. 14 is a block diagram of an embodiment of an electronic device provided herein;

FIG. 15 is a block diagram of an embodiment of a computer-readable storage medium provided herein.

Detailed Description

The following describes in detail the embodiments of the present application with reference to the drawings attached hereto.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present application.

The terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship. Further, the term "plurality" herein means two or more than two.

Referring to fig. 1, fig. 1 is a schematic flowchart illustrating an audio processing method according to an embodiment of the present disclosure. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 1 is not limited in this embodiment. As shown in fig. 1, the present embodiment includes:

step S11: and acquiring audio data acquired by the screen projection equipment.

In this embodiment, the screen projection device is connected to the source end device through the preset interface, and the screen projection device and the source end device are electrically connected through the preset interface, so that the screen projection device can transmit the acquired audio data to the source end device, the source end device can acquire the audio data acquired by the screen projection device, and the audio data acquired by the screen projection device can be processed conveniently in the following process. Optionally, the preset interface may be a USB-Type-C interface, and the USB-Type-C interface is a USB universal data interface, and is capable of implementing transmission of digital audio data, and transmitting the audio data to the source device in a predetermined format, where the predetermined format of the audio data includes, but is not limited to, 24K/24bit, and the like. It is understood that, in other embodiments, the preset interface may also be other types of interfaces, and may be specifically configured according to actual use requirements, and is not specifically limited herein.

The method of the embodiment is used for processing the audio data collected by the screen projection device, so that the source device includes but is not limited to a computer, a mobile phone, a tablet computer, and the like. In addition, the screen projection equipment has the screen projection function, so when the screen projection equipment realizes the screen projection function, the screen projection equipment can also comprise display equipment such as an intelligent television, and when the screen projection equipment is connected with the display equipment (such as the intelligent television) in a screen projection mode, a computer screen can be projected onto the screen of the display equipment, so that real-time screen projection is realized through wireless transmission.

In an embodiment, the state of a source device connected to the screen projection device through a preset interface is detected to determine which protocol related to the preset interface is used to acquire audio data acquired by the screen projection device. For example, it may be determined which protocol is used for audio data transmission by detecting whether the source device currently runs a custom program of the screen projection device. Of course, it is also possible to determine which protocol is used for audio data transmission by detecting other statuses of the source device. It is to be understood that, in other embodiments, it may also be determined by detecting the type and size of the audio data, a transmission protocol supported by the source device, and other manners, which protocol is used to obtain the audio data collected by the screen projection device, and is not limited in this respect.

In a specific embodiment, it is determined which protocol associated with the preset interface is used to acquire the audio data by detecting whether the source device runs a customized program of the screen projection device. Specifically, when the detection result is that the source end device runs a customized program of the screen projection device, audio data acquired by the screen projection device is acquired by adopting a first protocol; and when the detection result is that the source end device does not run the customized program of the screen projection device, acquiring the audio data acquired by the screen projection device by adopting a second protocol. The first protocol is a self-defined protocol of a preset interface, and the self-defined protocol is a protocol which can be written by a user according to actual use requirements; the second protocol is a standard protocol of the predetermined interface, that is, the second protocol is a normal protocol corresponding to the predetermined interface. Of course, in other embodiments, after detecting that the source device runs the customized program of the screen projection device, the second protocol may also be used to transmit the audio data to the source device, which may be specifically set according to the use requirement, and is not specifically limited herein. The customized program is a software program which is installed on the source end device and can transmit a control instruction to the screen projection device, the audio data can be sent to the server to be processed according to user requirements after the source end device receives the audio data, and the screen projection device and the customized program are matched with each other to process the audio data through the first protocol channel.

For example, as shown in fig. 2, fig. 2 is a schematic structural diagram of an embodiment of an audio data transmission protocol provided in the present application, where a preset interface is a USB-Type-C interface, and after detecting that a source end device has been installed and runs a customized program of a screen projection device, the screen projection device may transmit audio data to the source end device through a data channel of a custom protocol, that is, the custom protocol, so that the source end device obtains the audio data collected by the screen projection device; when the source end device is detected not to be provided with the customized program of the screen projection device or provided with the customized program of the screen projection device and not to run the customized program, the source end device is determined not to run the customized program of the screen projection device, for example, the source end device runs common programs such as an announcement conference, a nail, a WeChat and the like, and the screen projection device can transmit audio data to the source end device through a UAC channel of a USB-Type-C interface, namely a standard protocol channel of the USB-Type-C interface.

It should be noted that, in this embodiment, the screen projection device can be used for screen projection, and a microphone is integrated in the screen projection device, and the microphone can collect a voice signal, so that the screen projection device integrated with the microphone can collect audio data. That is to say, throw the screen equipment and collect screen and audio data collection integration equipment as an organic whole, increased the application scope of throwing the screen equipment, avoid throwing screen equipment application scope single for throw the screen equipment possess other functions except that throw the screen function. The screen projection equipment can be used for screen projection, recording and other operations in a conference, the number of hardware equipment is reduced, the aims of rapidness and integration of a conference process are achieved, and diversified requirements under a modern multi-person conference scene are met.

In one embodiment, the microphone of the screen projection device configuration is an array microphone made up of a plurality of microphones. Alternatively, it may be a microphone array consisting of 2, 3, 4, 6 or 8 microphones, etc. Alternatively, the microphone may be a silicon microphone, which has high sensitivity, is small and light-weight and can directly convert a sound signal into an electric energy signal; of course, the microphone may be another type of microphone such as a rod, and is not limited in particular.

For example, a microphone array composed of 6 microphones is taken as an example, and the 6 microphones are uniformly arranged on a circumference, so that the screen projection equipment can carry out sound acquisition in 360 degrees in all directions, the sound within 10 m range of the screen projection equipment can be acquired, and the quality of the radio service of the screen projection equipment is improved, so that when the screen projection equipment is applied to a business meeting with a large space and multiple persons, on one hand, no extra equipment is needed for recording, and the screen projection equipment has a recording function, so that the purpose of recording the whole process of the meeting can be achieved; on the other hand, far-field reception can be realized without external meeting microphone equipment, so that the sound of all the participants can be collected or the sound of all the participants can be collected and the other party in the meeting can listen successfully, and the meeting can be carried out smoothly. The sound sampling rate of the microphone array formed by 6 microphone arrays can reach 48K 16bit 2, the higher the sampling rate is, the higher the sound definition is, and the definition of the collected sound is improved. Furthermore, the microphone array formed by the 6 microphones also has a sound source positioning function and a noise reduction function, so that sound direction positioning and noise elimination can be accurately carried out, and the definition of audio data collected by the screen projection equipment and transmitted to the source end equipment is improved.

In addition, in order to improve the quality of the audio data output by the screen projection device, such as definition, so as to facilitate subsequent processing of the audio data, in an embodiment, the screen projection device further performs preprocessing such As Echo Cancellation (AEC), Automatic Gain Compensation (AGC), and narrow beam noise reduction on the acquired audio data, and specifically performs what kind of preprocessing on the audio data, which is not specifically limited herein, and may be specifically set according to actual use needs. The echo cancellation technology is a technology for canceling an echo signal in a received signal by estimating the size of the echo signal by using an adaptive method and then subtracting the estimated echo signal value from the received signal, and can ensure that sound emitted from a far end through a loudspeaker is cancelled, avoid that the far end sound signal and a near end sound signal are mixed and then transmitted to the far end together, and improve the experience of a user. The automatic gain compensation technology is a control method that the gain of an amplifying circuit is automatically adjusted along with the signal intensity, when the input signal is weaker, a linear amplifying circuit works to ensure the intensity of the output signal, and when the input signal reaches a certain intensity, a compression amplifying circuit is started to reduce the intensity of the output signal, so that the intensity of the output signal is kept in a certain range, and the volume of the audio data output by the screen projection equipment is ensured to be in the optimal listening range. The narrow beam noise reduction technique is a method for reducing a noise signal of a sound signal by reducing a sound beam width, thereby performing noise reduction processing on the sound signal.

Step S12: and processing the audio data by adopting different strategies based on the state detection result of the source end equipment.

In an embodiment, the processing policy of the audio data may be to store the audio data, identify the audio data, and the like, and the identifying the audio data may be based on transcription, translation, and the like of the audio data, and may be specifically set according to actual needs, and is not specifically limited herein.

In an embodiment, the status detection result is whether the source device currently runs a customized program of the screen projection device. The processing strategy adopted by the audio data is determined based on whether the source end equipment operates the customized program of the screen projection equipment currently, and when the source end equipment operates the customized program, the customized program can send the audio data transmitted by the screen projection equipment to the cloud and other servers for processing required by a user, so that the processing of the audio data is completed, and the application range of the screen projection equipment is expanded. That is to say, as long as it is determined that the source device electrically connected to the screen projecting device is installed and runs the customized program of the screen projecting device, the audio data can be processed by adopting the strategies of identification, transcription, translation and the like without adding other hardware devices, so that the screen projecting device has other functions besides the simple screen projecting function, and the application range of the screen projecting device is expanded. Furthermore, when the screen projection equipment and the customized program are applied to the business conference, the conference requirements such as recording, recording transfer, translation, conference record generation and the like can be completed by using the equipment system, namely, the conference requirements from the conference start to the conference end of a user can be completed by using the same equipment system, so that the business conference flow is rapid, complete and integrated, and the diversified requirements of the conference are met.

In an embodiment, the status detection result may also be whether the source device has a configuration rule for the screen projection device, so as to determine a processing policy for the audio data according to whether the source device has the configuration rule for the screen projection device, where if the source device has the configuration rule for the screen projection device, the audio data is processed by using the configuration rule, and if the source device does not have the configuration rule, the audio data is stored. The configuration rule may be specifically set according to a use requirement, and is not specifically limited herein, for example, the configuration rule may be to transcribe audio data collected by the screen projection device, and the like. The configuration rule corresponding to the screen projection device may be configured in a bottom-layer drive of the source device, or the configuration rule corresponding to the screen projection device may be configured by using a script file, which is not specifically limited herein. It is to be understood that, in other embodiments, the processing policy applied to the audio data may also be determined based on other detection results of the source device, and is not limited in particular herein.

For example, taking a configuration rule as an example for transcribing audio data acquired by a screen projection device, based on the configuration rule that a source device has the screen projection device, after the source device acquires the audio data acquired by the screen projection device, the source device calls the configuration rule to transcribe the audio data. If the configuration rule defines a transfer route of the audio data, for example, the source device performs transfer locally by using a local transfer model or uploads the audio data to the cloud device by using a cloud transfer model, the source device performs local transfer on the audio data or uploads the audio data to the cloud device for transfer according to the transfer route in the configuration rule.

In the above embodiment, the microphone is integrated in the screen projection device, and the microphone can collect voice signals, so that the screen projection device integrated with the microphone can collect audio data, and the application range of the screen projection device is enlarged; in addition, the processing strategy adopted for the audio data is determined based on the current state detection result of the source device. For example, the processing policy of the audio data is determined by whether the source device currently runs the customized program of the screen projection device, and when the source device runs the customized program, the customized program can send the audio data transmitted by the screen projection device to a server such as a cloud to complete processing of the audio data, so that the screen projection device and the customized program are combined, and the application range of the screen projection device can be further expanded. Furthermore, the screen projection equipment and the customized program of the screen projection equipment are applied to the conference, and under the condition that other hardware equipment is not needed to be added, audio data can be subjected to processing such as transcription and translation, namely, the conference requirement from the conference start to the conference end of a user can be met by using the same equipment system, so that the business conference flow is rapid, complete and integrated, and the diversified requirement of the conference is met.

Referring to fig. 3, fig. 3 is a schematic flowchart illustrating an audio processing method according to another embodiment of the present application. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 3 is not limited in this embodiment. As shown in fig. 3, in this embodiment, determining a processing policy to be used for audio data based on whether a source device runs a customized program of a screen projection device specifically includes:

step S31: and acquiring audio data acquired by the screen projection equipment.

Step S31 is similar to step S11, and will not be described herein.

Step S32: and detecting whether the source end equipment runs the customized program of the screen projection equipment.

In this embodiment, whether the source device runs the customized program of the screen projection device is detected to determine the processing policy adopted for the audio data. And if the detection result is that the source device does not run the customized program of the screen projection device, or the source device does not install the customized program of the screen projection device, and at this time, the uninstalled customized program may also be understood as the source device does not detect that the source device runs the customized program of the screen projection device, then step S33 is executed. When the detection result is that the source device runs the customized program of the screen projection device, step S34 is executed.

Step S33: the audio data is saved.

In a specific embodiment, the state detection result indicates that the source device does not run the customized program, that is, it is detected that the source device does not run the customized program of the screen projection device, and the audio data is saved. Specifically, when it is detected that the source device does not run the customized program of the screen projection device, the screen projection device transmits the audio data to the source device through a standard protocol channel of a preset interface, and since the source device does not run the customized program, and cannot perform policy operations such as transcription and translation on the audio data, the source device only stores the audio data at this time, and the screen projection device can be regarded as a microphone for sound collection in this embodiment, which is equivalent to only extending the audio collection function, i.e., the recording function, on the screen projection function of the screen projection device.

In one embodiment, the source device may save the audio data locally. It is to be understood that, in other embodiments, the source device may also store the audio data in the cloud, that is, the user may access the audio data through the cloud. In addition, when the user saves the audio data in the cloud, the user can also edit files such as the audio data on line by accessing the cloud, or the user can also share the files such as the audio data through the cloud, so that other users can share the files such as the audio data, and the cooperative office efficiency is improved.

Step S34: and optimizing to obtain the audio to be recognized based on the voice data and the loop data.

In this embodiment, the audio data may specifically include voice data and loop data, where the voice data is collected by a microphone integrated in the screen projection device, the loop data is generated by the screen projection device according to the collected voice data, and the loop data is used to eliminate an echo in the voice data subsequently, so that the voice data can be optimized by combining the voice data and the loop data to eliminate an echo signal in the voice data, so as to obtain an audio to be recognized. In a specific embodiment, the microphone integrated with the screen projection device is an array microphone composed of 6 microphones, and the sound signals collected by the 6 microphones are arranged and combined by a preset algorithm to obtain voice data and loop data, for example, 6 paths of voice data and 2 paths of loop data. It is understood that, in other embodiments, 3-way or multi-way equal loop data may also be generated, which is not specifically limited herein. In addition, in other embodiments, the audio data may also include other data besides the voice data and the loopback data, and may be specifically set according to actual use needs, which is not specifically limited herein.

In one embodiment, the customized program of the source end device optimizes the audio data to be recognized, for example, performs sound source positioning, noise reduction, automatic gain compensation, reverberation removal and other processing on the audio data to be recognized, so as to improve the optimization degree of the audio data, and further facilitate subsequent processing such as audio data recognition.

In addition, in an embodiment, voice enhancement processing may be performed on the audio data to be recognized, specifically, voice in the audio data is amplified, so that the voice is clearer in a noisy environment, and a conference can be performed smoothly under various conditions. It is to be understood that in other embodiments, other speech processing technologies may also be used to process the audio data to be recognized, and the processing may be specifically set according to actual use needs, and is not specifically limited herein.

In addition, before the audio data is transmitted and processed through the data channel of the custom protocol, the screen projection device and the custom program of the screen projection device need to be connected through the custom protocol. Specifically, the screen projection device is inserted into a USB interface of the source device; the source terminal device searches the pid and the vid of the screen projection device to establish connection with the screen projection device; the source end device monitors the insertion of the screen projection device through attach and monitors the extraction of the screen projection device through detach. After the screen projection device and the source device are successfully connected, the source device obtains the incodpoint of the screen projection device, controls the screen projection device to collect sound through the ondata command of the incodpoint, and obtains a pcm audio source file with a preset format size, such as a 24-bit pcm audio source file, when the screen projection device transmits the collected audio data to the source device.

In addition, since the source device acquires the 24-bit pcm audio source file through the screen projection device, before processing the audio data by the processing strategy correspondingly provided with the custom program, format conversion needs to be performed on the audio source file to obtain an audio stream that can be read by the identification module of the custom program of the screen projection device. Specifically, the acquired audio source file is input into the push of array to convert the pcm audio source file into 16 bits, and then the 16-bit pcm audio source file is subjected to buffer conversion to convert the 16-bit pcm audio source file into a 16-bit opus audio file, so that the audio data can be identified and processed in the future. The audio format converted from the audio source file is specifically set according to the requirements of a subsequent identification model on the file format, and is not specifically limited herein, and because the identification model related to the application can only identify a 16-bit opus audio file, a 24-bit pcm audio source file is converted into a 16-bit opus audio file in the above statement.

Referring to fig. 4, fig. 4 is a schematic flowchart of an embodiment of an upgrade screen projection device provided in the present application. In order to improve the use quality of the screen projection device, and thus improve the experience of the user, in an embodiment, after detecting that the state detection result of the source end device meets a preset condition, the screen projection device may further be upgraded, for example, the whole system of the screen projection device may be upgraded and updated, or some function of the screen projection device may be upgraded and updated, and the like. The condition detection result meeting the preset condition can be that the source end device runs a customized program of the screen projection device; or, the source device may also run a customized program of the screen projection device, and the source device receives an attempted upgrade instruction initiated by the user through the screen projection device; or, the source device may also run a customized program of the screen projection device, and the source device receives an upgrade inquiry message sent by the cloud device; alternatively, other preset conditions may also be used, and may be specifically set according to actual use needs, and are not specifically limited herein. The upgrading of the screen projection equipment specifically comprises the following substeps:

step S41: and acquiring the equipment identifier of the screen projection equipment.

In this embodiment, after establishing connection with the screen projection device, the source device may obtain a device identifier of the screen projection device, so as to determine which screen projection device is to be upgraded and updated subsequently. The device identification of the screen projection device can be regarded as the unique number of the screen projection device, and can be used for distinguishing different screen projection devices, so that the problem that when a plurality of screen projection devices are connected with the source end device, the screen projection devices are wrongly upgraded, and resource waste is caused is avoided.

In a specific embodiment, the customized program of the source device may read the version number of the screen projection device, that is, the device identifier, through the adb shell/usr/local/bin/usr _ tool version command to obtain the device identifier of the screen projection device, and certainly, in other embodiments, the source device may also read the version number of the screen projection device through other commands, which is not specifically limited herein.

Step S42: and acquiring the upgrading firmware suitable for the screen projection equipment based on the equipment identification.

In this embodiment, the upgrade firmware corresponding to the screen projection device is acquired from the cloud device through the acquired device identifier of the screen projection device, and the upgrade firmware is used for upgrading the screen projection device. The upgrading of the screen projection equipment can be the upgrading of the whole system of the screen projection equipment, or the upgrading of a certain function in the screen projection equipment, and corresponding upgrading firmware can be obtained from the cloud equipment according to actual use requirements.

In a specific embodiment, the whole system of the screen projection device is updated, the upgrade firmware of the screen projection device is obtained from the cloud device through a customized program running on the source device, for example, the upgrade firmware is an fw file, and the upgrade firmware is downloaded and stored locally, so that the whole system of the screen projection device is subsequently updated and updated by using the upgrade firmware.

Step S43: and upgrading the screen projection equipment based on the upgrading firmware.

In the embodiment, the screen projection equipment can be upgraded based on the upgrading firmware. Specifically, when the whole system of the screen projection equipment is upgraded and updated, the customization program of the screen projection equipment passes through

The method comprises the steps that an adb shell cp/usr/local/bin/pcba _ test _ tool/tmp command obtains the system permission of the screen projection equipment, then an upgrade firmware is transmitted to a tmp folder of the screen projection equipment through the adb push command, and then the upgrade command is executed through the adb shell cat/tmp/upgrade protocol, so that the version system of the screen projection equipment is upgraded, and the screen projection equipment is upgraded remotely and online.

In a specific implementation manner, three indicator lights of red, green and blue may be disposed on the screen projection device to inform a user of the current upgrading process and state of the screen projection device, so that the user can judge the upgrading process by observing the colors of the indicator lights of the screen projection device, where the blue light is used to indicate that the screen projection device is about to be upgraded, the red light is used to indicate that the screen projection device is in upgrading, and the green light is used to indicate that the screen projection device is upgraded, and can be normally used after being restarted.

Step S35: and acquiring the recognition result of the audio data.

The recognition result is obtained based on the audio to be recognized optimized in step S34, that is, the audio data mentioned below is the optimized audio data. In the embodiment, the optimized audio data is identified through the cloud equipment so as to obtain the identification result of the audio data. Alternatively, the recognition result may be a recognition text of the audio data, a conversation record, a translation audio of the audio data, and the like, and may be specifically set according to actual use needs, which is not specifically limited herein. The identification text of the audio data is the text format converted from the audio data, and can be a multilingual text; the session record is the conference record of the sub-roles generated based on the audio data, namely the conference summary; the translated audio of the audio data is the audio data in the second language different from the language of the audio data.

As shown in fig. 5, fig. 5 is a flowchart illustrating an embodiment of step S35 shown in fig. 3, where the step S35 includes the following sub-steps, and it should be noted that, if there is substantially the same result, the present embodiment is not limited to the flowchart shown in fig. 5.

Step S351: and sending the audio data to the cloud equipment.

In an embodiment, a network address of the cloud end device is predefined in a customized program run by the source end device, so that after the source end device receives audio data transmitted by the screen projection device and performs optimization such as preprocessing, the network module in the source end device can be used to send the optimized audio data to the cloud end device according to the network address of the cloud end device in the customized program. In other embodiments, the screen projection device may also be provided with a network module, so that the screen projection device can send the audio data to the cloud device by using its own network module according to the network address of the cloud device.

The audio identification model for identifying the audio data is integrated in the cloud device, so that the audio data can be identified after the cloud device receives the optimized audio data. Optionally, the audio recognition model includes, but is not limited to, an intelligent speech recognition model and a translation model, and is specifically configured according to actual use needs, and is not specifically limited herein.

Step S352: and receiving the recognition result of the cloud equipment on the audio data.

In this embodiment, after the cloud device identifies the audio data by using the audio identification model, the identification result is fed back to the customization program of the screen projection device, and the customization program receives the identification result of the cloud device on the audio data, so that the source device obtains the identification result of the audio data.

In addition, in one embodiment, the cloud device stores the recognition result of the audio data, so as to facilitate subsequent access, viewing, editing and the like; in addition, the data can be shared by other users through cloud equipment, and the cooperative office efficiency is improved.

Step S36: and displaying the recognition result on the preset equipment.

In one embodiment, the recognition result includes a recognition text of the audio data, that is, the audio recognition model converts the audio data into a text content. When the custom program receives the recognition result returned by the cloud device, the text content corresponding to the audio data can be displayed on the preset device, so that the subtitle content corresponding to the audio data is synchronously generated on the preset device, that is, the functions of recording and transcribing can be realized by setting the screen projection device and the custom program of the screen projection device, the audio data received in real time can be transcribed into the text content, and the requirements under scenes such as multi-user conferences are met. In other embodiments, the audio recognition model may also perform error correction processing on the recognized text content to improve the accuracy of the recognition result. Optionally, the preset device may be a source device, or may be a display device that is connected to the source device through a screen projection device, and may be specifically set according to actual use needs, and is not specifically limited herein.

The recognition text can be a first text which is directly recognized based on the audio data, and the first text is represented by a source language of the audio data; or the recognition text can also be a second text, the second text is represented in a first target language different from the source language, and the second text and the first text have the same semantic meaning; alternatively, the recognition text may be both the first text and the second text, i.e., the recognition text is both represented in the source language and a language different from the source language. That is, the recognition text may be a text in the source language or a translation text in the target language. Alternatively, the source language may be any language such as chinese, english, french, german, etc., and the first target language may be any language other than the source language. In other embodiments, any one or more languages other than the source language and the first target language may also be displayed at the same time, and the number of displayed languages may be specifically set according to actual use needs, which is not specifically limited herein.

For example, in a chinese-english conference, in order to facilitate efficient and flexible communication between two parties of the conference, subtitle content in chinese and english languages corresponding to audio data is synchronously generated on a preset device, so that the text recognized by the audio recognition model includes a first text expressed in chinese and a second text expressed in english.

In one embodiment, the audio Recognition model that converts the audio data into textual content in the source language may be an intelligent Speech Recognition model (ASR). It is to be understood that, in other embodiments, other network models may also be used to convert the audio data into the text content represented in the source language, and the text content may be specifically set according to actual usage needs, and is not specifically limited herein.

In one embodiment, the audio recognition model for converting the audio data into the translated text content of any target language is a translation model, wherein the translation model is not specifically limited and can be specifically set according to actual use requirements. Specifically, the cloud device identifies and obtains a first text of the audio data in a source language based on the received audio data by using an intelligent voice recognition model, then determines which two languages are inter-translated by obtaining a first target language, and finally obtains a second text represented by the first target language by translating the second text through a translation model, and transmits the second text to a custom program of the source device, so that the source device obtains the second text and displays the second text.

Step S37: a session record is obtained.

In the above embodiment, the screen projection device performs sound source localization on the collected audio data, and the source device further performs sound source localization on the audio data transmitted by the screen projection device, so that a speaking object to which the audio data belongs can be obtained based on the audio data. Therefore, in an embodiment, the recognition result may include not only the recognition text of the audio data but also the utterance object to which the audio data belongs, and since the functions of recording and rewriting may be implemented by setting the screen projection device and the customized program of the screen projection device, an instant conference summary may be generated and output based on the recognition text of the audio data and the utterance object to which the audio data belongs, and the instant conference summary at this time is the acquired session record.

In another embodiment, the audio data after the conference is finished, that is, the conference recording data of the whole conference, can complete accurate transcription of the conference content based on the conference recording data, and can generate the conference record of the sub-roles based on the whole conference based on the identification text of the conference recording data and the speaking objects to which the sounds in the recording data belong, where the conference record of the sub-roles at this time is the acquired session record.

In addition, in one embodiment, the session record or the accurate transcription based on the conference content can be acquired and then uploaded to the cloud device for storage, so as to facilitate subsequent access, viewing, editing and the like; in addition, the data can be shared by other users through cloud equipment, and the cooperative office efficiency is improved.

Step S38: and instructing the playing device to play the interpretation audio.

In one embodiment, the recognition result may include a translation audio of the audio data, the translation audio and the audio data having the same semantic meaning, that is, the audio recognition model of the cloud device may be capable of converting the audio data expressed in the source language into a translation audio expressed in a second target language different from the source language. The audio recognition model for generating the translation audio based on the audio data is a translation model, and the translation model is output in an audio format. As shown in fig. 6, fig. 6 is a schematic diagram of translated audio obtained based on audio data identification provided in the present application. Specifically, the source device receives the audio data, identifies the source language of the audio data through the language identification model, obtains the second target language to determine which two languages are translated with each other, and finally converts the audio data into translation audio expressed in the second target language through the translation model and transmits the translation audio back to the source device.

In the embodiment, after the translation audio processed by the cloud device on the audio data is obtained, the playing device is instructed to play the translation audio so as to realize simultaneous translation, and thus, efficient and flexible cross-language conference communication can be realized. The playing device may be a source device, a screen projection device, or a display device connected to the source device through the screen projection device, and may be specifically set according to actual use needs, and is not specifically limited herein.

For example, taking a screen projection device as a playing device as an example, the screen projection device is integrated with a speaker in addition to a microphone, so that the interpreted audio can be played. The loudspeaker in the screen projection equipment can be a high-fidelity loudspeaker or other types of loudspeakers, the high-fidelity loudspeaker can restore the human voice and the high-pitched part more truly and clearly, so that the screen projection equipment provides high-quality lift service, and the high-fidelity loudspeaker also supports the output of overweight and bass and the like.

It should be noted that, in the embodiment that only the whole course of the conference needs to be recorded in the form of a sound recording, the whole course of the conference may be uploaded to the cloud device after optimization for saving, sharing, editing, or the like, or directly saved locally without performing subsequent steps, that is, without performing steps S35-S38 and steps S351-S352.

In an implementation manner, as shown in fig. 7, fig. 7 is a schematic view of an embodiment of a cloud device provided in the present application. The user can store the data file to the cloud device, download the stored data file from the cloud device, edit the data file in the cloud device, and the like through the customization program running on the source device, wherein when the customization program runs on the source device, the audio data can be transcribed, translated, and the like, so that the data file comprises the audio data file and other data files generated based on the audio data file, and the like.

Specifically, the customization program of the screen projection device running on the source device can upload the audio data acquired by the screen projection device and the recognition result of the audio data to the cloud device, a user can log in a personal account to access the cloud device through the customization program, can download, delete, edit and the like files stored in the cloud device, and can upload other personal files to the cloud device for storage through the customization program of the screen projection device. In addition, some files stored on the cloud equipment can be shared through the custom program, so that other users can obtain the files related to sharing, the file sharing capability is realized, and the business office efficiency is improved.

In order to ensure the security and privacy of the file in the cloud device, in an embodiment, the extraction code and the file code may be further configured to limit the extraction of the conference file and the personal file from the cloud device, that is, the conference file can be extracted from the cloud device when the extraction code is available, and the personal file can be extracted from the cloud device when the file code is available.

In other embodiments, when the source device does not run the customized program, the source device itself may also directly access the cloud device, store the audio data file and/or other data files generated based on the configuration rule of the source device to the cloud device, download the stored data file from the cloud device, edit the data file in the cloud device, and so on.

Referring to fig. 8, fig. 8 is a schematic flowchart illustrating a screen projection method according to an embodiment of the present application. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 8 is not limited in this embodiment. As shown in fig. 8, the present embodiment includes:

step S81: and starting the temporary hotspot based on the preset distribution network information.

In this embodiment, the display device opens the temporary hotspot based on the preset distribution network information, wherein the first screen projection device has the preset distribution network information in advance, so that the first screen projection device can be connected to the temporary hotspot opened by the display device through the preset distribution network information, thereby realizing the screen projection connection between the first screen projection device and the display device. That is to say, first throw the screen equipment and can realize wireless screen at display device and throw the screen function, and this embodiment can support to throw and carry out wireless transmission between the screen equipment promptly, has improved mobility and portability between the equipment.

The preset distribution network information comprises a preset Service Set Identifier (SSID) and a preset password, the preset Service Set Identifier is a string of characters which uniquely name the wireless local area network, the preset Service Set Identifier is Set so that authentication is required when the user enters the temporary hotspot network, namely the user needs to perform the authentication on the preset password, and only the preset password corresponding to the preset Service Set Identifier can pass the authentication, so that unauthorized equipment can be prevented from entering the hotspot network. Since the preset distribution network information which is the same as that of the display device is prestored in the first screen projection device, the first screen projection device can complete authentication through the prestored preset password in the preset distribution network information so as to be connected with the temporary hotspot opened based on the preset distribution network information. It is to be understood that, in other embodiments, the preset distribution network information may also include other information, which may be specifically set according to actual use needs, and is not specifically limited herein.

Optionally, in an embodiment, the preset distribution network information may be a group, and when the preset distribution network information is a group, the display device may only be connected to one first screen projection device in a screen projection manner, that is, the first screen projection device and the display device are connected in a one-to-one manner. In other embodiments, a plurality of groups of different preset distribution network information can be configured at the display device, so that the display device can open a plurality of temporary hotspots based on the plurality of groups of different preset distribution network information, therefore, different preset distribution network information is pre-stored in a plurality of first screen projection devices, the pre-stored preset distribution network information in each first screen projection device corresponds to the preset distribution network information configured on the display device one by one, each first screen projection device can be connected to the temporary hotspots opened based on the corresponding preset distribution network information based on the preset distribution network information, that is, the screen projection of the first screen projection devices is connected to the same display device, the screen projection connection mode in one-to-many or many-to-one scenes is simplified, and the efficiency and flexibility of multi-user conferences are improved. The number of the first screen projection devices which can be connected with the same display device at the same time is not limited, and the first screen projection devices can be specifically arranged according to actual use requirements.

In an embodiment, the display device may open the temporary hotspot by connecting to an external hardware device, so that the first screen projection device is connected to the temporary hotspot, and screen projection connection between the first screen projection device and the display device is realized. In other embodiments, the display device may also start the temporary hotspot through built-in program software, so that the first screen projection device is connected to the temporary hotspot, and screen projection connection between the first screen projection device and the display device is realized.

Optionally, the display device may be a smart television, a mobile phone, a tablet computer, or the like, and may be specifically set according to actual use needs, which is not specifically limited herein.

Step S82: the method comprises the steps of obtaining an equipment identifier of a first screen projection device, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection device.

In the embodiment, after the first screen projection device and the display device realize screen projection connection through the temporary hotspot, the device identifier of the first screen projection device is obtained, the special target distribution network information corresponding to the first screen projection device is generated based on the device identifier of the first screen projection device, and the target distribution network information is sent to the first screen projection device, so that the first screen projection device can be subsequently connected with the display device in a screen projection manner based on the special target distribution network information of the first screen projection device.

In an embodiment, the device identifier of the first screen projection device may be a hardware address (MAC) of the first screen projection device, which is referred to as a MAC address for short, and the MAC address of the first screen projection device is an identifier representing the first screen projection device on the internet and has uniqueness, so that unique and unchangeable target distribution network information corresponding to the first screen projection device, that is, a target service set identifier and a target password corresponding to the target distribution network information can be generated based on the MAC address of the first screen projection device. It is to be understood that, in other embodiments, the target distribution network information corresponding to the first screen projection device may also be generated by using other device identifiers of the first screen projection device, and may be specifically set according to actual use needs, which is not specifically limited herein.

Step S83: and closing the temporary hotspot, and starting a screen projection hotspot special for the first screen projection equipment based on the target distribution network information.

After the display device generates the target distribution network information, the generated target distribution network information is sent to the corresponding first screen projection device, so that the first screen projection device can obtain the special target distribution network information generated based on the device identifier of the first screen projection device. Therefore, after the display device generates a hotspot based on the target distribution network information, the first screen projection device can be connected to the hotspot by using the target distribution network information. In this embodiment, after the display device sends the generated target distribution network information to the corresponding first screen projection device, the temporary hotspot corresponding to the first screen projection device is closed, and the screen projection hotspot dedicated to the first screen projection device is opened based on the target distribution network information.

In one embodiment, as shown in fig. 9, fig. 9 is a schematic view of an example of a first screen projection device and a display device screen projection connection provided in the present application. The display device can be connected with a second screen projecting device, the second screen projecting device is used for being in communication connection with the first screen projecting device, and at the moment, a screen projecting hotspot can be opened by the second screen projecting device connected with the display device, so that the first screen projecting device is connected to the screen projecting hotspot opened by the second screen projecting device, the first screen projecting device is connected with the display device through the screen projecting hotspot, and screen content of a source end device connected with the first screen projecting device can be projected onto the display device. That is, the wireless screen projection function can be realized without installing any other software, and the mobility and the portability between the devices are improved. In other embodiments, when the display device is not connected with the second screen projecting device, a network distribution program may be provided in the display device, a network module of the display device is configured through the network distribution program, the network module is used for being in communication connection with the first screen projecting device, and at this time, the screen projecting hotspot may be opened by the network module of the display device, so that the first screen projecting device is connected to the screen projecting hotspot opened by the network module of the display device, thereby realizing that the first screen projecting device is connected with the display device through the screen projecting hotspot, and further being capable of projecting the screen content of the source device connected with the first screen projecting device onto the display device.

In addition, as shown in fig. 9, in an embodiment, when the same display device is connected to a plurality of first screen projection devices for screen projection at the same time, the screen content of the source device connected to the plurality of first screen projection devices may also be displayed at the same time, that is, multi-user parallel wireless screen projection is supported. In other embodiments, the screen-shot content of each user can be displayed in turn by switching, so that efficient multi-user conference discussion is realized.

In an embodiment, after the source device is successfully connected to the display device through the first screen projection device, the display device receives and outputs screen projection data from the first screen projection device, so that screen content of the source device is successfully projected onto the display device. The screen projection data can include the recognition result of the audio data and image data, the image data is collected by the first screen projection device from a source device connected with the first screen projection device, and the audio data is collected by the first screen projection device.

In a specific embodiment, the recognition result may include a recognition text of the audio data, the recognition text includes at least one of a first text and a second text, the first text is directly recognized based on the audio data, the first text is represented in a source language of the audio data, the second text is represented in a first target language different from the source language, and the second text has the same semantic meaning as the first text. In other embodiments, the recognition result may also include an audio data transliteration audio, the audio data being expressed in a source language, the transliteration audio being expressed in a second target language different from the source language, and the transliteration audio having the same semantic meaning as the audio data.

In an implementation manner, as shown in fig. 7, fig. 7 is a schematic view of an embodiment of a cloud device provided in the present application. When the first screen projection device is connected with the display device in a screen projection mode through built-in program software of the display device, a user can access the cloud device through the built-in program software of the display device to store data files to the cloud device, download the stored data files from the cloud device, edit the data files in the cloud device and the like, wherein the data files comprise audio and video data files and other data files generated based on the audio data files as the display device can receive the audio and video data files from the source device and can transfer and translate the data files based on the audio data.

In other embodiments, when the first screen projection device is connected to the display device through a hardware device externally connected to the display device, the display device itself may also directly access the cloud device, store the audio/video data file and the data file such as transcription and translation based on the audio data in the cloud device, download the stored data file from the cloud device, and edit the data file in the cloud device.

Referring to fig. 10, fig. 10 is a schematic flowchart illustrating a screen projection method according to another embodiment of the present application. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 10 is not limited in this embodiment. As shown in fig. 10, the present embodiment includes:

step S1001: and driving the second screen projection equipment to start the temporary hotspot based on the preset distribution network information.

In this embodiment, as shown in fig. 9, the display device is connected to a second screen projection device, and preset distribution network information is prestored on the second screen projection device, so that the preset distribution network information is written into the second screen projection device to drive the second screen projection device to open a temporary hotspot, and therefore, the first screen projection device prestored with the preset distribution network information can be connected to the temporary hotspot opened by the second screen projection device based on the preset distribution network information, so that screen projection connection between the first screen projection device and the display device is realized.

Step S1002: the method comprises the steps of obtaining an equipment identifier of a first screen projection device, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection device.

Step S1002 is similar to step S82, and will not be described herein.

Step S1003: and driving a second screen projection device to start a screen projection hotspot based on the target distribution network information.

In this embodiment, after the second screen projection device generates the target distribution network information corresponding to the first screen projection device and sends the distribution network information to the corresponding first screen projection device, the first screen projection device disconnects the temporary hotspot, and the second screen projection device closes the temporary hotspot. And writing the target distribution network information into the electronic equipment, and driving the second screen projection equipment to start the screen projection hotspot by the electronic equipment by using the rewritten target distribution network information.

Step S1011: and driving a network module of the display equipment to open the temporary hotspot based on the preset distribution network information.

In this embodiment, the display device runs the distribution network program and is not connected to the second screen projection device, and the distribution network program is pre-stored with the preset distribution network information, so that the preset distribution network information is written into the distribution network program, so that the network module of the display device is driven by the distribution network program to open the temporary hotspot, and therefore, the first screen projection device pre-stored with the preset distribution network information can be connected to the temporary hotspot opened by the network module of the display device based on the preset distribution network information, and screen projection connection between the first screen projection device and the display device is realized.

In a specific embodiment, a user operates a distribution network program, writes a preset service set identifier and a preset password into the display device, and at this time, drives a network module of the display device to start a temporary hotspot by using the written preset service set identifier and the written preset password.

Step S1012: the method comprises the steps of obtaining an equipment identifier of a first screen projection device, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection device.

Step S1012 is similar to step S82 and will not be described herein.

Step S1013: and driving a network module to start a screen projection hotspot based on the target distribution network information.

In this embodiment, after the network distribution program generates the target network distribution information corresponding to the first screen projection device and sends the network distribution information to the corresponding first screen projection device, the first screen projection device disconnects the temporary hotspot, and the network module of the display device closes the temporary hotspot. And writing the target distribution network information into the electronic equipment, and driving the network module to open the screen projection hotspot by the electronic equipment by using the rewritten target distribution network information.

Step S102: and saving the generated target distribution network information as candidate distribution network information.

In this embodiment, after the first screen projection device and the display device complete pairing connection through the target distribution network information, the generated target distribution network information is stored to serve as candidate distribution network information, and then the screen projection hotspot corresponding to the first screen projection device which is expected to be connected with the screen projection device can be directly selected to be started according to a user, without repeating the step of connection pairing again.

In addition, the generated target distribution network information is stored, so that the electronic equipment can automatically start a screen projection hotspot based on the stored target distribution network information after being restarted.

Step S103: and opening a screen projection hotspot corresponding to the at least one candidate distribution network information based on the at least one candidate distribution network information selected by the user.

In this embodiment, the candidate distribution network information selected by the user may be one or more. When the candidate distribution network information selected by the user is one, starting a screen projection hotspot corresponding to the selected candidate distribution network information, so that a first screen projection device corresponding to the selected candidate distribution network information is connected to the screen projection hotspot, and the first screen projection device and the display device are in a one-to-one connection mode, and the display device receives and displays screen projection data of the first screen projection device corresponding to the candidate distribution network information selected by the user; when the candidate distribution network information selected by the user is multiple, a plurality of screen projection hot spots corresponding to the selected candidate distribution network information are started, the first screen projection equipment and the display equipment are in a many-to-one connection mode, the display equipment can receive and display screen projection data of each selected first screen projection equipment one by one through switching, or the display equipment can display the screen projection data of the selected first screen projection equipment in parallel, namely the screen projection data of all the selected first screen projection equipment are displayed on the same picture in the display equipment.

In addition, in order to avoid leakage of the distribution network information, in an embodiment, only the service set identifier of the distribution network information may be displayed at the display device, and the password corresponding to the service set identifier may be hidden.

Referring to fig. 11, fig. 11 is a schematic flowchart illustrating a screen projection method according to another embodiment of the present application. It should be noted that, if the result is substantially the same, the flow sequence shown in fig. 11 is not limited in this embodiment. As shown in fig. 11, in this embodiment, switching the screen-casting right by detecting the screen-casting state of the screen-casting device specifically includes:

step S110: and starting the temporary hotspot based on the preset distribution network information.

Step S110 is similar to step S81 and will not be described herein.

Step S111: the method comprises the steps of obtaining an equipment identifier of a first screen projection device, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection device.

Step S111 is similar to step S82, and will not be described herein.

Step S112: and closing the temporary hotspot, and starting a screen projection hotspot special for the first screen projection equipment based on the target distribution network information.

Step S112 is similar to step S83, and will not be described herein.

Step S113: responding to a screen projection request sent by the screen projection equipment, keeping screen projection connection with the current screen projection equipment, and detecting the screen projection state of the current screen projection equipment.

The first screen projection devices are in communication connection with the display device through respective dedicated screen projection hotspots, the first screen projection devices comprise current screen projection devices and screen projection waiting devices, the current screen projection devices are connected with the display device in a screen projection mode, and the rest screen projection waiting devices are not actually connected with the display device in a screen projection mode. In this embodiment, after receiving a screen projection request sent by a screen projection device, the screen projection state of the current screen projection device is detected first, and the screen projection state of the current screen projection device is not switched immediately.

Optionally, the screen projection state may be a screen projection maintaining state, or may be a screen abdicating state, that is, a screen abdicating right is given. In one embodiment, when a yielding request from the current screen projection equipment is detected, the screen projection state of the current screen projection equipment is determined to be a yielding screen projection state. In other embodiments, when a yielding request from the current screen projection device is not detected, the screen projection state of the current screen projection device is determined to be a screen projection maintaining state.

In a specific embodiment, a key may be disposed on the first screen projection device, a waiting key may be set in advance to indicate a screen projection request when the first screen projection device double-clicks, and a current screen projection device double-clicks indicate a yielding request, so that whether a screen projection request or a yielding request exists may be determined by detecting whether the first screen projection device has a key action. It can be understood that, in other embodiments, a screen-throwing request may be represented by a double-click button of the screen-throwing device, a single-screen button of the current screen-throwing device represents a yielding request, and how to embody the screen-throwing request or the yielding request of the screen-throwing device may be specifically set according to actual use needs, which is not specifically limited herein.

Step S114: and determining whether to switch to screen projection connection with the screen projection equipment waiting for screen projection based on the screen projection state.

In this embodiment, when a yielding request from the screen projection device in the current screen projection state is detected, that is, the screen projection state of the current screen projection device is a yielding screen projection state, it indicates that the current screen projection device may yield the screen projection right, and in order to avoid a screen projection error caused by an erroneous operation, a message to be confirmed is output to prompt a user to confirm whether to yield the screen projection. After the current screen projection equipment confirms, namely after the current screen projection equipment sends a confirmation yielding message, the display equipment is determined to be switched to be connected with the screen projection equipment waiting to be connected, and therefore switching of the screen projection right is achieved.

Referring to fig. 12, fig. 12 is a schematic block diagram illustrating an embodiment of an audio processing apparatus according to the present application. The audio processing device 120 includes: a data acquisition module 121 and a data processing module 122. The data obtaining module 121 is configured to obtain audio data collected by a screen projection device, where the screen projection device is connected to a source end device through a preset interface; the data processing module 122 is configured to process the audio data by using different strategies based on the status detection result of the source device.

Wherein the state detection result includes any one of: whether the source device runs the customized program of the screen projection device currently or not and whether the source device has a configuration rule for the screen projection device or not.

The state detection result comprises whether the source end equipment currently runs a customized program of the screen projection equipment; the data processing module 122 is configured to process the audio data by using different strategies based on the state detection result of the source device, and specifically includes: the detected state detection result comprises that the source end equipment runs a customized program to obtain the identification result of the audio data.

Wherein, the recognition result comprises a recognition text of the audio data; the data processing module 122 is configured to, after obtaining the identification result of the audio data, further include: displaying the recognition result on preset equipment; the preset device comprises at least one of a source device and a display device connected with the source device through a screen projection device in a screen projection mode.

Wherein, the recognition result also comprises a speaking object to which the audio data belongs; the data processing module 122 is configured to, after obtaining the identification result of the audio data, further include: and acquiring the conversation record based on the identification text of the audio data and the speaking object to which the audio data belongs.

The identification text comprises at least one of a first text and a second text; the first text is obtained through direct recognition based on the audio data, the first text is represented by the source language of the audio data, the second text is represented by the first target language different from the source language, and the second text and the first text have the same semantics.

Wherein, the recognition result comprises translation audio of the audio data; the data processing module 122 is configured to, after obtaining the identification result of the audio data, further include: instructing the playing device to play the interpretation audio; the playing device comprises any one of a source end device, a screen projection device and a display device connected with the source end device in a screen projection mode through the screen projection device, the audio data are expressed in a source language, the transliterated audio is expressed in a second target language different from the source language, and the transliterated audio and the audio data have the same semantic meaning.

Wherein, the audio data comprises voice data and loop back data; the data processing module 122 is configured to, before obtaining the recognition result of the audio data, further include: optimizing to obtain audio to be recognized based on the voice data and the loop data; and the recognition result is obtained based on the audio to be recognized.

The data obtaining module 121 is configured to obtain an identification result of the audio data, and specifically includes: sending the audio data to the cloud equipment; the cloud equipment is integrated with an audio recognition model, and the audio recognition model is used for recognizing audio data; and receiving the recognition result of the cloud equipment on the audio data.

The data processing module 122 is configured to process the audio data by using different strategies based on a state detection result of the source device, and specifically further includes: and detecting that the state detection result comprises that the source end equipment does not run the customized program, and storing the audio data.

The data obtaining module 121 is configured to obtain audio data collected by a screen projection device electrically connected to a source device, and specifically includes: and acquiring audio data by adopting different protocols related to the preset interface based on the state detection result.

The state detection result comprises whether the source end equipment currently runs a customized program of the screen projection equipment; the data obtaining module 121 is configured to obtain, based on the state detection result, audio data by using different protocols related to a preset interface, and specifically includes: under the condition that the state detection result comprises that the source end equipment runs the customized program, acquiring audio data by adopting a first protocol; the first protocol is a self-defined protocol of a preset interface; and/or, under the condition that the state detection result includes that the source end device does not run the customized program, acquiring audio data by adopting a second protocol; the second protocol is a standard protocol of a preset interface.

The data processing module 122 is further configured to detect that a state detection result meets a preset condition, and acquire an equipment identifier of the screen projection equipment; acquiring an upgrading firmware suitable for the screen projection equipment based on the equipment identification; and upgrading the screen projection equipment based on the upgrading firmware.

Wherein the data processing module 122 is further configured to at least one of: storing the data file to the cloud device; downloading a data file from the cloud equipment; and editing the data file in the cloud equipment.

Referring to fig. 13, fig. 13 is a schematic frame diagram of a screen projection device according to an embodiment of the present disclosure. The screen projection device 130 includes: a temporary hotspot module 131, a distribution network information module 132, and a screen projection hotspot module 133. The temporary hotspot module 131 is configured to open a temporary hotspot based on preset distribution network information, where the preset distribution network information is prestored in the first screen projection device, and the first screen projection device is connected to the temporary hotspot based on the preset distribution network information; the distribution network information module 132 is configured to obtain an equipment identifier of the first screen projection equipment, generate target distribution network information based on the equipment identifier, and send the target distribution network information to the first screen projection equipment; the screen-casting hotspot module 133 is configured to close the temporary hotspot, and open a screen-casting hotspot dedicated to the first screen-casting device based on the target distribution network information, where the first screen-casting device is connected to the screen-casting hotspot based on the target distribution network information.

The display equipment is connected with second screen projection equipment, and the second screen projection equipment prestores preset distribution network information; the temporary hotspot module 131 is configured to open a temporary hotspot based on preset distribution network information, and specifically includes: driving a second screen projection device to start a temporary hotspot based on preset distribution network information; the screen projection hotspot module 133 is configured to open a screen projection hotspot based on the target distribution network information, and specifically includes: and driving a second screen projection device to start a screen projection hotspot based on the target distribution network information.

The display equipment runs a distribution network program, and the distribution network program prestores preset distribution network information; the temporary hotspot module 131 is configured to open a temporary hotspot based on preset distribution network information, and specifically includes: driving a network module of the display device to open a temporary hotspot based on preset distribution network information; the screen projection hotspot module 133 is configured to open a screen projection hotspot based on the target distribution network information, and specifically includes: and driving a network module to start a screen projection hotspot based on the target distribution network information.

The screen projection device 130 further comprises a data execution module 134, wherein the data execution module 134 is configured to store the generated target distribution network information as candidate distribution network information; and opening a screen projection hotspot corresponding to the at least one candidate distribution network information based on the at least one candidate distribution network information selected by the user.

The first screen projection devices are in communication connection with the display device through respective dedicated screen projection hotspots, the first screen projection devices comprise current screen projection devices and screen projection waiting devices, and the current screen projection devices are in screen projection connection with the display device; the data execution module 134 is further configured to respond to a screen projection request sent by a screen projection device, maintain screen projection connection with a current screen projection device, and detect a screen projection state of the current screen projection device; wherein, the screen projection state comprises: keeping any one of the screen projection state and the screen abdicating projection state; and determining whether to switch to screen projection connection with the screen projection equipment waiting for screen projection based on the screen projection state.

The data execution module 134 is configured to detect a screen projection state of a current screen projection device, and specifically includes: detecting a yielding request from current screen projection equipment, and determining that a screen projection state is a yielding screen projection state; or, the screen projection state is determined to be the screen projection maintaining state without detecting a yielding request from the current screen projection equipment.

The data execution module 134 is configured to determine whether to switch to a screen projection connection with a screen projection waiting device based on a screen projection state, and specifically includes: outputting a message to be confirmed based on the screen throwing state as a transition screen throwing state; the message to be confirmed is used for prompting whether to confirm the screen of giving way; and responding to a confirmation yielding message sent by the current screen projection equipment, and determining to switch to screen projection connection with the screen projection equipment waiting for screen projection.

The screen projecting device 130 further comprises a data receiving module 135, wherein the data receiving module 135 is used for receiving and outputting screen projecting data from the first screen projecting device; the screen projection data comprise the identification result of the audio data and image data, the image data are collected by the first screen projection device from a source device connected with the first screen projection device, and the audio data are collected by the first screen projection device.

The recognition result comprises a recognition text of the audio data, the recognition text comprises at least one of a first text and a second text, the first text is directly recognized based on the audio data, the first text is represented by a source language of the audio data, the second text is represented by a first target language different from the source language, and the second text and the first text have the same semantic meaning; and/or the recognition result comprises transliterated audio of the audio data, the audio data is expressed in a source language, the transliterated audio is expressed in a second target language different from the source language, and the transliterated audio and the audio data have the same semantic meaning.

The screen projecting device 130 further includes a data processing module 136, and the data processing module 136 is further configured to at least one of: storing the data file to the cloud device; downloading a data file from the cloud equipment; and editing the data file in the cloud equipment.

Referring to fig. 14, fig. 14 is a schematic diagram of a frame of an embodiment of an electronic device provided in the present application. The electronic device 140 comprises a memory 141 and a processor 142 coupled to each other, wherein the memory 141 stores program instructions, and the processor 142 is configured to execute the program instructions to implement the steps in any of the audio processing method or the screen projection method embodiments described above. Specifically, the electronic device 140 may include, but is not limited to: desktop computers, notebook computers, servers, mobile phones, tablet computers, and the like, without limitation.

In particular, the processor 142 is configured to control itself and the memory 141 to implement the steps in any of the audio processing method or the screen projection method embodiments described above. Processor 142 may also be referred to as a CPU (Central Processing Unit). The processor 142 may be an integrated circuit chip having signal processing capabilities. The Processor 142 may also be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 142 may be commonly implemented by integrated circuit chips.

Referring to fig. 15, fig. 15 is a block diagram illustrating an embodiment of a computer-readable storage medium according to the present application. The computer-readable storage medium 150 stores program instructions 151 capable of being executed by a processor, the program instructions 151 being for implementing the steps in any of the audio processing method or screen projection method embodiments described above.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, and for brevity, will not be described again herein.

In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some interfaces, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

Claims

1. A screen projection method, comprising:

starting a temporary hotspot based on preset distribution network information; the first screen projection equipment is prestored with the preset distribution network information and is connected to the temporary hotspot based on the preset distribution network information;

acquiring an equipment identifier of the first screen projection equipment, generating target distribution network information based on the equipment identifier, and sending the target distribution network information to the first screen projection equipment;

closing the temporary hotspot, and opening a screen projection hotspot special for the first screen projection device based on the target distribution network information; the first screen projection equipment is connected to the screen projection hotspot based on the target distribution network information.

2. The method of claim 1, wherein the first screen projection device is connected to a display device through the screen projection hotspot, the display device is connected to a second screen projection device, and the second screen projection device is used for being in communication connection with the first screen projection device to realize the screen projection connection;

or the first screen projection device is connected with the display device in a screen projection mode through the screen projection hotspot, the display device runs a network distribution program, the network distribution program is used for configuring a network module of the display device, and the network module is in communication connection with the first screen projection device to achieve the screen projection connection.

3. The method according to claim 2, wherein the display device is connected to the second screen projection device, and the second screen projection device prestores the preset distribution network information; based on preset distribution network information, the temporary hotspot is opened, and the method comprises the following steps:

driving the second screen projection equipment to start the temporary hotspot based on the preset distribution network information;

the starting of the screen projection hotspot based on the target distribution network information comprises the following steps:

and driving the second screen projection equipment to start the screen projection hotspot based on the target distribution network information.

4. The method of claim 2, wherein the display device runs the distribution network program, and the distribution network program prestores the preset distribution network information; based on preset distribution network information, the temporary hotspot is opened, and the method comprises the following steps:

driving a network module of the display device to open the temporary hotspot based on the preset distribution network information;

and driving the network module to start the screen projection hotspot based on the target distribution network information.

5. The method of claim 1, further comprising:

saving the generated target distribution network information as candidate distribution network information;

and opening a screen projection hotspot corresponding to at least one candidate distribution network information based on the at least one candidate distribution network information selected by the user.

6. The method of claim 1, wherein a plurality of first screen projection devices are in communication connection with a display device through respective dedicated screen projection hotspots, the plurality of first screen projection devices include a current screen projection device and a waiting screen projection device, and the current screen projection device is in screen projection connection with the display device; the method further comprises the following steps:

responding to a screen projection request sent by the screen projection waiting equipment, keeping screen projection connection with the current screen projection equipment, and detecting the screen projection state of the current screen projection equipment; wherein the screen projection state comprises: keeping any one of the screen projection state and the screen abdicating projection state;

and determining whether to switch to screen projection connection with the screen projection waiting equipment or not based on the screen projection state.

7. The method of claim 6, wherein the detecting the screen-projecting state of the current screen-projecting device comprises:

detecting a yielding request from the current screen projection equipment, and determining that the screen projection state is the yielding screen projection state;

or, the screen projection state is determined to be the screen projection maintaining state without detecting a yielding request from the current screen projection equipment.

8. The method of claim 6, wherein the determining whether to switch to the screen-projection connection with the screen-projection waiting device based on the screen-projection state comprises:

outputting a message to be confirmed based on the screen throwing state as the screen giving state; the message to be confirmed is used for prompting whether to confirm the screen of giving way or not;

and responding to a confirmation yielding message sent by the current screen projection equipment, and determining to switch to the screen projection connection with the screen projection equipment waiting for screen projection.

9. The method of claim 1, further comprising:

receiving and outputting screen projection data from the first screen projection equipment;

the screen projection data comprise an identification result of audio data and image data, the image data is collected by the first screen projection device from a source device connected with the first screen projection device, and the audio data is collected by the first screen projection device.

10. The method of claim 9, wherein the recognition result comprises a recognition text of the audio data, the recognition text comprises at least one of a first text and a second text, the first text is directly recognized based on the audio data, the first text is represented in a source language of the audio data, the second text is represented in a first target language different from the source language, and the second text has the same semantic meaning as the first text;

and/or the recognition result comprises transliterated audio of the audio data, the audio data is expressed in a source language, the transliterated audio is expressed in a second target language different from the source language, and the transliterated audio and the audio data have the same semantic meaning.

11. The method of claim 1, further comprising at least one of:

storing the data file to the cloud device;

downloading the data file from the cloud device;

and editing the data file in the cloud equipment.

12. A screen projection apparatus, comprising:

the temporary hotspot module is used for starting a temporary hotspot based on preset distribution network information; the first screen projection equipment is prestored with the preset distribution network information and is connected to the temporary hotspot based on the preset distribution network information;

the distribution network information module is used for acquiring the equipment identifier of the first screen projection equipment, generating target distribution network information based on the equipment identifier and sending the target distribution network information to the first screen projection equipment;

the screen projection hotspot module is used for closing the temporary hotspot and opening a screen projection hotspot special for the first screen projection equipment based on the target distribution network information; the first screen projection equipment is connected to the screen projection hotspot based on the target distribution network information.

13. An electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is configured to execute the program instructions to implement the screen projection method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that program instructions executable by a processor for implementing the screen projection method of any one of claims 1 to 11 are stored.