CN114006971B - Screen-throwing window audio control method, equipment and computer readable storage medium - Google Patents
Screen-throwing window audio control method, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114006971B CN114006971B CN202111263096.7A CN202111263096A CN114006971B CN 114006971 B CN114006971 B CN 114006971B CN 202111263096 A CN202111263096 A CN 202111263096A CN 114006971 B CN114006971 B CN 114006971B
- Authority
- CN
- China
- Prior art keywords
- audio
- track
- screen
- game
- throwing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 115
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 19
- 238000012952 Resampling Methods 0.000 claims abstract description 17
- 230000006854 communication Effects 0.000 claims description 20
- 238000004891 communication Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 14
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000002955 isolation Methods 0.000 abstract description 26
- 230000000694 effects Effects 0.000 abstract description 11
- 230000006870 function Effects 0.000 description 16
- 238000010295 mobile communication Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000005266 casting Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/162—Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72427—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for supporting games or graphical animations
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Telephone Function (AREA)
Abstract
The invention discloses a method, equipment and a computer-readable storage medium for controlling audio of a projection window, wherein the method comprises the following steps: intercepting audio data belonging to a screen-throwing window process in the audio writing process of the thrown end; detecting a track type of the audio data; if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing; if the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Description
Technical Field
The present invention relates to the field of mobile communications, and in particular, to a method and apparatus for controlling audio of a screen window, and a computer readable storage medium.
Background
In the prior art, with the continuous development of intelligent terminal equipment, screen projection application based on the intelligent terminal equipment is more and more popular with users. Taking a mobile phone as an example, in the existing small window screen-throwing application process, the screen of the mobile phone cannot be influenced by the small window screen-throwing in the screen-throwing process, namely, perfect isolation of the screen-throwing screen and the screen used by the mobile phone is completely realized.
However, even if perfect isolation of the screen-casting screen and the mobile phone itself using screen is achieved, the two sources of sound are mixed, i.e. the sound heard by the screen casting is found to be a mixture of all the sounds of the whole system when the screen is cast. Specifically, when a user plays a game while playing music on the mobile phone, at this time, the same sound, i.e. the mixed sound of the music and the game, is heard by both the screen and the mobile phone, and is not isolated, so that the two sounds cannot be isolated from each other.
Therefore, there is a need for further sound isolation based on-screen isolation during the screen-on mode.
Disclosure of Invention
In order to solve the technical defects in the prior art, the invention provides a screen-projection window audio control method, which comprises the following steps:
And intercepting the audio data belonging to the screen-throwing window process in the audio writing process of the thrown end.
The track type of the audio data is detected.
And if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing.
If the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing.
Optionally, intercepting the audio data belonging to the projection window process in the audio writing process of the cast end includes:
and when the screen-throwing end and the screen-throwing end are in screen-throwing communication connection, monitoring a screen-throwing window started by the screen-throwing end.
And acquiring an application program in the screen-throwing window, and recording a process corresponding to the application program as the screen-throwing window process.
Optionally, the intercepting the audio data belonging to the process of the screen-throwing window in the audio writing process of the thrown end further includes:
when the screen-throwing end is in screen-throwing communication connection with the screen-throwing end, monitoring audio write-in data of the screen-throwing end.
When the audio data of the process of the screen throwing window exists in the audio writing data, the audio data is intercepted.
Optionally, the detecting the audio track type of the audio data includes:
and acquiring program information of the application program and current state information of the cast terminal.
Presetting the type of the audio track to be detected according to the program information and/or the state information.
Optionally, if the audio data is a single track type of a game track or a microphone track, transmitting the audio data to a projection end corresponding to the cast end for playing, including:
if the audio data is a single track type of the game track or the microphone track, detecting whether the game track belongs to the same frequency and the same sound track number or detecting whether the microphone track belongs to the same frequency and the same sound track number.
If the game sound tracks belong to the same frequency and the same sound channel number, or the microphone sound tracks belong to the same frequency and the same sound channel number, the audio data are transmitted to a screen throwing end corresponding to the thrown end for playing.
Optionally, if the audio data is a single track type of a game track or a microphone track, transmitting the audio data to a projection end corresponding to the cast end for playing, including:
If the game sound track does not belong to the same frequency or the same sound track number, or the microphone sound track does not belong to the same frequency or the same sound track number, synthesizing the game sound track or the microphone sound track according to a preset fixed frequency and sound track.
And transmitting the processed audio data to a screen throwing end corresponding to the thrown end for playing.
Optionally, if the audio data is a multiple track type of the game track and the microphone track, synthesizing and resampling the game track and the microphone track, and transmitting the processed audio data to the screen-throwing end for playing, including:
if the audio data is multiple track types of the game track and the microphone track, detecting the track types of the game track and the microphone track.
The game track is determined to include game background sounds and game body sounds that do not belong to the same frequency or the same number of channels, or the microphone track is determined to include teammate voices and own voices that do not belong to the same frequency or the same number of channels.
Optionally, if the audio data is a multiple track type of the game track and the microphone track, synthesizing and resampling the game track and the microphone track, and transmitting the processed audio data to the screen-throwing end for playing, and further including:
And presetting audio processing parameters according to the program information and/or the state information.
And synthesizing and resampling the game background sound, the game main body sound, the teammate voice and the self voice according to the audio processing parameters, and transmitting the processed audio data to the screen throwing end for playing.
The invention also provides a screen window audio control device which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the computer program realizes the steps of the screen window audio control method according to any one of the above when being executed by the processor.
The invention also provides a computer readable storage medium, on which a screen window audio control program is stored, which when executed by a processor implements the steps of the screen window audio control method as described in any one of the above.
According to the method, the device and the computer readable storage medium for controlling the audio of the screen throwing window, the audio data belonging to the screen throwing window process are intercepted in the audio writing process of the thrown end; detecting a track type of the audio data; if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing; if the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a schematic diagram of a hardware structure of a mobile terminal according to the present invention;
fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention;
FIG. 3 is a flow chart of a first embodiment of the screen window audio control method of the present invention;
FIG. 4 is a flow chart of a second embodiment of the present invention of a method for controlling audio in a projection window;
FIG. 5 is a flow chart of a third embodiment of the screen window audio control method of the present invention;
FIG. 6 is a flow chart of a fourth embodiment of the screen window audio control method of the present invention;
FIG. 7 is a flowchart of a fifth embodiment of the method for controlling audio of a projection window according to the present invention;
FIG. 8 is a flowchart of a sixth embodiment of a method for controlling audio of a projection window according to the present invention;
FIG. 9 is a flowchart of a seventh embodiment of a method for controlling audio of a projection window according to the present invention;
fig. 10 is a flowchart of an eighth embodiment of the screen window audio control method of the present invention.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present invention, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
The terminal may be implemented in various forms. For example, the terminals described in the present invention may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs, desktop computers, and the like.
The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present invention can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose.
Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal implementing various embodiments of the present invention, the mobile terminal 100 may include: an RF (Radio Frequency) unit 101, a WiFi module 102, an audio output unit 103, an a/V (audio/video) input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, a processor 110, and a power supply 111. Those skilled in the art will appreciate that the mobile terminal structure shown in fig. 1 is not limiting of the mobile terminal and that the mobile terminal may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
The following describes the components of the mobile terminal in detail with reference to fig. 1:
the radio frequency unit 101 may be used for receiving and transmitting signals during the information receiving or communication process, specifically, after receiving downlink information of the base station, processing the downlink information by the processor 110; and, the uplink data is transmitted to the base station. Typically, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication, global System for Mobile communications), GPRS (General Packet Radio Service ), CDMA2000 (Code Division Multiple Access, CDMA 2000), WCDMA (Wideband Code Division Multiple Access ), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access, time Division synchronous code Division multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, frequency Division Duplex Long term evolution), and TDD-LTE (Time Division Duplexing-Long Term Evolution, time Division Duplex Long term evolution), etc.
WiFi belongs to a short-distance wireless transmission technology, and a mobile terminal can help a user to send and receive e-mails, browse web pages, access streaming media and the like through the WiFi module 102, so that wireless broadband Internet access is provided for the user. Although fig. 1 shows a WiFi module 102, it is understood that it does not belong to the necessary constitution of a mobile terminal, and can be omitted entirely as required within a range that does not change the essence of the invention.
The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output (e.g., a call signal reception sound, a message reception sound, etc.) related to a specific function performed by the mobile terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.
The a/V input unit 104 is used to receive an audio or video signal. The a/V input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, the graphics processor 1041 processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound into audio data. The processed audio (voice) data may be converted into a format output that can be transmitted to the mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting the audio signal.
The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and the proximity sensor can turn off the display panel 1061 and/or the backlight when the mobile terminal 100 moves to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; as for other sensors such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured in the mobile phone, the detailed description thereof will be omitted.
The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 107 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile terminal. In particular, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 1071 or thereabout by using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a predetermined program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends the touch point coordinates to the processor 110, and can receive and execute commands sent from the processor 110. Further, the touch panel 1071 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. The user input unit 107 may include other input devices 1072 in addition to the touch panel 1071. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc., as specifically not limited herein.
Further, the touch panel 1071 may overlay the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or thereabout, the touch panel 1071 is transferred to the processor 110 to determine the type of touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of touch event. Although in fig. 1, the touch panel 1071 and the display panel 1061 are two independent components for implementing the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 may be integrated with the display panel 1061 to implement the input and output functions of the mobile terminal, which is not limited herein.
The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100. For example, the external devices may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.
Memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area that may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and a storage data area; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 109 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by running or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.
The mobile terminal 100 may further include a power source 111 (e.g., a battery) for supplying power to the respective components, and preferably, the power source 111 may be logically connected to the processor 110 through a power management system, so as to perform functions of managing charging, discharging, and power consumption management through the power management system.
Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described herein.
In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based will be described below.
Referring to fig. 2, fig. 2 is a schematic diagram of a communication network system according to an embodiment of the present invention, where the communication network system is an LTE system of a general mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an e-UTRAN (Evolved UMTS Terrestrial Radio Access Network ) 202, an epc (Evolved Packet Core, evolved packet core) 203, and an IP service 204 of an operator that are sequentially connected in communication.
Specifically, the UE201 may be the terminal 100 described above, and will not be described herein.
The E-UTRAN202 includes eNodeB2021 and other eNodeB2022, etc. The eNodeB2021 may be connected with other eNodeB2022 by a backhaul (e.g., an X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide access from the UE201 to the EPC 203.
EPC203 may include MME (Mobility Management Entity ) 2031, hss (Home Subscriber Server, home subscriber server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, pgw (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function, policy and tariff function entity) 2036, and so on. The MME2031 is a control node that handles signaling between the UE201 and EPC203, providing bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location registers (not shown) and to hold user specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034 and PGW2035 may provide IP address allocation and other functions for UE201, PCRF2036 is a policy and charging control policy decision point for traffic data flows and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).
IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem ), or other IP services, etc.
Although the LTE system is described above as an example, it should be understood by those skilled in the art that the present invention is not limited to LTE systems, but may be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.
Based on the above mobile terminal hardware structure and the communication network system, various embodiments of the method of the present invention are provided.
Example 1
Fig. 3 is a flowchart of a first embodiment of the screen-drop window audio control method of the present invention. A method for controlling audio of a projection window, the method comprising:
s1, intercepting audio data belonging to a screen projection window process in the audio writing process of the cast end.
S2, detecting the type of the audio track of the audio data.
And S3, if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing.
And S4, if the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing.
Alternatively, a mobile phone running an android operating system is described as an example. Specifically, first, the implementation idea of the present embodiment is to strip the sound of the screen-throwing small window, that is, output the sound of the screen-throwing small window to the currently established screen-throwing channel separately, without outputting the sound to the external channel of the current mobile phone.
In this embodiment, after determining the implementation thought, detecting a specific application of the screen-throwing widget in real time, that is, identifying a process corresponding to the specific application; the implementation manner is that in the audio tracking (AudioTrack) of the system, the process identification of the specific application is performed by a method (write) of writing audio.
In this embodiment, considering that all sounds to be played by the specific application of the screen are only performed by the method (write) of writing audio through the above-mentioned audio track (AudioTrack), the specific application of the screen is used as the basis of the method (write) interception of writing audio.
In this embodiment, if it is detected that the process of the specific application of screen projection is a small window process, the audio data transmitted by the process is intercepted. Thus, the audio data incoming by the above process is not transferred to the subsequent audio synthesis service (audioplayer). It will be appreciated that if the audio data transmitted by the above process is transferred to a subsequent audio synthesis service (audioplayer), the audio data transmitted by the process will be output to the external of the mobile phone itself, so that audio interference will be caused.
In this embodiment, the audio data transferred by the process is transferred to the established screen-throwing channel as an audio buffer, and specifically, the data transfer can be performed in a socket mode, so that the audio data transferred by the process can reach a screen-throwing end which establishes screen-throwing connection with the end to be thrown.
In the above embodiment, the interception of the sound of the small window is implemented and sent to the screen throwing end, and it can be understood that if the small window application has only one audio track (AudioTrack), that is, one audio track (AudioTrack) is a fixed frequency and channel, at this time, the direct data is transferred to the screen throwing computer end, so that the audio playing can be implemented. However, considering more specific applications of screen projection, there is not only one audio track, for example, a game program related to a flat elite, etc., but generally one game background audio track and one microphone speaking audio track, and the two audio tracks use completely different frequencies and channel numbers. Therefore, in this embodiment, the receiving end of the audio track is distinguished, that is, the audio track is distinguished for sending and receiving, then the received data is subjected to audio track synthesis and resampling, and finally the processed data is transferred to the screen-throwing end for playing.
It can be seen that in this embodiment, a track-tracking (AudioTrack) related process is skillfully used to distinguish the application sources of the sound, and then, in the synthesis stage of the mobile phone audio, the screen-throwing application sound and the non-screen-throwing application sound are separated, synthesized and output separately, so that an audio isolation effect unified with the picture isolation is achieved in the screen-throwing mode.
The method has the advantages that the audio data belonging to the screen-throwing window process is intercepted in the audio writing process of the thrown end; detecting a track type of the audio data; if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing; if the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example two
Fig. 4 is a flowchart of a second embodiment of the audio control method for a screen-projection window according to the present invention, based on the above embodiment, the intercepting audio data belonging to the screen-projection window process during the audio writing process of the cast end includes:
and S11, monitoring a screen throwing window started by the thrown end when the thrown end and the screen throwing end are in screen throwing communication connection.
S12, acquiring an application program in the screen-throwing window, and recording a process corresponding to the application program as the screen-throwing window process.
Optionally, in this embodiment, an application program in the screen-throwing window is acquired, and when it is determined that the screen-throwing window has the use authority of the microphone and the loudspeaker at the same time, a process corresponding to the application program is recorded as the screen-throwing window process.
The method has the beneficial effects that when the screen-throwing end and the screen-throwing end are in screen-throwing communication connection, the screen-throwing window started by the screen-throwing end is monitored; and acquiring an application program in the screen-throwing window, and recording a process corresponding to the application program as the screen-throwing window process. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example III
Fig. 5 is a flowchart of a third embodiment of the audio control method for a screen-projection window according to the present invention, based on the above embodiment, the method for intercepting audio data belonging to a screen-projection window process during an audio writing process of a cast end further includes:
and S13, monitoring audio write-in data of the thrown end when the thrown end and the thrown end are in communication connection.
S14, when the audio data of the screen throwing window process exists in the audio writing data, the audio data are intercepted.
Optionally, in this embodiment, the audio writing data of the thrown end includes program audio writing of a main window of the thrown end itself and/or program audio writing of a small window above the main window.
Optionally, in this embodiment, when the widget is not located above the main window, audio data of the screen-drop window process existing in the audio write data is directly disabled.
The method has the beneficial effects that when the screen-throwing communication connection is established between the thrown end and the screen-throwing end, audio write-in data of the thrown end are monitored; when the audio data of the process of the screen throwing window exists in the audio writing data, the audio data is intercepted. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example IV
Fig. 6 is a flowchart of a fourth embodiment of the audio control method for a projection window according to the present invention, based on the above embodiment, the detecting the track type of the audio data includes:
s21, acquiring program information of the application program and current state information of the cast terminal.
S22, presetting the type of the audio track to be detected according to the program information and/or the state information.
Optionally, in this embodiment, program information of the application program and current state information of the cast end are acquired, where the program information includes authority information of the program, for example, includes speaker authority and microphone authority at the same time, and further includes configuration information of the program, for example, includes a game voice setting module and a game background sound setting module at the same time, that is, it may be determined that the application may output audio data of a multiple track type according to the program information.
Optionally, in this embodiment, the state information includes an operation state of the main window, that is, it is determined whether the main window has a call requirement of a microphone or a speaker at the same time, so as to avoid audio confusion that may occur later.
The method and the device have the advantages that the track type to be detected is preset according to the program information and/or the state information by acquiring the program information of the application program and the current state information of the cast end. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example five
Fig. 7 is a flowchart of a fifth embodiment of a screen-projection window audio control method according to the present invention, based on the above embodiment, if the audio data is a single track type of a game track or a microphone track, the audio data is transferred to a screen-projection end corresponding to the screen-projection end for playing, including:
s31, if the audio data is a single track type of a game track or a microphone track, detecting whether the game track belongs to the same frequency and the same sound channel number, or detecting whether the microphone track belongs to the same frequency and the same sound channel number.
And S32, if the game sound tracks belong to the same frequency and the same sound channel number, or the microphone sound tracks belong to the same frequency and the same sound channel number, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing.
Optionally, in this embodiment, if it is detected that the main window has a call requirement of a microphone or a speaker at the same time, when the audio data is a game audio track or a single audio track type of the microphone audio track, it is further detected whether the game audio track belongs to the same frequency and the same number of channels, or whether the microphone audio track belongs to the same frequency and the same number of channels.
Optionally, in this embodiment, if only a game soundtrack exists and the game soundtrack belongs to the same frequency and the same number of channels, or only a microphone soundtrack exists and the microphone soundtrack belongs to the same frequency and the same number of channels, the audio data is transmitted to a projection terminal corresponding to the to-be-cast terminal for playing.
The method has the advantages that whether the game sound tracks belong to the same frequency and the same sound channel number or whether the microphone sound tracks belong to the same frequency and the same sound channel number is detected by detecting whether the audio data are single sound track types of the game sound tracks or the microphone sound tracks; if the game sound tracks belong to the same frequency and the same sound channel number, or the microphone sound tracks belong to the same frequency and the same sound channel number, the audio data are transmitted to a screen throwing end corresponding to the thrown end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example six
Fig. 8 is a flowchart of a sixth embodiment of a screen-projection window audio control method according to the present invention, based on the above embodiment, if the audio data is a single track type of a game track or a microphone track, the audio data is transferred to a screen-projection end corresponding to the dropped end for playing, and further includes:
And S33, if the game sound tracks do not belong to the same frequency or the same sound channel number, or the microphone sound tracks do not belong to the same frequency and the same sound channel number, synthesizing the game sound tracks or the microphone sound tracks according to the preset fixed frequency and the preset sound channel.
And S34, transmitting the processed audio data to a screen projection end corresponding to the projected end for playing.
Optionally, in this embodiment, if the game audio track and the microphone audio track exist simultaneously, and the game audio track does not belong to the same frequency or the same number of channels, and the microphone audio track does not belong to the same frequency or the same number of channels, the game audio track or the microphone audio track is synthesized according to a preset fixed frequency and channel.
The method has the advantages that when the game sound tracks do not belong to the same frequency or the same sound channel number, or the microphone sound tracks do not belong to the same frequency and the same sound channel number, the game sound tracks or the microphone sound tracks are synthesized according to preset fixed frequencies and sound channels; and transmitting the processed audio data to a screen throwing end corresponding to the thrown end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example seven
Fig. 9 is a flowchart of a seventh embodiment of a screen-drop window audio control method according to the present invention, based on the above embodiment, if the audio data is multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen-drop terminal for playing, where the method includes:
s41, if the audio data are multiple audio track types of the game audio track and the microphone audio track, detecting the audio track types of the game audio track and the microphone audio track.
S42, determining that the game sound track comprises game background sounds and game main sounds which do not belong to the same frequency or the same channel number, or determining that the microphone sound track comprises teammate voices and self voices which do not belong to the same frequency or the same channel number.
Optionally, in this embodiment, during the same time, it is determined that the game audio track includes a game background sound and a game main sound that do not belong to the same frequency or the same channel number, or the microphone audio track includes a teammate voice and a self voice that do not belong to the same frequency or the same channel number, so that when the audio trigger times are different, reception and transmission can be performed respectively, and corresponding audio data transmission is performed.
The method has the advantages that the audio track types of the game audio track and the microphone audio track are detected when the audio data are multiple audio track types of the game audio track and the microphone audio track; the game track is determined to include game background sounds and game body sounds that do not belong to the same frequency or the same number of channels, or the microphone track is determined to include teammate voices and own voices that do not belong to the same frequency or the same number of channels. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example eight
Fig. 10 is a flowchart of an eighth embodiment of a screen-drop window audio control method according to the present invention, based on the above embodiment, if the audio data is multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen-drop terminal for playing, further including:
s43, presetting audio processing parameters according to the program information and/or the state information.
S44, synthesizing and resampling the game background sound, the game main body sound, the teammate voice and the self voice according to the audio processing parameters, and transmitting the processed audio data to the screen throwing end for playing.
Optionally, in this embodiment, audio processing parameters are preset according to the program information and/or the state information, where the audio processing parameters include parameters for synthesizing and resampling the game background sound, the game main sound, the teammate voice, and the self voice, so as to meet multiple screen-throwing requirements of different screen-throwing applications on self data bandwidth, transmission delay, and the like.
The beneficial effects of the embodiment are that the audio processing parameters are preset through the program information and/or the state information; and synthesizing and resampling the game background sound, the game main body sound, the teammate voice and the self voice according to the audio processing parameters, and transmitting the processed audio data to the screen throwing end for playing. The humanized screen window audio control scheme is realized, so that an audio isolation effect unified with the picture isolation can be obtained in the screen mode, and the screen use experience of a user is enhanced.
Example nine
Based on the above embodiment, the present invention also proposes a device for controlling audio of a screen window, the device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program implementing the steps of the method for controlling audio of a screen window as defined in any one of the above when being executed by the processor.
It should be noted that the above device embodiments and method embodiments belong to the same concept, the specific implementation process of the device embodiments is detailed in the method embodiments, and technical features in the method embodiments are correspondingly applicable to the device embodiments, which are not repeated herein.
Examples ten
Based on the above embodiment, the present invention further provides a computer readable storage medium, where a screen window audio control program is stored, where the screen window audio control program, when executed by a processor, implements the steps of the screen window audio control method according to any one of the above embodiments.
It should be noted that the medium embodiment and the method embodiment belong to the same concept, the specific implementation process of the medium embodiment and the method embodiment are detailed, and technical features in the method embodiment are correspondingly applicable in the medium embodiment, which is not repeated herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiments of the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those having ordinary skill in the art without departing from the spirit of the present invention and the scope of the claims, which are to be protected by the present invention.
Claims (3)
1. A method for controlling audio of a projection window, the method comprising:
intercepting audio data belonging to a screen-throwing window process in the audio writing process of the thrown end;
detecting a track type of the audio data;
if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing;
if the audio data are multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing;
the intercepting the audio data belonging to the screen-throwing window process in the audio writing process of the thrown end comprises the following steps:
When the screen-throwing end and the screen-throwing end are in screen-throwing communication connection, monitoring a screen-throwing window started by the screen-throwing end;
acquiring an application program in the screen-throwing window, and recording a process corresponding to the application program as the screen-throwing window process;
when the screen-throwing end and the screen-throwing end are in screen-throwing communication connection, monitoring audio write-in data of the screen-throwing end;
when the audio data of the screen throwing window process exists in the audio writing data, the audio data is intercepted;
the detecting the audio track type of the audio data includes:
acquiring program information of the application program and current state information of the thrown terminal;
presetting the type of the audio track to be detected according to the program information and/or the state information;
if the audio data is a single audio track type of a game audio track or a microphone audio track, transmitting the audio data to a projection end corresponding to the projected end for playing, including:
if the audio data is a single track type of a game track or a microphone track, detecting whether the game track belongs to the same frequency and the same sound channel number or detecting whether the microphone track belongs to the same frequency and the same sound channel number;
If the game sound tracks belong to the same frequency and the same sound channel number, or the microphone sound tracks belong to the same frequency and the same sound channel number, transmitting the audio data to a screen throwing end corresponding to the thrown end for playing;
if the game sound track does not belong to the same frequency or the same sound track number, or the microphone sound track does not belong to the same frequency or the same sound track number, synthesizing the game sound track or the microphone sound track according to a preset fixed frequency and sound track;
transmitting the processed audio data to a screen throwing end corresponding to the thrown end for playing;
if the audio data is multiple audio track types of the game audio track and the microphone audio track, synthesizing and resampling the game audio track and the microphone audio track, and transmitting the processed audio data to the screen throwing end for playing, wherein the method comprises the following steps:
if the audio data is multiple track types of the game track and the microphone track, detecting the track types of the game track and the microphone track;
determining that the game soundtrack comprises game background sounds and game main sounds which do not belong to the same frequency or the same sound track number, or the microphone soundtrack comprises teammate voices and self voices which do not belong to the same frequency or the same sound track number;
Presetting audio processing parameters according to the program information and/or the state information;
and synthesizing and resampling the game background sound, the game main body sound, the teammate voice and the self voice according to the audio processing parameters, and transmitting the processed audio data to the screen throwing end for playing.
2. A drop window audio control device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the drop window audio control method of claim 1.
3. A computer readable storage medium, wherein a drop window audio control program is stored on the computer readable storage medium, which when executed by a processor, implements the steps of the drop window audio control method of claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111263096.7A CN114006971B (en) | 2021-10-28 | 2021-10-28 | Screen-throwing window audio control method, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111263096.7A CN114006971B (en) | 2021-10-28 | 2021-10-28 | Screen-throwing window audio control method, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114006971A CN114006971A (en) | 2022-02-01 |
| CN114006971B true CN114006971B (en) | 2024-03-19 |
Family
ID=79924542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111263096.7A Active CN114006971B (en) | 2021-10-28 | 2021-10-28 | Screen-throwing window audio control method, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114006971B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107682541A (en) * | 2017-09-27 | 2018-02-09 | 努比亚技术有限公司 | Audio control method, mobile terminal and the storage medium of screen prjection |
| CN108958684A (en) * | 2018-06-22 | 2018-12-07 | 维沃移动通信有限公司 | Throw screen method and mobile terminal |
| CN109889885A (en) * | 2019-02-27 | 2019-06-14 | 努比亚技术有限公司 | A kind of throwing screen control method, terminal and computer readable storage medium |
| CN110083328A (en) * | 2019-03-29 | 2019-08-02 | 北京奇艺世纪科技有限公司 | A kind of audio frequency playing method, device, electronic equipment and storage medium |
| WO2020063675A1 (en) * | 2018-09-27 | 2020-04-02 | 深圳市冠旭电子股份有限公司 | Smart loudspeaker box and method for using smart loudspeaker box |
| CN111984215A (en) * | 2020-08-19 | 2020-11-24 | Oppo广东移动通信有限公司 | Multimedia data playing method, terminal and storage medium |
| CN112423211A (en) * | 2020-10-26 | 2021-02-26 | 努比亚技术有限公司 | Multi-audio transmission control method, equipment and computer readable storage medium |
| CN113050916A (en) * | 2021-04-09 | 2021-06-29 | 深圳Tcl新技术有限公司 | Audio playing method, device and storage medium |
| CN113176869A (en) * | 2021-05-28 | 2021-07-27 | 努比亚技术有限公司 | Screen-projecting audio time delay control method and device and computer readable storage medium |
-
2021
- 2021-10-28 CN CN202111263096.7A patent/CN114006971B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107682541A (en) * | 2017-09-27 | 2018-02-09 | 努比亚技术有限公司 | Audio control method, mobile terminal and the storage medium of screen prjection |
| CN108958684A (en) * | 2018-06-22 | 2018-12-07 | 维沃移动通信有限公司 | Throw screen method and mobile terminal |
| WO2020063675A1 (en) * | 2018-09-27 | 2020-04-02 | 深圳市冠旭电子股份有限公司 | Smart loudspeaker box and method for using smart loudspeaker box |
| CN109889885A (en) * | 2019-02-27 | 2019-06-14 | 努比亚技术有限公司 | A kind of throwing screen control method, terminal and computer readable storage medium |
| CN110083328A (en) * | 2019-03-29 | 2019-08-02 | 北京奇艺世纪科技有限公司 | A kind of audio frequency playing method, device, electronic equipment and storage medium |
| CN111984215A (en) * | 2020-08-19 | 2020-11-24 | Oppo广东移动通信有限公司 | Multimedia data playing method, terminal and storage medium |
| CN112423211A (en) * | 2020-10-26 | 2021-02-26 | 努比亚技术有限公司 | Multi-audio transmission control method, equipment and computer readable storage medium |
| CN113050916A (en) * | 2021-04-09 | 2021-06-29 | 深圳Tcl新技术有限公司 | Audio playing method, device and storage medium |
| CN113176869A (en) * | 2021-05-28 | 2021-07-27 | 努比亚技术有限公司 | Screen-projecting audio time delay control method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114006971A (en) | 2022-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107613489B (en) | Wireless charging method, wireless hotspot device, terminal and storage medium | |
| CN112394896B (en) | Reverse screen-throwing frame rate control method, device and computer readable storage medium | |
| CN112346685B (en) | Screen-off and screen-projection control method and device and computer readable storage medium | |
| CN108683806B (en) | Anti-disturbing method, mobile terminal and computer readable storage medium | |
| CN107807767B (en) | Communication service processing method, terminal and computer readable storage medium | |
| CN112689191B (en) | Screen projection control method, terminal and computer readable storage medium | |
| CN109195213B (en) | Mobile terminal screen control method, mobile terminal and computer readable storage medium | |
| CN109088997B (en) | Game audio control method, terminal and computer readable storage medium | |
| CN112423076B (en) | Audio screen-throwing synchronous control method, equipment and computer readable storage medium | |
| CN108958936B (en) | Application program switching method, mobile terminal and computer readable storage medium | |
| CN109711830B (en) | Quick display method and device for two-dimension code payment, mobile terminal and storage medium | |
| CN112437472B (en) | Network switching method, equipment and computer readable storage medium | |
| CN112492450B (en) | Sound parameter regulation and control method, device and computer readable storage medium | |
| CN107678622B (en) | Application icon display method, terminal and storage medium | |
| CN113176869A (en) | Screen-projecting audio time delay control method and device and computer readable storage medium | |
| CN111970738B (en) | Network switching control method, device and computer readable storage medium | |
| CN112423211A (en) | Multi-audio transmission control method, equipment and computer readable storage medium | |
| CN109462829B (en) | Call transfer method, device and computer readable storage medium | |
| CN109561221B (en) | Call control method, device and computer readable storage medium | |
| CN109684020B (en) | Theme switching method, device and computer readable storage medium | |
| CN109151201B (en) | Anti-addiction method, mobile terminal and computer storage medium | |
| CN107682551B (en) | Control method, terminal and storage medium of intelligent power amplifier | |
| CN108076230B (en) | Call volume adjusting method, mobile terminal and computer readable storage medium | |
| CN112887776B (en) | Method, equipment and computer readable storage medium for reducing audio delay | |
| CN112947831B (en) | Screen-throwing reverse control method, mobile terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |