CN107682752A - Method, apparatus, system, terminal device and the storage medium that video pictures are shown - Google Patents
Method, apparatus, system, terminal device and the storage medium that video pictures are shown Download PDFInfo
- Publication number
- CN107682752A CN107682752A CN201710949829.XA CN201710949829A CN107682752A CN 107682752 A CN107682752 A CN 107682752A CN 201710949829 A CN201710949829 A CN 201710949829A CN 107682752 A CN107682752 A CN 107682752A
- Authority
- CN
- China
- Prior art keywords
- user
- data
- video
- user terminal
- change notification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
- Telephonic Communication Services (AREA)
Abstract
The invention discloses a kind of method that video pictures are shown, comprise the following steps：Receive the SOT state of termination change notification from server；User list is updated according to the SOT state of termination change notification of reception, the user list includes the user data made a speech；The corresponding interface of user data display in the user list.Method, apparatus, system, terminal device and the storage medium shown the invention also discloses a kind of video pictures, the flexibility of long-distance video picture switching can be improved, the process of Switch Video picture is simplified, the operation of user can be efficiently reduced.
Description
Technical field
The present invention relates to the instant video communications field, more particularly to a kind of video pictures show method, apparatus, system,
Terminal device and storage medium.
Background technology
In existing instant messaging application, video communication is a very extensive application.In the prior art, long-range
When Video chat or meeting, user is by terminal device and one or more other users Video chat, in chat process
In, more people's video windows or a certain personal video preview of full screen display can only be shown on the screen of terminal device simultaneously, when
When needing Switch Video picture, user needs to show the video window of corresponding people by switching push button.
If the screen of terminal device is not big enough, while shows that more people's video councils cause each window too small, it is unfavorable for seeing
Understand the details in video.If a certain personal video preview of full screen display, when needing to carry out picture switching, user
Need to carry out selection switching manually, cause complex operation, allow user to use inconvenience.
The content of the invention
In view of the above-mentioned problems, the method, apparatus shown it is an object of the invention to provide a kind of video pictures, system, end
End equipment and storage medium, the flexibility of long-distance video picture switching can be improved, simplify the operation of user.
In order to solve the above technical problems, the invention provides a kind of method that video pictures are shown, comprise the following steps：
Receive the SOT state of termination change notification from server；
User list is updated according to the SOT state of termination change notification of reception, the user list includes what is made a speech
User data；
The corresponding interface of user data display in the user list.
Preferably, the corresponding interface of user data display in the user list, is specifically included：
When the number of the user data in the user list is more than 1, the user list is shown in current interface；
When the number of the user data in the user list is equal to 1, according to from server and the user data
The first treated video data of corresponding compression, generates small window video preview in current interface.
Further, when the user data in the user list number be equal to 1 when, according to from server with
Video pictures corresponding to the user data, after small window video preview is generated in current interface, in addition to：
When according to from server user terminal change notification update the user list when, detect with currently it is small
User data corresponding to window video preview does not stop making a speech and detecting that the number of the user data in the user list is big
In 1, then the user list comprising newly-increased user data is shown in current interface.
Preferably, when detecting the event for clicking on the small window video preview, to the server request with it is described small
The second video data of processing is not compressed corresponding to window video preview；
The second video data returned from server according to request is received, and is generated entirely according to second video data
Shield display picture.
Present invention also offers a kind of method that video pictures are shown, comprise the following steps：
Receive the multi-medium data from each user terminal uploads；Wherein, the multi-medium data includes voice data；
Extract the audio frequency characteristics of the voice data of each user terminal uploads；
The floor status of each user terminal is determined according to the audio frequency characteristics, and according to the speech shape of each user terminal
State generates SOT state of termination change notification；
The SOT state of termination change notification is sent to each user terminal so that each user terminal according to
The SOT state of termination change notification renewal user list received, and the user data display in the user list is corresponding
Interface.
Preferably, the audio frequency characteristics comprise at least one of：Decibel value, tone color, tone, vocal print.
Preferably, the audio frequency characteristics are decibel value, then described to determine each user terminal according to the audio frequency characteristics
Floor status, and SOT state of termination change notification is generated according to the floor status of each user terminal, specifically include：
When the decibel value higher than specific threshold and duration be more than it is most short speak duration when, mark this user terminal
Floor status is to make a speech；
When the decibel value is less than specific threshold and the duration is more than maximum silence duration, this user terminal is marked
Floor status is made a speech for stopping；
SOT state of termination change notification is generated according to the floor status of each user terminal.
Further, effective audio number is selected from the voice data according to the floor status of each user terminal
According to, and the effectively voice data is sent to each user terminal.
Preferably, the multi-medium data also includes the video data of each user terminal uploads, then the switching is long-range
The method of video pictures also includes：
, will be corresponding with the user terminal when the quantity for detecting that floor status is the user terminal made a speech is 1
Video data is compressed processing, and the first video data that the compression is handled to obtain is sent to each user's end
End；
Sent and the user data pair in the user list of the user terminal from one of user terminal when receiving
When that answers is not compressed the request of the second video data of processing, second video data is sent to the user terminal.
Present invention also offers the device that a kind of video pictures are shown, it is characterised in that including：
State receiving module, for receiving the SOT state of termination change notification from server；
Update module, for updating user list, the user list according to the SOT state of termination change notification of reception
Including the user data made a speech；
Display control module, for the corresponding interface of user data display in the user list.
Present invention also offers the device that a kind of video pictures are shown, it is characterised in that including：
Data reception module, for receiving the multi-medium data from each user terminal uploads；Wherein, the multimedia
Data include voice data；
Extraction module, the audio frequency characteristics of the voice data for extracting each user terminal uploads；
Determining module, for determining the floor status of each user terminal according to the audio frequency characteristics, and according to each use
The floor status generation SOT state of termination change notification of family terminal；
Sending module, for the SOT state of termination change notification to be sent into each user terminal, so that described each
Individual user terminal updates user list, and the use in the user list according to the SOT state of termination change notification of reception
The corresponding interface of user data display.
Present invention also offers the system that a kind of video pictures are shown, it is characterised in that including at least two user terminals
And at least one server, wherein：
The user terminal, for the multi-medium data of collection to be sent into server；The multi-medium data includes sound
Frequency evidence；
The server, for receiving the multi-medium data from each user terminal uploads, extract more matchmakers
The audio frequency characteristics of voice data in volume data, the floor status of each user terminal, and root are determined according to the audio frequency characteristics
After the floor status generation SOT state of termination change notification of each user, the SOT state of termination change notification is sent to institute
State each user terminal；
The user terminal, it is additionally operable to receive the SOT state of termination change notification from the server, according to the terminal
Status change notification updates user list, and the corresponding interface of user data display in the user list；Wherein, it is described
User list includes the user data made a speech.
Present invention also offers a kind of terminal device, including processor, memory and be stored in the memory and
It is configured as, by the computer program of the computing device, realizing as described above during computer program described in the computing device
The method that video pictures described in middle any one are shown.
Present invention also offers a kind of computer-readable recording medium, the computer-readable recording medium includes storage
Computer program, wherein, equipment where controlling the computer-readable recording medium when the computer program is run performs
The method that video pictures as described in above-mentioned any one are shown.
The present invention provides method, apparatus, system, terminal device and the storage medium that a kind of video pictures are shown, user is whole
End according to the SOT state of termination change notification renewal user list from server, user data in the user list come
The corresponding interface of display, can improve the flexibility of long-distance video picture switching, efficiently reduce the operation of user, take full advantage of
The space of screen.
Brief description of the drawings
In order to illustrate more clearly of technical scheme, the required accompanying drawing used in embodiment will be made below
Simply introduce, it should be apparent that, drawings in the following description are only some embodiments of the present invention, general for this area
For logical technical staff, on the premise of not paying creative work, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is the schematic flow sheet for the method that the video pictures that first embodiment of the invention provides are shown.
Fig. 2 is a kind of schematic diagram of the display interface for the method that the video pictures that first embodiment of the invention provides are shown.
Fig. 3 is the signal of another display interface for the method that the video pictures that first embodiment of the invention provides are shown
Figure.
Fig. 4 is that the display result flow for the method that the video pictures that second embodiment of the invention provides are shown is intended to.
Fig. 5 is a kind of schematic diagram of the display interface for the method that the video pictures that second embodiment of the invention provides are shown.
Fig. 6 is the schematic flow sheet for the method that the video pictures that fourth embodiment of the invention provides are shown.
Fig. 7 is the structural representation for the device that the video pictures that seventh embodiment of the invention provides are shown.
Fig. 8 is the structural representation for the device that the video pictures that eighth embodiment of the invention provides are shown.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
Referring to Fig. 1, first embodiment of the invention provides a kind of method that video pictures are shown, can be in terminal device
Upper execution, and comprise the following steps：
S11, receive the SOT state of termination change notification from server.
In embodiments of the present invention, the terminal device can be smart mobile phone, tablet personal computer, personal computer, more matchmakers
Body player etc. has the electric terminal of interactive screen.As an example, during Video chat or video conference, Mei Gesuo
State terminal device upload itself collection voice data and video data to server, the server set according in each terminal
The audio frequency characteristics (such as decibel value) extracted in standby voice data, and generate terminal shape according to the audio frequency characteristics of each terminal device
State change notification and each terminal device is sent to, each terminal device receives the terminal from the server
Status change notification.
The SOT state of termination change notification refers to that the floor status of one or more terminal is to make a speech, or, institute
State SOT state of termination change notification and refer to that the floor status of one or more terminal is made a speech for stopping.
S12, user list is updated according to the SOT state of termination change notification of reception, the user list includes sending out
The user data of speech.
In embodiments of the present invention, each one user list of the terminal device self maintained, when server detects
To send notification to terminal device when making a speech, the terminal device receives the floor status of one or more terminal devices
After notice, user data is updated in the user list of itself, the user data that described will making a speech is added to the use
In the list of family.Certainly, when the floor status that the SOT state of termination change notification received is one or more terminal devices is stopping
During speech, the user data for stopping speech being deleted from the user list.
S13, the corresponding interface of user data display in the user list.
Specifically, when the number of the user data in the user list is more than 1, directly displayed in current interface described
User list.
When the number of the user data in the user list is equal to 1, according to from server and the user data
The first treated video data of corresponding compression, generates small window video preview in current interface.
As an example it is supposed that the user for currently having multiple terminal devices makes a speech at the same time, i.e., the user list of each terminal
In all contain multiple user data, it is to be understood that the video pictures of users that now all will need not making a speech
Shown on the screen of terminal device, need to only show that the user comprising the multiple user data arranges in current interface
Table, as shown in Figure 2.
As an example it is supposed that the user of an only terminal device (hereinafter referred to as first terminal equipment) exists in a period of time
During speech, i.e., the user data of first terminal equipment is only included in the user list on each terminal device, now server
The video data that first terminal equipment uploads is compressed processing, reduces the bit rate and resolution ratio of the video data, and
The first video data that the compression handles to obtain is sent to each terminal device, each terminal device is by first video
Video pictures corresponding to data generate small window video preview and shown in the current interface of each terminal device, such as Fig. 3 institutes
Show.In such case, it will be understood that if user corresponding to the video pictures being currently displayed in full screen is state be it is described just
In the user of speech, then any operation need not be carried out, it is not necessary to small window video preview corresponding to generation.
Furthermore, it is necessary to illustrate, in the embodiment of the present invention, the current interface of the terminal device can have following three kinds
Displaying scheme：
The video pictures of host are displayed in full screen by the 1st, system default selection in the current interface, for example,
In meeting, generally, comparatively host is the core of meeting, and therefore, in conference process, the terminal of host is set
The video data of the host of reception is sent to respectively by the standby video data for uploading host to server, the server
Individual terminal device, now each terminal device according to the video data of the host of reception by corresponding video pictures
It is displayed in full screen.
2nd, system, which can be given tacit consent to, shows main screen background in current interface, certainly, user can also self-defined current interface
The background picture of upper display.
3rd, video pictures corresponding with the small window video preview can also worked as prezone by system according to the wish of user
It is displayed in full screen on face.For example, when user clicks on the small window video preview, the user terminal is asked to server and institute
State video data corresponding to small window video preview and receive the video data, then by video pictures corresponding to the video data
It is displayed in full screen.
In summary, the invention provides a kind of method that video pictures are shown, user terminal is according to from server
SOT state of termination change notification updates the user list of itself and corresponding interface is shown according to the user list, can improve
The flexibility of long-distance video picture switching, efficiently reduces the operation of user.Small window video preview is generated in current interface, is led to
The mode of too small window video preview is realized to show the video pictures of speech user and automatically switches and show video window, fully
It make use of the space of screen.
Second embodiment of the invention：
On the basis of one embodiment, when the user data in the user list number be equal to 1 when, according to
From server video pictures corresponding with the user data, after small window video preview is generated in current interface, also wrap
Include：
When according to from server user terminal change notification update the user list when, detect with currently it is small
User data corresponding to window video preview does not stop making a speech and detecting that the number of the user data in the user list is big
In 1, then the user list comprising newly-increased user data is shown in current interface.
In embodiments of the present invention, as an example, referring to Fig. 4, assuming in Video chat, only first terminal equipment
User making a speech, then have small window video preprocessor corresponding with first terminal equipment on the current screen interface of each terminal device
Look at, now, in addition to first terminal equipment, the server detects the user of other one or more terminal devices
Also begin to make a speech, and send new SOT state of termination change notification to each terminal device, each terminal device according to
The new SOT state of termination change notification received updates the user list of itself, i.e., the user list of described each terminal device
The user data of other terminal devices for starting speech has all been newly increased, if the user of first terminal equipment also is continuing to make a speech,
The user data of first terminal equipment is also shown on i.e. described user list, then is shown below the first small window video preview
Second user list comprising newly-increased user data, as shown in Figure 5.
It is appreciated that when the user of first terminal equipment stops speech, will user corresponding with first terminal equipment
Data are deleted from the user list of each terminal, and close the first small window video preview.
Can be the name list of the user to make a speech on the user list or state is the user that is making a speech
The device name list of terminal.
Third embodiment of the invention：
On the basis of one embodiment, in addition to：
When detecting the event for clicking on the small window video preview, to server request and the small window video preprocessor
The second video data of processing is not compressed corresponding to looking at；
The second video data returned from server according to request is received, and is generated entirely according to second video data
Shield display picture.
In embodiments of the present invention, as an example, when second terminal equipment (can be first terminal equipment or
Other terminal devices) user pass through external input equipment (mouse or keyboard etc.) or touch-screen etc. and click on the small window video
Preview is at the region on present terminal device screen, second terminal device-to-server transmission pair and the small window video preprocessor
The request of complete video data corresponding to looking at, now the server will be on terminal device corresponding with the small window video preview
The second video data passed handles ground without any compression, is intactly sent to the second terminal equipment.Described second eventually
After end equipment receives the second video data, video pictures corresponding to the second video data are displayed in full screen on screen.
It should be noted that when the first user corresponding to the video pictures being displayed in full screen on the terminal device screen stops
During speech, the video pictures of the terminal device full screen display are still video pictures corresponding to first user, will not be certainly
Video pictures in dynamic switching current interface, for example, the video pictures of host will not be switched back into.
By the above-mentioned means, by way of small window video preview, the video fermentation of speech user is first shown, then by user
Decide whether video pictures being displayed in full screen according to the wish of oneself, it is ensured that the accuracy of video pictures switching,
Realize the technique effect of interaction close friend.
Fourth embodiment of the invention：
Referring to Fig. 6, fourth embodiment of the invention additionally provides a kind of method that video pictures are shown, can be in server
Upper execution, and comprise the following steps：
S21, gather the voice data of itself and video data from each user terminal uploads.
In embodiments of the present invention, each user terminal can upload voice data by wired or wireless mode and regard
Frequency evidence.As an example, in long-distance video, each user terminal for participating in long-distance video is carried or outside by terminal device
The sound of the microphone collection user connect, the video of camera collection user carried by terminal device or external are drawn
Face, and the sound (voice data) of user and video pictures (video data) are uploaded onto the server in real time, the server
The voice data and video data can be stored or handled.
S22, extract the audio frequency characteristics of the voice data of each user terminal uploads.
In embodiments of the present invention, the audio frequency characteristics comprise at least one of：Decibel value, tone color, tone, sound
Line.
As an example it is supposed that the user of one of terminal device makes a speech, then the terminal device sends the user
Sound (hereinafter referred to as the first voice data) to the server, the server is to first voice data that collects
Detected, extract the information volume in first voice data, and the information volume is changed into audio frequency characteristics (such as
Decibel value).
S23, the floor status of each user terminal is determined according to the audio frequency characteristics, and according to the hair of each user terminal
Speech state generates SOT state of termination change notification.
In embodiments of the present invention, as an example, the audio frequency characteristics are decibel value, specifically,
When the decibel value higher than specific threshold and duration be more than it is most short speak duration when, mark this user terminal
Floor status is to make a speech；
When the decibel value is less than specific threshold and the duration is more than maximum silence duration, this user terminal is marked
Floor status is made a speech for stopping；
SOT state of termination change notification is generated according to the floor status of each user terminal.
In embodiments of the present invention, specific threshold, most short duration of speaking are preset in memory cell in the server in advance
And maximum silence duration, the server by decibel value corresponding to each terminal device extracted and the specific threshold,
It is described it is most short speak duration and the maximum silence duration is compared, then judge the state change of each terminal device,
SOT state of termination change notification can be generated according to the state change of each terminal device.
As an example it is supposed that the user of third terminal equipment loquiturs and uploads the voice data of itself in real time described in
Server, if the server detects very big (the decibel value height of wave volume in the voice data that third terminal equipment uploads
In specific threshold) and the speech duration of the user exceed most short duration of speaking, then mark the state of third terminal equipment as just
Making a speech, and the notice that the user for generating third terminal equipment is making a speech；Certainly, it is assumed that the user of the 4th terminal device is saying
It is complete if oneself after pipe down, now, the server detects not having in the voice data that the 4th terminal device uploads
There are sound or wave volume very little (decibel value is less than specific threshold) and continue for some time and (be more than maximum silence duration),
Then the state of the terminal device of mark the 4th is stops making a speech, and the user for generating the 4th terminal device has stopped the logical of speech
Know.
It should be noted that if audio frequency characteristics are tone color, tone, vocal print, as an example, different voice datas has not
With tone color, the server can according to different tone colors come judge sound that people sends or can the object of sounding send
Sound.Certainly, the server can preset a specific frequency, and the server will extract from voice data
The frequency of tone is contrasted the user to determine making a speech with the specific frequency.Certainly, the server can be with root
The user to make a speech is determined according to the matching degree of vocal print.
S24, the SOT state of termination change notification is sent to each user terminal, so that each user terminal
User list is updated according to the SOT state of termination change notification of reception, and the user data in the user list is shown
Corresponding interface.
As an example it is supposed that first terminal status change notification is made a speech for the user of third terminal equipment, the clothes
The first terminal status change notification is sent to each terminal device by business device by wired or wireless mode, described each
After terminal receives the first terminal status change notification, increase the user of third terminal equipment on the user list of itself
Data；Certainly, it is assumed that second terminal status change notification stops speech for the user of the 4th terminal device, and the server is by the
Two SOT state of termination change notifications are sent to each terminal device, and each terminal device is receiving the second terminal state
After change notification, the user data of the 4th terminal device is deleted on the user list of itself.
Fifth embodiment of the invention：
In fourth embodiment, in general, the voice data that each terminal device gathers can be sent to each by server
Terminal device, terminal device playing audio data, but in some cases, the voice data of some terminal devices collection
It is probably invalid voice data (such as noise), if terminal device plays these voice datas, user can be influenceed
Audio experience.
Therefore, in the present embodiment, in addition to：
Effective voice data is selected from the voice data according to the floor status of each user terminal, and by described in
Effective voice data is sent to each user terminal.
The server can judge whether the voice data video is rubbish sound according to the audio frequency characteristics extracted
Frequency evidence, and rubbish voice data can be shielded or deleted.For example, it is assumed that the server detects the of extraction
When the decibel value of two voice datas exceedes default noise criteria, the second audio data is designated as rubbish voice data,
And the rubbish voice data is shielded, the quality of Video chat or video conference can be improved.The server will can remove
Other voice datas outside rubbish voice data are labeled as effective voice data.
As an example, the effectively voice data can refer to voice data corresponding with all users to speak.
The server ought to mark according to the extraction audio frequency characteristics (such as decibel value) from the voice data of each user terminal uploads
Preceding floor status is the user terminal made a speech, and voice data corresponding to the user terminal made a speech described in mark is
Effective voice data, the effective video data are sent to each terminal device, each terminal device having reception
Effect voice data plays out.For example, when the floor status of the server-tag third terminal equipment is makes a speech, will
Voice data corresponding with third terminal equipment is labeled as first effective voice data, and described first effective voice data is sent out
Deliver to each terminal device.Each terminal device plays out described first effective voice data of reception, that is, plays
The sound of third terminal equipment user.
Sixth embodiment of the invention：
On the basis of the 4th embodiment, in addition to：
, will be corresponding with the user terminal when the quantity for detecting that floor status is the user terminal made a speech is 1
Video data is compressed processing, and the first video data that the compression is handled to obtain is sent to each user's end
End；
Sent and the user data pair in the user list of the user terminal from one of user terminal when receiving
When that answers is not compressed the request of the second video data of processing, second video data is sent to the user terminal.
In embodiments of the present invention, the video pictures shown on small window video preview need not be very high resolution ratio, this
Sample can be effectively saved Internet resources and system resource.Video data can be compressed by server by video encoder
Processing.
As an example it is supposed that currently the user of only first terminal equipment speaks, the server is whole by first
The complete video data that end equipment uploads are compressed by video encoder, and the first video data that compression is handled to obtain
Each terminal device is sent to, and the video pictures of the first video data are generated the first small window video preprocessor by each terminal device
Look at.If now, the user of second terminal equipment clicks the first small window video preview, i.e. second terminal equipment is to the server
The complete video data of first terminal equipment are asked, after the server receives request, complete by first terminal equipment regards
According to second terminal equipment is sent to, the complete video data of reception are then displayed in full screen frequency by second terminal equipment.
Referring to Fig. 7, seventh embodiment of the invention additionally provides the device that a kind of video pictures are shown, including：
State receiving module 11, for receiving the SOT state of termination change notification from server；
Update module 12, for updating user list according to the SOT state of termination change notification of reception, the user list shows
Show the user data made a speech；
Display control module 13, corresponding interface is shown for the user data in the user list.
Preferably, the display control module 13 specifically includes：
First display control unit, for when the number of the user data in the user list is more than 1, working as prezone
Face shows the user list；
Second display control unit, for when the user data in the user list number be equal to 1 when, according to from
The corresponding with the user data of server compresses the first treated video data, and small window video preprocessor is generated in current interface
Look at.
Preferably, the display control module 13 is additionally operable to：
When according to from server user terminal change notification update the user list when, detect with currently it is small
User data corresponding to window video preview does not stop making a speech and detecting that the number of the user data in the user list is big
In 1, then the user list comprising newly-increased user data is shown in current interface.
Preferably, the display control module 13 is additionally operable to：
When detecting the event for clicking on the small window video preview, to server request and the small window video preprocessor
The second video data of processing is not compressed corresponding to looking at；
The second video data returned from server according to request is received, and is generated entirely according to second video data
Shield display picture.
Referring to Fig. 8, eighth embodiment of the invention additionally provides the device that a kind of video pictures are shown, including：
Data reception module 21, for receiving the multi-medium data from each user terminal uploads；Wherein, more matchmakers
Volume data includes voice data；
Extraction module 22, the audio frequency characteristics of the voice data for extracting each user terminal uploads；
Determining module 23, for determining the floor status of each user terminal according to the audio frequency characteristics, and according to each
The floor status generation SOT state of termination change notification of user terminal；
Sending module 24, for the SOT state of termination change notification to be sent into each user terminal, so that described
Each user terminal updates user list according to the SOT state of termination change notification of reception, and according in the user list
The corresponding interface of user data display.
Preferably, the audio frequency characteristics are decibel value, then the determining module 23 specifically includes：
First indexing unit, for being more than most short duration of speaking higher than specific threshold and duration when the decibel value
When, the floor status for marking this user terminal is to make a speech；
Second indexing unit, for being more than maximum silence duration less than specific threshold and duration when the decibel value
When, mark the floor status of this user terminal to be made a speech for stopping；
Notification unit, for generating SOT state of termination change notification according to the floor status of each user terminal.
Preferably, the determining module 23 is additionally operable to：
Effective voice data is obtained according to the floor status of each user terminal, and the effectively voice data is sent to
Each user terminal.
Preferably, the multi-medium data also includes the video data of each user terminal uploads, then the sending module
24 are additionally operable to：
, will video corresponding with the user terminal when the quantity that the state that detects is the user terminal made a speech is 1
Data are compressed processing, and the first treated video data of the compression is sent into each user terminal；
Sent and the user data pair in the user list of the user terminal from one of user terminal when receiving
When that answers is not compressed the request of the second video data of processing, second video data is sent to the user terminal.
Preferably, the audio frequency characteristics comprise at least one of：Decibel value, tone color, tone, vocal print.
Ninth embodiment of the invention additionally provides the system that a kind of video pictures are shown, including at least two user terminals and
At least one server, wherein：
The user terminal, for the multi-medium data of collection to be sent into server；The multi-medium data includes sound
Frequency evidence；
The server, for receiving the multi-medium data from each user terminal uploads, extract more matchmakers
The audio frequency characteristics of voice data in volume data, the floor status of each user terminal, and root are determined according to the audio frequency characteristics
After the floor status generation SOT state of termination change notification of each user, the SOT state of termination change notification is sent to institute
State each user terminal；
The user terminal, it is additionally operable to receive the SOT state of termination change notification from the server, according to the terminal
Status change notification updates user list, and the corresponding interface of user data display in the user list；Wherein, it is described
User list includes the user data made a speech.
Preferably, the audio frequency characteristics are decibel value, then the server, are additionally operable to when the decibel value is higher than certain threshold
Value and duration be more than it is most short speak duration when, mark the floor status of this user terminal to make a speech；When the decibel
When value is more than maximum silence duration less than specific threshold and duration, the floor status of this user terminal is marked to stop hair
Speech；SOT state of termination change notification is generated according to the floor status of each user terminal.
Preferably, the multi-medium data also includes the video data of each user terminal uploads, then the server, also
, will video corresponding with the user terminal for when it is that the quantity of user terminal made a speech is 1 to detect floor status
Data are compressed processing, and the first video data that the compression is handled to obtain is sent to each user terminal；
The user terminal, it is additionally operable to when the number of the user data in the user list is more than 1, in current interface
Show the user list；When the number of the user data in the user list is equal to 1, according to from server and this
The first treated video data of compression, generates small window video preview in current interface corresponding to user data.
Preferably, the user terminal, it is additionally operable to when according to the user terminal change notification renewal institute from server
When stating user list, detect that user data corresponding with currently small window video preview does not stop making a speech and detecting the use
The number of user data in the list of family is more than 1, then carries out the user list comprising newly-increased user data in current interface
Display.
Preferably, the user terminal, it is additionally operable to when detecting the event for clicking on the small window video preview, to described
Server asks the second video data for not being compressed processing corresponding with the small window video preview；
The server, it is additionally operable to work as the user's row received from the transmission of one of user terminal with the user terminal
When not being compressed the request of the second video data of processing corresponding to the user data in table, by second video data transmitting
It is sent to the user terminal；
The user terminal, it is additionally operable to receive the second video data returned according to request from server, and according to institute
State the second video data generation full screen display picture.
Preferably, the server, it is additionally operable to be selected from the voice data according to the floor status of each user terminal
Effective voice data is taken out, and the effectively voice data is sent to each user terminal.
Preferably, the audio frequency characteristics comprise at least one of：Decibel value, tone color, tone, vocal print.
Tenth embodiment of the invention additionally provides the terminal device that a kind of video pictures are shown.The video pictures of the embodiment
The terminal device of display includes：Processor, display, memory and it is stored in the survival wherein and can be in the processing
The computer program run on device, such as the program that video pictures are shown.It is real during computer program described in the computing device
Step in the embodiment of existing above-mentioned each video picture displaying method, such as the step S11 shown in Fig. 1.Or the processing
Device realizes the function of each unit in above-mentioned each device embodiment, such as the monitoring list shown in Fig. 7 when performing the computer program
Member 11.
Exemplary, the computer program can be divided into one or more modules, one or more of moulds
Block is stored in the memory, and by the computing device, to complete the present invention.One or more of modules can be with
It is the series of computation machine programmed instruction section that can complete specific function, the instruction segment is used to describe the computer program in institute
State the implementation procedure in the terminal device that video pictures are shown.
The terminal device that the video pictures are shown can be desktop PC, notebook, palm PC and high in the clouds clothes
The computing devices such as business device.The terminal device that the video pictures are shown may include, but be not limited only to, processor, memory, display
Device.It will be understood by those skilled in the art that the example for the terminal device that the schematic diagram, which is only video pictures, to be shown, not structure
The restriction for the terminal device that paired video pictures are shown, parts more more or less than diagram can be included, or combined some
Part, or different parts, such as the terminal device that the video pictures are shown can also include input-output equipment, network
Access device, bus etc..
Alleged processor can be CPU (Central Processing Unit, CPU), can also be it
His general processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other PLDs, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor can also be any conventional processor
Deng the processor is the control centre for the terminal device that the video pictures are shown, whole using various interfaces and connection
The various pieces for the terminal device that the individual video pictures are shown.
The memory can be used for storing the computer program and/or module, and the processor is by running or performing
The computer program and/or module being stored in the memory, and the data being stored in memory are called, described in realization
The various functions for the terminal device that video pictures are shown.The memory can mainly include storing program area and storage data field,
Wherein, storing program area can storage program area, application program (such as sound-playing function, text needed at least one function
Word translation function etc.) etc.；Storage data field can store uses created data (such as voice data, word according to mobile phone
Message data etc.) etc..In addition, memory can include high-speed random access memory, nonvolatile memory can also be included,
Such as hard disk, internal memory, plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure
Digital, SD) card, flash card (Flash Card), at least one disk memory, flush memory device or other volatibility are consolidated
State memory device.
Wherein, if the module that the terminal device that the video pictures are shown integrates is realized in the form of SFU software functional unit
And as independent production marketing or in use, it can be stored in a computer read/write memory medium.Based on such
Understand, the present invention realizes all or part of flow in above-described embodiment method, can also instruct phase by computer program
The hardware of pass is completed, and described computer program can be stored in a computer-readable recording medium, the computer program exists
When being executed by processor, can be achieved above-mentioned each embodiment of the method the step of.Wherein, the computer program includes computer journey
Sequence code, the computer program code can be source code form, object identification code form, executable file or some middle shapes
Formula etc..The computer-readable medium can include：Any entity or device, the note of the computer program code can be carried
Recording medium, USB flash disk, mobile hard disk, magnetic disc, CD, computer storage, read-only storage (ROM, Read-Only Memory),
Random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium
Deng.It should be noted that the content that the computer-readable medium includes can be real according to legislation in jurisdiction and patent
The requirement trampled carries out appropriate increase and decrease, such as in some jurisdictions, according to legislation and patent practice, computer-readable medium
Do not include electric carrier signal and telecommunication signal.
It should be noted that device embodiment described above is only schematical, wherein described be used as separating component
The unit of explanation can be or may not be physically separate, can be as the part that unit is shown or can also
It is not physical location, you can with positioned at a place, or can also be distributed on multiple NEs.Can be according to reality
Need to select some or all of module therein to realize the purpose of this embodiment scheme.In addition, device provided by the invention
In embodiment accompanying drawing, the annexation between module represents there is communication connection between them, specifically can be implemented as one or
A plurality of communication bus or signal wire.Those of ordinary skill in the art are without creative efforts, you can to understand
And implement.
Described above is the preferred embodiment of the present invention, it is noted that for those skilled in the art
For, under the premise without departing from the principles of the invention, some improvements and modifications can also be made, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (14)
1. a kind of method that video pictures are shown, it is characterised in that including：
Receive the SOT state of termination change notification from server；
User list is updated according to the SOT state of termination change notification of reception, the user list includes the user to make a speech
Data；
The corresponding interface of user data display in the user list.
2. the method that video pictures according to claim 1 are shown, it is characterised in that described according in the user list
The corresponding interface of user data display, specifically include：
When the number of the user data in the user list is more than 1, the user list is shown in current interface；
When the number of the user data in the user list is equal to 1, according to corresponding with the user data from server
Treated the first video data of compression, small window video preview is generated in current interface.
3. the method that video pictures according to claim 2 are shown, it is characterised in that when the use in the user list
When the number of user data is equal to 1, according to from server video pictures corresponding with the user data, given birth in current interface
Into after small window video preview, in addition to：
When updating the user list according to the user terminal change notification from server, detect and regarded with current small window
User data corresponding to frequency preview does not stop making a speech and detecting that the number of the user data in the user list is more than 1,
Then the user list comprising newly-increased user data is shown in current interface.
4. the method for switching long-distance video according to claim 2, it is characterised in that also include：
When detecting the event for clicking on the small window video preview, to server request and the small window video preview pair
The second video data without overcompression processing answered；
The second video data returned from server according to request is received, and it is full frame aobvious according to second video data generation
Show picture.
5. a kind of method that video pictures are shown, it is characterised in that including：
Receive the multi-medium data from each user terminal uploads；Wherein, the multi-medium data includes voice data；
Extract the audio frequency characteristics of the voice data of each user terminal uploads；
The floor status of each user terminal is determined according to the audio frequency characteristics, and is given birth to according to the floor status of each user terminal
Into SOT state of termination change notification；
The SOT state of termination change notification is sent to each user terminal, so that each user terminal is according to reception
SOT state of termination change notification renewal user list, and the corresponding boundary of user data display in the user list
Face.
6. the method that video pictures according to claim 5 are shown, it is characterised in that the audio frequency characteristics comprise at least with
It is one of lower：Decibel value, tone color, tone, vocal print.
7. the method that video pictures according to claim 5 are shown, it is characterised in that the audio frequency characteristics are decibel value,
The then floor status that each user terminal is determined according to the audio frequency characteristics, and according to the floor status of each user terminal
SOT state of termination change notification is generated, is specifically included：
When the decibel value higher than specific threshold and duration be more than it is most short speak duration when, mark the speech of this user terminal
State is to make a speech；
When the decibel value is less than specific threshold and the duration is more than maximum silence duration, the speech of this user terminal is marked
State is made a speech for stopping；
SOT state of termination change notification is generated according to the floor status of each user terminal.
8. the method that the video pictures according to claim 5 to 7 any one are shown, it is characterised in that also include：
Effective voice data is selected from the voice data according to the floor status of each user terminal, and will be described effective
Voice data is sent to each user terminal.
9. the method that video pictures according to claim 5 are shown, it is characterised in that the multi-medium data also includes each
The video data of individual user terminal uploads, the then method that the video pictures are shown also include：
, will video corresponding with the user terminal when the quantity for detecting that floor status is the user terminal made a speech is 1
Data are compressed processing, and the first video data that the compression is handled to obtain is sent to each user terminal；
When receiving, the user data in the user list for sending with the user terminal from one of user terminal is corresponding
When not being compressed the request of the second video data of processing, second video data is sent to the user terminal.
A kind of 10. device that video pictures are shown, it is characterised in that including：
State receiving module, for receiving the SOT state of termination change notification from server；
Update module, for updating user list according to the SOT state of termination change notification of reception, the user list includes
The user data made a speech；
Display control module, for the corresponding interface of user data display in the user list.
A kind of 11. device that video pictures are shown, it is characterised in that including：
Data reception module, for receiving the voice data of itself and video data from each user terminal uploads；
Extraction module, the audio frequency characteristics of the voice data for extracting each user terminal uploads；
Determining module, for determining the floor status of each user terminal according to the audio frequency characteristics, and it is whole according to each user
The floor status generation SOT state of termination change notification at end；
Sending module, for the SOT state of termination change notification to be sent into each user terminal, so that each use
Family terminal updates user list, and the number of users in the user list according to the SOT state of termination change notification of reception
According to the corresponding interface of display.
12. the system that a kind of video pictures are shown, it is characterised in that including at least two user terminals and at least one service
Device, wherein：
The user terminal, for the multi-medium data of collection to be sent into server；The multi-medium data includes audio number
According to；
The server, for receiving the multi-medium data from each user terminal uploads, extract the multimedia number
The audio frequency characteristics of voice data in, the floor status of each user terminal is determined according to the audio frequency characteristics, and according to institute
After the floor status generation SOT state of termination change notification for stating each user, the SOT state of termination change notification is sent to described each
Individual user terminal；
The user terminal, it is additionally operable to receive the SOT state of termination change notification from the server, according to the SOT state of termination
Change notification updates user list, and the corresponding interface of user data display in the user list；Wherein, the user
List includes the user data made a speech.
13. a kind of terminal device, including processor, memory and it is stored in the memory and is configured as by described
The computer program that device performs is managed, is realized described in the computing device during computer program as any one in claim 1 to 9
The method that video pictures described in are shown.
14. a kind of computer-readable recording medium, it is characterised in that the computer-readable recording medium includes the calculating of storage
Machine program, wherein, equipment where controlling the computer-readable recording medium when the computer program is run is performed as weighed
Profit requires the method that the video pictures in 1 to 9 described in any one are shown.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710949829.XA CN107682752B (en) | 2017-10-12 | 2017-10-12 | Method, device and system for displaying video picture, terminal equipment and storage medium |
| PCT/CN2017/116628 WO2019071808A1 (en) | 2017-10-12 | 2017-12-15 | Video image display method, apparatus and system, terminal device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710949829.XA CN107682752B (en) | 2017-10-12 | 2017-10-12 | Method, device and system for displaying video picture, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107682752A true CN107682752A (en) | 2018-02-09 |
| CN107682752B CN107682752B (en) | 2020-07-28 |
Family
ID=61139936
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710949829.XA Active CN107682752B (en) | 2017-10-12 | 2017-10-12 | Method, device and system for displaying video picture, terminal equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107682752B (en) |
| WO (1) | WO2019071808A1 (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109963107A (en) * | 2019-02-20 | 2019-07-02 | 视联动力信息技术股份有限公司 | A kind of display methods and system of audio, video data |
| CN110996021A (en) * | 2019-11-30 | 2020-04-10 | 咪咕文化科技有限公司 | Director switching method, electronic device and computer readable storage medium |
| CN112383738A (en) * | 2020-11-11 | 2021-02-19 | 浙江讯盟科技有限公司 | Multi-user audio and video conference method and system with low traffic and resource consumption |
| CN113596349A (en) * | 2021-07-26 | 2021-11-02 | 世邦通信股份有限公司 | Conference method, system, device and storage medium for automatic linkage of speech position and video |
| CN113784151A (en) * | 2020-06-10 | 2021-12-10 | 腾讯科技（深圳）有限公司 | Data processing method and device, computer equipment and storage medium |
| CN113923359A (en) * | 2021-10-13 | 2022-01-11 | 宁波米福软件有限公司 | Fusion method and fusion system for court trial field pictures and remote pictures |
| CN114697732A (en) * | 2020-12-30 | 2022-07-01 | 华为技术有限公司 | Shooting method, system and electronic equipment |
| CN113784151B (en) * | 2020-06-10 | 2024-05-17 | 腾讯科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20230065847A1 (en) * | 2021-08-31 | 2023-03-02 | International Business Machines Corporation | Network bandwidth conservation during video conferencing |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101442654A (en) * | 2008-12-26 | 2009-05-27 | 深圳华为通信技术有限公司 | Method, apparatus and system for switching video object of video communication |
| CN102404542A (en) * | 2010-09-09 | 2012-04-04 | 华为终端有限公司 | Method and device for adjusting display of images of participants in multi-screen video conference |
| CN102647578A (en) * | 2011-02-17 | 2012-08-22 | 鸿富锦精密工业（深圳）有限公司 | Video switching system and method |
| US20130010049A1 (en) * | 2011-07-08 | 2013-01-10 | Adel Mostafa | Negotiate multi-stream continuous presence |
| CN104038725A (en) * | 2010-09-09 | 2014-09-10 | 华为终端有限公司 | Method and device for adjusting conventioneer image display in multi-screen video conference |
| CN105791738A (en) * | 2014-12-15 | 2016-07-20 | 深圳Tcl新技术有限公司 | Method and device for adjusting video window in video conference |
| CN106063255A (en) * | 2014-02-27 | 2016-10-26 | 谷歌公司 | Displaying a presenter during a video conference |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297743A (en) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | Video conference display window adjusting method and video conference service equipment |
-
2017
- 2017-10-12 CN CN201710949829.XA patent/CN107682752B/en active Active
- 2017-12-15 WO PCT/CN2017/116628 patent/WO2019071808A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101442654A (en) * | 2008-12-26 | 2009-05-27 | 深圳华为通信技术有限公司 | Method, apparatus and system for switching video object of video communication |
| CN102404542A (en) * | 2010-09-09 | 2012-04-04 | 华为终端有限公司 | Method and device for adjusting display of images of participants in multi-screen video conference |
| CN104038725A (en) * | 2010-09-09 | 2014-09-10 | 华为终端有限公司 | Method and device for adjusting conventioneer image display in multi-screen video conference |
| CN102647578A (en) * | 2011-02-17 | 2012-08-22 | 鸿富锦精密工业（深圳）有限公司 | Video switching system and method |
| US20130010049A1 (en) * | 2011-07-08 | 2013-01-10 | Adel Mostafa | Negotiate multi-stream continuous presence |
| CN106063255A (en) * | 2014-02-27 | 2016-10-26 | 谷歌公司 | Displaying a presenter during a video conference |
| CN105791738A (en) * | 2014-12-15 | 2016-07-20 | 深圳Tcl新技术有限公司 | Method and device for adjusting video window in video conference |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109963107A (en) * | 2019-02-20 | 2019-07-02 | 视联动力信息技术股份有限公司 | A kind of display methods and system of audio, video data |
| CN110996021A (en) * | 2019-11-30 | 2020-04-10 | 咪咕文化科技有限公司 | Director switching method, electronic device and computer readable storage medium |
| CN113784151A (en) * | 2020-06-10 | 2021-12-10 | 腾讯科技（深圳）有限公司 | Data processing method and device, computer equipment and storage medium |
| CN113784151B (en) * | 2020-06-10 | 2024-05-17 | 腾讯科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium |
| CN112383738A (en) * | 2020-11-11 | 2021-02-19 | 浙江讯盟科技有限公司 | Multi-user audio and video conference method and system with low traffic and resource consumption |
| CN114697732A (en) * | 2020-12-30 | 2022-07-01 | 华为技术有限公司 | Shooting method, system and electronic equipment |
| WO2022143883A1 (en) * | 2020-12-30 | 2022-07-07 | 华为技术有限公司 | Photographing method and system, and electronic device |
| CN113596349A (en) * | 2021-07-26 | 2021-11-02 | 世邦通信股份有限公司 | Conference method, system, device and storage medium for automatic linkage of speech position and video |
| CN113923359A (en) * | 2021-10-13 | 2022-01-11 | 宁波米福软件有限公司 | Fusion method and fusion system for court trial field pictures and remote pictures |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019071808A1 (en) | 2019-04-18 |
| CN107682752B (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107682752A (en) | Method, apparatus, system, terminal device and the storage medium that video pictures are shown | |
| US9973506B2 (en) | Method and apparatus for clearing notification icon, and storage medium | |
| CN106254311A (en) | Live broadcasting method and device, live data streams methods of exhibiting and device | |
| US11247134B2 (en) | Message push method and apparatus, device, and storage medium | |
| CN108491147A (en) | A kind of man-machine interaction method and mobile terminal based on virtual portrait | |
| CN109378000A (en) | Voice awakening method, device, system, equipment, server and storage medium | |
| CN101867487A (en) | Call out the system and method that connects symbol management association centre with figure | |
| CN110209799A (en) | Message treatment method, the display methods of unread message, terminal | |
| CN103098042A (en) | System and method for social collection | |
| US20150120824A1 (en) | Communications Method, Client, and Terminal | |
| CN110716676A (en) | Message reminding method and terminal | |
| CN107436748A (en) | Handle method, apparatus, terminal device and the computer-readable recording medium of third-party application message | |
| CN107864297A (en) | Group's adding method, device, terminal and storage medium | |
| CN107222790A (en) | A kind of method, terminal and computer-readable recording medium for sending barrage | |
| CN108055404A (en) | Reminder announced method, apparatus, terminal and storage medium | |
| CN107659850A (en) | Media information processing method and device | |
| CN108616392A (en) | Intelligent terminal business configuration system and method | |
| EP2838225A1 (en) | Message based conversation function execution method and electronic device supporting the same | |
| CN105808231A (en) | System and method for recording script and system and method for playing script | |
| CN110213154A (en) | Instant communicating method, device, computer installation and storage medium | |
| CN110267051B (en) | Data processing method and device | |
| CN107105104A (en) | Terminal jingle bell reminding method and system | |
| CN109951504B (en) | Information pushing method and device, terminal and storage medium | |
| CN107346197A (en) | A kind of method for information display and device | |
| CN113827953B (en) | Game control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |