CN110418150A - Information prompting method, equipment, system and computer readable storage medium - Google Patents

Information prompting method, equipment, system and computer readable storage medium Download PDF

Info

Publication number
CN110418150A
CN110418150A CN201910639977.0A CN201910639977A CN110418150A CN 110418150 A CN110418150 A CN 110418150A CN 201910639977 A CN201910639977 A CN 201910639977A CN 110418150 A CN110418150 A CN 110418150A
Authority
CN
China
Prior art keywords
target
live
live stream
stream
event
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910639977.0A
Other languages
Chinese (zh)
Other versions
CN110418150B (en
Inventor
钟宜峰
张健
莫东松
张进
赵璐
马丹
马晓琳
王科
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Migu Cultural Technology Co Ltd
Original Assignee
Migu Cultural Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Migu Cultural Technology Co Ltd filed Critical Migu Cultural Technology Co Ltd
Priority to CN201910639977.0A priority Critical patent/CN110418150B/en
Publication of CN110418150A publication Critical patent/CN110418150A/en
Application granted granted Critical
Publication of CN110418150B publication Critical patent/CN110418150B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213Monitoring of end-user related data
    • H04N21/44218Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/488Data services, e.g. news ticker
    • H04N21/4882Data services, e.g. news ticker for displaying messages, e.g. warnings, reminders
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/63Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
    • H04N21/643Communication protocols
    • H04N21/6437Real-time Transport Protocol [RTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Databases & Information Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses an information prompting method, equipment, a system and a computer readable storage medium, relates to the technical field of live broadcast, and aims to solve the problem that information cannot be timely prompted to a user in the existing live broadcast scene. The method comprises the following steps: collecting a live broadcast image of a live broadcast scene; encoding the live broadcast image to obtain a live broadcast stream; under the condition that a target event occurs in the live broadcast scene, adding indication information of the target event in the live broadcast stream to obtain a target live broadcast stream; and sending the target live stream to a client, or sending the target live stream to a server. The embodiment of the invention can improve the timeliness of information prompt in a live broadcast scene.

Description

A kind of information cuing method, equipment, system and computer readable storage medium
Technical field
The present invention relates to direct seeding technique field more particularly to a kind of information cuing method, equipment, systems and computer-readable Storage medium.
Background technique
With the popularization of intelligent terminals, more and more users like watching the programme televised lives such as sports tournament by terminal. During live streaming, live streaming platform plays live signal.By taking football match as an example, the time of football match is long, and big Part-time is the non-excellent time.So, in the meantime, user may be absent-minded or handles other things, so as to cause ratio is missed The splendid moment of match.
According to the above analysis it is found that not being available the content that family understands programme televised live in time in the way of the prior art Information.
Summary of the invention
The embodiment of the present invention provides a kind of information cuing method, equipment, system and computer readable storage medium, to solve The problem of information can not be prompted the user in existing live scene in time.
In a first aspect, being applied to Image Acquisition end the embodiment of the invention provides a kind of information cuing method, comprising:
Acquire the live video of live scene;
The live video is encoded, live stream is obtained;
In the case where object event has occurred in determining the live scene, the target is added in the live stream The instruction information of event, obtains target live stream;
The target live stream is sent to client, or sends the target live stream to server-side, so that the clothes End be engaged in the client transmission target live stream, so that the client is broadcast live according to the target live stream When, the object event is prompted according to the instruction information.
Wherein, the object event includes: score change events;It is described that the object event is added in the live stream Instruction information, obtain target live stream, comprising:
It acquires in the live scene than distributional image;
It is analyzed than distributional image described;
In the case where determining that variation has occurred in score based on the analysis results, the score change events are had occurred in determination;
First identifier is added in the first predetermined position of the live stream, obtains the target live stream;First mark Knowledge, which is used to indicate, has occurred the score change events.
Wherein, the object event includes: penalty shot event;The finger that the object event is added in the live stream Show information, obtain target live stream, comprising:
Acquire the panoramic picture of the live scene;
Detect position of the official ball in the panoramic picture;
In the case where determining that the position of the official ball in the predetermined time does not change, it is determined that penalized described in generation Ball event;
Second identifier is added in the first predetermined position of the live stream, obtains the target live stream;Second mark Knowledge, which is used to indicate, has occurred the penalty shot event.
Wherein, the object event includes: penalty event;The finger that the object event is added in the live stream Show information, obtain target live stream, comprising:
Acquire the image judged in the live scene;
The image for whether occurring penalty apparatus in the image of the judge detected;
In the case where determining the image for the penalty apparatus occur, the penalty event is had occurred in determination;
In the first predetermined position of live stream addition third mark, the target live stream is obtained;The third mark Knowledge, which is used to indicate, has occurred the penalty event.
Wherein, the object event includes: target person occur;It is described that the object event is added in the live stream Instruction information, obtain target live stream, comprising:
Recognition of face is carried out to the live video;
The facial image recognized and preset facial image are compared;
In the case where comparison result meets predetermined condition, it is determined to be the target person;
The 4th mark of addition in the first predetermined position of the live stream obtains the target live stream, the 4th mark Knowledge, which is used to indicate, has there is the target person.
Wherein, it is determined to be after the target person described in the case where comparison result meets predetermined condition, The method also includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
Second aspect, the embodiment of the present invention provide a kind of information cuing method, are applied to server-side, comprising:
Receive the target live stream that Image Acquisition end is sent, wherein include the finger of object event in the target live stream Show information;
The target live stream is sent to client, so that the client is broadcast live according to the target live stream When, the object event is prompted according to the instruction information.
Wherein, in the case where the object event does not include target person occur, described to described in client transmission Before target live stream, the method also includes:
The target live stream is decoded, live video is obtained;
Recognition of face is carried out to the live video;
According to recognition of face as a result, having determined whether target person;
In the case where being determined to be the target person, the 4th mark of addition in the first predetermined position of the live stream Know, the 4th mark, which is used to indicate, the target person has occurred.
Wherein, described according to recognition of face as a result, having determined whether target person after, the method is also wrapped It includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
The third aspect, the embodiment of the present invention provide a kind of information cuing method, are applied to client, comprising:
Receive the target live stream at Image Acquisition end or server-side transmission, wherein include mesh in the target live stream The instruction information of mark event;
When being broadcast live according to the target live stream, the object event is mentioned according to the instruction information Show.
It is wherein, described that the object event is prompted according to the instruction information, comprising:
The prompt for obtaining user selects information;
In the case where prompt selection information and the instruction information matches, the preselected prompt side of user is obtained Formula;
Using the prompting mode, the object event is prompted.
Wherein, there is target person in instruction information instruction, and described including identifying in the prompting mode It is described to utilize the prompting mode when target person, the object event is prompted, comprising:
Determine the human face region of the target person;
In the human face region, the facial contour of the target person is identified.
Wherein, the human face region of the determination target person, comprising:
Obtain the first parameter for including in the second predetermined position of the target live stream;
According to first parameter, the central point of the facial contour is obtained in the human body image where the target person Normalized coordinate in frame;
The second parameter is obtained according to the normalized coordinate;
The human body image frame is divided at least one region, each region has corresponding matrix;
The matrix is traversed, in objective matrix, if the corresponding element value of second parameter is target value, the mesh Marking the corresponding target area of matrix is human face region.
Wherein, described in the human face region, identify the facial contour of the target person, comprising:
In the human face region, the corresponding pixel of facial contour is determined;
Highlight the target pixel points.
Fourth aspect, the embodiment of the invention provides a kind of information alert equipment, comprising: transceiver, memory, processor And it is stored in the computer program that can be run on the memory and on the processor;
The processor, for reading the step in the method for the realization of the program in memory as described in relation to the first aspect;Or Person realizes the step in the method as described in second aspect;Or step of the realization in the method as described in the third aspect.
5th aspect, the embodiment of the invention provides a kind of computer readable storage mediums, for storing computer program, The step in method as described in relation to the first aspect is realized when the computer program is executed by processor;Or realize such as second party Step in method described in face;Or step of the realization in the method as described in the third aspect.
6th aspect, the embodiment of the invention provides a kind of information presentation systems, comprising: Image Acquisition end and at least one Client;
Described image collection terminal encodes the live video for acquiring the live video of live scene, obtains Live stream;In the case where object event has occurred in determining the live scene, the target is added in the live stream The instruction information of event obtains target live stream, and sends the target live stream to the client;
The client, for receiving the target live stream of described image collection terminal transmission, and straight according to the target When broadcasting stream and being broadcast live, the object event is prompted according to the instruction information.
7th aspect, the embodiment of the invention provides a kind of information presentation systems, comprising: Image Acquisition end, server-side and At least one client;
Described image collection terminal encodes the live video for acquiring the live video of live scene, obtains Live stream;In the case where object event has occurred in determining the live scene, the target is added in the live stream The instruction information of event obtains target live stream, and sends the target live stream to the server-side;
The server-side, for sending the target live stream to the client;
The client, the target live stream sent for receiving the server-side, and according to the target live stream When being broadcast live, the object event is prompted according to the instruction information.
In embodiments of the present invention, the object event in live scene is identified using Image Acquisition end.Therefore, objective It, can be according to the instruction when being broadcast live according to the target live stream after family termination receives the live stream of server-side transmission Information prompts the object event.Therefore, using the scheme of the embodiment of the present invention, family can be used timely to understand live streaming The event occurred in scene, to improve the timeliness of information alert.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, needed in being described below to the embodiment of the present invention Attached drawing to be used is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, For those of ordinary skill in the art, without any creative labor, it can also obtain according to these attached drawings Obtain other attached drawings.
Fig. 1 (a) and Fig. 1 (b) is the schematic diagram of the information presentation system of the embodiment of the present invention respectively;
Fig. 2 is one of the flow chart of information cuing method provided in an embodiment of the present invention;
Fig. 3 (a) and Fig. 3 (b) is the schematic diagram at Image Acquisition end in system provided in an embodiment of the present invention respectively;
Fig. 4 is the two of the flow chart of information cuing method provided in an embodiment of the present invention;
Fig. 5 is the schematic diagram of server-side in system provided in an embodiment of the present invention;
Fig. 6 is the three of the flow chart of information cuing method provided in an embodiment of the present invention;
Fig. 7 is one of the structure chart of information presentation device provided in an embodiment of the present invention;
Fig. 8 is one of the structure chart of Second processing module in information presentation device provided in an embodiment of the present invention;
Fig. 9 is two of the structure chart of Second processing module in information presentation device provided in an embodiment of the present invention;
Figure 10 is three of the structure chart of Second processing module in information presentation device provided in an embodiment of the present invention;
Figure 11 is the two of the structure chart of information presentation device provided in an embodiment of the present invention;
Figure 12 is the three of the structure chart of information presentation device provided in an embodiment of the present invention;
Figure 13 is the four of the structure chart of information presentation device provided in an embodiment of the present invention;
Figure 14 is the structure chart of cue module in information presentation device provided in an embodiment of the present invention;
Figure 15 is the structure chart of prompting submodule in information presentation device provided in an embodiment of the present invention;
Figure 16 is one of the structure chart of information alert equipment provided in an embodiment of the present invention;
Figure 17 is the two of the structure chart of information alert equipment provided in an embodiment of the present invention;
Figure 18 is the three of the structure chart of information alert equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts Example, shall fall within the protection scope of the present invention.
It is the information presentation system schematic diagram of the embodiment of the present invention referring to Fig. 1 (a), Fig. 1 (a).It is described as shown in Fig. 1 (a) System includes: Image Acquisition end 101, server-side 102 and at least one client (such as mobile phone) 103.Wherein, the figure Picture collection terminal 101 encodes the live video for acquiring the live video of live scene, obtains live stream;True In the case where object event has occurred in the fixed live scene, the instruction letter of the object event is added in the live stream Breath obtains target live stream, and sends the target live stream to the server-side;The server-side 102 is used for the visitor Family end sends the target live stream;The client 103, the target live stream sent for receiving the server-side, and When being broadcast live according to the target live stream, the object event is prompted according to the instruction information.
Signal of broadcasting live is acquired by the equipment at Image Acquisition end, pushes RTMP (Real Time after coding Messaging Protocol, real-time messages transport protocol) it flow to server-side.Server-side carries out necessary processing (such as to video flowing Broadcast Control, transcoding), and live streaming is streamed to by client by CDN (Content Delivery Network, content distributing network) End.Client is decoded and plays to live stream using player.
It should be noted that another information presentation system of the embodiment of the present invention also may not include shown in Fig. 1 (a) Server-side.So, in this case, the data at Image Acquisition end will be transmitted directly to client.It is described as shown in Fig. 1 (b) Information presentation system includes: Image Acquisition end 104 and at least one client 105.Wherein, described image collection terminal 104, is used for The live video for acquiring live scene, encodes the live video, obtains live stream;In determining the live scene In the case where object event has occurred, the instruction information of the object event is added in the live stream, obtains target live streaming Stream, and the target live stream is sent to the client;The client 105 is sent for receiving described image collection terminal Target live stream, and when being broadcast live according to the target live stream, according to the instruction information to the object event It is prompted.
In the system shown in Fig. 1 (a) or Fig. 1 (b), Image Acquisition end, server-side and client can be respectively intended to execute Corresponding process performed by Image Acquisition end, server-side and client in following methods embodiment.It should be noted that due to Difference on system composition, under different system architectures, the function that Image Acquisition end, server-side and client are realized slightly has Difference.Hereinafter, being described in detail in the specific implementation of the embodiment of the present invention under above system framework in conjunction with different embodiments Journey.
Referring to fig. 2, Fig. 2 is the flow chart of information cuing method provided in an embodiment of the present invention, is applied to Image Acquisition end, As shown in Figure 2, comprising the following steps:
Step 201, the live video for acquiring live scene.
In embodiments of the present invention, live video, such as video camera can be acquired using image capture device in live streaming scene Deng.
Step 202 encodes the live video, obtains live stream.
In this step, arbitrary coding method can be used to encode live video, form live stream.
Step 203 in the case where object event has occurred in determining the live scene, is added in the live stream The instruction information of the object event, obtains target live stream.
It in embodiments of the present invention, can also be existing in live streaming in order to enable user timely understands the information in live scene Image capture devices of more of the field deployment for auxiliary, are used for collection event information.For example, can dispose respectively for acquiring score The image capture device of board information, for acquiring the image capture device of judge's image, the image for acquiring official ball is adopted Collect equipment etc..
The object event includes but is not limited to are as follows: target person occur in score change events, penalty shot event, penalty event Object etc..Wherein, score change events refer to being changed than distributional score;Penalty shot event is referred to according to judge's Judgement carries out penalty shot, such as corner-kick, free kick etc.;Penalty event refers to that judge compares the ruling of race into row using penalty tool, Such as famous event, yellow card event;There is goal task and refer to preset personage occur in live streaming picture, such as some Football star etc..Pass through the detection to different event and corresponding instruction information is set, the diversity of prompt can be enriched, thus more Be conducive to user and timely understand information in live scene.
Different object events can be determined by different modes.It is settable not at Image Acquisition end as shown in Fig. 3 (a) Same module, it is determined whether object event occurs.For example, settable score change detection module, for detecting whether ratio occurs Divide change events.For example, settable penalty shot event checking module, for detecting whether penalty shot event occurs.In practical applications, By taking football match as an example, the penalty shot event checking module can be corner-kick free kick detection module.For example, settable penalty thing Part detection module, for detecting whether penalty event occurs.In practical applications, by taking football match as an example, the penalty event Detection module can be red and yellow card detection module.For example, settable target person detection module, for detecting whether there is mesh Mark personage.
If the object event includes score change events, then this step compares specifically, acquiring in the live scene Distributional image, and analyzed than distributional image described.In the feelings for determining that score is changed based on the analysis results Under condition, the score change events are had occurred in determination.Then, first identifier is added in the first predetermined position of the live stream, Obtain the target live stream;The first identifier, which is used to indicate, has occurred the score change events.Wherein, of the invention real It applies in example, first predetermined position refers in the head information of RTMP protocol block, CSID (Chunk Stream ID, circulation road ID the position where).Wherein, CSID supports that the customized numberical range of user is [3,65599].
As shown in Fig. 3 (a) or Fig. 3 (b), the score change detection module may include score variation picture pick-up device, coding Equipment etc..Encoding device may include OCR (Optical Character Recognition, optical character identification) recognizer Submodule, encoding service submodule etc..The live streaming of picture pick-up device real-time capture is live than distributional image information (1 frame per second), and Image information is transferred on encoding device.OCR recognizer submodule is believed using the identification score of Tesseract-OCR algorithm Breath.When algorithm identifies score variation, encoding service submodule is in the head information of RTMP protocol block, by the way that CSID to be arranged " score variation " this event is identified for 101.
If the object event includes penalty shot event, then this step is specifically, acquire the panorama sketch of the live scene Picture, and detect position of the official ball in the panoramic picture.Determining the position of the official ball in the predetermined time not In the case where changing, it is determined that the penalty shot event occurs.Then, in the first predetermined position of live stream addition the Two marks, obtain the target live stream;The second identifier, which is used to indicate, has occurred the penalty shot event.
In embodiments of the present invention, the different situations of penalty shot event can use different marks and be identified, can be with Unified mark is carried out to penalty shot event this kind event.If be identified to the different situations in penalty shot event, it can In the case where the first penalty shot event has occurred, in the first predetermined position of the live stream, one mark of addition, obtains the mesh Live stream is marked, the mark, which is used to indicate, has occurred the first penalty shot event;In the case where the second penalty shot event has occurred, In the first predetermined position of the live stream, one mark of addition, obtains the target live stream, the mark is used to indicate generation The second penalty shot event.The mark for being used to indicate different penalty shot events above is different.Wherein, the first penalty shot event and second Penalty shot event can be according to any setting of difference of live scene.For example, the first penalty shot event can be point by taking football match as an example Ball event, the second penalty shot event can be corner-kick event, and two different penalty shot events are indicated with different marks.
As shown in Fig. 3 (a) or Fig. 3 (b), the penalty shot event checking module may include penalty shot event picture pick-up device, coding Equipment etc..Encoding device may include algorithm of target detection submodule, encoding service submodule etc..The picture pick-up device real-time capture whole audience Image information (1 frame per second), and image information is transferred on encoding device.Algorithm of target detection submodule utilizes SSD (Single Shot MultiBox Detector, the more box detectors of single-shot) algorithm of target detection, detection official ball is (such as foot Ball) position coordinates in the picture.By taking football match as an example, if the position of continuous 3 seconds or more footballs does not change, compile Code Attendant sub-module in the head information of RTMP protocol block, by by CSID be set as 102 come identify " corner-kick free kick " this Event.
If the object event includes penalty event, then what this step specifically, acquire was judged in the live scene Image, and detect in the image of the judge image of penalty apparatus whether occur.Determining the figure for the penalty apparatus occur As in the case where, the penalty event is had occurred in determination.Then, third mark is added in the first predetermined position of the live stream Know, obtains the target live stream;The third mark, which is used to indicate, has occurred the penalty event.
In embodiments of the present invention, the different situations of penalty event can use different marks and be identified, can be with Unified mark is carried out to penalty event this kind event.If be identified to the different situations in penalty event, it can In the case where the first penalty event has occurred, in the first predetermined position of the live stream, one mark of addition, obtains the mesh Live stream is marked, the mark, which is used to indicate, has occurred the first penalty event;In the case where the second penalty event has occurred, In the first predetermined position of the live stream, one mark of addition, obtains the target live stream, the mark is used to indicate generation The second penalty event.The mark for being used to indicate different penalty events above is different.Wherein, the first penalty event and second Penalty event can be according to any setting of difference of live scene.For example, the first penalty event can be red by taking football match as an example Board event, the second penalty event can be yellow card event, and two different penalty events are indicated with different marks.
As shown in Fig. 3 (a) or Fig. 3 (b), the penalty event checking module may include penalty event picture pick-up device, coding Equipment etc..Encoding device may include algorithm of target detection submodule, encoding service submodule etc..Picture pick-up device real-time capture judge Image information (1 frame per second), and image information is transferred on encoding device.By taking football match as an example, algorithm of target detection Module utilizes the red yellow board in SSD algorithm of target detection detection image.If detecting red yellow board in the picture, clothes are encoded Submodule be engaged in the head information of RTMP protocol block, it is " red card " and " yellow to identify respectively by setting 103 and 104 for CSID The two events of board ".
If the object event includes target person occur, then this step is specifically, carry out people to the live video Face identification, and the facial image recognized and preset facial image are compared.Meet predetermined condition in comparison result In the case of, it is determined to be the target person.Then, it in the 4th mark of the first predetermined position of live stream addition, obtains To the target live stream, the 4th mark, which is used to indicate, has there is the target person.
In order to enable accurately target person is prompted in client, it can also be in the human figure for target person occur As determining normalized coordinate of the central point of facial contour in the human body image frame where the target person, then in frame The first parameter is determined according to the normalized coordinate.Later, in the second predetermined position of live stream addition first ginseng Number, first parameter indicate relative position of the central point of the facial contour in live streaming picture.In this way, may be used So that client fast and accurately determines facial contour, and prompted.
Wherein, the second predetermined position can also be CSID in the head information of RTMP protocol block, but unlike that above-mentioned first Predetermined position.First predetermined position and the second predetermined position can be CSID adjacent in RTMP protocol block, can also have interval.
Assuming that absolute coordinate of the face frame detected using Face datection algorithm in the upper left corner of live streaming picture are as follows: (x0, y0), lower right corner absolute coordinate are as follows: (x1, y1), screen size is broadcast live are as follows: w*h, w indicate the length of live streaming picture, and h indicates live streaming The width of picture, w, h are positive integer.Here, carrying out following operation:
wp∈ [0,100], hp∈ [0,100].
Wherein, (wp, hp) indicate normalization after face frame central point coordinate.
Such as wpOr hpFor an integer, then respectively in wpOr hpIt is preceding to mend 0.Later, by wpAnd hpIt is spliced into four integers.In The most anteposition of four integers mends 1, obtains five integers, and set five integers for CSID field.Five integers As described first parameter.
In conjunction with shown in Fig. 3 (a), the target person detection module may include encoding device etc..Encoding device may include people Face detection algorithm submodule, face recognition algorithms submodule, encoding service submodule etc..Acquire the camera review of live video Information is transferred on encoding device.Face datection algorithm submodule and face recognition algorithms submodule need to carry out people to every frame image Face detection and recognition of face.Specifically, every frame image data is input to acquisition face frame in Face datection algorithm submodule.Example Such as, Face datection algorithm can be MTCNN (Multi-task convolutional neural network, multitask convolution Neural network) Face datection algorithm.Image information in face frame is intercepted, is input in face recognition algorithms submodule.For example, people Face recognizer can be facenet algorithm etc..When the face characteristic detected is low with the Euclidean distance of star's face planting modes on sink characteristic When given threshold, then it is assumed that the star occur in the frame image.Encoding service submodule is by the head information of RTMP protocol block Middle CSID is set as 111 (assuming that 111 represent above-mentioned star) to identify " target person X occur " this event.
It should be noted that the various values of above-mentioned CSID can be arbitrarily arranged, it is merely illustrative above.
Step 204 sends the target live stream to client, or sends the target live stream to server-side, with The server-side is set to send the target live stream to client, so that the client is according to the target live stream When being broadcast live, the object event is prompted according to the instruction information.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
In practical applications, to save computing resource, processing speed is improved, some computationally intensive modules can not be set It sets at Image Acquisition end.For example, target person detection module can be not arranged in Image Acquisition end, such as it may be provided at server-side Deng.At this point, the structure at Image Acquisition end can be as shown in Fig. 3 (b).
Referring to fig. 4, Fig. 4 is the flow chart of information cuing method provided in an embodiment of the present invention, is applied to server-side, such as schemes Shown in 4, comprising the following steps:
Step 401 receives the target live stream that Image Acquisition end is sent, wherein includes target in the target live stream The instruction information of event.
Wherein, the meaning of the object event can refer to the description of preceding method embodiment, indicate the specific instruction of information Mode can also refer to the description of previous embodiment.
Step 402 sends the target live stream to client, so that the client is according to the target live stream When being broadcast live, the object event is prompted according to the instruction information.
In server-side, in the case where the object event does not include target person occur, it also can detect whether occur Target person, thus the abundant prompting to the event in live scene.Specifically, server-side solves the target live stream Code obtains live video, and carries out recognition of face to the live video.According to recognition of face as a result, determining whether Target person.Then, in the case where being determined to be the target person, add in the first predetermined position of the live stream Add the 4th mark, the 4th mark, which is used to indicate, the target person has occurred.
Wherein, the first position includes CSID in the head information of RTMP protocol block.4th mark is, for example, 111 (assuming that 111 represent above-mentioned star), for identifying " target person X occur " this event.Alternatively, can also be any other setting Value.
Equally, in this embodiment, in order to enable accurately being prompted target person in client, can also occur In the human body image frame of target person, determine the central point of facial contour in the human body image frame where the target person Normalized coordinate, the first parameter is then determined according to the normalized coordinate.Later, in the second pre-determined bit of the live stream Addition first parameter is set, first parameter indicates opposite position of the central point of the facial contour in live streaming picture It sets.
Wherein it is determined that the mode of the first parameter can refer to the description of previous embodiment.Second predetermined position can also be RTMP CSID in the head information of protocol block, but unlike that the first above-mentioned predetermined position.First predetermined position and the second predetermined position It can be CSID adjacent in RTMP protocol block, can also have interval.
In conjunction with Fig. 5, server-side can include: decoding device/decoding service module, face detection module, face recognition module, Encoding device/encoding service module etc..
Server-side receives RTMP data flow, is decoded by decoding device/decoding service module, obtains every frame picture number According to.Face datection and recognition of face are carried out to every frame image.Every frame image data is input in face detection module and obtains people Face frame.For example, Face datection detection module, which can use MTCNN Face datection algorithm, carries out Face datection.It intercepts in face frame Image information is input in face recognition module.Face recognition module can use facenet algorithm etc. and carry out recognition of face. When the Euclidean distance of the face characteristic and star's face planting modes on sink characteristic that detect is lower than given threshold, then it is assumed that go out in the frame image The star is showed.Encoding device/encoding service module sets 111 (assuming that 111 generations for CSID in the head information of RTMP protocol block The above-mentioned star of table) identify " target person X occur " this event.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
It is the flow chart of information cuing method provided in an embodiment of the present invention referring to Fig. 6, Fig. 6, is applied to client, such as schemes Shown in 6, comprising the following steps:
Step 601, the target live stream for receiving Image Acquisition end or server-side transmission, wherein the target live stream In include object event instruction information.
Wherein, the meaning of the object event can refer to the description of preceding method embodiment, indicate the specific instruction of information Mode can also refer to the description of previous embodiment.
Step 602, when being broadcast live according to the target live stream, according to the instruction information to the object event It is prompted.
Wherein, in this step, to make the information of prompt be more in line with the demand of user, the prompt choosing of user can be obtained Select information.In the case where prompt selection information and the instruction information matches, the preselected prompting mode of user is obtained. Later, using the prompting mode, the object event is prompted.
In practical applications, user can carry out the setting of event prompting before live streaming starts and in live streaming progress.In Client, can be by all event prompts for supporting reminding business of system to user, including " score variation " mentioned above, " angle The events such as ball free kick ", " red and yellow card ", " star X appearance ", " star Y appearance ".So, which user can be arranged by choosing Event is reminded when occurring.Correspondingly, client can obtain the prompt selection information of user.
Meanwhile also prompting mode can be set according to the input of user.Wherein, prompting mode include any of the following or Person's various ways:
Concern star is marked with red contours line;Playing alert tones;Trigger mobile phone vibration;Screen specific position dodges three times Bright event type related text;Within the set time (in such as 5 seconds), it is (such as highlighted aobvious every 1 second that picture brightness is intermittently turned up Show 1 second), realize entire picture flicker effect;Player window shake;Other movements etc. with reminding effect.
Client player receives live stream and is decoded.The prompt for reading CSID field and user setting selects letter Breath.For example, if CSID is 101, and user selected " score variation " event of prompting, then triggers " score variation " event and mention It wakes up.Alerting pattern may include above one or more kinds of combinations, or according to the user's choice, be mentioned using what user chose Awake mode reminds user.
There is target person in instruction information instruction, and includes identifying the target person in the prompting mode When object, in this step, the human face region of the target person can first determine that.Then, in the human face region, institute is identified State the facial contour of target person.
When determining human face region, the first parameter for including in the second predetermined position of the target live stream, root are obtained According to first parameter, normalizing of the central point of the facial contour in the human body image frame where the target person is obtained Change coordinate.The second parameter is obtained further according to the normalized coordinate.Then, the human body image frame is divided at least one area Domain, each region have corresponding matrix.Then, the matrix is traversed, in objective matrix, if second parameter is corresponding Element value is target value, then the corresponding target area of the objective matrix is human face region.
For the human face region determined, in the human face region, the corresponding pixel of facial contour is determined, it is prominent aobvious Show the target pixel points.Such as use these pixels of certain color identifier.
When client detects that the expression of CSID field recognizes star X, continues to monitor CSID field, read above-mentioned First parameter, i.e. five integers.Then, latter four of five integers are split as two integer wpAnd hp, wp/ 100 and hp/100 Respectively indicate the normalized coordinate of the face frame central point of star X in the frame image.
Later, using Mask (mask)-RCNN (Region CNN, region convolutional neural networks) algorithm to the frame image into Row human body segmentation, obtains one or more mask matrixes having a size of w*h, and each mask matrix indicates a personage in the picture Region, shaped like:
Wherein, the corresponding matrix parameter of human region is 1, other regions are 0.All mask matrixes are traversed, if some Element in matrix:Then indicate that the matrix corresponds to star X.
Then, the pixel color that coordinate is following values is disposed as the color for having certain, such as red:
That is, traversing every a line in the matrix, finding out and meet xC, rThe smallest C greater than 0, the element then correspond to face wheel Wide leftmost edge;It finds out and meets xC, rMaximum C greater than 0, the element then correspond to the rightmost edges of facial contour.So far, really The pixel of facial contour is made, and it is identified with red.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
In conjunction with Fig. 3 (b) and Fig. 5, it is assumed that target face detection module is arranged in server-side.For carrying out football live streaming, The information cuing method of the embodiment of the present invention may include following steps:
Step S1, Image Acquisition end acquires image.
Image Acquisition end is provided with more picture pick-up devices, and main picture pick-up device is for capturing live game coverage key frame, i.e. user The picture finally watched;Score variation detection picture pick-up device is captured than distributional picture, one frame of pumping per second, is identified for score;Angle Ball free kick detection picture pick-up device captures entire court picture, one frame of pumping per second, for football detection positioning above court;It is red Yellow card detects picture pick-up device and captures judge's picture, and one frame of pumping per second is detected for red and yellow card.
Step S2, Image Acquisition end detecting event.
After race starts, main picture pick-up device starts to acquire real-time live broadcast picture, is transferred to the progress of collection terminal encoding device RTMP coding, forms RTMP live stream.Score variation detection picture pick-up device, corner-kick free kick detection picture pick-up device, red and yellow card inspection Surveying picture pick-up device can be to encoding device one frame image of transmission per second.The Tesseract-OCR identification being deployed on encoding device Algorithm, football algorithm of target detection (SSD), red and yellow card algorithm of target detection (SSD) respectively analyze these images, per second The position than fractional value, football on court can be obtained and whether judge lifts red and yellow card.
Step S3, the encoding device setting instruction information at Image Acquisition end.
CSID field in the head information of RTMP protocol block supports user customized, can value range be [3,65599].This Different event is identified by setting CSID field in inventive embodiments.
When the incident Detection Algorithm on the encoding device at Image Acquisition end detects score variation, encoding service will CSID field is set as 101;When detecting that football position does not change for continuous 3 seconds, i.e., it is any corner-kick will to occur for judgement Ball event, encoding service set 102 for CSID field;When detecting that judge lifts red card or when yellow card, encoding service is by word Section is respectively set to 103 and 104.
Step S4, server-side detects whether target person occur, and instruction information is arranged if occurring.
The RTMP live streaming of live streaming key frame is streamed to server-side.The decoding device of server-side is decoded RTMP stream, Image information is obtained, MTCNN Face datection algorithm and Facenet face recognition algorithms are transferred to, carries out the identification of star's face.Work as knowledge When the face being clipped in star library, CSID field is set star's number (such as 111 expression star X) by encoding service.
Wherein, the mode of mark can refer to the description of preceding method embodiment.
Step S5, client obtains the prompt selection information and alerting pattern of user setting.
User can carry out the prompting setting of event before live streaming starts and in live streaming progress.So, correspondingly, client The setting information of user can be obtained.All events for supporting reminding business can be seen in user, and including mentioned above, " score becomes Which event change ", " corner-kick free kick ", " red and yellow card ", " star X appearance ", " star Y appearance " etc., user are arranged by choosing It is reminded when generation.Client prompting acts the description that can refer to previous embodiment, when the generation of each event may be selected in user Which which or prompting is triggered to act.
Step S6, RTMP live stream is transferred to client by CDN.Event parsing module pair in client player The CSID field of RTMP stream is parsed, and is such as judged that the field values correspond to the preselected event of user, is then triggered user and set The prompting movement set.Wherein, star can refer to the description of previous embodiment with the method that red contours line marks.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
It is the structure chart of information presentation device provided in an embodiment of the present invention referring to Fig. 7, Fig. 7.The information presentation device Applied to Image Acquisition end.As shown in fig. 7, the information presentation device 700 includes:
Acquisition module 701, for acquiring the live video of live scene;First processing module 702, for the live streaming Image is encoded, and live stream is obtained;Second processing module 703, for target thing to have occurred in determining the live scene In the case where part, the instruction information of the object event is added in the live stream, obtains target live stream;Sending module 704, for sending the target live stream to the client transmission target live stream, or to server-side, so that the clothes End be engaged in the client transmission target live stream, so that the client is broadcast live according to the target live stream When, the object event is prompted according to the instruction information.
Optionally, stating object event includes: score change events;As shown in figure 8, the Second processing module 703 includes:
First acquisition submodule 7031, for acquiring in the live scene than distributional image;First analysis submodule 7032, for being analyzed than distributional image described;First determines submodule 7033, for determining based on the analysis results In the case that variation has occurred in score, the score change events are had occurred in determination;First acquisition submodule 7034, in institute The the first predetermined position addition first identifier for stating live stream, obtains the target live stream;The first identifier is used to indicate hair The score change events are given birth to.
Optionally, the object event includes: penalty event;As shown in figure 9, the Second processing module 703 includes: Two acquisition submodules 7035, acquire the image judged in the live scene;Detection sub-module 7036, for detecting the judge Image in whether there is the image of penalty apparatus;Second determines submodule 7037, for determining the penalty apparatus occur Image in the case where, determination the penalty event has occurred;Second acquisition submodule 7038, for the of the live stream Third mark is added in one predetermined position, obtains the target live stream;The third mark, which is used to indicate, has occurred the penalty Event.
Optionally, the object event includes: target person occur;As shown in Figure 10, the Second processing module 703 is wrapped It includes: identification submodule 7039, for carrying out recognition of face to the live video;Submodule 70310 is compared, for that will recognize Facial image and preset facial image be compared;Third determines submodule 70311, predetermined for meeting in comparison result In the case where condition, it is determined to be the target person;Third acquisition submodule 70312, for the of the live stream Addition the 4th mark in one predetermined position obtains the target live stream, and the 4th mark, which is used to indicate, the target has occurred Personage.
Optionally, on the basis of shown in Fig. 10, the Second processing module further include: the 4th determines submodule, is used for Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;5th determines submodule Block, for determining the first parameter according to the normalized coordinate;Submodule is added, for the second pre-determined bit in the live stream Addition first parameter is set, first parameter indicates opposite position of the central point of the facial contour in live streaming picture It sets.
The working principle of the device of that embodiment of the invention can refer to the description of preceding method embodiment.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
It is the structure chart of information presentation device provided in an embodiment of the present invention referring to Figure 11, Figure 11.The information alert dress It sets and is applied to server-side.As shown in figure 11, the information presentation device 1100 includes:
Receiving module 1101, for receiving the target live stream of Image Acquisition end transmission, wherein in the target live stream Instruction information including object event;Sending module 1102, for sending the target live stream to client, so that the visitor Family end prompts the object event according to the instruction information when being broadcast live according to the target live stream.
Optionally, in the case where the object event does not include target person occur, as shown in figure 12, described device is also Include: to obtain module 1103, for being decoded to the target live stream, obtains live video;Identification module 1104, is used for Recognition of face is carried out to the live video;First determining module 1105, for the result according to recognition of face, it is determined whether go out Target person is showed;First adding module 1106, in the case where being determined to be the target person, in the live streaming The 4th mark of the first predetermined position addition of stream, the 4th mark, which is used to indicate, has there is the target person.
Optionally, shown in Figure 12 on the basis of, described device may also include that the second determining module, for determining face Normalized coordinate of the central point of profile in the human body image frame where the target person;Third determining module is used for root The first parameter is determined according to the normalized coordinate;Second adding module, for being added in the second predetermined position of the live stream First parameter, first parameter indicate relative position of the central point of the facial contour in live streaming picture.
The working principle of the device of that embodiment of the invention can refer to the description of preceding method embodiment.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
It is the structure chart of information presentation device provided in an embodiment of the present invention referring to Figure 13, Figure 13.The information alert dress It sets and is applied to client.As shown in figure 13, the information presentation device 1300 includes:
Receiving module 1301, for receiving the target live stream at Image Acquisition end or server-side transmission, wherein the mesh Mark the instruction information in live stream including object event;Cue module 1302, for being carried out directly according to the target live stream Sowing time prompts the object event according to the instruction information.
Optionally, as shown in figure 14, the cue module 1302 includes:
First acquisition submodule 13021, the prompt for obtaining user select information;Second acquisition submodule 13022 is used In in the case where prompt selection information and the instruction information matches, the preselected prompting mode of user is obtained;Prompt Submodule 13023 prompts the object event for utilizing the prompting mode.
Optionally, as shown in figure 15, the prompting submodule 13023 comprises determining that unit 130231, in the finger Show that target person occurs in information instruction, and when the prompting mode includes identifying the target person, determines the mesh Mark the human face region of personage;Unit 130232 is identified, for identifying the face wheel of the target person in the human face region It is wide.
Wherein, the determination unit includes: the first acquisition subelement, for obtaining in the second pre- of the target live stream Positioning set including the first parameter;Second obtains subelement, for obtaining in the facial contour according to first parameter Normalized coordinate of the heart point in the human body image frame where the target person;Third obtains subelement, for according to Normalized coordinate obtains the second parameter;Subelement is divided, for the human body image frame to be divided at least one region, each Region has corresponding matrix;Subelement is determined, for traversing the matrix, in objective matrix, if second parameter pair The element value answered is target value, then the corresponding target area of the objective matrix is human face region.
The mark unit comprises determining that subelement, for determining the corresponding picture of facial contour in the human face region Vegetarian refreshments;Subelement is identified, for highlighting the target pixel points.
The working principle of the device of that embodiment of the invention can refer to the description of preceding method embodiment.
From the above, it can be seen that in embodiments of the present invention, using Image Acquisition end to the object event in live scene into Line identifier.Therefore, after client receives the live stream that server-side is sent, when being broadcast live according to the target live stream, The object event can be prompted according to the instruction information.Therefore, using the scheme of the embodiment of the present invention, family can be used The event occurred in live scene is timely understood, to improve the timeliness of information alert.
As shown in figure 16, the information alert equipment of the embodiment of the present invention is applied to Image Acquisition end, comprising: processor 1600, for reading the program in memory 1620, execute following process:
Acquire the live video of live scene;The live video is encoded, live stream is obtained;It is described straight determining It broadcasts in the case where object event has occurred in scene, the instruction information of the object event is added in the live stream, is obtained Target live stream;The target live stream is sent to client by transceiver 1610, or sends the target to server-side Live stream, so that the server-side sends the target live stream to client, so that the client is according to When target live stream is broadcast live, the object event is prompted according to the instruction information.
Transceiver 1610, for sending and receiving data under the control of processor 1600.
Wherein, in Figure 16, bus architecture may include the bus and bridge of any number of interconnection, specifically by processor The various circuits for the memory that 1600 one or more processors represented and memory 1620 represent link together.Total coil holder Structure can also link together various other circuits of such as peripheral equipment, voltage-stablizer and management circuit or the like, this It is all a bit it is known in the art, therefore, it will not be further described herein.Bus interface provides interface.Transceiver 1610 can be multiple element, that is, include transmitter and transceiver, provide for logical with various other devices over a transmission medium The unit of letter.Processor 1600 is responsible for management bus architecture and common processing, memory 1620 can store processor 1600 The used data when executing operation.
Processor 1600, which is responsible for management bus architecture and common processing, memory 1620, can store processor 1600 and exists Execute used data when operation.
The object event includes: score change events;Processor 1600 is also used to read the computer program, executes Following steps:
It acquires in the live scene than distributional image;
It is analyzed than distributional image described;
In the case where determining that variation has occurred in score based on the analysis results, the score change events are had occurred in determination;
First identifier is added in the first predetermined position of the live stream, obtains the target live stream;First mark Knowledge, which is used to indicate, has occurred the score change events.
The object event includes: penalty shot event;Processor 1600 is also used to read the computer program, executes as follows Step:
Acquire the panoramic picture of the live scene;
Detect position of the official ball in the panoramic picture;
In the case where determining that the position of the official ball in the predetermined time does not change, it is determined that penalized described in generation Ball event;
Second identifier is added in the first predetermined position of the live stream, obtains the target live stream;Second mark Knowledge, which is used to indicate, has occurred the penalty shot event.
The object event includes: penalty event;Processor 1600 is also used to read the computer program, executes as follows Step:
Acquire the image judged in the live scene;
The image for whether occurring penalty apparatus in the image of the judge detected;
In the case where determining the image for the penalty apparatus occur, the penalty event is had occurred in determination;
In the first predetermined position of live stream addition third mark, the target live stream is obtained;The third mark Knowledge, which is used to indicate, has occurred the penalty event.
The object event includes: target person occur;Processor 1600 is also used to read the computer program, executes Following steps:
Recognition of face is carried out to the live video;
The facial image recognized and preset facial image are compared;
In the case where comparison result meets predetermined condition, it is determined to be the target person;
The 4th mark of addition in the first predetermined position of the live stream obtains the target live stream, the 4th mark Knowledge, which is used to indicate, has there is the target person.
Processor 1600 is also used to read the computer program, executes following steps:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
As shown in figure 17, the information alert equipment of the embodiment of the present invention is applied to server-side, comprising: processor 1700 is used In reading the program in memory 1720, following process is executed:
The target live stream that Image Acquisition end is sent is received by transceiver 1710, wherein is wrapped in the target live stream Include the instruction information of object event;The target live stream is sent to client, so that the client is according to the target When live stream is broadcast live, the object event is prompted according to the instruction information.
Transceiver 1710, for sending and receiving data under the control of processor 1700.
Wherein, in Figure 17, bus architecture may include the bus and bridge of any number of interconnection, specifically by processor The various circuits for the memory that 1700 one or more processors represented and memory 1720 represent link together.Total coil holder Structure can also link together various other circuits of such as peripheral equipment, voltage-stablizer and management circuit or the like, this It is all a bit it is known in the art, therefore, it will not be further described herein.Bus interface provides interface.Transceiver 1710 can be multiple element, that is, include transmitter and transceiver, provide for logical with various other devices over a transmission medium The unit of letter.Processor 1700 is responsible for management bus architecture and common processing, memory 1720 can store processor 1700 The used data when executing operation.
Processor 1700, which is responsible for management bus architecture and common processing, memory 1720, can store processor 1700 and exists Execute used data when operation.
In the case where the object event does not include target person occur, processor 1700 is also used to read the calculating Machine program executes following steps:
The target live stream is decoded, live video is obtained;
Recognition of face is carried out to the live video;
According to recognition of face as a result, having determined whether target person;
In the case where being determined to be the target person, the 4th mark of addition in the first predetermined position of the live stream Know, the 4th mark, which is used to indicate, the target person has occurred.
Processor 1700 is also used to read the computer program, executes following steps:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
Processor 1700 is also used to read the computer program, executes following steps:
As shown in figure 18, the information alert equipment of the embodiment of the present invention is applied to client, comprising: processor 1800 is used In reading the program in memory 1820, following process is executed:
The target live stream at Image Acquisition end or server-side transmission is received by transceiver 1810, wherein the target It include the instruction information of object event in live stream;When being broadcast live according to the target live stream, believed according to the instruction Breath prompts the object event.
Transceiver 1810, for sending and receiving data under the control of processor 1800.
Wherein, in Figure 18, bus architecture may include the bus and bridge of any number of interconnection, specifically by processor The various circuits for the memory that 1800 one or more processors represented and memory 1820 represent link together.Total coil holder Structure can also link together various other circuits of such as peripheral equipment, voltage-stablizer and management circuit or the like, this It is all a bit it is known in the art, therefore, it will not be further described herein.Bus interface provides interface.Transceiver 1810 can be multiple element, that is, include transmitter and receiver, provide for logical with various other devices over a transmission medium The unit of letter.For different user equipmenies, user interface 1830, which can also be, the interface for needing equipment external is inscribed, even The equipment connect includes but is not limited to keypad, display, loudspeaker, microphone, control stick etc..
Processor 1800, which is responsible for management bus architecture and common processing, memory 1820, can store processor 1800 and exists Execute used data when operation.
There is target person in instruction information instruction, and includes identifying the target person in the prompting mode When object, processor 1800 is also used to read the computer program, executes following steps:
The prompt for obtaining user selects information;
In the case where prompt selection information and the instruction information matches, the preselected prompt side of user is obtained Formula;
Using the prompting mode, the object event is prompted.
Processor 1800 is also used to read the computer program, executes following steps:
Determine the human face region of the target person;
In the human face region, the facial contour of the target person is identified.
Processor 1800 is also used to read the computer program, executes following steps:
Obtain the first parameter for including in the second predetermined position of the target live stream;
According to first parameter, the central point of the facial contour is obtained in the human body image where the target person Normalized coordinate in frame;
The second parameter is obtained according to the normalized coordinate;
The human body image frame is divided at least one region, each region has corresponding matrix;
The matrix is traversed, in objective matrix, if the corresponding element value of second parameter is target value, the mesh Marking the corresponding target area of matrix is human face region.
Processor 1800 is also used to read the computer program, executes following steps:
In the human face region, the corresponding pixel of facial contour is determined;
Highlight the target pixel points.
In addition, the computer readable storage medium of the embodiment of the present invention, for storing computer program, the computer journey Sequence can be executed by processor and perform the steps of
Acquire the live video of live scene;
The live video is encoded, live stream is obtained;
In the case where object event has occurred in determining the live scene, the target is added in the live stream The instruction information of event, obtains target live stream;
The target live stream is sent to client, or sends the target live stream to server-side, so that the clothes End be engaged in the client transmission target live stream, so that the client is broadcast live according to the target live stream When, the object event is prompted according to the instruction information.
Wherein, the object event includes: score change events;It is described that the object event is added in the live stream Instruction information, obtain target live stream, comprising:
It acquires in the live scene than distributional image;
It is analyzed than distributional image described;
In the case where determining that variation has occurred in score based on the analysis results, the score change events are had occurred in determination;
First identifier is added in the first predetermined position of the live stream, obtains the target live stream;First mark Knowledge, which is used to indicate, has occurred the score change events.
Wherein, the object event includes: penalty shot event;The finger that the object event is added in the live stream Show information, obtain target live stream, comprising:
Acquire the panoramic picture of the live scene;
Detect position of the official ball in the panoramic picture;
In the case where determining that the position of the official ball in the predetermined time does not change, it is determined that penalized described in generation Ball event;
Second identifier is added in the first predetermined position of the live stream, obtains the target live stream;Second mark Knowledge, which is used to indicate, has occurred the penalty shot event.
Wherein, the object event includes: penalty event;The finger that the object event is added in the live stream Show information, obtain target live stream, comprising:
Acquire the image judged in the live scene;
The image for whether occurring penalty apparatus in the image of the judge detected;
In the case where determining the image for the penalty apparatus occur, the penalty event is had occurred in determination;
In the first predetermined position of live stream addition third mark, the target live stream is obtained;The third mark Knowledge, which is used to indicate, has occurred the penalty event.
Wherein, the object event includes: target person occur;It is described that the object event is added in the live stream Instruction information, obtain target live stream, comprising:
Recognition of face is carried out to the live video;
The facial image recognized and preset facial image are compared;
In the case where comparison result meets predetermined condition, it is determined to be the target person;
The 4th mark of addition in the first predetermined position of the live stream obtains the target live stream, the 4th mark Knowledge, which is used to indicate, has there is the target person.
Wherein, it is determined to be after the target person described in the case where comparison result meets predetermined condition, The method also includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
In addition, the computer readable storage medium of the embodiment of the present invention, for storing computer program, the computer journey Sequence can be executed by processor and perform the steps of
Receive the target live stream that Image Acquisition end is sent, wherein include the finger of object event in the target live stream Show information;
The target live stream is sent to client, so that the client is broadcast live according to the target live stream When, the object event is prompted according to the instruction information.
Wherein, in the case where the object event does not include target person occur, described to described in client transmission Before target live stream, the method also includes:
The target live stream is decoded, live video is obtained;
Recognition of face is carried out to the live video;
According to recognition of face as a result, having determined whether target person;
In the case where being determined to be the target person, the 4th mark of addition in the first predetermined position of the live stream Know, the 4th mark, which is used to indicate, the target person has occurred.
Wherein, described according to recognition of face as a result, having determined whether target person after, the method is also wrapped It includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the face wheel Relative position of the wide central point in live streaming picture.
In addition, the computer readable storage medium of the embodiment of the present invention, for storing computer program, the computer journey Sequence can be executed by processor and perform the steps of
Receive the target live stream at Image Acquisition end or server-side transmission, wherein include mesh in the target live stream The instruction information of mark event;
When being broadcast live according to the target live stream, the object event is mentioned according to the instruction information Show.
It is wherein, described that the object event is prompted according to the instruction information, comprising:
The prompt for obtaining user selects information;
In the case where prompt selection information and the instruction information matches, the preselected prompt side of user is obtained Formula;
Using the prompting mode, the object event is prompted.
Wherein, there is target person in instruction information instruction, and described including identifying in the prompting mode It is described to utilize the prompting mode when target person, the object event is prompted, comprising:
Determine the human face region of the target person;
In the human face region, the facial contour of the target person is identified.
Wherein, the human face region of the determination target person, comprising:
Obtain the first parameter for including in the second predetermined position of the target live stream;
According to first parameter, the central point of the facial contour is obtained in the human body image where the target person Normalized coordinate in frame;
The second parameter is obtained according to the normalized coordinate;
The human body image frame is divided at least one region, each region has corresponding matrix;
The matrix is traversed, in objective matrix, if the corresponding element value of second parameter is target value, the mesh Marking the corresponding target area of matrix is human face region.
Wherein, described in the human face region, identify the facial contour of the target person, comprising:
In the human face region, the corresponding pixel of facial contour is determined;
Highlight the target pixel points.
In several embodiments provided herein, it should be understood that disclosed method and apparatus, it can be by other Mode realize.For example, the apparatus embodiments described above are merely exemplary, for example, the division of the unit, only For a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components can combine Or it is desirably integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed phase Coupling, direct-coupling or communication connection between mutually can be through some interfaces, the INDIRECT COUPLING or communication of device or unit Connection can be electrical property, mechanical or other forms.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit It is that the independent physics of each unit includes, can also be integrated in one unit with two or more units.Above-mentioned integrated list Member both can take the form of hardware realization, can also realize in the form of hardware adds SFU software functional unit.
The above-mentioned integrated unit being realized in the form of SFU software functional unit can store and computer-readable deposit at one In storage media.Above-mentioned SFU software functional unit is stored in a storage medium, including some instructions are used so that a computer Equipment (can be personal computer, server or the network equipment etc.) executes receiving/transmission method described in each embodiment of the present invention Part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (Read-Only Memory, abbreviation ROM), random access memory (Random Access Memory, abbreviation RAM), magnetic or disk etc. are various can store The medium of program code.
The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art For, without departing from the principles of the present invention, it can also make several improvements and retouch, these improvements and modifications It should be regarded as protection scope of the present invention.

Claims (18)

1. a kind of information cuing method is applied to Image Acquisition end characterized by comprising
Acquire the live video of live scene;
The live video is encoded, live stream is obtained;
In the case where object event has occurred in determining the live scene, the object event is added in the live stream Instruction information, obtain target live stream;
The target live stream is sent to client, or sends the target live stream to server-side, so that the server-side The target live stream is sent to client, so that the client is according to the target live stream when being broadcast live, The object event is prompted according to the instruction information.
2. the method according to claim 1, wherein the object event includes: score change events;It is described The instruction information that the object event is added in the live stream, obtains target live stream, comprising:
It acquires in the live scene than distributional image;
It is analyzed than distributional image described;
In the case where determining that variation has occurred in score based on the analysis results, the score change events are had occurred in determination;
First identifier is added in the first predetermined position of the live stream, obtains the target live stream;The first identifier is used The score change events have occurred in instruction.
3. the method according to claim 1, wherein the object event includes: penalty shot event;It is described described The instruction information that the object event is added in live stream, obtains target live stream, comprising:
Acquire the panoramic picture of the live scene;
Detect position of the official ball in the panoramic picture;
In the case where determining that the position of the official ball in the predetermined time does not change, it is determined that the penalty shot thing occurs Part;
Second identifier is added in the first predetermined position of the live stream, obtains the target live stream;The second identifier is used The penalty shot event has occurred in instruction.
4. the method according to claim 1, wherein the object event includes: penalty event;It is described described The instruction information that the object event is added in live stream, obtains target live stream, comprising:
Acquire the image judged in the live scene;
The image for whether occurring penalty apparatus in the image of the judge detected;
In the case where determining the image for the penalty apparatus occur, the penalty event is had occurred in determination;
In the first predetermined position of live stream addition third mark, the target live stream is obtained;The third mark is used The penalty event has occurred in instruction.
5. the method according to claim 1, wherein the object event includes: target person occur;It is described The instruction information that the object event is added in the live stream, obtains target live stream, comprising:
Recognition of face is carried out to the live video;
The facial image recognized and preset facial image are compared;
In the case where comparison result meets predetermined condition, it is determined to be the target person;
The 4th mark of addition in the first predetermined position of the live stream, obtains the target live stream, and the 4th mark is used There is the target person in instruction.
6. according to the method described in claim 5, it is characterized in that, described the case where comparison result meets predetermined condition Under, it is determined to be after the target person, the method also includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the facial contour Relative position of the central point in live streaming picture.
7. a kind of information cuing method is applied to server-side characterized by comprising
Receive the target live stream that Image Acquisition end is sent, wherein the instruction in the target live stream including object event is believed Breath;
The target live stream is sent to client, so that the client is according to the target live stream when being broadcast live, The object event is prompted according to the instruction information.
8. the method according to the description of claim 7 is characterized in that not including the feelings for target person occur in the object event Under condition, before the target live stream to client transmission, the method also includes:
The target live stream is decoded, live video is obtained;
Recognition of face is carried out to the live video;
According to recognition of face as a result, having determined whether target person;
In the case where being determined to be the target person, the 4th mark of addition in the first predetermined position of the live stream, 4th mark, which is used to indicate, has there is the target person.
9. according to the method described in claim 8, it is characterized in that, in the result according to recognition of face, it is determined whether go out After having showed target person, the method also includes:
Determine normalized coordinate of the central point of facial contour in the human body image frame where the target person;
The first parameter is determined according to the normalized coordinate;
First parameter is added in the second predetermined position of the live stream, first parameter indicates the facial contour Relative position of the central point in live streaming picture.
10. a kind of information cuing method is applied to client characterized by comprising
Receive the target live stream at Image Acquisition end or server-side transmission, wherein include target thing in the target live stream The instruction information of part;
When being broadcast live according to the target live stream, the object event is prompted according to the instruction information.
11. according to the method described in claim 10, it is characterized in that, it is described according to the instruction information to the object event It is prompted, comprising:
The prompt for obtaining user selects information;
In the case where prompt selection information and the instruction information matches, the preselected prompting mode of user is obtained;
Using the prompting mode, the object event is prompted.
12. according to the method for claim 11, which is characterized in that there is target person in instruction information instruction, It is described to utilize the prompting mode and when the prompting mode includes identifying the target person, to the object event It is prompted, comprising:
Determine the human face region of the target person;
In the human face region, the facial contour of the target person is identified.
13. according to the method for claim 12, which is characterized in that the human face region of the determination target person, packet It includes:
Obtain the first parameter for including in the second predetermined position of the target live stream;
According to first parameter, the central point of the facial contour is obtained in the human body image frame where the target person Normalized coordinate;
The second parameter is obtained according to the normalized coordinate;
The human body image frame is divided at least one region, each region has corresponding matrix;
The matrix is traversed, in objective matrix, if the corresponding element value of second parameter is target value, the target square The corresponding target area of battle array is human face region.
14. according to the method for claim 12, which is characterized in that it is described in the human face region, identify the target The facial contour of personage, comprising:
In the human face region, the corresponding pixel of facial contour is determined;
Highlight the target pixel points.
15. a kind of information alert equipment, comprising: transceiver, memory, processor and be stored on the memory and can be in institute State the computer program run on processor;It is characterized in that,
The processor is realized for reading the program in memory as in method described in any one of claims 1 to 6 Step;Or step of the realization in the method as described in any one of claim 7 to 9;Or realize as claim 10 to Step in method described in any one of 14.
16. a kind of computer readable storage medium, for storing computer program, which is characterized in that the computer program quilt It realizes when processor executes such as the step in method described in any one of claims 1 to 6;Or realize such as claim 7 Step into method described in any one of 9;Or realization is in the method as described in any one of claim 10 to 14 Step.
17. a kind of information presentation system characterized by comprising Image Acquisition end and at least one client;
Described image collection terminal encodes the live video for acquiring the live video of live scene, obtains live streaming Stream;In the case where object event has occurred in determining the live scene, the object event is added in the live stream Instruction information, obtain target live stream, and send the target live stream to the client;
The client, for receiving the target live stream of described image collection terminal transmission, and according to the target live stream When being broadcast live, the object event is prompted according to the instruction information.
18. a kind of information presentation system characterized by comprising Image Acquisition end, server-side and at least one client;
Described image collection terminal encodes the live video for acquiring the live video of live scene, obtains live streaming Stream;In the case where object event has occurred in determining the live scene, the object event is added in the live stream Instruction information, obtain target live stream, and send the target live stream to the server-side;
The server-side, for sending the target live stream to the client;
The client, the target live stream sent for receiving the server-side, and carried out according to the target live stream When live streaming, the object event is prompted according to the instruction information.
CN201910639977.0A 2019-07-16 2019-07-16 Information prompting method, equipment, system and computer readable storage medium Active CN110418150B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910639977.0A CN110418150B (en) 2019-07-16 2019-07-16 Information prompting method, equipment, system and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910639977.0A CN110418150B (en) 2019-07-16 2019-07-16 Information prompting method, equipment, system and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN110418150A true CN110418150A (en) 2019-11-05
CN110418150B CN110418150B (en) 2022-07-01

Family

ID=68361648

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910639977.0A Active CN110418150B (en) 2019-07-16 2019-07-16 Information prompting method, equipment, system and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN110418150B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112312156A (en) * 2020-11-06 2021-02-02 云南腾云信息产业有限公司 Live broadcast scene reminding method, device, equipment and storage medium
WO2022121871A1 (en) * 2020-12-08 2022-06-16 北京字节跳动网络技术有限公司 Information display and processing methods and apparatuses, device, and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102595191A (en) * 2012-02-24 2012-07-18 央视国际网络有限公司 Method and device for searching sport events in sport event videos
CN106254927A (en) * 2016-08-15 2016-12-21 网易乐得科技有限公司 A kind of information processing method and device
WO2017200849A1 (en) * 2016-05-19 2017-11-23 Scenera, Inc. Scene marking
CN109040773A (en) * 2018-07-10 2018-12-18 武汉斗鱼网络科技有限公司 A kind of video improvement method, apparatus, equipment and medium
US10198819B2 (en) * 2015-11-30 2019-02-05 Snap Inc. Image segmentation and modification of a video stream
CN110012348A (en) * 2019-06-04 2019-07-12 成都索贝数码科技股份有限公司 A kind of automatic collection of choice specimens system and method for race program

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102595191A (en) * 2012-02-24 2012-07-18 央视国际网络有限公司 Method and device for searching sport events in sport event videos
US10198819B2 (en) * 2015-11-30 2019-02-05 Snap Inc. Image segmentation and modification of a video stream
WO2017200849A1 (en) * 2016-05-19 2017-11-23 Scenera, Inc. Scene marking
CN106254927A (en) * 2016-08-15 2016-12-21 网易乐得科技有限公司 A kind of information processing method and device
CN109040773A (en) * 2018-07-10 2018-12-18 武汉斗鱼网络科技有限公司 A kind of video improvement method, apparatus, equipment and medium
CN110012348A (en) * 2019-06-04 2019-07-12 成都索贝数码科技股份有限公司 A kind of automatic collection of choice specimens system and method for race program

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112312156A (en) * 2020-11-06 2021-02-02 云南腾云信息产业有限公司 Live broadcast scene reminding method, device, equipment and storage medium
WO2022121871A1 (en) * 2020-12-08 2022-06-16 北京字节跳动网络技术有限公司 Information display and processing methods and apparatuses, device, and medium
US11962866B2 (en) 2020-12-08 2024-04-16 Beijing Bytedance Network Technology Co., Ltd. Information display and processing methods and apparatuses, device, and medium

Also Published As

Publication number Publication date
CN110418150B (en) 2022-07-01

Similar Documents

Publication Publication Date Title
CN106228628B (en) Check-in system, method and device based on face recognition
CN104618803B (en) Information-pushing method, device, terminal and server
CN110298231B (en) Method and system for judging goal of basketball game video
CN110392274B (en) Information processing method, equipment, client, system and storage medium
CN107124662B (en) Video live broadcast method and device, electronic equipment and computer readable storage medium
CN107454437B (en) Video annotation method and device and server
Yu et al. Current and emerging topics in sports video processing
CN103581705A (en) Method and system for recognizing video program
CN112312231B (en) Video image coding method and device, electronic equipment and medium
CN1750618A (en) Method of viewing audiovisual documents on a receiver, and receiver for viewing such documents
CN202998337U (en) Video program identification system
JP4577774B2 (en) Sports video classification device and log generation device
CN109299326B (en) Video recommendation method, device and system, electronic equipment and storage medium
EP1938208A1 (en) Face annotation in streaming video
CN112581627A (en) System and apparatus for user-controlled virtual camera for volumetric video
CN109729390A (en) A kind of IPTV program monitoring method, apparatus and system
CN110443115A (en) Face identification method, device, computer equipment and readable storage medium storing program for executing
CN110418150A (en) Information prompting method, equipment, system and computer readable storage medium
CN111444822B (en) Object recognition method and device, storage medium and electronic device
CN105872537A (en) Video playing method, device and system
CN109472230B (en) Automatic athlete shooting recommendation system and method based on pedestrian detection and Internet
CN108881119B (en) Method, device and system for video concentration
CN110287934B (en) Object detection method and device, client and server
Nieto et al. An automatic system for sports analytics in multi-camera tennis videos
JP2003143546A (en) Method for processing football video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant