WO2019000793A1 - 直播中打码方法及装置、电子设备及存储介质 - Google Patents
直播中打码方法及装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2019000793A1 WO2019000793A1 PCT/CN2017/111241 CN2017111241W WO2019000793A1 WO 2019000793 A1 WO2019000793 A1 WO 2019000793A1 CN 2017111241 W CN2017111241 W CN 2017111241W WO 2019000793 A1 WO2019000793 A1 WO 2019000793A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- area
- coded
- video image
- next frame
- image
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 239000013598 vector Substances 0.000 claims abstract description 84
- 230000004044 response Effects 0.000 claims abstract description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 11
- 238000000605 extraction Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 9
- 239000003086 colorant Substances 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20182—Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
Definitions
- the present invention relates to the field of live broadcast technologies, and in particular, to a method and device for encoding in a live broadcast, an electronic device, and a storage medium.
- the object of the present invention is to provide a method for coding in a live broadcast, so as to realize real-time coding during the live broadcast process to block content that the user does not want to be broadcasted.
- Another object of the present invention is to provide a live recording device for real-time coding during live broadcast to shield content that the user does not want to be broadcasted.
- Another object of the present invention is to provide an electronic device for real-time coding during a live broadcast to shield content that the user does not want to be broadcasted.
- Another object of the present invention is to provide a storage medium to realize real-time coding during a live broadcast to shield content that the user does not want to be broadcasted.
- an embodiment of the present invention provides a method for coding in a live broadcast, which is applied to an electronic device, and the method includes:
- the area of the first frame of the video image to be coded is matched with the next frame of the video image according to the first feature vector, and the area with the highest degree of matching with the area to be coded in the next frame of the video image is selected as the area image;
- the area image is coded to achieve coding in the live broadcast.
- the embodiment of the present invention further provides a coding device in a live broadcast, which is applied to an electronic device, where the device includes a determining module, an extracting module, a matching module, and a coding module, where
- a determining module configured to determine an area to be coded in the video image of the first frame in response to a sliding operation of the user
- An extraction module configured to extract a color feature and a texture feature of the area to be coded, to generate a first feature vector
- a matching module configured to, according to the first feature vector, match the area to be coded in the first frame of the video image to the next frame of the video image, and select the region with the highest matching degree in the video frame of the next frame to be coded.
- the area is an area image
- the coding module is used for coding the area image to achieve coding in the live broadcast.
- an embodiment of the present invention provides an electronic device, where the electronic device includes:
- the device is installed in the memory, and the device is installed in the memory and includes one or more software function modules executed by the processor, and the in-live recording device includes:
- a determining module configured to determine an area to be coded in the video image of the first frame in response to a sliding operation of the user
- An extraction module configured to extract color features and texture features of the code-required area, and generate a first feature vector
- a matching module configured to sequentially match the area to be coded in the first frame video image with the next frame video image according to the first feature vector, and select the next frame video image and the required
- the area with the highest matching degree of the coded area is the area image
- the coding module is configured to code the image of the area to perform coding in the live broadcast.
- an embodiment of the present invention further provides a storage medium, where the computer instruction is stored in the storage medium, wherein the computer instruction performs the above-mentioned live recording method when being read and executed.
- An electronic device and a live recording method, device, and storage medium are provided in an embodiment of the present invention.
- the live coding method, device, and storage medium are applied to an electronic device, and the live coding method includes a sliding operation in response to the user. Determining an area to be coded in the video image of the first frame, extracting color features and texture features of the area to be coded, and generating a first feature vector. And the area of the first frame of the video image to be coded is matched with the next frame of the video image according to the first feature vector, and the area of the next frame of the video image that matches the area to be coded is selected to be the highest.
- the area is an area image, and the area image is coded to achieve coding in the live broadcast. Through the program to achieve the live broadcast process The user does not want to take pictures in real time to improve the user experience.
- FIG. 1 is a block diagram showing the structure of an electronic device according to a preferred embodiment of the present invention.
- FIG. 2 is a schematic flowchart diagram of a method for coding in a live broadcast according to an embodiment of the present invention.
- FIG. 3 is a schematic flowchart diagram of sub-steps of a coding method in a live broadcast according to an embodiment of the present invention.
- FIG. 4 is a schematic flow chart showing the sub-steps of step S103 provided by the embodiment of the present invention.
- FIG. 5 is a schematic flowchart diagram of another method for coding in a live broadcast according to an embodiment of the present invention.
- FIG. 6 is a schematic flowchart diagram of another sub-step of a coding method in a live broadcast according to an embodiment of the present invention.
- FIG. 7 is a schematic diagram of functional modules of a coding device in a live broadcast according to an embodiment of the present invention.
- FIG. 8 is a schematic diagram of a submodule of a matching module according to an embodiment of the present invention.
- the method, device and storage medium for recording in a live broadcast provided by the embodiments of the present invention are all applied to an electronic device, and the electronic device may be, but not limited to, an electronic device such as a desktop computer, a tablet computer, or a mobile phone.
- FIG. 1 is a block diagram showing the structure of an electronic device 200 that can be applied to an embodiment of the present invention.
- the electronic device 200 includes a live recording device 100, a memory 102, a memory controller 103, one or more (only one shown) processor 104, a peripheral interface 105, a radio frequency module 106, The audio module 107, the touch screen 108, and the like.
- the in-live recording device 100 includes at least one software function module that can be stored in the memory 102 or in an operating system (OS) of the electronic device 200 in the form of software or firmware. .
- OS operating system
- the memory 102 can be used to store software programs and modules, such as program instructions/modules corresponding to the in-live recording device and method in the embodiment of the present invention.
- the processor 104 executes by executing a software program and a module stored in the memory 102.
- Memory 102 can include high speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. Access to the memory 102 by the processor 104 and other possible components can be performed under the control of the memory controller 103.
- non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. Access to the memory 102 by the processor 104 and other possible components can be performed under the control of the memory controller 103.
- Peripheral interface 105 couples various input/output devices to processor 104 and memory 102.
- peripheral interface 105, processor 104, and memory controller 103 can be implemented in a single chip. In other instances, they can be implemented by separate chips.
- the radio frequency module 106 is configured to receive and transmit electromagnetic waves, and realize mutual conversion between electromagnetic waves and electric signals, thereby communicating with a communication network or other devices.
- the audio module 107 provides an audio interface to the user, which may include one or more microphones, one or more speakers, and audio circuitry.
- the touch screen 108 provides an output and input interface simultaneously between the electronic device 200 and the user.
- the touch screen 108 supports single-point and multi-touch operations.
- the touch screen 108 can be a capacitive touch screen or resistive touch that supports single-point and multi-touch operations. Control screen and so on. Supporting single-point and multi-touch operations is that the touch screen 108 can sense the simultaneous touch operation from one or more positions on the touch screen, and hand over the sensed multi-touch operation to the touch operation.
- the processor 104 performs processing.
- FIG. 1 is merely illustrative, and the electronic device 200 may further include more or less components than those shown in FIG. 1, or have a different configuration than that shown in FIG.
- the components shown in Figure 1 can be implemented in hardware, software, or a combination thereof.
- the electronic device and the live recording method, device and storage medium provided by the embodiments of the present invention provide a new live recording method.
- the electronic device and the live coding device and method can be applied to platforms such as an Android operating system, an iOS operating system, and a Windows Phone operating system.
- the live broadcast coded device proposed by the embodiment of the present invention The method can be applied to an application installation client of the electronic device 200, and the client encodes the content that the user who wants to broadcast live does not want to appear, so as to fully protect the privacy of the live broadcast user, thereby reaching the anchor of the anchor platform. purpose.
- FIG. 2 is a schematic flowchart of a coding method in a live broadcast according to an embodiment of the present invention, where the coding method in the live broadcast includes:
- Step S110 determining an area to be coded in the video image of the first frame in response to the sliding operation of the user.
- the user When the user starts live broadcast, you can set the area to be coded according to your needs. Specifically, when the user does not want the advertisement trademark to appear, the user can set a rectangular frame that can include the advertisement trademark by sliding on the screen, and the electronic device responds to the sliding operation of the user, and sets the area corresponding to the rectangular frame to be coded.
- the area of course, the user can also set a circular frame or a regular polygonal frame that can include the advertising trademark by sliding on the screen, which is not limited herein.
- Step S120 extracting color features and texture features of the area to be coded to generate a first feature vector.
- the dimension of the three primary color ratio values (ie, RGB values) of each pixel in the area to be coded is converted from 3D to 11 dimensions.
- the current three primary color ratio values of each pixel can only be characterized. Red, green, and blue colors
- the color of the three primary color ratio values of each pixel is refined to 11 types, such as black, blue, and yellow. , gray, pink, red, white, etc., so that when extracting the color features of the area to be coded, it is possible to adaptively select according to the actual color distribution of the area to be coded.
- the ratio of the three primary colors of each pixel is adaptively reduced from 11 to 2 to extract the significant color features in the area to be coded.
- the texture feature of the region to be coded is extracted by the HOG (Histograms of Oriented Gradients) feature extraction method, and the first feature vector is generated by extracting the color feature and the texture feature.
- HOG Heistograms of Oriented Gradients
- Step S130 The coded area in the first frame of the video image is sequentially matched with the next frame of the video image according to the first feature vector, and the video image in the next frame and the code to be coded are selected.
- the area with the highest area matching is the area image.
- step S130 includes:
- Step S1301 The area to be coded is sequentially slidably matched with the next frame of the video image.
- the area to be coded is matched from the top to the bottom in the next frame video image, and from the left to the right, the area of the next frame video image with the same size as the area to be coded is matched, and the area to be coded is next.
- the length of the sliding in the frame video image can be set according to actual needs. It should be noted that the smaller the length of the sliding, the longer the time is. The longer the sliding length may lead to inaccurate matching.
- Step S1302 sequentially extract color features and texture features in the image of the next frame video image that are consistent with the size of the area to be coded, and generate a plurality of second feature vectors.
- Step S1303 Calculating sequentially according to the first feature vector and the second feature vector by using a standard mean correlation matching algorithm The matching degree between the area of the next frame video image that matches the size of the area to be coded and the area to be coded.
- R(x, y) is the matching degree of the area corresponding to the size of the area to be coded in the video image of the next frame and the area to be coded
- (x, y) is the pixel point in the video image of the next frame.
- Coordinates, (i, j) are the coordinates of the pixel points in the area to be coded
- T is the first feature vector of the area image to be coded
- I is the second feature vector in the next frame video image
- (w , h) is the coordinate formed by the width and height of the area to be coded
- (m, n) is the coordinate used when the area to be coded is traversed.
- Step S1304 The region with the highest matching degree of the plurality of regions in the next frame video image that is consistent with the size of the region to be coded is selected as the region image.
- Step S1304 indicates that the area image is most relevant to the area to be coded, and is an area in the next frame of the video image that needs to be coded.
- step S140 the area image is coded to implement coding in the live broadcast.
- FIG. 2 is a schematic flowchart of a sub-step of step S140 of a method for recording a live broadcast according to an embodiment of the present invention.
- the step S140 includes:
- Step S141 selecting a maximum pixel value and a minimum pixel value of a plurality of pixel points adjacent to each pixel point in the area image.
- a maximum pixel value and a minimum pixel value of a plurality of pixel points in an area adjacent to each pixel point in the area image are selected, and a size of an area adjacent to each pixel point may be set as needed.
- Step S142 setting a pixel value of the pixel point to an average value of the maximum pixel value and the minimum pixel value to implement coding in a live broadcast.
- the coding method in the live broadcast includes:
- Step S210 determining an area to be coded in the video image of the first frame in response to the sliding operation of the user.
- the user When the user starts live broadcast, you can set the area to be coded according to your needs. Specifically, when the user does not want the advertisement trademark to appear, the user can set a rectangular frame that can include the advertisement trademark by sliding on the screen, and the electronic device responds to the sliding operation of the user, and sets the area corresponding to the rectangular frame to be coded. Area.
- Step S220 extracting color features and texture features of the area to be coded to generate a first feature vector.
- the dimension of the three primary color ratio values (ie, RGB values) of each pixel in the area to be coded is converted from 3D to 11 dimensions.
- the current three primary color ratio values of each pixel can only be characterized. Red, green, and blue colors
- the color of the three primary color ratio values of each pixel is refined to 11 types, such as black, blue, and yellow. , gray, pink, red, white, etc., so that when extracting the color features of the area to be coded, it is possible to adaptively select according to the actual color distribution of the area to be coded.
- the ratio of the three primary colors of each pixel is adaptively reduced from 11 to 2 to extract the significant color features in the area to be coded.
- the texture feature of the region to be coded is extracted by the HOG (Histograms of Oriented Gradients) feature extraction method, and the first feature vector is generated by extracting the color feature and the texture feature.
- HOG Heistograms of Oriented Gradients
- Step S230 collecting pixel values of each pixel in the next frame video image.
- Step S240 establishing a first histogram of the next frame video image according to the pixel value of each pixel in the next frame video image.
- the first histogram is established based on the pixel value of each pixel of the next frame of the video image and the number of pixels of the same pixel value.
- Step S250 Collect pixel values of each pixel in the area to be coded.
- Step S260 establishing a second histogram of the area to be coded according to the pixel value of each pixel in the area to be coded.
- a second histogram is established according to the pixel value of each pixel in the area to be coded and the number of pixels of the same pixel value.
- Step S270 obtaining a third histogram by calculating a ratio of the second histogram and the first histogram.
- Step S280 calculating a distribution probability of each pixel in the next frame video image in the third histogram.
- step S290 an area in which the distribution point of the pixel in the next frame video image is greater than the preset threshold in the third histogram is selected as an area that needs to be matched with the area to be coded in sequence.
- the probability that the pixel point in the next frame of the video image is greater than the preset threshold in the third histogram, it indicates that the area where the pixel is located is closer to the color of the area to be coded, and the next frame is selected.
- the area in which the pixel points in the video image are distributed in the third histogram with a probability greater than the preset threshold is an area that needs to be matched with the area to be coded, thereby reducing the amount of calculation of the matching operation.
- Step S310 sequentially, according to the first feature vector, the area to be coded in the first frame video image A pixel in a video image is matched in a region in which the probability of distribution in the third histogram is greater than a preset threshold, and a region in the video image of the next frame that has the highest matching degree with the region to be coded is selected as Area image.
- the area to be coded is sequentially slidably matched with the area in the video image of the next frame in which the probability of distribution in the third histogram is greater than a preset threshold, that is, the area to be coded is from the top to the area in the area.
- a preset threshold that is, the area to be coded is from the top to the area in the area.
- the matching is performed from left to right, and the length of the sliding area of the area to be coded in the area can be set according to actual needs. It should be noted that the smaller the length of the sliding, the longer the time is, the sliding A length that is too long may result in inaccurate matching.
- FIG. 6 is a schematic flowchart of a sub-step of step S310 of another method for recording a live broadcast according to an embodiment of the present invention.
- the step S310 includes:
- Step S311 extracting color features and texture features of the regions in the next frame video image whose distribution probability is greater than a preset threshold in the third histogram, to generate a second feature vector.
- the color feature and the texture feature of the region are extracted to generate a second feature vector.
- Step S312 performing matching operations on the first feature vector and the second feature vector by using a standard mean correlation matching algorithm, and selecting a second feature vector in the next frame video image and the region to be coded.
- the area where the first feature vector has the highest degree of matching is the area image.
- a standard mean correlation matching algorithm is used to sequentially calculate the matching degree between the region of the next frame video image and the size of the area to be coded, and the specific standard mean correlation matching algorithm is :
- R(x, y) is the matching degree of the area corresponding to the size of the area to be coded in the video image of the next frame and the area to be coded
- (x, y) is the pixel point in the video image of the next frame.
- Coordinates, (i, j) are the coordinates of the pixel points in the area to be coded
- T is the first feature vector of the area image to be coded
- I is the second feature vector in the next frame video image
- (w , h) is the coordinate formed by the width and height of the area to be coded
- (m, n) is the coordinate used when the area to be coded is traversed.
- Selecting the region with the highest matching degree of the multiple regions in the next frame video image that is the same size as the region to be coded is the region image, indicating that the region image is most relevant to the region to be coded, and is needed in the next frame video image. Coded area.
- step S320 the area image is coded to achieve coding in the live broadcast.
- FIG. 7 is a schematic diagram of a functional module of a coding device 100 in a live broadcast according to an embodiment of the present invention.
- the coding device 100 includes a determination module 110, an extraction module 120, a first acquisition module 130, and a first setup.
- the determining module 110 is configured to determine an area of the first frame of the video image to be coded in response to the sliding operation of the user.
- step S110 and step S210 may be performed by the determining module 110.
- the extracting module 120 is configured to extract color features and texture features of the area to be coded, and generate a first feature vector.
- step S120 and step S220 may be performed by the extraction module 120.
- the first acquisition module 130 is configured to collect pixel values of each pixel in the next frame of the video image.
- step S230 may be performed by the first collection module 130.
- the first establishing module 140 is configured to establish a first histogram of the next frame video image according to the pixel value of each pixel in the next frame video image.
- step S240 may be performed by the first establishing module 140.
- the second collecting module 150 is configured to collect pixel values of each pixel in the area to be coded.
- step S250 may be performed by the second collection module 150.
- the second establishing module 160 is configured to establish a second histogram of the area to be coded according to a pixel value of each pixel in the area to be coded.
- step S260 may be performed by the second establishing module 160.
- the first calculating module 170 is configured to obtain a third histogram by calculating a ratio of the second histogram and the first histogram.
- step S270 may be performed by the first computing module 170.
- the second calculating module 180 is configured to calculate a distribution probability of each pixel in the next frame video image in the third histogram.
- step S280 can be performed by the second calculation module 180.
- the selection module 190 is configured to select an area in which the probability of the pixel in the next frame video image is greater than the preset threshold in the third histogram is an area that needs to be matched with the area to be coded in sequence.
- step S290 can be performed by the selection module 190.
- the matching module 200 is configured to: according to the first feature vector, the area of the first frame video image to be coded The next time, the video image of the next frame is matched, and the area with the highest matching degree with the area to be coded in the video image of the next frame is selected as the area image.
- steps S130, S310-S312 may be performed by the matching module 200.
- the matching module 200 may include a sliding matching sub-module 211 , a second feature vector generating sub-module 212 , a matching degree calculating sub-module 213 , and an area image selecting sub-module 214 .
- the sliding matching sub-module 211 can be used to perform step 1301
- the second feature vector generating sub-module 212 can be used to perform step 1302
- the matching degree calculating sub-module 213 can be used to perform step 1303, and the region image selecting sub-module 214 can be used.
- Step 1304 is performed.
- the sliding matching sub-module 211 is configured to sequentially perform sliding matching on the area to be coded with the next frame of the video image.
- the sliding matching sub-module 211 is configured to match the area to be coded from top to bottom in the next frame video image, and from the left to the right, and match the area of the next frame video image with the same size as the area to be coded. .
- the second feature vector generation sub-module 212 is configured to sequentially extract color features and texture features in the image of the next frame video image that are consistent with the size of the area to be coded, and generate a plurality of second feature vectors.
- the matching degree calculation sub-module 213 is configured to sequentially calculate, according to the first feature vector and the second feature vector, a matching degree of the area corresponding to the size of the area to be coded and the area to be coded in the video image of the next frame by using a standard mean correlation matching algorithm.
- the matching degree calculation sub-module 213 is configured to use the formula
- the degree of matching between the area corresponding to the size of the area to be coded and the area to be coded in the video image of the next frame is sequentially calculated, where R(x, y) is the same as the size of the area to be coded in the video image of the next frame.
- the matching degree between the area and the area to be coded (x, y) is the coordinates of the pixel points in the video image of the next frame, (i, j) is the coordinates of the pixel points in the area to be coded, and T is required to be played.
- the first feature vector of the region image of the code, I is the second feature vector in the next frame video image, and (w, h) is the width sum of the region to be coded.
- the coordinates formed by the high, (m, n) are the coordinates used for the traversal of the area to be coded.
- the area image selection sub-module 214 is configured to select an area of the next frame video image that has the highest matching degree of the plurality of areas that are in the same size as the area to be coded as the area image.
- the coding module 210 is configured to code the area image to implement coding in the live broadcast.
- steps S140-S142 and step S320 may be performed by the coding module 210.
- an embodiment of the present invention further provides a storage medium, where the computer instruction stores a computer instruction, wherein the computer instruction executes the above-mentioned live recording method when being read and executed. Since it has been described in detail in the live broadcast coding method, it will not be described here.
- the embodiments of the present invention provide a method, a device, and a storage medium for encoding in an electronic device and a live broadcast.
- the coding method, device, and storage medium in the live broadcast are applied to an electronic device, and the coding method in the live broadcast includes a response.
- the sliding operation of the user determines an area to be coded in the video image of the first frame, extracts color features and texture features of the area to be coded, and generates a first feature vector. And the area of the first frame of the video image to be coded is matched with the next frame of the video image according to the first feature vector, and the area of the next frame of the video image that matches the area to be coded is selected to be the highest.
- the area is an area image, and the area image is coded to achieve coding in the live broadcast.
- each block of the flowchart or block diagram can represent a module, a program segment, or a portion of code that includes one or more of the Executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may also occur in a different order than those illustrated in the drawings.
- each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or action. Or it can be implemented by a combination of dedicated hardware and computer instructions.
- each functional module in each embodiment of the present invention may be integrated to form a separate part, or each module may exist separately, or two or more modules may be integrated to form a separate part.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
一种直播中打码方法及装置、电子设备及存储介质，该直播中打码方法及装置应用于电子设备，该直播中打码方法包括响应用户的滑动操作确定第一帧视频图像中的需打码的区域（S210），提取该需打码区域的颜色特征和纹理特征，生成第一特征向量（S220）。根据第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取该下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像（S310），对该区域图像进行打码，以实现直播中打码（S320）。通过该方法及装置实现对直播过程中用户不想出镜的画面实时打码，以提高用户体验感。
Description
相关申请的交叉引用
本申请要求于2017年06月30日提交中国专利局的申请号为2017105204920、名称为“直播中打码方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本发明涉及直播技术领域，具体而言，涉及直播中打码方法及装置、电子设备及存储介质。
随着直播行业的快速发展，直播的场景越来越丰富，直播的内容也越来越出现多样化的形式。在主播进行直播的过程中，往往很容易出现有些不太合适的内容，这些内容包括但不限于：(1)某些不必要的广告牌、商标等；(2)某些少儿不宜的暴露或者血腥内容；(3)某些不愿意将真实面目展现给观众的。这些直播内容在直播过程中，因为直播没有后期剪辑的原因，不太方便去除。因此，提供一种能在直播过程中实时打码的方法是十分必要的。
发明内容
本发明的目的在于提供一种直播中打码方法，以实现在直播过程中实时打码，以屏蔽用户不想被直播的内容。
本发明的另一目的在于提供一种直播中打码装置，以实现在直播过程中实时打码，以屏蔽用户不想被直播的内容。
本发明的另一目的在于提供一种电子设备，以实现在直播过程中实时打码，以屏蔽用户不想被直播的内容。
本发明的另一目的在于提供一种存储介质，以实现在直播过程中实时打码，以屏蔽用户不想被直播的内容。
为了实现上述目的，本发明实施例采用的技术方案如下：
第一方面，本发明实施例提供了一种直播中打码方法，应用于电子设备，方法包括：
响应用户的滑动操作确定第一帧视频图像中的需打码的区域；
提取需打码的区域的颜色特征和纹理特征，生成第一特征向量；
根据第一特征向量将第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取下一帧视频图像中与需打码的区域匹配度最高的区域为区域图像；
对区域图像进行打码，以实现直播中打码。
第二方面，本发明实施例还提供了一种直播中打码装置，应用于电子设备，装置包括确定模块、提取模块、匹配模块及打码模块，其中，
确定模块，用于响应用户的滑动操作确定第一帧视频图像中的需打码的区域；
提取模块，用于提取需打码区域的颜色特征和纹理特征，生成第一特征向量；
匹配模块，用于根据第一特征向量将第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取下一帧视频图像中与需打码的区域匹配度最高的区域为区域图像；
打码模块，用于对区域图像进行打码，以实现直播中打码。
第三方面，本发明实施例提供了一种电子设备，所述电子设备包括：
存储器；
处理器；及
直播中打码装置，所述装置安装于所述存储器中并包括一个或多个由所述处理器执行的软件功能模块，所述直播中打码装置包括：
确定模块，用于响应用户的滑动操作确定第一帧视频图像中的需打码的区域；
提取模块，用于提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量；
匹配模块，用于根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像；
打码模块，用于对所述区域图像进行打码，以实现直播中打码。
第四方面，本发明实施例还提供了一种存储介质，所述存储介质中存储有计算机指令，其中，所述计算机指令在被读取并运行时执行上述的直播中打码方法。
本发明实施例提供的一种电子设备与直播中打码方法、装置及存储介质，该直播中打码方法、装置及存储介质应用于电子设备，该直播中打码方法包括响应用户的滑动操作确定第一帧视频图像中的需打码的区域，提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量。根据第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像，对所述区域图像进行打码，以实现直播中打码。通过本方案实现对直播过程中
用户不想出镜的画面实时打码，以提高用户体验感。
为使本发明的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下
为了更清楚地说明本发明实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本发明较佳实施例提供的电子设备的结构框图；
图2示出了本发明实施例提供的一种直播中打码方法的流程示意图。
图3示出了本发明实施例提供的一种直播中打码方法的子步骤的流程示意图。
图4示出了本发明实施例提供的步骤S103的子步骤的流程示意图。
图5示出了本发明实施例提供的另一种直播中打码方法的流程示意图。
图6示出了本发明实施例提供的另一种直播中打码方法的子步骤的流程示意图。
图7示出了本发明实施例提供的一种直播中打码装置的功能模块示意图。
图8示出了本发明实施例提供的一种匹配模块的子模块的示意图。
图示：100-直播中打码装置；110-确定模块；120-提取模块；130-第一采集模块；140-第一建立模块；150-第二采集模块；160-第二建立模块；170-第一计算模块；180-第二计算模块；190-选取模块；200-匹配模块；210-打码模块；211-滑动匹配子模块；212-第二特征向量生成子模块；213-匹配度计算子模块；214-区域图像选取子模块。
下面将结合本发明实施例中附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本发明实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本发明的实施例的详细描述并非旨在限制要求保护的本发明的范围，而是仅仅表示本发明的选定实施例。基于本发明的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。同时，在本发明的描述中，术语“第一”、“第二”等仅用于区分描述，而不能理解为指示或暗示相对重要性。
本发明实施例提供的一种直播中打码方法、装置及存储介质均应用于电子设备，该电子设备可以是，但不限于，台式电脑、平板电脑、手机等电子设备。
图1示出了一种可应用于本发明实施例中的电子设备200的结构框图。如图1所示，电子设备200包括直播中打码装置100、存储器102、存储控制器103，一个或多个(图中仅示出一个)处理器104、外设接口105、射频模块106、音频模块107、触控屏幕108等。所述直播中打码装置100包括至少一个可以以软件或固件(firmware)的形式存储于所述存储器102中或固化在所述电子设备200的操作系统(operating system，OS)中的软件功能模块。
存储器102可用于存储软件程序以及模块，如本发明实施例中的直播中打码装置及方法所对应的程序指令/模块，处理器104通过运行存储在存储器102内的软件程序以及模块，从而执行各种功能应用以及数据处理，如本发明实施例提供的直播中打码方法。
存储器102可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。处理器104以及其他可能的组件对存储器102的访问可在存储控制器103的控制下进行。
外设接口105将各种输入/输出装置耦合至处理器104以及存储器102。在一些实施例中，外设接口105、处理器104以及存储控制器103可以在单个芯片中实现。在其他一些实例中，他们可以分别由独立的芯片实现。
射频模块106用于接收以及发送电磁波，实现电磁波与电信号的相互转换，从而与通讯网络或者其他设备进行通讯。
音频模块107向用户提供音频接口，其可包括一个或多个麦克风、一个或者多个扬声器以及音频电路。
触控屏幕108在电子设备200与用户之间同时提供一个输出及输入界面。在本实施例中，所述触控屏幕108支持单点和多点触控操作，例如，该触控屏幕108可为支持单点和多点触控操作的电容式触控屏或电阻式触控屏等。支持单点和多点触控操作是触控屏幕108能感应到来自该触控屏上一个或多个位置处同时产生的触控操作，并将该感应到的多点触控操作交由处理器104进行处理。
可以理解，图1所示的结构仅为示意，电子设备200还可包括比图1中所示更多或者更少的组件，或者具有与图1所示不同的配置。图1中所示的各组件可以采用硬件、软件或其组合实现。
本发明实施例提出的电子设备与直播中打码方法、装置及存储介质，提供了一种新的直播中打码方法。该电子设备及直播中打码装置与方法可适用于Android操作系统、iOS操作系统、Windows Phone操作系统等平台。具体地，本发明实施例提出的直播中打码装
置、方法可应用于电子设备200的应用程序安装客户端，该客户端以将进行直播的用户不想要出镜的内容进行打码，以充分保护直播用户的隐私，进而达到吸引该主播平台主播的目的。
请参照图2，是本发明实施例提供的一种直播中打码方法的流程示意图，该直播中打码方法包括：
步骤S110，响应用户的滑动操作确定第一帧视频图像中的需打码的区域。
当用户开始直播时，可根据自己的需要设置需要打码的区域。具体为，当用户不想要广告商标出镜时，用户可以通过在屏幕上滑动设置能够包括该广告商标的矩形框，该电子设备响应用户的滑动操作，将该矩形框对应的区域设置为需打码的区域，当然地，用户还可以通过在屏幕上滑动设置能够包括该广告商标的圆形框或者正多边形框，在此不做限制。
步骤S120，提取所述需打码的区域的颜色特征和纹理特征，生成第一特征向量。
将该需打码的区域中的每一个像素点的三原色比例值(即RGB值)的维数由3维，转换为11维，换句话说，目前每一个像素点的三原色比例值只能表征红、绿、蓝三种颜色，将三原色比例值的维数由3维转换为11维之后，将每一个像素点的三原色比例值表征的颜色细化到11种，如黑色、蓝色、黄色、灰色、粉色、红色、白色等，以使得在提取需打码区域的颜色特征时，能自适应地根据需打码的区域的实际颜色分布情况进行选择。同时通过降维的思想，自适应地将每一个像素点的三原色比例值由11维降到2维，以提取需打码区域中的显著颜色特征。
通过HOG(Histograms of Oriented Gradients)特征提取方法提取该需打码的区域的纹理特征，通过提取的该颜色特征和纹理特征，生成第一特征向量。
步骤S130，根据所述第一特征向量将所述第一帧视频图像中的需打码区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像。
如图4所示，具体地，步骤S130包括：
步骤S1301：将需打码的区域依次与下一帧视频图像进行滑动匹配。
即将需打码的区域在下一帧视频图像中从上往下，从左往右依次与下一帧视频图像中与需打码的区域大小相同的区域进行匹配，该需打码的区域在下一帧视频图像中滑动的长度可以根据实际需要进行设置，需要注意的是，该滑动的长度越小，则耗时更长，该滑动的长度越长，则可能导致匹配不精准。
步骤S1302：依次提取下一帧视频图像中与需要打码的区域的大小一致的图像中颜色特征和纹理特征，生成多个第二特征向量。
步骤S1303：根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算
下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度。
其中，具体的标准均值相关匹配算法为：
其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
步骤S1304：选取下一帧视频图像中与需打码的区域大小一致的多个区域的匹配度最高的区域为区域图像。
其中，步骤S1304表明该区域图像与需打码的区域最相关，是下一帧视频图像中需要打码的区域。
步骤S140，对所述区域图像进行打码，以实现直播中打码。
请参照图2，是本发明实施例提供的一种直播中打码方法的步骤S140的子步骤的流程示意图，该步骤S140包括：
步骤S141，选取所述区域图像中与每一像素点邻近的多个像素点的最大像素值以及最小像素值。
选取所述区域图像中与每一像素点邻近的区域中的多个像素点的最大像素值和最小像素值，与每一像素点邻近的区域的大小可根据需要进行设置。
步骤S142，设置所述像素点的像素值为所述最大像素值和所述最小像素值的均值，以实现直播中打码。
将区域图像中的每一个像素点的像素值设置为邻近区域的最大像素值和最小像素值的均值，以将区域图像中的每一个像素点的像素值与邻近区域的像素点的像素值进行关联，实现模糊的视觉效果，进一步实现直播中打码的效果。
请参照图5，是本发明实施例提供的另一种直播中打码方法的流程示意图，该直播中打码方法包括：
步骤S210，响应用户的滑动操作确定第一帧视频图像中的需打码的区域。
当用户开始直播时，可根据自己的需要设置需要打码的区域。具体为，当用户不想要广告商标出镜时，用户可以通过在屏幕上滑动设置能够包括该广告商标的矩形框，该电子设备响应用户的滑动操作，将该矩形框对应的区域设置为需打码的区域。
步骤S220，提取所述需打码的区域的颜色特征和纹理特征，生成第一特征向量。
将该需打码的区域中的每一个像素点的三原色比例值(即RGB值)的维数由3维，转换为11维，换句话说，目前每一个像素点的三原色比例值只能表征红、绿、蓝三种颜色，将三原色比例值的维数由3维转换为11维之后，将每一个像素点的三原色比例值表征的颜色细化到11种，如黑色、蓝色、黄色、灰色、粉色、红色、白色等，以使得在提取需打码区域的颜色特征时，能自适应地根据需打码的区域的实际颜色分布情况进行选择。同时通过降维的思想，自适应地将每一个像素点的三原色比例值由11维降到2维，以提取需打码区域中的显著颜色特征。
通过HOG(Histograms of Oriented Gradients)特征提取方法提取该需打码的区域的纹理特征，通过提取的该颜色特征和纹理特征，生成第一特征向量。
步骤S230，采集下一帧视频图像中的每一像素点的像素值。
步骤S240，依据所述下一帧视频图像中的每一像素点的像素值建立下一帧视频图像的第一直方图。
根据下一帧视频图像的每一像素点的像素值以及相同像素值的像素点的个数建立第一直方图。
步骤S250，采集所述需打码的区域中每一像素点的像素值。
步骤S260，依据所述需打码的区域中每一像素点的像素值建立所述需打码的区域的第二直方图。
根据所述需打码的区域中每一像素点的像素值以及相同像素值的像素点的个数建立第二直方图。
步骤S270，通过计算所述第二直方图和第一直方图的比率得到第三直方图。
步骤S280，计算下一帧视频图像中的每一个像素点在所述第三直方图中的分布概率。
步骤S290，选取下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域为需依次与所述需打码的区域进行匹配的区域。
若下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值，则表明该像素点所在的区域与需打码的区域的颜色较为接近，则选取该下一帧视频图像中的像素点在第三直方图中分布概率大于预设阈值的区域为需要与需打码的区域进行匹配的区域，从而减少匹配运算的计算量。
步骤S310，根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下
一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像。
将需打码的区域依次与下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域进行滑动匹配，即将需打码的区域在该区域中从上往下，从左往右进行匹配，该需打码的区域在该区域的滑动的长度可以根据实际需要进行设置，需要注意的是，该滑动的长度越小，则耗时更长，该滑动的长度约长，则可能导致匹配不精准。
请参照图6，是本发明实施例提供的另一种直播中打码方法的步骤S310的子步骤的流程示意图，该步骤S310包括：
步骤S311，提取所述下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域的颜色特征和纹理特征，生成第二特征向量。
即提取该区域的颜色特征和纹理特征，生成第二特征向量。
步骤S312，采用标准均值相关匹配算法对所述第一特征向量和所述第二特征向量进行匹配运算，选取所述下一帧视频图像中的第二特征向量与所述需打码的区域的第一特征向量匹配度最高的区域为区域图像。
根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，具体的标准均值相关匹配算法为：
其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
选取下一帧视频图像中与需打码的区域大小一致的多个区域的匹配度最高的区域为区域图像，表明该区域图像与需打码的区域最相关，是下一帧视频图像中需要打码的区域。
步骤S320，对所述区域图像进行打码，以实现直播中打码。
选取所述区域图像中与每一像素点邻近的区域中的多个像素点的最大像素值和最小像
素值，与每一像素点邻近的区域的大小可根据需要进行设置。将区域图像中的每一个像素点的像素值设置为邻近区域的最大像素值和最小像素值的均值，以将区域图像中的每一个像素点的像素值与邻近区域的像素点的像素值进行关联，实现模糊的视觉效果，进一步实现直播中打码的效果。
请参照图7，是本发明实施例提供的一种直播中打码装置100的功能模块示意图，该直播中打码装置100包括确定模块110、提取模块120、第一采集模块130、第一建立模块140、第二采集模块150、第二建立模块160、第一计算模块170、第二计算模块180、选取模块190、匹配模块200、打码模块210，其中：
确定模块110，用于响应用户的滑动操作确定第一帧视频图像中的需打码的区域。
在本发明实施例中，步骤S110和步骤S210可以由确定模块110执行。
提取模块120，用于提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量。
在本发明实施例中，步骤S120和步骤S220可以由提取模块120执行。
第一采集模块130，用于采集下一帧视频图像中的每一像素点的像素值。
在本发明实施例中，步骤S230可以由第一采集模块130执行。
第一建立模块140，用于根据所述下一帧视频图像中的每一像素点的像素值建立下一帧视频图像的第一直方图。
在本发明实施例中，步骤S240可以由第一建立模块140执行。
第二采集模块150，用于采集所述需打码的区域中每一像素点的像素值。
在本发明实施例中，步骤S250可以由第二采集模块150执行。
第二建立模块160，用于依据所述需打码的区域中每一像素点的像素值建立所述需打码的区域的第二直方图。
在本发明实施例中，步骤S260可以由第二建立模块160执行。
第一计算模块170，用于通过计算所述第二直方图和第一直方图的比率得到第三直方图。
在本发明实施例中，步骤S270可以由第一计算模块170执行。
第二计算模块180，用于计算下一帧视频图像中的每一个像素点在所述第三直方图中的分布概率。
在本发明实施例中，步骤S280可以由第二计算模块180执行。
选取模块190，用于选取下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域为需依次与所述需打码的区域进行匹配的区域。
在本发明实施例中，步骤S290可以由选取模块190执行。
匹配模块200，用于根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依
次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像。
在本发明实施例中，步骤S130、S310～S312可以由匹配模块200执行。
其中，如图8所示，匹配模块200可以包括滑动匹配子模块211、第二特征向量生成子模块212、匹配度计算子模块213以及区域图像选取子模块214。
其中，滑动匹配子模块211可以用于执行步骤1301，第二特征向量生成子模块212可以用于执行步骤1302，匹配度计算子模块213可以用于执行步骤1303，区域图像选取子模块214可以用于执行步骤1304。
具体地，滑动匹配子模块211用于将需打码的区域依次与下一帧视频图像进行滑动匹配。
具体地滑动匹配子模块211用于将需打码的区域在下一帧视频图像中从上往下，从左往右依次与下一帧视频图像中与需打码的区域大小相同的区域进行匹配。
第二特征向量生成子模块212用于依次提取下一帧视频图像中与需要打码的区域的大小一致的图像中颜色特征和纹理特征，生成多个第二特征向量。
匹配度计算子模块213用于根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度。
具体地，匹配度计算子模块213用于依据算式
依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和
高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
区域图像选取子模块214用于选取下一帧视频图像中与需打码的区域大小一致的多个区域的匹配度最高的区域为区域图像。
打码模块210，用于对所述区域图像进行打码，以实现直播中打码。
在本发明实施例中，步骤S140～S142和步骤S320可以由打码模块210执行。
另外，本发明实施例还提供了一种存储介质，所述存储介质中存储有计算机指令，其中，所述计算机指令在被读取并运行时执行上述的直播中打码方法。由于在直播打码方法中已经详细描述，在此不再赘述。
综上所述，本发明实施例提供一种电子设备与直播中打码方法、装置及存储介质，该直播中打码方法、装置及存储介质应用于电子设备，该直播中打码方法包括响应用户的滑动操作确定第一帧视频图像中的需打码的区域，提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量。根据第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像，对所述区域图像进行打码，以实现直播中打码。通过本方案实现对直播过程中用户不想出镜的画面实时打码，以充分保护直播用户的隐私，进而达到吸引该主播平台主播的目的，且提高用户体验感。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，也可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，附图中的流程图和框图显示了根据本发明的多个实施例的装置、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现方式中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
另外，在本发明各个实施例中的各功能模块可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或两个以上模块集成形成一个独立的部分。
所述功能如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计
算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。
通过本方案实现对直播过程中用户不想出镜的画面实时打码，以充分保护直播用户的隐私，进而达到吸引该主播平台主播的目的，且提高用户体验感。
Claims (20)
- 一种直播中打码方法，应用于电子设备，其特征在于，所述方法包括：响应用户的滑动操作确定第一帧视频图像中的需打码的区域；提取所述需打码的区域的颜色特征和纹理特征，生成第一特征向量；根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像；对所述区域图像进行打码，以实现直播中打码。
- 如权利要求1所述的直播中打码方法，其特征在于，所述对所述区域图像进行打码，以实现直播中打码的步骤包括：选取所述区域图像中与每一像素点邻近的多个像素点的最大像素值以及最小像素值；设置所述像素点的像素值为所述最大像素值和所述最小像素值的均值，以实现直播中打码。
- 如权利要求1或2所述的直播中打码方法，其特征在于，所述根据所述特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像之前还包括步骤：采集下一帧视频图像中的每一像素点的像素值；依据所述下一帧视频图像中的每一像素点的像素值建立下一帧视频图像的第一直方图；采集所述需打码的区域中每一像素点的像素值；依据所述需打码的区域中每一像素点的像素值建立所述需打码的区域的第二直方图；通过计算所述第二直方图和第一直方图的比率得到第三直方图；计算下一帧视频图像中的每一个像素点在所述第三直方图中的分布概率；选取下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域为需依次与所述需打码的区域进行匹配的区域。
- 如权利要求3所述的直播中打码方法，其特征在于，所述根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像的步骤包括：根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像。
- 如权利要求4所述的直播中打码方法，其特征在于，所述根据所述第一特征向量将 所述第一帧视频图像中的需打码的区域依次与下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像的步骤包括：提取所述下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域的颜色特征和纹理特征，生成第二特征向量；采用标准均值相关匹配算法对所述第一特征向量和所述第二特征向量进行匹配运算，得出下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度；选取所述下一帧视频图像中的第二特征向量与所述需打码的区域的第一特征向量匹配度最高的区域为区域图像。
- 如权利要求5所述的直播中打码方法，其特征在于，所述采用标准均值相关匹配算法对所述第一特征向量和所述第二特征向量进行匹配运算，得出下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度的步骤包括：
依据算式依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
- 如权利要求1-3任一所述的直播中打码方法，其特征在于，所述根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像的步骤包括：将需打码的区域依次与下一帧视频图像进行滑动匹配；依次提取下一帧视频图像中与需要打码的区域的大小一致的图像中颜色特征和纹理特 征，生成多个第二特征向量；根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度；选取下一帧视频图像中与需打码的区域大小一致的多个区域的匹配度最高的区域为区域图像。
- 如权利要求7所述的直播中打码方法，其特征在于，将需打码的区域依次与下一帧视频图像进行滑动匹配的步骤包括：将需打码的区域在下一帧视频图像中从上往下，从左往右依次与下一帧视频图像中与需打码的区域大小相同的区域进行匹配。
- 如权利要求7或8所述的直播中打码方法，其特征在于，所述根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度的步骤包括：
依据算式依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
- 一种直播中打码装置，应用于电子设备，其特征在于，所述装置包括：确定模块，配置成响应用户的滑动操作确定第一帧视频图像中的需打码的区域；提取模块，配置成提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量；匹配模块，配置成根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最 高的区域为区域图像；打码模块，配置成对所述区域图像进行打码，以实现直播中打码。
- 如权利要求10所述的直播中打码装置，其特征在于，所述打码模块还配置成：选取所述区域图像中与每一像素点邻近的多个像素点的最大像素值以及最小像素值；设置所述像素点的像素值为所述最大像素值和所述最小像素值的均值，以实现直播中打码。
- 如权利要求10或11所述的直播中打码装置，其特征在于，所述装置还包括：第一采集模块，配置成采集下一帧视频图像中的每一像素点的像素值；第一建立模块，配置成根据所述下一帧视频图像中的每一像素点的像素值建立下一帧视频图像的第一直方图；第二采集模块，配置成采集所述需打码的区域中每一像素点的像素值；第二建立模块，配置成依据所述需打码的区域中每一像素点的像素值建立所述需打码的区域的第二直方图；第一计算模块，配置成通过计算所述第二直方图和第一直方图的比率得到第三直方图；第二计算模块，配置成计算下一帧视频图像中的每一个像素点在所述第三直方图中的分布概率；选取模块，配置成选取下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域为需依次与所述需打码的区域进行匹配的区域。
- 如权利要求12所述的直播中打码装置，其特征在于，所述匹配模块配置成：根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像。
- 如权利要求13所述的直播中打码装置，其特征在于，所述匹配模块配置成：提取所述下一帧视频图像中的像素点在所述第三直方图中分布概率大于预设阈值的区域的颜色特征和纹理特征，生成第二特征向量；采用标准均值相关匹配算法对所述第一特征向量和所述第二特征向量进行匹配运算，得出下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度；选取所述下一帧视频图像中的第二特征向量与所述需打码的区域的第一特征向量匹配度最高的区域为区域图像。
- 如权利要求14所述的直播中打码装置，其特征在于，所述匹配模块配置成
依据算式依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
- 如权利要求10-12任一所述的直播中打码装置，其特征在于，所述匹配模块包括：滑动匹配子模块，配置成将需打码的区域依次与下一帧视频图像进行滑动匹配；第二特征向量生成子模块，配置成依次提取下一帧视频图像中与需要打码的区域的大小一致的图像中颜色特征和纹理特征，生成多个第二特征向量；匹配度计算子模块，配置成根据第一特征向量以及第二特征向量采用标准均值相关匹配算法依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度；区域图像选取子模块，配置成选取下一帧视频图像中与需打码的区域大小一致的多个区域的匹配度最高的区域为区域图像。
- 如权利要求16所述的直播中打码装置，其特征在于，所述滑动匹配子模块配置成：将需打码的区域在下一帧视频图像中从上往下，从左往右依次与下一帧视频图像中与需打码的区域大小相同的区域进行匹配。
- 如权利要求16或17所述的直播中打码装置，其特征在于，所述匹配度计算子模块配置成：
依据算式依次计算下一帧视频图像中与需打码区域大小一致的区域与需打码区域的匹配度，其中，R(x,y)为下一帧视频图像中与需打码的区域大小一致的区域与需打码区域的匹配度，(x,y)为下一帧视频图像中的像素点的坐标，(i,j)为需打码的区域中的像素点的坐标，T为需打码的区域图像的第一特征向量，I为下一帧视频图像中的第二特征向量，(w,h)为需打码的区域的宽和高形成的坐标，(m,n)为需打码的区域遍历时用的坐标。
- 一种电子设备，其特征在于，所述电子设备包括：存储器；处理器；及直播中打码装置，所述装置安装于所述存储器中并包括一个或多个由所述处理器执行的软件功能模块，所述直播中打码装置包括：确定模块，配置成响应用户的滑动操作确定第一帧视频图像中的需打码的区域；提取模块，配置成提取所述需打码区域的颜色特征和纹理特征，生成第一特征向量；匹配模块，配置成根据所述第一特征向量将所述第一帧视频图像中的需打码的区域依次与下一帧视频图像进行匹配，选取所述下一帧视频图像中与所述需打码的区域匹配度最高的区域为区域图像；打码模块，配置成对所述区域图像进行打码，以实现直播中打码。
- 一种存储介质，其特征在于，所述存储介质中存储有计算机指令，其中，所述计算机指令在被读取并运行时执行如权利要求1-9中任一权项所述的直播中打码方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710520492.0 | 2017-06-30 | ||
| CN201710520492.0A CN107274373B (zh) | 2017-06-30 | 2017-06-30 | 直播中打码方法及装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2019000793A1 true WO2019000793A1 (zh) | 2019-01-03 |
Family
ID=60070067
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2017/111241 WO2019000793A1 (zh) | 2017-06-30 | 2017-11-16 | 直播中打码方法及装置、电子设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107274373B (zh) |
| WO (1) | WO2019000793A1 (zh) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107274373B (zh) * | 2017-06-30 | 2018-08-03 | 武汉斗鱼网络科技有限公司 | 直播中打码方法及装置 |
| CN108900895B (zh) * | 2018-08-23 | 2021-05-18 | 深圳码隆科技有限公司 | 一种对视频流的目标区域的屏蔽方法及其装置 |
| CN108897899A (zh) * | 2018-08-23 | 2018-11-27 | 深圳码隆科技有限公司 | 一种对视频流的目标区域的定位方法及其装置 |
| CN109759713B (zh) * | 2019-01-15 | 2024-03-22 | 北京缔佳医疗器械有限公司 | 基于ccd图像识别的快速打标方法及快速打标系统 |
| CN110428365B (zh) * | 2019-07-25 | 2022-10-25 | 贵阳朗玛信息技术股份有限公司 | 一种基于云端的打码方法及装置 |
| CN111836065B (zh) * | 2020-07-14 | 2022-04-29 | 北京场景互娱传媒科技有限公司 | 一种直播商标自动隐藏的智能方法 |
| CN111898587A (zh) * | 2020-08-14 | 2020-11-06 | 广州盈可视电子科技有限公司 | 一种视频打码处理方法和装置 |
| CN112788359B (zh) * | 2020-12-30 | 2023-05-09 | 北京达佳互联信息技术有限公司 | 直播处理方法、装置、电子设备及存储介质 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6463174B1 (en) * | 1998-11-17 | 2002-10-08 | Electronics And Telecommunications Research Institute | Macroblock-based segmentation and background mosaicking method |
| CN101742228A (zh) * | 2008-11-19 | 2010-06-16 | 新奥特硅谷视频技术有限责任公司 | 一种应用于数字法庭中的预处理方法和系统 |
| CN105812921A (zh) * | 2016-04-26 | 2016-07-27 | Tcl海外电子（惠州）有限公司 | 控制媒体信息播放的方法及终端 |
| CN106507110A (zh) * | 2016-12-06 | 2017-03-15 | 北京华夏电通科技有限公司 | 庭审证人画面及视频运动画面保护方法、装置和系统 |
| CN106550243A (zh) * | 2016-12-09 | 2017-03-29 | 武汉斗鱼网络科技有限公司 | 直播视频处理方法、装置及电子设备 |
| CN107274373A (zh) * | 2017-06-30 | 2017-10-20 | 武汉斗鱼网络科技有限公司 | 直播中打码方法及装置 |
-
2017
- 2017-06-30 CN CN201710520492.0A patent/CN107274373B/zh active Active
- 2017-11-16 WO PCT/CN2017/111241 patent/WO2019000793A1/zh active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6463174B1 (en) * | 1998-11-17 | 2002-10-08 | Electronics And Telecommunications Research Institute | Macroblock-based segmentation and background mosaicking method |
| CN101742228A (zh) * | 2008-11-19 | 2010-06-16 | 新奥特硅谷视频技术有限责任公司 | 一种应用于数字法庭中的预处理方法和系统 |
| CN105812921A (zh) * | 2016-04-26 | 2016-07-27 | Tcl海外电子（惠州）有限公司 | 控制媒体信息播放的方法及终端 |
| CN106507110A (zh) * | 2016-12-06 | 2017-03-15 | 北京华夏电通科技有限公司 | 庭审证人画面及视频运动画面保护方法、装置和系统 |
| CN106550243A (zh) * | 2016-12-09 | 2017-03-29 | 武汉斗鱼网络科技有限公司 | 直播视频处理方法、装置及电子设备 |
| CN107274373A (zh) * | 2017-06-30 | 2017-10-20 | 武汉斗鱼网络科技有限公司 | 直播中打码方法及装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107274373A (zh) | 2017-10-20 |
| CN107274373B (zh) | 2018-08-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019000793A1 (zh) | 直播中打码方法及装置、电子设备及存储介质 | |
| CN106254933B (zh) | 字幕提取方法及装置 | |
| CN109272459B (zh) | 图像处理方法、装置、存储介质及电子设备 | |
| US10803554B2 (en) | Image processing method and device | |
| CN109639982B (zh) | 一种图像降噪方法、装置、存储介质及终端 | |
| US11037275B2 (en) | Complex architecture for image processing | |
| Du et al. | Saliency-guided color-to-gray conversion using region-based optimization | |
| US9251613B2 (en) | Systems and methods for automatically applying effects based on media content characteristics | |
| US20150326845A1 (en) | Depth value restoration method and system | |
| CN105243371A (zh) | 一种人脸美颜程度的检测方法、系统及拍摄终端 | |
| CN109977952B (zh) | 基于局部最大值的候选目标检测方法 | |
| CN104618803A (zh) | 信息推送方法、装置、终端及服务器 | |
| JP2003058894A (ja) | ピクセル化された画像をセグメント化するための方法および装置 | |
| WO2017096946A1 (zh) | 图像高频信息的定位方法和装置 | |
| US20140212037A1 (en) | Image processing apparatus, image processing method, and computer readable medium | |
| JP6015267B2 (ja) | 画像処理装置、画像処理プログラム、これを記録したコンピュータ読み取り可能な記録媒体、および、画像処理方法 | |
| US20150278605A1 (en) | Apparatus and method for managing representative video images | |
| WO2023046112A1 (zh) | 文档图像增强方法、装置及电子设备 | |
| US9305603B2 (en) | Method and apparatus for indexing a video stream | |
| WO2023078284A1 (zh) | 图片渲染方法、装置、设备、存储介质和程序产品 | |
| CN109618098A (zh) | 一种人像面部调整方法、装置、存储介质及终端 | |
| CN106548117B (zh) | 一种人脸图像处理方法和装置 | |
| CN105447846B (zh) | 一种图像处理方法及电子设备 | |
| US9036921B2 (en) | Face and expression aligned movies | |
| CN113112422A (zh) | 图像处理方法、装置、电子设备、计算机可读介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17915755 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 17915755 Country of ref document: EP Kind code of ref document: A1 |