CN111428568B - Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium - Google Patents
Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111428568B CN111428568B CN202010119603.9A CN202010119603A CN111428568B CN 111428568 B CN111428568 B CN 111428568B CN 202010119603 A CN202010119603 A CN 202010119603A CN 111428568 B CN111428568 B CN 111428568B
- Authority
- CN
- China
- Prior art keywords
- picture
- pictures
- frame
- living
- data packet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 67
- 238000003672 processing method Methods 0.000 title claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 70
- 238000000605 extraction Methods 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 34
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000009432 framing Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 12
- 238000007906 compression Methods 0.000 description 13
- 230000006835 compression Effects 0.000 description 11
- 238000001727 in vivo Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
- G06V40/45—Detection of the body part being alive
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
Abstract
The invention relates to living body detection of a biological identification technology, and provides a living body video picture processing method, a living body video picture processing device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving a detection request of a living body video and storing the living body video into a memory; acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet; performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures; decoding the target picture to obtain a decoded target picture; and feeding back the decoding target picture to a bottom layer interface. The device, the computer equipment and the storage medium are used for processing the living video pictures based on the steps of the method. The invention directly reduces intermediate processing links in the operation in the memory, improves the living video arrangement processing performance, improves the processing speed of detecting and decoding the video by the living video, and saves the performance loss of the GPU.
Description
Technical Field
The present invention relates to the field of biological recognition technology, and in particular, to a method and apparatus for processing video pictures in living body detection, and a readable storage medium.
Background
Portrait recognition is a mature biological recognition technology and is widely applied to the fields of finance, education, public security, social services, electronic commerce and the like. However, the face is easily duplicated with a video or a photo and then fraudulently recognized, so that security is problematic. Video live detection is currently the primary solution to this security problem.
In the prior art video living body detection, a video is required to decode pictures and sound, but the living body detection only needs to detect the decoded pictures, and does not detect the decoded sound. When the confidence of living body is detected for the pictures, all video pictures are not detected, but only about 3-6 seconds of video is detected, the video rate is calculated by 25 frames/second, the total frame number of the video detected by the living body is about 100 frames, the time for decoding and format converting one picture by using a CPU is about 20-40 milliseconds, the time for 100 frames of pictures is about 2000-4000 milliseconds, and the time is long and the resources of an operating system are wasted. If the video is downloaded to the local disk for processing, the IO access amount between the disk and the memory is increased, and the processing speed is reduced.
In addition, conventional video in-vivo detection decoders often process video in a file format, and cannot directly detect memory video data. In addition, part of video detected by living bodies is sourced from mobile terminal equipment such as mobile phones, tablet computers and the like, shooting angles are different, video living body detection and decoding need to rotate images, and CPU resources and processing time are also consumed. All the above conditions can lead to reduced video live detection efficiency and poor user experience.
Accordingly, there is a need for improvements in the art in an effort to obtain a better user experience.
Disclosure of Invention
Based on this, it is necessary to provide a processing method, apparatus and readable storage medium for video pictures in live detection to improve the efficiency of existing video live detection, in view of the existing problems.
A living video picture processing method, the method comprising:
receiving a detection request of a living body video and storing the living body video into a memory;
acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet;
performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures;
Decoding the target picture to obtain a decoded target picture;
and feeding back the decoding target picture to a bottom layer interface.
In one embodiment, the method further comprises: the step of decoding the target picture to obtain a decoded target picture further includes:
acquiring picture angle information of the decoding target picture;
and performing picture rotation operation according to the picture angle information to adjust the decoding target picture to be a picture with a normal angle.
In one embodiment, the method further comprises: the step of extracting frames from the pictures in the original live video data packet according to the number of the pictures in the original live video data packet to obtain a target picture comprises the following steps:
acquiring the number and numbering of pictures in the original living video data packet;
deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
In one embodiment, the method further comprises: the step of decoding the target picture to obtain a decoded target picture comprises the following steps:
acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
Performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains complete picture information in the original living video;
and decoding the target picture after the frame extraction detection to obtain a decoded target picture.
A living-body video picture processing apparatus, the living-body video picture processing apparatus comprising:
the receiving unit is used for receiving the detection request of the living video and storing the living video into the memory;
an original living video data packet calculation unit configured to calculate the number of pictures in the original living video data packet;
the image frame extracting unit is used for extracting frames of the images in the original living video data packet according to the number of the images calculated by the original living video data packet calculating unit so as to obtain target images;
and the picture decoding unit is used for decoding and restoring the target picture extracted by the picture frame extracting unit to obtain a decoded target picture, and feeding the decoded target picture back to the bottom layer interface.
In one embodiment, the apparatus further comprises:
and the picture angle processing unit is used for acquiring the picture angle information of the decoding target picture output by the picture decoding unit, and carrying out picture rotation operation according to the picture angle information so as to adjust the decoding target picture into a picture with a normal angle.
In one embodiment, the picture frame extraction unit further includes:
the picture number numbering module is used for obtaining the number of pictures in the original living video data packet and numbering the pictures in the original living video data packet;
and the picture frame extraction processing module is used for deleting the preset number of pictures with the picture numbers at the head and the tail in the original living video data packet numbered by the picture number numbering module, and equidistantly extracting the remaining pictures to obtain a target picture.
In one embodiment, the picture decoding unit further comprises:
the reference frame acquisition module is used for acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
the picture frame extraction detection module is used for carrying out frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame acquired by the reference frame acquisition module;
and the picture decoding module is used for decoding and restoring the target picture output by the picture frame extraction detection module and feeding back the decoded target picture to a bottom layer interface.
A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the method described above.
A storage medium storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method described above.
The method, the device, the computer equipment and the storage medium for processing the living video picture receive the living video detection request and store the living video into the memory; and calculating the number of pictures contained in the living video to be detected, removing frame pictures at the head and the tail of a picture sequence in the living video, extracting the frame pictures at equal distance to obtain a target picture, detecting the target picture according to a reference frame in a video compression technology to ensure that the obtained target picture contains complete information in the original living video, decoding the target picture after the frame extraction detection to finally obtain a decoded target picture, and outputting the decoded target picture to a bottom interface. The operations of picture frame extraction and decoding are directly carried out in the memory, so that the operation of downloading the living video into the disk is avoided, the IO (Input and Output) operation of the disk is reduced, the intermediate processing links are reduced by the operation directly in the memory, the living video finishing processing performance is improved, the processing speed of detecting and decoding the video by the living video is further improved, and the performance loss of the GPU is saved. In addition, before outputting the decoding target picture to the bottom layer interface, the decoding target picture can be processed in picture angle information, the picture with an abnormal visual angle is processed in angle rotation, the picture with a normal visual angle is obtained and output to the bottom layer interface, the processing pressure of a bottom layer SDK (Soft Development Kit software development kit) is further reduced, and the accuracy of living video detection is improved.
Drawings
FIG. 1 is an environmental diagram of an implementation of a method of processing a living video picture provided in one embodiment;
FIG. 2 is a block diagram of the internal architecture of a computer device in one embodiment;
FIG. 3 is a flow chart of a method of processing a live video picture in one embodiment;
FIG. 4 is a flowchart of a method of processing a live video picture in another embodiment;
FIG. 5 is a flowchart of a method for extracting frames from a picture to obtain a target picture according to the number of pictures in an original in-vivo video data packet according to one embodiment;
FIG. 6 is a flow chart of decoding a target picture to obtain a decoded target picture in one embodiment;
fig. 7 is a block diagram showing the structure of a living body video picture processing apparatus in one embodiment;
fig. 8 is a block diagram showing the structure of a living body video picture processing apparatus in another embodiment;
FIG. 9 is a block diagram of a picture frame extraction unit in one embodiment;
fig. 10 is a block diagram of the structure of a picture decoding unit in one embodiment.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Fig. 1 is a diagram of an implementation environment of a living video picture processing method provided in one embodiment, as shown in fig. 1, in which a computer device 110 and a terminal 120 are included.
The computer device 110 is a living body video image processing device, for example, a computer device such as a computer used by a tester, and a living body video image processing tool is mounted on the computer device 110. The terminal 120 is provided with an application that needs to perform the processing of the living video picture, when the processing of the living video picture is needed, a tester can send out a living video picture processing request at the terminal 120, the living video picture processing request carries a living video picture processing identifier, the computer equipment 110 receives the living video picture processing request, a test script corresponding to the living video picture processing identifier in the computer equipment 110 is obtained according to the living video picture processing identifier, then the test script is executed by using a living video picture processing tool, the application on the terminal 120 is tested, and a living video picture processing result corresponding to the test script is obtained.
It should be noted that, the terminal 120 and the computer device 110 may be, but not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a server, and the like. The computer device 110 and the terminal 120 may be connected by bluetooth, USB (Universal Serial Bus ) or other communication connection, which is not limited herein.
FIG. 2 is a schematic diagram of the internal structure of a computer device in one embodiment. As shown in fig. 2, the computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer device stores an operating system, a database and a computer program, the database can store a control information sequence, and the computer program can enable the processor to realize a living video picture processing method when being executed by the processor. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, cause the processor to perform a method of processing a living video picture. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in fig. 2 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
As shown in fig. 3, in one embodiment, a method is proposed, which may be applied to the computer device 110 described above, and specifically may include the following steps S302 to S310:
step S302, receiving a detection request of a living body video and storing the living body video into a memory;
in this embodiment, the detection request for receiving the live video is generally performed by a computer device, and in this embodiment, a background server is used, however, the background server is not limited to the server, and other computer devices as described above may also be used. In the video living detection technology, a background server bears video decoding operation, a video living detector is arranged at a background server end, and after the video living detector receives a detection request of living videos, the video living detector can acquire an original living video data packet and store the original living video data packet into a memory. The processing speed of the original living video data can be improved by storing the original living video data packet into the memory for operation, because the direct operation speed of the data in the memory is far higher than the operation of downloading the original living video data into the disk, the processing speed of the original living video data is improved.
In some embodiments, the original live video data packets may also be saved to a non-volatile storage medium for processing.
Step S304, an original living video data packet is obtained, and the number of pictures in the original living video data packet is calculated;
in the embodiment of the present invention, after the video decoder acquires the original live video data packet, the number of frame pictures included in the original live video data packet needs to be calculated first. In the video live detection technology, the number of video data required to be detected is about 3-6 seconds, and the video data can be determined according to different video code rates. For example, the video of about 3-6 seconds contains 90 frame pictures, in this embodiment, the frame pictures contained in the video are simply called pictures, and the video decoder obtains 90 pictures in the original living video data packet.
Step S306, frames are extracted from the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures;
in the method, the frame extraction is carried out on the pictures in the original living video data packet as key points, namely the key frame pictures in the original living video data packet are extracted, so that the number of pictures processed by a video decoder is reduced, the video decoder processes necessary pictures, and unnecessary pictures are not processed, thereby saving system resources and accelerating the processing speed.
Fig. 5 shows a flowchart of extracting frames from a picture according to the number of pictures in an original live video data packet to obtain a target picture in one embodiment, which may specifically include the following steps S502 to S506:
step S502, obtaining and numbering the number of pictures in the original living video data packet;
after the video decoder obtains the number of pictures in the original live video data packet, the pictures in the original live video data packet need to be numbered sequentially, for example, the original live video data packet obtained by the video decoder contains 90 frames of pictures, i.e. the 90 frames of pictures are numbered sequentially from 0 to 89.
Step S504, deleting the preset number of pictures numbered at the beginning and the end;
for pictures in an original in-vivo video data packet, there must be a key frame and a non-key frame, in the existing video compression technology field, each frame represents a still image, and in the actual compression process, various algorithms are adopted to reduce the data capacity, and IPB is the most common means. I frame is also called intra picture (I frame), which represents key frame, and belongs to intra-frame compression, decoding can be completed only by picture data of the frame, P frame represents difference between the frame and previous key frame (P frame), and decoding needs to use picture buffered before to overlap difference defined by the frame to generate final picture. B frame is two-way difference frame, namely B frame records the difference between the frame and the front and back frames, decodes B frame, obtains the previous buffer picture, decodes the later picture, and generates the final picture by superposition of the front and back pictures and the frame picture. The B frame compression rate is high, and the CPU load is heavy during decoding, so the invention adopts the key frame I frame and the non-key frame P frame to calculate.
In this embodiment, the first and last preset pictures of the group of pictures in the original live video data packet need to be deleted, specifically, for 90 original live video data packets, 15 first and last pictures need to be deleted, where 15 first and last deleted pictures are to prevent the original live video data received by the video decoder from being false pictures, and the deleted preset number of pictures are also obtained according to experience values, so that those skilled in the art can know to perform appropriate adjustment according to the frame pictures in the received original live video data packet. For an original living video data packet of 90 pictures, after deleting the first and last 15 pictures, 60 frames of pictures are left.
Step S506, equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
After deleting the preset number of the first and last pictures in the original live video data packet, equidistant picture extraction operation is needed to be performed on the remaining pictures, for example, 60 frames of pictures remain after deleting the preset number of the first and last pictures in the original live video data packet of 90 pictures, equidistant n= (90-15×2)/(6+1) =8 is performed on the 60 frames of pictures, 6 pictures are needed to be extracted according to the distance n being 8, namely, the frames of 6 frames of pictures in the original live video data packet, such as 24 th frame (15+8), 32 th frame (24+8), 40 th frame (32+8), 48 th frame (40+8), 56 th frame (48+8), 64 th frame (56+8) and the like, are respectively extracted from the 15 th frame of the picture group in the original live video data packet, and the target pictures are obtained after equidistant extraction of the pictures in the original live video data packet.
In some embodiments, the target picture obtained by frame extraction detects the target picture according to the key frame I frame and the non-key frame P frame in the video compression technology, so as to ensure that the extracted target picture is correct, accurately represent the original living video, avoid malicious attack with high probability, and improve detection performance and response effect.
Step S308, decoding the target picture to obtain a decoded target picture;
in this embodiment, after obtaining a suitable target picture, the picture may be decoded according to the video compression technique of the original in-vivo video data packet to obtain a decoded target picture.
Fig. 6 shows a flowchart of decoding a target picture to obtain a decoded target picture in an embodiment, which may specifically include the following steps S602 to 606:
step S602, obtaining a key frame I frame and a non-key frame P frame in the original living video data packet;
step S604, performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains the complete picture information in the original living video;
step S606, decoding the target picture after the frame extraction detection to obtain a decoded target picture.
Step S310, the decoding target picture is fed back to the bottom layer interface.
In this embodiment, the video decoder outputs the resulting decoding target picture in the memory to the underlying interface, such as the underlying SDK (Software Development Kit software development kit), for in-vivo detection to complete the in-vivo video detection request from the SDK.
Fig. 4 shows another embodiment, which proposes a method for processing a living video picture, where the method may be applied to the computer device 110, and may specifically include the following steps for further optimizing a technical solution based on the method for processing a living video picture:
step S402, receiving a detection request of a living body video and storing the living body video into a memory;
in this embodiment, the detection request for receiving the live video is generally performed by a computer device, and in this embodiment, a background server is used, however, the background server is not limited to the server, and other computer devices as described above may also be used. In the video living detection technology, a background server bears video decoding operation, a video living detector is arranged at a background server end, and after the video living detector receives a detection request of living videos, the video living detector can acquire an original living video data packet and store the original living video data packet into a memory. The processing speed of the original living video data can be improved by storing the original living video data packet into the memory for operation, because the direct operation speed of the data in the memory is far higher than the operation of downloading the original living video data into the disk, the processing speed of the original living video data is improved.
In some embodiments, the original live video data packets may also be saved to a non-volatile storage medium for processing.
Step S404, an original living video data packet is obtained, and the number of pictures in the original living video data packet is calculated;
in the embodiment of the present invention, after the video decoder acquires the original live video data packet, the number of frame pictures included in the original live video data packet needs to be calculated first. In the video live detection technology, the number of video data required to be detected is about 3-6 seconds, and the video data can be determined according to different video code rates. For example, the video of about 3-6 seconds contains 90 frame pictures, in this embodiment, the frame pictures contained in the video are simply called pictures, and the video decoder obtains 90 pictures in the original living video data packet.
Step S406, frame extraction is carried out on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures;
in the method, the frame extraction is carried out on the pictures in the original living video data packet as key points, namely the key frame pictures in the original living video data packet are extracted, so that the number of pictures processed by a video decoder is reduced, the video decoder processes necessary pictures, and unnecessary pictures are not processed, thereby saving system resources and accelerating the processing speed.
Fig. 5 shows a flowchart of extracting frames from a picture according to the number of pictures in an original live video data packet to obtain a target picture in one embodiment, which may specifically include the following steps S502 to S506:
step S502, obtaining and numbering the number of pictures in the original living video data packet;
after the video decoder obtains the number of pictures in the original live video data packet, the pictures in the original live video data packet need to be numbered sequentially, for example, the original live video data packet obtained by the video decoder contains 90 frames of pictures, i.e. the 90 frames of pictures are numbered sequentially from 0 to 89.
Step S504, deleting the preset number of pictures numbered at the beginning and the end;
for pictures in an original in-vivo video data packet, there must be a key frame and a non-key frame, in the existing video compression technology field, each frame represents a still image, and in the actual compression process, various algorithms are adopted to reduce the data capacity, and IPB is the most common means. I frame is also called intra picture (I frame), which represents key frame, and belongs to intra-frame compression, decoding can be completed only by picture data of the frame, P frame represents difference between the frame and previous key frame (P frame), and decoding needs to use picture buffered before to overlap difference defined by the frame to generate final picture. B frame is two-way difference frame, namely B frame records the difference between the frame and the front and back frames, decodes B frame, obtains the previous buffer picture, decodes the later picture, and generates the final picture by superposition of the front and back pictures and the frame picture. The B frame compression rate is high, and the CPU load is heavy during decoding, so the invention adopts the key frame I frame and the non-key frame P frame to calculate.
In this embodiment, the first and last preset pictures of the group of pictures in the original live video data packet need to be deleted, specifically, for 90 original live video data packets, 15 first and last pictures need to be deleted, where 15 first and last deleted pictures are to prevent the original live video data received by the video decoder from being false pictures, and the deleted preset number of pictures are also obtained according to experience values, so that those skilled in the art can know to perform appropriate adjustment according to the frame pictures in the received original live video data packet. For an original living video data packet of 90 pictures, after deleting the first and last 15 pictures, 60 frames of pictures are left.
Step S506, equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
After deleting the preset number of the first and last pictures in the original live video data packet, equidistant picture extraction operation is needed to be performed on the remaining pictures, for example, 60 frames of pictures remain after deleting the preset number of the first and last pictures in the original live video data packet of 90 pictures, equidistant n= (90-15×2)/(6+1) =8 is performed on the 60 frames of pictures, 6 pictures are needed to be extracted according to the distance n being 8, namely, the frames of 6 frames of pictures in the original live video data packet, such as 24 th frame (15+8), 32 th frame (24+8), 40 th frame (32+8), 48 th frame (40+8), 56 th frame (48+8), 64 th frame (56+8) and the like, are respectively extracted from the 15 th frame of the picture group in the original live video data packet, and the target pictures are obtained after equidistant extraction of the pictures in the original live video data packet.
In some embodiments, the target picture obtained by frame extraction detects the target picture according to the key frame I frame and the non-key frame P frame in the video compression technology, so as to ensure that the extracted target picture is correct, accurately represent the original living video, avoid malicious attack with high probability, and improve detection performance and response effect.
Step S408, decoding the target picture to obtain a decoded target picture;
in this embodiment, after obtaining a suitable target picture, the picture may be decoded according to the video compression technique of the original in-vivo video data packet to obtain a decoded target picture.
Fig. 6 shows a flowchart of decoding a target picture to obtain a decoded target picture in an embodiment, which may specifically include the following steps S602 to 606:
step S602, obtaining a key frame I frame and a non-key frame P frame in the original living video data packet;
step S604, performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains the complete picture information in the original living video;
step S606, decoding the target picture after the frame extraction detection to obtain a decoded target picture.
Step S410, obtaining the picture angle information of the decoding target picture, and performing picture rotation operation according to the picture angle information to adjust the decoding target picture to a picture with a normal angle;
in this embodiment, for an original live video, some original live videos originate from a mobile terminal, for example, a mobile phone, and when angles of rotation are different when the mobile phone shoots the video, the shot angles are also different, that is, the video is an abnormal viewing angle. At this time, the decoded target picture needs to be further rotated, i.e. the decoded target picture is transposed by 90 degrees or 270 degrees, or mirrored by 180 degrees, rotated to a normal viewing angle and then fed back to the bottom interface. Of course, the rotation operation is not arbitrarily rotated, and needs to be performed based on angle information of the original living video. Specifically, matrix inversion or mirroring and other operation processes can be adopted to obtain a decoding target picture of a normal view. For example, the picture is a two-dimensional array, the RGB format picture with width W and height H can be expressed as array [ W ] [ H ], the element values of the array comprise (R, G, B) three values, the RGB format image is a 2-dimensional 3-layer matrix, the matrix inversion is to inversely set the array [ W ] [ H ] matrix as array [ H ] [ W ], and the 3-layer image carries out the same matrix inversion operation.
Step S412, feeding back the decoding target picture to the bottom layer interface.
In this embodiment, the video decoder outputs the resulting decoding target picture in the memory to the underlying interface, such as the underlying SDK (Software Development Kit software development kit), for in-vivo detection to complete the in-vivo video detection request from the SDK.
As shown in fig. 7, in one embodiment, a living video picture processing apparatus is provided, which may be integrated in the computer device 110 described above, and may specifically include a receiving unit 702, an original living video data packet calculating unit 704, a picture frame extracting unit 706, and a picture decoding unit 708.
A receiving unit 702, configured to receive a detection request of a living video and store the living video into a memory;
an original living video data packet calculation unit 704 for calculating the number of pictures in the original living video data packet;
a picture frame extracting unit 706, configured to extract frames from the pictures in the original live video data packet according to the number of pictures calculated by the original live video data packet calculating unit, so as to obtain a target picture;
and the picture decoding unit 708 is configured to decode and restore the target picture extracted by the picture frame extracting unit to obtain a decoded target picture, and feed back the decoded target picture to the bottom layer interface.
As shown in fig. 9, in one embodiment, the picture frame extraction unit 706 further includes a picture number numbering module 706A and a picture frame extraction processing module 706B.
A picture number numbering module 706A, configured to number the obtained number of pictures in the original live video data packet, and number the pictures in the original live video data packet;
the picture frame extraction processing module 706B is configured to delete a preset number of pictures with picture numbers at the beginning and the end in the original living video data packet numbered by the picture number numbering module, and perform equidistant picture extraction on the remaining pictures to obtain a target picture.
As shown in fig. 10, in one embodiment, the picture decoding unit 708 further includes a reference frame acquisition module 708A, a picture extraction frame detection module 708B, and a picture decoding module 708C.
A reference frame acquisition module 708A configured to acquire a key frame I frame and a non-key frame P frame in the original living video data packet;
the picture frame extraction detection module 708B is configured to perform frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame acquired by the reference frame acquisition module;
the picture decoding module 708C is configured to decode and restore the target picture output by the picture frame extraction detection module, and feed back the decoded target picture to a bottom layer interface.
Fig. 8 shows a living video picture processing apparatus according to another embodiment, which may be integrated in the computer device 110, and specifically may include a receiving unit 802, an original living video packet computing unit 804, a picture frame extracting unit 806, a picture angle processing unit 808, and a picture decoding unit 810 for further optimization based on the living video picture processing apparatus.
A receiving unit 802, configured to receive a detection request of a living video and store the living video into a memory;
an original living video data packet calculation unit 804 for calculating the number of pictures in the original living video data packet;
a picture frame extracting unit 806, configured to extract frames from the pictures in the original live video data packet according to the number of pictures calculated by the original live video data packet calculating unit, so as to obtain a target picture;
a picture angle processing unit 808, configured to obtain picture angle information of the decoding target picture output by the picture decoding unit, and perform a picture rotation operation according to the picture angle information to adjust the decoding target picture to a picture with a normal angle;
And the picture decoding unit 810 is configured to decode and restore the target picture extracted by the picture frame extracting unit to obtain a decoded target picture, and feed back the decoded target picture to the bottom layer interface.
In the embodiment of the living video picture processing device shown in fig. 8, the picture frame extracting unit 806 is the same as the living video picture processing device described above, and also includes a picture number numbering module and a picture frame extracting processing module, and is not illustrated, but reference is made to fig. 9 because it is the same as the picture frame extracting unit 706 shown in fig. 9.
The picture number numbering module is used for obtaining the number of pictures in the original living video data packet and numbering the pictures in the original living video data packet;
and the picture frame extraction processing module is used for deleting the preset number of pictures with the picture numbers at the head and the tail in the original living video data packet numbered by the picture number numbering module, and equidistantly extracting the remaining pictures to obtain a target picture.
Similarly, the picture decoding unit 810 is the same as the aforementioned living video picture processing device, and also includes a reference frame acquisition module, a picture frame extraction detection module, and a picture decoding module, and is not illustrated, but can refer to fig. 10, because it is the same as the picture decoding unit 708 shown in fig. 10.
The reference frame acquisition module is used for acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
the picture frame extraction detection module is used for carrying out frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame acquired by the reference frame acquisition module;
and the picture decoding module is used for decoding and restoring the target picture output by the picture frame extraction detection module and feeding back the decoded target picture to a bottom layer interface.
In one embodiment, a computer device is presented, the computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program: receiving a detection request of a living body video and storing the living body video into a memory; acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet; performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures; decoding the target picture to obtain a decoded target picture; and feeding back the decoding target picture to a bottom layer interface.
In one embodiment, the processor, when executing the computer readable instructions, further performs the steps of: the step of extracting frames from the pictures in the original live video data packet according to the number of the pictures in the original live video data packet to obtain a target picture comprises the following steps:
acquiring the number and numbering of pictures in the original living video data packet;
deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
In one embodiment, the step of decoding the target picture to obtain a decoded target picture includes:
acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains complete picture information in the original living video;
and decoding the target picture after the frame extraction detection to obtain a decoded target picture.
In another embodiment, a computer device is also presented, the computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of: receiving a detection request of a living body video and storing the living body video into a memory; acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet; performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures; decoding the target picture to obtain a decoded target picture; acquiring picture angle information of the decoding target picture, and performing picture rotation operation according to the picture angle information to adjust the decoding target picture into a picture with a normal angle; and feeding back the decoding target picture to a bottom layer interface.
In one embodiment, the processor, when executing the computer readable instructions, further performs the steps of: the step of extracting frames from the pictures in the original live video data packet according to the number of the pictures in the original live video data packet to obtain a target picture comprises the following steps:
acquiring the number and numbering of pictures in the original living video data packet;
deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
In one embodiment, the step of decoding the target picture to obtain a decoded target picture includes:
acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains complete picture information in the original living video;
and decoding the target picture after the frame extraction detection to obtain a decoded target picture.
In one embodiment, a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a detection request of a living body video and storing the living body video into a memory; acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet; performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures; decoding the target picture to obtain a decoded target picture; and feeding back the decoding target picture to a bottom layer interface.
In one embodiment, the processor, when executing the computer readable instructions, further performs the steps of: the step of extracting frames from the pictures in the original live video data packet according to the number of the pictures in the original live video data packet to obtain a target picture comprises the following steps:
acquiring the number and numbering of pictures in the original living video data packet;
deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
In one embodiment, the step of decoding the target picture to obtain a decoded target picture includes:
acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains complete picture information in the original living video;
and decoding the target picture after the frame extraction detection to obtain a decoded target picture.
In another embodiment, a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a detection request of a living body video and storing the living body video into a memory; acquiring an original living video data packet, and calculating the number of pictures in the original living video data packet; performing frame extraction on the pictures in the original living video data packet according to the number of the pictures in the original living video data packet so as to obtain target pictures; decoding the target picture to obtain a decoded target picture; acquiring picture angle information of the decoding target picture, and performing picture rotation operation according to the picture angle information to adjust the decoding target picture into a picture with a normal angle; and feeding back the decoding target picture to a bottom layer interface.
In one embodiment, the processor, when executing the computer readable instructions, further performs the steps of: the step of extracting frames from the pictures in the original live video data packet according to the number of the pictures in the original live video data packet to obtain a target picture comprises the following steps:
acquiring the number and numbering of pictures in the original living video data packet;
deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
In one embodiment, the step of decoding the target picture to obtain a decoded target picture includes:
acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame, so as to ensure that the target picture contains complete picture information in the original living video;
and decoding the target picture after the frame extraction detection to obtain a decoded target picture.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (8)
1. A method of processing a living video picture, the method comprising:
receiving a detection request of a living body video and storing the living body video into a memory;
acquiring an original living video data packet from the memory, and calculating the number of pictures in the original living video data packet;
equidistance frame extraction is carried out on the pictures in the original living video data packet in the memory according to the number of the pictures in the original living video data packet so as to obtain target pictures;
Acquiring a key frame I frame and a non-key frame P frame in the original living video data packet from the memory, and performing frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame to ensure that the target picture contains complete picture information in the original living video;
decoding the target picture after the frame extraction detection in the memory to obtain a decoded target picture;
and feeding back the decoding target picture to a bottom layer interface.
2. The method of processing a living video picture according to claim 1, wherein the step of decoding the target picture to obtain a decoded target picture further comprises:
and acquiring the picture angle information of the decoding target picture, and performing picture rotation operation according to the picture angle information to adjust the decoding target picture into a picture with a normal angle.
3. The living video picture processing method according to claim 1 or 2, wherein the step of framing pictures in the original living video data packet according to the number of pictures in the original living video data packet to obtain a target picture includes:
acquiring the number and numbering of pictures in the original living video data packet;
Deleting the preset number of pictures numbered at the head and the tail;
and equidistantly extracting the remaining pictures of the preset number of pictures from which the head and the tail are deleted, so as to obtain a target picture.
4. A living-body video picture processing apparatus, characterized by comprising:
the receiving unit is used for receiving the detection request of the living video and storing the living video into the memory;
an original living video data packet calculation unit configured to calculate the number of pictures in an original living video data packet in the memory;
the picture frame extracting unit is used for carrying out equidistant frame extraction on pictures in the original living video data packet in the memory according to the number of the pictures calculated by the original living video data packet calculating unit so as to obtain a target picture;
the picture decoding unit is used for decoding and restoring the target picture extracted by the picture frame extracting unit in the memory to obtain a decoded target picture, and feeding the decoded target picture back to a bottom layer interface; wherein,
the picture decoding unit further includes:
the reference frame acquisition module is used for acquiring a key frame I frame and a non-key frame P frame in the original living video data packet;
The picture frame extraction detection module is used for carrying out frame extraction detection on the target picture according to the information of the key frame I frame and the non-key frame P frame acquired by the reference frame acquisition module;
and the picture decoding module is used for decoding and restoring the target picture output by the picture frame extraction detection module and feeding back the decoded target picture to a bottom layer interface.
5. The living video picture processing apparatus according to claim 4, wherein the apparatus further comprises:
and the picture angle processing unit is used for acquiring the picture angle information of the decoding target picture output by the picture decoding unit, and carrying out picture rotation operation according to the picture angle information so as to adjust the decoding target picture into a picture with a normal angle.
6. The living video picture processing apparatus according to claim 4 or 5, wherein the picture frame extracting unit further comprises:
the picture number numbering module is used for obtaining the number of pictures in the original living video data packet and numbering the pictures in the original living video data packet;
and the picture frame extraction processing module is used for deleting the preset number of pictures with the picture numbers at the head and the tail in the original living video data packet numbered by the picture number numbering module, and equidistantly extracting the remaining pictures to obtain a target picture.
7. A computer device comprising a memory and a processor, the memory having stored therein computer readable instructions which, when executed by the processor, cause the processor to perform the steps of the method of any of claims 1 to 3.
8. A storage medium storing computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of the method of any of claims 1 to 3.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010119603.9A CN111428568B (en) | 2020-02-26 | 2020-02-26 | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium |
| PCT/CN2020/136434 WO2021169501A1 (en) | 2020-02-26 | 2020-12-15 | Living body video picture processing method and apparatus, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010119603.9A CN111428568B (en) | 2020-02-26 | 2020-02-26 | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111428568A CN111428568A (en) | 2020-07-17 |
| CN111428568B true CN111428568B (en) | 2024-04-09 |
Family
ID=71551594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010119603.9A Active CN111428568B (en) | 2020-02-26 | 2020-02-26 | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111428568B (en) |
| WO (1) | WO2021169501A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428568B (en) * | 2020-02-26 | 2024-04-09 | 平安科技（深圳）有限公司 | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium |
| CN112866799B (en) * | 2020-12-31 | 2023-08-11 | 百果园技术(新加坡)有限公司 | Video frame extraction processing method, device, equipment and medium |
| CN114394100B (en) * | 2022-01-12 | 2024-04-05 | 深圳力维智联技术有限公司 | Unmanned patrol car control system and unmanned car |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108024159A (en) * | 2017-12-07 | 2018-05-11 | 上海网达软件股份有限公司 | A kind of generation method and system of video file thumbnail |
| CN110472487A (en) * | 2019-07-03 | 2019-11-19 | 平安科技（深圳）有限公司 | Living body user detection method, device, computer equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10142707B2 (en) * | 2016-02-25 | 2018-11-27 | Cyberlink Corp. | Systems and methods for video streaming based on conversion of a target key frame |
| CN106791480A (en) * | 2016-11-30 | 2017-05-31 | 努比亚技术有限公司 | A kind of terminal and video skimming creation method |
| CN108377399B (en) * | 2018-03-07 | 2019-03-19 | 广州图普网络科技有限公司 | Live video stream code-transferring method, device and computer readable storage medium |
| CN111428568B (en) * | 2020-02-26 | 2024-04-09 | 平安科技（深圳）有限公司 | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium |
-
2020
- 2020-02-26 CN CN202010119603.9A patent/CN111428568B/en active Active
- 2020-12-15 WO PCT/CN2020/136434 patent/WO2021169501A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108024159A (en) * | 2017-12-07 | 2018-05-11 | 上海网达软件股份有限公司 | A kind of generation method and system of video file thumbnail |
| CN110472487A (en) * | 2019-07-03 | 2019-11-19 | 平安科技（深圳）有限公司 | Living body user detection method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111428568A (en) | 2020-07-17 |
| WO2021169501A1 (en) | 2021-09-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111428568B (en) | Living-body video picture processing method, living-body video picture processing device, computer equipment and storage medium | |
| US11609968B2 (en) | Image recognition method, apparatus, electronic device and storage medium | |
| CN108805047B (en) | Living body detection method and device, electronic equipment and computer readable medium | |
| US10776609B2 (en) | Method and system for facial recognition | |
| US20110311100A1 (en) | Method, Apparatus and Computer Program Product for Providing Object Tracking Using Template Switching and Feature Adaptation | |
| US20220180477A1 (en) | Video super-resolution processing method and apparatus | |
| CN111985281B (en) | Image generation model generation method and device and image generation method and device | |
| CN112991180B (en) | Image stitching method, device, equipment and storage medium | |
| US20210397822A1 (en) | Living body detection method, apparatus, electronic device, storage medium and program product | |
| US20240153041A1 (en) | Image processing method and apparatus, computer, readable storage medium, and program product | |
| CN115346278A (en) | Image detection method, device, readable medium and electronic equipment | |
| CN107733874B (en) | Information processing method, information processing device, computer equipment and storage medium | |
| CN111274996B (en) | Face picture feature comparison method, device, computer equipment and storage medium | |
| CN113256643A (en) | Portrait segmentation model training method, storage medium and terminal equipment | |
| US10282633B2 (en) | Cross-asset media analysis and processing | |
| CN114973293A (en) | Similarity judgment method, key frame extraction method, device, medium and equipment | |
| CN115190303A (en) | Cloud desktop image processing method and system and related equipment | |
| CN109493349B (en) | Image feature processing module, augmented reality equipment and corner detection method | |
| CN115937338B (en) | Image processing method, device, equipment and medium | |
| CN116567296B (en) | Video picture processing method, device, computer equipment and storage medium | |
| CN116993620B (en) | Deblurring method and electronic equipment | |
| US20220279241A1 (en) | Method and device for recognizing images | |
| CN115883768A (en) | Screen sharing method and device, computer equipment and storage medium | |
| CN116668736A (en) | Video generation method, device, equipment and storage medium | |
| CN117710775A (en) | Image training pair generation method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40032044 Country of ref document: HK |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |