CN114170553A - Image processing method and device and electronic equipment - Google Patents
Image processing method and device and electronic equipment Download PDFInfo
- Publication number
- CN114170553A CN114170553A CN202111502823.0A CN202111502823A CN114170553A CN 114170553 A CN114170553 A CN 114170553A CN 202111502823 A CN202111502823 A CN 202111502823A CN 114170553 A CN114170553 A CN 114170553A
- Authority
- CN
- China
- Prior art keywords
- video frame
- image processing
- processing
- video
- stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 204
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000004044 response Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 238000003709 image segmentation Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 25
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000010365 information processing Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
In the image processing method, the image processing device and the electronic device disclosed by the embodiment of the disclosure, in the process of processing each video frame in a video stream, the process of processing the video frame is divided into two image processing stages for processing, when the image processing of the second stage is performed, the image processing of the first stage does not need to be vacant, and when the image processing of the first stage is performed, the image processing of the second stage also does not need to be vacant, so that the efficiency of processing the video stream can be improved. Meanwhile, the first-stage image processing and the second-stage image processing can be carried out simultaneously, and the second video frame can be processed immediately after the first video frame is processed, so that the time interval between the video frame processing results can be shortened, therefore, the frame skipping processing is not needed, and the accuracy of the processing process can be improved.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to an image processing method and apparatus, and an electronic device.
Background
With the development of scientific technology, some data processing may also be performed by using a cloud server, for example, image recognition, limb detection, and the like by using a cloud. However, when data processing is performed at the cloud, data is processed when data is received, and then, after the data processing is completed, data transmission is continued to be waited; after the data is delivered, the data processing can be performed again.
Disclosure of Invention
This disclosure is provided to introduce concepts in a simplified form that are further described below in the detailed description. This disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
The embodiment of the disclosure provides an image processing method, an image processing device and an electronic device, which can divide a video stream processing process into a first-stage image processing and a second-stage image processing in the video stream processing process, and the first-stage image processing and the second-stage image processing can be performed simultaneously, so that the efficiency of processing the video stream is improved, all video frames in the video stream can be processed, and the accuracy of processing the video stream can be improved.
In a first aspect, an embodiment of the present disclosure provides an image processing method, including: generating a video frame queue, wherein the step of generating the video frame queue comprises: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue; responding to the completion of second-stage image processing on a first video frame in the video frame queue, and acquiring a second video frame one bit behind the first video frame from the video frame sequence; and carrying out second-stage image processing on the second video frame.
In a second aspect, an embodiment of the present disclosure provides an image processing apparatus, including: an obtaining unit configured to generate a video frame queue, wherein the step of generating the video frame queue includes: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue; an obtaining unit, configured to obtain, from the video frame sequence, a second video frame one bit after a first video frame in the video frame queue in response to completion of second-stage image processing on the first video frame in the video frame queue; and the processing unit is used for carrying out second-stage image processing on the second video frame.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the image processing method according to the first aspect.
In a fourth aspect, the disclosed embodiments provide a computer readable medium, on which a computer program is stored, which when executed by a processor, implements the steps of the image processing method as described above in the first aspect.
According to the image processing method, the image processing device and the electronic equipment provided by the embodiment of the disclosure, in the process of processing each video frame in a video stream, the processing process of the video frame is divided into two image processing stages for processing, when the second-stage image processing is performed, the first-stage image processing does not need to be vacant, and when the first-stage image processing is performed, the second-stage image processing also does not need to be vacant, so that the efficiency of processing the video stream can be improved. Meanwhile, the first-stage image processing and the second-stage image processing can be carried out simultaneously, and the second video frame can be processed immediately after the first video frame is processed, so that the time interval between the video frame processing results can be shortened, therefore, the frame skipping processing is not needed, and the accuracy of the processing process can be improved.
Drawings
The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.
FIG. 1 is a flow diagram for one embodiment of an image processing method according to the present disclosure;
FIG. 2 is a process flow diagram of another embodiment of an image processing method according to the present disclosure;
FIG. 3 is a process flow diagram of yet another embodiment of an image processing method according to the present disclosure;
FIG. 4 is a schematic block diagram of one embodiment of an image processing apparatus according to the present disclosure;
FIG. 5 is an exemplary system architecture to which the image processing method of one embodiment of the present disclosure may be applied;
fig. 6 is a schematic diagram of a basic structure of an electronic device provided according to an embodiment of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.
It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.
The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.
It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.
The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.
Referring to fig. 1, a flow of one embodiment of an image processing method according to the present disclosure is shown. The image processing method can be applied to terminal equipment or a cloud server. The image processing method as shown in fig. 1 includes the steps of:
Here, the generating of the video frame queue may include: and carrying out first-stage image processing on each video frame in the received video stream, and caching the processed video frame according to the receiving time to obtain a video frame queue.
As an example, when identifying information characterized by a video stream, identification is usually required for each video frame in the video stream, and before identifying the video frame, the video frame may be preprocessed to improve subsequent identification efficiency. While the first stage of image processing may be understood as pre-processing before identifying the video frames. For example, the first stage image processing includes: image gray scale conversion, image normalization processing, image enhancement processing and the like. That is, after the video frame is subjected to the first stage image processing, the efficiency when the subsequent video frame is identified can be improved.
As an example, the video frames in the video frame queue are video frames that have undergone the first stage image processing.
As an example, the second stage process may be understood as an identification process of the video frame, for example, information specifically included in the video frame image may be identified.
As an example, when the first video frame is performing the second-stage image processing, at this time, the video frame in the video stream that has not been performed with the first-stage image processing may continue to perform the first-stage image processing, that is, the first-stage image processing and the second-stage image processing may be performed simultaneously.
As an example, the processing duration of the image processing of the video frame in the second stage is generally longer than the processing duration of the image processing of the video frame in the first stage. Therefore, after the first video frame completes the second stage image processing, a plurality of video frames may be already stored in the video frame queue, and at this time, the second video frame after the first video frame may be directly acquired.
And 103, performing second-stage image processing on the second video frame.
As an example, after the second video frame is acquired, the second stage image processing may be performed on the second video frame.
In the related art, in the process of processing each video frame in the video stream, all processing flows of one video frame are processed first, and then the next video frame is processed, and in the process, since the second video frame may have been discarded when the first video frame is processed, it is only possible to wait for the transmission of the third video frame and then continue the processing. In this way, errors occur in the processing result of the obtained video stream, and the total processing time of the video frames is longer.
It can be seen that, in the process of processing each video frame in the video stream in the present application, the process of processing the video frame is divided into two image processing stages for processing, and when the second stage image processing is performed, the first stage image processing does not need to be performed in an empty state, and when the first stage image processing is performed, the second stage image processing also does not need to be performed in an empty state, so that the efficiency of processing the video stream can be improved.
Meanwhile, the first-stage image processing and the second-stage image processing can be carried out simultaneously, and after the first video frame processing is finished, the second video frame can be processed immediately, so that the time interval between the obtained video frame processing results can be shortened, therefore, frame skipping recognition is not needed, and the recognition accuracy can be improved.
For convenience of understanding, the differences between the related art and the present application can be described with reference to fig. 2 and fig. 3, fig. 2 is a schematic diagram of a processing flow of processing a video stream according to the related art, as shown in fig. 2, after receiving the video stream, a video frame of the video stream can be processed, and the processing flow can include a first-stage image processing and a second-stage image processing, for example, after the first-stage image processing and the second-stage image processing of the video frame a are completed; the video frame may be acquired again, and since the video frame B may have passed at this time, it is necessary to wait for the video frame C, and after the video frame C is acquired, the first-stage image processing and the second-stage image processing may be performed on the video frame C. As can be seen from fig. 2, the time interval between the processing result a corresponding to the video frame a and the processing result C corresponding to the video frame C is equal to two input frame intervals, and the video frame image B has not been processed yet. In fig. 2, T1 and T2 can be understood as video frame intervals, that is, after the processing result a of the video frame a is obtained, it is necessary to wait for the duration of T21 to obtain the video frame C, and the video frame C can be processed to obtain the processing result C of the video frame C. At this time, the time interval between the processing result a corresponding to the video frame a and the processing result C corresponding to the video frame C is T3, and T3 is T1+ T2.
With continuing reference to fig. 3, fig. 3 is a schematic view of a processing flow of processing a video stream in the present application, as shown in fig. 3, after receiving the video stream, each video frame in the video stream may be subjected to a first-stage image processing, and a video frame queue is generated from the video frames that have completed the first-stage image processing, and when a video frame is included in the video frame queue, a second-stage image processing may be started; if the video frame D completes the first-stage image processing, the second-stage image processing of the video frame D may be started, and at this time, the first-stage image processing of the video frame E may also be started at the same time; after the video frame D completes the second-stage image processing, the video certificate E that completes the first-stage image processing may be immediately acquired, and then the second-stage image processing may be performed on the video frame E that completes the first-stage image processing. As can be seen from fig. 3, the time interval between the parsing result corresponding to the video frame D and the parsing result corresponding to the video frame E is equal to the processing time of the second stage image processing, and the single processing time of the second stage image processing is obviously less than the interval between two input video frames. And in this way, all video frames of the video stream can be processed. Whereas T4 and T5 may be understood as the time interval between video frames for which the first stage of video frame processing is completed, T6 may be understood as the time interval between the processing result a corresponding to video a and the processing result B corresponding to video frame B, where T6 is equal to the processing time duration of video frame B for which the first stage of image processing is completed. That is, T6< T4+ T5.
Therefore, the image processing method disclosed by the application can not only split the processing process of the video frame into two stages for processing, but also process the two stages simultaneously, so that the processing efficiency of the video frame is improved, and the video frame subjected to the first-stage image processing can be cached, so that all the video frames in the video stream can be processed.
In some embodiments, the execution subject of steps 101 to 103 may be a cloud server, and the video stream processed by the cloud server is received to the terminal device.
As an example, after the terminal device obtains the video stream, the video stream may be transmitted to the cloud server, and since the computing capability of the cloud server is usually stronger than that of the terminal device, the video stream is transmitted to the cloud server for processing, so that not only can the processing result of the video stream be obtained faster, but also the computing resource of the terminal device can be saved.
In some embodiments, the cloud server may generate a processing result corresponding to the video stream in response to determining that each video frame in the video stream completes the second stage of image processing; and may send the processing result to the corresponding terminal device.
Here, the processing result may be used to characterize the information indicated in the video stream.
As an example, after receiving the video stream, the cloud server may process the video stream to obtain a processing result of the video frame, so that only the processing result needs to be returned to the terminal device.
As an example, the processing result may include: and the sub-processing result corresponds to each video frame image.
As an example, after each video frame is subjected to the second stage processing, one sub-processing result may be obtained.
In some embodiments, the execution subject of steps 101 to 103 may be a cloud server, and the video stream processed by the server includes the image of the target category.
In some embodiments, the images of the target category may include any of: limb images and gesture images.
As an example, when a limb image, a gesture image, or the like is included in the video stream, it may be necessary to determine a motion posture of a limb in the video stream, or determine a gesture posture in the video stream, or the like; and because the calculation resources required for determining the motion gestures and the gesture gestures are large, when the video frames in the video stream are judged to comprise the target type images, the video stream can be sent to the cloud server. Therefore, the limb movement gesture or gesture in the video stream can be obtained quickly.
In some embodiments, when the terminal device determines that the amount of information in the video stream that needs to be processed is large, the video stream may be sent to the cloud for processing, and when the terminal device determines that the amount of information in the video stream is small, the terminal device may directly process the video stream.
In some embodiments, each video frame in the video stream includes a limb image, at which point the pose result may be generated based on the video frames for which second stage image processing is completed in response to determining that each video frame in the video stream completes second stage image processing.
Here, the gesture result is used to indicate the motion gesture of the limb in the video stream.
In some embodiments, after step 102 (in response to completing the second stage image processing on the first video frame in the video frame queue, obtaining a second video frame one bit after the first video frame from the video frame sequence), the first image result corresponding to the first video frame may also be saved; and deleting the first video frame in the video frame queue.
As an example, after processing each video frame, one image result may be obtained, and after processing all the video frames in the video stream, the processing result of the video flow may be obtained, that is, after processing all the video frames in the video stream, the processing result corresponding to the video stream may be obtained. For example, when a video frame in a video stream includes a limb image, then limb dynamics may be obtained.
In some embodiments, the first stage image processing may include at least any one of: image gray scale conversion, image normalization processing, image enhancement processing and the like.
As an example, a video frame may be subjected to gray-scale conversion first, so that the influence of color on subsequent image recognition may be reduced, and the amount of data that needs to be processed may be reduced. Then, the video frame can be continuously normalized, and then the image can be enhanced, so that the characteristics of the image are more obvious. The image can be conveniently identified in the second stage image processing process.
In some embodiments, the second stage image processing may include at least any of: image feature extraction, image segmentation, image recognition, and the like.
As an example, the second stage process may be understood as processing the video frame that completes the first stage process using a preset model. The type of the specifically adopted model can be set according to actual conditions, for example, a gesture image needs to be recognized, and the preset model can be a model for recognizing a gesture in the image. It should be noted that there are many ways to train an image recognition model, and for the sake of brevity of the description, the specific training way of the preset model is not limited here, and only needs to be reasonably selected according to the actual situation.
As an example, the second stage image processing may be understood as performing recognition processing on an image, for example, an image of a limb is included in the image, and the second stage image processing may be understood as performing recognition on a characterized limb form of the limb image.
With further reference to fig. 4, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an image processing apparatus, which corresponds to the embodiment of the image processing method shown in fig. 1, and which is particularly applicable in various electronic devices.
As shown in fig. 4, the image processing apparatus of the present embodiment includes: a generating unit 401, configured to generate a video frame queue, where the step of generating the video frame queue includes: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue; an obtaining unit 402, configured to, in response to that second-stage image processing is performed on a first video frame in the video frame queue, obtain, from the video frame sequence, a second video frame that is one bit after the first video frame; a processing unit 403, configured to perform a second-stage image processing on the second video frame.
In some embodiments, the execution subject of the method includes a cloud server, and the video stream processed by the cloud server is received from a terminal device.
In some embodiments, the cloud server generates a processing result corresponding to the video stream in response to determining that each video frame in the video stream completes a second stage of image processing; and sending the processing result to the corresponding terminal equipment.
In some embodiments, the execution subject of the method includes a cloud server, and the video stream processed by the cloud server includes an image of a target category; wherein the image of the target category includes any one of: limb images and gesture images.
In some embodiments, the apparatus further comprises a deleting unit 404 configured to save a first image result corresponding to the first video frame; and deleting the first video frame in the video frame queue.
In some embodiments, the first stage image processing includes at least any one of: image gray level conversion, image normalization processing and image enhancement processing.
In some embodiments, the second stage image processing may include at least any of: image feature extraction, image segmentation and image identification.
Referring to fig. 5, fig. 5 illustrates an exemplary system architecture to which the image processing method of one embodiment of the present disclosure may be applied.
As shown in fig. 5, the system architecture may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 may be the medium used to provide communication links between the terminal devices 501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 501, 502, 503 may interact with a server 505 over a network 504 to receive or send messages or the like. The terminal devices 501, 502, 503 may have various client applications installed thereon, such as a web browser application, a search-type application, and a news-information-type application. The client application in the terminal device 501, 502, 503 may receive the instruction of the user, and complete the corresponding function according to the instruction of the user, for example, add the corresponding information in the information according to the instruction of the user.
The terminal devices 501, 502, 503 may be hardware or software. When the terminal devices 501, 502, 503 are hardware, they may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like. When the terminal devices 501, 502, and 503 are software, they can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The server 505 may be a server providing various services, for example, receiving an information acquisition request sent by the terminal device 501, 502, 503, and acquiring the presentation information corresponding to the information acquisition request in various ways according to the information acquisition request. And the relevant data of the presentation information is sent to the terminal equipment 501, 502, 503.
It should be noted that the information processing method provided by the embodiment of the present disclosure may be executed by a terminal device, and accordingly, the image processing apparatus may be provided in the terminal devices 501, 502, and 503. In addition, the information processing method provided by the embodiment of the present disclosure may also be executed by the server 505, and accordingly, an information processing apparatus may be provided in the server 505.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to fig. 6, shown is a schematic diagram of an electronic device (e.g., a terminal device or a server of fig. 5) suitable for use in implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, the electronic device may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: generating a video frame queue, wherein the step of generating the video frame queue comprises: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue; responding to the completion of second-stage image processing on a first video frame in the video frame queue, and acquiring a second video frame one bit behind the first video frame from the video frame sequence; and carrying out second-stage image processing on the second video frame.
Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the generation unit 401 may also be described as a "unit for generating a video frame queue".
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.
Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims (10)
1. An image processing method, comprising:
generating a video frame queue, wherein the step of generating the video frame queue comprises: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue;
in response to the completion of the second-stage image processing on the first video frame in the video frame queue, acquiring a second video frame one bit behind the first video frame from the video frame sequence;
and carrying out second-stage image processing on the second video frame.
2. The method according to claim 1, wherein the execution subject of the method comprises a cloud server, and the video stream processed by the cloud server is received from the terminal device.
3. The method of claim 2, further comprising:
generating a processing result corresponding to the video stream in response to determining that each video frame in the video stream completes the second-stage image processing;
and sending the processing result to corresponding terminal equipment.
4. The method according to claim 1, wherein the execution subject of the method comprises a cloud server, wherein the video stream processed by the cloud server comprises images of the target category;
wherein the image of the target category comprises any one of:
limb images and gesture images.
5. The method of claim 1, wherein after the second stage video frame processing is completed in response to a first video frame in the sequence of video frames, the method further comprises:
saving a first image result corresponding to the first video frame; and deleting the first video frame in the video frame queue.
6. The method of claim 1, wherein the first stage image processing comprises at least any one of:
image gray level conversion, image normalization processing and image enhancement processing.
7. The method of claim 1, wherein the second stage image processing comprises at least one of:
image feature extraction, image segmentation and image identification.
8. An image processing apparatus characterized by comprising:
a generating unit, configured to generate a video frame queue, wherein the step of generating the video frame queue includes: performing first-stage image processing on each video frame in the received video stream, and caching the video frames subjected to the first-stage image processing according to receiving time to obtain a video frame queue;
the acquisition unit is used for responding to the completion of second-stage image processing on a first video frame in the video frame queue, and acquiring a second video frame one bit behind the first video frame from the video frame sequence;
and the processing unit is used for carrying out second-stage image processing on the second video frame.
9. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.
10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111502823.0A CN114170553A (en) | 2021-12-09 | 2021-12-09 | Image processing method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111502823.0A CN114170553A (en) | 2021-12-09 | 2021-12-09 | Image processing method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114170553A true CN114170553A (en) | 2022-03-11 |
Family
ID=80485232
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111502823.0A Pending CN114170553A (en) | 2021-12-09 | 2021-12-09 | Image processing method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170553A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170111532A1 (en) * | 2012-01-12 | 2017-04-20 | Kofax, Inc. | Real-time processing of video streams captured using mobile devices |
| CN106686278A (en) * | 2016-12-31 | 2017-05-17 | 华中科技大学 | Heterogeneous processor parallel coordination processing method facing real-time object identification |
| CN109544598A (en) * | 2018-11-21 | 2019-03-29 | 电子科技大学 | Method for tracking target, device and readable storage medium storing program for executing |
| EP3499900A2 (en) * | 2018-05-31 | 2019-06-19 | Beijing Baidu Netcom Science and Technology Co., Ltd. | Video processing method, apparatus and device |
| CN111079669A (en) * | 2019-12-20 | 2020-04-28 | 京东方科技集团股份有限公司 | Image processing method, device and storage medium |
| CN111462060A (en) * | 2020-03-24 | 2020-07-28 | 湖南大学 | Method and device for detecting standard section image in fetal ultrasonic image |
| CN112291483A (en) * | 2020-10-29 | 2021-01-29 | 京东方科技集团股份有限公司 | Video pushing method and system, electronic equipment and readable storage medium |
| WO2021179804A1 (en) * | 2020-03-11 | 2021-09-16 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium, and electronic apparatus |
| CN113408380A (en) * | 2021-06-07 | 2021-09-17 | 深圳小湃科技有限公司 | Video image adjusting method, device and storage medium |
| CN113473126A (en) * | 2020-03-31 | 2021-10-01 | 阿里巴巴集团控股有限公司 | Video stream processing method and device, electronic equipment and computer readable medium |
-
2021
- 2021-12-09 CN CN202111502823.0A patent/CN114170553A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170111532A1 (en) * | 2012-01-12 | 2017-04-20 | Kofax, Inc. | Real-time processing of video streams captured using mobile devices |
| CN106686278A (en) * | 2016-12-31 | 2017-05-17 | 华中科技大学 | Heterogeneous processor parallel coordination processing method facing real-time object identification |
| EP3499900A2 (en) * | 2018-05-31 | 2019-06-19 | Beijing Baidu Netcom Science and Technology Co., Ltd. | Video processing method, apparatus and device |
| CN109544598A (en) * | 2018-11-21 | 2019-03-29 | 电子科技大学 | Method for tracking target, device and readable storage medium storing program for executing |
| CN111079669A (en) * | 2019-12-20 | 2020-04-28 | 京东方科技集团股份有限公司 | Image processing method, device and storage medium |
| WO2021179804A1 (en) * | 2020-03-11 | 2021-09-16 | Oppo广东移动通信有限公司 | Image processing method, image processing device, storage medium, and electronic apparatus |
| CN111462060A (en) * | 2020-03-24 | 2020-07-28 | 湖南大学 | Method and device for detecting standard section image in fetal ultrasonic image |
| CN113473126A (en) * | 2020-03-31 | 2021-10-01 | 阿里巴巴集团控股有限公司 | Video stream processing method and device, electronic equipment and computer readable medium |
| CN112291483A (en) * | 2020-10-29 | 2021-01-29 | 京东方科技集团股份有限公司 | Video pushing method and system, electronic equipment and readable storage medium |
| CN113408380A (en) * | 2021-06-07 | 2021-09-17 | 深圳小湃科技有限公司 | Video image adjusting method, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110298413B (en) | Image feature extraction method and device, storage medium and electronic equipment | |
| CN110413812B (en) | Neural network model training method and device, electronic equipment and storage medium | |
| CN112364860B (en) | Training method and device of character recognition model and electronic equipment | |
| CN110826567B (en) | Optical character recognition method, device, equipment and storage medium | |
| CN111784712B (en) | Image processing method, device, equipment and computer readable medium | |
| CN110781373B (en) | List updating method and device, readable medium and electronic equipment | |
| CN111459364B (en) | Icon updating method and device and electronic equipment | |
| CN110516159A (en) | A kind of information recommendation method, device, electronic equipment and storage medium | |
| CN113449070A (en) | Multimodal data retrieval method, device, medium and electronic equipment | |
| CN111968648B (en) | Voice recognition method and device, readable medium and electronic equipment | |
| CN112257478A (en) | Code scanning method, device, terminal and storage medium | |
| CN113191257B (en) | Order of strokes detection method and device and electronic equipment | |
| CN110378282B (en) | Image processing method and device | |
| CN111311358A (en) | Information processing method and device and electronic equipment | |
| CN113628097A (en) | Image special effect configuration method, image recognition method, image special effect configuration device and electronic equipment | |
| CN111324405A (en) | Character display method and device and electronic equipment | |
| CN112418233B (en) | Image processing method and device, readable medium and electronic equipment | |
| CN115269978A (en) | Video tag generation method, device, equipment and medium | |
| CN114170553A (en) | Image processing method and device and electronic equipment | |
| CN114004229A (en) | Text recognition method and device, readable medium and electronic equipment | |
| CN111399730A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111680754A (en) | Image classification method and device, electronic equipment and computer-readable storage medium | |
| CN113240108A (en) | Model training method and device and electronic equipment | |
| CN111538552A (en) | Gift panel configuration method and device, readable medium and electronic equipment | |
| CN111258670B (en) | Method and device for managing component data, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |