CN116483549B - Task scheduling method and device for intelligent camera system, camera and storage medium - Google Patents
Task scheduling method and device for intelligent camera system, camera and storage medium Download PDFInfo
- Publication number
- CN116483549B CN116483549B CN202310747625.3A CN202310747625A CN116483549B CN 116483549 B CN116483549 B CN 116483549B CN 202310747625 A CN202310747625 A CN 202310747625A CN 116483549 B CN116483549 B CN 116483549B
- Authority
- CN
- China
- Prior art keywords
- task
- processed
- priority
- camera system
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012545 processing Methods 0.000 claims abstract description 86
- 238000001514 detection method Methods 0.000 claims description 25
- 230000008569 process Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 6
- 230000004044 response Effects 0.000 abstract description 16
- 238000010586 diagram Methods 0.000 description 7
- 238000005286 illumination Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 239000000047 product Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 239000012467 final product Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the technical field of intelligent cameras, and provides a task scheduling method and device of an intelligent camera system, a camera and a storage medium, wherein the method comprises the following steps: receiving a task to be processed issued by an application layer; determining task priority corresponding to the task to be processed according to task processing parameters of the task to be processed; according to the task priority, dynamically updating task priority queues corresponding to all the tasks to be processed currently and the tasks to be executed currently in the camera system; and scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues. According to the method, the corresponding task priority is calculated through the task processing parameters of the task to be processed, so that the dynamic scheduling of the task including the currently executed task in the camera system is realized, the response time of the camera system is shortened, and the resource utilization rate of the camera system is improved.
Description
Technical Field
The present invention relates to the technical field of intelligent cameras, and in particular, to a task scheduling method and apparatus for an intelligent camera, a camera, and a storage medium.
Background
At present, embedded software task scheduling generally adopts a preemptive method of first-come-first-obtained, that is, a task which acquires computing resources first will monopolize computing resources until the operation is finished. The traditional method has the advantages of being simple to implement, and ensuring the computing efficiency and the resource utilization rate to a certain extent when the computing resources are abundant.
However, for most embedded camera platforms, due to energy consumption, cost, volume and other considerations, only a single-core dedicated computing acceleration hardware, such as a single-core GPU or DSP, often has a situation of insufficient computing resources, and at this time, if a simple preemptive-by-first-come scheduling method is used, the system may wait for some tasks with low real-time requirements, long operation time and low priority to finish, but cannot perform tasks with high real-time requirements, short operation time and high priority in time, thereby seriously reducing the response time and resource utilization rate of the system, causing problems of frame loss, long delay and the like, and affecting the use experience of the user on the product.
Therefore, it is necessary to solve the problems of insufficient response capability, long time consumption and low utilization rate of system resources of the camera system in the conventional task scheduling method of the camera system.
Disclosure of Invention
The invention provides a task scheduling method, a task scheduling device, a camera and a storage medium of an intelligent camera system, which are used for overcoming the defects of insufficient response capability, long time consumption and low system resource utilization rate of a camera system in the conventional task scheduling method of the camera system, accelerating the response time of the camera system and improving the resource utilization rate.
In one aspect, the present invention provides a task scheduling method for an intelligent camera system, including: receiving a task to be processed issued by an application layer; determining a task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed; according to the task priority, dynamically updating task priority queues corresponding to all tasks to be processed currently and tasks to be executed currently in the camera system; and scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues.
Further, the determining the task priority corresponding to the task to be processed according to the task processing parameter of the task to be processed includes: determining task processing parameters corresponding to the task to be processed; according to the task processing parameters and the weight values corresponding to the task processing parameters, calculating to obtain task priorities corresponding to the tasks to be processed; the task processing parameters at least comprise the triggering frequency of the task to be processed, the task waiting time, the task remaining time and the task switching loss time.
Further, the calculating, according to the task processing parameters and the weight values corresponding to the task processing parameters, the task priority corresponding to the task to be processed includes: acquiring the surrounding environment perceived by the intelligent camera system, and hardware attribute and load attribute of the intelligent camera system; and dynamically acquiring a weight value corresponding to the task processing parameter according to the surrounding environment, the hardware attribute and the load attribute.
Further, the dynamically updating the task priority queues corresponding to all the tasks to be processed currently and the tasks to be executed currently in the camera system according to the task priorities includes: and determining that the task waiting time length of the task to be processed reaches the set time length, and improving the priority ordering of the task to be processed in the task priority queue.
Further, the scheduling of all the current tasks to be processed and the current tasks to be executed based on the updated task priority queue includes: acquiring the task priority of a currently executed task, wherein the task priority does not belong to the highest priority of a task priority queue; determining that the task switching loss time length of the currently executed task is lower than a set threshold value, and/or determining that the task remaining time length of the currently executed task is higher than a time length threshold value; interrupting the current executing task, switching to a task to be processed corresponding to the highest priority, and starting to execute.
Further, a formula for calculating the task priority corresponding to the task to be processed is as follows:
；
wherein, the liquid crystal display device comprises a liquid crystal display device,for the task priority of the task to be processed, +.>For the trigger frequency of the task to be processed, +.>Task waiting time for a task to be processed, +.>The remaining time length for the task to be processed, +.>Lost time for task switching of the task to be processed, < >>Respectively->And +.>Corresponding weight values.
Further, the task to be processed comprises one or more of a camera control task, an image processing task, an image enhancement task, a full-image target detection task, a target tracking task, a key point positioning task, a target snapshot task, a target identification task and a feature comparison task.
In a second aspect, the present invention further provides a task scheduling device of an intelligent camera system, including: the task to be processed receiving module is used for receiving the task to be processed issued by the application layer; the task priority determining module is used for determining the task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed; the task priority queue updating module is used for dynamically updating task priority queues corresponding to all the current tasks to be processed and the current executing tasks in the camera system according to the task priorities; and the task scheduling module is used for scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queue.
In a third aspect, the present invention further provides a camera, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the task scheduling method of the smart camera system as described in any one of the above when executing the program.
In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a task scheduling method of a smart camera system as described in any of the above.
According to the task scheduling method of the intelligent camera system, task priorities corresponding to the tasks to be processed are determined according to task processing parameters of the tasks to be processed by receiving the tasks to be processed issued by an application layer, and task priority queues corresponding to all the tasks to be processed currently and the tasks to be executed currently in the camera system are dynamically updated according to the task priorities, so that all the tasks to be processed currently and the tasks to be executed currently are scheduled based on the updated task priority queues. According to the method, the corresponding task priority is calculated through the task processing parameters of the task to be processed, so that the dynamic scheduling of the task including the currently executed task in the camera system is realized, the response time of the camera system is shortened, and the resource utilization rate of the camera system is improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a logic framework of a task scheduling method of an intelligent camera system provided by the invention;
fig. 2 is a flow chart of a task scheduling method of the intelligent camera system provided by the invention;
fig. 3 is a schematic structural diagram of a task scheduling device of the intelligent camera system provided by the invention;
fig. 4 is a schematic structural diagram of a camera according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, in the existing embedded software task scheduling, a preemptive method of first-come-first-get is generally adopted, that is, the task that first acquires the computing resource will monopolize the computing resource until the operation is finished.
The traditional method has the advantages that the implementation is simple, and when the computing resources are abundant, the computing efficiency and the resource utilization rate can be ensured to a certain extent; however, when the computing resources are insufficient, the scheduling method may cause the system to wait for the completion of some tasks with low real-time requirements, long operation time and low priority, but cannot perform tasks with high real-time requirements, short operation time and high priority in time, so that the response time and the resource utilization rate of the system are seriously reduced, the problems of frame loss, long delay and the like are caused, and the use experience of users on products is affected.
In consideration of the task scheduling method, the invention provides a new task scheduling method, namely a task scheduling method of an intelligent camera system.
Before describing the task scheduling method of the intelligent camera system, first, an overall logic framework for implementing the task scheduling method is introduced,
specifically, fig. 1 shows a schematic diagram of a logic framework of a task scheduling method of an intelligent camera system provided by the invention.
The task scheduling method of the intelligent camera system is realized under the framework of the intelligent camera system. As shown in fig. 1, the logic framework of the task scheduling method of the intelligent camera system is composed of three parts, namely an application layer, a scheduling layer and a computing resource layer, and the application layer issues a task processing request to the scheduling layer, and the scheduling layer performs task scheduling on all tasks to be processed in the camera system, including the currently executed task, based on the bottom computing resource, so as to speed up the response time of the camera system and improve the resource utilization rate of the camera system.
In the scheduling process, the scheduling layer mainly considers two factors of the level, namely environment awareness and hardware state awareness.
Specifically, on one hand, the intelligent camera system can dynamically sense the surrounding environment, so that adjustment of a scheduling strategy is performed according to the sensed surrounding environment, and the adjustment of the scheduling strategy is mainly characterized in that corresponding weight parameters are dynamically adjusted according to different surrounding environments such as illumination.
In another aspect, the smart camera system has different hardware characteristics and loads, and the smart camera system may load balance among multiple hardware.
For example, a smart camera system comprises a DSP (Digital Signal Processing, digital signal processor) and an ARM, and different tasks to be processed are executed with different hardware efficiency, which is represented by the task remaining duration Tr hereinafter.
If one hardware is heavy in load, the corresponding weight parameter can be dynamically improved, so that the task to be processed can be executed on other hardware more, namely, the scheduling layer can intelligently adjust the scheduling strategy according to the hardware characteristics and the hardware load of the intelligent camera system.
The task scheduling method of the intelligent camera system provided by the invention is developed by taking the scheduling layer as an execution main body, and specifically, fig. 2 shows a flow diagram of the task scheduling method of the intelligent camera system provided by the invention.
As shown in fig. 2, the method includes:
s210, receiving a task to be processed issued by an application layer.
It should be noted that the task scheduling method provided by the invention is applicable to intelligent camera systems. In a specific embodiment, the task scheduling method is applied to a face recognition camera system.
Different from the common camera system, the intelligent camera system has more kinds and the more number of the tasks to be processed, and each task to be processed is mutually connected, so that a certain processing logic relationship exists.
It can be understood that when the application layer has a task processing requirement, a task processing request is issued to the scheduling layer, according to the task processing request, a task to be processed that needs to be processed can be obtained, and after the scheduling layer receives the task to be processed issued by the application layer, step S220 is executed.
And receiving the task to be processed issued by the application layer, wherein the task to be processed issued by the application layer can be received at the same time, the tasks to be processed issued by the application layer can be received at the same time, the task to be processed issued by the application layer can be received according to the set time, and the task to be processed issued by the application layer can be received in real time.
The task to be processed may be any task to be processed in the camera system, and may be specifically determined according to actual requirements, which is not specifically limited herein.
For example, in one particular embodiment, the tasks to be processed include one or more of a camera control task, an image processing task, an image enhancement task, a full-view object detection task, an object tracking task, a key point positioning task, an object snapshot task, an object recognition task, and a feature alignment task.
In another specific embodiment, the intelligent camera system is a face recognition camera system, and the task to be processed includes one or more of a camera control task, an image processing task, an image enhancement task, a full-image face detection task, a face tracking task, a key point positioning task, a face snapshot task, a face recognition task, and a feature comparison task.
S220, determining the task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed.
It can be understood that, further, based on receiving the task to be processed issued by the application layer in step S210, the task priority corresponding to each task to be processed is calculated according to the task processing parameters of the task to be processed.
Specifically, each task to be processed in the camera system has corresponding task processing parameters, the number of the task processing parameters can be one or more, and according to the task processing parameters, the task priority corresponding to each task to be processed can be calculated by combining the weight values corresponding to the parameters.
The task processing parameter may be any parameter related to task processing from the time of receiving the task to the time of ending the task execution, for example, a task processing duration, a task waiting duration, and a trigger frequency, which are not limited herein.
The task priority is a determined value for describing the degree of importance of a plurality of tasks to be processed in the camera system, i.e. the processed priority.
S230, dynamically updating task priority queues corresponding to all the tasks to be processed currently and the tasks to be executed currently in the camera system according to the task priorities.
It can be understood that, based on determining the task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed in step S220, further, according to the task priority obtained by calculation, dynamic update of the task priority queue can be performed on all tasks in the camera system, including the task to be processed and the task currently executed.
Specifically, for each task to be processed issued by the application layer, the corresponding task priority is calculated, so that task priorities of all tasks in the camera system can be ordered according to the task priorities, and an updated task priority queue is obtained.
It should be noted that, before the latest task to be processed is issued to the scheduling layer, there is a task priority queue in the camera system, and after the task priority queue is dynamically updated, the latest task to be processed may be arranged at the end of the task priority queue which is not updated, may be arranged in the task priority queue which is not updated, and may also be arranged at the head of the task priority queue which is not updated.
It should be further noted that, the update of the task priority queue is a dynamic adjustment process, not only aiming at the scheduling of the task to be processed, but also aiming at the currently executed task, the execution of the currently executed task can be interrupted according to the actual situation.
S240, scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues.
It can be understood that, based on dynamically updating the task priority queues corresponding to all the current tasks to be processed and the current tasks to be executed in the camera system in step S230, further, the current tasks to be processed and the current tasks to be executed may be scheduled based on the updated task priority queues.
Specifically, as described above, before the latest task to be processed is issued to the scheduling layer, there is a task priority queue in the camera system, and after the task priority queue is dynamically updated, the latest task to be processed may be arranged at the end of the non-updated task priority queue, may be arranged in the non-updated task priority queue (without a queue header), and may be arranged at the head of the non-updated task priority queue.
When the latest task to be processed is positioned at the tail end of the priority queue of the non-updated task, the latest task to be processed does not influence the scheduling of each previous task to be processed, and the camera system schedules all the previous tasks in the camera system according to the priority queue of the non-updated task and processes the latest task to be processed finally.
When the latest task to be processed is arranged in the priority queue of the task which is not updated, the latest task to be processed affects the priority ordering of the prior task to be processed, but does not affect the execution of the currently executed task, the camera system can continue to execute the currently executed task, and all the tasks to be processed in the camera system are scheduled according to the updated task priority queue.
When the latest task to be processed is arranged at the head of the priority queue of the non-updated task, the task priority of the latest task to be processed is the highest, at this time, all tasks in the priority queue of the non-updated task may be affected, whether the current executing task is interrupted or not needs to be judged according to actual conditions, and the latest task to be processed is switched to and executed, so that the scheduling of all the tasks to be processed in the camera system, namely the current executing task, is completed.
In this embodiment, by receiving a task to be processed issued by an application layer, determining a task priority corresponding to the task to be processed according to task processing parameters of the task to be processed, dynamically updating task priority queues corresponding to all current tasks to be processed and current executing tasks in a camera system according to the task priorities, and further scheduling all current tasks to be processed and current executing tasks based on the updated task priority queues. According to the method, the corresponding task priority is calculated through the task processing parameters of the task to be processed, so that the dynamic scheduling of the task including the currently executed task in the camera system is realized, the response time of the camera system is shortened, and the resource utilization rate of the camera system is improved.
On the basis of the above embodiment, further, determining, according to task processing parameters of the task to be processed, a task priority corresponding to the task to be processed includes: determining task processing parameters corresponding to a task to be processed; and calculating to obtain the task priority corresponding to the task to be processed according to the task processing parameters and the weight values corresponding to the task processing parameters.
It can be understood that, according to task processing parameters of the task to be processed, task priorities corresponding to the task to be processed are calculated, specifically, first task processing parameters of the task to be processed are determined, each task processing parameter corresponds to a corresponding weight value, and then according to the task processing parameters and the corresponding weight values, the task priorities of the task to be processed are calculated, so as to obtain the task priorities corresponding to the task to be processed.
The task processing parameters at least comprise the triggering frequency of the task to be processed, the task waiting time, the task remaining time and the task switching loss time.
The trigger frequency of the task to be processed is determined in advance according to the requirements of the final product. For example, face cameras require fluency in video images, at least 25 frames per second, and the real-time nature of the image processing task is at most 40ms. As another example, face recognition tasks require at least 1 per second, and the real-time performance of such tasks is 1000ms.
The task waiting time length and the task remaining time length are dynamically updated in the scheduling process, wherein the task waiting time length refers to the waiting time length of the task to be processed in the task priority queue, and the task remaining time length refers to how long the task to be processed can complete the task.
The task switch loss time is related to the hardware capability of the camera system and is determined in advance. For example, using shared memory DSP or GPU (Graphics Processing Unit, graphics processor) hardware, task switching loss times are relatively short; if the shared memory is not available, the data needs to be copied from the main memory during task switching, and the task switching loss time is longer.
In a development, when calculating the task priority corresponding to the task to be processed, the following aspects may be considered:
(1) Different tasks to be processed have different trigger frequencies. For example, at a speed of 30 frames of images per second, the image processing task may request 30 times per second; the full-image face detection task requests 2-3 times per second; the face tracking task requests 30 times per second; the face recognition task requests 1 time per second.
Taking an image processing task as an example, the image processing task needs to complete calculation within 30ms to process the next frame of image, otherwise, the phenomenon of frame loss and jitter of the image occurs.
That is, the higher the trigger frequency of the task to be processed, the higher the real-time requirement of the task, and the higher the task priority can be obtained.
(2) Different tasks to be processed have different times when processing and when waiting for processing.
On the one hand, the tasks to be processed, when being processed, require different processing lengths. For example, the image processing task processing needs to occupy about 20ms of the DSP, the full-image face detection task processing needs to occupy about 100ms of the DSP, the ARM is about 100ms, and the face tracking task processing needs to occupy about 10ms of the ARM.
If the task to be processed is interrupted during execution, the processing time at the moment is the residual time of the task. The shorter the required length for processing the task to be processed, the higher the task priority it gets.
In another aspect, from the time when the task to be processed is received by the self-scheduling layer to the time when the task to be processed is executed, each task to be processed has a corresponding task waiting duration. The longer the task waiting time is, the higher the task priority can be according to the set rule, so as to avoid the long-time low-priority task from being unable to be executed.
(3) The length of time it takes to interrupt the currently executing task (i.e., the task switch lost length) also needs to be taken into consideration.
Specifically, the time loss of task switching corresponding to the current execution task is interrupted, including the time length of saving the site and the time length of loading the new task codes and data.
It should be noted that, the longer the task switching loss time length is, the greater the possibility that the task switching loss time length is not interrupted, so as to avoid the problem that the calculation process is frequently interrupted and repeated switching occurs, the weight value corresponding to the task switching loss time length can be properly adjusted.
Based on the above, according to the task processing parameters and the weight values corresponding to the task processing parameters, the task priority corresponding to the task to be processed is calculated, specifically, the formula for calculating the task priority corresponding to the task to be processed is as follows:
；
wherein, the liquid crystal display device comprises a liquid crystal display device,for the task priority of the task to be processed, +.>For the trigger frequency of the task to be processed, +.>Task waiting time for a task to be processed, +.>The remaining time length for the task to be processed, +.>Lost time for task switching of the task to be processed, < >>Respectively->And +.>Corresponding weight values.
It should be noted that the number of the substrates,weight value +.>Determining from the lighting environment, including->Weight value +.>Determining +.>Weight value +.>Determined from camera system stability.
In one embodiment, according to the task processing parameters and the weight values corresponding to the task processing parameters, the task priority corresponding to the task to be processed is calculated, which includes: acquiring the surrounding environment perceived by the intelligent camera system, and the hardware attribute and the load attribute of the intelligent camera system; and dynamically acquiring a weight value corresponding to the task processing parameter according to the surrounding environment, the hardware attribute and the load attribute.
It can be understood that the surrounding environment perceived by the intelligent camera system is obtained, specifically, the intelligent camera system can dynamically perceive the surrounding environment, so that the weight value of the corresponding parameter to be processed is dynamically adjusted, specifically, the corresponding weight value is increased or decreased, according to the perceived surrounding environment.
Wherein the ambient environment includes, but is not limited to, illumination intensity, illumination direction, and illumination color.
The method comprises the steps of obtaining hardware attributes and load attributes of an intelligent camera system, specifically, the intelligent camera system has different hardware attributes and load attributes, and carrying out load balancing on multiple hardware in the camera system according to the hardware attributes and the load attributes perceived in real time so as to better schedule tasks to be processed in the intelligent camera system, thereby improving the resource utilization rate of the intelligent camera system.
And dynamically acquiring a weight value corresponding to the task processing parameter according to the surrounding environment, the hardware attribute and the load attribute.
For example, in a camera system with higher real-time requirements, the value of the weight can be adjusted to be high, so that more execution opportunities can be obtained for the task to be processed.
For example, when the illumination environment is poor, the calculation instantaneity of the camera control algorithm can be improved, so that the algorithms such as exposure enhancement, multi-frame wide dynamic and the like can obtain higher task priority, and the adaptability of the camera to the illumination environment is improved.
For another example, when there is a high requirement on the face recognition speed, the weight value may be increased to improve the response time of face detection, key point positioning, snap shot, and face recognition.
It should be noted that, in the above formula, R is the trigger frequency of the task to be processed, and different trigger frequencies also represent real-time performance of the task to be processed, so that the parameter may be replaced by corresponding real-time performance.
In the embodiment, the task priority corresponding to the task to be processed is obtained by determining the task processing parameters corresponding to the task to be processed and calculating according to the task processing parameters and the weight values corresponding to the task processing parameters, so that various different application scenes of the intelligent camera system are fully considered, dynamic and flexible scheduling of the tasks including the currently executed task in the camera system is realized, different scheduling requirements can be met, response time of the camera system is shortened, and resource utilization rate of the camera system is improved.
Based on the above embodiment, further, according to the task priority, dynamically updating task priority queues corresponding to all currently pending tasks and currently executing tasks in the camera system includes: and determining that the task waiting time length of the task to be processed reaches the set time length, and improving the priority ordering of the task to be processed in the task priority queue.
It will be appreciated that there may be some tasks to be processed that are prioritized in the task priority queue and therefore may not be available for execution for a long period of time due to the lower task priority.
In consideration of this, in this embodiment, when the task waiting time length of the task to be processed reaches the set time length, the priority ordering of the task to be processed in the task priority queue is promoted.
The set duration may be set according to practical situations, for example, in a specific embodiment, the set duration is 1 minute.
The set duration may be set to be multiple, different set durations correspond to different priority ranks, specifically, the longer the set duration is, the higher the priority ranks of the tasks to be processed can be lifted.
In this embodiment, by improving the priority ordering of the task to be processed in the task priority queue under the condition that it is determined that the task waiting time of the task to be processed reaches the set time, the problem that some low-priority tasks cannot wait for a long time to obtain the opportunity to be executed can be effectively avoided.
Based on the above embodiment, further, based on the updated task priority queue, scheduling all the tasks to be processed and the tasks to be executed currently includes: acquiring the task priority of a currently executed task, wherein the task priority does not belong to the highest priority of a task priority queue; determining that the task switching loss time length of the currently executed task is lower than a set threshold value, and/or determining that the residual time length of the currently executed task is higher than a time length threshold value; interrupting the current execution task, switching to the task to be processed corresponding to the highest priority, and starting to execute.
It can be understood that when the priority of the latest issued task to be processed is higher than the priorities of all tasks to be processed issued previously and the tasks currently executed, whether the tasks currently executed are interrupted or not needs to be judged according to the actual situation, and the latest task to be processed is switched to be executed, so that the scheduling of all the tasks to be processed in the camera system, namely the tasks currently executed, is completed.
Specifically, determining a current execution task, acquiring a task priority corresponding to the current execution task, and judging whether to interrupt the current execution task under the condition that the task priority of the current execution task does not belong to the highest priority of the updated task priority queue.
In an embodiment, the task switching duration of the currently executed task is taken as a consideration, when the task switching loss duration of the currently executed task is lower than a set threshold, the currently executed task is interrupted, the task to be processed corresponding to the highest priority is switched, and execution is started.
The setting threshold may be set according to actual situations, and is not specifically limited herein. For example, in one specific embodiment, the threshold is set to 9ms.
In another embodiment, the task remaining time length of the currently executed task is taken as a consideration, when the task remaining time length of the currently executed task is higher than a time length threshold, the currently executed task is interrupted, and the task to be processed corresponding to the highest priority is switched to and starts to be executed.
The time period threshold may be set according to practical situations, and is not specifically limited herein. For example, in one particular embodiment, the duration threshold is 1 second.
In yet another embodiment, the task remaining time length and the task switching loss time length of the currently executed task are taken as consideration at the same time, when the task switching loss time length of the currently executed task is lower than a set threshold value and the corresponding task remaining time length is higher than the time length threshold value, the currently executed task is interrupted, and the task to be processed corresponding to the highest priority is switched to and starts to be executed.
In this embodiment, by acquiring the task priority of the currently executing task, where the task priority does not belong to the highest priority in the task priority queue, under the condition that it is determined that the task switching duration of the currently executing task is lower than the set threshold and/or the remaining time duration of the currently executing task is higher than the duration threshold, the currently executing task is interrupted, and the task to be processed corresponding to the highest priority is switched to and starts to be executed, so that dynamic and flexible scheduling of tasks including the currently executing task in the camera system is realized, response time of the camera system is accelerated, and resource utilization rate of the camera system is improved.
Based on the above embodiment, further, based on the updated task priority queue, scheduling all the tasks to be processed and the tasks to be executed currently includes: determining the current execution task as a full-graph target detection task, and reducing the trigger frequency of the current execution task to a set trigger frequency; and detecting the target by combining the target tracking task.
It will be appreciated that any scheduling policy that adjusts the order of execution by task priority will allow higher priority tasks to execute faster, lower priority tasks to execute slower, and the trigger frequency of the full graph object detection tasks is lower, and the time required for their corresponding processing is longer, resulting in lower corresponding task priorities.
In this regard, in order to make up for the problem that the response time of the full-view object detection task is too long, the processing procedure of the full-view object detection task is improved.
Specifically, under the condition that the camera system is determined to be executing the full-image target detection task, the triggering frequency of the currently executed full-image target detection task is reduced to the set triggering frequency, and a simplified target tracking task with weaker functions and lower precision is added, so that the target tracking task is matched with the slow full-image target detection task, and the target tracking task carries out small-range rapid target detection around the target based on the latest full-image detection result, so that the real-time target detection effect can be realized, and the problem of overlong response time of the full-image target detection task is solved.
The setting trigger frequency may be set according to actual situations, and is not specifically limited herein.
For example, in a specific embodiment, under the condition that it is determined that the camera system is executing the full-image target detection task, the trigger frequency of the full-image target detection task is reduced to 1 time per second, the corresponding task priority is lower, other tasks to be processed are executed with saved time and calculation force, and a fast and low-precision target tracking algorithm is added at the same time, and the task of target detection is completed by running 30 times per second, so as to compensate the problem that the detection time is slow.
In this embodiment, the trigger frequency of the currently executed task is reduced to the set trigger frequency by determining that the currently executed task is a full-image target detection task, and then the target is detected in combination with the target tracking task. The method solves the problem of overlong response time of the full-image target detection task by improving the processing process of the full-image target detection task.
Fig. 3 shows a schematic structural diagram of a task scheduling device of a camera system provided by the invention. As shown in fig. 3, the apparatus includes: a task to be processed receiving module 310, configured to receive a task to be processed issued by an application layer; a task priority determining module 320, configured to determine a task priority corresponding to the task to be processed according to a task processing parameter of the task to be processed; a task priority queue updating module 330, configured to dynamically update task priority queues corresponding to all tasks to be processed currently and tasks to be executed currently in the camera system according to the task priorities; and a task scheduling module 340, configured to schedule all the current tasks to be processed and the current executing tasks based on the updated task priority queue.
In this embodiment, the task to be processed issued by the application layer is received by the task to be processed receiving module 310, the task priority determining module 320 determines the task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed, the task priority queue updating module 330 dynamically updates the task priority queues corresponding to all the tasks to be processed currently and the tasks to be executed currently in the camera system according to the task priorities, and the task scheduling module 340 schedules all the tasks to be processed currently and the tasks to be executed currently based on the updated task priority queues. The device calculates the corresponding task priority through the task processing parameters of the task to be processed, so that the dynamic scheduling of the task including the currently executed task in the camera system is realized, the response time of the camera system is shortened, and the resource utilization rate of the camera system is improved.
It should be noted that, the task scheduling device of the intelligent camera system provided by the present invention and the task scheduling method of the intelligent camera system described above may be referred to correspondingly, and are not described herein again.
Fig. 4 illustrates a physical structure diagram of a camera, as shown in fig. 4, which may include: processor 410, communication interface (communications Interface) 420, memory 430 and communication bus 440, wherein processor 410, communication interface 420 and memory 430 communicate with each other via communication bus 440. The processor 410 may invoke logic instructions in the memory 430 to perform a task scheduling method of the smart camera system, the method comprising: receiving a task to be processed issued by an application layer; determining a task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed; according to the task priority, dynamically updating task priority queues corresponding to all tasks to be processed currently and tasks to be executed currently in the camera system; and scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues.
Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the task scheduling method of the smart camera provided by the above methods, the method comprising: receiving a task to be processed issued by an application layer; determining a task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed; according to the task priority, dynamically updating task priority queues corresponding to all tasks to be processed currently and tasks to be executed currently in the camera system; and scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A task scheduling method for an intelligent camera system, comprising:
receiving a task to be processed issued by an application layer;
determining task priority corresponding to the task to be processed according to task processing parameters of the task to be processed, wherein the task processing parameters at least comprise trigger frequency, task waiting time, task remaining time and task switching loss time of the task to be processed;
according to the task priority, dynamically updating task priority queues corresponding to all current tasks to be processed and current executing tasks in a camera system, wherein the updating of the task priority queues is a dynamic adjustment process, and in the process, the execution of the current executing tasks can be interrupted according to actual conditions for the tasks to be processed and the current executing tasks;
Scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues, and adjusting a scheduling strategy through environment awareness and hardware state awareness in the scheduling process;
when the latest task to be processed is positioned at the tail end or the middle of the priority queue of the task which is not updated, the current executing task is not interrupted;
when the latest task to be processed is positioned at the head of the task priority queue which is not updated, interrupting the currently executed task;
the task priority queues which are not updated are task priority queues corresponding to all tasks in the camera system before the latest task to be processed is issued.
2. The task scheduling method of the intelligent camera system according to claim 1, wherein the determining, according to the task processing parameter of the task to be processed, a task priority corresponding to the task to be processed includes:
determining task processing parameters corresponding to the task to be processed;
and calculating to obtain the task priority corresponding to the task to be processed according to the task processing parameters and the weight values corresponding to the task processing parameters.
3. The task scheduling method of the intelligent camera system according to claim 2, wherein the calculating, according to the task processing parameters and the weight values corresponding to the task processing parameters, the task priority corresponding to the task to be processed includes:
Acquiring the surrounding environment perceived by the intelligent camera system, and hardware attribute and load attribute of the intelligent camera system;
and dynamically acquiring a weight value corresponding to the task processing parameter according to the surrounding environment, the hardware attribute and the load attribute.
4. The task scheduling method of the intelligent camera system according to claim 1, wherein dynamically updating task priority queues corresponding to all currently pending tasks and currently executing tasks in the camera system according to the task priorities includes:
and determining that the task waiting time length of the task to be processed reaches the set time length, and improving the priority ordering of the task to be processed in the task priority queue.
5. The task scheduling method of the intelligent camera system according to claim 1, wherein the scheduling of all the current tasks to be processed and the current tasks to be executed based on the updated task priority queue includes:
acquiring the task priority of a currently executed task, wherein the task priority does not belong to the highest priority of a task priority queue;
determining that the task switching loss time length of the currently executed task is lower than a set threshold value, and/or determining that the task remaining time length of the currently executed task is higher than a time length threshold value;
Interrupting the current executing task, switching to a task to be processed corresponding to the highest priority, and starting to execute.
6. The task scheduling method of an intelligent camera system according to claim 1, wherein a formula for calculating a task priority corresponding to the task to be processed is as follows:
the method comprises the steps of determining a task priority of a task to be processed, triggering frequency of the task to be processed, task waiting time of the task to be processed, task remaining time of the task to be processed, task switching loss time of the task to be processed, and corresponding weight values.
7. The task scheduling method of an intelligent camera system according to any one of claims 1 to 6, wherein the task to be processed includes one or more of a camera control task, an image processing task, an image enhancement task, a full-view target detection task, a target tracking task, a key point positioning task, a target snapshot task, a target recognition task, and a feature comparison task.
8. A task scheduling device of an intelligent camera system, comprising:
the task to be processed receiving module is used for receiving the task to be processed issued by the application layer;
The task priority determining module is used for determining the task priority corresponding to the task to be processed according to the task processing parameters of the task to be processed, wherein the task processing parameters at least comprise the triggering frequency, the task waiting time, the task remaining time and the task switching loss time of the task to be processed;
the task priority queue updating module is used for dynamically updating task priority queues corresponding to all current tasks to be processed and current executing tasks in the camera system according to the task priorities, wherein the updating of the task priority queues is a dynamic adjustment process, and in the process, the execution of the current executing tasks can be interrupted according to actual conditions for the tasks to be processed and the current executing tasks at the same time;
the task scheduling module is used for scheduling all the current tasks to be processed and the current executing tasks based on the updated task priority queues, and the scheduling strategy is adjusted through environment awareness and hardware state awareness in the scheduling process;
when the latest task to be processed is positioned at the tail end or the middle of the priority queue of the task which is not updated, the current executing task is not interrupted;
When the latest task to be processed is positioned at the head of the task priority queue which is not updated, interrupting the currently executed task;
the task priority queues which are not updated are task priority queues corresponding to all tasks in the camera system before the latest task to be processed is issued.
9. A camera comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the task scheduling method of the smart camera system of any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the task scheduling method of the smart camera system of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310747625.3A CN116483549B (en) | 2023-06-25 | 2023-06-25 | Task scheduling method and device for intelligent camera system, camera and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310747625.3A CN116483549B (en) | 2023-06-25 | 2023-06-25 | Task scheduling method and device for intelligent camera system, camera and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116483549A CN116483549A (en) | 2023-07-25 |
| CN116483549B true CN116483549B (en) | 2023-09-19 |
Family
ID=87221828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310747625.3A Active CN116483549B (en) | 2023-06-25 | 2023-06-25 | Task scheduling method and device for intelligent camera system, camera and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116483549B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116795517B (en) * | 2023-08-25 | 2023-11-07 | 中国人民解放军国防科技大学 | Multi-strategy self-adaptive asynchronous task scheduling method, system and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8203956B1 (en) * | 2008-08-28 | 2012-06-19 | Raytheon Bbn Technologies Corp. | Method and apparatus providing a precedence drop quality of service (PDQoS) |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN107423120A (en) * | 2017-04-13 | 2017-12-01 | 阿里巴巴集团控股有限公司 | Method for scheduling task and device |
| CN111143045A (en) * | 2019-12-11 | 2020-05-12 | 青岛海尔科技有限公司 | Task scheduling method and device of smart home operating system and storage medium |
| CN113535367A (en) * | 2021-09-07 | 2021-10-22 | 北京达佳互联信息技术有限公司 | Task scheduling method and related device |
| CN113900821A (en) * | 2021-10-27 | 2022-01-07 | Oppo广东移动通信有限公司 | Task processing method and device, storage medium and electronic equipment |
| CN114579285A (en) * | 2022-04-29 | 2022-06-03 | 武汉深之度科技有限公司 | Task running system and method and computing device |
| CN114840349A (en) * | 2022-07-04 | 2022-08-02 | 东声(苏州)智能科技有限公司 | Distributed task scheduling method of AI (Artificial Intelligence) intelligent camera and AI intelligent camera system |
| CN115048201A (en) * | 2022-06-13 | 2022-09-13 | 青岛聚看云科技有限公司 | Server and task scheduling method based on interrupt |
| CN115237556A (en) * | 2022-06-23 | 2022-10-25 | 哲库科技(北京)有限公司 | Scheduling method and device, chip, electronic equipment and storage medium |
| CN115454589A (en) * | 2022-07-29 | 2022-12-09 | 天翼云科技有限公司 | Task scheduling method and device and Kubernetes scheduler |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8539498B2 (en) * | 2007-05-17 | 2013-09-17 | Alcatel Lucent | Interprocess resource-based dynamic scheduling system and method |
| US8146107B2 (en) * | 2007-07-10 | 2012-03-27 | Mitel Networks Corporation | Virtual machine environment for interfacing a real time operating system environment with a native host operating system |
-
2023
- 2023-06-25 CN CN202310747625.3A patent/CN116483549B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8203956B1 (en) * | 2008-08-28 | 2012-06-19 | Raytheon Bbn Technologies Corp. | Method and apparatus providing a precedence drop quality of service (PDQoS) |
| CN107102894A (en) * | 2017-04-07 | 2017-08-29 | 百度在线网络技术（北京）有限公司 | Method for scheduling task, device and system |
| CN107423120A (en) * | 2017-04-13 | 2017-12-01 | 阿里巴巴集团控股有限公司 | Method for scheduling task and device |
| CN111143045A (en) * | 2019-12-11 | 2020-05-12 | 青岛海尔科技有限公司 | Task scheduling method and device of smart home operating system and storage medium |
| CN113535367A (en) * | 2021-09-07 | 2021-10-22 | 北京达佳互联信息技术有限公司 | Task scheduling method and related device |
| CN113900821A (en) * | 2021-10-27 | 2022-01-07 | Oppo广东移动通信有限公司 | Task processing method and device, storage medium and electronic equipment |
| CN114579285A (en) * | 2022-04-29 | 2022-06-03 | 武汉深之度科技有限公司 | Task running system and method and computing device |
| CN115048201A (en) * | 2022-06-13 | 2022-09-13 | 青岛聚看云科技有限公司 | Server and task scheduling method based on interrupt |
| CN115237556A (en) * | 2022-06-23 | 2022-10-25 | 哲库科技(北京)有限公司 | Scheduling method and device, chip, electronic equipment and storage medium |
| CN114840349A (en) * | 2022-07-04 | 2022-08-02 | 东声(苏州)智能科技有限公司 | Distributed task scheduling method of AI (Artificial Intelligence) intelligent camera and AI intelligent camera system |
| CN115454589A (en) * | 2022-07-29 | 2022-12-09 | 天翼云科技有限公司 | Task scheduling method and device and Kubernetes scheduler |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116483549A (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10101910B1 (en) | Adaptive maximum limit for out-of-memory-protected web browser processes on systems using a low memory manager | |
| CN116483549B (en) | Task scheduling method and device for intelligent camera system, camera and storage medium | |
| US8370846B2 (en) | Task execution device and method | |
| US20160210174A1 (en) | Hybrid Scheduler and Power Manager | |
| EP3610352A1 (en) | Intra-frame real-time frequency control | |
| CN107506234B (en) | Virtual machine scheduling method and device | |
| KR101978457B1 (en) | Adjusting thread priority to improve throughput between peer-to-peer (p2p) devices | |
| CN109165103B (en) | Frame rate control method, device, terminal and storage medium | |
| CN112486642B (en) | Resource scheduling method, device, electronic equipment and computer readable storage medium | |
| CN109710330B (en) | Method and device for determining running parameters of application program, terminal and storage medium | |
| CN108681439A (en) | Uniform display methods based on frame per second control | |
| US8132171B2 (en) | Method of controlling thread access to a synchronization object | |
| EP3278220A1 (en) | Power aware scheduling and power manager | |
| CN114217993A (en) | Method, system, terminal device and storage medium for controlling thread pool congestion | |
| CN114490048A (en) | Task execution method and device, electronic equipment and computer storage medium | |
| CN111338803B (en) | Thread processing method and device | |
| JPH02300939A (en) | Semaphore operation system | |
| CN113434303A (en) | Batch-processed remote sensing image intelligent processing model prediction performance optimization system and method | |
| WO2023193527A1 (en) | Thread execution method and apparatus, electronic device, and computer-readable storage medium | |
| CN115686758B (en) | VirtIO-GPU performance controllable method based on frame statistics | |
| CN115952866A (en) | Inference method, computer equipment and medium for artificial intelligence inference framework | |
| CN114579279A (en) | Task scheduling method and device | |
| CN113783999B (en) | Method and device for controlling terminal display and computer readable medium | |
| CN113849282A (en) | OpenRESTY reverse proxy queuing processing method based on dynamic weight | |
| CN114546910A (en) | Access control method, device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |