CN112416570A

CN112416570A - Picture stream access method and device, picture processing system and electronic equipment

Info

Publication number: CN112416570A
Application number: CN202011106422.9A
Authority: CN
Inventors: 张亮; 李盼伟
Original assignee: Beijing Kuangshi Technology Co Ltd
Current assignee: Beijing Kuangshi Technology Co Ltd; Beijing Megvii Technology Co Ltd
Priority date: 2020-10-15
Filing date: 2020-10-15
Publication date: 2021-02-26

Abstract

The invention relates to a picture stream access method, a picture stream access device, a picture processing system and electronic equipment. The method comprises the steps of acquiring an index for representing the picture processing pressure of a picture processing system in real time; and then, dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the indexes and the corresponding preset indexes. In the process, when the image processing pressure of the image processing system is higher, the access quantity of the gateway layer to the image stream can be reduced, so that the normal operation of the image processing system can be ensured; when the image processing pressure of the image processing system is low, the access number of the gateway layer to the image stream can be increased, so that the image processing resources of the image processing system are fully utilized.

Description

Picture stream access method and device, picture processing system and electronic equipment

Technical Field

The application belongs to the field of data processing, and particularly relates to a picture stream access method, a picture stream access device, a picture processing system and electronic equipment.

Background

In an actual application scenario, a gateway layer of the picture processing system may receive a large number of picture synchronization requests within a period of time, that is, the gateway layer may have a situation of high concurrent requests, or may receive a small number of picture synchronization requests within a period of time. However, the processing capability of the conventional picture processing system for pictures accessed by the gateway layer is relatively constant, and when there is a high concurrent request to the gateway layer, the picture processing pressure of the entire system may be large, which further causes the system not to operate normally, and when there are few picture synchronization requests received by the gateway layer for a long time, the picture processing pressure of the entire system may be too low, which further causes the system not to fully utilize the picture processing capability thereof, which causes the system not to operate in an optimal state.

Disclosure of Invention

In view of the above, an object of the present application is to provide a picture stream access method, a picture processing system, an electronic device, and a computer-readable storage medium, which dynamically adjust the number of access to picture streams by a gateway layer according to the current picture processing capability of the picture processing system, so as to ensure that the picture processing system is kept in an optimal operation state on the premise of being able to operate normally.

The embodiment of the application is realized as follows:

in a first aspect, an embodiment of the present application provides a picture stream access method, which is applied to a service layer of a picture processing system, where the picture processing system further includes a gateway layer in communication connection with the service layer, and the gateway layer is used to access a picture stream, and the method includes: acquiring an index for representing the picture processing pressure of the picture processing system in real time; and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the index and the corresponding preset index.

In the process, the service layer can control the access strategy of the gateway layer to the picture stream according to the picture processing pressure of the picture processing system in real time, so that the access quantity to the picture stream is controlled, and the picture processing system is kept in the optimal running state as far as possible on the premise of normal running.

With reference to the embodiment of the first aspect, in a possible implementation manner, the picture processing system further includes an algorithm layer in communication connection with the service layer, where the index includes a first index for characterizing picture feature extraction capability of the algorithm layer, and a preset index corresponding to the first index includes a first index conventional threshold for characterizing conventional picture feature extraction capability of the algorithm layer and a first index maximum threshold for characterizing maximum picture feature extraction capability of the algorithm layer, where the first index conventional threshold is smaller than the first index maximum threshold; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index includes: and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation among the first index, the first index conventional threshold and the first index maximum threshold.

The stress condition of the picture processing system can be measured through the picture feature extraction capability corresponding to the algorithm layer.

With reference to the embodiment of the first aspect, in a possible implementation manner, the gateway layer includes a plurality of receiving threads for accessing the picture stream, a picture included in the picture stream has a priority set in advance, and the dynamically adjusting the access policy of the gateway layer to the picture stream according to a size relationship between the first index, the first index normal threshold, and the first index maximum threshold includes: when the first index is determined to be larger than the first index conventional threshold and smaller than the first index maximum threshold, controlling the gateway layer to close part of first target receiving threads; when the first index is determined to be larger than or equal to the first index maximum threshold value, controlling the gateway layer to close all first target receiving threads; the priority of the picture accessed by the first target receiving thread is lower than the priority of the pictures accessed by other receiving threads.

When the image processing pressure of the image processing system is higher, partial or all receiving threads used for accessing the low-priority images are closed, so that the number of access of a gateway layer to image streams can be reduced, the image processing pressure of the image processing system is reduced, and the image processing system can be ensured to operate normally.

With reference to the embodiment of the first aspect, in a possible implementation manner, after the controlling the gateway layer to close all target receiving threads, or after the controlling the gateway layer to close a part of first target receiving threads, the method further includes: controlling the gateway layer portion to restart/fully restart the first target receiving thread that has been shut down upon determining that the first metric is less than the first metric conventional threshold.

After closing part or all of the receiving threads for accessing the low-priority pictures, the picture processing pressure of the picture processing system is relieved, and the state with lower pressure can be recovered, at the moment, the receiving threads closed previously can be gradually recovered, so that the access quantity of the gateway layer to the picture stream is increased, and the picture processing resources of the picture processing system are fully utilized.

With reference to the embodiment of the first aspect, in a possible implementation manner, the index includes a second index used for representing a currently processed quantity of concurrent requests by the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold used for representing a currently conventionally processable quantity of concurrent requests by the gateway layer; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index includes: and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the second index and the second index conventional threshold.

The stress of the picture processing system can be measured by the number of currently processed concurrent requests corresponding to the gateway layer.

With reference to the embodiment of the first aspect, in a possible implementation manner, the index further includes a second index used for representing the currently processed quantity of concurrent requests by the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold used for representing the currently conventionally processable quantity of concurrent requests by the gateway layer; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index further comprises: and when the first index is determined to be smaller than or equal to the first index conventional threshold, dynamically adjusting the access strategy of the gateway layer to the picture flow according to the size relation between the second index and the second index conventional threshold.

The stress condition of the picture processing system can be measured by the picture feature extraction capability corresponding to the algorithm layer and the number of the currently processed concurrent requests corresponding to the gateway layer.

With reference to the embodiment of the first aspect, in a possible implementation manner, a priority is preset for a picture included in the picture stream, where the receiving thread includes a first target receiving thread and a second target receiving thread, and a priority of a picture accessed by the first target receiving thread is lower than a priority of a picture accessed by the second target receiving thread; the second index comprises a third index corresponding to the first target receiving thread and a fourth index corresponding to the second target receiving thread; the second index normal threshold value comprises a normal threshold value corresponding to the third index and a normal threshold value corresponding to the fourth index; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the second index and the second index conventional threshold includes: when one of the third index and the fourth index is determined to be smaller than or equal to the corresponding conventional threshold value, and the other index is determined to be larger than the corresponding conventional threshold value, whether the number of receiving threads corresponding to the other index is increased or not is determined. With reference to the embodiment of the first aspect, in a possible implementation manner, the preset index corresponding to the second index further includes a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the fourth index; the determining whether to increase the number of receiving threads corresponding to the another indicator includes: judging whether an unallocated idle receiving thread exists in the gateway layer; if so, setting the idle receiving thread as a receiving thread corresponding to the other index; if the other index is the fourth index and the fourth index is larger than the maximum threshold corresponding to the fourth index, scheduling part or all of the receiving threads in the first target receiving thread as the second target receiving thread.

When the idle receiving thread exists, the idle receiving thread is preferentially set to be the type corresponding to the receiving thread with the highest pressure, and when the idle receiving thread does not exist and the receiving thread with the highest pressure is the second target receiving thread used for accessing the high-priority picture, the first target receiving thread used for accessing the low-priority picture is scheduled to be used as the second target receiving thread, so that the current value of the maximum threshold value corresponding to the fourth index is increased, meanwhile, the processing capacity of the second target receiving thread is increased, the processing pressure of the second target receiving thread is reduced, and the processing progress of the high-priority picture is guaranteed.

With reference to the embodiment of the first aspect, in a possible implementation manner, the dynamically adjusting, according to a size relationship between the second index and the second index normal threshold, an access policy of the gateway layer to the picture stream further includes: increasing the number of at least one of the first target receiving thread and the second target receiving thread when it is determined that the third metric and the fourth metric are both greater than the corresponding conventional thresholds.

With reference to the embodiment of the first aspect, in a possible implementation manner, the preset index corresponding to the second index further includes a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the third index and a maximum threshold value corresponding to the fourth index; the increasing the number of at least one of the first target receiving thread and the second target receiving thread when it is determined that the third metric and the fourth metric are both greater than the corresponding regular threshold includes: judging whether an unallocated idle receiving thread exists in the gateway layer; if so, determining the allocation strategy of the idle receiving thread according to the third index, the conventional threshold corresponding to the third index, the maximum threshold corresponding to the third index, the fourth index, the conventional threshold corresponding to the fourth index and the maximum threshold corresponding to the fourth index.

And when the first target receiving thread and the second target receiving thread both have pressure and have unallocated idle receiving threads, allocating the idle receiving threads according to the actual pressure conditions of the first target receiving thread and the second target receiving thread.

With reference to the embodiment of the first aspect, in a possible implementation manner, the determining an allocation policy of the idle receiving thread includes: if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread; if the third index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, and the fourth index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, setting one part of the idle receiving threads as the second target receiving thread, and setting the other part of the idle receiving threads as the first target receiving thread; if the fourth index is greater than the corresponding maximum threshold value, and the third index is greater than the corresponding conventional threshold value and smaller than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread; and if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding conventional threshold value and less than the corresponding maximum threshold value, setting the idle receiving thread as the first target receiving thread.

I.e. the processing progress of the second target receiving thread which processes the high priority picture is preferentially guaranteed.

With reference to the embodiment of the first aspect, in a possible implementation manner, after the setting the idle receiving thread as the second target receiving thread, the method further includes: and determining that the fourth index is greater than the corresponding maximum threshold value, and scheduling part or all of the first target receiving threads as the second target receiving threads.

When the pressure of the second target receiving thread processing the high-priority picture is large, if the idle receiving thread is set as the second target receiving thread preferentially and the pressure of the second target receiving thread is still not reduced to be within a proper range, the first target receiving thread processing the low-priority picture can be scheduled as the second target receiving thread.

With reference to the embodiment of the first aspect, in a possible implementation manner, after the setting the idle receiving thread as the first target receiving thread, the method further includes: and determining that the third index is larger than the corresponding conventional threshold and smaller than the corresponding maximum threshold and remaining idle receiving threads still exist, and setting the remaining idle receiving threads as the second target receiving threads.

When the pressure of a first target receiving thread for processing a low-priority picture is large, if the idle receiving thread is set as the first target receiving thread preferentially, the pressure of the first target receiving thread is reduced to a proper range, and the rest idle receiving threads exist, the rest idle receiving threads are set as second target receiving threads, so that the pressure of the second target receiving threads is reduced.

With reference to the embodiment of the first aspect, in a possible implementation manner, after the determining whether there is an unallocated idle receiving thread in the gateway layer, the method further includes: if the fourth index is larger than the corresponding maximum threshold value, scheduling part or all of the receiving threads in the first target receiving threads as the second target receiving threads.

When no redundant idle receiving thread exists, the first target receiving thread for processing the low-priority picture can be dispatched to the second target receiving thread as long as the pressure of the second target receiving thread is high to a certain degree, so that the processing progress of the high-priority picture is ensured.

In a second aspect, an embodiment of the present application provides an image stream access apparatus, which is applied to a service layer of an image processing system, where the image processing system further includes a gateway layer in communication connection with the service layer, and the gateway layer is used to access an image stream, and the apparatus includes: the device comprises an acquisition module and an adjustment module. The acquisition module is used for acquiring an index for representing the image processing pressure of the image processing system in real time; and the adjusting module is used for dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the index and the corresponding preset index.

With reference to the second aspect, in a possible implementation manner, the picture processing system further includes an algorithm layer in communication connection with the service layer, where the indexes include a first index for characterizing picture feature extraction capability of the algorithm layer, and the preset index corresponding to the first index includes a first index conventional threshold for characterizing conventional picture feature extraction capability of the algorithm layer and a first index maximum threshold for characterizing maximum picture feature extraction capability of the algorithm layer, where the first index conventional threshold is smaller than the first index maximum threshold; the adjusting module is configured to dynamically adjust an access policy of the gateway layer to the picture stream according to a size relationship between the first indicator, the first indicator conventional threshold, and the first indicator maximum threshold.

With reference to the second aspect, in a possible implementation manner, the gateway layer includes multiple receiving threads for accessing the picture stream, where a priority is preset for pictures included in the picture stream, and the adjusting module is configured to control the gateway layer to close a part of the first target receiving threads when it is determined that the first indicator is greater than the first indicator normal threshold and smaller than the first indicator maximum threshold; when the first index is determined to be larger than or equal to the first index maximum threshold value, controlling the gateway layer to close all first target receiving threads; the priority of the picture accessed by the first target receiving thread is lower than the priority of the pictures accessed by other receiving threads.

With reference to the second aspect, in a possible implementation manner, the apparatus further includes a restart module, configured to control the gateway layer to partially restart/completely restart the shutdown first target receiving thread when it is determined that the first metric is smaller than the first metric conventional threshold.

With reference to the embodiment of the second aspect, in a possible implementation manner, the index includes a second index used for representing the number of concurrent requests currently processed by the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold used for representing the number of currently conventionally processable concurrent requests by the gateway layer; and the adjusting module is used for dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relationship between the second index and the second index conventional threshold value.

With reference to the embodiment of the second aspect, in a possible implementation manner, the index further includes a second index used for characterizing the number of concurrent requests currently processed by the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold used for characterizing the number of currently conventionally processable concurrent requests by the gateway layer; and the adjusting module is configured to dynamically adjust an access policy of the gateway layer to the picture stream according to a size relationship between the second indicator and the second indicator normal threshold when it is determined that the first indicator is smaller than or equal to the first indicator normal threshold.

With reference to the second aspect embodiment, in a possible implementation manner, the pictures included in the picture stream are preset with priorities, the receiving threads include a first target receiving thread and a second target receiving thread, and the priority of the pictures accessed by the first target receiving thread is lower than the priority of the pictures accessed by the second target receiving thread; the second index comprises a third index corresponding to the first target receiving thread and a fourth index corresponding to the second target receiving thread; the second index normal threshold value comprises a normal threshold value corresponding to the third index and a normal threshold value corresponding to the fourth index; the adjusting module is configured to determine whether to increase the number of receiving threads corresponding to another indicator when it is determined that one of the third indicator and the fourth indicator is smaller than or equal to a corresponding conventional threshold and the other indicator is greater than the corresponding conventional threshold.

With reference to the embodiment of the second aspect, in a possible implementation manner, the preset index corresponding to the second index further includes a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the fourth index; the adjusting module is used for judging whether an unallocated idle receiving thread exists in the gateway layer; if so, setting the idle receiving thread as a receiving thread corresponding to the other index; if the other index is the fourth index and the fourth index is larger than the maximum threshold corresponding to the fourth index, scheduling part or all of the receiving threads in the first target receiving thread as the second target receiving thread.

With reference to the second aspect embodiment, in a possible implementation manner, the adjusting module is further configured to increase the number of at least one receiving thread of the first target receiving thread and the second target receiving thread when it is determined that the third index and the fourth index are both greater than the corresponding regular threshold.

With reference to the second aspect, in a possible implementation manner, the preset indexes corresponding to the second index further include a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the third index and a maximum threshold value corresponding to the fourth index; the adjusting module is used for judging whether an unallocated idle receiving thread exists in the gateway layer; if so, determining the allocation strategy of the idle receiving thread according to the third index, the conventional threshold corresponding to the third index, the maximum threshold corresponding to the third index, the fourth index, the conventional threshold corresponding to the fourth index and the maximum threshold corresponding to the fourth index.

With reference to the second aspect, in a possible implementation manner, the adjusting module is configured to set the idle receiving thread as the second target receiving thread if the third indicator is greater than the corresponding maximum threshold and the fourth indicator is greater than the corresponding maximum threshold; if the third index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, and the fourth index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, setting one part of the idle receiving threads as the second target receiving thread, and setting the other part of the idle receiving threads as the first target receiving thread; if the fourth index is greater than the corresponding maximum threshold value, and the third index is greater than the corresponding conventional threshold value and smaller than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread; and if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding conventional threshold value and less than the corresponding maximum threshold value, setting the idle receiving thread as the first target receiving thread.

With reference to the second aspect, in a possible implementation manner, the adjusting module is configured to determine that the fourth indicator is greater than a corresponding maximum threshold, and schedule part or all of the first target receiving threads as the second target receiving threads.

With reference to the second aspect embodiment, in a possible implementation manner, the adjusting module is configured to determine that the third indicator is greater than the corresponding regular threshold and smaller than the corresponding maximum threshold and there are remaining idle receiving threads, and set the remaining idle receiving threads as the second target receiving threads.

With reference to the second aspect embodiment, in a possible implementation manner, the adjusting module is configured to, when it is determined that there is no unallocated idle receiving thread in the gateway layer, schedule a part or all of the receiving threads in the first target receiving threads as the second target receiving threads if the fourth index is greater than the corresponding maximum threshold.

In a third aspect, an embodiment of the present application further provides an electronic device, including: a memory and a processor, the memory and the processor connected; the memory is used for storing programs; the processor calls a program stored in the memory to perform the method of the first aspect embodiment and/or any possible implementation manner of the first aspect embodiment.

In a fourth aspect, the present application further provides a non-transitory computer-readable storage medium (hereinafter, referred to as a computer-readable storage medium), on which a computer program is stored, where the computer program is executed by a computer to perform the method in the foregoing first aspect and/or any possible implementation manner of the first aspect.

In a fifth aspect, an embodiment of the present application further provides a picture processing system, including a service layer, and a gateway layer and an algorithm layer, which are in communication connection with the service layer, where the gateway layer is used to access a picture stream; the algorithm layer is used for extracting the characteristics of the accessed picture flow; the service layer is used for acquiring indexes, used for representing the picture processing pressure of the picture processing system, of the gateway layer and/or the algorithm layer, and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the indexes and the corresponding preset indexes.

Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and drawings.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts. The foregoing and other objects, features and advantages of the application will be apparent from the accompanying drawings. Like reference numerals refer to like parts throughout the drawings. The drawings are not intended to be to scale as practical, emphasis instead being placed upon illustrating the subject matter of the present application.

Fig. 1 shows a schematic structural diagram of a picture processing system according to an embodiment of the present application.

Fig. 2 shows a flowchart of a picture stream access method provided in an embodiment of the present application.

Fig. 3 shows a block diagram of a picture stream access apparatus according to an embodiment of the present application.

Fig. 4 shows a schematic structural diagram of an electronic device provided in an embodiment of the present application.

Icon: 10-a picture processing system; 11-gateway layer; 12-a service layer; 13-algorithm layer; 14-a base layer; 100-an electronic device; 110-a processor; 120-a memory; 400-picture stream access means; 410-an obtaining module; 420-adjusting module.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, relational terms such as "first," "second," and the like may be used solely in the description herein to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Further, the term "and/or" in the present application is only one kind of association relationship describing the associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone.

In addition, the defects (which cause the system not to operate normally or cause the system not to operate in the best state) existing in the prior art are the results obtained after the applicant has conducted practical and careful study, and therefore, the discovery process of the above defects and the solutions proposed by the embodiments of the present application to the above defects in the following text should be considered as contributions of the applicant to the present application.

In order to solve the above problem, embodiments of the present application provide a picture stream access method, a picture processing system, an electronic device, and a computer-readable storage medium, which are beneficial to keeping the picture processing system in an optimal operating state on the premise that the picture processing system can operate normally.

The technology can be realized by adopting corresponding software, hardware and a combination of software and hardware. The following describes embodiments of the present application in detail.

First, a system architecture of a picture processing system 10 for implementing a picture stream access method and apparatus according to an embodiment of the present application is described with reference to fig. 1. The picture processing system 10 may include a gateway layer 11, a service layer 12, an algorithm layer 13, and a base layer 14.

The service layer 12 establishes communication connection with the gateway layer 11 and the algorithm layer 13, respectively, and the base layer establishes communication connection with the gateway layer 11, the service layer 12 and the algorithm layer 13, respectively.

In some embodiments, the gateway layer 11, the service layer 12, and the algorithm layer 13 may be different servers or server clusters, respectively, and in other embodiments, the gateway layer 11 and the service layer 12 may belong to the same server or server cluster, and the algorithm layer belongs to another server or server cluster.

Base layer 14 is maintained within picture processing system 10 in the form of a database. Optionally, Kafka (distributed publish-subscribe messaging system) for caching pictures, message middleware for data transmission, and the like may be deployed in the base layer 14.

A receiving thread is deployed in the gateway layer 11, and is configured to process a picture synchronization request initiated by an external picture access device (e.g., a camera, an external picture access system, etc.), so as to access a picture included in the picture synchronization request, and place the accessed picture in Kafka.

In some embodiments, the gateway layer 11 may access different categories of pictures, such as access passerby pictures and basement pictures (the more important pictures). At this time, one part of the receiving threads may be used for accessing the passerby picture, and the other part of the receiving threads may be used for accessing the basement picture.

Within the algorithm layer 13 is deployed a picture feature extraction component. The algorithm layer 13 may subscribe to Kafka at a preset speed to obtain a picture in Kafka, and perform feature extraction on the obtained picture through the picture feature extraction component. In addition, after the algorithm layer 13 extracts features from the picture, the extracted features are again stored in Kafka.

A warehousing thread, a service scheduling component, and a service monitoring component are deployed within the service layer 12.

The service layer 12 may subscribe to Kafka to obtain the extracted features, and store the extracted features and/or the corresponding pictures in a storage thread.

It is noted that in some embodiments of the present application, the number of binned threads is proportional to the number of receiving threads.

Furthermore, the service layer 12 can know the load condition of the gateway layer 11 and the algorithm layer 13 through the service monitoring component. Wherein, for the gateway layer 11, the load condition thereof, namely, the load condition has a positive correlation with the number (qps) of the image synchronization concurrent requests accessed by the gateway layer 11 in unit time; for the algorithm layer 13, the load condition has a positive correlation with the number of pictures that the algorithm layer 13 is performing feature extraction.

In this embodiment, after obtaining the load conditions of the gateway layer 11 and the algorithm layer 13, the service layer 12 may determine the picture processing pressure of the picture processing system 10 based on the load conditions of the gateway layer 11 and the algorithm layer 13, and dynamically adjust the access policy of the gateway layer 11 to the picture stream through the service scheduling component according to the current picture processing pressure, so as to ensure that the picture processing system 10 is kept in the optimal operating state on the premise of being able to operate normally.

The following describes a process of the service layer 12 dynamically adjusting the access policy of the gateway layer 11 to the picture stream through the service scheduling component, in conjunction with the picture stream access method shown in fig. 2.

Referring to fig. 2, an embodiment of the present application provides a picture stream access method applied to the service layer 12 of the picture processing system 10. The method comprises the following steps.

Step S110: and acquiring an index for representing the picture processing pressure of the picture processing system in real time.

Step S120: and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the index and the corresponding preset index.

As mentioned above, the picture processing system 10 may include a gateway layer 11, a service layer 12, and an algorithm layer 13. For example, in an optional embodiment, the picture processing pressure of the picture processing system 10 may be determined according to the picture feature extraction capability of the algorithm layer 13, in another optional embodiment, the picture processing pressure of the picture processing system 10 may be determined according to the currently processed quantity of concurrent requests of the gateway layer 11, and in yet another optional embodiment, the picture processing pressure of the picture processing system 10 may be determined by simultaneously combining the picture feature extraction capability of the algorithm layer 13 and the currently processed quantity of concurrent requests of the gateway layer 11.

That is to say, for the picture processing system 10, the index for representing the picture processing pressure includes a plurality of conditions, and correspondingly, the parameter included in the index for each condition is different, and the preset index corresponding to the condition may be preset, so as to form a one-to-one correspondence relationship.

After obtaining the index under each condition, the service layer 12 may compare the currently obtained index with a corresponding preset index, so as to determine the current picture processing pressure of the picture processing system 10, so as to dynamically adjust the access policy of the gateway layer 11 to the picture stream according to the current picture processing pressure.

Optionally, the process of the service layer 12 dynamically adjusting the access policy of the gateway layer 11 to the picture stream may be that the service layer 12 controls the gateway layer 11 to reduce the access to the picture stream when it is determined that the picture processing pressure of the picture processing system 10 is relatively high, so as to reduce the picture processing pressure of the picture processing system 10; or the service layer 12 may control the gateway layer 11 to increase access to the picture stream when it is determined that the picture processing pressure of the picture processing system 10 is low, so as to fully utilize the picture processing resources of the picture processing system 10; the service layer 12 may control the gateway layer 11 to reduce the access to the picture stream when determining that the picture processing pressure of the picture processing system 10 is large, and the service layer 12 may control the gateway layer 11 to increase the access to the picture stream when determining that the picture processing pressure of the picture processing system 10 is small.

As an alternative embodiment, when determining that the picture processing pressure of the picture processing system 10 is characterized by the picture feature extraction capability of the algorithm layer 13 (assumed to be represented by the first index a), the index includes the first index a. Accordingly, in this embodiment, the preset index corresponding to the first index a includes the first index normal threshold value Amed, and includes the first index maximum threshold value Amax.

In this embodiment, the access policy of the gateway layer 11 to the picture stream may be dynamically adjusted through the size relationship between A, Amed and Amax.

The first index conventional threshold Amed is used for representing the conventional image feature extraction capability of the algorithm layer 13 under a general condition, the first index maximum threshold Amax is used for representing the maximum image feature extraction capability of the algorithm layer 13 under a limit condition, and the Amed is less than the Amax. It is noted that the magnitudes of Amed and Amax are configured in advance by the worker according to the calculation power of the algorithm layer 13.

As mentioned above, the gateway layer 11 includes a plurality of receiving threads for accessing the picture stream. Based on the principle of the current limiting algorithm (e.g., leaky bucket algorithm, token bucket algorithm, etc.), the effect of reducing the upper limit of the receiving speed of the interface for accessing the picture stream can be achieved by closing part of the receiving threads, so that when the interface detects that the picture accessing speed exceeds the upper limit of the receiving speed, the picture exceeding the upper limit of the receiving speed is discarded.

In some embodiments, when the service layer 12 determines that the first metric a is greater than the first metric conventional threshold Amed, it may determine that the picture processing pressure of the picture processing system 10 is large. At this time, the current limitation on the speed of the accessed pictures can be realized by closing part of the receiving threads, so that the number of the accessed pictures is reduced, and the picture feature extraction pressure of the algorithm layer 13 can be relieved.

In addition, in other embodiments, when the receiving thread is used to access different classes of pictures, priorities may be pre-set for the different classes of pictures included in the picture stream. As in the above example, the gateway layer 11 may access the passerby picture and the bottom library picture (the more important picture), and the priority of the bottom library picture is higher than that of the passerby picture. It will be appreciated that low priority pictures are less important than high priority pictures and may be discarded if necessary. At this time, for convenience of distinction, a receiving thread for accessing a low-priority picture among receiving threads included in the gateway layer 11 is referred to as a first target receiving thread, and a receiving thread for accessing a high-priority picture is referred to as a second target receiving thread.

In this embodiment, the first index a is larger than the first index normal threshold Amed, and may be classified into the following two cases: (1) the first index A is positioned between the conventional threshold Amed of the first index and the maximum threshold Amax of the first index (Amed is less than A and less than Amax); (2) the first index A is larger than or equal to the maximum threshold value Amax of the first index (A is larger than or equal to Amax). And both of the above situations represent the presence of picture processing pressure in the picture processing system 10. At this time, the service layer 12 may take different measures to control the gateway layer 11 to reduce the access to the picture stream to relieve the stress.

For example, for the case where Amed < a < Amax, it is described that the algorithm layer 13 has the pressure of picture feature extraction. Based on the above current limiting algorithm, the number of access threads of the gateway layer 11 can be reduced appropriately, so as to reduce the number of accesses to the picture stream by the gateway layer 11 appropriately. At this time, since the priority of the picture accessed by the first target receiving thread is lower than the priority of the picture accessed by the second target receiving thread, the processing progress of the high-priority picture should be preferentially ensured, and therefore, the gateway layer 11 may be controlled to close part of the first target receiving threads. Corresponding to the example above, a receiving thread for accessing a passerby picture may be partially closed.

For example, for the case of A ≧ Amax, the pressure of the image feature extraction of the algorithm layer 13 is described to be large. Based on the above current limiting algorithm, the number of receiving threads of the gateway layer 11 can be greatly reduced, so as to greatly reduce the number of access of the gateway layer 11 to the picture stream. At this time, since the priority of the picture accessed by the first target receiving thread is lower than the priority of the picture accessed by the second target receiving thread, the processing progress of the high-priority picture should be preferentially ensured, and therefore, the gateway layer 11 may be controlled to close all the first target receiving threads. Corresponding to the example above, all receiving threads for accessing the passerby picture may be turned off.

As can be seen from the above description, when it is determined that the algorithm layer 13 has pressure according to the value of a, it is considered to preferentially close the first target receiving thread for accessing the low-priority picture, so as to ensure the access progress of the high-priority picture.

The number of the first target receiving threads that need to be turned off in the above text can be determined by the formula N ═ a-Amed)/Amed ] × [ (maximum number of receiving threads-current number of receiving threads) ]. The maximum number of receiving threads is used to represent the number of receiving threads that can be deployed at most by gateway layer 11, and the current number of receiving threads is used to represent the number of receiving threads that have been deployed currently by gateway layer 11.

When the gateway layer 11 turns off the receiving thread (possibly turning off part of the first target receiving thread, or possibly turning off all of the first target receiving threads), it can reduce the access of the subsequent pictures, so that after a certain time, the value of a obtained by the service layer 12 will gradually decrease until a < Amed. When A is less than Amed, the algorithm layer 13 is shown to have the remaining capacity to perform feature extraction on more pictures. At this time, as an optional implementation manner, the service layer 12 may control the gateway layer 11 to restart the above-mentioned part of the closed first target receiving threads or restart all the above-mentioned closed first target receiving threads, so that the part of or all the receiving threads that are previously closed start to access a new picture again, so as to increase the number of the pictures to be accessed, thereby fully utilizing the idle calculation power of the algorithm layer 13 and improving the overall efficiency of the picture processing system 10.

The number of the first target receiving threads needing to be restarted can be determined by the formula N ═ Amed-a)/Amed × [ (maximum number of receiving threads-current number of receiving threads) ]. The maximum number of receiving threads is used to represent the number of receiving threads that can be deployed at most by gateway layer 11, and the current number of receiving threads is used to represent the number of receiving threads that have been deployed currently by gateway layer 11.

In some embodiments, the access policy of the gateway layer 11 to the picture stream may be determined according to the first index a and the preset index corresponding to the first index a, as described above.

Of course, in some embodiments, the access policy of the gateway layer 11 to the picture stream may be determined by the second index B and a preset index corresponding to the second index B. The preset index corresponding to the second index B includes a second index normal threshold Bmed and a second index maximum threshold Bmax.

The second index B is used to represent the number of concurrent requests currently processed by the gateway layer 11; the second standard conventional threshold Bmed is used to characterize the conventional processable concurrent request quantity that the gateway layer 11 can provide on the basis of the currently deployed receiving thread quantity; the second indicated maximum threshold Bmax is used to characterize the maximum number of processable concurrent requests that the gateway layer 11 can provide based on the number of receiving threads that have been currently deployed.

It is worth noting that as the number of receiving threads currently deployed by the gateway layer 11 changes, the values of Bmed and Bmax change accordingly. The staff may configure and store the values of Bmed and Bmax in advance according to the actual number of receiving threads deployed in the gateway layer 11, or may automatically calculate the values of Bmed and Bmax according to the actual number of receiving threads currently deployed in the gateway layer 11. After acquiring the number of currently deployed receiving threads of the gateway layer 11, the service layer 12 may refer to pre-stored data, and acquire Bmed and Bmax that are matched with the number of currently deployed receiving threads.

In this embodiment, the pressure condition of the algorithm layer 13 may be ignored, and the access policy of the gateway layer 11 to the picture stream may be dynamically adjusted directly according to the size relationship between the second index B, the second index normal threshold Bmed, and the second index maximum threshold Bmax.

Of course, in some embodiments, the access policy of the gateway layer 11 to the picture stream may also be determined by combining the first index a, the preset index corresponding to the first index a, the second index B, and the preset index corresponding to the second index B.

In this embodiment, when it is determined that a > Amed, the access policy of the gateway layer 11 to the picture stream may still be dynamically adjusted according to the manner described above, which is not described herein again; when it is determined that a is not greater than Amed, it is described that the algorithm layer 13 has a residual force to perform feature extraction on more pictures, and the influence on the picture processing pressure of the picture processing system 10 is small, and at this time, the access policy of the gateway layer 11 to the picture stream can be dynamically adjusted according to the magnitude relation between the second index B, the second index conventional threshold Bmed, and the second index maximum threshold Bmax, so that the overall picture processing efficiency of the picture processing system 10 is improved.

The following describes a process of dynamically adjusting the access policy of the gateway layer 11 for the picture stream according to the size relationship between the second index B, the second index normal threshold Bmed, and the second index maximum threshold Bmax.

Of course, it is worth pointing out that the process may be applied to the above-described implementation that dynamically adjusts the access policy of the gateway layer 11 to the picture stream directly according to the size relationship between the second index B, the second index normal threshold Bmed, and the second index maximum threshold Bmax; the method may also be applied to the above-described implementation that determines the access policy of the gateway layer 11 to the picture stream by combining the first index a, the preset index corresponding to the first index a, the second index B, and the preset index corresponding to the second index B.

As mentioned above, the pictures included in the picture stream have priorities set in advance, the receiving threads deployed by the gateway layer 11 include a first target receiving thread and a second target receiving thread, and the priority of the pictures accessed by the first target receiving thread is lower than the priority of the pictures accessed by the second target receiving thread. In the following, an example will be described in which the first target receiving thread is used to access a passerby picture, and the second target receiving thread is used to access a basement picture.

On this premise, the second indicator B includes a third indicator B3 corresponding to the first target receiving thread and a fourth indicator B4 corresponding to the second target receiving thread.

The third index B3 represents the quantity of concurrent requests processed by all first target receiving threads currently used for accessing the passerby picture; the fourth index B4 characterizes the number of concurrent requests processed by all second target receiving threads currently used for accessing the base library picture.

Correspondingly, the second index conventional threshold value Bmed comprises a conventional threshold value Bmed3 corresponding to the third index and a conventional threshold value Bmed4 corresponding to the fourth index; the second index maximum threshold value Bmax includes a maximum threshold value Bmax3 corresponding to the third index and a maximum threshold value Bmax4 corresponding to the fourth index.

Wherein, the conventional threshold Bmed3 corresponding to the third index is used to characterize the conventional processable concurrent request number that the gateway layer 11 can provide on the basis of all the first target receiving thread numbers that have been currently deployed; the conventional threshold Bmed4 corresponding to the fourth index is used to characterize the conventional processable concurrent request number that the gateway layer 11 can provide based on all the second target receiving threads currently deployed.

The maximum threshold Bmax3 corresponding to the third index is used to characterize the maximum number of processable concurrent requests that can be provided by the gateway layer 11 on the basis of all the first target receiving threads currently deployed; the conventional threshold Bmax4 corresponding to the fourth criterion is used to characterize the maximum number of processable concurrent requests that the gateway layer 11 can provide on the basis of all the second target number of receiving threads that have been currently deployed.

Of course, Bmax3 is greater than Bmed3, and Bmax4 is greater than Bmed 4.

It is to be noted that, as the number of first target receiving threads and the number of second target receiving threads that have been currently deployed by the gateway layer 11 change, the values of Bmed3, Bmed4, Bmax3, and Bmax4 change accordingly, and the number of first target receiving threads is in positive correlation with the values of Bmed3 and Bmax3, and the number of second target receiving threads is in positive correlation with the values of Bmed4 and Bmax 4.

The dynamically adjusting the access policy of the gateway layer 11 to the picture stream according to the size relationship between the second index B, the second index normal threshold Bmed, and the second index maximum threshold Bmax may include the following several embodiments:

the first implementation mode comprises the following steps: when the service layer 12 determines that one of the third metric B3 and the fourth metric B4 is smaller than the corresponding normal threshold and the other metric is greater than the corresponding normal threshold, it indicates that the metric smaller than the corresponding normal threshold corresponds to no pressure in the receiving thread and the metric greater than the corresponding normal threshold corresponds to no pressure in the receiving thread.

For example, when the third indicator B3 is less than the corresponding regular threshold Bmed3(B3 < Bmed3) and the fourth indicator is greater than the corresponding regular threshold Bmed4(B4 > Bmed4), it indicates that there is currently some pressure for the second target receiving thread for accessing the basement picture, and there is currently no pressure for the first target receiving thread for accessing the passerby picture.

For example, when the fourth metric B4 is less than the corresponding regular threshold Bmed4(B4 < Bmed4) and the third metric is greater than the corresponding regular threshold Bmed3(B3 > Bmed3), it indicates that there is currently some pressure for the first target receiving thread for accessing the passerby picture and there is currently no pressure for the second target receiving thread for accessing the basement picture.

It is noted that for B4 > Bmed4, two cases of Bmed4 < B4 < Bmax4 and B4 ≧ Bmax4 can be included; for B3 > Bmed3, two cases of Bmed3 < B3 < Bmax3 and B3 ≧ Bmax3 can be included.

When the service layer 12 determines that one of the third metric B3 and the fourth metric B4 is less than the corresponding conventional threshold and the other metric is greater than the corresponding conventional threshold, it may be determined whether to increase the number of receiving threads corresponding to the other metric.

The number of receiving threads corresponding to another index is increased, that is, other receiving threads are used as supplementary receiving threads, and the supplementary receiving threads are set as receiving threads corresponding to another index.

Alternatively, it may be determined first whether there is an unallocated idle receive thread within gateway layer 11.

An unallocated idle receive thread here refers to a receive thread within gateway layer 11 that is neither a first targeted receive thread nor a second targeted receive thread.

In some embodiments, when there are unallocated idle receiving threads, it may be determined that the number of receiving threads corresponding to another metric needs to be increased, i.e., the idle receiving threads may be set as receiving threads corresponding to another metric.

At this time, a specific index of another index may be omitted, for example, when an idle receive line exists in the gateway layer 11 and a second target receive thread needs to be added, the idle receive thread in the gateway layer 11 is set as the second target receive thread to increase the number of the second target receive threads; when an idle receiving line exists in the gateway layer 11 and the first target receiving thread needs to be added, the idle receiving thread in the gateway layer 11 is set as the first target receiving thread to increase the number of the first target receiving threads.

In addition, the number of receiving threads that needs to be increased in the above description may be determined by the formula N ═ Amed-a)/Amed ] × [ (maximum number of receiving threads — current number of receiving threads) ]. The maximum number of receiving threads is used to represent the number of receiving threads that can be deployed at most by gateway layer 11, and the current number of receiving threads is used to represent the number of receiving threads that have been deployed currently by gateway layer 11.

As the number of receiving threads changes, the corresponding values of Bmed and Bmax change accordingly.

When the number of the second target receiving threads is increased, the value of Bmed4 and the value of Bmax4 are correspondingly increased, at this time, if B4 > Bmax4 before the number of the second target receiving threads is increased, after Bmax4 is changed, B4 may be less than current Bmax 4; if Bmed4 < B4 < Bmax4 is increased before the number of second target receiving threads is increased, after Bmed4 and Bmax4 are changed, B4 < current Bmed4 may be caused to be decreased, so that the pressure of receiving threads in the gateway layer 11 for accessing the base library picture is made to be evenly reduced, and normal access of the base library picture is ensured.

Similarly, when the number of the first target receiving threads is increased, the normal access of the passerby pictures can be ensured.

In some embodiments, when there are no unallocated idle receiving threads. If the other index is the third index B3, since the first target receiving thread corresponding to the third index B3 is used to access the less important passerby picture, and since there are no receiving threads available to supplement the number of the first target receiving threads, it is possible to determine that the number of receiving threads corresponding to the other index is not increased without adjusting the data of the current first target receiving thread. If the other index is the fourth index B4, and B4 is greater than the maximum threshold Bmax4 corresponding to B4 (B4 > Bmax4), since the second target receiving thread corresponding to B4 is used to access the basement picture, and the priority of the basement picture is higher than that of the passerby picture, at this time, part or all of the first target receiving threads used to access the passerby picture may be scheduled as the second target receiving threads, so as to increase the number of the second target receiving threads.

The second embodiment: when the service layer 12 determines that both the third metric B3 and the fourth metric B4 are greater than the corresponding regular thresholds (i.e., B3 > Bmed3 and B4 > Bmed4), indicating that there is pressure between the first targeted receiving thread and the second targeted receiving thread, the gateway layer 11 may be controlled to increase the number of at least one receiving thread of the first targeted receiving thread and the second targeted receiving thread.

In some embodiments, it may be determined first whether there are any unallocated idle receive threads within gateway layer 11.

When it is determined that there is no unallocated idle receiving thread, if the fourth metric B4 is greater than Bmax4, it indicates that the pressure of the second target receiving thread is large. Since the priority of the basement picture accessed by the second target receiving thread is higher than that of the passerby picture accessed by the first target receiving thread, in order to preferentially ensure the normal access of the basement picture, the first target receiving thread can be successively scheduled as the second target receiving thread to increase the number of the second target receiving thread until B4 < the current Bmax4, or until all the first target receiving threads are scheduled as the second target receiving threads, so that B4 < the current Bmax4 is not caused.

It is worth noting that as the number of second target receiving threads increases, Bmax4 increases accordingly. Optionally, when B4 < current Bmax4 is detected, which indicates that the access pressure of the second target receiving thread reaches an acceptable range, the scheduling of the first target receiving thread as the second target receiving thread may be stopped.

When the idle receiving threads which are not allocated exist, the allocation strategy of the idle receiving threads is determined according to the third index B3, the conventional threshold Bmed3 corresponding to the third index, the maximum threshold Bmax3 corresponding to the third index, the fourth index B4, the conventional threshold Bmed4 corresponding to the fourth index and the maximum threshold Bmax4 corresponding to the fourth index.

If the third indicator B3 is greater than the corresponding maximum threshold Bmax3 and the fourth indicator B4 is greater than the corresponding maximum threshold Bmax4, it indicates that the access pressures of the first target receiving thread and the second target receiving thread are both large, but since the priority of accessing pictures by the second target receiving thread is higher than the priority of accessing pictures by the first target receiving thread, idle receiving threads can be preferentially set as the second target receiving thread one by one, that is, access of pictures with high priority is preferentially guaranteed.

Alternatively, after the idle receiving thread is successively set as the second target receiving thread, Bmax4 is increased, and if B4 is detected to be smaller than the current Bmax4 and there are remaining idle receiving threads at this time, the remaining idle receiving threads may be set as the first target receiving threads.

Optionally, if B4 is not made smaller than the current Bmax4 after all idle receiving threads are successively set as the second target receiving threads, that is, B4 is always larger than the current Bmax4, the first target receiving thread may be successively scheduled as the second target receiving thread to increase the number of the second target receiving threads until B4 < the current Bmax 4; or until the first target receiving thread is all scheduled as the second target receiving thread, does not have B4 < current Bmax 4.

If the third indicator B3 is greater than the corresponding normal threshold Bmed3 and less than the corresponding maximum threshold Bmax3, and the fourth indicator B4 is greater than the corresponding normal threshold Bmed4 and less than the corresponding maximum threshold Bmax4, it indicates that the first target receiving thread and the second target receiving thread both have acceptable access pressure. At this time, a part of the idle receive threads may be set as the second target receive thread, and another part of the idle receive threads may be set as the first target receive thread.

Of course, since the second targeted receiving thread is more important than the first targeted receiving thread, most of the idle receiving threads may be set as the second targeted receiving thread.

If the fourth indicator B4 is greater than the corresponding maximum threshold Bmax4, and the third indicator B3 is greater than the corresponding regular threshold Bmed3 and less than the corresponding maximum threshold Bmax3, it indicates that the first target receiving thread has an acceptable access pressure, and the second target receiving thread has a greater pressure. At this point, the idle receive thread may be successively set as the second target receive thread to increase the number of second target receive threads until B4 < current Bmax 4.

Optionally, if B4 < the current Bmax4 is not set after all idle receiving threads are successively set as the second target receiving threads, that is, B4 is always greater than the current Bmax4, the first target receiving thread may be successively scheduled as the second target receiving thread to increase the number of the second target receiving threads until B4 < the current Bmax 4; or until the first target receiving thread is all scheduled as the second target receiving thread, does not have B4 < current Bmax 4.

If the third indicator B3 is greater than the corresponding maximum threshold Bmax3, and the fourth indicator B4 is greater than the corresponding regular threshold Bmed4 and less than the corresponding maximum threshold Bmax4, it indicates that the acceptable access pressure exists in the second target receiving thread, and the greater pressure exists in the first target receiving thread. At this point, the idle receive thread may be successively set as the first target receive thread to increase the number of first target receive threads until B3 < current Bmax 3.

Of course, if B3 < current Bmax3 is not set after all idle receiving threads are set as the first target receiving threads, then the adjustment of the number of first target receiving threads is abandoned.

Optionally, after the idle receiving thread is set as the first target receiving thread, if it is detected that the current Bmed3 < B3 < the current Bmax3 and there are remaining idle receiving threads, at this time, the remaining idle receiving threads may be set as the second target receiving threads.

A data buffer pool is deployed in the algorithm layer 13, and when the algorithm layer 13 executes a buffer mechanism, the picture feature extraction component of the algorithm layer 13 does not execute feature operation immediately after obtaining a picture, but stores the obtained picture in the data buffer pool. When the image feature extraction component of the algorithm layer 13 acquires the image and performs feature extraction, the number of the images stored in the data buffer pool and the image storage speed stored in the data buffer pool can be ignored, and the image can be acquired from the data buffer pool all the time according to the speed capable of being processed by the image feature extraction component to perform feature extraction, so that the normal operation of the image feature extraction component is ensured.

In some embodiments, the algorithm layer 13 of the picture processing system 10 may always execute the buffering mechanism, or execute the buffering mechanism when the service layer 12 determines that the picture processing pressure of the picture processing system 10 is large.

According to the picture stream access method provided by the embodiment of the application, indexes used for representing the picture processing pressure of a picture processing system are obtained in real time; and then, dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the indexes and the corresponding preset indexes. In the process, when the image processing pressure of the image processing system is higher, the access quantity of the gateway layer to the image stream can be reduced, so that the image processing pressure of the image processing system is reduced, and the normal operation of the image processing system is ensured; when the image processing pressure of the image processing system is low, the access quantity of the gateway layer to the image stream can be increased, so that the image processing resources of the image processing system are fully utilized, the image processing system is kept in the optimal state, and the working efficiency of the image processing system is improved.

As shown in fig. 3, an image stream access apparatus 400 is further provided in the embodiment of the present application, and is applied to a service layer of an image processing system, where the image processing system further includes a gateway layer in communication connection with the service layer, and the gateway layer is used for accessing an image stream.

The picture stream access apparatus 400 may include: an acquisition module 410 and an adjustment module 420.

An obtaining module 410, configured to obtain, in real time, an index representing a picture processing pressure of the picture processing system;

an adjusting module 420, configured to dynamically adjust an access policy of the gateway layer to the picture stream according to a size relationship between the index and a corresponding preset index.

In a possible implementation manner, the picture processing system further includes an algorithm layer in communication connection with the service layer, where the index includes a first index for characterizing picture feature extraction capability of the algorithm layer, a preset index corresponding to the first index includes a first index conventional threshold for characterizing conventional picture feature extraction capability of the algorithm layer, and a first index maximum threshold for characterizing maximum picture feature extraction capability of the algorithm layer, and the first index conventional threshold is smaller than the first index maximum threshold;

the adjusting module 420 is configured to dynamically adjust an access policy of the gateway layer to the picture stream according to a size relationship among the first indicator, the first indicator conventional threshold, and the first indicator maximum threshold.

In a possible implementation manner, the gateway layer includes multiple receiving threads for accessing the picture stream, the pictures included in the picture stream are preset with priorities, and the adjusting module 420 is configured to control the gateway layer to close a part of first target receiving threads when it is determined that the first indicator is greater than the first indicator normal threshold and less than the first indicator maximum threshold; when the first index is determined to be larger than or equal to the first index maximum threshold value, controlling the gateway layer to close all first target receiving threads; the priority of the picture accessed by the first target receiving thread is lower than the priority of the pictures accessed by other receiving threads.

In one possible implementation, the apparatus further includes a restart module configured to control the gateway layer to partially restart/completely restart the shutdown first target receiving thread when the first metric is determined to be less than the first metric normal threshold.

In a possible implementation manner, the index includes a second index for representing the currently processed quantity of concurrent requests of the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold for representing the currently conventionally processable quantity of concurrent requests of the gateway layer; the adjusting module 420 is configured to dynamically adjust the access policy of the gateway layer to the picture stream according to a size relationship between the second indicator and the second indicator normal threshold.

In a possible implementation manner, the index further includes a second index used for representing the currently processed quantity of concurrent requests of the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold used for representing the currently conventionally processable quantity of concurrent requests of the gateway layer; the adjusting module 420 is configured to dynamically adjust an access policy of the gateway layer to the picture stream according to a size relationship between the second indicator and the second indicator normal threshold when it is determined that the first indicator is smaller than or equal to the first indicator normal threshold.

In a possible implementation manner, the pictures included in the picture stream are preset with priorities, the receiving threads include a first target receiving thread and a second target receiving thread, and the priority of the pictures accessed by the first target receiving thread is lower than the priority of the pictures accessed by the second target receiving thread; the second index comprises a third index corresponding to the first target receiving thread and a fourth index corresponding to the second target receiving thread; the second index normal threshold value comprises a normal threshold value corresponding to the third index and a normal threshold value corresponding to the fourth index; the adjusting module 420 is configured to determine whether to increase the number of receiving threads corresponding to another indicator when it is determined that one indicator of the third indicator and the fourth indicator is smaller than or equal to a corresponding normal threshold and the other indicator is greater than the corresponding normal threshold.

In a possible implementation, the preset index corresponding to the second index further includes a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the fourth index; the adjusting module 420 is configured to determine whether an unallocated idle receiving thread exists in the gateway layer; if so, setting the idle receiving thread as a receiving thread corresponding to the other index; if the other index is the fourth index and the fourth index is larger than the maximum threshold corresponding to the fourth index, scheduling part or all of the receiving threads in the first target receiving thread as the second target receiving thread.

In a possible implementation manner, the adjusting module 420 is further configured to increase the number of at least one of the first target receiving thread and the second target receiving thread when it is determined that the third index and the fourth index are both greater than the corresponding regular threshold.

In a possible implementation manner, the preset indexes corresponding to the second index further include a second index maximum threshold value used for characterizing a current maximum processable concurrent request number of the gateway layer, where the second index maximum threshold value includes a maximum threshold value corresponding to the third index and a maximum threshold value corresponding to the fourth index; the adjusting module 420 is configured to determine whether an unallocated idle receiving thread exists in the gateway layer; if so, determining the allocation strategy of the idle receiving thread according to the third index, the conventional threshold corresponding to the third index, the maximum threshold corresponding to the third index, the fourth index, the conventional threshold corresponding to the fourth index and the maximum threshold corresponding to the fourth index.

In a possible implementation manner, the adjusting module 420 is configured to set the idle receiving thread as the second target receiving thread if the third indicator is greater than the corresponding maximum threshold and the fourth indicator is greater than the corresponding maximum threshold; if the third index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, and the fourth index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, setting one part of the idle receiving threads as the second target receiving thread, and setting the other part of the idle receiving threads as the first target receiving thread; if the fourth index is greater than the corresponding maximum threshold value, and the third index is greater than the corresponding conventional threshold value and smaller than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread; and if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding conventional threshold value and less than the corresponding maximum threshold value, setting the idle receiving thread as the first target receiving thread.

In a possible implementation manner, the adjusting module 420 is configured to determine that the fourth indicator is greater than the corresponding maximum threshold, and schedule part or all of the first target receiving threads as the second target receiving threads.

In a possible implementation, the adjusting module 420 is configured to determine that the third indicator is greater than the corresponding regular threshold and less than the corresponding maximum threshold and there are remaining idle receiving threads, and set the remaining idle receiving threads as the second target receiving threads.

In a possible implementation manner, the adjusting module 420 is configured to, when it is determined that there is no unallocated idle receiving thread in the gateway layer, schedule part or all of the receiving threads in the first target receiving threads as the second target receiving threads if the fourth index is greater than the corresponding maximum threshold.

The picture stream access apparatus 400 provided in the embodiment of the present application has the same implementation principle and the same technical effect as those of the foregoing method embodiments, and for the sake of brief description, no mention is made in the apparatus embodiment, and reference may be made to the corresponding contents in the foregoing method embodiments.

In addition, an embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computer, the method for accessing a picture stream as described above is performed.

In addition, please refer to fig. 4, an electronic device 100 is further provided in an embodiment of the present application. The electronic device 100 may include a processor 110 and a memory 120. It should be noted that the components and structure of electronic device 100 shown in FIG. 4 are exemplary only, and not limiting, and electronic device 100 may have other components and structures as desired.

The processor 110, memory 120, and other components that may be present in the electronic device 100 are electrically connected to each other, directly or indirectly, to enable the transfer or interaction of data. For example, the processor 110, the memory 120, and other components that may be present may be electrically coupled to each other via one or more communication buses or signal lines.

The above structure of the server to which the gateway layer 11, the service layer 12 or the algorithm layer 13 belongs may be as shown in fig. 4.

When the electronic device 100 is the service layer 12, the memory 120 is used to store a program, for example, a program corresponding to the above-mentioned picture stream access method or the above-mentioned picture stream access apparatus is stored. Optionally, when the picture stream access device is stored in the memory 120, the picture stream access device includes at least one software functional module that can be stored in the memory 120 in the form of software or firmware (firmware).

Optionally, the software function module included in the picture stream access apparatus may also be solidified in an Operating System (OS) of the electronic device 100.

The processor 110 is configured to execute an executable module stored in the memory 120, such as a software functional module or a computer program included in the picture stream access apparatus. When the processor 110 receives the execution instruction, it may execute the computer program, for example, to perform: acquiring an index for representing the picture processing pressure of the picture processing system in real time; and dynamically adjusting the access strategy of the gateway layer 11 to the picture stream according to the size relationship between the index and the corresponding preset index.

Of course, the method disclosed in any of the embodiments of the present application can be applied to the processor 110, or implemented by the processor 110.

In summary, the method, the apparatus, the image processing system, the electronic device, and the computer-readable storage medium for accessing the image stream according to the embodiments of the present invention obtain an index representing an image processing pressure of the image processing system in real time; and then, dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the indexes and the corresponding preset indexes. In the process, when the image processing pressure of the image processing system is higher, the access quantity of the gateway layer to the image stream can be reduced, so that the image processing pressure of the image processing system is reduced, and the normal operation of the image processing system is ensured; when the image processing pressure of the image processing system is low, the access quantity of the gateway layer to the image stream can be increased, so that the image processing resources of the image processing system are fully utilized, the image processing system is kept in the optimal state, and the working efficiency of the image processing system is improved.

It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application or portions thereof that substantially contribute to the prior art may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a notebook computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.

Claims

1. A picture stream access method is applied to a service layer of a picture processing system, the picture processing system further comprises a gateway layer in communication connection with the service layer, the gateway layer is used for accessing a picture stream, and the method comprises the following steps:

acquiring an index for representing the picture processing pressure of the picture processing system in real time;

and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the index and the corresponding preset index.

2. The method according to claim 1, wherein the picture processing system further comprises an algorithm layer in communication connection with the service layer, the indexes comprise a first index for characterizing picture feature extraction capability of the algorithm layer, the preset index corresponding to the first index comprises a first index conventional threshold value for characterizing conventional picture feature extraction capability of the algorithm layer and a first index maximum threshold value for characterizing maximum picture feature extraction capability of the algorithm layer, and the first index conventional threshold value is smaller than the first index maximum threshold value; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index includes:

and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation among the first index, the first index conventional threshold and the first index maximum threshold.

3. The method according to claim 2, wherein the gateway layer includes multiple receiving threads for accessing the picture stream, the pictures included in the picture stream are pre-prioritized, and the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the first indicator, the first indicator normal threshold, and the first indicator maximum threshold includes:

when the first index is determined to be larger than the first index conventional threshold and smaller than the first index maximum threshold, controlling the gateway layer to close part of first target receiving threads;

when the first index is determined to be larger than or equal to the first index maximum threshold value, controlling the gateway layer to close all first target receiving threads;

the priority of the picture accessed by the first target receiving thread is lower than the priority of the pictures accessed by other receiving threads.

4. The method of claim 3, wherein after the controlling the gateway layer to turn off all target receive threads or after the controlling the gateway layer to turn off a portion of a first target receive thread, the method further comprises:

controlling the gateway layer portion to restart/fully restart the first target receiving thread that has been shut down upon determining that the first metric is less than the first metric conventional threshold.

5. The method according to claim 1, wherein the index includes a second index for representing the currently processed number of concurrent requests of the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold for representing the currently conventionally processable number of concurrent requests of the gateway layer; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index includes:

and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the second index and the second index conventional threshold.

6. The method according to claim 2 or 3, wherein the index further includes a second index for characterizing the number of concurrent requests currently processed by the gateway layer, and the preset index corresponding to the second index includes a second index conventional threshold for characterizing the number of conventionally processable concurrent requests currently processed by the gateway layer; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the index and the corresponding preset index further comprises:

and when the first index is determined to be smaller than or equal to the first index conventional threshold, dynamically adjusting the access strategy of the gateway layer to the picture flow according to the size relation between the second index and the second index conventional threshold.

7. The method according to claim 5 or 6, wherein the gateway layer includes a plurality of receiving threads for accessing the picture stream, the pictures included in the picture stream are preset with priorities, the receiving threads include a first target receiving thread and a second target receiving thread, and the priority of the pictures accessed by the first target receiving thread is lower than the priority of the pictures accessed by the second target receiving thread; the second index comprises a third index corresponding to the first target receiving thread and a fourth index corresponding to the second target receiving thread; the second index normal threshold value comprises a normal threshold value corresponding to the third index and a normal threshold value corresponding to the fourth index; the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the second index and the second index conventional threshold includes:

when one of the third index and the fourth index is determined to be smaller than or equal to the corresponding conventional threshold value, and the other index is determined to be larger than the corresponding conventional threshold value, whether the number of receiving threads corresponding to the other index is increased or not is determined.

8. The method according to claim 7, wherein the preset metrics corresponding to the second metric further include a second metric maximum threshold value for characterizing a current maximum number of processable concurrent requests of the gateway layer, the second metric maximum threshold value including a maximum threshold value corresponding to the fourth metric;

the determining whether to increase the number of receiving threads corresponding to the another indicator includes:

judging whether an unallocated idle receiving thread exists in the gateway layer;

if so, setting the idle receiving thread as a receiving thread corresponding to the other index;

if the other index is the fourth index and the fourth index is larger than the maximum threshold corresponding to the fourth index, scheduling part or all of the receiving threads in the first target receiving thread as the second target receiving thread.

9. The method of claim 7, wherein the dynamically adjusting the access policy of the gateway layer to the picture stream according to the size relationship between the second indicator and the second indicator normal threshold further comprises:

increasing the number of at least one of the first target receiving thread and the second target receiving thread when it is determined that the third metric and the fourth metric are both greater than the corresponding conventional thresholds.

10. The method according to claim 9, wherein the preset metrics corresponding to the second metric further include a second metric maximum threshold value for characterizing a current maximum number of processable concurrent requests of the gateway layer, and the second metric maximum threshold value includes a maximum threshold value corresponding to the third metric and a maximum threshold value corresponding to the fourth metric;

the increasing the number of at least one of the first target receiving thread and the second target receiving thread when it is determined that the third metric and the fourth metric are both greater than the corresponding regular threshold includes:

if so, determining the allocation strategy of the idle receiving thread according to the third index, the conventional threshold corresponding to the third index, the maximum threshold corresponding to the third index, the fourth index, the conventional threshold corresponding to the fourth index and the maximum threshold corresponding to the fourth index.

11. The method of claim 10, wherein determining the allocation policy for the idle receiving thread comprises:

if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread;

if the third index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, and the fourth index is greater than the corresponding conventional threshold and less than the corresponding maximum threshold, setting one part of the idle receiving threads as the second target receiving thread, and setting the other part of the idle receiving threads as the first target receiving thread;

if the fourth index is greater than the corresponding maximum threshold value, and the third index is greater than the corresponding conventional threshold value and smaller than the corresponding maximum threshold value, setting the idle receiving thread as the second target receiving thread;

and if the third index is greater than the corresponding maximum threshold value and the fourth index is greater than the corresponding conventional threshold value and less than the corresponding maximum threshold value, setting the idle receiving thread as the first target receiving thread.

12. The method of claim 11, wherein after the setting the idle receive thread as the second target receive thread, the method further comprises:

and determining that the fourth index is greater than the corresponding maximum threshold value, and scheduling part or all of the first target receiving threads as the second target receiving threads.

13. The method of claim 11, wherein after the setting the idle receive thread as the first target receive thread, the method further comprises:

and determining that the third index is larger than the corresponding conventional threshold and smaller than the corresponding maximum threshold and remaining idle receiving threads still exist, and setting the remaining idle receiving threads as the second target receiving threads.

14. The method of claim 10, wherein after said determining whether there are any unallocated idle receive threads within the gateway layer, the method further comprises:

if the fourth index is larger than the corresponding maximum threshold value, scheduling part or all of the receiving threads in the first target receiving threads as the second target receiving threads.

15. An apparatus for accessing a picture stream, applied to a service layer of a picture processing system, wherein the picture processing system further includes a gateway layer in communication connection with the service layer, and the gateway layer is configured to access the picture stream, the apparatus comprising:

the acquisition module is used for acquiring an index for representing the image processing pressure of the image processing system in real time;

and the adjusting module is used for dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the index and the corresponding preset index.

16. An electronic device, comprising: a memory and a processor, the memory and the processor connected;

the memory is used for storing programs;

the processor calls a program stored in the memory to perform the method of any of claims 1-14.

17. A computer-readable storage medium, on which a computer program is stored which, when executed by a computer, performs the method of any one of claims 1-14.

18. A picture processing system is characterized by comprising a service layer, a gateway layer and an algorithm layer which are in communication connection with the service layer,

the gateway layer is used for accessing the picture stream;

the algorithm layer is used for extracting the characteristics of the accessed picture flow;

the service layer is used for acquiring indexes, used for representing the picture processing pressure of the picture processing system, of the gateway layer and/or the algorithm layer, and dynamically adjusting the access strategy of the gateway layer to the picture stream according to the size relation between the indexes and the corresponding preset indexes.