WO2022082361A1

WO2022082361A1 - Image information processing method and terminal device

Info

Publication number: WO2022082361A1
Application number: PCT/CN2020/121894
Authority: WO
Inventors: 邓宝华
Original assignee: 深圳市锐明技术股份有限公司
Priority date: 2020-10-19
Filing date: 2020-10-19
Publication date: 2022-04-28
Also published as: CN112425149B; CN112425149A

Abstract

The present application provides an image information processing method, comprising: each time a first data transmission module obtains image data, the first data transmission module sends the image data at least two image information identification devices, respectively; for each image information identification device, the first data transmission module stores the latest processed data returned by the image information identification device in a preset memory space; a second data transmission module obtains a processing data set from the preset memory space at a preset period; each time a second data processing module obtains K processing data sets in total, the second data processing module sends the K processing data sets to an image signal processor. The method solves the problem that after image identification algorithms identify different image features, transmission frequencies at which the different image features are asynchronously transmitted to an image signal processor are different, and therefore, the image signal processor needs to perform dense information interaction.

Description

Image information processing method and terminal device

technical field

The present application belongs to the technical field of image information processing, and in particular, relates to an image information processing method, a terminal device, and a computer-readable storage medium.

Background technique

In practical applications, in order to better meet the needs of different application scenarios, after acquiring the image data, it is often necessary to use some image recognition algorithms to identify the image, such as face area, eye area or other characteristic areas, to determine The region of interest in the image, thereby determining the image processing parameters such as exposure parameters for the image.

technical problem

At present, in order to obtain more accurate image feature information, it is often necessary to rely on image recognition algorithms to identify image features, but the introduction of image recognition algorithms often results in a delay in data processing; and if more than one image recognition algorithm is introduced , then, different image recognition algorithms often have different processing cycles for data, so that after each image recognition algorithm recognizes different image features, the transmission frequency of asynchronous transmission to the image signal processor is different, so that the image signal processor needs to perform Intensive information interaction leads to a large consumption of corresponding transmission resources and storage resources.

technical solutions

The embodiments of the present application provide an image information processing method, a terminal device, and a computer-readable storage medium, which can solve the problem that after each image recognition algorithm recognizes different image features, the transmission frequencies of asynchronous transmission to the image signal processor are different, so that the image signal The processor needs to perform intensive information interaction, which leads to the problem of large consumption of corresponding transmission resources and storage resources.

In a first aspect, an embodiment of the present application provides an image information processing method, which is applied to a terminal device, where the terminal device includes a first data transmission module, a second data transmission module, and an image signal processor, and the image information processing method include:

Each time the first data transmission module acquires image data, it sends the image data to at least two image information recognition devices respectively, so as to obtain the processing respectively returned by each of the image information recognition devices for the image data data;

For each image information recognition device, the last processing data returned by the image information recognition device is stored in a preset memory space through the first data transmission module, and the latest processing data is used as the the current processing data corresponding to the image information identification device;

Through the second data transmission module, a set of processing data is acquired from the preset memory space at a preset period, wherein each preset period corresponds to a set of processing data, and one set of processing data is included in a corresponding preset period. Set a period, the current processing data corresponding to each of the image information recognition devices obtained by the second data transmission module from the preset memory space;

The second data processing module sends the K processing data sets to the image signal processor each time the K processing data sets are acquired cumulatively, where K is a positive integer.

In a second aspect, an embodiment of the present application provides a terminal device, including a first data transmission module, a second data transmission module, and an image signal processor, wherein:

The first data transmission module is configured to: each time image data is acquired, send the image data to at least two image information recognition devices respectively, so as to obtain the respective image data of each of the image information recognition devices for the image data. returned processed data;

The first data transmission module is further configured to: for each image information identification device, store the latest processing data returned by the image information identification device in a preset memory space through the first data transmission module, and use the latest processing data as the current processing data corresponding to the image information recognition device;

The second data transmission module is used for: acquiring a processing data set from the preset memory space at a preset cycle, wherein each preset cycle corresponds to a processing data set, and one of the processing data sets is included in the corresponding processing data set. A preset period, the current processing data corresponding to each of the image information recognition devices obtained by the second data transmission module from the preset memory space;

The second data processing module is further configured to send the K processing data sets to the image signal processor each time the K processing data sets are acquired cumulatively, where K is a positive integer.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, implements the image information processing method according to the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer program product that, when the computer program product runs on a terminal device, enables the terminal device to execute the image information processing method described above in the first aspect.

beneficial effect

In this embodiment of the present application, each time the first data transmission module acquires image data, the first data transmission module sends the image data to at least two image information recognition apparatuses, respectively, so as to obtain the target data of each of the image information recognition apparatuses for the image. The processing data returned by the data respectively; then, for each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the preset memory space through the first data transmission module, and The most recent processing data is used as the current processing data corresponding to the image information recognition device; at this time, the first data transmission module can respectively carry out information interaction with each of the image information recognition devices to obtain and respectively The latest processing data returned by each image information recognition device is stored. In addition, in this embodiment of the present application, the second data transmission module may acquire a processing data set from the preset memory space at a preset period, wherein each preset period corresponds to a processing data set, and each preset period corresponds to a processing data set, and each The processing data set is included in the corresponding preset period, and the second data transmission module obtains the current processing data corresponding to each of the image information recognition devices from the preset memory space; and, in the Each time the second data processing module acquires K processing data sets accumulatively, it sends the K processing data sets to the image signal processor, so that through the second data transmission module, at different time nodes, The latest processing data corresponding to each of the image information recognition devices is collected as the processing data set corresponding to the corresponding time node, and then each processing data set is sent to the image signal processing at a relatively stable frequency The image signal processor can also obtain the required image features at a relatively stable speed and perform image processing, thereby avoiding intensive information interaction and saving transmission resources and storage resources.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only for the present application. In some embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic diagram of an exemplary information interaction of a first data transmission module, a second data transmission module, an image signal processor, and an image information recognition device provided by an embodiment of the present application;

2 is a schematic flowchart of an image information processing method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of an exemplary information interaction among the second data transmission module, the first data transmission module, and each image information identification device provided by an embodiment of the present application.

4 is a schematic flowchart of another image information processing method provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a terminal device according to an embodiment of the present application.

Embodiments of the present invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

In order to illustrate the above-mentioned technical solutions of the present application, the following specific embodiments are used for description.

For example, in the field of vehicle security, driver behavior recognition (Driving State Montoring, DSM) cameras film the driver's driving behavior. At this time, it is necessary to obtain the image feature information (such as feature point coordinate information, etc.) of the face area and facial details such as the eye area and the glasses area, so as to better expose the area of interest such as the face, and obtain better quality Good shot image.

At present, in order to obtain more accurate image feature information, it is often necessary to rely on image recognition algorithms to identify image features, but the introduction of image recognition algorithms often results in a delay in data processing; and if more than one image recognition algorithm is introduced , then, different image recognition algorithms often have different processing cycles for data, so that after each image recognition algorithm recognizes different image features, the transmission frequency of asynchronous transmission to the image signal processor is different, resulting in corresponding transmission resources and storage. Resource consumption is high.

However, through the embodiment of the present application, the first data transmission module can respectively conduct information interaction with each of the image information recognition devices, so as to obtain and respectively store the latest processing data returned by each image information recognition device; The second data transmission module conveniently collects the latest processing data corresponding to each of the image information recognition devices at different time nodes, as the processing data set corresponding to the corresponding time node, and then processes each data set. The data set is sent to the image signal processor at a relatively stable frequency, so that the image signal processor can obtain the required image features and perform image processing at a relatively stable speed, thereby avoiding intensive information interaction, and can Save transmission resources and storage resources.

The image information processing method in the embodiment of the present application can be applied to a terminal device, and the specific type of the terminal device is not limited in any way.

Exemplarily, the terminal device may be a vehicle-mounted device, a server, a desktop computer, a mobile phone, a tablet computer, a wearable device, an augmented reality (AR)/virtual reality (VR) device, a laptop, a super One of a mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA) and the like.

In this embodiment of the present application, the terminal device may include a first data transmission module, a second data transmission module, and an image signal processor.

Wherein, the first data transmission module may be a certain hardware module in the terminal device, or may be a set of at least two hardware modules in the terminal device, or may also be a hardware module in the terminal device A certain part of a specified hardware module used to implement a specified function, etc. The second data transmission module may be a certain hardware module in the terminal device, or may be a set of at least two hardware modules in the terminal device, or may be a specified device in the terminal device. A certain part of a hardware module used to implement a specified function, etc. It should be noted that, the first data transmission module and the second data transmission module may have different independent hardware structures, and may also be located in the same hardware structure and correspond to different parts of the same hardware structure. For example, the first data transmission module and the second data transmission module may be different data processing units in the same hardware structure to run different threads and so on respectively.

The image signal processor (Image Signal Processor, ISP) can be used to process the image data output by the image sensor to obtain the output image for display. In the embodiment of the present application, the image signal processor may perform processing such as exposure on the image data in combination with the processing data returned by each of the image information recognition devices for the image data.

In the embodiments of the present application, for any of the image information recognition apparatuses, the image information recognition apparatus may be set in the terminal device, and may be an external device capable of information transmission with the terminal device, for example, it may be External devices such as cloud servers. An image recognition algorithm may be deployed in the image information recognition device to perform image recognition on the input image data.

The number of the image information identification devices may be set according to actual scene requirements, which is not limited herein.

As shown in FIG. 1 , it is a schematic diagram of an exemplary information interaction of the first data transmission module, the second data transmission module, the image signal processor and the image information identification device.

As shown in FIG. 1 , the first data transmission module can exchange information with each image information recognition device, and the first data transmission module and the second data transmission module can exchange information, and the second data transmission module can exchange information with each other. Information interaction can be performed between the transmission module and the image signal processor.

FIG. 2 shows a flowchart of an image information processing method provided by an embodiment of the present application.

Specifically, as shown in FIG. 2 , the image information processing method may include:

Step S201, each time the first data transmission module acquires image data, it sends the image data to at least two image information recognition devices respectively, so as to obtain the respective images of the image information recognition devices for the image data. The returned processed data.

In the embodiments of the present application, for example, the image data may be acquired by an image sensor in the camera of the terminal device, or may be transmitted to the terminal device through other devices communicatively connected to the terminal device. ; Alternatively, the image data may also be local image data pre-stored in the terminal device or the like. The specific acquisition manner of the image data is not limited herein.

In an embodiment, the image data may be image data acquired in real time by the second data transmission module and sent to the first data transmission module at a preset period.

An image recognition algorithm may be deployed in any of the image information recognition apparatuses, and the image recognition algorithms in different image information recognition apparatuses may be different. The type of the image recognition algorithm can be determined according to actual scene requirements.

Exemplarily, if the image data is image data collected in a vehicle-mounted security system, the image information required by the image signal processor may include one or more of the following information:

1. Image lighting information. Exemplarily, the image illumination information may be used to indicate whether the image is in a backlight state, a dark light state, a daytime state, a night scene state, or the like.

2. Face area information. The face area information may include face edge coordinates, facial feature point coordinates, and other feature point coordinates, mask information, and the like.

3. Eye information. The eye information may include, for example, the coordinates of the center of the human eye, the width and height of the human eye, and the coordinates of the outline of the eye.

4. The glasses information. The glasses information may include judgment information about whether to wear glasses, information about the type of glasses, judgment information about whether to reflect light, and the like.

5. Facial orientation information. The face orientation information may include judgment information on the front face, the left face, or the right face.

Then, correspondingly, in some embodiments, any one of the image information identification devices is used to identify at least one of image illumination information, face area information, eye information, glasses information, and face orientation information, and Different image information identification devices are used to identify different information.

Of course, according to scene requirements, the image information identification device can also be used to identify other image information.

In this embodiment of the present application, the first data transmission module sends the image data to the at least two image information recognition apparatuses. The sending strategy of the image data may be determined according to the source of the image data, the data type, the user's preset information, and so on. Sure. For example, if the image data may come from two cameras, then if the image data comes from camera A, the image data can be transmitted to the image information recognition device a and the image information recognition device b. And if the image data comes from the camera B, the image data can be transmitted to the image information recognition device a and the image information recognition device c.

Step S202, for each image information recognition device, store the latest processing data returned by the image information recognition device in a preset memory space through the first data transmission module, and store the latest processing data in a preset memory space. The data is used as the current processing data corresponding to the image information identification device.

In the embodiment of the present application, the first data transmission module may be respectively associated with each image information recognition device, and respectively acquire and store the current processing data corresponding to each image information recognition device, so that the preset memory space is The latest processing data corresponding to each image information recognition device can be stored in the .

There may be various specific storage structures in the preset memory space. Exemplarily, the preset memory space includes memory subspaces corresponding to each image information recognition device one-to-one, so that the currently processed data corresponding to each image information recognition device can be stored without interfering with each other.

Step S203, through the second data transmission module, obtain a processing data set from the preset memory space at a preset cycle, wherein each preset cycle corresponds to a processing data set, and one of the processing data sets is included in the processing data set. In the corresponding preset period, the second data transmission module obtains the current processing data corresponding to each of the image information recognition devices from the preset memory space.

In the embodiment of the present application, since the preset memory space can store the latest processing data corresponding to each image information recognition device, the second data transmission module can conveniently collect each of the data at different time nodes. The latest processing data respectively corresponding to the image information recognition device is used as the processing data set corresponding to the corresponding time node.

Step S204, each time the second data processing module acquires K processing data sets accumulatively, sends the K processing data sets to the image signal processor, where K is a positive integer.

The K processing data sets may be set according to actual scene requirements.

In some embodiments, the value of K is determined according to the preset period and the operating speed of the image signal processor.

For example, if the preset period is 60ms, and each data processing process of the image signal processor takes 300ms, then the value of K can be 300/60=5, that is, the second Each time the data processing module acquires 5 processing data sets accumulatively, it sends the 5 processing data sets to the image signal processor.

In this embodiment of the present application, after the second data processing module acquires K processing data sets accumulatively each time, the K processing data sets are sent to the image signal processor, which can save storage resources and avoid memory data The problem that hardware resources are occupied by a large number of hardware resources caused by the intensive interaction, thereby saving transmission resources and storage resources.

In addition, through the embodiments of the present application, the image signal processor can obtain the required image features at a relatively stable speed and perform image processing, thereby stably outputting the processed image and improving the output image quality.

In some embodiments, the preset memory space includes memory subspaces corresponding to each image information recognition device one-to-one, and the memory subspace is used to store the latest processing returned by the corresponding image information recognition device data, each memory subspace includes two paired memory blocks, one of the two paired memory blocks is marked as the first mark, and the other one of the two paired memory blocks is marked as second identification;

For each image information recognition device, the last processing data returned by the image information recognition device is stored in a preset memory space through the first data transmission module, and the latest processing data is stored in the preset memory space. As the current processing data corresponding to the image information recognition device, it includes:

For each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the target memory block through the first data transmission module, as the current processing data of the image information recognition device, Wherein, the target memory block is the memory block corresponding to the first identifier in the memory subspace corresponding to the image information recognition device, and the target memory block after the storage is completed does not store any memory other than the most recent one. other rounds of processing data;

After the storage is completed, the identification of the target memory block after the storage is completed is updated to the second identification, and the identification of the memory block that is paired with the target memory block is updated to the first identification;

The acquiring, by the second data transmission module, the processing data set from the preset memory space at a preset period includes:

Through the second data transmission module, each of the current image data is acquired from each of the memory blocks corresponding to the second identifier in the preset memory space at a preset period to obtain the processing data set.

As shown in FIG. 3 , it is a schematic diagram of an exemplary information interaction among the second data transmission module, the first data transmission module and each image information recognition device.

Wherein, for the image information identification device 1 , the first data transmission module includes a memory subspace corresponding to the image information identification device 1 . The memory subspace may include two paired memory blocks, one of the two paired memory blocks corresponding to the first identifier buf1, and the other one of the two paired memory blocks corresponding to the identifier. For the second identifier buf0.

It can be understood that the memory block identified as the first identification buf1 is a memory block that can write the processing data transmitted by the image information identification device 1, and the memory block identified as the second identification buf0 is capable of being read by the second data transmission module. fetched memory block. At this time, the memory space used for writing data and the memory space used for reading data are separated to avoid operation conflicts caused by writing data and reading data at the same time.

In addition, in the embodiment of the present application, for each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the target memory block through the first data transmission module, and after the storage is completed, The identification of the target memory block after the storage is completed is updated to the second identification, and the identification of the memory block paired with the target memory block is updated to the first identification, after completing the storage of the latest processing data, By changing the mark of the memory block, the processing data in the memory block that can be read by the second data transmission module can be quickly and easily updated, avoiding the need for simultaneous write and read operations in the same memory space. The latency of read and write operations due to waiting for data to be written.

In some embodiments, each time the first data transmission module acquires image data, the image data is respectively sent to at least two image information recognition apparatuses, so as to acquire the image data of each of the image information recognition apparatuses. The processed data returned by the data, including:

Send real-time image data to the first data transmission module by the second data transmission module at a preset period;

Each time the first data transmission module acquires the image data sent by the second data transmission module, the image data is respectively sent to at least two image information recognition devices, so as to obtain each of the image information recognition devices The processing data returned respectively for the image data;

Each time the second data transmission module sends real-time image data to the first data transmission module, it acquires a processing data set from the preset memory space.

In the embodiment of the present application, the second data transmission module sends real-time image data to the first data transmission module at a preset period, and each time the real-time image data is sent to the first data transmission module at the same time , acquiring the processing data set from the preset memory space can reduce the delay between the acquired processing data set and the sent real-time image data, improve the efficiency of information interaction, and reduce the impact on information transmission resources. occupancy.

In some embodiments, as shown in FIG. 4 , after the K processing data sets are sent to the image signal processor, the method further includes:

Step S401, after each time the image signal processor receives the K processing data sets, determines a region of interest in the image data acquired in real time according to the K processing data sets;

Step S402, performing exposure processing on the image data acquired in real time according to the region of interest.

In this embodiment of the present application, the region of interest (region of interest, ROI) can be used to outline the area to be processed in the form of boxes, circles, ellipses, irregular polygons, etc. The content of the region of interest can be determined according to the actual scene and the corresponding image recognition algorithm. For example, in an application scenario of vehicle security, the region of interest may be a face region.

The region of interest may be determined based on at least a portion of the processing data in the K sets of processing data. For example, it can be determined according to the facial contour feature point coordinates, the human eye feature point coordinates, and mask information in the processed data. There may be various specific determination methods. For example, the smallest square area that can include the face contour feature points can be used as the region of interest; in addition, the distance from the edge of the region of interest to the human face can also be determined according to the distance between the camera and the human face. Specify the distance between feature points, etc.

In some embodiments, the step S401 may include:

After each time the image signal processor receives the K processing data sets, the initial region of interest in the real-time acquired image data is determined according to the K processing data sets;

Determine the estimated offset information of the initial region of interest according to the K processing data sets and the N historical processing data sets, where N is a positive integer;

According to the estimated offset information and the initial region of interest, the region of interest in the image data acquired in real time is determined.

Wherein, for the K processing data sets received by the image signal processor each time, when determining the region of interest in the image data acquired in real time according to the K processing data sets, the K processing data sets are There is a delay in the aggregation relative to the image data acquired in real time by the image signal processor. For example, it takes 300ms for the second data processing module to acquire K pieces of processing data sets accumulatively each time, and one processing cycle for the image data by the image information recognition device is 30ms, then, the K pieces of processing data The delay of the data set relative to the image data acquired in real time by the image signal processor is at least 330ms.

It can be seen that the initial region of interest determined according to the K processing data sets may have an offset relative to the real region of interest of the image data acquired in real time.

In this embodiment of the present application, the estimated offset information of the initial region of interest may be determined according to the K processing data sets and the N historical processing data sets. Wherein, the number of the historical processing data sets may be determined according to factors such as precision requirements and the value of K.

In some embodiments, the estimated offset information includes a compensation region;

The determining, according to the estimated offset information and the initial region of interest, the region of interest in the image data obtained in real time includes:

combining the compensation region and the initial region of interest to obtain the region of interest;

The performing exposure processing on the image data obtained in real time according to the region of interest includes:

Exposure processing is performed on the image data acquired in real time according to the first weight corresponding to the compensation area and the second weight corresponding to the initial region of interest.

In this embodiment of the present application, the first weight may be used to indicate the exposure level corresponding to the compensation area, and the second weight may be used to indicate the exposure level corresponding to the initial region of interest. The values of the first weight and the second weight may be predetermined. Exemplarily, the first weight may be smaller than the second weight, for example, the first weight may be 0.5 times the second weight, and at this time, the exposure degree of the initial region of interest is relatively high, The exposure of the compensation area is slightly weaker relative to the initial area of interest, so that the initial area of interest is more prominent, and the exposure of the compensation area is slightly increased relative to other uninteresting areas, so that the exposure Processing is more accurate.

In some embodiments, the determining the estimated offset information of the initial region of interest according to the K processing data sets and the N historical processing data sets includes:

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, the first moving speed of the target object corresponding to the historical period is calculated, wherein the target object and the initial associated with the region of interest;

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second moving speed of the target object corresponding to the target period;

determining the target acceleration of the target object according to the first moving speed and the second moving speed;

According to the target acceleration, the estimated offset information of the initial region of interest is determined.

Wherein, the first moving speed may include a first lateral speed and/or a first longitudinal speed, the second moving speed may include a second lateral speed and/or a second longitudinal speed, and accordingly, the target acceleration is also Lateral target acceleration and/or longitudinal target acceleration may be included.

In this embodiment of the present application, the first moving speed of the target object corresponding to the historical period can be calculated, so as to obtain the movement of the target object in the historical period; The second moving speed corresponding to the period, so as to obtain the movement of the target object in the target period, so that the movement of the photographed target object between two different time periods can be compared to estimate the target. The moving trend of the object is used to determine the estimated offset information.

Wherein, exemplarily, for the historical period, the coordinates corresponding to a certain specified feature point in each processing data set corresponding to the historical period can be obtained, so as to obtain the specified feature point in the historical period. The corresponding image position information at different time points, so as to obtain the first average speed of the specified feature point in the historical period, and then the first average speed can be used as the first moving speed. For the target period, similarly, the second moving speed is obtained by the above-mentioned method for obtaining the first moving speed.

Then, according to the first moving speed and the second moving speed, the target acceleration of the target object may be determined, so that the target acceleration according to the second moving speed relative to the first moving speed is determined by the target acceleration. According to the change situation, the moving trend of the target object can be estimated to determine the estimated offset information.

For example, if the first lateral velocity is in the same direction as the second lateral velocity, and the first lateral velocity is greater than the second lateral velocity, it is possible to estimate where the target object is located along the second lateral velocity slow down in the direction. At this time, if the second lateral velocity is greater than a preset velocity threshold, the compensation area can be obtained by expanding along the direction of the second lateral velocity on the basis of the initial region of interest; If the second lateral velocity is less than the preset velocity threshold, it can be estimated that the target object may move in the opposite direction. Therefore, on the basis of the initial region of interest, the direction of the second lateral velocity and the The two directions of reversal of the lateral velocity are extended simultaneously to obtain the compensation region.

However, if the first lateral velocity and the second lateral velocity are in different directions, and the first lateral velocity is greater than the second lateral velocity, it can be estimated that the target object is located along the second lateral velocity. direction acceleration. At this time, the initial region of interest may be expanded along the direction of the second lateral velocity to obtain the compensation region.

Exemplarily, the magnitude of the second lateral velocity is multiplied by a preset coefficient to obtain the width extended by the compensation region along a specified lateral direction.

It can be understood that, for the first longitudinal speed and the second longitudinal speed, the compensation area in the longitudinal direction can also be obtained by the above method.

In some embodiments, the image information processing method further includes:

determining the first distance between the camera that collected the image data and the target object in the historical period, and determining the second distance between the camera and the target object in the target period;

The calculating, according to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, calculates the first moving speed of the target object corresponding to the historical period, including:

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, calculate the first initial moving speed of the target object corresponding to the historical period;

obtaining the first moving speed according to the first distance and a preset distance-pixel displacement mapping relationship;

The calculating, according to the target period corresponding to the K processing data sets and the K processing data sets, the second movement speed of the target object corresponding to the target period, including:

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second initial moving speed of the target object corresponding to the target period;

The second moving speed is obtained according to the second distance and a preset distance-pixel displacement mapping relationship.

In the embodiment of the present application, when the distance between the camera and the target object is different, the corresponding relationship between the pixel displacement in the collected image and the moving speed of the target object in the real scene is also different.

Therefore, the distance-pixel displacement mapping relationship can be established in advance, so as to unify the mapping relationship between the pixel displacement between different images to the three-dimensional space based on the distance between the camera and the target object, and then according to the unified mapping relationship The pixel displacement after the mapping relationship determines the first moving speed and the second moving speed.

Wherein, for example, for a certain specified length in the three-dimensional space, a mapping relationship between different distances and pixel displacements may be determined in advance to obtain the distance-pixel displacement mapping relationship. At this time, for a certain distance, by querying the distance-pixel displacement mapping relationship, it can be determined how many pixel displacements in the image captured by the camera at the distance can indicate the specified length in the three-dimensional space. It should be noted that, the distance-pixel displacement mapping relationship can also be regarded as a distance-pixel moving speed mapping relationship.

Therefore, after the first initial movement speed is obtained by calculation, the first movement speed can be obtained according to the first distance and the preset distance-pixel displacement mapping relationship, and the second initial movement speed is obtained after calculation After the speed, the second moving speed is obtained according to the second distance and a preset distance-pixel displacement mapping relationship. At this time, the obtained first moving speed and the second moving speed can be equivalently regarded as moving speeds in the same space coordinate system, thereby ensuring the accuracy of the preset offset information.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

As shown in FIG. 5 , the terminal device 5 in this embodiment includes: a first data transmission module 501, a second data transmission module 502 and an image signal processor 503, wherein:

The first data transmission module 501 is configured to: each time image data is acquired, send the image data to at least two image information recognition devices respectively, so as to obtain the information about the image data by each of the image information recognition devices. The processed data returned separately;

The first data transmission module 501 is further configured to: for each image information identification device, store the latest processing data returned by the image information identification device in a preset memory space through the first data transmission module 501 , and use the latest processing data as the current processing data corresponding to the image information recognition device;

The second data transmission module 502 is configured to: obtain a processing data set from the preset memory space at a preset cycle, wherein each preset cycle corresponds to a processing data set, and one of the processing data sets is included in the corresponding processing data set. The preset period, the second data transmission module 502 obtains the current processing data corresponding to each of the image information recognition devices from the preset memory space;

The second data processing module is further configured to send the K processing data sets to the image signal processor 503 each time the K processing data sets are acquired cumulatively, where K is a positive integer.

Optionally, the preset memory space includes memory subspaces corresponding to each image information recognition device one-to-one, and the memory subspace is used to store the latest processing data returned by the corresponding image information recognition device, Each memory subspace includes two paired memory blocks, one of the two paired memory blocks is identified as the first identification, and the other of the two paired memory blocks is identified as the second identification;

The first data transmission module 501 is used for:

For each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the target memory block through the first data transmission module 501 as the current processing data of the image information recognition device , wherein the target memory block is the memory block corresponding to the first identifier in the memory subspace corresponding to the image information recognition device, and the target memory block after the storage is completed does not store any data except for the most recent memory block. other rounds of processing data;

The second data transmission module 502 is used for:

Each of the current image data is acquired from each of the memory blocks corresponding to the second identifier in the preset memory space at a preset period to obtain the processing data set.

Optionally, the second data transmission module 502 is used for:

Send real-time image data to the first data transmission module 501 with a preset period;

The first data transmission module 501 is used for:

Each time the image data sent by the second data transmission module 502 is acquired, the image data is respectively sent to at least two image information recognition devices, so as to obtain the information about the image data by each of the image information recognition devices. The processed data returned separately;

The second data transmission module 502 is also used for:

Each time the second data transmission module 502 sends real-time image data to the first data transmission module 501, it acquires a processing data set from the preset memory space.

Optionally, the value of K is determined according to the preset period and the running speed of the image signal processor 503 .

Optionally, the image signal processor 503 is used for:

After each receiving the K processing data sets, determine the region of interest in the image data obtained in real time according to the K processing data sets;

According to the region of interest, exposure processing is performed on the image data acquired in real time.

Optionally, the image signal processor 503 is used for:

After each receiving the K processing data sets, determine the initial region of interest in the image data obtained in real time according to the K processing data sets;

Optionally, the estimated offset information includes a compensation area;

The image signal processor 503 is used for:

Optionally, the image signal processor 503 is used for:

The first data transmission module 501 may include at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, so that the first data transmission module 501 may implement any of the above. Steps performed on the first data transmission module 501 in each image information processing method embodiment.

The second data transmission module 502 may include at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, so that the second data transmission module 502 may implement any of the above Steps performed on the second data transmission module 502 in each image information processing method embodiment.

The image signal processor 503 may also include at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, so that the image signal processor 503 can implement any of the above Steps performed on the image signal processor 503 in the embodiment of the image information processing method.

It should be noted that, the first data transmission module 501 , the second data transmission module 502 and the image signal processor 503 may respectively include more or less components according to requirements. The components included in the data transmission module 501 , the second data transmission module 502 and the image signal processor 503 are not limited herein.

Embodiments of the present application further provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be implemented.

The embodiments of the present application provide a computer program product, when the computer program product runs on a terminal device, so that the terminal device can implement the steps in the foregoing method embodiments when executed.

If the above-mentioned integrated units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above-mentioned embodiments, which can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program is in When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code may be in the form of source code, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the photographing device/terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory ( RAM, Random Access Memory), electrical carrier signals, telecommunication signals, and software distribution media. For example, U disk, mobile hard disk, disk or CD, etc. In some jurisdictions, under legislation and patent practice, computer readable media may not be electrical carrier signals and telecommunications signals.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that the above-mentioned embodiments can still be used for The recorded technical solutions are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the present application. within the scope of protection of the application.

Claims

An image information processing method, characterized in that it is applied to a terminal device, wherein the terminal device includes a first data transmission module, a second data transmission module, and an image signal processor, and the image information processing method includes:

Each time the first data transmission module acquires image data, it sends the image data to at least two image information recognition devices respectively, so as to obtain the processing respectively returned by each of the image information recognition devices for the image data data;

For each image information recognition device, the last processing data returned by the image information recognition device is stored in a preset memory space through the first data transmission module, and the latest processing data is used as the the current processing data corresponding to the image information identification device;

Through the second data transmission module, a set of processing data is acquired from the preset memory space at a preset period, wherein each preset period corresponds to a set of processing data, and one set of processing data is included in a corresponding preset period. Set a period, the current processing data corresponding to each of the image information recognition devices obtained by the second data transmission module from the preset memory space;

The second data processing module sends the K processing data sets to the image signal processor each time the K processing data sets are acquired cumulatively, where K is a positive integer.
The image information processing method according to claim 1, wherein the preset memory space includes memory subspaces corresponding to each image information recognition device one-to-one, and the memory subspaces are used for storing corresponding images The latest processing data returned by the information identification device, each memory subspace includes two paired memory blocks, one of the two paired memory blocks corresponds to the first identifier, and the two paired memory blocks correspond to the first Another corresponding identifier in the memory block is the second identifier;

For each image information recognition device, the last processing data returned by the image information recognition device is stored in a preset memory space through the first data transmission module, and the latest processing data is stored in the preset memory space. As the current processing data corresponding to the image information recognition device, it includes:

For each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the target memory block through the first data transmission module, as the current processing data of the image information recognition device, Wherein, the target memory block is the memory block corresponding to the first identifier in the memory subspace corresponding to the image information recognition device, and the target memory block after the storage is completed does not store any memory other than the most recent one. other rounds of processing data;

After the storage is completed, the identification of the target memory block after the storage is completed is updated to the second identification, and the identification of the memory block that is paired with the target memory block is updated to the first identification;

The acquiring, by the second data transmission module, the processing data set from the preset memory space at a preset period includes:

Through the second data transmission module, each of the current image data is acquired from each of the memory blocks corresponding to the second identifier in the preset memory space at a preset period to obtain the processing data set.
The image information processing method according to claim 1, wherein each time the first data transmission module obtains image data, the first data transmission module sends the image data to at least two image information recognition devices respectively, so as to obtain the respective image data. The processing data respectively returned by the image information identification device for the image data include:

Send real-time image data to the first data transmission module by the second data transmission module at a preset period;

Each time the first data transmission module acquires the image data sent by the second data transmission module, the image data is respectively sent to at least two image information recognition devices, so as to obtain each of the image information recognition devices The processing data returned respectively for the image data;

The acquiring, by the second data transmission module, the processing data set from the preset memory space at a preset period includes:

Each time the second data transmission module sends real-time image data to the first data transmission module, it acquires a processing data set from the preset memory space.
The image information processing method according to claim 1, wherein the value of K is determined according to the preset period and the running speed of the image signal processor.
The image information processing method according to any one of claims 1 to 4, wherein after the K processing data sets are sent to the image signal processor, the method further comprises:

After each time the image signal processor receives the K processing data sets, the region of interest in the real-time acquired image data is determined according to the K processing data sets;

According to the region of interest, exposure processing is performed on the image data acquired in real time.
The image information processing method according to claim 5, wherein after each time the image signal processor receives the K processing data sets, the image data obtained in real time is determined according to the K processing data sets Areas of interest in , including:

After each time the image signal processor receives the K processing data sets, the initial region of interest in the real-time acquired image data is determined according to the K processing data sets;

Determine the estimated offset information of the initial region of interest according to the K processing data sets and the N historical processing data sets, where N is a positive integer;

According to the estimated offset information and the initial region of interest, the region of interest in the image data acquired in real time is determined.
The image information processing method according to claim 6, wherein the estimated offset information comprises a compensation area;

The determining, according to the estimated offset information and the initial region of interest, the region of interest in the image data obtained in real time includes:

combining the compensation region and the initial region of interest to obtain the region of interest;

The performing exposure processing on the image data obtained in real time according to the region of interest includes:

Exposure processing is performed on the image data acquired in real time according to the first weight corresponding to the compensation area and the second weight corresponding to the initial region of interest.
The image information processing method according to claim 6, wherein, determining the estimated offset information of the initial region of interest according to the K processing data sets and the N historical processing data sets, comprising:

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, the first moving speed of the target object corresponding to the historical period is calculated, wherein the target object and the initial associated with the region of interest;

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second moving speed of the target object corresponding to the target period;

determining the target acceleration of the target object according to the first moving speed and the second moving speed;

According to the target acceleration, the estimated offset information of the initial region of interest is determined.
The image information processing method according to claim 8, further comprising:

determining the first distance between the camera that collected the image data and the target object in the historical period, and determining the second distance between the camera and the target object in the target period;

The calculating, according to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, calculates the first moving speed of the target object corresponding to the historical period, including:

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, calculate the first initial moving speed of the target object corresponding to the historical period;

obtaining the first moving speed according to the first distance and a preset distance-pixel displacement mapping relationship;

The calculating, according to the target period corresponding to the K processing data sets and the K processing data sets, the second movement speed of the target object corresponding to the target period, including:

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second initial moving speed of the target object corresponding to the target period;

The second moving speed is obtained according to the second distance and a preset distance-pixel displacement mapping relationship.
A terminal device, comprising a first data transmission module, a second data transmission module and an image signal processor, wherein:

The first data transmission module is configured to: each time image data is acquired, send the image data to at least two image information recognition devices respectively, so as to obtain the respective image data of each of the image information recognition devices for the image data. returned processed data;

The first data transmission module is further configured to: for each image information identification device, store the latest processing data returned by the image information identification device in a preset memory space through the first data transmission module, and use the latest processing data as the current processing data corresponding to the image information recognition device;

The second data transmission module is used for: acquiring a processing data set from the preset memory space at a preset cycle, wherein each preset cycle corresponds to a processing data set, and one of the processing data sets is included in the corresponding processing data set. A preset period, the current processing data corresponding to each of the image information recognition devices obtained by the second data transmission module from the preset memory space;

The second data processing module is further configured to send the K processing data sets to the image signal processor each time the K processing data sets are acquired cumulatively, where K is a positive integer.
The terminal device according to claim 10, wherein the preset memory space includes memory subspaces corresponding to each image information recognition device one-to-one, and the memory subspaces are used to store corresponding image information recognition devices. The latest processing data returned by the device, each memory subspace includes two paired memory blocks, one of the two paired memory blocks corresponds to the first identifier, and the two paired memory blocks Another corresponding identification in is the second identification;

The first data transmission module is used for:

For each image information recognition device, the most recent processing data returned by the image information recognition device is stored in the target memory block through the first data transmission module, as the current processing data of the image information recognition device, Wherein, the target memory block is the memory block corresponding to the first identifier in the memory subspace corresponding to the image information recognition device, and the target memory block after the storage is completed does not store any memory other than the most recent one. other rounds of processing data;

After the storage is completed, the identification of the target memory block after the storage is completed is updated to the second identification, and the identification of the memory block that is paired with the target memory block is updated to the first identification;

The second data transmission module is used for:

Each of the current image data is acquired from each of the memory blocks corresponding to the second identifier in the preset memory space at a preset period to obtain the processing data set.
The terminal device according to claim 10, wherein the second data transmission module is used for:

sending real-time image data to the first data transmission module with a preset period;

The first data transmission module is used for:

Each time the image data sent by the second data transmission module is acquired, the image data is respectively sent to at least two image information recognition devices, so as to obtain the respective image data of each of the image information recognition devices for the image data. returned processed data;

The second data transmission module is also used for:

Each time the second data transmission module sends real-time image data to the first data transmission module, it acquires a processing data set from the preset memory space.
The terminal device according to claim 10, wherein the value of K is determined according to the preset period and the running speed of the image signal processor.
The terminal device according to any one of claims 10 to 13, wherein the image signal processor is used for:

After each receiving the K processing data sets, determine the region of interest in the image data obtained in real time according to the K processing data sets;

According to the region of interest, exposure processing is performed on the image data acquired in real time.
The terminal device according to claim 14, wherein the image signal processor is used for:

After each receiving the K processing data sets, determine the initial region of interest in the image data obtained in real time according to the K processing data sets;

Determine the estimated offset information of the initial region of interest according to the K processing data sets and the N historical processing data sets, where N is a positive integer;

According to the estimated offset information and the initial region of interest, the region of interest in the real-time acquired image data is determined.
The terminal device of claim 15, wherein the estimated offset information comprises a compensation area;

The image signal processor is used for:

combining the compensation region and the initial region of interest to obtain the region of interest;

Exposure processing is performed on the image data acquired in real time according to the first weight corresponding to the compensation area and the second weight corresponding to the initial region of interest.
The terminal device according to claim 15, wherein the image signal processor is used for:

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, the first moving speed of the target object corresponding to the historical period is calculated, wherein the target object and the initial associated with the region of interest;

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second moving speed of the target object corresponding to the target period;

determining the target acceleration of the target object according to the first moving speed and the second moving speed;

According to the target acceleration, the estimated offset information of the initial region of interest is determined.
The terminal device according to claim 17, wherein the image signal processor is used for:

determining the first distance between the camera that collected the image data and the target object in the historical period, and determining the second distance between the camera and the target object in the target period;

According to the historical period corresponding to the N historical processing data sets and the N historical processing data sets, calculate the first initial moving speed of the target object corresponding to the historical period;

obtaining the first moving speed according to the first distance and a preset distance-pixel displacement mapping relationship;

The calculating, according to the target period corresponding to the K processing data sets and the K processing data sets, the second moving speed corresponding to the target object in the target period, including:

According to the target period corresponding to the K processing data sets and the K processing data sets, calculate the second initial moving speed of the target object corresponding to the target period;

The second moving speed is obtained according to the second distance and a preset distance-pixel displacement mapping relationship.
A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the image information processing method according to any one of claims 1 to 9 is implemented .