CN112784084A

CN112784084A - Image processing method and device and electronic equipment

Info

Publication number: CN112784084A
Application number: CN201911084740.7A
Authority: CN
Inventors: 刘宝龙; 陈永健; 孙凯; 李名杨
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-11-08
Filing date: 2019-11-08
Publication date: 2021-05-11
Anticipated expiration: 2039-11-08
Also published as: CN112784084B

Abstract

The application discloses an image processing method, which comprises the following steps: carrying out road component object target detection on an image to be processed to obtain a target road component object in the image to be processed; obtaining traffic use information to which the target road component object belongs; obtaining content information in the target road component object; and determining the traffic use and content information combination category to which the target road component object belongs according to the traffic use information to which the target road component object belongs and the content information in the target road component object. According to the image processing method, after the target road component object in the image to be processed is obtained, the combination category of the traffic purpose and the content information of the target road component object is further determined according to the traffic purpose information of the target road component object and the content information of the target road component object, and therefore the accuracy of classification of the target road component object in the image is improved.

Description

Image processing method and device and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to an image processing method and apparatus, and an electronic device.

Background

In the process of completing tasks such as high-precision map making, auxiliary positioning and the like, a large amount of classification and warehousing work of road components in the image can be involved. The method has the advantages that accurate classification of road components in the images can be realized, and tasks such as high-precision map making and auxiliary positioning can be accurately completed.

The existing method for classifying road components in an image generally comprises the following steps: firstly, detecting a target road component in an image through a target road component detection algorithm model; then, the image features of the road components in the target road component, such as color features, texture features and the like, are extracted, and the target road component is classified according to the image features of the road components, so that the classification result of the target road component is obtained. However, when classifying the target road components in the image according to the image features of the target road components, there is a problem that the classification result is wrong due to insufficient discrimination of the image features between the target road components of partially different classes.

Disclosure of Invention

The application provides an image processing method, an image processing device, an electronic device and a storage medium, which are used for improving the accuracy of classification of a target road component object in an image.

The application provides an image processing method, comprising the following steps:

carrying out road component object target detection on an image to be processed to obtain a target road component object in the image to be processed;

obtaining traffic use information to which the target road component object belongs;

obtaining content information in the target road component object;

and determining the traffic use and content information combination category to which the target road component object belongs according to the traffic use information to which the target road component object belongs and the content information in the target road component object.

Optionally, the performing, on the image to be processed, the target detection of the road component object to obtain the target road component object in the image to be processed includes:

obtaining a characteristic diagram corresponding to the target road component object according to the characteristic diagram corresponding to the image to be processed;

and obtaining the target road component object according to the characteristic diagram corresponding to the target road component object.

Optionally, the obtaining the target road component object according to the feature map corresponding to the target road component object includes: and according to the feature map corresponding to the target road component object, obtaining the target road component object and the position information of the target road component object in the image to be processed.

Optionally, the obtaining traffic usage information to which the target road component object belongs includes:

obtaining traffic use characteristic data of a characteristic diagram corresponding to the target road component object;

and obtaining the traffic use information of the target road component object according to the traffic use characteristic data of the characteristic diagram corresponding to the target road component object.

Optionally, the obtaining traffic use information to which the target road component object belongs according to the traffic use feature data of the feature map corresponding to the target road component object includes:

acquiring preset traffic use characteristic data, wherein the preset traffic use characteristic data is characteristic data corresponding to traffic use information to which the target road component object belongs;

and obtaining the traffic use information of the target road component object according to the traffic use characteristic data of the characteristic diagram corresponding to the target road component object and the preset traffic use characteristic data.

Optionally, obtaining traffic use information to which the target road component object belongs according to the traffic use feature data of the feature map corresponding to the target road component object and the preset traffic use feature data includes:

acquiring preset traffic use characteristic data contained in traffic use characteristic data of a characteristic diagram corresponding to the target road component object;

and obtaining the traffic use information of the target road component object according to the preset traffic use characteristic data contained in the traffic use characteristic data of the characteristic diagram corresponding to the target road component object.

Optionally, the obtaining content information in the target road component object includes:

acquiring content characteristic data of a characteristic diagram corresponding to the target road component object;

and obtaining the content information in the target road component object according to the content characteristic data of the characteristic diagram corresponding to the target road component object.

Optionally, the obtaining content information in the target road component object according to the content feature data of the feature map corresponding to the target road component object includes:

acquiring a characteristic diagram corresponding to content information according to content characteristic data of the characteristic diagram corresponding to the target road component object;

and according to the feature map corresponding to the content information, obtaining the content information and the position information of the content information in the target road component object.

Optionally, the content information in the target road component object at least includes one of numerical information, letter information, character information, and symbol information.

Optionally, the obtaining a feature map corresponding to the target road component object according to the feature map corresponding to the image to be processed includes:

obtaining a feature map corresponding to a candidate target road component object according to the feature map corresponding to the image to be processed;

and obtaining the characteristic diagram corresponding to the target road component object according to the characteristic diagram corresponding to the candidate target road component object.

Optionally, the obtaining a feature map corresponding to the target road component object according to the feature map corresponding to the candidate target road component object includes:

obtaining image characteristic data of a characteristic diagram corresponding to the candidate target road component object;

and obtaining the characteristic diagram corresponding to the target road component object according to the image characteristic data of the characteristic diagram corresponding to the candidate target road component object.

Optionally, the obtaining the feature map corresponding to the target road component object according to the image feature data of the feature map corresponding to the candidate target road component object includes:

acquiring the preset image characteristic data, wherein the preset image characteristic data is image characteristic data required to be included in the image characteristic data of the characteristic diagram corresponding to the target road component object;

and obtaining the characteristic diagram corresponding to the target road component object according to the image characteristic data of the characteristic diagram corresponding to the candidate target road component object and the preset image characteristic data.

Optionally, the obtaining the feature map corresponding to the target road component object according to the image feature data of the feature map corresponding to the candidate target road component object and the preset image feature data includes:

judging whether the image characteristic data of the characteristic diagram corresponding to the candidate target road component object contains the preset image characteristic data or not;

and if so, taking the feature map corresponding to the candidate target road component object as the feature map corresponding to the target road component object.

Optionally, if the image feature data of the feature map corresponding to the candidate target road component object does not include the preset image feature data, the feature map corresponding to the candidate target road component object is filtered.

Optionally, the determining, according to the traffic usage information to which the target road component object belongs and the content information in the target road component object, a category of a combination of traffic usage and content information to which the target road component object belongs includes:

according to the traffic use information of the target road component object, obtaining the traffic use category of the target road component object;

and determining the traffic use and content information combination category to which the target road component object belongs according to the traffic use category to which the target road component object belongs and the content information in the target road component object.

Optionally, the obtaining the traffic use category to which the target road component object belongs according to the traffic use information to which the target road component object belongs includes:

acquiring a corresponding relation of traffic purpose categories, wherein the corresponding relation of the traffic purpose categories is a preset corresponding relation between traffic purpose information and the traffic purpose categories;

and obtaining the traffic purpose category to which the target road component object belongs according to the traffic purpose information to which the target road component object belongs and the corresponding relation of the traffic purpose category.

Optionally, the determining, according to the traffic use category to which the target road component object belongs and the content information in the target road component object, a traffic use and content information combination category to which the target road component object belongs includes:

acquiring character data in the content information;

and determining the traffic use and content information combination category to which the target road component object belongs according to the character data in the content information and the traffic use category to which the target road component object belongs.

Optionally, the determining, according to the character data in the content information and the traffic use category to which the target road component object belongs, a traffic use and content information combination category to which the target road component object belongs includes: and determining the target road component objects with different character data in the content information as the traffic purpose and content information combination categories to which the different target road component objects belong, and acquiring the content information categories of the target road component objects under the traffic purpose categories to which the target road component objects belong.

Optionally, the method further includes: and outputting the traffic use and content information combination category to which the target road component object belongs.

Optionally, the image to be processed is a road scene image to be processed, and the target road component object is a traffic signboard.

Optionally, the method further includes: obtaining an initial image; and carrying out distortion removal processing on the initial image to obtain the image to be processed.

In another aspect of the present application, there is provided an image processing apparatus including:

the target road component object obtaining unit is used for carrying out road component object target detection on an image to be processed to obtain a target road component object in the image to be processed;

a traffic-use information obtaining unit for obtaining traffic-use information to which the target road component object belongs;

a content information obtaining unit for obtaining content information in the target road component object;

and the combination category determining unit is used for determining the combination category of the traffic use and the content information of the target road component object according to the traffic use information of the target road component object and the content information of the target road component object.

In another aspect of the present application, an electronic device is provided, including:

a processor;

a memory for storing a program of an image processing method, the electronic device executing the following steps after being powered on and running the program of the image processing method through the processor:

obtaining content information in the target road component object;

In another aspect of the present application, there is provided a storage medium storing a program of an electronic image processing method, the program being executed by a processor to perform the steps of:

obtaining content information in the target road component object;

In another aspect of the present application, an image processing method is provided, including:

sending an image to be processed to a server;

and acquiring the traffic purpose and content information combination category to which the target road component object in the image to be processed output by the server belongs.

the image to be processed sending unit is used for sending the image to be processed to the server;

and the combined category obtaining unit is used for obtaining the traffic purpose and content information combined category to which the target road component object in the image to be processed output by the server belongs.

a processor;

sending an image to be processed to a server;

In another aspect of the present application, there is provided an image processing system including: a server side and a client side;

the server is used for obtaining the image to be processed sent by the client; carrying out road component object target detection on an image to be processed to obtain a target road component object in the image to be processed; obtaining traffic use information to which the target road component object belongs; obtaining content information in the target road component object; determining the traffic use and content information combination category of the target road component object according to the traffic use information of the target road component object and the content information in the target road component object; outputting the traffic purpose and content information combination category to which the target road component object belongs;

the client is used for sending the image to be processed to the server; and acquiring the traffic use and content information combination category to which the target road component object output by the server belongs.

Compared with the prior art, the method has the following advantages:

according to the image processing method, after the target road component object in the image to be processed is obtained, the traffic purpose and content information combination category of the target road component object is further determined according to the traffic purpose information of the target road component object and the content information of the target road component object. The content information of the target road component is the main difference point between different target road components with higher similarity, so that the traffic purpose information of the target road component object and the content information in the target road component are utilized to classify the target road component in the image to be processed, and the accuracy of classifying the target road component object in the image can be improved.

Drawings

Fig. 1 is a schematic diagram of a first application scenario provided in the present application.

Fig. 2 is a schematic diagram of a second application scenario embodiment provided in the present application.

Fig. 3 is a flowchart of an image processing method provided in the first embodiment of the present application.

Fig. 4 is a flowchart of a method for obtaining a feature map corresponding to a target road component object according to a first embodiment of the present application.

Fig. 5 is a flowchart of a method for obtaining traffic usage information according to a first embodiment of the present application.

Fig. 6 is a flowchart of a method for obtaining content information in a road component object according to a first embodiment of the present application.

Fig. 7 is a flowchart of a method for determining a traffic purpose category to which a target road component object belongs according to the first embodiment of the present application.

Fig. 8 is a schematic diagram of an image processing apparatus provided in a second embodiment of the present application.

Fig. 9 is a schematic diagram of an electronic device provided in an embodiment of the present application.

Fig. 10 is a flowchart of an image processing method provided in a fifth embodiment of the present application.

Fig. 11 is a schematic diagram of an image processing apparatus provided in a sixth embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather construed as limited to the embodiments set forth herein.

In order to show the present application more clearly, an application scenario of the image processing method provided in the embodiment of the present application is introduced first.

Some embodiments provided in the present application may be applied to internal processing of a server, as shown in fig. 1, which is a schematic diagram of a first application scenario embodiment provided in the present application.

After obtaining the image to be processed 102, the server 101 firstly performs road component object target detection on the image to be processed to obtain a first target road component object 102-1 and a second target road component object 102-2 in the image to be processed 102; secondly, traffic use information to which the target road component object 102-1 and the second target road component object 102-2 belong is obtained; thirdly, content information 102-11 in the target road component object is obtained; finally, the traffic use and content information combination category to which the target road component object 102-1 and the second target road component object 102-2 belong is determined based on the traffic use information to which the target road component object 102-1 and the second target road component object 102-2 belong and the content information in the target road component object 102-1. The image to be processed 102 is a road scene image, and the target road component object 102-1 is a traffic signboard.

Some embodiments provided by the present application may also be applied to a scenario in which a server interacts with a client, as shown in fig. 2, which is a schematic diagram of a second application scenario embodiment provided by the present application.

Firstly, establishing connection between a server 201 and a client 202; secondly, the client 202 sends the image to be processed to the server 201; thirdly, after obtaining the image to be processed, the server 201 performs road component object target detection on the image to be processed to obtain a target road component object in the image to be processed; obtaining traffic use information to which a target road component object belongs; obtaining content information in a target road component object; determining the traffic use and content information combination category of the target road component object according to the traffic use information of the target road component object and the content information in the target road component object; finally, the server 201 sends the traffic purpose and content information combination category to which the target road component object belongs to the client 202.

It should be noted that the two application scenarios described above are only two embodiments of the application scenarios of the image processing method provided in the present application, and the two application scenario embodiments are provided to facilitate understanding of the image processing method provided in the present application, and are not used to limit the image processing method provided in the present application. The image processing method provided by the application can also be applied to other scenes, such as internal processing of a client and the like, and is not repeated here.

First embodiment

A first embodiment of the present application provides a method for processing road scene images, which is described below with reference to fig. 3 to 6.

Step S301, carrying out road component object target detection on the image to be processed to obtain a target road component object in the image to be processed.

The image to be processed in the first embodiment of the present application is generally a road scene image including a target road component, and may also be another image including a target road component. Common road components include: a traffic signboard, a road signboard, etc., and the target road component in the first embodiment of the present application is generally referred to as a traffic signboard.

Before the road component object target detection is performed on the image to be processed, the image to be processed needs to be obtained, and specifically, an initial image can be obtained firstly; and then carrying out distortion removal processing on the initial image so as to obtain an image to be processed.

The method comprises the following steps of carrying out road component object target detection on an image to be processed, and obtaining a target road component object in the image to be processed: obtaining a characteristic diagram corresponding to a target road component object according to the characteristic diagram corresponding to the image to be processed; and obtaining the target road component object according to the characteristic diagram corresponding to the target road component object. The obtaining of the target road component object according to the feature map corresponding to the target road component object includes: and according to the characteristic diagram corresponding to the target road component object, obtaining the target road component object and the position information of the target road component object in the image to be processed.

When obtaining the feature map corresponding to the image to be processed, generally: firstly, performing convolution feature extraction on an image to be processed through a CNN (Convolutional Neural Networks) to obtain feature maps of different scales output by a Backbone network Backbone (Neural network model) of the CNN; and then, fusing Feature maps of different scales output by a Backbone network Backbone (neural network) model of the CNN through an FPN (Feature template Structure) algorithm, thereby obtaining a Feature map corresponding to the image to be processed.

Referring to fig. 4, a flowchart of a method for obtaining a feature map corresponding to a target road component object according to a feature map corresponding to the target road component object is shown.

Step S401, according to the feature map corresponding to the image to be processed, obtaining the feature map corresponding to the candidate target road component object.

In the first embodiment of the present application, when obtaining the feature map corresponding to the candidate target road component object according to the feature map corresponding to the image to be processed, generally, an RPN (region candidate network) algorithm is first used to extract a region of the candidate target road component object from the feature map corresponding to the image to be processed obtained in step S301, and then a feature map corresponding to the candidate target road component object is obtained in a region feature aggregation manner.

Step S402, according to the feature map corresponding to the candidate target road component object, obtaining the feature map corresponding to the target road component object.

Obtaining a feature map corresponding to the target road component object according to the feature map corresponding to the candidate target road component object, wherein the feature map comprises: acquiring image characteristic data of a characteristic diagram corresponding to the candidate target road component object; and obtaining the characteristic diagram corresponding to the target road component object according to the image characteristic data of the characteristic diagram corresponding to the candidate target road component object.

Specifically, obtaining the feature map corresponding to the target road component object according to the image feature data of the feature map corresponding to the candidate target road component object includes: acquiring preset image characteristic data, wherein the preset image characteristic data is image characteristic data required to be included in the image characteristic data of the characteristic diagram corresponding to the target road component object; and obtaining the characteristic diagram corresponding to the target road component object according to the image characteristic data and the preset image characteristic data of the characteristic diagram corresponding to the candidate target road component object.

When the feature map corresponding to the target road component object is obtained according to the image feature data and the preset image feature data of the feature map corresponding to the candidate target road component object, it is required to first determine whether the image feature data of the feature map corresponding to the candidate target road component object includes the preset image feature data, and then further obtain the feature map corresponding to the target road component object according to whether the image feature data of the feature map corresponding to the candidate target road component object includes the preset image feature data. Specifically, if the image feature data of the feature map corresponding to the candidate target road component object includes preset image feature data, the feature map corresponding to the candidate target road component object is used as the feature map corresponding to the target road component object; and if the image characteristic data of the characteristic diagram corresponding to the candidate target road component object does not contain preset image characteristic data, filtering the characteristic diagram corresponding to the candidate target road component object.

Step S402 in the first embodiment of the present application may be implemented based on a Region-CNN (Region-CNN; target detection based on CNN) algorithm, where input of the Region-CNN (Region-CNN; target detection based on CNN) algorithm is a feature map corresponding to a candidate target road component object, and output is position information of the target road component object and the target road component object in an image to be processed.

In step S302, traffic use information to which the target road component object belongs is obtained.

The traffic use information to which the target road component object belongs is information for explaining an application function of the target road component object in traffic, and taking the target road component object as a traffic signboard for example, the traffic use information to which the traffic signboard belongs is information for explaining the application function of the target road component object in traffic, such as: the traffic use information of the traffic signboard with the height limit of 2m is used for explaining that the traffic signboard with the height limit of 2m is applied to the traffic signboard with the height limit.

Referring to fig. 5, a flowchart of a method for obtaining traffic usage information in a first embodiment of the present application is shown.

Step S501, obtaining a characteristic diagram corresponding to the target road component object according to the characteristic diagram corresponding to the image to be processed.

In step S501, please refer to steps S401 to S402 for a detailed process of obtaining a feature map corresponding to a target road component object according to a feature map corresponding to an image to be processed.

Step S502, obtaining the traffic use characteristic data of the characteristic diagram corresponding to the target road component object.

After the feature map corresponding to the target road component object is obtained, the traffic-use feature data of the feature map corresponding to the target road component object may be further extracted, so as to obtain the traffic-use feature data of the feature map corresponding to the target road component object.

In step S503, the traffic use feature data of the feature map corresponding to the target road component object is used to obtain the traffic use information to which the target road component object belongs.

The specific process of obtaining the traffic use information to which the target road component object belongs according to the traffic use feature data of the feature map corresponding to the target road component object is as follows: acquiring preset traffic use characteristic data, wherein the preset traffic use characteristic data is characteristic data corresponding to traffic use information to which a target road component object belongs; and obtaining the traffic use information of the target road component object according to the traffic use characteristic data of the characteristic diagram corresponding to the target road component object and the preset traffic use characteristic data. The method for acquiring the traffic use information of the target road component object according to the traffic use characteristic data of the characteristic diagram corresponding to the target road component object and the preset traffic use characteristic data comprises the following steps: acquiring preset traffic use characteristic data contained in traffic use characteristic data of a characteristic diagram corresponding to a target road component object; and obtaining the traffic use information of the target road component object according to preset traffic use characteristic data contained in the traffic use characteristic data of the characteristic diagram corresponding to the target road component object.

In step S303, content information in the target road component object is obtained.

The content information in the target road component object is such as: the "50" in the traffic signboard of the "speed limit 50",

and 2 in the traffic signboard with the height limit of 2 m. The process of obtaining the content information in the target road component object in the first embodiment of the present application is a process of performing Character Recognition on the target road component object, for example, obtaining the content information in the target road component object by an OCR (Optical Character Recognition) technology. The content information in the target road component object is obtained through a character recognition technology, and the specific process is as follows:

referring to fig. 6, a flowchart of a method for obtaining content information in a target road component object according to a first embodiment of the present application is shown.

Step S601, obtaining a characteristic diagram corresponding to the target road component object according to the characteristic diagram corresponding to the image to be processed.

In step S602, content feature data of a feature map corresponding to the target road component object is obtained.

Step S603, obtaining content information in the target road component object according to the content feature data of the feature map corresponding to the target road component object.

Obtaining content information in the target road component object according to content feature data of a feature map corresponding to the target road component object, wherein the content feature data comprises: acquiring a characteristic diagram corresponding to content information according to content characteristic data of the characteristic diagram corresponding to the target road component object; and obtaining the content information and the position information of the content information in the target road component object according to the feature map corresponding to the content information. Wherein the content information in the target road component object at least includes one of numerical information, letter information, character information, and symbol information. Taking a target road component as a speed-limiting traffic signboard as an example, the speed-limiting traffic signboard has multiple types, content information in different types of speed-limiting traffic signboards is different, and content information in some speed-limiting traffic signboards may only contain numerical information, such as: the content information in some speed-limiting traffic signboards comprises not only numerical information but also text information, such as 60km/h and 5km for speed-limiting driving.

Step S304, according to the traffic use information of the target road component object and the content information in the target road component object, determining the traffic use and content information combination category of the target road component object.

In the image processing method provided in the first embodiment of the present application, the traffic purpose category to which the target road component object belongs needs to be obtained according to the traffic purpose information to which the target road component object belongs. In the image processing method according to the first embodiment of the present application, since the traffic use category to which the target road component object belongs is obtained in step S302, after the traffic use information to which the target road component object belongs is generally obtained, in step S303, the traffic use category to which the target road component object belongs can be further determined based on the traffic use information to which the target road component object belongs.

The target road component object is divided according to the traffic purpose category to which the target road component object belongs, for example, a traffic signboard with a speed limit of 50, a traffic signboard with a speed limit of 60 and a traffic signboard with a speed limit of 80 are divided into traffic signboards with speed limits, and a traffic signboard with a height limit of 2m, a traffic signboard with a height limit of 2.5m and a traffic signboard with a height limit of 3m are divided into traffic signboards with height limits. Referring to fig. 7, a flowchart of a method for determining a traffic usage category to which a target road component object belongs according to traffic usage information to which the target road component object belongs is provided in a first embodiment of the present application.

Step S701, obtaining a corresponding relation of traffic purpose categories, wherein the corresponding relation of the traffic purpose categories is a preset corresponding relation between traffic purpose information and the traffic purpose categories.

Step S702, according to the traffic purpose information and traffic purpose category corresponding relation of the target road component object, obtaining the traffic purpose category of the target road component object.

After the traffic purpose category to which the target road component object belongs is obtained, it is necessary to further determine the traffic purpose and content information combination category to which the target road component object belongs according to the traffic purpose category to which the target road component object belongs and the content information in the target road component object. Specifically, determining a traffic use and content information combination category to which the target road component object belongs according to the traffic use category to which the target road component object belongs and the content information in the target road component object includes: acquiring character data in the content information; and determining the traffic use and content information combination category to which the target road component object belongs according to the character data in the content information and the traffic use category to which the target road component object belongs.

When determining the traffic purpose and content information combination category to which the target road component object belongs, it is necessary to determine the target road component object having different character data in the content information as belonging to the traffic purpose and content information combination category to which the different target road component object belongs, and obtain the content information category of the target road component object under the traffic purpose category to which the target road component object belongs. Taking the content information as numerical information as an example, the numerical value "60" is composed of a numerical character "6" and a numerical character "0". In the first embodiment of the present application, the process when obtaining the content information is to obtain character data in the content information, and determine numerical information or text information corresponding to the content information according to the character data in the content information.

For example, although the traffic signboard of the "speed limit 50", the traffic signboard of the "speed limit 60", and the traffic signboard of the "speed limit 80" are all the "speed limit type" traffic signboards, the image processing method provided in the first embodiment of the present application divides the traffic signboard of the "speed limit 50", the traffic signboard of the "speed limit 60", and the "speed limit 80" into different "speed limit type" traffic signboards. That is, detailed numerical information or text information in the "speed limit type" traffic signboard is determined according to character data, the traffic signboard with the speed limit 50 is divided into the "speed limit type" traffic signboard, the traffic signboard with the speed limit value of "50", the traffic signboard with the speed limit 60 "is divided into the" speed limit type "traffic signboard, the traffic signboard with the speed limit value of" 60 ", the traffic signboard with the speed limit 80" is divided into the "speed limit type" traffic signboard, and the traffic signboard with the speed limit value of "80".

According to the image processing method, after the target road component object in the image to be processed is obtained, the traffic purpose and content information combination category of the target road component object are further determined according to the traffic purpose information of the target road component object and the content information of the target road component object. The content information of the target road component is the main difference point between different target road components with higher similarity, so that the traffic purpose information of the target road component object and the content information in the target road component are utilized to classify the target road component in the image to be processed, and the accuracy of classifying the target road component object in the image can be improved.

The image processing method provided in the first embodiment of the present application further includes: and outputting the traffic use and content information combination category to which the target road component object belongs. Since the image processing method provided in the first embodiment of the present application can obtain the position information of the target road component in the image to be processed while determining the target road component in the image to be processed, the position information of the target road component in the image to be processed can be output while outputting the traffic use and content information combination category to which the target road component object belongs.

Second embodiment

Corresponding to the image processing method provided in the first embodiment of the present application, a second embodiment of the present application also provides an image processing apparatus. Since the apparatus embodiment is substantially similar to the method first embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.

The image processing apparatus includes:

a target road component object obtaining unit 801, configured to perform road component object target detection on an image to be processed, so as to obtain a target road component object in the image to be processed;

a traffic-use information obtaining unit 802 for obtaining traffic-use information to which the target road component object belongs;

a content information obtaining unit 803 for obtaining content information in the target road component object;

a combination category determining unit 804, configured to determine a traffic use and content information combination category to which the target road component object belongs according to the traffic use information to which the target road component object belongs and the content information in the target road component object.

Optionally, the target road component object obtaining unit 801 is specifically configured to obtain a feature map corresponding to the target road component object according to the feature map corresponding to the image to be processed; and obtaining the target road component object according to the characteristic diagram corresponding to the target road component object.

Optionally, the traffic purpose information obtaining unit is specifically configured to obtain a feature map corresponding to the target road component object according to the feature map corresponding to the image to be processed; obtaining traffic use characteristic data of a characteristic diagram corresponding to the target road component object; and obtaining the traffic use information of the target road component object according to the traffic use characteristic data of the characteristic diagram corresponding to the target road component object.

Optionally, the content information obtaining unit 803 is specifically configured to obtain a feature map corresponding to the target road component object according to the feature map corresponding to the image to be processed; acquiring content characteristic data of a characteristic diagram corresponding to the target road component object; and obtaining the content information in the target road component object according to the content characteristic data of the characteristic diagram corresponding to the target road component object.

Optionally, the combination category determining unit 804 is specifically configured to obtain the traffic use category to which the target road component object belongs according to the traffic use information to which the target road component object belongs; and determining the traffic use and content information combination category to which the target road component object belongs according to the traffic use category to which the target road component object belongs and the content information in the target road component object.

acquiring character data in the content information;

Optionally, the image processing apparatus further includes: and the combined category output unit is used for outputting the traffic purpose and content information combined category to which the target road component object belongs.

Optionally, the image processing apparatus further includes:

an initial image obtaining unit for obtaining an initial image;

and the to-be-processed image obtaining unit is used for carrying out distortion removal processing on the initial image to obtain the to-be-processed image.

Third embodiment

A third embodiment of the present application provides an electronic device corresponding to the image processing method provided in the first embodiment of the present application.

As shown in fig. 9, fig. 9 is a schematic view of an electronic device according to an embodiment of the present application. The electronic device includes:

a processor 901; and

a memory 902 for storing a program of an image processing method, the apparatus performing the following steps after being powered on and running the program of the image processing method by the processor:

obtaining content information in the target road component object;

It should be noted that, for the detailed description of the electronic device provided in the third embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.

Fourth embodiment

In accordance with an image processing method provided in the first embodiment of the present application, a fourth embodiment of the present application provides a storage medium storing a program of the image processing method, the program being executed by a processor to perform the steps of:

obtaining content information in the target road component object;

It should be noted that, for the detailed description of the storage medium provided in the fourth embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.

Fifth embodiment

In the first embodiment described above, an image processing method is provided, and correspondingly, a fifth embodiment of the present application provides another image processing method. Since this embodiment of the image processing method is basically similar to the first embodiment of the method, it is relatively simple to describe, and for the relevant points, refer to the partial description of the embodiment of the method. The method embodiments described below are merely illustrative.

Referring to fig. 10, fig. 10 is a flowchart illustrating an image processing method according to a fifth embodiment of the present disclosure.

Step S1001, sending the image to be processed to the server. The image to be processed is a road scene image to be processed.

Step S1002, obtaining the traffic purpose and content information combination category to which the target road component object in the image to be processed output by the server belongs. Wherein the target road component object is a traffic signboard.

Sixth embodiment

In the fifth embodiment described above, an image processing method is provided, and correspondingly, a sixth embodiment of the present application provides an image processing apparatus. Since the apparatus embodiment is substantially similar to the method fifth embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.

Referring to fig. 11, fig. 11 is a schematic diagram of an image processing apparatus according to a sixth embodiment of the present application.

The image processing apparatus includes:

a to-be-processed image sending unit 1101 configured to send a to-be-processed image to a server;

a combination category obtaining unit 1102, configured to obtain a traffic purpose and content information combination category to which a target road component object in the image to be processed output by the server belongs.

Seventh embodiment

A seventh embodiment of the present application provides an electronic apparatus, corresponding to the image processing method provided in the fifth embodiment of the present application.

As shown in fig. 9, the electronic apparatus includes:

a processor 901; and

a memory 902 for storing a program of an image processing method, which executes the following steps after the apparatus is powered on and the program of the image processing method is executed by the processor:

sending an image to be processed to a server;

It should be noted that, for the detailed description of the electronic device provided in the seventh embodiment of the present application, reference may be made to the related description of the fifth embodiment of the present application, and details are not described here again.

Eighth embodiment

In accordance with an eighth embodiment of the present invention, in correspondence with the image processing method provided in the fifth embodiment of the present invention, there is provided a storage medium storing a program of the image processing method, the program being executed by a processor to perform the steps of:

sending an image to be processed to a server;

It should be noted that, for the detailed description of the storage medium provided in the eighth embodiment of the present application, reference may be made to the related description of the fifth embodiment of the present application, and details are not described here again.

Ninth embodiment

In the first embodiment described above, an image processing method is provided, and correspondingly, a ninth embodiment of the present application provides another image processing system. Since the image processing system in the ninth embodiment is basically similar to the first embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment. The device embodiments described below are merely illustrative.

Referring to fig. 2 again, the image processing system includes: server 201 and client 202

The server 201 is configured to obtain an image to be processed sent by the client 202; carrying out road component object target detection on an image to be processed to obtain a target road component object in the image to be processed; obtaining traffic use information to which the target road component object belongs; obtaining content information in the target road component object; determining the traffic use and content information combination category of the target road component object according to the traffic use information of the target road component object and the content information in the target road component object; outputting the traffic purpose and content information combination category to which the target road component object belongs;

the client 202 is configured to send an image to be processed to the server 201; and acquiring the traffic use and content information combination category to which the target road component object output by the server 201 belongs.

Tenth embodiment

Corresponding to the image processing method provided in the first embodiment of the present application, a tenth embodiment of the present application also provides another image processing method. Since this method embodiment is substantially similar to the first method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The method embodiments described below are merely illustrative.

The image processing method provided in the tenth embodiment of the present application may be used for processing a road scene image obtained by a vehicle-mounted image capturing device during a driving process of a vehicle, for example, an automatic driving vehicle needs to obtain the road scene image through the vehicle-mounted image capturing device during the driving process, and an automatic navigation is implemented by combining with a vehicle-mounted navigation system. In the tenth embodiment of the application, the image to be processed is a road scene image obtained by the vehicle-mounted image acquisition device in the driving process of the automatic driving automobile.

After obtaining the road scene image, the image processing system in the vehicle may perform road component object target detection on the image to be processed first, so as to obtain a target road component object in the image to be processed. Then, the traffic purpose category to which the target road component object belongs is obtained, and the driving mode of the autonomous vehicle is adjusted according to the obtained traffic purpose category to which the target road component object belongs, if the traffic purpose category to which the target road component object belongs is obtained as the "speed limit category", the driving mode of the autonomous vehicle is adjusted to the speed limit mode, and after the driving mode of the autonomous vehicle is adjusted to the speed limit mode, the autonomous vehicle starts to adjust the driving speed.

In order to further determine the driving speed to which the autonomous vehicle needs to be adjusted, it is also necessary to obtain the content information in the target road component object and determine the driving speed to which the autonomous vehicle needs to be adjusted based on the content information in the target road component object. Specifically, the process of obtaining the content information in the target road component object is as follows: according to the traffic purpose category to which the target road component object belongs, a content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs is obtained, wherein the content information obtaining model is a model for identifying content information in the target road component according to the traffic purpose category to which the target road component object belongs; and obtaining the content information in the target road component object according to the content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs.

The method for obtaining the content information in the target road component object according to the content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs comprises the following steps: and obtaining the content information corresponding to the traffic purpose category to which the target road component object belongs in the target road component object according to the content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs. The content information in the target road component object includes at least one of numerical information, letter information, character information, and symbol information.

Eleventh embodiment

Corresponding to one of the image processing methods provided in the first embodiment of the present application, an eleventh embodiment of the present application also provides another image processing method. Since this method embodiment is substantially similar to the first method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The method embodiments described below are merely illustrative.

The image processing method provided in the eleventh embodiment of the present application may be used for processing a road scene image obtained by a vehicle-mounted image capturing device during a driving process of a vehicle, for example, an automatic driving vehicle needs to obtain the road scene image through the vehicle-mounted image capturing device during the driving process, and an automatic navigation is implemented by combining with a vehicle-mounted navigation system. Since the vehicle needs to recognize the road scene image quickly after acquiring the road scene image during driving, the automatic driving automobile can acquire what kind of operation the automatic driving automobile needs to perform in advance and start to perform the operation. However, in order to be able to further determine how the autonomous vehicle should perform the operation, it is necessary to further obtain the content information in the target road component object and determine how the autonomous vehicle should perform the operation based on the content information in the target road component object. Since the running speed of the vehicle during running is often fast, the image processing method provided in the eleventh embodiment of the present application needs to be able to quickly obtain the content information in the target road component object. Therefore, the image processing method provided in the eleventh embodiment of the present application obtains the content information in the target road component object in the following manner:

firstly, a traffic purpose category to which a target road component object of an image to be processed belongs is obtained through a primary neural network model, and the primary neural network model is used for carrying out road component object target detection on the image to be processed, obtaining the target road component object in the image to be processed and obtaining a model of the traffic purpose category to which the target road component object belongs. Obtaining only the model for the traffic purpose category to which the target road component object belongs can ensure that the autonomous vehicle can more quickly obtain what operation the autonomous vehicle needs to perform.

Then, a secondary neural network model for identifying the content information in the target road component is obtained in the primary neural network model according to the traffic use category to which the target road component object belongs.

And finally, acquiring content information in the target road component object through a secondary neural network model.

Although the present invention has been described with reference to the preferred embodiments, it should be understood that the scope of the present invention is not limited to the embodiments described above, and that various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the present invention.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage media, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transmyedia), such as modulated data signals and carrier waves.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims

1. An image processing method, comprising:

obtaining content information in the target road component object;

2. The image processing method according to claim 1, wherein the road component object target detection is performed on the image to be processed to obtain a target road component object in the image to be processed, and the method comprises:

3. The image processing method according to claim 2, wherein the obtaining the target road component object from the feature map corresponding to the target road component object includes: and according to the feature map corresponding to the target road component object, obtaining the target road component object and the position information of the target road component object in the image to be processed.

4. The image processing method according to claim 1, wherein the obtaining traffic use information to which the target road component object belongs includes:

5. The image processing method according to claim 4, wherein the obtaining of the traffic use information to which the target road component object belongs based on the traffic use feature data of the feature map corresponding to the target road component object includes:

6. The image processing method according to claim 5, wherein obtaining traffic use information to which the target road component object belongs based on the traffic use feature data of the feature map corresponding to the target road component object and the preset traffic use feature data comprises:

7. The image processing method according to claim 1, wherein the obtaining content information in the target road component object comprises:

8. The image processing method according to claim 7, wherein the obtaining content information in the target road component object from content feature data of a feature map corresponding to the target road component object comprises:

9. The image processing method according to claim 7, wherein the content information in the target road component object includes at least one of numerical information, letter information, character information, and symbol information.

10. The image processing method according to any one of claims 2, 4 and 7, wherein the obtaining the feature map corresponding to the target road component object according to the feature map corresponding to the image to be processed comprises:

11. The image processing method according to claim 10, wherein the obtaining a feature map corresponding to the target road component object from a feature map corresponding to the candidate target road component object comprises:

12. The image processing method according to claim 11, wherein the obtaining of the feature map corresponding to the target road component object from the image feature data of the feature map corresponding to the candidate target road component object includes:

13. The image processing method according to claim 12, wherein the obtaining the feature map corresponding to the target road component object according to the image feature data of the feature map corresponding to the candidate target road component object and the preset image feature data comprises:

14. The image processing method according to claim 13, wherein if the image feature data of the feature map corresponding to the candidate target road component object does not include the preset image feature data, the feature map corresponding to the candidate target road component object is filtered.

15. The image processing method according to claim 1, wherein the determining a traffic use and content information combination category to which the target road component object belongs, from the traffic use information to which the target road component object belongs and the content information in the target road component object, comprises:

16. The image processing method according to claim 15, wherein the obtaining of the traffic use category to which the target road component object belongs, based on the traffic use information to which the target road component object belongs, comprises:

17. The image processing method according to claim 15, wherein the determining a traffic use and content information combination category to which the target road component object belongs, from the traffic use category to which the target road component object belongs and the content information in the target road component object, comprises:

acquiring character data in the content information;

18. The image processing method according to claim 17, wherein said determining a traffic use and content information combination category to which the target road component object belongs, based on the character data in the content information and the traffic use category to which the target road component object belongs, comprises: and determining the target road component objects with different character data in the content information as the traffic purpose and content information combination categories to which the different target road component objects belong, and acquiring the content information categories of the target road component objects under the traffic purpose categories to which the target road component objects belong.

19. The image processing method according to claim 1, further comprising: and outputting the traffic use and content information combination category to which the target road component object belongs.

20. The image processing method according to claim 1, wherein the image to be processed is a road scene image to be processed, and the target road component object is a traffic signboard.

21. The image processing method according to claim 1, further comprising:

obtaining an initial image;

and carrying out distortion removal processing on the initial image to obtain the image to be processed.

22. An image processing apparatus characterized by comprising:

23. An electronic device, comprising:

a processor;

obtaining content information in the target road component object;

24. A storage medium storing a program of an electronic image processing method, the program being executed by a processor to perform the steps of:

obtaining content information in the target road component object;

25. An image processing method, comprising:

sending an image to be processed to a server;

26. The image processing method according to claim 25, wherein the image to be processed is a road scene image to be processed, and the target road component object is a traffic signboard.

27. An image processing apparatus characterized by comprising:

28. An electronic device, comprising:

a processor;

sending an image to be processed to a server;

29. A storage medium storing a program of an electronic image processing method, the program being executed by a processor to perform the steps of:

sending an image to be processed to a server;

30. An image processing system, comprising: a server side and a client side;

31. An image processing method, comprising:

obtaining a traffic purpose category to which the target road component object belongs;

according to the traffic purpose category to which the target road component object belongs, obtaining a content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs, wherein the content information obtaining model is a model for identifying content information in the target road component according to the traffic purpose category to which the target road component object belongs;

and obtaining the content information in the target road component object according to the content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs.

32. The image processing method according to claim 31, wherein the obtaining of the content information in the target road component object based on the content information obtaining model corresponding to the traffic use category to which the target road component object belongs includes:

and obtaining the content information corresponding to the traffic purpose category to which the target road component object belongs in the target road component object according to the content information obtaining model corresponding to the traffic purpose category to which the target road component object belongs.

33. The image processing method of claim 31, wherein the content information in the target road component object includes at least one of numerical information, alphabetical information, textual information, and symbolic information.

34. The image processing method according to claim 31, further comprising: and determining the traffic use and content information combination category to which the target road component object belongs according to the traffic use information to which the target road component object belongs and the content information in the target road component object.

35. An image processing method, comprising:

the method comprises the steps that a traffic purpose category to which a target road component object of an image to be processed belongs is obtained through a primary neural network model, wherein the primary neural network model is used for carrying out road component object target detection on the image to be processed, obtaining the target road component object in the image to be processed and obtaining a model of the traffic purpose category to which the target road component object belongs;

according to the traffic purpose category to which the target road component object belongs, a secondary neural network model used for identifying content information in the target road component is obtained in the primary neural network model;

and obtaining the content information in the target road component object through the secondary neural network model.