CN113516145A

CN113516145A - Image processing and vehicle information providing method, apparatus and storage medium

Info

Publication number: CN113516145A
Application number: CN202011399516.XA
Authority: CN
Inventors: 陈均炫; 谢贤海; 邓兵
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-12-01
Filing date: 2020-12-01
Publication date: 2021-10-19
Anticipated expiration: 2040-12-01
Also published as: CN113516145B

Abstract

The embodiment of the application provides an image processing and vehicle information providing method, equipment and a storage medium. In the embodiment of the application, for two images including an object, the similarity between the objects included in the two images can be determined by combining the visual features of the two images and the difference area between the two images, and when the similarity meets the set condition, the information of one object can be provided according to the information of the other object. The method and the device have the advantages that the visual characteristics and the difference areas of the two images are combined simultaneously, the difference and the same between the two objects can be displayed more visually and accurately, and further, when the two objects are identified to meet the similarity condition, the information of one object can be accurately provided based on the information of the other object, so that the accuracy of information acquisition is remarkably improved.

Description

Image processing and vehicle information providing method, apparatus and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method, a device, and a storage medium for image processing and vehicle information providing.

Background

With the development of video monitoring technology, the monitoring of vehicle traffic conditions in cities is more and more extensive. In vehicle traffic monitoring, vehicles can be monitored and tracked through monitoring videos. For example, for an illegally-driven vehicle, license plate information, a driving track and the like of the vehicle can be tracked through a monitoring video. In practical applications, due to factors such as a shooting angle and shading, the vehicle license plate information cannot be accurately tracked from the image a 1. For such a problem, it is possible to trace back other images that have been captured, and find an image a2 that is most similar to the image a1 from the other images based on the image a1 and the feature vectors of the other images, and take the vehicle information in the image a2 as the vehicle information in the image a 1. However, the error rate of the vehicle information searched by the method is high, and the accuracy rate of information acquisition is low.

Disclosure of Invention

Aspects of the present disclosure provide an image processing and vehicle information providing method, apparatus, and storage medium to improve accuracy of information acquisition significantly.

An embodiment of the present application provides an image processing method, including: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first object and the second object according to the visual features and the difference region; and when the similarity of the first object and the second object meets the set condition, outputting the information of the first object according to the information of the second object.

The embodiment of the present application further provides a vehicle information providing method, including: receiving a first image uploaded by a terminal device, wherein the first image comprises a first vehicle of which license plate information is to be determined; acquiring a second image, wherein the second image comprises a second vehicle and license plate information thereof; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first vehicle and the second vehicle according to the visual features and the difference area; and when the similarity of the first vehicle and the second vehicle meets the set condition, returning the license plate information of the second vehicle to the terminal equipment as the license plate information of the first vehicle.

An embodiment of the present application further provides an image processing method, including: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; splicing the first image and the second image into a target image, wherein the target image comprises visual features in the first image and the second image; marking a difference region between the first image and the second image in the target image; and outputting information of the second object as information of the first object and outputting the target image in a case where the visual feature in the difference region satisfies the setting condition.

An embodiment of the present application further provides an image processing apparatus, including: a memory and a processor; a memory for storing a computer program; a processor coupled with the memory for executing the computer program for: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first object and the second object according to the visual features and the difference region; and when the similarity of the first object and the second object meets the set condition, outputting the information of the first object according to the information of the second object.

An embodiment of the present application further provides a vehicle information providing apparatus, including: a memory and a processor; a memory for storing a computer program; a processor coupled with the memory for executing the computer program for: receiving a first image uploaded by a terminal device, wherein the first image comprises a first vehicle of which license plate information is to be determined; acquiring a second image, wherein the second image comprises a second vehicle and license plate information thereof; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first vehicle and the second vehicle according to the visual features and the difference area; and when the similarity of the first vehicle and the second vehicle meets the set condition, returning the license plate information of the second vehicle to the terminal equipment as the license plate information of the first vehicle.

An embodiment of the present application further provides an image processing apparatus, including: a memory and a processor; a memory for storing a computer program; a processor coupled with the memory for executing the computer program for: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; splicing the first image and the second image into a target image, wherein the target image comprises visual features in the first image and the second image; marking a difference region between the first image and the second image in the target image; and outputting information of the second object as information of the first object and outputting the target image in a case where the visual feature in the difference region satisfies the setting condition.

Embodiments of the present application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the image processing method and the vehicle information providing method provided by the embodiments of the present application.

In the embodiment of the application, for two images including an object, the similarity between the objects included in the two images can be determined by combining the visual features of the two images and the difference area between the two images, and when the similarity meets the set condition, the information of one object can be provided according to the information of the other object. The method and the device have the advantages that the visual characteristics and the difference areas of the two images are combined simultaneously, the difference and the same between the two objects can be displayed more visually and accurately, and further, when the two objects are identified to meet the similarity condition, the information of one object can be accurately provided based on the information of the other object, so that the accuracy of information acquisition is remarkably improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1a is a schematic diagram of an image processing system according to an exemplary embodiment of the present disclosure;

FIG. 1b is a thermodynamic diagram illustrating the formation of a differential mark between a first image and a second image according to an embodiment of the present disclosure;

fig. 2a is a schematic structural diagram of a high-speed charging system according to an exemplary embodiment of the present application;

fig. 2b is a schematic structural diagram of an urban traffic monitoring system according to an exemplary embodiment of the present application;

fig. 3 is a schematic flowchart of an image processing method according to an exemplary embodiment of the present application;

FIG. 4 is a schematic flow chart diagram illustrating a vehicle information providing method according to an exemplary embodiment of the present application;

FIG. 5a is a block diagram of another exemplary image processing method according to an exemplary embodiment of the present disclosure;

FIG. 5b is a schematic flow chart of another image processing method provided in an exemplary embodiment of the present application;

fig. 6a is a schematic structural diagram of an image processing apparatus according to an exemplary embodiment of the present application;

fig. 6b is a schematic structural diagram of a vehicle information providing device according to an exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the embodiment of the application, for two images including an object, the similarity between the objects included in the two images can be determined by combining the visual features of the two images and the difference region between the two images, and when the similarity meets a set condition, information of another object can be provided according to the information of one object. The method and the device have the advantages that the visual characteristics and the difference areas of the two images are combined simultaneously, the difference and the same between the two objects can be displayed more visually and accurately, and further, when the two objects are identified to meet the similarity condition, the information of one object can be accurately provided based on the information of the other object, so that the accuracy of information acquisition is remarkably improved.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1a is a schematic structural diagram of an image processing system according to an exemplary embodiment of the present application, and as shown in fig. 1a, the image processing system 100 includes: a terminal apparatus 101 and an image processing apparatus 102.

The image processing system 100 provided by the embodiment of the application can be applied to any scene with image processing requirements, for example, a scene with image searching, a scene with highway toll audit or a scene with urban vehicle monitoring. In this embodiment, the terminal apparatus 101 may acquire an image including an object. The objects in the image may be any item of a vehicle, a person, furniture, a cup, etc. In some cases, the terminal device may need to acquire information of an object from an image, and the information of the object may be different according to the object. For example, if the object is a vehicle, the information may be license plate information; if the object is a person, the information may be information of five sense organs of the person; the object is a cup, and the information may be pattern information of the cup, etc. In practical applications, due to a shooting angle, a shooting technology, or occlusion, information of an object in an image is often blurred or invisible, and the terminal device 101 needs to perform some subsequent operations by using the information of the object in the image, for example, determining an illegal vehicle by using license plate information of a vehicle in the image, or determining a criminal suspect by using facial information of a person in the image. Based on this, the terminal device 101 may send an image with blurred or invisible object information to the image processing device 102, the image processing device 102 searches for another image similar to the image based on the image with blurred or invisible object information, and acquires information of the object required by the terminal device 101 according to information of the object in the other image. For convenience of distinction and description, an image in which object information is blurred or invisible is made as a first image, an object included in the first image is referred to as a first object, and information of the first object is to be determined.

After the image processing device 102 receives the first image, at least one image similar to or the same as the first image may be acquired, and the at least one image is recorded as a second image, the second image includes a second object, and information of the second object is known. In the present embodiment, the embodiment in which the image processing apparatus 102 acquires the second image is not limited. In an alternative embodiment, the image processing system 100 further comprises: the image database 103 is used for storing images and related information thereof, wherein when the images are stored, the images can be reduced into a feature vector with a fixed length and stored. Based on this, the image processing apparatus 102 may calculate a feature vector of the first image; at least one image having the same or similar feature vector as the first image is selected as the second image from the known images stored in the image database 103 based on the feature vector of the first image. In another alternative embodiment, the image processing apparatus 102 may randomly select at least one image from the known images stored in the image database 103 as the second image. In yet another alternative embodiment, the image processing apparatus 102 may take the known images stored in the image database 103 as the second images, respectively.

In the present embodiment, after the image processing apparatus 102 acquires the first image and the second image, the visual features in the first image and the second image may be extracted, and the difference region between the first image and the second image may be detected. The visual features extracted from the first image or the second image may be data describing or describing features of an object included in the first image or the second image, may be local images including the object in the first image or the second image, and may also be corner points, key points, feature points, or the like in the images. For example, if the object included in the first image or the second image is a vehicle, the visual feature extracted may be a headlight of the vehicle, a license plate of the vehicle, or a wiper of the vehicle. The difference region between the first image and the second image refers to a region where there is a significant difference in the objects contained in the first image and the second image.

In this embodiment, the sequence of extracting the visual features in the first image and the second image and detecting the difference region between the first image and the second image is not limited, for example, the visual features in the first image and the second image may be extracted first, and then the difference region between the first image and the second image may be detected; for another example, the difference region between the first image and the second image may be detected, and then the visual features in the first image and the second image may be extracted. Alternatively, the two operations of "extracting visual features in the first image and the second image" and "detecting a difference region between the first image and the second image" may be performed in parallel.

Whichever of the above embodiments, the image processing apparatus 102 may determine the similarity between the first object and the second object based on the extracted visual feature and the detected difference region; when the similarity between the first object and the second object satisfies the setting condition, the information of the first object is determined according to the information of the second object and the information of the first object is output to the terminal device 101. The set condition may be that the similarity exceeds a set threshold, for example, the similarity exceeds 80%, 90%, or 95%, which is not limited.

In the present embodiment, the embodiment in which the image processing apparatus 102 outputs the information of the first object based on the information of the second object is not limited. In an alternative embodiment, the information of the second object may be directly used as the information of the first object, and the information of the first object may be output. For example, the first image is an image including a vehicle, the first object is the vehicle, the information of the first object is the license plate number of the vehicle, but the license plate number cannot be clearly displayed in the first image; the second image is an image including a vehicle, the second object is a vehicle, and the license plate number of the vehicle is clear, so that when the similarity between the second image and the first image exceeds 95%, the vehicle in the second image and the vehicle in the first image can be considered to be the same vehicle, and the license plate number of the vehicle in the second image can be directly used as the license plate number of the vehicle in the first image and output. In another optional embodiment, object information associated with the second object is searched for as information of the first object according to the information of the second object, and the information of the first object is output. For example, the first image and the second image are both images including a vehicle, the information of the first object refers to driver information of the vehicle included in the first image, the information of the second object is a license plate number of the vehicle included in the second image, and when the similarity between the first image and the second image exceeds 99%, the vehicle in the second image and the vehicle in the first image can be considered to be the same vehicle, so that the license plate number of the vehicle included in the second image is acquired, the information of the driver corresponding to the license plate number is searched based on the license plate number, and the driver information is used as the driver information of the vehicle included in the first image and is output.

In an optional embodiment, to improve the accuracy of determining the similarity between the first object and the second object, before extracting the visual features in the first image and the second image, the image processing device 102 may further align the first image and the second image according to an alignment relationship between the first object and the second object, where the alignment refers to a position of the same keypoint in the first image being the same as or corresponding to a position of the same keypoint in the second image, and the keypoint is a feature point in the first image or the second image that can represent the object included in the keypoint, for example, may be a feature point on the object.

In this embodiment, an embodiment of aligning the first image and the second image according to the alignment relationship of the first object and the second object is not limited. In an optional embodiment, the object categories included in the first image and the second image are known, in this case, a designated key point may be preset, where the designated key point is an object key point in an object category to which the first object and the second object belong, and is a feature point capable of representing commonality between objects in the object category; accordingly, the alignment relationship refers to positional correspondence of the specified key points. The number of the assigned key points of the belonging object category may be 1, or may be multiple, for example, 2, 5, or 10. For example, assuming that the object categories to which the first object and the second object belong are vehicle categories, the designated key points are key points characterizing the features of the vehicle, which are simply referred to as vehicle key points, and fig. 2a and 2b show schematic diagrams of the vehicle key points in the first image and the second image, in which 10 designated key points are taken as an example for illustration, but not limited thereto. In this embodiment, the image template is marked with the positions of the designated key points under the object categories to which the first object and the second object belong, the positions of the designated key points in the image template can be flexibly set according to requirements, and the number of the designated key points in the image template may be 1, or may be multiple, for example, 2, 5, or 10.

Based on the above, the image processing apparatus 102 may align both the first image and the second image with the image template according to the positions of the specified keypoints in the first image and the second image and the positions of the specified keypoints in the image template to align the first image and the second image. Alternatively, a Thin Plate Spline (TPS) transformation may be performed on the first image and the second image, and the TPS transformation is used to deform the specified keypoints in the first image and the second image to the positions of the specified keypoints in the image template, and simultaneously provide the deformation in the whole space, so as to align the first image and the second image, as shown in fig. 2a and 2b, which shows the deformed images after the first image and the second image are aligned to the image template.

In this embodiment, after aligning the first image and the second image, both the first image and the second image can be spliced into the target image as two image regions. The splicing mode can be, but is not limited to, up-down splicing, left-right splicing, front-back splicing and the like. In this embodiment, after obtaining the target image, feature extraction may be performed on two image regions in the target image to obtain visual features of the first image and the second image; and detecting the difference of two image areas in the target image to obtain the difference area between the first image and the second image. The target image can be input into the neural network model for visual feature extraction and difference region detection. The neural network model may be, but is not limited to: convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Neural Networks (DNNs), residual Networks, and the like. Taking the CNN model as an example, the CNN model is divided into two branches, one is a Feature branch, and the Feature branch is used for extracting visual features of a target image; the other is an Attention (Attention) branch, the Attention branch obtains a difference region needing important Attention by rapidly scanning a global target image, the difference region is a focus of Attention, and subsequently more Attention resources are put into the difference region to obtain visual features of more difference regions.

In this embodiment, the embodiment of determining the similarity between the first object and the second object based on the visual features and the difference region is not limited, and will be exemplified below.

In an alternative embodiment, the extracted visual features and the detected difference region may be input to a neural network model for similarity detection, so as to obtain a similarity between the first object and the second object. Optionally, the visual features and the difference region may be input into a Fully Connected layer (FC) in the CNN model, each neuron in the Fully Connected layer is Fully Connected to all neurons in a previous layer, and the Fully Connected layer may integrate feature information of the difference region, so as to perform fine-grained feature comparison on the difference region to obtain a similarity between the first object and the second object.

In another alternative embodiment, the similarity of the visual features in the first image and the second image may be calculated; calculating the difference degree between the first object and the second object according to the number and/or the positions of the difference areas; and correcting the similarity of the visual characteristics according to the difference degree to obtain the similarity between the first object and the second object.

In yet another alternative embodiment, the visual feature located in the difference region is selected from the visual features, and the similarity between the first object and the second object is calculated based on the visual feature located in the difference region.

In some optional embodiments of the present application, after performing difference detection on two image regions in the target image to obtain a difference region between the first image and the second image, the image processing device 102 may mark the difference region in the target image and output an image after marking the difference region, so as to realize visualization of the difference region, and may more intuitively and accurately represent similarities and differences between two objects.

In this embodiment, the embodiment of marking the difference region in the target image is not limited. According to the different marking modes, the output images with different marking areas are different. For example, the difference region may be directly marked with marking information, and the marking information may be lines having different visualization properties, wherein the visualization properties of the lines include at least one of color, line type, and line width. For another example, the difference region may be displayed by a thermodynamic diagram, which may display the difference region of the first image and the second image in a highlighted form, wherein darker colors indicate greater differences, and lighter colors indicate smaller differences. The color depth cannot be reflected in the thermodynamic diagram shown in fig. 1b, and for the convenience of distinguishing, areas with large differences in the first image and the second image are circled by lines, so that the effect similar to or identical to the color distinguishing is achieved.

In the present embodiment, the embodiment in which the image processing apparatus 102 outputs the image after marking the disparity region is not limited. In an alternative embodiment, the image processing apparatus 102 may output the image marked with the difference area to a manager of the image processing apparatus 102, so that the manager performs manual verification on the similarity between the first image and the second image based on the marking information; correcting the similarity obtained by the image processing device 102 by using the similarity obtained by manual verification, wherein the correction mode can be that the similarity obtained by manual verification and the similarity obtained by the image processing device 102 are subjected to weighted summation to obtain the final similarity; or, the similarity obtained through manual verification is directly used as the similarity between the final first image and the second image, which is not limited. In another optional embodiment, the image processing device 102 may output the image marked with the difference area to the terminal device 101, and the image is output by the terminal device 101 to a worker at the side of the terminal device 101, so that the worker performs manual verification on the similarity between the first image and the second image based on the marking information; and correcting the similarity obtained by the terminal equipment 101 by using the manually verified similarity. The similarity of the first image and the second image is corrected in a manual verification mode, so that the accuracy of similarity judgment can be improved, and the effect of outputting the information of the first object according to the information of the second object in the follow-up process is improved.

In the embodiment of the application, the physical areas of the first image and the second image are aligned by detecting the designated key points in the first image and the second image, the first image and the second image are spliced into the target image as two image areas after alignment, the difference area between the first image and the second image is obtained by performing visual feature extraction and difference area detection on the target image, so that fine-grained feature comparison is performed in the difference area to obtain the similarity between the first image and the second image, meanwhile, the obtained similarity is corrected by performing manual verification in a visual difference area mode, the accuracy of similarity judgment of the first object and the second object is greatly improved in the whole process, and the effect of upper-layer application such as track restoration or monitoring is improved.

In the embodiment, the implementation form of the terminal device 101 is different according to different application scenarios, for example, in a high-speed toll collection system, the terminal device 101 may be implemented as a toll terminal at a high-speed toll station; in the urban traffic monitoring system, the terminal device 101 may be implemented as a monitoring terminal. This will be explained in detail below.

Scenario example 1: high-speed charging system

Fig. 2a is a schematic structural diagram of a high-speed charging system 200a, which includes: the system comprises a charging terminal 201a, a cloud server 202a, an image acquisition device 203a and an image database 204 a. The charging terminal 201a may be, but is not limited to, a desktop computer, a notebook computer, or a smart phone; the image capturing device 203a may be, but is not limited to, a camera, an electronic eye, or the like; the image database 204a may be a storage system such as a Data warehouse or a Data Lake (Data Lake); in fig. 2a, the charging terminal 201a is a desktop computer, and the image capturing device 203a is a camera, but the charging terminal is not limited thereto.

In the high-speed toll collection system 200a, the image acquisition device 203a is used for acquiring images of passing vehicles at an entrance and an exit of a high-speed road and providing the acquired images to the toll terminal 201a, the toll terminal 201a receives the images of the passing vehicles at the entrance and the exit, license plate information in the images is identified, a payment account number associated with the image and a credit level of a user are determined according to the license plate information, and the toll terminal 201a calculates fees to be paid by the vehicles based on the credit level of the user and the positions of the entrance and the exit of the high-speed road and registers payment records of the vehicles in the payment account number for subsequent check. The image acquired by the image acquisition device 203a and the related information thereof may also be stored in the image database 204 a. The image information collected by the image collecting device 203a may be directly uploaded to the image database 204a by the image collecting device 203a, or the image collected by the image collecting device 203a may be uploaded to the image database 204a by the charging terminal 201 a; the latter is illustrated in fig. 2a as an example.

In practical application, because of shooting angle, weather or shielding and the like, the license plate information of the vehicle in the image acquired by the image acquisition device 203a may be blurred or invisible, in order to ensure that the charging operation for the vehicle can still be completed under such a condition, the cloud server 202a is added to the system of the embodiment, and is responsible for providing the license plate information contained in the image in which the license plate information may be blurred or invisible for the charging terminal 201 a. Specifically, after acquiring an image of a vehicle whose license plate information is unclear or invisible, the charging terminal 201a may provide the image to the cloud server 202a, as shown in fig. 2 a. For convenience of description, an image containing unclear or invisible license plate information is referred to as a first vehicle image, and a vehicle contained in the first image is referred to as a first vehicle, the license plate information of which is to be determined.

Specifically, as shown in fig. 2a, after receiving the first vehicle image, the cloud server 202a selects at least one image that is the same as or similar to the first vehicle image from the image database, and records the image as the second vehicle image. The second vehicle image comprises a second vehicle and license plate information thereof. In an alternative embodiment, the image database stores the feature vectors of the vehicle images in addition to the vehicle images. Based on this, one way for the cloud server 202a to select the second vehicle image from the image database includes: calculating a feature vector of the first vehicle image, and selecting at least one image with the feature vector same as or similar to the first vehicle image from the known images as a second vehicle image based on the feature vector of the first vehicle image.

As shown in fig. 2a, the similarity between the first vehicle in the first vehicle image and the second vehicle in the second vehicle image is calculated. The cloud server 202a may calculate a similarity between the first vehicle in the first vehicle image and the second vehicle in the second vehicle image using a neural network model. Specifically, a first vehicle image and a second vehicle image may be fed into a neural network model, where the first vehicle image and the second vehicle image may be respectively aligned with an image template. In this embodiment, an image template is maintained in advance, and the template includes designated key points and positions thereof; cloud server 202a may detect the locations of specified keypoints in the first vehicle image and the second vehicle image; aligning the first vehicle image and the second vehicle image with the image template according to the positions of the specified key points in the first vehicle image and the second vehicle image and the positions of the specified key points in the image template; the alignment is to perform deformation processing such as stretching, extending, enlarging or reducing on the first vehicle image or the second vehicle image so that the positions of the same designated key points can be completely overlapped when the first vehicle image or the second vehicle image and the image template are overlapped.

Further, as shown in fig. 2a, after the alignment, the first vehicle image and the second vehicle image are spliced into the target image as two image areas, and specifically, the first vehicle image and the second vehicle image may be vertically spliced together through a splicing function (e.g., an hconcat or vconcat function) to obtain the target image; further, different operations are carried out on the target image through two branches, the first branch is a Feature branch, Feature extraction can be carried out on two image areas in the target image, and visual features of the first vehicle image and the second vehicle image are obtained; the second branch is an Attention (Attention) branch, and difference detection can be carried out on two image areas in the target image to obtain a difference area between the first vehicle image and the second vehicle image; and further, inputting the visual features and the difference region into a full connection layer of the neural network model for similarity detection to obtain the similarity between the first vehicle and the second vehicle.

As shown in fig. 2a, when the cloud server 202a determines that the similarity between the first vehicle and the second vehicle meets the set condition, the license plate information of the second vehicle is used as the license plate information of the first vehicle, and the license plate information of the first vehicle is output to the charging terminal 201a, so that the charging terminal 201a performs subsequent charging or auditing work based on the license plate information.

Optionally, the high-speed charging system may further include a charging server, the charging terminal may transmit the first vehicle image to the charging server, the charging server forwards the first vehicle image to the cloud server, the cloud server receives the first vehicle image and obtains a second vehicle image similar to or identical to the first vehicle image, when the similarity between the second vehicle image and the first vehicle image satisfies a set condition, the license plate information of the second vehicle is used as the license plate information of the first vehicle, and the license plate information of the first vehicle is returned to the charging server, and the charging server provides the charging terminal with a fee to be charged based on the information.

Scenario example 2: urban traffic monitoring system

Fig. 2b is a schematic structural diagram of an urban traffic monitoring system 200b, which includes: the system comprises a monitoring terminal 201b, a cloud server 202b, an image acquisition device 203b and an image database 204 a. The monitoring terminal 201b may be, but is not limited to, a desktop computer, a notebook computer, or a smart phone; the image capturing device 203b may be, but is not limited to, a camera or an electronic eye, etc.; the image database 204b may be a storage system such as a data warehouse or a data lake; in fig. 2b, the monitor terminal 201b is a desktop computer, and the image capturing device 203b is a camera, but the present invention is not limited thereto.

The urban traffic monitoring system 200b is an important component of a traffic guidance system, provides the most intuitive reflection of the field situation, and is a basic guarantee for implementing accurate scheduling, wherein the image acquisition device 203b is used for acquiring video images of monitoring points of key sites, and provides the acquired image data to the monitoring terminal 201b, so that the monitoring terminal 201b integrates or issues information, and the like, so that traffic guidance managers can timely and accurately monitor and track traffic violations, traffic jams, traffic accidents and other emergencies, and correspondingly adjust various system control parameters and guidance scheduling strategies. The image acquired by the image acquisition device 203b and the related information thereof may also be stored in the image database 204 b. The image information collected by the image collecting device 203b can be directly uploaded to the image database 204b, or the image collected by the image collecting device 203a can be uploaded to the image database 204b by the monitoring terminal 201 b; the latter is illustrated in fig. 2b as an example.

In practical application, because of shooting angle, weather or shielding and the like, the license plate information of the vehicle in the image acquired by the image acquisition device 203b may be blurred or invisible, and in order to ensure that the collection of the vehicle information can still be completed under such a condition, the cloud server 202b is added to the system of the embodiment and is responsible for providing the license plate information contained in the image in which the license plate information may be blurred or invisible for the monitoring terminal 201 b. Specifically, after acquiring the vehicle image with unclear or invisible license plate information, the monitoring terminal 201b may provide the image to the cloud server 202b, as shown in fig. 2 b. For convenience of description, an image containing unclear or invisible license plate information is referred to as a first vehicle image, and a vehicle contained in the first vehicle image is referred to as a first vehicle, the license plate information of which is to be determined.

Specifically, as shown in fig. 2b, after receiving the first vehicle image, the cloud server 202b selects at least one image that is the same as or similar to the first vehicle image from the image database, and records the image as the second vehicle image. The second vehicle image comprises a second vehicle and license plate information thereof. In an alternative embodiment, the image database stores the feature vectors of the vehicle images in addition to the vehicle images. Based on this, one way for the cloud server 202b to select the second vehicle image from the image database includes: calculating a feature vector of the first vehicle image, and selecting at least one image with the feature vector same as or similar to the first vehicle image from the known images as a second vehicle image based on the feature vector of the first vehicle image.

As shown in fig. 2b, the similarity between the first vehicle in the first vehicle image and the second vehicle in the second vehicle image is calculated. The cloud server 202b may calculate a similarity between the first vehicle in the first vehicle image and the second vehicle in the second vehicle image using a neural network model. Specifically, a first vehicle image and a second vehicle image may be fed into a neural network model, where the first vehicle image and the second vehicle image may be respectively aligned with an image template. In this embodiment, an image template is maintained in advance, and the template includes designated key points and positions thereof; cloud server 202b may detect the locations of the specified keypoints in the first vehicle image and the second vehicle image; aligning the first vehicle image and the second vehicle image with the image template according to the positions of the specified key points in the first vehicle image and the second vehicle image and the positions of the specified key points in the image template; the alignment is to perform deformation processing such as stretching, extending, enlarging or reducing on the first vehicle image or the second vehicle image so that the positions of the same designated key points can be completely overlapped when the first vehicle image or the second vehicle image and the image template are overlapped.

Further, as shown in fig. 2b, after the alignment, the first vehicle image and the second vehicle image are spliced into the target image as two image areas, and specifically, the first vehicle image and the second vehicle image may be vertically spliced together through a splicing function (e.g., an hconcat or vconcat function) to obtain the target image; further, different operations are carried out on the target image through two branches, the first branch is a Feature branch, Feature extraction can be carried out on two image areas in the target image, and visual features of the first vehicle image and the second vehicle image are obtained; the second branch is an Attention (Attention) branch, and difference detection can be carried out on two image areas in the target image to obtain a difference area between the first vehicle image and the second vehicle image; and further, inputting the visual features and the difference region into a full connection layer of the neural network model for similarity detection to obtain the similarity between the first vehicle and the second vehicle.

As shown in fig. 2b, when the similarity between the first vehicle and the second vehicle is determined to meet the set condition, the cloud server 202b takes the license plate information of the second vehicle as the license plate information of the first vehicle, and outputs the license plate information of the first vehicle to the monitoring terminal 201b, so that the monitoring terminal 201b performs subsequent information integration based on the license plate information.

Optionally, the urban traffic monitoring system may further include a monitoring server, the monitoring terminal may transmit the first vehicle image to the monitoring server, the monitoring server forwards the first vehicle image to the cloud server, the cloud server receives the first vehicle image and obtains a second vehicle image similar to or identical to the first vehicle image, when the similarity between the second vehicle image and the first vehicle image satisfies a set condition, the license plate information of the second vehicle is used as the license plate information of the first vehicle, and the license plate information of the first vehicle is returned to the monitoring server, and the monitoring server performs subsequent information integration based on the information and provides the integrated information to the monitoring terminal 201 b.

Fig. 3 is a schematic flowchart of an image processing method according to an exemplary embodiment of the present application; as shown in fig. 3, the method includes:

s301, acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object;

s302, extracting visual features in the first image and the second image, and detecting a difference area between the first image and the second image;

s303, determining the similarity between the first object and the second object according to the visual features and the difference area;

and S304, when the similarity of the first object and the second object meets the set condition, outputting the information of the first object according to the information of the second object.

In an optional embodiment, before extracting the visual features in the first image and the second image, the method provided in this embodiment further includes: aligning the first image and the second image according to the alignment relation of the first object and the second object; after the alignment, the first image and the second image are stitched into the target image as two image regions.

In an alternative embodiment, aligning the first image and the second image according to the alignment relationship of the first object and the second object includes: detecting positions of a designated key point in the first image and the second image, wherein the designated key point is an object key point under object categories to which the first object and the second object belong; the first image and the second image are both aligned with the image template based on the locations of the specified keypoints in the first image and the second image, and the locations of the specified keypoints in the image template.

In an alternative embodiment, extracting visual features in the first image and the second image comprises: performing feature extraction on two image areas in the target image to obtain visual features of the first image and the second image; detecting a difference region between the first image and the second image, comprising: and carrying out difference detection on two image areas in the target image to obtain a difference area between the first image and the second image.

In an optional embodiment, the method provided in this embodiment further includes: and marking a difference area in the target image, and outputting the image marked with the difference area.

In this embodiment, the embodiment of determining the similarity between the first object and the second object based on the visual feature and the difference region is not limited. In an optional embodiment, inputting the visual features and the difference region into a neural network model for similarity detection to obtain the similarity between the first object and the second object; in another alternative embodiment, the similarity of the visual features in the first image and the second image is calculated; calculating the difference degree between the first object and the second object according to the number and/or the positions of the difference areas; correcting the similarity of the visual features according to the difference degree to obtain the similarity between the first object and the second object; in a further alternative embodiment, the visual features located in the difference region are selected from the visual features, and the similarity between the first object and the second object is calculated based on the visual features located in the difference region.

In an optional embodiment, outputting the information of the first object according to the information of the second object includes: taking the information of the second object as the information of the first object and outputting the information of the first object; or searching object information associated with the second object as the information of the first object according to the information of the second object, and outputting the information of the first object.

In an alternative embodiment, acquiring the first image and the second image comprises: receiving a first image uploaded by terminal equipment, and calculating a feature vector of the first image; and selecting at least one image with the characteristic vector same as or similar to the first image from the known images as a second image according to the characteristic vector of the first image.

FIG. 4 is a schematic flow chart diagram illustrating a vehicle information providing method according to an exemplary embodiment of the present application; as shown in fig. 4, the method includes:

s401, receiving a first image uploaded by a terminal device, wherein the first image comprises a first vehicle of which license plate information is to be determined;

s402, acquiring a second image, wherein the second image comprises a second vehicle and license plate information thereof;

s403, extracting visual features in the first image and the second image, and detecting a difference area between the first image and the second image;

s404, determining the similarity between the first vehicle and the second vehicle according to the visual features and the difference area;

s405, when the similarity between the first vehicle and the second vehicle meets a set condition, the license plate information of the second vehicle is used as the license plate information of the first vehicle and returned to the terminal device.

In an alternative embodiment, acquiring the second image comprises: calculating a feature vector of the first image; and selecting at least one image with the characteristic vector same as or similar to the first image from the known vehicle images as a second image according to the characteristic vector of the first image.

In an optional embodiment, the terminal device is a toll terminal at a high-speed toll station; or, the monitoring terminal is a monitoring terminal in the urban traffic monitoring system.

FIG. 5a is a block diagram of another exemplary image processing method according to an exemplary embodiment of the present disclosure; fig. 5b is a schematic flowchart of another image processing method according to an exemplary embodiment of the present application. As shown in fig. 5b, the method comprises:

s501, acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object;

s502, splicing the first image and the second image into a target image, wherein the target image comprises visual features in the first image and the second image;

s503, marking a difference area between the first image and the second image in the target image;

and S504, when the visual features in the difference area meet the set conditions, outputting the information of the second object as the information of the first object, and outputting the target image.

As shown in fig. 5a, the image processing apparatus may acquire a first image including a first object and a second image including a second object; information of the first object is pending and information of the second object is known; according to different application scenarios, the implementation of obtaining the first image and the second image may be different, and in addition, the implementation of the first object and the second object and the information thereof may be different, and for related examples, reference may be made to the foregoing embodiments, and details are not described here.

Further, as shown in fig. 5a, the image processing apparatus may stitch the first image and the second image into the target image, and mark a difference region between the first image and the second image in the target image, thereby obtaining the target image including the visual features and the difference region in the first image and the second image. Optionally, before stitching the first image and the second image, the first image and the second image may be aligned according to an alignment relationship of the first object and the second object. For a detailed implementation of the alignment and splicing, reference may be made to the foregoing embodiments, and details are not repeated here.

In this embodiment, the image processing apparatus may determine whether the first object and the second object are similar according to the visual feature in the marked difference region in the target image. Specifically, it may be determined whether or not the visual feature in the difference region satisfies the setting condition, and when the visual feature in the difference region satisfies the setting condition, it indicates that the similarity between the first object and the second object is high, and both of them may be considered as the same object, so that the information of the second object may be output as the information of the first object. Further, as shown in fig. 5a, the present embodiment may also output a target image including visual features of the first image and the second image and a difference region between the first image and the second image. For the implementation of obtaining the visual features and the difference regions, reference may be made to the foregoing embodiments, which are not described herein again.

For a user, the similarity of the first object and the second object can be manually verified by combining the marked difference region in the target image; and correcting the similarity result of the two objects obtained by the image processing equipment by using the similarity verified manually. For example, if the result of the manual verification is that the first object and the second object have a significant difference, that is, the first object and the second object are not similar, the information of the second image output by the image processing device is discarded, and the second object similar to the first object can be found again, so as to obtain the information of the first object based on the information of the second object; if the first object and the second object are similar as a result of the manual verification, the information of the second object output by the image processing device can be directly used as the information of the first object. The similarity of the first image and the second image is corrected in a manual verification mode, so that the accuracy of similarity judgment can be improved, and the quality and the effect of corresponding operation performed according to information output by the image processing equipment in the follow-up process are guaranteed.

In the embodiment of the application, the visual characteristics of the two images and the difference area between the two images can be combined aiming at the two images containing the object, and the visual characteristics and the difference area of the two images are combined at the same time, so that the difference and the similarity between the two objects can be more intuitively and accurately embodied, further, when the two objects are identified to meet the similarity condition, the information of the other object can be accurately provided on the basis of the information of the one object, and the accuracy of information acquisition is remarkably improved.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps S301 to S303 may be device a; for another example, the execution subject of steps S301 and S302 may be device a, and the execution subject of step S303 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations may be executed out of the order they appear herein or in parallel, and the order of the operations, such as S301, S302, etc., is merely used to distinguish between the various operations, and the order itself does not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

Fig. 6a is a schematic structural diagram of an image processing apparatus according to an exemplary embodiment of the present application. As shown in fig. 6a, the image processing apparatus includes: a memory 54 and a processor 55.

A memory 54 for storing a computer program and may be configured to store other various data to support operations on the image processing apparatus. Examples of such data include instructions for any application or method operating on an image processing device.

The memory 54 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

A processor 55 coupled to the memory 54 for executing computer programs in the memory 54 for: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first object and the second object according to the visual features and the difference region; and when the similarity of the first object and the second object meets the set condition, outputting the information of the first object according to the information of the second object.

In an alternative embodiment, prior to extracting the visual features in the first and second images, the processor 55 is further configured to: aligning the first image and the second image according to the alignment relation of the first object and the second object; after the alignment, the first image and the second image are stitched into the target image as two image regions.

In an alternative embodiment, the processor 55, when aligning the first image and the second image according to the alignment relationship between the first object and the second object, is specifically configured to: detecting positions of a designated key point in the first image and the second image, wherein the designated key point is an object key point under object categories to which the first object and the second object belong; the first image and the second image are both aligned with the image template based on the locations of the specified keypoints in the first image and the second image, and the locations of the specified keypoints in the image template.

In an alternative embodiment, the processor 55, when extracting the visual features in the first image and the second image, is specifically configured to: performing feature extraction on two image areas in the target image to obtain visual features of the first image and the second image; the processor 55, when detecting the difference region between the first image and the second image, is specifically configured to: and carrying out difference detection on two image areas in the target image to obtain a difference area between the first image and the second image.

In an alternative embodiment, processor 55 is further configured to: and marking a difference area in the target image, and outputting the image marked with the difference area.

In an alternative embodiment, the processor 55, when determining the similarity between the first object and the second object based on the visual characteristics and the difference region, is specifically configured to: inputting the visual characteristics and the difference region into a neural network model for similarity detection to obtain the similarity between the first object and the second object; or calculating the similarity of the visual characteristics in the first image and the second image; calculating the difference degree between the first object and the second object according to the number and/or the positions of the difference areas; correcting the similarity of the visual features according to the difference degree to obtain the similarity between the first object and the second object; or selecting the visual features in the difference area from the visual features, and calculating the similarity between the first object and the second object according to the visual features in the difference area.

In an optional embodiment, when outputting the information of the first object according to the information of the second object, the processor 55 is specifically configured to: taking the information of the second object as the information of the first object and outputting the information of the first object; or searching object information associated with the second object as the information of the first object according to the information of the second object, and outputting the information of the first object.

In an alternative embodiment, the processor 55, when acquiring the first image and the second image, is specifically configured to: receiving a first image uploaded by terminal equipment, and calculating a feature vector of the first image; and selecting at least one image with the characteristic vector same as or similar to the first image from the known images as a second image according to the characteristic vector of the first image.

The image processing device provided by the embodiment of the application can determine the similarity between the objects contained in the two images by combining the visual characteristics of the two images and the difference area between the two images aiming at the two images containing the objects, and can provide the information of one object according to the information of the other object when the similarity meets the set condition. The method and the device have the advantages that the visual characteristics and the difference areas of the two images are combined simultaneously, the difference and the same between the two objects can be displayed more visually and accurately, and further, when the two objects are identified to meet the similarity condition, the information of one object can be accurately provided based on the information of the other object, so that the accuracy of information acquisition is remarkably improved.

Further, as shown in fig. 6a, the image processing apparatus further includes: communication components 56, display 57, power components 58, audio components 59, and the like. Only some of the components are schematically shown in fig. 6a and it is not meant that the image processing apparatus comprises only the components shown in fig. 6 a. It should be noted that the components within the dashed box in fig. 6a are optional components, not necessary components, and may be determined according to the product form of the image processing apparatus.

The image processing device of this embodiment may be implemented as a terminal device such as a desktop computer, a notebook computer, or a smart phone, or may be a server device such as a conventional server, a cloud server, or a server array. If the image processing device of this embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, etc., the image processing device may include components within a dashed line frame in fig. 6 a; if the image processing apparatus of the present embodiment is implemented as a server-side apparatus such as a conventional server, a cloud server, or a server array, the components within the dashed box in fig. 6a may not be included.

Accordingly, the present application also provides a computer readable storage medium storing a computer program, where the computer program can implement the steps that can be executed by the image processing apparatus in the method embodiment of fig. 3 described above when executed.

The embodiment of the present application further provides an image processing apparatus, the implementation structure of which is the same as or similar to that of the image processing apparatus shown in fig. 6a, and which can be implemented with reference to the structure of the image processing apparatus shown in fig. 6 a. The image processing apparatus provided by the present embodiment differs from the image processing apparatus in the embodiment shown in fig. 6a mainly in that: the functions performed by the processor to execute the computer programs stored in the memory are different. For the image processing apparatus provided in this embodiment, the processor thereof executes the computer program stored in the memory, and is operable to: acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object; splicing the first image and the second image into a target image, wherein the target image comprises visual features in the first image and the second image; marking a difference region between the first image and the second image in the target image; and outputting information of the second object as information of the first object and outputting the target image in a case where the visual feature in the difference region satisfies the setting condition.

Accordingly, the present application further provides a computer readable storage medium storing a computer program, and the computer program can implement the steps that can be executed by the image processing apparatus in the method embodiment of fig. 5 b.

Fig. 6b is a schematic structural diagram of a vehicle information providing device according to an exemplary embodiment of the present application. As shown in fig. 6b, the vehicle information providing apparatus includes: a memory 64 and a processor 65.

The memory 64 is used to store a computer program, and may be configured to store other various data to support operations on the vehicle information providing apparatus. Examples of such data include instructions for any application or method operating on the vehicle information providing device.

The memory 64 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

A processor 65, coupled to the memory 64, for executing computer programs in the memory 64 for: receiving a first image uploaded by a terminal device, wherein the first image comprises a first vehicle of which license plate information is to be determined; acquiring a second image, wherein the second image comprises a second vehicle and license plate information thereof; extracting visual features in the first image and the second image, and detecting a difference region between the first image and the second image; determining a similarity between the first vehicle and the second vehicle according to the visual features and the difference area; and when the similarity of the first vehicle and the second vehicle meets the set condition, returning the license plate information of the second vehicle to the terminal equipment as the license plate information of the first vehicle.

In an alternative embodiment, the processor 65 is specifically configured to: acquiring a second image comprising: calculating a feature vector of the first image; and selecting at least one image with the characteristic vector same as or similar to the first image from the known vehicle images as a second image according to the characteristic vector of the first image.

According to the vehicle information providing device provided by the embodiment of the application, for two images containing objects, the similarity between the objects contained in the two images can be determined by combining the visual features of the two images and the difference area between the two images, and when the similarity meets the set condition, the information of one object can be provided according to the information of the other object. The method and the device have the advantages that the visual characteristics and the difference areas of the two images are combined simultaneously, the difference and the same between the two objects can be displayed more visually and accurately, and further, when the two objects are identified to meet the similarity condition, the information of one object can be accurately provided based on the information of the other object, so that the accuracy of information acquisition is remarkably improved.

Further, as shown in fig. 6b, the image processing apparatus further includes: communication components 66, display 67, power components 68, audio components 69, and the like. Only some of the components are schematically shown in fig. 6b, and it is not meant that the image processing apparatus comprises only the components shown in fig. 6 b. It should be noted that the components within the dashed box in fig. 6b are optional components, not necessary components, and may be determined according to the product form of the image processing apparatus.

The image processing device of this embodiment may be implemented as a terminal device such as a desktop computer, a notebook computer, or a smart phone, or may be a server device such as a conventional server, a cloud server, or a server array. If the image processing device of this embodiment is implemented as a terminal device such as a desktop computer, a notebook computer, a smart phone, etc., the image processing device may include components within a dashed line frame in fig. 6 b; if the image processing apparatus of the present embodiment is implemented as a server-side apparatus such as a conventional server, a cloud server, or a server array, the components within the dashed box in fig. 6b may not be included.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing a computer program, where the computer program can implement the steps that can be executed by the vehicle information providing device in the above-mentioned vehicle information providing method embodiments.

The communication components of fig. 6a and 6b described above are configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

The displays in fig. 6a and 6b described above include screens, which may include Liquid Crystal Displays (LCDs) and Touch Panels (TPs). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

The power supply components of fig. 6a and 6b described above provide power to the various components of the device in which the power supply components are located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

The audio components of fig. 6a and 6b described above may be configured to output and/or input audio signals. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image processing method, comprising:

acquiring a first image and a second image, wherein the first image comprises a first object, and the second image comprises a second object;

extracting visual features in the first image and the second image and detecting a difference region between the first image and the second image;

determining a similarity between the first object and the second object according to the visual features and the difference region;

and when the similarity of the first object and the second object meets a set condition, outputting the information of the first object according to the information of the second object.

2. The method of claim 1, further comprising, prior to extracting visual features in the first image and the second image:

aligning the first image and the second image according to the alignment relation of the first object and the second object;

after the alignment, the first image and the second image are stitched into a target image as two image regions.

3. The method of claim 2, wherein aligning the first image and the second image according to the alignment relationship of the first object and the second object comprises:

detecting positions of specified key points in the first image and the second image, wherein the specified key points are object key points under object categories to which the first object and the second object belong;

aligning both the first image and the second image with the image template according to the positions of the designated keypoints in the first image and the second image and the positions of the designated keypoints in the image template.

4. The method according to claim 2 or 3,

the extracting visual features in the first image and the second image comprises: performing feature extraction on two image areas in the target image to obtain visual features of the first image and the second image;

the detecting a region of difference between the first image and the second image comprises: and carrying out difference detection on two image areas in the target image to obtain a difference area between the first image and the second image.

5. The method of claim 4, further comprising:

and marking the difference region in the target image, and outputting the image marked with the difference region.

6. The method of any one of claims 1-3, wherein determining the similarity between the first object and the second object based on the visual feature and the region of difference comprises:

inputting the visual characteristics and the difference region into a neural network model for similarity detection to obtain the similarity between the first object and the second object;

or

Calculating the similarity of the visual features in the first image and the second image; calculating a difference degree between the first object and the second object according to the number and/or the position of the difference areas; correcting the similarity of the visual features according to the difference to obtain the similarity between the first object and the second object;

or

Selecting visual features located in the difference area from the visual features, and calculating the similarity between the first object and the second object according to the visual features located in the difference area.

7. The method according to any one of claims 1-3, wherein outputting the information of the first object according to the information of the second object comprises:

taking the information of the second object as the information of the first object, and outputting the information of the first object;

or

And searching object information associated with the second object according to the information of the second object to be used as the information of the first object, and outputting the information of the first object.

8. The method of any of claims 1-3, wherein acquiring the first image and the second image comprises:

receiving a first image uploaded by terminal equipment, and calculating a feature vector of the first image;

and selecting at least one image with the characteristic vector same as or similar to the first image from the known images as a second image according to the characteristic vector of the first image.

9. A vehicle information providing method characterized by comprising:

receiving a first image uploaded by a terminal device, wherein the first image comprises a first vehicle of which license plate information is to be determined;

acquiring a second image, wherein the second image comprises a second vehicle and license plate information thereof;

determining a similarity between the first vehicle and the second vehicle according to the visual features and the difference region;

and when the similarity between the first vehicle and the second vehicle meets a set condition, returning the license plate information of the second vehicle to the terminal equipment as the license plate information of the first vehicle.

10. The method of claim 9, wherein acquiring a second image comprises:

calculating a feature vector of the first image;

and selecting at least one image with the characteristic vector same as or similar to the first image from the known vehicle images as a second image according to the characteristic vector of the first image.

11. The method according to claim 9 or 10, wherein the terminal device is a charging terminal in a high-speed charging system; or, the monitoring terminal is a monitoring terminal in the urban traffic monitoring system.

12. An image processing method, comprising:

stitching the first image and the second image into a target image, wherein the target image comprises visual features in the first image and the second image;

marking a difference region between the first image and the second image in the target image; and

and outputting the information of the second object as the information of the first object and outputting the target image when the visual features in the difference region meet set conditions.

13. An image processing apparatus characterized by comprising: a memory and a processor;

the memory for storing a computer program;

the processor, coupled with the memory, to execute the computer program to:

14. A vehicle information providing apparatus characterized by comprising: a memory and a processor;

the memory for storing a computer program;

the processor, coupled with the memory, to execute the computer program to:

15. An image processing apparatus characterized by comprising: a memory and a processor;

the memory for storing a computer program;

the processor, coupled with the memory, to execute the computer program to:

16. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 12.