WO2021072886A1 - 图像风格迁移方法、装置、设备及存储介质 - Google Patents

图像风格迁移方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021072886A1
WO2021072886A1 PCT/CN2019/119118 CN2019119118W WO2021072886A1 WO 2021072886 A1 WO2021072886 A1 WO 2021072886A1 CN 2019119118 W CN2019119118 W CN 2019119118W WO 2021072886 A1 WO2021072886 A1 WO 2021072886A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
night
mapping relationship
preset
content
Prior art date
Application number
PCT/CN2019/119118
Other languages
English (en)
French (fr)
Inventor
王义文
王健宗
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021072886A1 publication Critical patent/WO2021072886A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • This application relates to the field of image processing, in particular to image style transfer methods, devices, equipment and storage media.
  • neural style transfer which transfers the artistic style of artistic works to daily life In the photo, it has become a computer vision task that is highly valued in academia and industry.
  • image style transfer is to make directional changes to the texture, color, content, etc. of the image, so that the image changes from one style to another, for example, to transfer the style of a photo of a person to obtain an image with oil painting style. , Or transfer the style of the landscape photos taken under dim light conditions to obtain images under brighter light conditions. Therefore, semantic labeling of night scenes has become a hot research direction.
  • image semantic annotation has become a research hotspot in the field of image processing and computer vision.
  • the inventor realizes that the current main method for solving semantic segmentation is to use a large number of annotations to train deep neural networks.
  • This supervised learning scheme has been successful on images with good daylight conditions, but for other unfavorable lighting conditions, it can be The scalability is very poor, so it cannot meet the all-weather visual recognition needs of many outdoor applications. For example, at night and in bad weather, the quality of the collected license plate images is poor and cannot be used as a valuable training sample.
  • Image segmentation is a crucial preprocessing for image recognition and computer vision. Correct recognition cannot be achieved without correct segmentation.
  • the only basis for image segmentation is the brightness and color of the pixels in the image, which makes the computer encounter various difficulties when automatically processing the segmentation. For example, uneven lighting, the influence of noise, the presence of unclear parts and shadows in the image, etc. often cause segmentation errors, and it is impossible to effectively identify the vehicles, license plates and roads in the business.
  • the main purpose of this application is to solve the technical problems of inaccurate labeling of image training samples collected under unfavorable lighting conditions and poor scalability.
  • the first aspect of the present application provides an image style transfer method, including: acquiring a day content image set and a night image reference set, the number of the day content image set and the night image reference set The number is equal, the day content image set is a real image set collected and annotated according to a preset service; an auxiliary image reference set is preset according to the night image reference set, and the auxiliary image reference set is the scene and the night image
  • the scenes of the reference set are consistent, and the style is a daytime preset image set; according to a preset algorithm, feature matching is performed on the daytime content image set and the auxiliary image reference set to obtain a first mapping relationship.
  • the mapping relationship is used to indicate the image correspondence relationship between the daytime content image set and the auxiliary image reference set; calculation is performed according to the first mapping relationship to obtain a second mapping relationship, and the second mapping relationship is used for Indicate the image correspondence relationship between the day content image set and the night image reference set; perform style transfer on the day content image set and the night image reference set according to the second mapping relationship to obtain a target An image set, where the target image set is a marked night image training sample.
  • a second aspect of the present application provides an image style transfer device, including: an acquiring unit, configured to acquire a day content image set and a night image reference set, the number of the day content image set and the night image reference set The number is equal, the day content image set is a real image set collected and annotated according to a preset service; the setting unit is configured to preset an auxiliary image reference set according to the night image reference set, and the auxiliary image reference set is a scene A preset image set that is consistent with the scene of the night image reference set and whose style is daytime; the matching unit is configured to perform feature matching between the daytime content image set and the auxiliary image reference set according to a preset algorithm, Obtain a first mapping relationship, where the first mapping relationship is used to indicate the image correspondence relationship between the daytime content image set and the auxiliary image reference set; a calculation unit is used to perform calculations based on the first mapping relationship To obtain a second mapping relationship, where the second mapping relationship is used to indicate the image correspondence relationship between the daytime content image set and the night image
  • a third aspect of the present application provides an image style transfer device, including: a memory and at least one processor, the memory stores instructions, the memory and the at least one processor are interconnected through a wire; the at least one processor The device invokes the instructions in the memory, so that the image style transfer device executes the method described in the first aspect.
  • the fourth aspect of the present application provides a computer-readable storage medium having instructions stored in the computer-readable storage medium, which when run on a computer, cause the computer to execute the method described in the first aspect.
  • a day content image set and a night image reference set are acquired, the number of the day content image set is equal to the number of the night image reference set, and the day content image set is based on a predetermined Set the real image set collected and annotated by the business; preset an auxiliary image reference set according to the night image reference set, the auxiliary image reference set is the scene consistent with the scene of the night image reference set, and the style is the daytime preset Set image set; according to a preset algorithm, the day content image set and the auxiliary image reference set are matched to obtain a first mapping relationship, and the first mapping relationship is used to indicate the day content image set and The image correspondence relationship between the auxiliary image reference sets; calculation is performed according to the first mapping relationship to obtain a second mapping relationship, and the second mapping relationship is used to indicate the day content image set and the night image Image correspondence between reference sets; according to the second mapping relationship, perform style transfer on the day content image set and the night image reference set to obtain a target image set, and the target image
  • the auxiliary image reference set is used as a pairing bridge between the day content image set and the night image reference set, which facilitates the feature extraction and matching of the day content image set and the night image reference set, and the marked day
  • the real image is converted into a synthesized night image, and the synthesized night image is used as a training sample for semantic segmentation, avoiding the workload of relabeling, and improving the accuracy and scalability of night image semantic segmentation.
  • FIG. 1 is a schematic diagram of an embodiment of an image style transfer method in an embodiment of the application
  • FIG. 2 is a schematic diagram of another embodiment of the image style transfer method in the embodiment of the application.
  • FIG. 3 is a schematic diagram of an embodiment of an image style transfer device in an embodiment of the application.
  • FIG. 4 is a schematic diagram of another embodiment of the image style transfer device in the embodiment of the application.
  • FIG. 5 is a schematic diagram of an embodiment of an image style transfer device in an embodiment of the application.
  • the embodiments of the present application provide an image style transfer method, device, equipment, and storage medium, which are used to facilitate the comparison of the day content image set by using the auxiliary image reference set as the pairing bridge between the day content image set and the night image reference set.
  • Feature extraction and matching with the night image reference set convert the labeled real day image into a synthesized night image, and use the synthesized night image as a training sample for semantic segmentation, avoiding the workload of relabeling and improving the semantics of night images Accuracy and scalability of segmentation.
  • An embodiment of the image style transfer method in the embodiment of the present application includes:
  • the number of day content image sets is equal to the number of night image reference sets
  • the day content image set is a real image set collected and annotated according to a preset service
  • the server obtains the day content image set and the night image reference set.
  • the number of day content image sets is equal to the number of night image reference sets.
  • the marked semantic features in the daytime content image set include vehicles, license plates, and roads, which are not specifically limited here.
  • the preset business includes loss assessment business and vehicle claims business, and the specifics are not limited here.
  • the day content image set is a group of photos taken in the daytime scene
  • the night image reference set is a group of photos taken in the night scene.
  • the number of images contained in the day content image set and the number of night image reference sets are The same, but the order of the scenes is not a one-to-one relationship.
  • the auxiliary image reference set is a preset image set whose scene is consistent with the scene of the night image reference set, and whose style is daytime;
  • the server presets the auxiliary image reference set according to the night image reference set.
  • the auxiliary image reference set is a preset image set whose scene is consistent with the scene of the night image reference set and whose style is daytime.
  • the preset image set refers to the preset image set based on the night image.
  • the auxiliary image reference set preset by the scene of the reference set, the server presets the scene of the auxiliary image reference set according to the scene of the night image reference set, and the server according to the unique identification of each image in the night image reference set and the uniqueness of the auxiliary image reference set
  • the identifier is mapped, and the preset mapping relationship is obtained.
  • K is a positive integer, and has the same scene as the night image reference set R, that is, K is equal to N
  • the preset mapping relationship is obtained as B, where B: ⁇ 1,2...,K ⁇ 1,2...,N ⁇ , that is, B is the mapping of the night image set of the same scene.
  • style transfer uses similar semantic content to span two images.
  • the server can obtain more accurate output by using paired image pairs with similar advanced features.
  • similar semantic content such as crows and swallows, cars and bus.
  • Due to the problems of a large number of semantic categories, mutual occlusion, low-level visual feature recognition and uneven illumination in the night image reference set if the server directly extracts and matches features based on the day content image set and the night image reference set, it will cause some problems.
  • the feature recognition is not accurate, so the server selects an auxiliary image reference set for the server to perform feature extraction and matching.
  • the server performs feature matching between the day content image set and the auxiliary image reference set according to a preset algorithm to obtain a first mapping relationship, which is used to indicate the image correspondence relationship between the day content image set and the auxiliary image reference set. Specifically, the server performs feature extraction on the first image in the daytime content image set according to a preset algorithm to obtain the first feature; the server performs feature extraction on the second image in the auxiliary image reference set according to the preset algorithm to obtain the second feature, Among them, the first feature and the second feature are features with a deeper number of convolutional layers; the server separately calculates the value of each first image in the day content image set and multiple second images in the auxiliary image reference set according to the first and second features Multiple similarities, and the maximum similarity between each first image in the day content image set and multiple second images in the auxiliary image reference set is set as the maximum similarity, and the maximum similarity is used to indicate the basis similarity The maximum value of determines the image matching pair of the daytime content image set and the auxiliary image reference set. Further
  • the server performs calculations according to the first mapping relationship to obtain a second mapping relationship.
  • the second mapping relationship is used to indicate the image correspondence relationship between the day content image set and the night image reference set.
  • the server confirms that the preset mapping relationship from the image auxiliary reference set to the night image reference set is B: ⁇ 1,2,3 ,4,5 ⁇ 5,4,3,2,1 ⁇
  • the server confirms that the first mapping relationship from the daytime content image set to the image auxiliary reference set is A': ⁇ 1,2,3,4,5 ⁇ ⁇ 5,4,3,2,1 ⁇
  • the server obtains the second mapping relationship between the day content image set and the night image reference set as A: ⁇ 1,2,3,4,5 ⁇ 1,2 ,3,4,5 ⁇ .
  • the server performs style transfer on the day content image set and the night image reference set according to the second mapping relationship to obtain the target image set, and the target image set is the marked night image training sample.
  • the server transfers the style of each image in the night image reference set to the content of each corresponding day image in the day content image set to obtain the target image set.
  • the target image set is the marked night image training sample, for example, for A daytime content image that includes a license plate in a daytime scene, the license plate is ABCD, the color of the license plate is bright blue, and the corresponding one includes another license plate in the night scene is 12? ?
  • the last two digits of the license plate are blurry, and the background color of the license plate is black and gray.
  • the target night image is obtained as the license plate ABCD with the background color of black and gray. It is understandable that the target night image set includes the content of the day image set and the style of the night image set, and the content of the target image set is already labeled content, so the workload of relabeling is avoided.
  • the auxiliary image reference set is used as a pairing bridge between the day content image set and the night image reference set, which facilitates the feature extraction and matching of the day content image set and the night image reference set, and the marked day
  • the real image is converted into a synthetic night image
  • the synthetic night image is used as a training sample for semantic segmentation, avoiding the workload of relabeling, and improving the accuracy and scalability of night image semantic segmentation.
  • FIG. 2 another embodiment of the image style transfer method in the embodiment of the present application includes:
  • the number of day content image sets is equal to the number of night image reference sets, and the day content image sets are real image sets collected and labeled according to preset services;
  • the server obtains the day content image set and the night image reference set.
  • the number of day content image sets is equal to the number of night image reference sets.
  • the day content image set is a real image set collected and labeled according to a preset service. Among them, the preset business includes loss assessment business and vehicle claims settlement business, which are not specifically limited here.
  • the day content image set is a group of photos taken in the daytime scene
  • the night image reference set is a group of photos taken in the night scene.
  • the number of images contained in the day content image set and the number of night image reference sets are The same, but the order of the scenes is not a one-to-one relationship.
  • the server generates the first query statement according to the number of day content image sets, the number of night image reference sets, and the structured query language SQL language rules; the server executes the first query statement to obtain the day content image set and night image reference
  • the number of day content image collections is equal to the number of night image reference collections, and the day content image collections are real image collections collected and labeled according to preset services.
  • the marked semantic features in the daytime content image set include vehicles, license plates, and roads, which are not specifically limited here.
  • the auxiliary image reference set is a preset image set whose scene is consistent with the scene of the night image reference set, and whose style is daytime;
  • the server presets an auxiliary image reference set according to the night image reference set
  • the auxiliary image reference set is a scene consistent with the scene of the night image reference set
  • the style is a preset image set during the day.
  • the preset image set means that the auxiliary image reference set is preset according to the scene of the night image reference set, and the unique identification of each image in the night image reference set and the unique identification of the auxiliary image reference set are mapped to obtain the preset Mapping relations.
  • K is a positive integer, and has the same scene as the night image reference set R, that is, K is equal to N
  • the preset mapping relationship is obtained as B, where B: ⁇ 1,2...,K ⁇ 1,2...,N ⁇ , that is, B is the mapping of the night image set of the same scene.
  • the server reads the preset mapping relationship from the preset data table.
  • the preset mapping relationship is used to indicate that the mapping relationship is preset according to the unique identifier of the night image reference set and the unique identifier of the auxiliary image reference set;
  • the relationship and SQL language rules generate a second query statement; the server executes the second query statement to obtain an auxiliary image reference set.
  • the auxiliary image reference set is that the scene is consistent with the scene of the night image reference set, and the style is the daytime preset image set. For example, for a night image N in the night image reference set, the scene location of N is c, then the image corresponding to B in the auxiliary reference set is a day image including location c.
  • style transfer uses similar semantic content to span two images.
  • the server can obtain more accurate output by using paired image pairs with similar advanced features.
  • similar semantic content such as crows and swallows, cars and Bus etc.
  • the night image reference set has a large number of semantic categories, mutual occlusion, weak recognition of low-level visual features, and uneven lighting, if the server directly extracts and matches features based on the day content image set and the night image reference set, some feature recognition Inaccurate, so the server selects an auxiliary image reference set for the server to perform feature extraction and matching.
  • the server performs feature matching on the daytime content image set and the auxiliary image reference set according to the preset algorithm to obtain the first mapping relationship, and stores the first mapping relationship in the preset mapping data table.
  • the first mapping relationship is used to indicate the daytime The image correspondence between the content image set and the auxiliary image reference set.
  • the server performs feature extraction on the first image in the daytime content image set according to a preset algorithm to obtain the first feature; the server performs feature extraction on the second image in the auxiliary image reference set according to the preset algorithm to obtain the second feature,
  • the server uses a scale-invariant feature transformation algorithm as a preset algorithm to perform feature extraction on the first image and the second image; the server separately calculates each first image and multiple second images according to the first feature and the second feature Multiple similarities between the two to obtain the maximum similarity; the server determines the image matching relationship between the daytime content image set and the auxiliary image reference set according to the maximum similarity, obtains the first mapping relationship, and stores the first mapping relationship in Preset mapping data table.
  • F(I i ) is used to indicate feature extraction of images in the daytime content image set
  • F(R' k ) is used to indicate feature extraction of images in the auxiliary image reference set
  • ⁇ is used to indicate similarity.
  • the server calculates the daytime content image set and the auxiliary image reference set. As long as the features of the two pictures, most of the features in the same area can find similarities to each other, the two pictures can be considered similar. Furthermore, if most of the feature blocks in the same area of the two images are similar, it is confirmed that the two images are similar.
  • the similarity of the two features is measured by calculating the cosine distance between the features.
  • the server determines the first mapping relationship according to the maximum similarity. For example, after the server processes the daytime content image set I and the auxiliary image reference set R', the first mapping relationship A'is obtained: ⁇ 1,2...,M ⁇ 1,2...,K ⁇ , Where M and K are equal.
  • the server reads the preset mapping relationship and the first mapping relationship from the preset mapping data table. Specifically, the server generates the third query statement according to the preset mapping data table and the SQL syntax rules; the server executes the third query statement to obtain the query result, and the query result includes the preset mapping relationship and the first mapping relationship.
  • SQL structured query language
  • the server performs matrix multiplication calculation according to the preset mapping relationship and the first mapping relationship to obtain a second mapping relationship.
  • the second mapping relationship is used to indicate the paired image of the day content image set and the night image reference set.
  • the server presets the mapping relationship B: ⁇ 1,2,...,K ⁇ 1,2,...,N ⁇ and the first mapping relationship
  • the second mapping relationship is obtained as A: ⁇ 1,2,...,M ⁇ 1,2,...,N ⁇ .
  • the second mapping relationship uses the unique identifier of the paired image of the day content image set and the night image reference set for relationship mapping, that is, 1, 2, ... M are the unique identifiers of the day content image. 1, 2, ... N is the unique identification of the night image.
  • the server determines the paired image between the day content image set and the night image reference set according to the second mapping relationship, and the paired image includes the day content image and the night image. Specifically, the server analyzes the second mapping relationship to obtain the mapping identifier of the paired image.
  • the mapping identifier of the paired image includes the unique identifier of the day content image and the unique identifier of the night image; the server reads the daytime image according to the mapping identifier of the paired image.
  • the server obtains the second mapping relationship as A: ⁇ 1,2,...,M ⁇ 1,2,...,N ⁇ , and the server parses the second mapping relationship to obtain the image of the daytime content
  • the unique identifiers are 1, 2,...M
  • the unique identifiers of night images are 1, 2,...N, where 1, 2,...M and 1, 2,...N constitute the mapping identification of the paired image, then
  • the unique ID 1 of the day content image and the unique ID 1 of the night image are mapped to each other, the unique ID of the day content image 2 and the unique ID of the night image 3 are mapped to each other, and so on, the unique ID of the day content image
  • the identifier M and the unique identifier N of the night image are a mutual mapping relationship.
  • the server uses a preset deep convolutional neural network model to perform style transfer on the paired images according to the corresponding relationship to obtain the target image set, which is the marked night image training sample. Specifically, the server determines the size of the day content image, and generates a Gaussian white noise image according to the size of the day content image; the server inputs the day content image, night image, and Gaussian white noise image to the preset deep convolutional neural network model
  • the preset deep convolutional neural network model includes a content constraint feature extraction layer and a style constraint feature extraction layer; the server calculates the daytime content image and Gaussian white noise image in the content constraint feature extraction layer to obtain the content constraint layer loss function; The server calculates the night image and Gaussian white noise image in the style constraint feature extraction layer to obtain the style constraint layer loss function; the server accumulates the content constraint layer loss function and the style constraint layer loss function to obtain the total loss function; the server uses The gradient descent algorithm iteratively updates the total loss function to obtain the target
  • the target image set is a simulated image close to the real image, which improves the quality of the night image training samples in the preset business.
  • the target night image set of the marked vehicles and roads is used as the night image training sample.
  • the night image training samples train the semantic segmentation model, and perform semantic segmentation on the vehicle image to be recognized through the trained semantic segmentation model to obtain accurate vehicle and license plate segmentation results.
  • the target image set includes the content of the day content image set and the style of the night image set.
  • the license plate is ABCD
  • the color of the license plate is bright blue.
  • the corresponding one including the night scene and another license plate is 12? ?
  • the last two digits of the license plate are blurry
  • the background color of the license plate is black and gray.
  • the target night image is obtained as the license plate ABCD with the background color of black and gray. Further, the target night image set is used for semantic segmentation training to avoid the workload of relabeling.
  • the preset deep convolutional neural network model can use VGG-19.
  • the VGG-19 model refers to a 19-layer super-resolution test sequence (visual geometry group, VGG) network, including 16 convolutional layers and 5 Feature space provided by a pooling layer. Change the weight of the normalized network proportionally, and the average activation value of each layer of convolution filter in the image and position is equal to 1.
  • the auxiliary image reference set is used as a pairing bridge between the day content image set and the night image reference set, which facilitates the feature extraction and matching of the day content image set and the night image reference set, and the marked day
  • the real image is converted into a synthesized night image, and the synthesized night image is used as a training sample for semantic segmentation, avoiding the workload of relabeling, and improving the accuracy and scalability of night image semantic segmentation.
  • An embodiment of the image style transfer device in the embodiment of the application includes:
  • the acquiring unit 301 is used to acquire a day content image set and a night image reference set, the number of day content image sets is equal to the number of night image reference sets, and the day content image sets are real images collected and labeled according to preset services set;
  • the setting unit 302 is configured to preset an auxiliary image reference set according to the night image reference set, the auxiliary image reference set is a scene consistent with the scene of the night image reference set, and the style is a daytime preset image set;
  • the matching unit 303 is configured to perform feature matching between the day content image set and the auxiliary image reference set according to a preset algorithm to obtain a first mapping relationship, and the first mapping relationship is used to indicate the relationship between the day content image set and the auxiliary image reference set Correspondence of images;
  • the calculation unit 304 is configured to perform calculation according to the first mapping relationship to obtain a second mapping relationship, where the second mapping relationship is used to indicate the image correspondence relationship between the day content image set and the night image reference set;
  • the style transfer unit 305 is configured to perform style transfer on the day content image set and the night image reference set according to the second mapping relationship to obtain a target image set, and the target image set is a marked night image training sample.
  • the auxiliary image reference set is used as a pairing bridge between the day content image set and the night image reference set, which facilitates the feature extraction and matching of the day content image set and the night image reference set, and the marked day
  • the real image is converted into a synthesized night image, and the synthesized night image is used as a training sample for semantic segmentation, avoiding the workload of relabeling, and improving the accuracy and scalability of night image semantic segmentation.
  • FIG. 4 another embodiment of the image style transfer device in the embodiment of the present application includes:
  • the acquiring unit 301 is used to acquire a day content image set and a night image reference set, the number of day content image sets is equal to the number of night image reference sets, and the day content image sets are real images collected and labeled according to preset services set;
  • the setting unit 302 is configured to pre-set an auxiliary image reference set according to the night image reference set, the auxiliary image reference set is a scene that is consistent with the scene of the night image reference set, and the style is a daytime preset image set;
  • the matching unit 303 is configured to perform feature matching between the day content image set and the auxiliary image reference set according to a preset algorithm to obtain a first mapping relationship, and the first mapping relationship is used to indicate the relationship between the day content image set and the auxiliary image reference set Correspondence of images;
  • the calculation unit 304 is configured to perform calculation according to the first mapping relationship to obtain a second mapping relationship, where the second mapping relationship is used to indicate the image correspondence relationship between the day content image set and the night image reference set;
  • the style transfer unit 305 is configured to perform style transfer on the day content image set and the night image reference set according to the second mapping relationship to obtain a target image set, and the target image set is a marked night image training sample.
  • the matching unit 303 may also be specifically configured to:
  • the image matching relationship between the daytime content image set and the auxiliary image reference set is determined to obtain the first mapping relationship, and the first mapping relationship is stored in the preset mapping data table.
  • calculation unit 304 may also be specifically configured to:
  • the matrix multiplication calculation is performed according to the preset mapping relationship and the first mapping relationship to obtain the second mapping relationship, and the second mapping relationship is used to indicate the image correspondence relationship between the day content image set and the night image reference set.
  • the transfer learning unit 305 may further include:
  • the determining subunit 3051 is configured to determine a paired image between the day content image set and the night image reference set according to the second mapping relationship, and the paired image includes the day content image and the night image;
  • the style transfer subunit 3052 is used to perform style transfer on the paired images according to the corresponding relationship through a preset deep convolutional neural network model to obtain a target image set, which is a marked night image training sample.
  • style transfer subunit 3052 may also be specifically used for:
  • the night image and Gaussian white noise image are calculated to obtain the style constraint layer loss function
  • the gradient descent algorithm is used to iteratively update the total loss function to obtain the target image set, which is the marked night image training sample.
  • the obtaining unit 301 may also be specifically configured to:
  • the first query sentence is executed to obtain the day content image set and the night image reference set.
  • the day content image set is the real image set collected and labeled according to the preset service.
  • the setting unit 302 may also be specifically configured to:
  • the preset mapping relationship is used to instruct to preset the auxiliary image reference set according to the night image reference set;
  • the second query sentence is executed to obtain the auxiliary image reference set.
  • the auxiliary image reference set is a scene consistent with the scene of the night image reference set, and the style is a preset image set for daytime.
  • the auxiliary image reference set is used as a pairing bridge between the day content image set and the night image reference set, which facilitates the feature extraction and matching of the day content image set and the night image reference set, and the marked day
  • the real image is converted into a synthesized night image, and the synthesized night image is used as a training sample for semantic segmentation, avoiding the workload of relabeling, and improving the accuracy and scalability of night image semantic segmentation.
  • FIG. 5 is a schematic structural diagram of an image style transfer device provided by an embodiment of the present application.
  • the image style transfer device 500 may have relatively large differences due to different configurations or performance, and may include one or more processors (central processing units).
  • a CPU 501 for example, one or more processors
  • a memory 509 for example, one or more storage devices
  • storage media 508 for example, one or more storage devices
  • the memory 509 and the storage medium 508 may be short-term storage or persistent storage.
  • the program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the image style transfer device.
  • the processor 501 may be configured to communicate with the storage medium 508, and execute a series of instruction operations in the storage medium 508 on the image style transfer device 500.
  • the image style transfer device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and/or one or more operating systems 505, such as Windows Serve , Mac OS X, Unix, Linux, FreeBSD, etc.
  • operating systems 505 such as Windows Serve , Mac OS X, Unix, Linux, FreeBSD, etc.
  • FIG. 5 does not constitute a limitation on the image style transfer device, and may include more or less components than those shown in the figure, or a combination of certain components, or different components. The layout of the components.
  • the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium.
  • the computer-readable storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer executes the following steps:
  • the number of the day content image set is equal to the number of the night image reference set
  • the day content image set is a real image collected and annotated according to a preset service set
  • the auxiliary image reference set is a preset image set whose scene is consistent with the scene of the night image reference set, and whose style is daytime;
  • mapping relationship is used to indicate the image correspondence relationship between the daytime content image set and the night image reference set

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

一种图像风格迁移方法，该图像风格迁移方法包括：获取日间内容图像集和夜间图像参考集（101）；根据夜间图像参考集预先设置辅助图像参考集（102）；根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系（103）；根据第一映射关系进行计算，得到第二映射关系（104）；根据第二映射关系对日间内容图像集和夜间图像参考集进行风格迁移学习，得到目标图像集，目标图像集为已标注的夜间图像训练样本（105）。该方法以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，将已标注的日间真实图像集转换为目标图像集，将目标图像集作为语义分割的训练样本，提高夜间图像语义分割的准确性。

Description

图像风格迁移方法、装置、设备及存储介质
本申请要求于2019年10月18日提交中国专利局、申请号为201910990747.9、发明名称为“图像风格迁移方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在申请中。
技术领域
本申请涉及图像处理领域，尤其涉及图像风格迁移方法、装置、设备及存储介质。
背景技术
随着科技技术迅速发展，在深度学习研究领域，使用卷积神经网络将一张图片的语义内容与不同风格融合起来的过程被称为神经风格迁移，将具有艺术作品上的艺术风格转移到日常照片上，成为在学术界和工业界中非常受重视的计算机视觉任务。
图像风格迁移的目的是要对图像的纹理、色彩、内容等进行定向的改变，使得图像由一种风格变化为另外一种风格，例如，将人的照片进行风格迁移，得到具有油画风格的图像，或者将光线较昏暗条件下拍摄得到的风景照片进行风格迁移，得到在光线较明亮条件下的图像。因此，对于夜间场景语义标注已成为棘手可热的研究方向。
图像语义标注作为图像场景理解的核心之一，已成为图像处理与计算机视觉领域的研究热点。发明人意识到目前解决语义分割的主要方法是使用大量注释来训练深度神经网络，这一监督学习方案在日间光照条件良好的图像上取得了成功，但对于其他光照条件不利的环境下，可扩展性很差，因此不能满足许多户外应用全天候的视觉识别需求，例如，在夜间和恶劣天气下，采集的车牌图像质量差，不能作为有价值的训练样本。图像分割是图像识别和计算机视觉至关重要的预处理，没有正确的分割就不可能有正确的识别。图像分割仅有的依据是图像中像素的亮度及颜色，使得计算机自动处理分割时，遇到各种困难。例如，光照不均匀、噪声的影响、图像中存在不清晰的部分以及阴影等，常常发生分割错误，不能对业务中的车辆、车牌以及道路进行有效识别。
发明内容
本申请的主要目的在于解决了在光照条件不利环境下采集的图像训练样本标注不准确，可扩展性差的技术问题。
为实现上述目的，本申请第一方面提供了一种图像风格迁移方法，包括：获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹 配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
本申请第二方面提供了一种图像风格迁移装置，包括：获取单元，用于获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；设置单元，用于根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；匹配单元，用于根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；计算单元，用于根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；风格迁移单元，用于根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
本申请第三方面提供了一种图像风格迁移设备，包括：存储器和至少一个处理器，所述存储器中存储有指令，所述存储器和所述至少一个处理器通过线路互联；所述至少一个处理器调用所述存储器中的所述指令，以使得所述图像风格迁移设备执行上述第一方面所述的方法。
本申请的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有指令，当其在计算机上运行时，使得计算机执行上述第一方面所述的方法。
本申请提供的技术方案中，获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。本申请实施例中， 通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，并将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
附图说明
图1为本申请实施例中图像风格迁移方法的一个实施例示意图；
图2为本申请实施例中图像风格迁移方法的另一个实施例示意图；
图3为本申请实施例中图像风格迁移装置的一个实施例示意图；
图4为本申请实施例中图像风格迁移装置的另一个实施例示意图；
图5为本申请实施例中图像风格迁移设备的一个实施例示意图。
具体实施方式
本申请实施例提供了一种图像风格迁移方法、装置、设备及存储介质，用于通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，并将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例进行描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”或“具有”及其任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
为便于理解，下面对本申请实施例的具体流程进行描述，请参阅图1，本申请实施例中图像风格迁移方法的一个实施例包括：
101、获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集；
服务器获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集。具体的，服务器获取已标注的日间内容图像集I＝{I i:i＝1,2...,M}和夜间图像参考集R＝{R r:r＝1,2...,N}，其中，M和N相等并且均为正整数,日间内容图像集中已标注的语义特征包括车辆、车牌和道路，具体此处不做限定。
需要说明的是，预置业务包括定损业务、车辆理赔业务，具体此处不做限 定。日间内容图像集为在白天场景下拍摄的一组照片，夜间图像参考集为在夜晚场景下拍摄的一组照片，日间内容图像集包含的图像个数与夜间图像参考集的个数是相等的，但是场景顺序并不是一一对应的关系。
102、根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集；
服务器根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集，其中，预置图像集是指根据夜间图像参考集的场景预先设置的辅助图像参考集，服务器根据夜间图像参考集的场景预设辅助图像参考集的场景，服务器根据夜间图像参考集的每一张图像的唯一标识和辅助图像参考集的唯一标识进行映射，得到预置映射关系。例如，对于辅助图像参考集R'＝{R' k:k＝1,2...,K}，K为正整数，与夜间图像参考集R具有相同的场景，也就是K与N相等，得到预置映射关系为B，其中，B:{1,2...,K}→{1,2...,N}，也就是B为同一场景夜间图像集的映射。通过局部化基础事实，R' k粗略地描绘了R B(k)中的相同图像，服务器将辅助图像参考集作为日间图像集和夜间图像集映射的桥梁。
可以理解的是，风格迁移是以相似的语义内容去跨越两个图像，服务器使用具有类似高级特征的配对图像对能够获取更准确的输出，其中，相似的语义内容如，乌鸦和燕子、汽车和公交车。由于夜间图像参考集存在大量语义类别、互相遮挡、低层视觉特征辨识力较弱以及不均匀光照等问题，若服务器直接根据日间内容图像集和夜间图像参考集进行特征提取和匹配，会导致有些特征识别不准确，因此服务器选定一个辅助图像参考集便于服务器进行特征提取与匹配。
103、根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系；
服务器根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系。具体的，服务器根据预置算法对日间内容图像集中的第一图像进行特征提取，得到第一特征；服务器根据预置算法对辅助图像参考集中的第二图像进行特征提取，得到第二特征，其中，第一特征和第二特征为卷积层数更深的特征；服务器根据第一特征和第二特征分别计算日间内容图像集中每个第一图像与辅助图像参考集中多个第二图像的多个相似度，并将日间内容图像集中每个第一图像与辅助图像参考集中多个第二图像的多个相似度的最大值设置为最大相似度，最大相似度用于指示依据相似度的最大值确定所述日间内容图像集与所述辅助图像参考集的图像匹配对，进一步地，服务器根据图像匹配对确定第一映射关系，因此，第一映射关系就是基于两个图像的相似度匹配计算得到的。
104、根据第一映射关系进行计算，得到第二映射关系，第二映射关系用于指示日间内容图像集和夜间图像参考集之间的图像对应关系；
服务器根据第一映射关系进行计算，得到第二映射关系，第二映射关系用于指示日间内容图像集和夜间图像参考集之间的图像对应关系。例如，对于日间内容图像集I＝{I i:i＝1,2,3,4,5}、夜间图像参考集R＝{R r:r＝1,2,3,4,5}和图像辅助参考集R'＝{R' k:k＝1,2,3,4,5}，服务器确认图像辅助参考集到夜间图像参考集的预置映射关系为B:{1,2,3,4,5}→{5,4,3,2,1}，服务器确认日间内容图像集至图像辅助参考集的第一映射关系为A':{1,2,3,4,5}→{5,4,3,2,1}，则服务器得到日间内容图像集和夜间图像参考集的第二映射关系为A:{1,2,3,4,5}→{1,2,3,4,5}。
105、根据第二映射关系对日间内容图像集和夜间图像参考集进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
服务器根据第二映射关系对日间内容图像集和夜间图像参考集进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。服务器将夜间图像参考集中每张图像的风格迁移至日间内容图像集中相对应的每张日间图像的内容上，得到目标图像集，目标图像集为已标注的夜间图像训练样本，例如，针对一张包括白天场景下车牌的日间内容图像，车牌为ABCD，车牌的颜色为亮蓝色，相对应的一张包括夜间场景的另一个车牌为12？？的夜间图像，该车牌的后两位比较模糊，车牌背景颜色为黑灰色，服务器根据训练好的模型进行图像风格迁移后，得到目标夜间图像为背景颜色是黑灰色的车牌ABCD。可以理解的是，目标夜间图像集包括日间图像集的内容和夜间图像集的风格，同时目标图像集的内容为已标注的内容，则避免重新标注的工作量。
本申请实施例中，通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
请参阅图2，本申请实施例中图像风格迁移方法的另一个实施例包括：
201、获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集；
服务器获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集。其中，预置业务包括定损业务、车辆理赔业务，具体此处不做限定。日间内容图像集为在白天场景下拍摄的一组照片，夜间图像参考集为在夜晚场景下拍摄的一组照片，日间内容图像集包含的图像个数与夜间图像参考集的个数是相等的，但是场景顺序并不是一一对应的关系。
具体的，服务器根据日间内容图像集的数量、夜间图像参考集的数量和结 构化查询语言SQL语言规则生成第一查询语句；服务器执行第一查询语句，得到日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集。例如，服务器获取已标注的日间内容图像集I＝{I i:i＝1,2...,M}和夜间图像参考集R＝{R r:r＝1,2...,N}，其中，M和N相等并且均为正整数,日间内容图像集中已标注的语义特征包括车辆、车牌和道路，具体此处不做限定。
202、根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集；
服务器根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集。其中，预置图像集是指根据夜间图像参考集的场景预先设置辅助图像参考集，并根据夜间图像参考集的每一张图像的唯一标识和辅助图像参考集的唯一标识进行映射，得到预置映射关系。例如，对于辅助图像参考集R'＝{R' k:k＝1,2...,K}，K为正整数，与夜间图像参考集R具有相同的场景，也就是K与N相等，得到预置映射关系为B，其中，B:{1,2...,K}→{1,2...,N}，也就是B为同一场景夜间图像集的映射。通过局部化基础事实，R' k粗略地描绘了R B(k)中的相同图像，服务器将辅助图像参考集作为日间图像集和夜间图像集映射的桥梁。
具体的，服务器从预置数据表中读取预置映射关系，预置映射关系用于指示根据夜间图像参考集的唯一标识和辅助图像参考集的唯一标识预先设置映射关系；服务器根据预置映射关系和SQL语言规则生成第二查询语句；服务器执行第二查询语句，得到辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集。例如，对于夜间图像参考集中的一张夜间图像N，N的场景地点为c，则辅助参考集中与B相对应的图像是一张包括地点为c的日间图像。
需要说明的是，风格迁移是以相似的语义内容去跨越两个图像，服务器使用具有类似高级特征的配对图像对能够获取更准确的输出，其中，相似的语义内容如，乌鸦和燕子、汽车和公交车等。由于夜间图像参考集存在大量语义类别、互相遮挡、低层视觉特征辨识力较弱以及不均匀光照等问题，若服务器直接根据日间内容图像集和夜间图像参考集进行特征提取和匹配，有些特征识别不准确，因此服务器选定一个辅助图像参考集便于服务器进行特征提取与匹配。
203、根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，并将第一映射关系存储到预置映射数据表中，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系；
服务器根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，并将第一映射关系存储到预置映射数据表中，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系。具体的， 服务器根据预置算法对日间内容图像集中的第一图像进行特征提取，得到第一特征；服务器根据预置算法对辅助图像参考集中的第二图像进行特征提取，得到第二特征，可选的，服务器采用尺度不变特征变换算法作为预置算法对第一图像和第二图像进行特征提取；服务器根据第一特征和第二特征分别计算每个第一图像与多个第二图像之间的多个相似度，得到最大相似度；服务器根据最大相似度确定日间内容图像集和辅助图像参考集之间的图像匹配关系，得到第一映射关系，并将第一映射关系存储到预置映射数据表中。进一步地，以上过程为
Figure PCTCN2019119118-appb-000001
其中，F(I i)用于指示对日间内容图像集中的图像进行特征提取，F(R' k)用于指示对辅助图像参考集中的图像进行特征提取，ρ用于表示相似度。
可以理解的是，服务器对日间内容图像集和辅助图像参考集进行计算，只要两张图片的特征中，相同区域的大部分特征能找到彼此相似的点就可以认为两张图片是相似的。进一步地，两张图像同一区域的特征块中大部分是相似的，就确认两张图像是相似的。可选的，通过计算特征之间的余弦距离衡量两个特征的相似度。服务器根据最大相似度确定第一映射关系。例如，服务器对日间内容图像集I和辅助图像参考集R'进行处理后，得到第一映射关系A':{1,2...,M}→{1,2...,K}，其中，M与K相等。
204、从预置映射数据表中读取预置映射关系和第一映射关系；
服务器从预置映射数据表中读取预置映射关系和第一映射关系。具体的，服务器根据预置映射数据表和SQL语法规则生成第三查询语句；服务器执行第三查询语句，得到查询结果，该查询结果包括预置映射关系和第一映射关系。
需要说明的是，结构化查询语言(structured query language，SQL)是一种数据库查询和程序设计语言，用于存取数据、查询数据、更新数据和管理关系数据库系统。
205、根据预置映射关系和第一映射关系进行矩阵乘法计算，得到第二映射关系，第二映射关系用于指示日间内容图像集与夜间图像参考集的配对图像；
服务器根据预置映射关系和第一映射关系进行矩阵乘法计算，得到第二映射关系，第二映射关系用于指示日间内容图像集与夜间图像参考集的配对图像。其中，该预置映射公式为A＝BΟA'，其中，B为预置映射关系，A'为第一映射关系，A为第二映射关系。例如，服务器根据预置映射关系B:{1,2,...,K}→{1,2,...,N}和第一映射关系
Figure PCTCN2019119118-appb-000002
得到第二映射关系为A:{1,2,...,M}→{1,2,...,N}。
需要说明的是，第二映射关系采用日间内容图像集与夜间图像参考集的配对图像的唯一标识进行关系映射，也就是说，1、2、……M为日间内容图像的唯一标识，1、2、……N为夜间图像的唯一标识。
206、根据第二映射关系确定日间内容图像集与夜间图像参考集之间的配对图像，配对图像包括日间内容图像和夜间图像；
服务器根据第二映射关系确定日间内容图像集与夜间图像参考集之间的配对图像，配对图像包括日间内容图像和夜间图像。具体的，服务器对第二映射关系进行解析，得到配对图像的映射标识，配对图像的映射标识包括日间内容图像的唯一标识和夜间图像的唯一标识；服务器根据配对图像的映射标识读取日间内容图像集与夜间图像参考集之间的配对图像。例如，服务器获取第二映射关系为A:{1,2,...,M}→{1,2,...,N}，服务器将第二映射关系解析后，得到日间内容图像的唯一标识为1、2、……M，夜间图像的唯一标识为1、2、……N，其中，1、2、……M与1、2、……N构成配对图像的映射标识，则日间内容图像的唯一标识1与夜间图像的唯一标识1互为映射关系，日间内容图像的唯一标识2与夜间图像的唯一标识3互为映射关系，以此类推，日间内容图像的唯一标识M与夜间图像的唯一标识N互为映射关系。
207、通过预置深度卷积神经网络模型对配对图像按照对应关系进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
服务器通过预置深度卷积神经网络模型对配对图像按照对应关系进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。具体的，服务器确定日间内容图像的尺寸，并根据日间内容图像的尺寸生成高斯白噪声图像；服务器将日间内容图像、夜间图像和高斯白噪声图像输入到预置深度卷积神经网络模型中，预置深度卷积神经网络模型包括内容约束特征提取层和风格约束特征提取层；服务器在内容约束特征提取层对日间内容图像和高斯白噪声图像进行计算，得到内容约束层损失函数；服务器在风格约束特征提取层对夜间图像和高斯白噪声图像进行计算，得到风格约束层损失函数；服务器对内容约束层损失函数和风格约束层损失函数进行累加计算，得到总的损失函数；服务器采用梯度下降算法迭代更新总的损失函数，得到目标图像集，目标图像集为已标注的夜间图像训练样本。目标图像集为接近真实图像的模拟图像，提高了预置业务中夜间图像训练样本的质量，例如，在车险理赔业务中，将已标注车辆和道路的目标夜间图像集作为夜间图像训练样本，根据夜间图像训练样本对语义分割模型进行训练，并通过训练好的语义分割模型对待识别的车辆图像进行语义分割，得到准确的车辆和车牌分割结果。
可以理解的是，该目标图像集包括日间内容图像集的内容和夜间图像集的风格例如，针对一张包括白天场景下车牌的日间内容图像，车牌为ABCD，车牌的颜色为亮蓝色，相对应的一张包括夜间场景的另一个车牌为12？？的夜间图像，该车牌的后两位比较模糊，车牌背景颜色为黑灰色，服务器根据训练好的模型进行图像风格迁移后，得到目标夜间图像为背景颜色是黑灰色的车牌ABCD。进一步地，该目标夜间图像集用于语义分割训练，避免重新标注的工作量。
需要说明的是，预置深度卷积神经网络模型可采用VGG-19，VGG-19模型是指19层的超分辨率测试序列(visual geometry group，VGG)网络，包含16个卷积层和5个池化层提供的特征空间。按比例改变权重规范化网络，每层卷积滤波器在图像和位置上平均激活值就等于1。
本申请实施例中，通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，并将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
上面对本申请实施例中图像风格迁移方法进行了描述，下面对本申请实施例中图像风格迁移装置进行描述，请参阅图3，本申请实施例中图像风格迁移装置的一个实施例包括：
获取单元301，用于获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集；
设置单元302，用于根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集；
匹配单元303，用于根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系；
计算单元304，用于根据第一映射关系进行计算，得到第二映射关系，第二映射关系用于指示日间内容图像集和夜间图像参考集之间的图像对应关系；
风格迁移单元305，用于根据第二映射关系对日间内容图像集和夜间图像参考集进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
本申请实施例中，通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，并将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
请参阅图4，本申请实施例中图像风格迁移装置的另一个实施例包括：
获取单元301，用于获取日间内容图像集和夜间图像参考集，日间内容图像集的数量与夜间图像参考集的数量相等，日间内容图像集为根据预置业务采集并标注的真实图像集；
设置单元302，用于根据夜间图像参考集预先设置辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像 集；
匹配单元303，用于根据预置算法对日间内容图像集和辅助图像参考集进行特征匹配，得到第一映射关系，第一映射关系用于指示日间内容图像集和辅助图像参考集之间的图像对应关系；
计算单元304，用于根据第一映射关系进行计算，得到第二映射关系，第二映射关系用于指示日间内容图像集和夜间图像参考集之间的图像对应关系；
风格迁移单元305，用于根据第二映射关系对日间内容图像集和夜间图像参考集进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
可选的，匹配单元303还可以具体用于：
根据预置算法对日间内容图像集中的第一图像进行特征提取，得到第一特征；
根据预置算法对辅助图像参考集中的第二图像进行特征提取，得到第二特征；
根据第一特征和第二特征分别计算每个第一图像与多个第二图像之间的多个相似度，得到最大相似度；
根据最大相似度计算确定日间内容图像集和辅助图像参考集之间的图像匹配关系，得到第一映射关系，并将第一映射关系存储到预置映射数据表中。
可选的，计算单元304还可以具体用于：
从预置映射数据表中读取预置映射关系和第一映射关系；
根据预置映射关系和第一映射关系进行矩阵乘法计算，得到第二映射关系，第二映射关系用于指示日间内容图像集与夜间图像参考集之间的图像对应关系。
可选的，迁移学习单元305还可以进一步包括：
确定子单元3051，用于根据第二映射关系确定日间内容图像集与夜间图像参考集之间的配对图像，配对图像包括日间内容图像和夜间图像；
风格迁移子单元3052，用于通过预置深度卷积神经网络模型对配对图像按照对应关系进行风格迁移，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
可选的，风格迁移子单元3052还可以具体用于：
确定日间内容图像的尺寸，并根据日间内容图像的尺寸生成高斯白噪声图像；
将日间内容图像、夜间图像和高斯白噪声图像输入到预置深度卷积神经网络模型中，预置深度卷积神经网络模型包括内容约束特征提取层和风格约束特征提取层；
在内容约束特征提取层对日间内容图像和高斯白噪声图像进行计算，得到内容约束层损失函数；
在风格约束特征提取层对夜间图像和高斯白噪声图像进行计算，得到风格约束层损失函数；
对内容约束层损失函数和风格约束层损失函数进行累加计算，得到总的损失函数；
采用梯度下降算法迭代更新总的损失函数，得到目标图像集，目标图像集为已标注的夜间图像训练样本。
可选的，获取单元301还可以具体用于：
根据日间内容图像集的数量、夜间图像参考集的数量和结构化查询语言SQL语言规则生成第一查询语句；
执行第一查询语句，得到日间内容图像集和夜间图像参考集，日间内容图像集为根据预置业务采集并标注的真实图像集。
可选的，设置单元302还可以具体用于：
从预置映射数据表中读取预置映射关系，预置映射关系用于指示根据夜间图像参考集预先设置辅助图像参考集；
根据预置映射关系和SQL语言规则生成第二查询语句；
执行第二查询语句，得到辅助图像参考集，辅助图像参考集为场景与夜间图像参考集的场景一致，并且风格为日间的预置图像集。
本申请实施例中，通过以辅助图像参考集作为日间内容图像集和夜间图像参考集的配对桥梁，便于对日间内容图像集和夜间图像参考集的特征提取和匹配，将已标注的日间真实图像转换为合成的夜间图像，并将合成的夜间图像作为语义分割的训练样本，避免重新标注的工作量，提高夜间图像语义分割的准确性和可扩展性。
上面图3和图4从模块化功能实体的角度对本申请实施例中的图像风格迁移装置进行详细描述，下面从硬件处理的角度对本申请实施例中图像风格迁移设备进行详细描述。
图5是本申请实施例提供的一种图像风格迁移设备的结构示意图，该图像风格迁移设备500可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器(central processing units，CPU)501(例如，一个或一个以上处理器)和存储器509，一个或一个以上存储应用程序507或数据506的存储介质508(例如一个或一个以上海量存储设备)。其中，存储器509和存储介质508可以是短暂存储或持久存储。存储在存储介质508的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对图像风格迁移设备中的一系列指令操作。更进一步地，处理器501可以设置为与存储介质508通信，在图像风格迁移设备500上执行存储介质508中的一系列指令操作。
图像风格迁移设备500还可以包括一个或一个以上电源502，一个或一个以上有线或无线网络接口503，一个或一个以上输入输出接口504，和/或，一个或一个以上操作系统505，例如Windows Serve，Mac OS X，Unix，Linux， FreeBSD等等。本领域技术人员可以理解，图5中示出的图像风格迁移设备结构并不构成对图像风格迁移设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
本申请还提供一种计算机可读存储介质，该计算机可读存储介质可以为非易失性计算机可读存储介质，也可以为易失性计算机可读存储介质。计算机可读存储介质存储有计算机指令，当所述计算机指令在计算机上运行时，使得计算机执行如下步骤：
获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；
根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；
根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；
根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；
根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种图像风格迁移方法，包括：
    获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；
    根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；
    根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；
    根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；
    根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  2. 根据权利要求1所述的图像风格迁移方法，所述根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系包括：
    根据预置算法对所述日间内容图像集中的第一图像进行特征提取，得到第一特征；
    根据所述预置算法对所述辅助图像参考集中的第二图像进行特征提取，得到第二特征；
    根据所述第一特征和所述第二特征分别计算每个第一图像与多个第二图像之间的多个相似度，得到最大相似度；
    根据所述最大相似度确定所述日间内容图像集和所述辅助图像参考集之间的图像匹配关系，得到第一映射关系，并将第一映射关系存储到预置映射数据表中。
  3. 根据权利要求2所述的图像风格迁移方法，所述根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系包括：
    从所述预置映射数据表中读取所述预置映射关系和所述第一映射关系；
    根据所述预置映射关系和所述第一映射关系进行矩阵乘法计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集与所述夜间图像参考集之间的图像对应关系。
  4. 根据权利要求1所述的图像风格迁移方法，所述根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本包括：
    根据所述第二映射关系确定所述日间内容图像集与所述夜间图像参考集之间的配对图像，所述配对图像包括日间内容图像和夜间图像；
    通过预置深度卷积神经网络模型对所述配对图像按照对应关系进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  5. 根据权利要求4所述的图像风格迁移方法，所述通过预置深度卷积神经网络模型对所述配对图像按照对应关系进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本包括：
    确定所述日间内容图像的尺寸，并根据所述日间内容图像的尺寸生成高斯白噪声图像；
    将所述日间内容图像、所述夜间图像和所述高斯白噪声图像输入到所述预置深度卷积神经网络模型中，所述预置深度卷积神经网络模型包括内容约束特征提取层和风格约束特征提取层；
    在所述内容约束特征提取层对所述日间内容图像和所述高斯白噪声图像进行计算，得到内容约束层损失函数；
    在所述风格约束特征提取层对所述夜间图像和所述高斯白噪声图像进行计算，得到风格约束层损失函数；
    对所述内容约束层损失函数和所述风格约束层损失函数进行累加计算，得到总的损失函数；
    采用梯度下降算法迭代更新所述总的损失函数，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  6. 根据权利要求1所述的图像风格迁移方法，所述获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集包括：
    根据所述日间内容图像集的数量、所述夜间图像参考集的数量和结构化查询语言SQL语言规则生成第一查询语句；
    执行所述第一查询语句，得到所述日间内容图像集和所述夜间图像参考集，所述日间内容图像集为根据预置业务采集并标注的真实图像集。
  7. 根据权利要求3或者6所述的图像风格迁移方法，所述根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集包括：
    从所述预置映射数据表中读取所述预置映射关系，所述预置映射关系用于 指示根据所述夜间图像参考集的唯一标识和辅助图像参考集的唯一标识预先设置映射关系；
    根据所述预置映射关系和所述SQL语言规则生成第二查询语句；
    执行所述第二查询语句，得到所述辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集。
  8. 一种图像风格迁移装置，所述图像风格迁移装置包括：
    获取单元，用于获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；
    设置单元，用于根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；
    匹配单元，用于根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；
    计算单元，用于根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；
    风格迁移单元，用于根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  9. 根据权利要求8所述的图像风格迁移装置，所述匹配单元具体用于：
    根据预置算法对所述日间内容图像集中的第一图像进行特征提取，得到第一特征；
    根据所述预置算法对所述辅助图像参考集中的第二图像进行特征提取，得到第二特征；
    根据所述第一特征和所述第二特征分别计算每个第一图像与多个第二图像之间的多个相似度，得到最大相似度；
    根据所述最大相似度确定所述日间内容图像集和所述辅助图像参考集之间的图像匹配关系，得到第一映射关系，并将第一映射关系存储到预置映射数据表中。
  10. 根据权利要求9所述的图像风格迁移装置，所述计算单元具体用于：
    从所述预置映射数据表中读取所述预置映射关系和所述第一映射关系；
    根据所述预置映射关系和所述第一映射关系进行矩阵乘法计算，得到第二 映射关系，所述第二映射关系用于指示所述日间内容图像集与所述夜间图像参考集之间的图像对应关系。
  11. 根据权利要求8所述的图像风格迁移装置，所述风格迁移单元包括：
    确定子单元，用于根据所述第二映射关系确定所述日间内容图像集与所述夜间图像参考集之间的配对图像，所述配对图像包括日间内容图像和夜间图像；
    风格迁移子单元，用于通过预置深度卷积神经网络模型对所述配对图像按照对应关系进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  12. 根据权利要求11所述的图像风格迁移装置，所述风格迁移子单元具体用于：
    确定所述日间内容图像的尺寸，并根据所述日间内容图像的尺寸生成高斯白噪声图像；
    将所述日间内容图像、所述夜间图像和所述高斯白噪声图像输入到所述预置深度卷积神经网络模型中，所述预置深度卷积神经网络模型包括内容约束特征提取层和风格约束特征提取层；
    在所述内容约束特征提取层对所述日间内容图像和所述高斯白噪声图像进行计算，得到内容约束层损失函数；
    在所述风格约束特征提取层对所述夜间图像和所述高斯白噪声图像进行计算，得到风格约束层损失函数；
    对所述内容约束层损失函数和所述风格约束层损失函数进行累加计算，得到总的损失函数；
    采用梯度下降算法迭代更新所述总的损失函数，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  13. 根据权利要求8所述的图像风格迁移装置，所述获取单元具体用于：
    根据所述日间内容图像集的数量、所述夜间图像参考集的数量和结构化查询语言SQL语言规则生成第一查询语句；
    执行所述第一查询语句，得到所述日间内容图像集和所述夜间图像参考集，所述日间内容图像集为根据预置业务采集并标注的真实图像集。
  14. 根据权利要求10或者13所述的图像风格迁移装置，所述设置单元具体用于：
    根据所述预置映射关系和所述SQL语言规则生成第二查询语句；
    执行所述第二查询语句，得到所述辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集。
  15. 一种图像风格迁移设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机可读指令，所述处理器执行所述计算机可读指令时实现如下步骤：
    获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；
    根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；
    根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；
    根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；
    根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  16. 根据权利要求15所述的图像风格迁移设备，所述处理器执行所述计算机可读指令实现所述根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系时，包括以下步骤：
    根据预置算法对所述日间内容图像集中的第一图像进行特征提取，得到第一特征；
    根据所述预置算法对所述辅助图像参考集中的第二图像进行特征提取，得到第二特征；
    根据所述第一特征和所述第二特征分别计算每个第一图像与多个第二图像之间的多个相似度，得到最大相似度；
    根据所述最大相似度确定所述日间内容图像集和所述辅助图像参考集之间的图像匹配关系，得到第一映射关系，并将第一映射关系存储到预置映射数据表中。
  17. 根据权利要求16所述的图像风格迁移设备，所述处理器执行所述计算机可读指令实现所述根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系时，包括以下步骤：
    从所述预置映射数据表中读取所述预置映射关系和所述第一映射关系；
    根据所述预置映射关系和所述第一映射关系进行矩阵乘法计算，得到第二 映射关系，所述第二映射关系用于指示所述日间内容图像集与所述夜间图像参考集之间的图像对应关系。
  18. 根据权利要求15所述的图像风格迁移设备，所述处理器执行所述计算机可读指令实现所述根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本时，包括以下步骤：
    根据所述第二映射关系确定所述日间内容图像集与所述夜间图像参考集之间的配对图像，所述配对图像包括日间内容图像和夜间图像；
    通过预置深度卷积神经网络模型对所述配对图像按照对应关系进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  19. 根据权利要求18所述的图像风格迁移设备，所述处理器执行所述计算机可读指令实现所述通过预置深度卷积神经网络模型对所述配对图像按照对应关系进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本时，包括以下步骤：
    确定所述日间内容图像的尺寸，并根据所述日间内容图像的尺寸生成高斯白噪声图像；
    将所述日间内容图像、所述夜间图像和所述高斯白噪声图像输入到所述预置深度卷积神经网络模型中，所述预置深度卷积神经网络模型包括内容约束特征提取层和风格约束特征提取层；
    在所述内容约束特征提取层对所述日间内容图像和所述高斯白噪声图像进行计算，得到内容约束层损失函数；
    在所述风格约束特征提取层对所述夜间图像和所述高斯白噪声图像进行计算，得到风格约束层损失函数；
    对所述内容约束层损失函数和所述风格约束层损失函数进行累加计算，得到总的损失函数；
    采用梯度下降算法迭代更新所述总的损失函数，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
  20. 一种计算机可读存储介质，所述计算机可读存储介质中存储计算机指令，当所述计算机指令在计算机上运行时，使得计算机执行如下步骤：
    获取日间内容图像集和夜间图像参考集，所述日间内容图像集的数量与所述夜间图像参考集的数量相等，所述日间内容图像集为根据预置业务采集并标注的真实图像集；
    根据所述夜间图像参考集预先设置辅助图像参考集，所述辅助图像参考集为场景与所述夜间图像参考集的场景一致，并且风格为日间的预置图像集；
    根据预置算法对所述日间内容图像集和所述辅助图像参考集进行特征匹配，得到第一映射关系，所述第一映射关系用于指示所述日间内容图像集和所述辅助图像参考集之间的图像对应关系；
    根据所述第一映射关系进行计算，得到第二映射关系，所述第二映射关系用于指示所述日间内容图像集和所述夜间图像参考集之间的图像对应关系；
    根据所述第二映射关系对所述日间内容图像集和所述夜间图像参考集进行风格迁移，得到目标图像集，所述目标图像集为已标注的夜间图像训练样本。
PCT/CN2019/119118 2019-10-18 2019-11-18 图像风格迁移方法、装置、设备及存储介质 WO2021072886A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910990747.9 2019-10-18
CN201910990747.9A CN110880016B (zh) 2019-10-18 2019-10-18 图像风格迁移方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021072886A1 true WO2021072886A1 (zh) 2021-04-22

Family

ID=69727968

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/119118 WO2021072886A1 (zh) 2019-10-18 2019-11-18 图像风格迁移方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN110880016B (zh)
WO (1) WO2021072886A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113837133A (zh) * 2021-09-29 2021-12-24 维沃移动通信有限公司 摄像头数据迁移方法及其装置
CN114463992A (zh) * 2022-02-11 2022-05-10 超级视线科技有限公司 夜间路侧停车管理视频转换方法以及装置
CN114972749A (zh) * 2022-04-28 2022-08-30 北京地平线信息技术有限公司 用于处理语义分割模型的方法、装置、介质和设备
CN115588070A (zh) * 2022-12-12 2023-01-10 南方科技大学 一种三维图像风格化迁移方法及终端

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111639525A (zh) * 2020-04-22 2020-09-08 上海擎感智能科技有限公司 一种感知算法的训练方法、装置及计算机存储介质
CN111986302A (zh) * 2020-07-23 2020-11-24 北京石油化工学院 一种基于深度学习的图像风格迁移方法及装置
CN111913863B (zh) * 2020-08-07 2023-10-17 北京达佳互联信息技术有限公司 统计模型建立方法、夜间模式页面生成方法、装置及设备
CN112634282B (zh) * 2020-12-18 2024-02-13 北京百度网讯科技有限公司 图像处理方法、装置以及电子设备
CN113723457A (zh) * 2021-07-28 2021-11-30 浙江大华技术股份有限公司 图像识别方法和装置、存储介质及电子装置
CN114511488B (zh) * 2022-02-19 2024-02-27 西北工业大学 一种夜间场景的日间风格可视化方法
US20240177456A1 (en) * 2022-11-24 2024-05-30 Industrial Technology Research Institute Object detection method for detecting one or more objects using a plurality of deep convolution neural network layers and object detection apparatus using the same method and non-transitory storage medium thereof

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180373999A1 (en) * 2017-06-26 2018-12-27 Konica Minolta Laboratory U.S.A., Inc. Targeted data augmentation using neural style transfer
US20190244329A1 (en) * 2018-02-02 2019-08-08 Nvidia Corporation Photorealistic Image Stylization Using a Neural Network Model

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109508580B (zh) * 2017-09-15 2022-02-25 阿波罗智能技术(北京)有限公司 交通信号灯识别方法和装置
US10467820B2 (en) * 2018-01-24 2019-11-05 Google Llc Image style transfer for three-dimensional models
CN108596830B (zh) * 2018-04-28 2022-04-22 国信优易数据股份有限公司 一种图像风格迁移模型训练方法以及图像风格迁移方法
CN109919829B (zh) * 2019-01-17 2023-12-26 北京达佳互联信息技术有限公司 图像风格迁移方法、装置和计算机可读存储介质
CN110310222A (zh) * 2019-06-20 2019-10-08 北京奇艺世纪科技有限公司 一种图像风格迁移方法、装置、电子设备及存储介质

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180373999A1 (en) * 2017-06-26 2018-12-27 Konica Minolta Laboratory U.S.A., Inc. Targeted data augmentation using neural style transfer
US20190244329A1 (en) * 2018-02-02 2019-08-08 Nvidia Corporation Photorealistic Image Stylization Using a Neural Network Model

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LIU DONGJING: "Research on Image Style Transfer Based on Semantic Matching and Style Sampling", INFORMATION SCIENCE AND TECHNOLOGY, CHINESE MASTER’S THESES FULL-TEXT DATABASE, 1 January 2019 (2019-01-01), XP055801187 *
RUDER MANUEL; DOSOVITSKIY ALEXEY; BROX THOMAS: "Artistic Style Transfer for Videos and Spherical Images", INTERNATIONAL JOURNAL OF COMPUTER VISION., KLUWER ACADEMIC PUBLISHERS, NORWELL., US, vol. 126, no. 11, 21 April 2018 (2018-04-21), US, pages 1199 - 1219, XP036589840, ISSN: 0920-5691, DOI: 10.1007/s11263-018-1089-z *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113837133A (zh) * 2021-09-29 2021-12-24 维沃移动通信有限公司 摄像头数据迁移方法及其装置
CN114463992A (zh) * 2022-02-11 2022-05-10 超级视线科技有限公司 夜间路侧停车管理视频转换方法以及装置
CN114972749A (zh) * 2022-04-28 2022-08-30 北京地平线信息技术有限公司 用于处理语义分割模型的方法、装置、介质和设备
CN114972749B (zh) * 2022-04-28 2024-03-19 北京地平线信息技术有限公司 用于处理语义分割模型的方法、装置、介质和设备
CN115588070A (zh) * 2022-12-12 2023-01-10 南方科技大学 一种三维图像风格化迁移方法及终端
CN115588070B (zh) * 2022-12-12 2023-03-14 南方科技大学 一种三维图像风格化迁移方法及终端

Also Published As

Publication number Publication date
CN110880016B (zh) 2022-07-15
CN110880016A (zh) 2020-03-13

Similar Documents

Publication Publication Date Title
WO2021072886A1 (zh) 图像风格迁移方法、装置、设备及存储介质
Chen et al. Feature detection and description for image matching: from hand-crafted design to deep learning
Raja et al. Color object detection based image retrieval using ROI segmentation with multi-feature method
US9858472B2 (en) Three-dimensional facial recognition method and system
CN104599275B (zh) 基于概率图模型的非参数化的rgb-d场景理解方法
US8892542B2 (en) Contextual weighting and efficient re-ranking for vocabulary tree based image retrieval
CN110662484A (zh) 用于全身测量结果提取的系统和方法
Chen et al. Large-scale structure from motion with semantic constraints of aerial images
CN108875602A (zh) 监控环境下基于深度学习的人脸识别方法
CN110866953A (zh) 地图构建方法及装置、定位方法及装置
CN110704712A (zh) 基于图像检索的场景图片拍摄位置范围识别方法及系统
US20230041943A1 (en) Method for automatically producing map data, and related apparatus
WO2020151148A1 (zh) 基于神经网络的黑白照片色彩恢复方法、装置及存储介质
CN110516707B (zh) 一种图像标注方法及其装置、存储介质
CN111695431A (zh) 一种人脸识别方法、装置、终端设备及存储介质
CN103886013A (zh) 一种基于网络视频监控中的智能图像检索系统
Guo et al. Image esthetic assessment using both hand-crafting and semantic features
CN113159043A (zh) 基于语义信息的特征点匹配方法及系统
CN111062260A (zh) 一种面部整容推荐方案自动生成方法
CN108875828A (zh) 一种相似图像的快速匹配方法和系统
CN110046669B (zh) 基于素描图像的半耦合度量鉴别字典学习的行人检索方法
Wang et al. Pedestrian detection in infrared image based on depth transfer learning
CN114579794A (zh) 特征一致性建议的多尺度融合地标图像检索方法及系统
CN114612612A (zh) 人体姿态估计方法及装置、计算机可读介质、电子设备
CN112836611A (zh) 确定身体部位语义图、模型训练和行人重识别方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19949302

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19949302

Country of ref document: EP

Kind code of ref document: A1