CN113688746A - Outdoor advertisement picture recognition system and method based on neural network - Google Patents
Outdoor advertisement picture recognition system and method based on neural network Download PDFInfo
- Publication number
- CN113688746A CN113688746A CN202110995155.3A CN202110995155A CN113688746A CN 113688746 A CN113688746 A CN 113688746A CN 202110995155 A CN202110995155 A CN 202110995155A CN 113688746 A CN113688746 A CN 113688746A
- Authority
- CN
- China
- Prior art keywords
- picture
- outdoor
- value
- image
- advertisement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 22
- 238000011161 development Methods 0.000 claims abstract description 42
- 238000012549 training Methods 0.000 claims description 36
- 238000011176 pooling Methods 0.000 claims description 21
- 238000010586 diagram Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000013461 design Methods 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000007689 inspection Methods 0.000 description 4
- 229910052754 neon Inorganic materials 0.000 description 2
- GKAOGPIIYCISHV-UHFFFAOYSA-N neon atom Chemical compound [Ne] GKAOGPIIYCISHV-UHFFFAOYSA-N 0.000 description 2
- 241001503987 Clematis vitalba Species 0.000 description 1
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention discloses an outdoor advertisement picture identification system and method based on a neural network, wherein the system comprises a main control development board, a GPS module, a relay module, a camera module, a transceiving module, a server and a client; the camera module is connected with the main control development board through the relay module, and transmits the picture image to the main control development board after collecting the outdoor advertisement picture image; the collected time is controlled by the relay module; the GPS module is connected with the main control development board and sends the position of the outdoor advertisement corresponding to the GPS module to the main control development board; the main control development board is connected with the transceiving module, the outdoor advertising picture image and the position data positioned by the GPS module are sent to the server through the transceiving module, and the server analyzes, processes and stores the received data; the server is installed in a remote central control room; the client is connected with the server. The invention can monitor the condition of outdoor advertisement pictures in real time and reduce the operation cost.
Description
Technical Field
The invention relates to the technical field of Internet of things, in particular to an outdoor advertisement picture identification system and method based on a neural network.
Background
The outdoor billboard is commonly used in outdoor scenes such as urban roads, expressways, building floors and the like, and is characterized in that the billboard is of a steel structure, the billboard is usually 18 meters or more away from the ground, and once the advertising picture is abnormal, a professional climber with high-altitude operation needs to repair the advertising picture or repair an advertising spot lamp;
because the billboard is in an outdoor environment, wind, rain and the billboard is easily affected by extreme weather (such as typhoon, thunderstorm and the like), if the advertising picture is damaged or a mechanical device fails, the advertising effect of an advertiser is often seriously affected, even in appointed advertising picture release time, malicious replacement and bad industry of repeated sale are easily caused, and further the trust between an advertising media company and the advertiser is reduced, and the problem of service doubt is caused; meanwhile, the outdoor billboard of the type has wide distribution area and long ground clearance, the cost of manual inspection and monitoring time is high, the accuracy is not high, and the advertisement picture is abnormal and can not be notified in time; in addition, the type of billboard is provided with a lamp tube or a neon lamp, whether the lamp is normally lighted on time or not can not be known in time by means of manual inspection, and the lighting time adjustment of the spot lamp requires manual on-site time control (at least twice per year), so that certain workload is brought to operation and maintenance, the operation cost is improved, and meanwhile, the working efficiency is reduced.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide an outdoor advertising picture recognition system based on a neural network, so that the condition of an outdoor advertising picture can be known without manual inspection, the working efficiency can be improved, and the operation cost can be reduced.
In order to achieve the purpose, the technical scheme provided by the invention is as follows:
an outdoor advertising picture recognition system and method based on neural network, including main control development board, GPS module, relay module, camera module, receiving and dispatching module, server and customer end;
wherein the content of the first and second substances,
the camera module is connected with the main control development board through the relay module, and transmits the picture image to the main control development board after collecting the outdoor advertisement picture image; the collected time is controlled by the relay module;
the GPS module is connected with the main control development board and sends the position of the outdoor advertisement corresponding to the GPS module to the main control development board;
the main control development board is connected with the transceiver module, the outdoor advertisement picture image and the position data positioned by the GPS module are sent to the server through the transceiver module, and the server analyzes, processes and stores the received data;
the server is installed in a remote central control room;
the client is connected with the server, and the outdoor advertising picture is remotely monitored through the server;
the camera module adopts a 1.6mm ultra-wide angle fisheye camera module;
the client is any one of a smart phone, a tablet computer, a notebook computer and a desktop computer.
The transceiver module adopts any one of 4G and 5G, NB-IoT;
and the main control development board processes the chip by adopting RK 3288.
In order to achieve the above object, the present invention further provides an outdoor advertisement picture recognition method based on a neural network, comprising the steps of:
s1, the main control development board carries out self-checking, the GPS module and the transceiver module are operated, the server receives the state codes of the modules, and online monitoring of the modules is realized;
s2, the client updates the positioning information of the billboard, and the user sets the snapshot time point, the snapshot interval time, the advertisement release time period of the target advertiser and the lighting time of the billboard;
s3, the master control development board acquires the current world time, judges whether to enter a preset image capture time point, if not, the relay module and the camera module continue to enter a normal standby working state, the GPS module reports the current longitude and latitude information to the server every one hour, if the uploaded data and the previous recorded data have deviation, the outdoor advertising board is judged to be displaced, and the client prompts abnormity; if the preset image snapshot time point is judged to be entered, the main control development board sends an instruction to start the relay module, the camera module is electrified to operate, and snapshot return of image data of the corresponding outdoor advertising picture is carried out;
s4, the snap-shot image is transmitted back to the main control development board, after the image data are completely received, the relay module is closed, and the next snap-shot time point is waited; the image data is sent to a server through a transceiver module;
and S5, finally, carrying out outdoor advertisement picture recognition based on the neural network model, and recognizing whether the outdoor advertisement picture is an outdoor advertisement picture, whether the outdoor advertisement picture is damaged, and whether a spotlight arranged on the billboard normally lights on time.
Further, when the outdoor advertising frame recognition based on the neural network is carried out, whether the outdoor advertising frame recognition model, the outdoor advertising frame breakage recognition model and the advertising spot lamp recognition model are constructed and trained to respectively recognize whether the outdoor advertising frame is an outdoor advertising frame, whether the outdoor advertising frame is broken and whether a spot lamp arranged on the advertising board normally lights on time.
Further, the specific process of training the outdoor advertisement frame recognition model is as follows:
s01, converting the initial image into gray scale, marking pixels, wherein 1 represents white and 0 represents black, extracting four positions from the initial image as the characteristics of the outdoor advertising range, and forming convolution kernels;
s02, graying the snap-shot image, marking the pixel characteristics with 1, 0, taking any element value in the convolution kernel, multiplying the element value by any pixel value of the snap-shot image, obtaining a new mark value when the result is the convolution value marked by the current pixel, and obtaining a new pixel value, also called a window, after averaging a plurality of values;
s03, after obtaining a window value, sliding the window to the right with the step length of 2, continuing to calculate a new characteristic pixel value, and after applying convolution kernel calculation to the snapshot image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s04, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the initial image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the initial image;
s05, performing multiple operations through a Softmax classification function, and adding probability values of all pixel regions to obtain an average, wherein the average is a similarity value of the snapshot image and the initial image;
s06, and circulating steps S01-S05 for multiple times, wherein image data captured each time are used as a training set for training an outdoor advertisement frame recognition model through convolution, average pooling, full connection network and full graph probability average until the outdoor advertisement frame recognition model converges.
Further, the specific process of training the outdoor advertising picture breakage recognition model is as follows:
s-1, collecting outdoor advertisement damaged picture images manually, and adding the picture images into a training library;
s-2, selecting and graying a first picture image from a training library, marking pixels, wherein 1 represents white and 0 represents black, and then extracting four positions from the first picture image as the characteristics of an outdoor advertisement range to serve as a convolution kernel;
s-3, selecting and graying a second picture image from a training library, marking the pixel characteristics by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value of the current pixel mark;
s-4, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and obtaining a brand new characteristic image after applying convolution kernel calculation to the second picture image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-5, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the first picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the first picture image;
s-6, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the second picture image and the first picture image;
and S-7, circulating the steps S-2 to S-6 for multiple times, and carrying out convolution-average pooling-full connection network-full map probability average until the outdoor advertising picture breakage identification model converges.
Further, the specific process of training the identification model of the advertisement spotlight is as follows:
s-01, obtaining outdoor advertisement picture images in batch, and adding the outdoor advertisement picture images into a training library;
s-02, selecting a first outdoor advertisement picture image from a training library, and manually marking coordinates of the positions of all advertisement spot lamps;
s-03, graying the marked first outdoor advertisement picture image, marking pixels, wherein 1 represents white and 0 represents black, and extracting characteristic values of position coordinates of all spot lights of the first outdoor advertisement picture image to serve as convolution kernels;
s-04, selecting a second outdoor advertisement picture image from the training library, marking a pixel characteristic by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value marked by the current pixel, and averaging a plurality of values to obtain a new pixel value which is also called a window;
s-05, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and after applying convolution kernel calculation to a second outdoor advertisement picture image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-06, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the second outdoor advertisement picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the second outdoor advertisement picture image;
s-07, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the first outdoor advertisement picture image and the second outdoor advertisement picture image;
and S-08, repeating the steps from S-02 to S-07 for multiple times, and performing convolution-average pooling-full connection network-full map probability average until the identification model of the advertisement spotlight is converged.
Further, the process of identifying the outdoor advertisement picture change comprises the following steps:
1) uploading an advertisement design picture of a target outdoor billboard by a client, and filling an advertisement picture publishing time period; meanwhile, the camera module corresponding to the target outdoor billboard carries out snapshot on the advertisement picture image to obtain a target advertisement picture image;
2) uploading the obtained target advertisement picture image to a server, judging the average value of probability values of the advertisement picture image through a trained outdoor advertisement frame identification model, if the average value is less than 0.95, judging that the target advertisement picture image is a non-outdoor billboard feature picture, sending a judgment result to a client side, and giving an alarm; if the average value is judged to be greater than or equal to 0.95, a cosine similarity function is executed, if the cosine value is greater than or equal to 0.93, the advertisement design picture is consistent with the snap-shot target advertisement picture image, and the picture replacement risk does not exist; otherwise, the advertisement design picture is considered to be not in accordance with the captured target advertisement picture image, and the picture replacement risk exists;
3) and the client receives the final picture comparison result and judges whether the current advertisement picture is changed in the advertisement publishing period according to the appointed advertisement picture publishing time.
Further, the process of identifying the screen breakage includes:
matching an outdoor advertisement picture damage identification model by taking the characteristic value of a target advertisement picture image as a cosine similarity function vector input value, if the output cosine value is less than 0.95, judging that the target advertisement picture has a damage fault, sending result data to a client server, and pushing an alarm message to a user; and if the cosine value is greater than or equal to 0.95, judging that the target billboard picture is not damaged.
Compared with the prior art, the principle and the advantages of the scheme are as follows:
1. the scheme is based on the mode of the Internet of things, and the picture change or damage of the outdoor advertisement and the working condition of the spotlight are remotely monitored, so that the picture condition of the advertisement can be known without manual inspection, the working efficiency is improved, and the operation cost is reduced.
2. Specifically, a relay module is connected between a main control development board and an image acquisition module, and the image data of the advertisement pictures are acquired at regular time and uploaded to a server through a receiving and sending module to analyze, calculate and compare the images, so that whether the advertisement pictures are damaged or replaced is automatically judged, and the monitoring accuracy and the advertisement release trust problem between the advertisement pictures and an advertiser are greatly improved.
3. Specifically, a GPS module is installed at each outdoor advertisement position, so that the central control room receives the position of each outdoor advertisement, and if the advertisement picture of a certain advertisement billboard is monitored to be abnormal, the client can receive the advertisement picture in real time and inform the nearest staff of overhauling in the past.
4. The outdoor advertising pictures are identified based on the neural network model, and the identification accuracy is guaranteed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for the embodiments or the technical solutions in the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of an outdoor advertising picture recognition system based on a neural network according to the present invention;
FIG. 2 is a schematic flow chart of an outdoor advertisement picture recognition method based on neural network according to the present invention;
FIG. 3 is a schematic flow chart of training an outdoor advertisement frame recognition model in the outdoor advertisement frame recognition method based on the neural network according to the present invention;
FIG. 4 is a schematic flow chart of training an outdoor advertising picture breakage recognition model in the outdoor advertising picture recognition method based on the neural network according to the present invention;
FIG. 5 is a schematic flow chart of training an advertisement spot light recognition model in the outdoor advertisement picture recognition method based on the neural network according to the present invention.
Reference numerals:
1-master control development board; 2-a GPS module; 3-a relay module; 4-a camera module; 5-a transceiver module; 6-a server; 7-client side.
Detailed Description
The invention will be further illustrated with reference to specific examples:
as shown in fig. 1, an outdoor advertisement picture recognition system based on a neural network includes a main control development board 1, a GPS module 2, a relay module 3, a camera module 4, a transceiver module 5, a server 6, and a client 7.
Wherein the content of the first and second substances,
the camera module 4 is connected with the main control development board 1 through the relay module 3, and transmits the picture image to the main control development board 1 after collecting the outdoor advertisement picture image; the collected time is controlled by the relay module 3;
the GPS module 2 is connected with the main control development board 1 and sends the position of the outdoor advertisement corresponding to the GPS module to the main control development board 1;
the main control development board 1 is connected with a transceiver module 5, the outdoor advertisement picture image and the position data positioned by the GPS module 2 are sent to a server 6 through the transceiver module 5, and the server 6 analyzes, processes and stores the received data;
the server 6 is installed in a remote central control room;
the client 7 is connected with the server 6, and the outdoor advertising picture is remotely monitored through the server 6.
Specifically, in the present embodiment, the camera module 4 employs a 1.6mm ultra-wide angle fisheye camera module. The client 7 is a smart phone. The transceiver module 5 adopts a 5G network. The main control development board 1 processes the chip by adopting RK 3288.
As shown in fig. 2, the working principle of the present embodiment is as follows:
s1, the main control development board 1 carries out self-checking, the GPS module 2 and the transceiver module 5 are operated, the server 6 receives the state codes of the modules, and online monitoring of the modules is realized;
s2, the client 7 updates the positioning information of the billboard, and the user sets the snapshot time point, the snapshot interval time, the advertisement release time period of the target advertiser and the lighting time of the billboard;
s3, the master control development board 1 acquires the current world time, judges whether to enter a preset image capturing time point, and if not, the relay module 3 and the camera module 4 continue to enter a normal standby working state; the GPS module 2 reports the current longitude and latitude information to the server 6 every hour, if the uploaded data is deviated from the previous recorded data, the outdoor advertising board is judged to be displaced, and the client 7 prompts abnormity; if the preset image snapshot time point is judged to be entered, the main control development board 1 sends an instruction to start the relay module 3, the camera module 4 is electrified to operate, and snapshot return of image data of the corresponding outdoor advertisement picture is carried out;
s4, the captured image is transmitted back to the main control development board 1, after the image data are completely received, the relay module 3 is closed, and the next capturing time point is waited; the image data is sent to the server 6 through the transceiver module 5;
and S5, finally, carrying out outdoor advertisement picture identification based on the neural network model.
In the above, when the outdoor advertisement picture based on the neural network is identified, whether the outdoor advertisement picture is the outdoor advertisement picture, whether the outdoor advertisement picture is broken or not and whether a lamp tube or a neon lamp arranged on the advertisement board is normally lighted on time or not are respectively identified through the constructed and trained outdoor advertisement frame identification model, the outdoor advertisement picture breakage identification model and the advertisement spotlight identification model.
As shown in fig. 3, the specific process of training the outdoor advertisement frame recognition model is as follows:
s01, converting the initial image into gray scale, marking pixels, wherein 1 represents white and 0 represents black, extracting four positions from the initial image as the characteristics of the outdoor advertising range, and forming convolution kernels;
s02, graying the snap-shot image, marking the pixel characteristics with 1, 0, taking any element value in the convolution kernel, multiplying the element value by any pixel value of the snap-shot image, obtaining a new mark value when the result is the convolution value marked by the current pixel, and obtaining a new pixel value, also called a window, after averaging a plurality of values;
s03, after obtaining a window value, sliding the window to the right with the step length of 2, continuing to calculate a new characteristic pixel value, and after applying convolution kernel calculation to the snapshot image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s04, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the initial image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the initial image;
s05, performing multiple operations through a Softmax classification function, and adding probability values of all pixel regions to obtain an average, wherein the average is a similarity value of the snapshot image and the initial image;
s06, and circulating steps S01-S05 for multiple times, wherein image data captured each time are used as a training set for training an outdoor advertisement frame recognition model through convolution, average pooling, full connection network and full graph probability average until the outdoor advertisement frame recognition model converges.
As shown in fig. 4, the specific process of training the outdoor advertisement picture breakage recognition model is as follows:
s-1, collecting outdoor advertisement damaged picture images manually, and adding the picture images into a training library;
s-2, selecting and graying a first picture image from a training library, marking pixels, wherein 1 represents white and 0 represents black, and then extracting four positions from the first picture image as the characteristics of an outdoor advertisement range to serve as a convolution kernel;
s-3, selecting and graying a second picture image from a training library, marking the pixel characteristics by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value of the current pixel mark;
s-4, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and obtaining a brand new characteristic image after applying convolution kernel calculation to the second picture image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-5, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the first picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the first picture image;
s-6, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the second picture image and the first picture image;
and S-7, circulating the steps S-2 to S-6 for multiple times, and carrying out convolution-average pooling-full connection network-full map probability average until the outdoor advertising picture breakage identification model converges.
As shown in fig. 5, the specific process of training the identification model of the advertisement spotlight is as follows:
s-01, obtaining outdoor advertisement picture images in batch, and adding the outdoor advertisement picture images into a training library;
s-02, selecting a first outdoor advertisement picture image from a training library, and manually marking coordinates of the positions of all advertisement spot lamps;
s-03, graying the marked first outdoor advertisement picture image, marking pixels, wherein 1 represents white and 0 represents black, and extracting characteristic values of position coordinates of all spot lights of the first outdoor advertisement picture image to serve as convolution kernels;
s-04, selecting a second outdoor advertisement picture image from the training library, marking a pixel characteristic by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value marked by the current pixel, and averaging a plurality of values to obtain a new pixel value which is also called a window;
s-05, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and after applying convolution kernel calculation to a second outdoor advertisement picture image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-06, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the second outdoor advertisement picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the second outdoor advertisement picture image;
s-07, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the first outdoor advertisement picture image and the second outdoor advertisement picture image;
and S-08, repeating the steps from S-02 to S-07 for multiple times, and performing convolution-average pooling-full connection network-full map probability average until the identification model of the advertisement spotlight is converged.
The process of identifying the outdoor advertising picture change comprises the following steps:
1) the client 7 uploads an advertisement design picture of a target outdoor billboard, and fills in an advertisement picture publishing time period; meanwhile, the camera module 4 corresponding to the target outdoor billboard carries out snapshot on the advertisement picture image to obtain a target advertisement picture image;
2) uploading the obtained target advertisement picture image to a server 6, judging the average value of probability values of the advertisement picture image through a trained outdoor advertisement frame identification model, if the average value is less than 0.95, judging that the target advertisement picture image is a non-outdoor billboard feature picture, sending a judgment result to a client 7, and giving an alarm; if the average value is judged to be greater than or equal to 0.95, a cosine similarity function is executed, if the cosine value is greater than or equal to 0.93, the advertisement design picture is consistent with the snap-shot target advertisement picture image, and the picture replacement risk does not exist; otherwise, the advertisement design picture is considered to be not in accordance with the captured target advertisement picture image, and the picture replacement risk exists;
3) the client 7 receives the final picture comparison result, and judges whether the current advertisement picture is changed in the advertisement publishing period according to the appointed advertisement picture publishing time.
The process of identifying the picture breakage comprises the following steps:
matching an outdoor advertisement picture breakage recognition model by taking the characteristic value of the target advertisement picture image as a cosine similarity function vector input value, if the output cosine value is less than 0.95, judging that the target advertisement picture has a breakage fault, sending result data to a client server 6, and pushing an alarm message to a user; and if the cosine value is greater than or equal to 0.95, judging that the target billboard picture is not damaged.
The picture change or the damage condition of the outdoor advertisement is remotely monitored based on the mode of the Internet of things, so that the picture condition of the advertisement can be known without manual patrol, the working efficiency is improved, and the operation cost is reduced. Then, a relay module 3 is connected between the main control development board 1 and the image acquisition module, and the advertisement picture image data is acquired at regular time and uploaded to a server 6 through a transceiver module 5 to be analyzed, calculated and compared, so that whether the advertisement picture is damaged or replaced is automatically judged, and the monitoring accuracy and the advertisement release trust problem between the advertisement picture and an advertiser are greatly improved. And in addition, the GPS module 2 is arranged at each outdoor advertisement position, so that the central control room receives the position of each outdoor advertisement, and if the advertisement picture of a certain advertisement signboard is monitored to be abnormal, the client 7 can receive the advertisement picture in real time and inform the nearest staff of overhauling in the past. And finally, the outdoor advertisement picture is identified based on the neural network model, so that the identification accuracy is ensured.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that variations based on the shape and principle of the present invention should be covered within the scope of the present invention.
Claims (9)
1. An outdoor advertising picture recognition system based on a neural network is characterized by comprising a main control development board (1), a GPS module (2), a relay module (3), a camera module (4), a transceiving module (5), a server (6) and a client (7);
wherein the content of the first and second substances,
the camera module (4) is connected with the main control development board (1) through the relay module (3), and transmits the picture image to the main control development board (1) after collecting the outdoor advertisement picture image; the collected time is controlled by the relay module (3);
the GPS module (2) is connected with the main control development board (1) and sends the position of the outdoor advertisement corresponding to the GPS module to the main control development board (1);
the main control development board (1) is connected with the transceiver module (5), the outdoor advertisement picture image and the position data positioned by the GPS module (2) are sent to the server (6) through the transceiver module (5), and the server (6) analyzes, processes and stores the received data;
the server (6) is installed in a remote central control room;
the client (7) is connected with the server (6), and the outdoor advertising picture is remotely monitored through the server (6).
2. The outdoor advertising picture recognition system based on the neural network as claimed in claim 1, wherein the camera module (4) adopts a 1.6mm ultra wide angle fisheye camera module;
the client (7) is any one of a smart phone, a tablet personal computer, a notebook computer and a desktop computer;
the transceiver module (5) adopts any one of 4G, 5G, NB-IoT;
and the main control development board (1) processes the chip by adopting RK 3288.
3. An outdoor advertisement picture identification method based on a neural network is characterized by comprising the following steps:
s1, the main control development board carries out self-checking, the GPS module and the transceiver module are operated, the server receives the state codes of the modules, and online monitoring of the modules is realized;
s2, the client updates the positioning information of the billboard, and the user sets the snapshot time point, the snapshot interval time, the advertisement release time period of the target advertiser and the lighting time of the billboard;
s3, the master control development board acquires the current world time, judges whether to enter a preset image capture time point, if not, the relay module and the camera module continue to enter a normal standby working state, the GPS module reports the current longitude and latitude information to the server every one hour, if the uploaded data and the previous recorded data have deviation, the outdoor advertising board is judged to be displaced, and the client prompts abnormity; if the preset image snapshot time point is judged to be entered, the main control development board sends an instruction to start the relay module, the camera module is electrified to operate, and snapshot return of image data of the corresponding outdoor advertising picture is carried out;
s4, the snap-shot image is transmitted back to the main control development board, after the image data are completely received, the relay module is closed, and the next snap-shot time point is waited; the image data is sent to a server through a transceiver module;
and S5, finally, carrying out outdoor advertisement picture recognition based on the neural network model, and recognizing whether the outdoor advertisement picture is an outdoor advertisement picture, whether the outdoor advertisement picture is damaged, and whether a spotlight arranged on the billboard normally lights on time.
4. The method as claimed in claim 3, wherein when the neural network-based outdoor advertising picture is identified, the outdoor advertising frame identification model, the outdoor advertising picture breakage identification model and the advertising spot lamp identification model which are constructed and trained respectively identify whether the outdoor advertising picture is an outdoor advertising picture, whether the outdoor advertising picture is broken and whether a spot lamp installed on the advertising board is normally lighted on time.
5. The method for recognizing the outdoor advertising picture based on the neural network as claimed in claim 4, wherein the specific process of training the outdoor advertising frame recognition model is as follows:
s01, converting the initial image into gray scale, marking pixels, wherein 1 represents white and 0 represents black, extracting four positions from the initial image as the characteristics of the outdoor advertising range, and forming convolution kernels;
s02, graying the snap-shot image, marking the pixel characteristics with 1, 0, taking any element value in the convolution kernel, multiplying the element value by any pixel value of the snap-shot image, obtaining a new mark value when the result is the convolution value marked by the current pixel, and obtaining a new pixel value, also called a window, after averaging a plurality of values;
s03, after obtaining a window value, sliding the window to the right with the step length of 2, continuing to calculate a new characteristic pixel value, and after applying convolution kernel calculation to the snapshot image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s04, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the initial image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the initial image;
s05, performing multiple operations through a Softmax classification function, and adding probability values of all pixel regions to obtain an average, wherein the average is a similarity value of the snapshot image and the initial image;
s06, and circulating steps S01-S05 for multiple times, wherein image data captured each time are used as a training set for training an outdoor advertisement frame recognition model through convolution, average pooling, full connection network and full graph probability average until the outdoor advertisement frame recognition model converges.
6. The method for recognizing the outdoor advertising picture based on the neural network as claimed in claim 4, wherein the specific process of training the outdoor advertising picture breakage recognition model is as follows:
s-1, collecting outdoor advertisement damaged picture images manually, and adding the picture images into a training library;
s-2, selecting and graying a first picture image from a training library, marking pixels, wherein 1 represents white and 0 represents black, and then extracting four positions from the first picture image as the characteristics of an outdoor advertisement range to serve as a convolution kernel;
s-3, selecting and graying a second picture image from a training library, marking the pixel characteristics by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value of the current pixel mark;
s-4, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and obtaining a brand new characteristic image after applying convolution kernel calculation to the second picture image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-5, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the first picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the first picture image;
s-6, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the second picture image and the first picture image;
and S-7, circulating the steps S-2 to S-6 for multiple times, and carrying out convolution-average pooling-full connection network-full map probability average until the outdoor advertising picture breakage identification model converges.
7. The method for identifying the outdoor advertising picture based on the neural network as claimed in claim 4, wherein the specific process of training the identification model of the advertising spot lamp is as follows:
s-01, obtaining outdoor advertisement picture images in batch, and adding the outdoor advertisement picture images into a training library;
s-02, selecting a first outdoor advertisement picture image from a training library, and manually marking coordinates of the positions of all advertisement spot lamps;
s-03, graying the marked first outdoor advertisement picture image, marking pixels, wherein 1 represents white and 0 represents black, and extracting characteristic values of position coordinates of all spot lights of the first outdoor advertisement picture image to serve as convolution kernels;
s-04, selecting a second outdoor advertisement picture image from the training library, marking a pixel characteristic by using 1, 0, taking any element value in a convolution kernel, multiplying the pixel value by any pixel value of the second picture image, and obtaining a new mark value when the result is the convolution value marked by the current pixel, and averaging a plurality of values to obtain a new pixel value which is also called a window;
s-05, after obtaining a window value, sliding the window to the right with the step length of 2, continuously calculating a new characteristic pixel value, and after applying convolution kernel calculation to a second outdoor advertisement picture image, obtaining a brand new characteristic image; then, adopting an average pooling method to sequentially slide the characteristic values of the characteristic graph to the right and taking an average value;
s-06, obtaining a characteristic diagram after multiple times of pooling; then, applying a full-connection network to the feature map, adopting a Softmax classification function, and outputting a probability value of each corresponding category, wherein the closer the probability value is to 1, the closer the pixel region is to the second outdoor advertisement picture image, and otherwise, the closer the probability value is to 0, the closer the pixel region is to the second outdoor advertisement picture image;
s-07, performing multiple operations by a Softmax classification function, and adding probability values of all pixel regions to obtain an average number, wherein the average number is a similarity value of the first outdoor advertisement picture image and the second outdoor advertisement picture image;
and S-08, repeating the steps from S-02 to S-07 for multiple times, and performing convolution-average pooling-full connection network-full map probability average until the identification model of the advertisement spotlight is converged.
8. The method as claimed in claim 5, wherein the process of identifying the outdoor advertising scene change comprises:
1) uploading an advertisement design picture of a target outdoor billboard by a client, and filling an advertisement picture publishing time period; meanwhile, the camera module corresponding to the target outdoor billboard carries out snapshot on the advertisement picture image to obtain a target advertisement picture image;
2) uploading the obtained target advertisement picture image to a server, judging the average value of probability values of the advertisement picture image through a trained outdoor advertisement frame identification model, if the average value is less than 0.95, judging that the target advertisement picture image is a non-outdoor billboard feature picture, sending a judgment result to a client side, and giving an alarm; if the average value is judged to be greater than or equal to 0.95, a cosine similarity function is executed, if the cosine value is greater than or equal to 0.93, the advertisement design picture is consistent with the snap-shot target advertisement picture image, and the picture replacement risk does not exist; otherwise, the advertisement design picture is considered to be not in accordance with the captured target advertisement picture image, and the picture replacement risk exists;
3) and the client receives the final picture comparison result and judges whether the current advertisement picture is changed in the advertisement publishing period according to the appointed advertisement picture publishing time.
9. The method as claimed in claim 6, wherein the process of identifying the picture breakage comprises:
matching an outdoor advertisement picture damage identification model by taking the characteristic value of a target advertisement picture image as a cosine similarity function vector input value, if the output cosine value is less than 0.95, judging that the target advertisement picture has a damage fault, sending result data to a client server, and pushing an alarm message to a user; and if the cosine value is greater than or equal to 0.95, judging that the target billboard picture is not damaged.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110995155.3A CN113688746A (en) | 2021-08-27 | 2021-08-27 | Outdoor advertisement picture recognition system and method based on neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110995155.3A CN113688746A (en) | 2021-08-27 | 2021-08-27 | Outdoor advertisement picture recognition system and method based on neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688746A true CN113688746A (en) | 2021-11-23 |
Family
ID=78583412
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110995155.3A Pending CN113688746A (en) | 2021-08-27 | 2021-08-27 | Outdoor advertisement picture recognition system and method based on neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688746A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201142013Y (en) * | 2008-01-08 | 2008-10-29 | 卢旭东 | Monitoring and warning system of outdoor advertising lamp board |
| US20100066484A1 (en) * | 2008-09-15 | 2010-03-18 | Commtiva Technology Corporation | Remote monitor/control for billboard lighting or standby power system |
| CN102421064A (en) * | 2011-11-10 | 2012-04-18 | 厦门雅迅网络股份有限公司 | Method for monitoring vehicle displacement based on global positioning system (GPS) and communication base station and system thereof |
| CN106408024A (en) * | 2016-09-20 | 2017-02-15 | 四川大学 | Method for extracting lung lobe contour from DR image |
| CN109283994A (en) * | 2018-09-10 | 2019-01-29 | 广东小天才科技有限公司 | Electricity saving method, device and the smart machine of smart machine |
| CN109520481A (en) * | 2018-12-31 | 2019-03-26 | 广东博媒广告传播有限公司 | It is a kind of for monitoring the monitoring of the advertisement system of outdoor advertising picture |
| CN109544864A (en) * | 2019-01-21 | 2019-03-29 | 郑州航空工业管理学院 | Billboard risk online monitoring alarm device |
| CN109727293A (en) * | 2018-12-31 | 2019-05-07 | 广东博媒广告传播有限公司 | A kind of outdoor media light automatic recognition system |
| KR102048908B1 (en) * | 2019-07-09 | 2019-11-26 | 김상보 | Complete enumeration Management System and Method of Signboard using the GPS |
| CN112381104A (en) * | 2020-11-16 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Image identification method and device, computer equipment and storage medium |
-
2021
- 2021-08-27 CN CN202110995155.3A patent/CN113688746A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201142013Y (en) * | 2008-01-08 | 2008-10-29 | 卢旭东 | Monitoring and warning system of outdoor advertising lamp board |
| US20100066484A1 (en) * | 2008-09-15 | 2010-03-18 | Commtiva Technology Corporation | Remote monitor/control for billboard lighting or standby power system |
| CN102421064A (en) * | 2011-11-10 | 2012-04-18 | 厦门雅迅网络股份有限公司 | Method for monitoring vehicle displacement based on global positioning system (GPS) and communication base station and system thereof |
| CN106408024A (en) * | 2016-09-20 | 2017-02-15 | 四川大学 | Method for extracting lung lobe contour from DR image |
| CN109283994A (en) * | 2018-09-10 | 2019-01-29 | 广东小天才科技有限公司 | Electricity saving method, device and the smart machine of smart machine |
| CN109520481A (en) * | 2018-12-31 | 2019-03-26 | 广东博媒广告传播有限公司 | It is a kind of for monitoring the monitoring of the advertisement system of outdoor advertising picture |
| CN109727293A (en) * | 2018-12-31 | 2019-05-07 | 广东博媒广告传播有限公司 | A kind of outdoor media light automatic recognition system |
| CN109544864A (en) * | 2019-01-21 | 2019-03-29 | 郑州航空工业管理学院 | Billboard risk online monitoring alarm device |
| KR102048908B1 (en) * | 2019-07-09 | 2019-11-26 | 김상보 | Complete enumeration Management System and Method of Signboard using the GPS |
| CN112381104A (en) * | 2020-11-16 | 2021-02-19 | 腾讯科技（深圳）有限公司 | Image identification method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| AYŞE ÖZDEMIR等: "EFFECTS OF MENTHA SPICATA L. EXTRACTS ON HORMONAL REGULATION OF ENERGY METABOLISM IN RATS WITH HYPERCHOLESTEROLEMIA AND HYPERLIPIDEMIA", 《FRESENIUS ENVIRONMENTAL BULLETIN》, vol. 28, no. 10, 31 December 2019 (2019-12-31), pages 7314 - 7318 * |
| 孟鹏涛等: "基于i.MX6的智慧广告一体机的应用设计", 《单片机与嵌入式系统应用》, vol. 18, no. 01, 31 December 2018 (2018-12-31), pages 57 - 60 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10628890B2 (en) | Visual analytics based vehicle insurance anti-fraud detection | |
| US10511676B2 (en) | Image analysis system for property damage assessment and verification | |
| CN108109385B (en) | System and method for identifying and judging dangerous behaviors of power transmission line anti-external damage vehicle | |
| CN108537122B (en) | Image fusion acquisition system containing meteorological parameters and image storage method | |
| CN112166439A (en) | True-to-composite image domain transfer | |
| CN111047818A (en) | Forest fire early warning system based on video image | |
| JP2004537790A (en) | Moving object evaluation system and method | |
| JP2004534315A (en) | Method and system for monitoring moving objects | |
| CN111523397B (en) | Intelligent lamp post visual identification device, method and system and electronic equipment thereof | |
| CN109413411B (en) | Black screen identification method and device of monitoring line and server | |
| CN111680606B (en) | Low-power-consumption water level remote measuring system based on artificial intelligence cloud identification water gauge | |
| CN111639530B (en) | Method and system for detecting and identifying power transmission tower and insulator of power transmission line | |
| CN111446920A (en) | Photovoltaic power station monitoring method, device and system | |
| CN112966665A (en) | Pavement disease detection model training method and device and computer equipment | |
| CN114241338A (en) | Building measuring method, device, equipment and storage medium based on image recognition | |
| CN113552656B (en) | Rainfall intensity monitoring method and system based on outdoor image multi-space-time fusion | |
| CN115082813A (en) | Detection method, unmanned aerial vehicle, detection system and medium | |
| CN113705442A (en) | Outdoor large-board advertising picture monitoring and identifying system and method | |
| CN116310922A (en) | Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium | |
| CN208335208U (en) | Image co-registration acquisition system comprising meteorologic parameter | |
| CN113688746A (en) | Outdoor advertisement picture recognition system and method based on neural network | |
| CN117294818A (en) | Building site panoramic monitoring method for airport construction | |
| KR102457470B1 (en) | Apparatus and Method for Artificial Intelligence Based Precipitation Determination Using Image Analysis | |
| CN115690505A (en) | Photovoltaic module fault detection method and device, computer equipment and storage medium | |
| CN113569956A (en) | Mountain fire disaster investigation and identification method based on AI algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |