CN111275039B - Water gauge character positioning method, device, computing equipment and storage medium - Google Patents
Water gauge character positioning method, device, computing equipment and storage medium Download PDFInfo
- Publication number
- CN111275039B CN111275039B CN202010055194.0A CN202010055194A CN111275039B CN 111275039 B CN111275039 B CN 111275039B CN 202010055194 A CN202010055194 A CN 202010055194A CN 111275039 B CN111275039 B CN 111275039B
- Authority
- CN
- China
- Prior art keywords
- water gauge
- character
- gauge character
- neural network
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
- Character Input (AREA)
Abstract
The invention relates to the technical field of image recognition, and provides a water gauge character positioning method, a device, computing equipment and a storage medium, wherein the water gauge character positioning method comprises the following steps: acquiring a plurality of water gauge character images to be identified, which are specific to the same ship and have time attributes and space position attributes; generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the space position attribute; inputting the image set into a pre-trained convolutional neural network, and acquiring the water gauge character position in the water gauge character image to be identified by the convolutional neural network in combination with the time attribute and the space position attribute of the water gauge character image in the image set; and carrying out identification processing on the water gauge characters in the water gauge character image to be identified. By the implementation of the invention, the problem of lower accuracy of water gauge character positioning in the water gauge character positioning method in the prior art can be solved.
Description
Technical Field
The present invention relates to the field of image recognition technologies, and in particular, to a water gauge character positioning method, a device, a computing device, and a storage medium.
Background
With the continuous rise of economic level, free trade is developing, and a large number of ships are required to transport goods for trade. At present, a water gauge character is generally arranged on a ship, and a detector can detect the draft of the ship according to the draft state of the water gauge character, however, due to the limitations of the position of the ship, the ship volume, the detection time and the like, a method for detecting the water gauge character manually is complex, and the labor cost is high.
Meanwhile, along with the continuous development of image recognition technology, the accuracy and efficiency of image recognition are improved continuously, the image recognition technology is applied to detection and positioning of water gauge characters and is widely applied, in the prior art, generally, a photographed clearer water gauge character image is firstly obtained, then the water gauge character image is recognized and positioned through a deep learning network, and therefore the position of a water gauge character is marked.
Although the water gauge character positioning method can be used for positioning the water gauge characters, when the ship is in a complex and changeable environment, serious noise is easy to generate in the water gauge character positioning process, and the accuracy of positioning the water gauge characters is greatly reduced. In summary, the water gauge character positioning method in the prior art has the problem of low water gauge character positioning accuracy.
Disclosure of Invention
The embodiment of the application provides a water gauge character positioning method, a device, computing equipment and a storage medium, which are used for solving the problem that the water gauge character positioning accuracy is low in the water gauge character positioning method in the prior art.
The first embodiment of the invention provides a water gauge character positioning method, which comprises the following steps:
acquiring a plurality of water gauge character images to be identified, which are specific to the same ship and have time attributes and space position attributes;
generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the space position attribute;
inputting the image set into a pre-trained convolutional neural network, and acquiring the water gauge character position in the water gauge character image to be identified by the convolutional neural network in combination with the time attribute and the space position attribute of the water gauge character image in the image set;
and carrying out identification processing on the water gauge characters in the water gauge character image to be identified.
A second embodiment of the present invention provides a water gauge character locating device, comprising:
the water gauge character image acquisition module is used for aiming at a plurality of water gauge character images to be identified, which have time attributes and space position attributes, of the same ship;
the image set acquisition module is used for generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute;
the system comprises a water gauge character position acquisition module, a water gauge character position recognition module and a water gauge character recognition module, wherein the water gauge character position acquisition module is used for inputting an image set into a convolutional neural network, and the water gauge character position in a water gauge character image to be recognized is obtained by combining a time attribute and a space position attribute of the water gauge character image in the image set through the convolutional neural network trained in advance;
the mark processing module is used for carrying out mark processing on the water gauge characters in the water gauge character image to be identified.
A third embodiment of the present invention provides a computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of a water gauge character locating method provided by the first embodiment of the present invention when the computer program is executed by the processor.
A fourth embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of a water gauge character positioning method provided by the first embodiment of the present invention.
In the water gauge character positioning method, the device, the computing equipment and the storage medium, firstly, a plurality of water gauge character images to be recognized with time attribute and space position attribute for the same ship are obtained, then, an image set is generated by the water gauge character images to be recognized according to the time attribute and the space position attribute, then, the image set is input into a convolutional neural network trained in advance, the convolutional neural network combines the time attribute and the space position attribute of the water gauge character images in the image set to obtain the water gauge character positions in the water gauge character images to be recognized, and finally, the water gauge characters in the water gauge character images to be recognized are identified. In the invention, the convolutional neural network is combined with the space-time information of the water gauge character image to position the water gauge character in the water gauge character image so as to fully consider different application environments and backgrounds, and the water gauge character position is obtained from the complex image in real time, thereby effectively improving the accuracy of positioning the water gauge character and solving the problem of lower water gauge character positioning accuracy in the water gauge character positioning method in the prior art.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of an application environment of a water gauge character locating method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a water gauge character locating method of a first embodiment of the present invention;
FIG. 3 is a flow chart of step 13 of the water gauge character locating method of the first embodiment of the present invention;
FIG. 4 is a schematic diagram of the structure of a convolutional layer in a convolutional neural network in a first embodiment of the present invention;
FIG. 5 is yet another flow chart of a water gauge character locating method of a first embodiment of the present invention;
FIG. 6 is a block diagram of a water gauge character locating method of a second embodiment of the present invention;
FIG. 7 is a block diagram of a computing device provided by a third embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The water gauge character positioning method provided by the first embodiment of the invention can be applied to an application environment as shown in fig. 1, wherein a computing device acquisition device communicates with a computing device through a network. The method comprises the steps that computing equipment obtains a plurality of water gauge character images to be recognized, which are obtained by collecting equipment and have time attributes and space position attributes, according to the time attributes and the space position attributes, the water gauge character images to be recognized are generated into an image set, the image set is input into a convolutional neural network trained in advance, the convolutional neural network is combined with the time attributes and the space position attributes of the water gauge character images in the image set to obtain the water gauge character positions in the water gauge character images to be recognized, identification processing is carried out on the water gauge characters in the water gauge character images to be recognized, and an identification result is sent to a client side for a user to check. The collecting device comprises a shooting device with a camera, for example, the collecting device can be a notebook computer, a smart phone, a tablet personal computer, an unmanned aerial vehicle, a monitoring camera and the like with various cameras. The computing device may be implemented as a stand-alone server or as a cluster of servers. In addition, the communication mode between the acquisition device and the computing device can be, but is not limited to, wired (serial port, network port), wireless (2.4 g,5.8g, 4 th and 5 th generation mobile communication, satellite), and the like.
In a first embodiment of the present invention, as shown in fig. 2, a water gauge character positioning method is provided, and the method is applied to the computing device in fig. 1 for illustration, and includes the following steps 11 to 14.
Step 11: and acquiring a plurality of water gauge character images to be identified, which are specific to the same ship and have time attributes and space position attributes.
The time attribute may be time point information of shooting the water gauge character image, and the spatial position attribute may be azimuth, focal length and other information of shooting the water gauge character image. Specifically, a plurality of water gauge character images containing water gauge characters can be obtained from videos shot for the same ship. In addition, the time attribute and the spatial position attribute in the present embodiment should be determined depending on the time and the spatial position at the time of capturing the character image of the water gauge.
It should be noted that, in this embodiment, in order to be able to more easily obtain the spatial position attribute in the water gauge character image, the position of the camera is generally not changed when capturing the image or video containing the water gauge character, and images of different spatial sizes containing the water gauge character are obtained by changing the size of the focal length at different time points.
Step 12: and generating an image set from the plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute.
Specifically, the water gauge character images with different spatial position attributes are generated into an image set according to a time sequence, and the image set is shown in the following formula (1):
L(t)＝[l 1 (t),l 2 (t),...,l k (t),...,l K (t)] (1)
where t represents time, l (t) represents water gauge character image, and k represents different focus indexes.
For example, l 1 (t) can be a water gauge character image with 11-point focus index of 1, l 2 (t) can be a water gauge character image with 11-point focus index of 2, l 3 (t) may be a water gauge character image with an 11-point focus index of 3. It should be noted that there may be a plurality of water gauge character images of focal length indexes at a time point, and each time point may generate an image set as shown in the above formula (1), and in addition, the time point may be a time or a time period.
Step 13: inputting the image set into a pre-trained convolutional neural network, and acquiring the water gauge character position in the water gauge character image to be identified by the convolutional neural network in combination with the time attribute and the space position attribute of the water gauge character image in the image set.
The pixel information of each water gauge character image in the image set is specifically input to a pre-trained convolutional neural network, and the pixel information can be information of a single channel, a three channel, a six channel and the like of the water gauge character image, which is not particularly limited. In the process of positioning the water gauge character position in the water gauge character image, the convolutional neural network should combine the time attribute and the space position attribute of each water gauge character image to position and track each water gauge character in each water gauge character image so as to position the water gauge character.
It should be noted that in this embodiment, the convolutional neural network should include at least a convolutional layer and a fully-connected layer. The full-connection layer positions the water gauge characters in the water gauge character images according to the characteristics obtained by the convolution layer.
Further, as an implementation manner of the present embodiment, as shown in fig. 3, the above step 13 may specifically include the following steps 131 to 133.
Step 131: the first layer convolution layer, the second layer convolution layer and the third layer convolution layer in the pre-trained convolution neural network respectively extract first features of water gauge character images at different spatial positions at the same time point.
The method comprises the steps of obtaining time information and space position information of water gauge character images according to an image set, wherein the water gauge character images in different space positions can be water gauge character images with different focal lengths. In addition, the first feature is a shallow feature obtained from pixel information of the water gauge character image. For example, the first characteristic may be a water gauge character brightness, a water gauge character sharpness, a water gauge character duty cycle, a water body area, and so on.
Further, as shown in fig. 4, in one implementation manner of this embodiment, a plurality of first-layer convolution layers are disposed in the convolutional neural network, the plurality of first-layer convolution layers are respectively in one-to-one correspondence with a plurality of water gauge character images at different spatial positions at the same time point, and the first-layer convolution layers extract features of the water gauge character images corresponding to the first-layer convolution layers.
Specifically, taking three first-layer convolution layers as an example, the first-layer convolution layers include a first-layer convolution layer a, a first-layer convolution layer b, and a first-layer convolution layer c. The water gauge character image with the focal length index of k=1 is subjected to feature extraction through a first layer of convolution layer a, the feature extracted through a second layer of convolution layer a is subjected to feature extraction through a first layer of convolution layer b, the water gauge character image with the focal length index of k=2 is subjected to feature extraction through a first layer of convolution layer c, and the feature extracted through the first layer of convolution layer a and the feature extracted through the first layer of convolution layer c are subjected to feature extraction through a third layer of convolution layer.
In addition, when the same time point further comprises a water gauge character image with focal length index k=4, k=5 and k=6, the water gauge character image with focal length index k=4 extracts features through the first layer convolution layer a, the features extracted through the second layer convolution layer a are re-extracted, the water gauge character image with focal length index k=5 extracts features through the first layer convolution layer b, the water gauge character image with focal length index k=6 extracts features through the first layer convolution layer c, and the features extracted through the third layer convolution layer and the features extracted through the first layer convolution layer a are re-extracted.
That is, when the number of the first convolution layers is set to be the complete number, the number of the first convolution layers should be fixed, taking the number of the first convolution layers as three as an example in this embodiment, the number of the first convolution layers a, the second convolution layers b, and the third convolution layers c will be increased by increasing the number of the processed water gauge character images, and the number of the first convolution layers will not be increased.
In addition, it should be noted that in the present embodiment, the features of the respective water gauge character images should be taken as a whole, and the time information and the spatial position information in the respective water gauge character images should be taken as reference information for extracting the features. In addition, in the present embodiment, the features obtained by extracting the second layer convolution layer and the third layer convolution layer are taken as the first features.
In this embodiment, since the number of the input water gauge character images is large and the data size is large, the convolution layer is set according to the method shown in fig. 4, so that the effective first feature can be effectively extracted, the data dimension is reduced, and the calculation amount of the subsequent convolution layer in the convolution neural network is reduced.
Step 132: and extracting and fusing the first features by a fourth layer of convolution layer and a fifth layer of convolution layer in the pre-trained convolution neural network to obtain second features.
Wherein the second feature is a deep feature obtained from the first feature. The depth features may be waterline, a reflection area, a water gauge character handwriting ambiguity, and so on. As shown in fig. 4, the fourth and fifth layers of convolution extract and fuse the first features obtained by the second and third layers of convolution to obtain the second features.
Step 133: and respectively positioning the water gauge characters in the multiple water gauge character images according to the second characteristics by the full-connection layer in the pre-trained convolutional neural network to obtain the positions of the water gauge characters.
The method specifically comprises the steps that a full-connection layer in a pre-trained convolutional neural network obtains all second features extracted by a fifth layer convolutional layer, and the water gauge character position in a water gauge character image is calculated and obtained according to the weight of each feature in the full-connection layer. It should be noted that in this embodiment, two fully connected layers should be provided in the pre-trained convolutional neural network, where the first fully connected layer locates the position of the water gauge character according to the second feature and the feature weight, and the second fully connected layer relocates the position of the water gauge character.
In this embodiment, through implementation of steps 131 to 133, features of the water gauge character images are extracted according to time attributes and spatial location attributes of each water gauge character image, so that influence of loss of image information on positioning results when the water gauge character images are scaled in a convolutional neural network is effectively reduced, specifically, for features of smaller characters of the water gauge characters, the size of focal distances is adjusted at different time points, and the water gauge characters in the water gauge character images are positioned by utilizing focal distance information.
Step 14: and carrying out identification processing on the water gauge characters in the water gauge character image to be identified.
The water gauge characters in the water gauge character image can be specifically identified, and the mode of identification processing is not particularly limited.
It should be noted that in the process of image recognition by the convolutional neural network, downsampling processing is generally required for the image, but the downsampling processing is performed for the image, so that the quality of the image is lower, when the water gauge character image subjected to the zooming processing is obtained, the smaller water gauge character is larger, the situation that the water gauge character is smaller after being reduced and the lost information quantity is larger is effectively prevented, and then the water gauge character image is more effectively recognized, meanwhile, when the camera zooms, the resolution of the image is unchanged, but the photographed water gauge character is larger, and the water gauge character can still keep a certain size and information quantity after the downsampling processing, so that accurate recognition is facilitated.
Through implementation of the steps 11 to 14, the convolutional neural network is combined with the time attribute and the space attribute of the water gauge character image to position the water gauge character, so that the accuracy of positioning the water gauge character image can be effectively improved, and the problem of low positioning accuracy of the water gauge character in the water gauge character positioning method in the prior art is effectively solved.
Further, as an implementation manner of this embodiment, as shown in fig. 5, obtaining a pre-trained convolutional neural network may include the following steps 21 to 25.
Step 21: and acquiring a plurality of sample water gauge character images with time attribute and space position attribute aiming at the same ship.
The sample water gauge character image should contain multiple categories, and each category should have positive and negative samples for testing.
Further, as an implementation manner of this embodiment, the sample water gauge character image includes at least one of water gauge character images photographed in different directions, water gauge character images set at different time points, water gauge character images with water body reflection, and water gauge character images without water body reflection. The more the number of sample water gauge character images, the more the variety, the better the training effect of the convolutional neural network.
In this embodiment, by setting that the sample water gauge character image includes at least one of a water gauge character image photographed in different directions, a water gauge character image set at different time points, a water gauge character image with water body reflection, and a water gauge character image without water body reflection, the convolutional neural network can identify a specific environment in which the water gauge character exists when the characteristics of the sample water gauge character image are extracted subsequently, and positioning accuracy is improved.
Step 22: and generating a sample image set from the plurality of sample water gauge character images according to the time attribute and the spatial position attribute.
Since the method of generating the sample image set in step 22 is similar to the method of generating the image set in step 12 described above, the description thereof will be omitted.
Step 23: and inputting the sample image set into a convolutional neural network, and acquiring the water gauge character position in the sample water gauge character image by the convolutional neural network in combination with the time attribute and the space position attribute of the sample water gauge character image in the sample image set.
Since the method for positioning the position of the water gauge character in the sample water gauge character image in step 23 is similar to the method for positioning the position of the water gauge character in the water gauge character image in step 13, the description thereof will not be repeated here.
Step 24: and calculating the predicted water gauge character position positioning results of the water gauge character images of the plurality of samples through the loss function to obtain the quality evaluation coefficient of the convolutional neural network.
The quality evaluation coefficient is used for evaluating the quality of the convolutional neural network, and the step 24 specifically obtains a predicted positioning frame for marking the predicted water gauge character position of the sample water gauge image according to the predicted water gauge character position, then obtains an actual positioning frame for marking the predicted water gauge character position of the sample water gauge image, and finally calculates the predicted positioning frame and the actual positioning frame through softmax logistic regression to obtain the quality evaluation coefficient of the convolutional neural network.
Further, as an implementation manner of the present embodiment, the quality evaluation coefficient is obtained according to softmax logistic regression calculation as shown in the following formula (2):
L(x,c,l,g)＝
1/N(L conf (x,c)+a 1 *L loc1 (x,l,g)+a 2 *L loc2 (x,l,g)+…+a n *L locn (x,l,g)) (2)
wherein N represents the prior character frame number matched with the actual character frame, x represents the matching indication matrix, c represents the confidence level, L represents the predicted character frame, g represents the actual character frame, L locn Representing the position loss of the nth predicted character frame, L conf Representing the Softmax logistic regression loss,a n representing the weight of the n-th predicted frame position penalty.
Further, L locn Specifically, the difference between the center coordinates and the width and height of the nth predicted character frame and the actual character frame is represented.
In this embodiment, the quality evaluation coefficient is obtained by calculation according to the above (2), and the softmax logistic regression is combined with the convolutional neural network to evaluate the degree of the convolutional neural network.
It should be noted that in this embodiment, a plurality of sample water gauge character images belonging to different spatial position attributes with different focal lengths are used for inputting, and when prediction for a certain type of problem is inaccurate, a sample water gauge character image related to the certain type of problem can be input for the certain type of problem. For example, the number of sample water gauge character images with the back image is reduced, when training is performed on most of sample water gauge character images without the back image, the convolutional neural network extracts the characteristics of the sample water gauge character images with the back image, suppresses the back image character characteristics of the sample water gauge character images with the back image and the character projection, as shown by L in the formula (2) loc1 (x,l,g)、L loc2 (x,l,g)、L loc3 (x,l,g)、…L locn (x, l, g) only one sample water gauge character image containing the reflection, and only a few reflection characters in the reflection sample water gauge character image, the influence on the whole loss function is negligible, and the reflection character cannot be detected as a result. By the training method, the problem of inaccurate prediction aiming at a certain type of problem can be effectively solved.
Step 25: when the quality evaluation coefficient does not reach a preset threshold, the feature weight of each feature in the convolutional neural network is adjusted, and the sample image set is repeatedly input into the convolutional neural network to obtain the predicted water gauge character position, until the quality evaluation coefficient reaches the preset threshold, so that the convolutional neural network trained in advance is obtained.
The preset threshold value can be preset, when the quality evaluation coefficient reaches the preset threshold value, the current convolutional neural network is represented to meet the requirement for positioning the water gauge characters, and the current convolutional neural network is determined to be a pre-trained convolutional neural network for positioning the subsequent water gauge characters.
Further, as an implementation manner of this embodiment, in the step 25, the feature weights of the features in the convolutional neural network may be specifically adjusted by an accelerated gradient descent algorithm.
The characteristic weights of the various characteristics in the convolutional neural network can be specifically adjusted according to the following formula (3) and formula (4):
V t+1 ＝u*V t -gradient L (W t +u*V t ) (3)
W t+1 ＝W t +V t+1 (4)
Where t represents the index of each cycle parameter update, such as t=0, t=1, t=2, v t+1 Is the gradient value of the next cycle, V t Represents the gradient value of the current cycle, L represents the loss function, W t Represents the weight of the current cycle, u represents the basic weight, a represents the total weight, and the weight W of the next cycle t+1 Is composed of the current weight W t Gradient value V of next cycle t+1 Obtained by the formula (4).
In this embodiment, the feature weights of the features in the convolutional neural network are adjusted by the accelerating gradient descent algorithm, so that the speed of updating the feature weights can be effectively increased, and the training process is accelerated.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
A second embodiment of the present invention provides a water gauge character positioning device, where the water gauge character positioning device corresponds to the water gauge character positioning method provided in the first embodiment one by one.
Further, as shown in fig. 6, the water gauge character positioning device includes a water gauge character image acquisition module 41, an image set acquisition module 42, a water gauge character position acquisition module 43, and an identification processing module 44. The functional modules are described in detail as follows:
a water gauge character image obtaining module 41, configured to target a plurality of water gauge character images to be identified of the same ship, where the water gauge character images have a time attribute and a spatial position attribute;
an image set obtaining module 42, configured to generate an image set from a plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute;
the water gauge character position obtaining module 43 is configured to input the image set into a convolutional neural network, and the pre-trained convolutional neural network obtains a water gauge character position in a water gauge character image to be identified by combining a time attribute and a spatial position attribute of the water gauge character image in the image set;
the identification processing module 44 is configured to perform identification processing on the water gauge characters in the water gauge character image to be identified.
Further, as an implementation manner of the present embodiment, the water gauge character position obtaining module 43 may include a first feature obtaining unit, a second feature obtaining unit, and a water gauge character position obtaining unit. The functional units are described in detail as follows:
the first characteristic acquisition unit is used for respectively extracting first characteristics of water gauge character images at different spatial positions at the same time point from a first layer of convolution layer, a second layer of convolution layer and a third layer of convolution layer in the pre-trained convolution neural network;
the second feature acquisition unit is used for extracting and fusing the first features by a fourth layer convolution layer and a fifth layer convolution layer in the pre-trained convolutional neural network to obtain second features;
and the water gauge character position acquisition unit is used for respectively positioning water gauge characters in the multiple water gauge character images according to the second characteristics by the full-connection layer in the pre-trained convolutional neural network to obtain the water gauge character positions.
Further, as an implementation manner of the present embodiment, the first feature acquisition unit includes a first feature acquisition subunit. The first feature acquisition subunit is described in detail as follows:
the first characteristic acquisition subunit is configured to provide a plurality of first layer convolution layers in the convolutional neural network, and respectively correspond the plurality of first layer convolution layers to the plurality of water gauge character images at different spatial positions at the same time point one by one, where the first layer convolution layers extract characteristics of the water gauge character images corresponding to the first layer convolution layers.
Further, as an implementation manner of this embodiment, the water gauge character positioning device further includes: the system comprises a sample water gauge character image acquisition module, a sample image collection acquisition module, a sample water gauge character position acquisition module, a quality evaluation coefficient acquisition module and a convolutional neural network determination module. The functional modules are described in detail as follows:
the sample water gauge character image acquisition module is used for acquiring a plurality of sample water gauge character images with time attribute and space position attribute for the same ship;
the sample image set acquisition module is used for generating a sample image set from a plurality of sample water gauge character images according to the time attribute and the spatial position attribute;
the system comprises a sample water gauge character position acquisition module, a convolution neural network and a water gauge character position acquisition module, wherein the sample water gauge character position acquisition module is used for inputting a sample image set into the convolution neural network, and the convolution neural network acquires the water gauge character position in the sample water gauge character image by combining the time attribute and the space position attribute of the sample water gauge character image in the sample image set;
the quality evaluation coefficient acquisition module is used for calculating the predicted water gauge character position positioning results of the water gauge character images of the plurality of samples through the loss function to obtain the quality evaluation coefficient of the convolutional neural network;
and the convolutional neural network determining module is used for adjusting the feature weights of all the features in the convolutional neural network when the quality evaluation coefficient does not reach a preset threshold value, and repeatedly inputting the sample image set into the convolutional neural network to obtain the predicted water gauge character position until the quality evaluation coefficient reaches the preset threshold value so as to obtain the convolutional neural network trained in advance.
Further, as an implementation manner of the present embodiment, the quality evaluation coefficient acquisition module may include a quality evaluation coefficient acquisition unit. The quality evaluation coefficient acquisition unit is described in detail as follows:
a quality evaluation coefficient acquisition unit for calculating a quality evaluation coefficient calculated according to the following formula (2):
L(x,c,l,g)＝
1/N(L conf (x,c)+a 1 *L loc1 (x,l,g)+a 2 *L loc2 (x,l,g)+…+a n *L locn (x,l,g)) (2)
wherein N represents the prior character frame number matched with the actual character frame, x represents the matching indication matrix, c represents the confidence level, L represents the predicted character frame, g represents the actual character frame, L locn Representing the position loss of the nth predicted character frame, L conf Represents a Softmax logistic regression loss, a n Representing the weight of the n-th predicted frame position penalty.
Further, L locn Specifically, the difference between the center coordinates and the width and height of the nth predicted character frame and the actual character frame is represented.
Further, as an implementation manner of the present embodiment, the convolutional neural network determination module may include a feature weight adjustment unit. The detailed description of the feature weight adjustment unit is as follows:
and the characteristic weight adjusting unit is used for adjusting the characteristic weights of all the characteristics in the convolutional neural network through an accelerating gradient descent algorithm.
Further, as an implementation manner of this embodiment, the sample water gauge character image includes at least one of water gauge character images photographed in different directions, water gauge character images set at different time points, water gauge character images with water body reflection, and water gauge character images without water body reflection.
For specific limitations of the water gauge character positioning device, reference may be made to the above limitation of the water gauge character positioning method, and no further description is given here. The various modules/units in the above-described water gauge character locating device may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computing device, or may be stored in software in a memory in the computing device, so that the processor may invoke and execute operations corresponding to the above modules.
A third embodiment of the present invention provides a computing device whose internal structural diagram may be as shown in fig. 7. The computing device includes a processor, memory, a network interface, and a database connected by a system bus. Wherein the processor of the computing device is configured to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computing device is used to store data involved in the water gauge character locating method. The network interface of the computing device is for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements the water gauge character locating method provided by the first embodiment of the present invention.
A fourth embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the water gauge character locating method provided by the first embodiment of the present invention, such as steps 11 to 13 shown in fig. 2, steps 131 to 133 shown in fig. 3, and steps 21 to 25 shown in fig. 5. Alternatively, the computer program when executed by a processor implements the functions of the modules/units of the water gauge character positioning method provided in the first embodiment described above. In order to avoid repetition, a description thereof is omitted.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (8)
1. The water gauge character positioning method is characterized by comprising the following steps of:
acquiring a plurality of water gauge character images to be identified, which are specific to the same ship and have time attributes and space position attributes;
generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute;
inputting the image set into a pre-trained convolutional neural network, wherein the convolutional neural network combines the time attribute and the space position attribute of the water gauge character image in the image set to obtain the water gauge character position in the water gauge character image to be identified;
performing identification processing on the water gauge characters in the water gauge character image to be identified;
wherein obtaining the pre-trained convolutional neural network comprises:
acquiring a plurality of sample water gauge character images with time attribute and space position attribute for the same ship;
generating a sample image set from a plurality of sample water gauge character images according to the time attribute and the spatial position attribute;
inputting the sample image set into the convolutional neural network, and obtaining the water gauge character position in the sample water gauge character image by the convolutional neural network in combination with the time attribute and the space position attribute of the sample water gauge character image in the sample image set;
calculating the predicted water gauge character position positioning results of a plurality of sample water gauge character images through a loss function to obtain a quality evaluation coefficient of the convolutional neural network;
when the quality evaluation coefficient does not reach a preset threshold, adjusting the feature weight of each feature in the convolutional neural network, and repeatedly inputting the sample image set into the convolutional neural network to obtain a predicted water gauge character position, until the quality evaluation coefficient reaches the preset threshold, so as to obtain the convolutional neural network trained in advance;
wherein the quality assessment coefficient is calculated according to the following formula (1):
L(x,c,l,g)＝
1/N(L conf (x,c)+a 1 *L loc1 (x,l,g)+a 2 *L loc2 (x,l,g)+…+a n *L locn (x, l, g)) (1) wherein N represents the same actual character as that of the actual characterThe prior character frame number of the frame matching, x represents the matching indication matrix, c represents the confidence level, L represents the predicted character frame, g represents the actual character frame and L locn Representing the position loss of the nth predicted character frame, L conf Represents a Softmax logistic regression loss, a n Weight representing n-th predicted frame position penalty, a 1 Weight representing loss of 1 st predicted frame position, a 2 Representing the weight of the 2 nd predicted frame position penalty.
2. The method of claim 1, wherein the convolutional neural network combining the temporal attribute and the spatial location attribute of the water gauge character image in the image set to obtain a water gauge character location in the water gauge character image comprises:
the first layer convolution layer, the second layer convolution layer and the third layer convolution layer in the pre-trained convolution neural network respectively extract first features of the water gauge character images at different spatial positions at the same time point;
extracting and fusing the first features by a fourth layer of convolution layer and a fifth layer of convolution layer in the pre-trained convolution neural network to obtain second features;
and respectively positioning the water gauge characters in the water gauge character images according to the second characteristics by the full-connection layer in the pre-trained convolutional neural network to obtain the positions of the water gauge characters.
3. The method for positioning water gauge characters according to claim 2, wherein a plurality of first layer convolution layers are arranged in the convolutional neural network, the first layer convolution layers are respectively in one-to-one correspondence with a plurality of water gauge character images at different spatial positions at the same time point, and the first layer convolution layers extract features of the water gauge character images corresponding to the first layer convolution layers.
4. The waterbar character positioning method according to claim 1, wherein the feature weights of the features in the convolutional neural network are adjusted by an accelerated gradient descent algorithm.
5. The method of claim 1, wherein the sample water gauge character image comprises at least one of water gauge character images photographed in different directions, water gauge character images set at different time points, water gauge character images with water body reflection, and water gauge character images without water body reflection.
6. A water gauge character locating device, comprising:
the water gauge character image acquisition module is used for aiming at a plurality of water gauge character images to be identified, which have time attributes and space position attributes, of the same ship;
the image set acquisition module is used for generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the space position attribute;
the water gauge character position acquisition module is used for inputting the image set into a convolutional neural network, and the pre-trained convolutional neural network acquires the water gauge character position in the water gauge character image to be identified by combining the time attribute and the space position attribute of the water gauge character image in the image set;
the mark processing module is used for carrying out mark processing on the water gauge characters in the water gauge character image to be identified;
wherein obtaining the pre-trained convolutional neural network comprises:
the sample water gauge character image acquisition module is used for acquiring a plurality of sample water gauge character images with time attribute and space position attribute for the same ship;
the sample image set acquisition module is used for generating a sample image set from a plurality of sample water gauge character images according to the time attribute and the spatial position attribute;
the sample water gauge character position acquisition module is used for inputting the sample image set into the convolutional neural network, and the convolutional neural network is used for acquiring the water gauge character position in the sample water gauge character image by combining the time attribute and the space position attribute of the sample water gauge character image in the sample image set;
the quality evaluation coefficient acquisition module is used for calculating the position positioning results of the predicted water gauge characters of the plurality of sample water gauge character images through a loss function to obtain the quality evaluation coefficient of the convolutional neural network;
the convolutional neural network determining module is used for adjusting the feature weights of all the features in the convolutional neural network when the quality evaluation coefficient does not reach a preset threshold value, and repeatedly inputting the sample image set into the convolutional neural network to obtain a predicted water gauge character position, until the quality evaluation coefficient reaches the preset threshold value, so as to obtain the convolutional neural network trained in advance;
wherein the quality assessment coefficient is calculated according to the following formula (1):
L(x,c,l,g)＝
1/N(L conf (x,c)+a 1 *L loc1 (x,l,g)+a 2 *L loc2 (x,l,g)+…+a n *L locn (x, L, g)) (1), wherein N represents the number of prior character frames matching the actual character frame, x represents the matching indication matrix, c represents the confidence level, L represents the predicted character frame, g represents the actual character frame, L locn Representing the position loss of the nth predicted character frame, L conf Represents a Softmax logistic regression loss, a n Weight representing n-th predicted frame position penalty, a 1 Weight representing loss of 1 st predicted frame position, a 2 Representing the weight of the 2 nd predicted frame position penalty.
7. A computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the water gauge character locating method of any one of claims 1 to 5.
8. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the waterbar character positioning method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055194.0A CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055194.0A CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275039A CN111275039A (en) | 2020-06-12 |
| CN111275039B true CN111275039B (en) | 2023-05-16 |
Family
ID=71001683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010055194.0A Active CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275039B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663811B (en) * | 2022-03-24 | 2022-10-28 | 中国水利水电科学研究院 | Water surface line extraction method using water gauge reflection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107516096A (en) * | 2016-06-15 | 2017-12-26 | 阿里巴巴集团控股有限公司 | A kind of character identifying method and device |
| WO2018090641A1 (en) * | 2016-11-15 | 2018-05-24 | 平安科技（深圳）有限公司 | Method, apparatus and device for identifying insurance policy number, and computer-readable storage medium |
| CN108875722A (en) * | 2017-12-27 | 2018-11-23 | 北京旷视科技有限公司 | Character recognition and identification model training method, device and system and storage medium |
| CN109977980A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of method for recognizing verification code and device |
-
2020
- 2020-01-17 CN CN202010055194.0A patent/CN111275039B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107516096A (en) * | 2016-06-15 | 2017-12-26 | 阿里巴巴集团控股有限公司 | A kind of character identifying method and device |
| WO2018090641A1 (en) * | 2016-11-15 | 2018-05-24 | 平安科技（深圳）有限公司 | Method, apparatus and device for identifying insurance policy number, and computer-readable storage medium |
| CN108875722A (en) * | 2017-12-27 | 2018-11-23 | 北京旷视科技有限公司 | Character recognition and identification model training method, device and system and storage medium |
| CN109977980A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of method for recognizing verification code and device |
Non-Patent Citations (1)
| Title |
|---|
| 李翊 ; 兰华勇 ; 严华 ; .基于图像处理和BP神经网络的水位识别研究.人民黄河.(12),第18-21页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275039A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109034078B (en) | Training method of age identification model, age identification method and related equipment | |
| CN112990432B (en) | Target recognition model training method and device and electronic equipment | |
| US20200372243A1 (en) | Image processing method and apparatus, facial recognition method and apparatus, and computer device | |
| WO2020228446A1 (en) | Model training method and apparatus, and terminal and storage medium | |
| WO2020186678A1 (en) | Three-dimensional map constructing method and apparatus for unmanned aerial vehicle, computer device, and storage medium | |
| US11093737B2 (en) | Gesture recognition method and apparatus, electronic device, and computer-readable storage medium | |
| CN109871821B (en) | Pedestrian re-identification method, device, equipment and storage medium of self-adaptive network | |
| CN110807491A (en) | License plate image definition model training method, definition detection method and device | |
| CN111612822B (en) | Object tracking method, device, computer equipment and storage medium | |
| CN111553182A (en) | Ship retrieval method and device and electronic equipment | |
| CN111144398A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| CN112884782B (en) | Biological object segmentation method, apparatus, computer device, and storage medium | |
| CN112949453B (en) | Training method of smoke and fire detection model, smoke and fire detection method and equipment | |
| CN111047088A (en) | Prediction image acquisition method and device, computer equipment and storage medium | |
| CN110717449A (en) | Vehicle annual inspection personnel behavior detection method and device and computer equipment | |
| CN111832561B (en) | Character sequence recognition method, device, equipment and medium based on computer vision | |
| CN110660078A (en) | Object tracking method and device, computer equipment and storage medium | |
| CN113706481A (en) | Sperm quality detection method, sperm quality detection device, computer equipment and storage medium | |
| CN111695572A (en) | Ship retrieval method and device based on convolutional layer feature extraction | |
| CN113065593A (en) | Model training method and device, computer equipment and storage medium | |
| CN111275039B (en) | Water gauge character positioning method, device, computing equipment and storage medium | |
| CN109961103B (en) | Training method of feature extraction model, and image feature extraction method and device | |
| CN113780145A (en) | Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium | |
| CN112926616B (en) | Image matching method and device, electronic equipment and computer readable storage medium | |
| CN114596515A (en) | Target object detection method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |