CN111275039A - Water gauge character positioning method and device, computing equipment and storage medium - Google Patents
Water gauge character positioning method and device, computing equipment and storage medium Download PDFInfo
- Publication number
- CN111275039A CN111275039A CN202010055194.0A CN202010055194A CN111275039A CN 111275039 A CN111275039 A CN 111275039A CN 202010055194 A CN202010055194 A CN 202010055194A CN 111275039 A CN111275039 A CN 111275039A
- Authority
- CN
- China
- Prior art keywords
- water gauge
- gauge character
- character
- neural network
- convolutional neural
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
- Character Input (AREA)
Abstract
The invention relates to the technical field of image recognition, and provides a water gauge character positioning method, a device, a computing device and a storage medium, wherein the water gauge character positioning method comprises the following steps: acquiring a plurality of water gauge character images to be recognized, which have time attributes and spatial position attributes and aim at the same ship; generating an image set from a plurality of water gauge character images to be recognized according to the time attribute and the spatial position attribute; inputting the image set into a pre-trained convolutional neural network, and obtaining the water gauge character position in the water gauge character image to be recognized by the convolutional neural network by combining the time attribute and the space position attribute of the water gauge character image in the image set; and carrying out identification processing on the water gauge characters in the water gauge character image to be recognized. By implementing the method, the problem of low water gauge character positioning accuracy in a water gauge character positioning method in the prior art can be solved.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method and a device for positioning water gauge characters, computing equipment and a storage medium.
Background
With the increasing economic level and the continuous development of free trade, a large amount of ships are required for transporting goods for trade. At present, the water gauge characters are generally arranged on a ship, and a detector can detect the draft of the ship according to the draft state of the water gauge characters, however, due to the limitations of the position of the ship, the size of the ship, the detection time and the like, the method for manually detecting the water gauge characters is complex, and the labor cost is high.
Simultaneously, along with the continuous development of image recognition technology, image recognition's precision and efficiency also are constantly promoting, and the detection location of being applied to the water gauge character with image recognition technology has obtained extensive application, generally acquires the comparatively clear water gauge character image of shooting earlier among the prior art, then discerns the location to this water gauge character image through the degree of depth learning network to mark out the position of water gauge character.
Although the water gauge character positioning method can realize positioning of the water gauge characters, when the ship is in a complicated and changeable environment, serious noise is easily generated in the water gauge character positioning process, and the accuracy of positioning the water gauge characters is greatly reduced. In summary, the water gauge character positioning method in the prior art has the problem of low water gauge character positioning accuracy.
Disclosure of Invention
The embodiment of the application provides a water gauge character positioning method, a water gauge character positioning device, computing equipment and a storage medium, and aims to solve the problem that the water gauge character positioning accuracy is low in the water gauge character positioning method in the prior art.
A first embodiment of the present invention provides a method for positioning a water gauge character, including:
acquiring a plurality of water gauge character images to be recognized, which have time attributes and spatial position attributes and aim at the same ship;
generating an image set from a plurality of water gauge character images to be recognized according to the time attribute and the spatial position attribute;
inputting the image set into a pre-trained convolutional neural network, and obtaining the water gauge character position in the water gauge character image to be recognized by the convolutional neural network by combining the time attribute and the space position attribute of the water gauge character image in the image set;
and carrying out identification processing on the water gauge characters in the water gauge character image to be recognized.
A second embodiment of the present invention provides a water gauge character positioning device, including:
the water gauge character image acquisition module is used for aiming at a plurality of water gauge character images to be recognized of the same ship and having time attributes and spatial position attributes;
the image set acquisition module is used for generating an image set from a plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute;
the water gauge character position acquisition module is used for inputting the image set into the convolutional neural network, and the pre-trained convolutional neural network combines the time attribute and the space position attribute of the water gauge character images in the image set to acquire the water gauge character position in the water gauge character image to be recognized;
and the identification processing module is used for identifying the water gauge characters in the water gauge character image to be recognized.
A third embodiment of the present invention provides a computing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for positioning water ruler characters provided by the first embodiment of the present invention when executing the computer program.
A fourth embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the method for positioning a water gauge character provided by the first embodiment of the present invention are realized.
In the water gauge character positioning method, the water gauge character positioning device, the computing equipment and the storage medium, firstly, a plurality of water gauge character images to be recognized with time attributes and space position attributes for the same ship are obtained, then, an image set is generated by the water gauge character images to be recognized according to the time attributes and the space position attributes, then, the image set is input to a pre-trained convolutional neural network, the convolutional neural network combines the time attributes and the space position attributes of the water gauge character images in the image set to obtain the water gauge character positions in the water gauge character images to be recognized, and finally, the water gauge characters in the water gauge character images to be recognized are subjected to identification processing. In the invention, the convolutional neural network is combined with the space-time information of the water gauge character image to position the water gauge character in the water gauge character image, so that different application environments and backgrounds are fully considered, the position of the water gauge character is obtained from the complex image in real time, the accuracy of positioning the water gauge character is effectively improved, and the problem of low accuracy of positioning the water gauge character in the water gauge character positioning method in the prior art is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive labor.
FIG. 1 is a schematic diagram of an application environment of a method for positioning a water gauge character according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for positioning a water gauge character according to a first embodiment of the present invention;
FIG. 3 is a flowchart of step 13 of the water gauge character positioning method according to the first embodiment of the present invention;
FIG. 4 is a diagram illustrating the structure of convolutional layers in a convolutional neural network according to a first embodiment of the present invention;
FIG. 5 is yet another flowchart of a water gauge character positioning method of the first embodiment of the present invention;
FIG. 6 is a block diagram illustrating a method for positioning water gauge characters according to a second embodiment of the present invention;
fig. 7 is a block diagram of a computing device according to a third embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The method for positioning water gauge characters provided by the first embodiment of the present invention can be applied to an application environment as shown in fig. 1, in which a collection device of a computing device communicates with the computing device through a network. The method comprises the steps that a computing device obtains a plurality of water gauge character images to be recognized, which are obtained by a collecting device and have time attributes and space position attributes, for the same ship, the image sets are generated according to the time attributes and the space position attributes, the image sets are input into a pre-trained convolutional neural network, the convolutional neural network combines the time attributes and the space position attributes of the water gauge character images in the image sets to obtain the water gauge character positions in the water gauge character images to be recognized, the water gauge characters in the water gauge character images to be recognized are subjected to identification processing, and identification results are sent to a client side to be checked by a user. Wherein, collection equipment is including the shooting device that has the camera, for example, collection equipment can be that computing equipment is various notebook computer, smart mobile phone, panel computer, unmanned aerial vehicle, surveillance camera etc. that have the camera. The computing device may be implemented as a stand-alone server or as a server cluster of multiple servers. In addition, the communication mode between the acquisition device and the computing device may be, but is not limited to, wired (serial port, internet access), wireless (2.4G, 5.8G, 4 th and 5 th generation mobile communication, satellite), and the like.
In a first embodiment of the present invention, as shown in fig. 2, a method for positioning a water gauge character is provided, which is described by taking the example of applying the method to the computing device in fig. 1, and includes the following steps 11 to 14.
Step 11: and acquiring a plurality of water gauge character images to be recognized, which have time attributes and spatial position attributes and aim at the same ship.
The time attribute may be information of a time point when the water gauge character image is shot, and the spatial position attribute may be information of an azimuth, a focal length, and the like when the water gauge character image is shot. Specifically, a plurality of water gauge character images including water gauge characters can be acquired from videos shot for the same ship. In addition, the time attribute and the spatial position attribute in the present embodiment should be determined in accordance with the time and the spatial position when the water gauge character image is captured.
It should be noted that, in this embodiment, in order to more easily acquire the spatial position attribute in the water gauge character image, when the image or video containing the water gauge character is captured, the position of the camera is generally not changed, and the images with different spatial sizes containing the water gauge character are obtained by changing the size of the focal length at different time points.
Step 12: and generating an image set from the plurality of water gauge character images to be recognized according to the time attribute and the spatial position attribute.
Specifically, the water ruler character images with different spatial position attributes are generated into an image set according to a time sequence, as shown in the following formula (1):
L(t)＝[l1(t),l2(t),...,lk(t),...,lK(t)](1)
wherein t represents time, l (t) represents a water ruler character image, and k represents different focal length indexes.
For example, |1(t) may be a water ruler character image with 11 points focal length index of 1, l2(t) may be a water ruler character image with 11 points focal length index of 2,/3(t) may be a water gauge character image with 11-point focal length index of 3. It should be noted that, a plurality of focus-indexed water gauge character images may exist at a time point, each time point may generate an image set as shown in the above formula (1), and in addition, the time point may be used as a time instant or a time period.
Step 13: and inputting the image set into a pre-trained convolutional neural network, and combining the time attribute and the space position attribute of the water gauge character image in the image set by the convolutional neural network to obtain the water gauge character position in the water gauge character image to be recognized.
Specifically, the pixel information of each water ruler character image in the image set is input to a pre-trained convolutional neural network, and the pixel information may be information of a single channel, three channels, six channels, and the like of the water ruler character image, which is not limited specifically here. In the process of positioning the water gauge character positions in the water gauge character images, the convolutional neural network is used for positioning and tracking each water gauge character in each water gauge character image by combining the time attribute and the space position attribute of each water gauge character image so as to position the water gauge character.
It should be noted that in this embodiment, the convolutional neural network should at least contain convolutional layers and fully-connected layers. The convolution layer obtains pixel information of each water gauge character image, extracts and fuses features of the water gauge character image according to the pixel information of each water gauge character image, and the full-connection layer positions the water gauge character position in the water gauge character image according to the features obtained by the convolution layer.
Further, as an implementation manner of this embodiment, as shown in fig. 3, the step 13 may specifically include the following steps 131 to 133.
Step 131: and respectively extracting first features of the water gauge character images at different spatial positions at the same time point by using the first layer of convolutional layer, the second layer of convolutional layer and the third layer of convolutional layer in the pre-trained convolutional neural network.
Specifically, time information and spatial position information of the water gauge character images are obtained according to the image set, and the water gauge character images at different spatial positions can be water gauge character images with different focal lengths. In addition, the first feature is a shallow feature obtained from pixel information of the water gauge character image. For example, the first characteristic may be water gauge character brightness, water gauge character sharpness, water gauge character aspect, water body area, and the like.
Further, as an embodiment of the present embodiment, as shown in fig. 4, a plurality of first layer convolutional layers are provided in the convolutional neural network, the plurality of first layer convolutional layers are respectively in one-to-one correspondence with a plurality of water gauge character images at different spatial positions at the same time point, and the first layer convolutional layers extract features of the water gauge character images corresponding to the first layer convolutional layers.
Specifically, taking the example of the first layer of buildup layers being three, the first layer of buildup layers includes a first layer of buildup layer a, a first layer of buildup layer b, and a first layer of buildup layer c. The water gauge character image with the focal length index k being 1 is subjected to feature extraction by a first layer of convolution layer a, the features extracted by the first layer of convolution layer a are re-extracted by a second layer of convolution layer, the water gauge character image with the focal length index k being 2 is subjected to feature extraction by a first layer of convolution layer b, the water gauge character image with the focal length index k being 3 is subjected to feature extraction by a first layer of convolution layer c, and the features extracted by the first layer of convolution layer a and the features extracted by the first layer of convolution layer c are re-extracted by a third layer of convolution layer.
When the same time point further includes a water gauge character image with a focal length index k being 4, k being 5, and k being 6, the water gauge character image with the focal length index k being 4 is subjected to feature extraction by the first layer convolution layer a, and the features extracted by the first layer convolution layer a are re-extracted by the second layer convolution layer, the water gauge character image with the focal length index k being 5 is subjected to feature extraction by the first layer convolution layer b, the water gauge character image with the focal length index k being 6 is subjected to feature extraction by the first layer convolution layer c, and the features extracted by the first layer convolution layer a and the features extracted by the first layer convolution layer c are re-extracted by the third layer convolution layer.
That is, when the number of the first layer convolutional layers is already set, and the focal length index k is increased, the number of the first layer convolutional layers should be fixed, and taking the number of the first layer convolutional layers as three in this embodiment as an example, the increase of the focal length index k increases the number of the first layer convolutional layers a, the second layer convolutional layers b, and the third layer convolutional layers c processing the water gauge character image, and does not increase the number of the first layer convolutional layers.
In addition, it should be noted that, in the present embodiment, the features of each water gauge character image should be taken as a whole, and the time information and the spatial position information in each water gauge character image should be taken as the feature extraction reference information. In addition, in the present embodiment, the features obtained by extraction of the second-layer convolution layer and the third-layer convolution layer are taken as the first features.
In this embodiment, because the number of input water gauge character images is large and the data size is large, the convolutional layers are set according to the method shown in fig. 4, so that the data dimensionality is reduced while the effective first feature can be effectively extracted, and the calculation amount of subsequent convolutional layers in the convolutional neural network is reduced.
Step 132: and extracting and fusing the first features by a fourth convolutional layer and a fifth convolutional layer in the pre-trained convolutional neural network to obtain second features.
Wherein the second feature is a deep feature obtained from the first feature. The depth features may be waterline, areas of reflection, degree of blur of water gauge character handwriting, and the like. As shown in fig. 4, the fourth and fifth convolutional layers extract and fuse the first features obtained from the second and third convolutional layers to obtain the second features.
Step 133: and respectively positioning the water gauge characters in the multiple water gauge character images by the fully connected layer in the pre-trained convolutional neural network according to the second characteristic to obtain the positions of the water gauge characters.
And calculating to obtain the water gauge character position in the water gauge character image according to each characteristic weight in the full-connection layer. It should be noted that, in this embodiment, two fully-connected layers should be provided in the pre-trained convolutional neural network, where the first fully-connected layer locates the water gauge character position according to the second feature and the feature weight, and the second fully-connected layer relocates the water gauge character position.
In this embodiment, through the implementation of the above steps 131 to 133, the features of the water gauge character image are extracted according to the time attribute and the spatial position attribute of each water gauge character image, so as to effectively reduce the influence of the loss of image information on the positioning result when the water gauge character image is zoomed in the convolutional neural network, specifically, for the feature that the character of the water gauge character is small, the focal length is adjusted at different time points, and the water gauge character in the water gauge character image is positioned by using the focal length information.
Step 14: and carrying out identification processing on the water gauge characters in the water gauge character image to be recognized.
The water gauge characters in the water gauge character image can be specifically identified, and the identification processing mode is not specifically limited.
It should be noted that, in the process of identifying an image by a convolutional neural network, it is usually necessary to perform downsampling processing on the image, but the quality of the image is low after downsampling the image, when a water gauge character image subjected to zoom processing is obtained, a small water gauge character can be made large, the situation that the water gauge character is reduced and the lost information amount is large is effectively prevented from occurring, and further the water gauge character image is identified more effectively.
Through the implementation of the steps 11 to 14, the convolutional neural network positions the water gauge characters by combining the time attribute and the space attribute of the water gauge character image, so that the precision of positioning the water gauge character image can be effectively improved, and the problem of low water gauge character positioning precision of the water gauge character positioning method in the prior art is effectively solved.
Further, as an implementation manner of this embodiment, as shown in fig. 5, obtaining a pre-trained convolutional neural network may include the following steps 21 to 25.
Step 21: acquiring a plurality of sample water gauge character images with time attributes and spatial position attributes for the same ship.
Wherein, the sample water gauge character image should contain multiple categories, each category should have positive and negative samples for testing.
Further, as an implementation manner of this embodiment, the sample water gauge character image includes at least one of water gauge character images photographed in different orientations, water gauge character images set at different time points, water gauge character images with water reflection, and water gauge character images without water reflection. The more the number of the sample water gauge character images is, the more the types are, and the better the training effect of the convolutional neural network is.
In this embodiment, the sample water gauge character image is set to contain at least one of water gauge character images shot in different directions, water gauge character images set at different time points, water gauge character images with water reflection and water gauge character images without water reflection, so that the convolutional neural network can identify a specific environment in which water gauge characters exist when the characteristics of the sample water gauge character images are subsequently extracted, and the positioning accuracy is improved.
Step 22: and generating a sample image set from the plurality of sample water gauge character images according to the time attribute and the spatial position attribute.
Since the method for generating the sample image set in step 22 is similar to the method for generating the image set in step 12, the detailed description is omitted here.
Step 23: and inputting the sample image set into a convolutional neural network, and combining the time attribute and the spatial position attribute of the sample water gauge character image in the sample image set by the convolutional neural network to obtain the water gauge character position in the sample water gauge character image.
Since the method for positioning the water gauge character position in the sample water gauge character image in step 23 is similar to the method for positioning the water gauge character position in the water gauge character image in step 13, the detailed description is omitted here.
Step 24: and calculating the positioning result of the predicted water gauge character positions of the plurality of sample water gauge character images through the loss function to obtain a quality evaluation coefficient of the convolutional neural network.
In step 24, a prediction positioning frame used for marking the predicted water gauge character position of the sample water gauge image is obtained according to the predicted water gauge character position, an actual positioning frame used for marking the predicted water gauge character position of the sample water gauge image is obtained, and finally the prediction positioning frame and the actual positioning frame are calculated through softmax logistic regression to obtain the quality evaluation coefficient of the convolutional neural network.
Further, as an implementation manner of the present embodiment, the quality evaluation coefficient obtained according to the softmax logistic regression calculation is as shown in the following formula (2):
L(x,c,l,g)＝
1/N(Lconf(x,c)+a1*Lloc1(x,l,g)+a2*Lloc2(x,l,g)+…+an*Llocn(x,l,g)) (2)
wherein N represents the number of prior character frames matched with the actual character frame, x represents a matching indication matrix, c represents confidence, L represents a predicted character frame, g represents the actual character frame, and L representslocnRepresenting the loss of position of the nth predicted character box, LconfRepresents the Softmax logistic regression loss, anA weight representing the loss of the nth predicted box position.
Further, LlocnSpecifically, the difference between the center coordinates and the width and height of the nth predicted character frame and the actual character frame is expressed.
In the present embodiment, the quality evaluation coefficient is obtained by the calculation in (2) above, and softmax logistic regression is combined with the convolutional neural network to evaluate the degree of goodness and badness of the convolutional neural network.
It should be noted that, in this embodiment, a plurality of sample water gauge character images with different focal lengths and belonging to different spatial position attributes are used for inputting, and when prediction is not accurate for a certain class of problems, a sample water gauge character image related to the class of problems may be input for the class of problems. For example, the number of the sample water gauge character images with the introduction of the reflection is reduced, and when most of the sample water gauge character images without the reflection are trained, the convolutional neural network extracts the features of the sample water gauge character images without the reflection, suppresses the reflection character features of the sample water gauge character images with the reflection and the character projection, and the L shown in formula (2) is used as the reflection character features of the sample water gauge character images with the character projectionloc1(x,l,g)、Lloc2(x,l,g)、Lloc3(x,l,g)、…Llocn(x,l,g) Only one sample water gauge character image containing the reflection is provided, and the reflection characters in the reflection sample water gauge character image are few, so that the influence on the whole loss function is small and negligible, and the reflection characters cannot be detected in the result. By the training method, the problem of inaccurate prediction for a certain class of problems can be effectively solved.
Step 25: and when the quality evaluation coefficient does not reach the preset threshold value, adjusting the feature weight of each feature in the convolutional neural network, repeatedly inputting the sample image set into the convolutional neural network to obtain the predicted water gauge character position until the quality evaluation coefficient reaches the preset threshold value to obtain the pre-trained convolutional neural network.
The preset threshold value can be preset, when the quality evaluation coefficient reaches the preset threshold value, the current convolutional neural network meets the requirement for positioning the water gauge characters, and the current convolutional neural network is determined as a pre-trained convolutional neural network for positioning the subsequent water gauge characters.
Further, as an implementation manner of this embodiment, in step 25, the feature weight of each feature in the convolutional neural network may be specifically adjusted by an accelerated gradient descent algorithm.
Specifically, the feature weight of each feature in the convolutional neural network may be adjusted according to the following equations (3) and (4):
Vt+1＝u*Vt-a gradient L (W)t+u*Vt) (3)
Wt+1＝Wt+Vt+1(4)
Where t represents the index of each cyclic parameter update, such as t-0, t-1, t-2, Vt+1Is the gradient value, V, of the next cycletRepresenting the gradient value of the cycle, L representing the loss function, WtRepresents the weight of the current cycle, u represents the basis weight, a represents the total weight, and W represents the weight of the next cyclet+1Is determined by the weight W of this timetAnd the gradient value V of the next cyclet+1The result is obtained from equation (4).
In the embodiment, the feature weight of each feature in the convolutional neural network is adjusted through the accelerated gradient descent algorithm, so that the updating speed of the feature weight can be effectively increased, and the training process is accelerated.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
A second embodiment of the present invention provides a water gauge character positioning device, which corresponds to the water gauge character positioning method provided in the first embodiment one to one.
Further, as shown in fig. 6, the water gauge character positioning apparatus includes a water gauge character image acquisition module 41, an image set acquisition module 42, a water gauge character position acquisition module 43, and an identification processing module 44. The functional modules are explained in detail as follows:
the water gauge character image acquisition module 41 is used for acquiring a plurality of water gauge character images to be recognized, which have time attributes and spatial position attributes, of the same ship;
the image set acquisition module 42 is configured to generate an image set from the multiple water gauge character images to be recognized according to the time attribute and the spatial position attribute;
the water gauge character position acquiring module 43 is configured to input the image set to a convolutional neural network, and the pre-trained convolutional neural network combines a time attribute and a space position attribute of the water gauge character image in the image set to acquire a water gauge character position in the water gauge character image to be recognized;
and the identification processing module 44 is used for identifying the water gauge characters in the water gauge character image to be recognized.
Further, as an implementation manner of the present embodiment, the water gauge character position acquiring module 43 may include a first feature acquiring unit, a second feature acquiring unit, and a water gauge character position acquiring unit. The functional units are explained in detail as follows:
the first feature acquisition unit is used for respectively extracting first features of the water gauge character images at different spatial positions at the same time point from a first layer of convolutional layer, a second layer of convolutional layer and a third layer of convolutional layer in a pre-trained convolutional neural network;
the second characteristic acquisition unit is used for extracting and fusing the first characteristics by a fourth convolutional layer and a fifth convolutional layer in the pre-trained convolutional neural network to obtain second characteristics;
and the water gauge character position acquisition unit is used for positioning the water gauge characters in the multiple water gauge character images according to the second characteristics by the fully-connected layer in the pre-trained convolutional neural network to obtain the water gauge character positions.
Further, as an implementation manner of the present embodiment, the first feature acquisition unit includes a first feature acquisition subunit. The first feature acquisition subunit is described in detail as follows:
the first feature obtaining subunit is used for setting a plurality of first-layer convolutional layers in the convolutional neural network, enabling the plurality of first-layer convolutional layers to correspond to the plurality of water gauge character images at different spatial positions at the same time point one by one, and extracting features of the water gauge character images corresponding to the first-layer convolutional layers by the first-layer convolutional layers.
Further, as an implementation manner of this embodiment, the water gauge character positioning apparatus further includes: the device comprises a sample water gauge character image acquisition module, a sample image set acquisition module, a sample water gauge character position acquisition module, a quality evaluation coefficient acquisition module and a convolutional neural network determination module. The functional modules are explained in detail as follows:
the system comprises a sample water gauge character image acquisition module, a time attribute acquisition module and a space position attribute acquisition module, wherein the sample water gauge character image acquisition module is used for acquiring a plurality of sample water gauge character images which have time attributes and space position attributes and aim at the same ship;
the sample image set acquisition module is used for generating a sample image set from the plurality of sample water gauge character images according to the time attribute and the spatial position attribute;
the system comprises a sample water gauge character position acquisition module, a convolution neural network and a data processing module, wherein the sample water gauge character position acquisition module is used for inputting a sample image set into the convolution neural network, and the convolution neural network is used for acquiring a water gauge character position in the sample water gauge character image by combining a time attribute and a space position attribute of the sample water gauge character image in the sample image set;
the quality evaluation coefficient acquisition module is used for calculating the positioning results of the predicted water gauge character positions of the multiple sample water gauge character images through the loss function to obtain the quality evaluation coefficient of the convolutional neural network;
and the convolutional neural network determining module is used for adjusting the feature weight of each feature in the convolutional neural network when the quality evaluation coefficient does not reach the preset threshold value, repeatedly inputting the sample image set into the convolutional neural network to obtain the predicted water gauge character position until the quality evaluation coefficient reaches the preset threshold value, and thus obtaining the pre-trained convolutional neural network.
Further, as an implementation manner of the present embodiment, the quality evaluation coefficient acquisition module may include a quality evaluation coefficient acquisition unit. The quality evaluation coefficient acquisition unit is explained in detail as follows:
a quality evaluation coefficient acquisition unit for calculating a quality evaluation coefficient calculated according to the following formula (2):
L(x,c,l,g)＝
1/N(Lconf(x,c)+a1*Lloc1(x,l,g)+a2*Lloc2(x,l,g)+…+an*Llocn(x,l,g)) (2)
wherein N represents the number of prior character frames matched with the actual character frame, x represents a matching indication matrix, c represents confidence, L represents a predicted character frame, g represents the actual character frame, and L representslocnRepresenting the loss of position of the nth predicted character box, LconfRepresents the Softmax logistic regression loss, anA weight representing the loss of the nth predicted box position.
Further, LlocnSpecifically, the difference between the center coordinates and the width and height of the nth predicted character frame and the actual character frame is expressed.
Further, as an implementation manner of the present embodiment, the convolutional neural network determining module may include a feature weight adjusting unit. The detailed description of the feature weight adjustment unit is as follows:
and the characteristic weight adjusting unit is used for adjusting the characteristic weight of each characteristic in the convolutional neural network through an accelerated gradient descent algorithm.
Further, as an implementation manner of this embodiment, the sample water gauge character image includes at least one of water gauge character images photographed in different orientations, water gauge character images set at different time points, water gauge character images with water reflection, and water gauge character images without water reflection.
For the specific definition of the water gauge character positioning device, reference may be made to the above definition of the water gauge character positioning method, which is not described herein again. The modules/units in the water gauge character positioning device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computing device, and can also be stored in a memory in the computing device in a software form, so that the processor can call and execute operations corresponding to the modules.
A third embodiment of the present invention provides a computing device, the internal structure of which may be as shown in fig. 7. The computing device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computing device is configured to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computing device is used for storing data involved in the water gauge character positioning method. The network interface of the computing device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement the water gauge character positioning method provided by the first embodiment of the present invention.
A fourth embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, the computer program, when executed by a processor, implementing the steps of the water gauge character positioning method provided by the first embodiment of the present invention, such as steps 11 to 13 shown in fig. 2, steps 131 to 133 shown in fig. 3, and steps 21 to 25 shown in fig. 5. Alternatively, the computer program, when executed by the processor, implements the functions of the modules/units of the water gauge character positioning method provided in the first embodiment described above. To avoid repetition, further description is omitted here.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A water gauge character positioning method is characterized by comprising the following steps:
acquiring a plurality of water gauge character images to be recognized, which have time attributes and spatial position attributes and aim at the same ship;
generating an image set from the plurality of water gauge character images to be recognized according to the time attribute and the spatial position attribute;
inputting the image set into a pre-trained convolutional neural network, wherein the convolutional neural network combines the time attribute and the spatial position attribute of the water gauge character image in the image set to obtain the water gauge character position in the water gauge character image to be recognized;
and identifying the water gauge characters in the water gauge character image to be recognized.
2. The water ruler character positioning method of claim 1, wherein the convolutional neural network combining the temporal attributes and the spatial location attributes of the water ruler character images in the image set to obtain water ruler character locations in the water ruler character images comprises:
respectively extracting first features of the water gauge character images at different spatial positions at the same time point by using a first layer of convolutional layer, a second layer of convolutional layer and a third layer of convolutional layer in the pre-trained convolutional neural network;
extracting and fusing the first features by a fourth convolutional layer and a fifth convolutional layer in the pre-trained convolutional neural network to obtain second features;
and the full-connection layer in the pre-trained convolutional neural network respectively positions the water gauge characters in the water gauge character images according to the second characteristics to obtain the positions of the water gauge characters.
3. The method according to claim 2, wherein a plurality of first-layer convolutional layers are provided in the convolutional neural network, the plurality of first-layer convolutional layers are respectively in one-to-one correspondence with a plurality of water gauge character images at different spatial positions at the same time point, and the first-layer convolutional layers extract features of the water gauge character images corresponding to the first-layer convolutional layers.
4. The method of claim 1, wherein obtaining the pre-trained convolutional neural network comprises:
acquiring a plurality of sample water gauge character images with time attributes and spatial position attributes for the same ship;
generating a sample image set from the plurality of sample water gauge character images according to the time attribute and the spatial position attribute;
inputting the sample image set to the convolutional neural network, the convolutional neural network combining the temporal attributes and the spatial location attributes of the sample water gauge character images in the sample image set to obtain water gauge character locations in the sample water gauge character images;
calculating the positioning result of the predicted water gauge character positions of the sample water gauge character images through a loss function to obtain a quality evaluation coefficient of the convolutional neural network;
and when the quality evaluation coefficient does not reach a preset threshold value, adjusting the feature weight of each feature in the convolutional neural network, and repeatedly inputting the sample image set into the convolutional neural network to obtain a predicted water gauge character position until the quality evaluation coefficient reaches the preset threshold value to obtain the pre-trained convolutional neural network.
5. The water gauge character positioning method according to claim 4, wherein the quality evaluation coefficient is calculated according to the following formula (1):
L(x,c,l,g)＝1/N(Lconf(x,c)+a1*Lloc1(x,l,g)+a2*Lloc2(x,l,g)+…+an*Llocn(x,l,g))(1)
wherein N represents the number of prior character frames matched with the actual character frame, x represents a matching indication matrix, c represents confidence, L represents a predicted character frame, g represents the actual character frame, and L representslocnRepresenting the loss of position of the nth predicted character box, LconfRepresents the Softmax logistic regression loss, anA weight representing the loss of the nth predicted box position.
6. The method of claim 5, wherein the feature weights of the features in the convolutional neural network are adjusted by an accelerated gradient descent algorithm.
7. The water gauge character positioning method according to claim 4, wherein the sample water gauge character images comprise at least one of water gauge character images photographed in different orientations, water gauge character images set at different time points, water gauge character images with water body reflection, and water gauge character images without water body reflection.
8. A water gauge character positioning device, comprising:
the water gauge character image acquisition module is used for aiming at a plurality of water gauge character images to be recognized of the same ship and having time attributes and spatial position attributes;
the image set acquisition module is used for generating an image set from the plurality of water gauge character images to be identified according to the time attribute and the spatial position attribute;
the water gauge character position acquisition module is used for inputting the image set into a convolutional neural network, and the pre-trained convolutional neural network is combined with the time attribute and the space position attribute of the water gauge character image in the image set to obtain the water gauge character position in the water gauge character image to be recognized;
and the identification processing module is used for identifying the water gauge characters in the water gauge character image to be identified.
9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of the water ruler character positioning method of any of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the water gauge character positioning method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055194.0A CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010055194.0A CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111275039A true CN111275039A (en) | 2020-06-12 |
| CN111275039B CN111275039B (en) | 2023-05-16 |
Family
ID=71001683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010055194.0A Active CN111275039B (en) | 2020-01-17 | 2020-01-17 | Water gauge character positioning method, device, computing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111275039B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663811A (en) * | 2022-03-24 | 2022-06-24 | 中国水利水电科学研究院 | Water surface line extraction method using water gauge reflection |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107516096A (en) * | 2016-06-15 | 2017-12-26 | 阿里巴巴集团控股有限公司 | A kind of character identifying method and device |
| WO2018090641A1 (en) * | 2016-11-15 | 2018-05-24 | 平安科技（深圳）有限公司 | Method, apparatus and device for identifying insurance policy number, and computer-readable storage medium |
| CN108875722A (en) * | 2017-12-27 | 2018-11-23 | 北京旷视科技有限公司 | Character recognition and identification model training method, device and system and storage medium |
| CN109977980A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of method for recognizing verification code and device |
-
2020
- 2020-01-17 CN CN202010055194.0A patent/CN111275039B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107516096A (en) * | 2016-06-15 | 2017-12-26 | 阿里巴巴集团控股有限公司 | A kind of character identifying method and device |
| WO2018090641A1 (en) * | 2016-11-15 | 2018-05-24 | 平安科技（深圳）有限公司 | Method, apparatus and device for identifying insurance policy number, and computer-readable storage medium |
| CN108875722A (en) * | 2017-12-27 | 2018-11-23 | 北京旷视科技有限公司 | Character recognition and identification model training method, device and system and storage medium |
| CN109977980A (en) * | 2017-12-28 | 2019-07-05 | 航天信息股份有限公司 | A kind of method for recognizing verification code and device |
Non-Patent Citations (1)
| Title |
|---|
| 李翊;兰华勇;严华;: "基于图像处理和BP神经网络的水位识别研究" * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114663811A (en) * | 2022-03-24 | 2022-06-24 | 中国水利水电科学研究院 | Water surface line extraction method using water gauge reflection |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111275039B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112990432B (en) | Target recognition model training method and device and electronic equipment | |
| KR102591961B1 (en) | Model training method and device, and terminal and storage medium for the same | |
| CN108960211B (en) | Multi-target human body posture detection method and system | |
| WO2020186678A1 (en) | Three-dimensional map constructing method and apparatus for unmanned aerial vehicle, computer device, and storage medium | |
| US11093737B2 (en) | Gesture recognition method and apparatus, electronic device, and computer-readable storage medium | |
| CN111353512B (en) | Obstacle classification method, obstacle classification device, storage medium and computer equipment | |
| CN111814902A (en) | Target detection model training method, target identification method, device and medium | |
| CN109871821B (en) | Pedestrian re-identification method, device, equipment and storage medium of self-adaptive network | |
| CN111144398A (en) | Target detection method, target detection device, computer equipment and storage medium | |
| JP2022540101A (en) | POSITIONING METHOD AND APPARATUS, ELECTRONIC DEVICE, COMPUTER-READABLE STORAGE MEDIUM | |
| CN111553182A (en) | Ship retrieval method and device and electronic equipment | |
| CN110660078A (en) | Object tracking method and device, computer equipment and storage medium | |
| CN112884782A (en) | Biological object segmentation method, apparatus, computer device and storage medium | |
| CN111814846B (en) | Training method and recognition method of attribute recognition model and related equipment | |
| CN115797735A (en) | Target detection method, device, equipment and storage medium | |
| CN111275039A (en) | Water gauge character positioning method and device, computing equipment and storage medium | |
| CN113537020A (en) | Complex SAR image target identification method based on improved neural network | |
| CN115115552B (en) | Image correction model training method, image correction device and computer equipment | |
| CN112699809B (en) | Vaccinia category identification method, device, computer equipment and storage medium | |
| CN112926616A (en) | Image matching method and device, electronic equipment and computer-readable storage medium | |
| CN115761888A (en) | Tower crane operator abnormal behavior detection method based on NL-C3D model | |
| CN115984646A (en) | Distributed target detection method and device for remote sensing cross-satellite observation and satellite | |
| CN110751163A (en) | Target positioning method and device, computer readable storage medium and electronic equipment | |
| CN112488053B (en) | Face recognition method, device, robot and storage medium | |
| CN112101303B (en) | Image data processing method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |