WO2023231022A1

WO2023231022A1 - Image recognition method, self-moving device and storage medium

Info

Publication number: WO2023231022A1
Application number: PCT/CN2022/096975
Authority: WO
Inventors: 王雷; 陈熙
Original assignee: 深圳市正浩创新科技股份有限公司
Priority date: 2022-06-02
Filing date: 2022-06-02
Publication date: 2023-12-07
Also published as: CN115151950A

Abstract

An image recognition method, comprising: acquiring an image feature map of an image to be recognized, the image feature map comprising a plurality of image channels; sampling pixel points of the image channels to obtain a plurality of channel feature maps corresponding to the image channels, the total number of pixel points of the plurality of channel feature maps corresponding to the image channels being equal to the total number of the pixel points of the image channels; using all the channel feature maps corresponding to the image feature map as an input feature map of convolution processing; and performing a preset number of times of convolution processing on the input feature map to obtain the recognition result of said image.

Description

Image recognition methods, mobile devices and storage media

Technical field

The present application relates to the field of artificial intelligence, and in particular to an image recognition method, a mobile device and a computer-readable storage medium.

Background technique

The statements herein merely provide background information relevant to the present application and do not necessarily constitute exemplary techniques.

With the continuous advancement of computer technology and artificial intelligence technology, automatic working systems for mobile devices have slowly begun to enter people's lives. When mobile devices are working, they usually need to collect images in real time and segment and identify the image areas to achieve functions such as determining the work area and planning the path within the work area. For example, panoramic segmentation or instance segmentation in deep learning methods can be used to segment the collected images into working areas and non-working areas, thereby controlling the work of the robot based on the identified working areas. However, this method of image segmentation and recognition requires a large amount of calculation and has low segmentation efficiency.

Contents of the invention

Various embodiments of the present application provide an image recognition method, a mobile device and a storage medium.

In a first aspect, this application provides an image recognition method, which method includes:

Obtain an image feature map of the image to be recognized, where the image feature map includes multiple image channels;

The pixels of each image channel are sampled to obtain multiple channel feature maps corresponding to each image channel; the total number of pixels of the multiple channel feature maps corresponding to each image channel is equal to the pixels of each image channel. Total number of points;

Use all channel feature maps corresponding to the image feature map as input feature maps for convolution processing;

Perform preset subconvolution processing on the input feature map to obtain the recognition result of the image to be recognized.

In a second aspect, this application also provides an image recognition device, which includes:

An image acquisition module configured to acquire an image feature map of the image to be recognized, where the image feature map includes multiple image channels;

a channel sampling module configured to sample the pixels of each image channel to obtain multiple channel feature maps corresponding to each image channel; the total number of pixels of the multiple channel feature maps corresponding to each image channel Equal to the total number of pixels in each image channel;

An input determination module configured to use all channel feature maps corresponding to the image feature map as input feature maps for convolution processing;

The convolution processing module is configured to perform preset subconvolution processing on the input feature map to obtain the recognition result of the image to be recognized.

In a third aspect, this application also provides a self-mobile device, the computer device includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and execute the The computer program implements the above image recognition method.

In a fourth aspect, the present application also provides a computer-readable storage medium that stores a computer program. When executed by a processor, the computer program causes the processor to implement the above-mentioned image recognition method. .

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below. Other features, objects and advantages of the application will become apparent from the description, drawings and claims.

Description of the drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, which are of great significance to this field. Ordinary technicians can also obtain other drawings based on these drawings without exerting creative work.

Figure 1 is a schematic flow chart of an image recognition method provided by an embodiment of the present application.

Figure 2 is a schematic diagram of an image feature map provided by an embodiment of the present application.

FIG. 3a is a schematic diagram of a channel diagram of an image channel provided by an embodiment of the present application.

Figure 3b is a schematic diagram of the channel feature map corresponding to the image channel provided by the embodiment of the present application.

Figure 4 is a schematic flow chart for obtaining a channel characteristic map provided by an embodiment of the present application.

Figure 5a is a schematic diagram of an image to be recognized provided by an embodiment of the present application.

Figure 5b is a schematic diagram of a recognition result of an image to be recognized provided by an embodiment of the present application.

Figure 6 is a schematic block diagram of an image recognition device provided by an embodiment of the present application.

FIG. 7 is a schematic structural block diagram of a mobile device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all contents and operations/steps, nor are they necessarily performed in the order described. For example, some operations/steps can also be decomposed, combined or partially merged, so the actual order of execution may change according to actual conditions.

It should be understood that the terminology used in the specification of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly dictates otherwise.

It will also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Embodiments of the present application provide an image recognition method, device, mobile device and storage medium. Image recognition methods can reduce the calculation amount of image recognition, thereby improving the efficiency of image recognition. Among them, the self-mobile equipment can be small self-mobile equipment such as lawn mowers, patrol robots, and mine-clearing robots, or self-mobile equipment for sanitation and cleaning, self-mobile food delivery equipment, and self-mobile equipment for agricultural sowing.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The following embodiments and features in the embodiments may be combined with each other without conflict.

Please refer to Figure 1, which is a schematic flow chart of an image recognition method provided by an embodiment of the present application. This image recognition method obtains the channel feature map by sampling the image feature map of the image to be recognized according to the image channel of the image to be recognized, and uses the channel feature map as the input of the convolution process to obtain the recognition result, which can reduce the need for convolution processing. The size of the channel feature map input at the same time can reduce the calculation amount of image recognition and improve the efficiency of image recognition.

As shown in Figure 1, the image recognition method specifically includes steps S101 to S104.

S101. Obtain the image feature map of the image to be recognized. The image feature map includes multiple image channels.

The image to be recognized captured by the camera device can be obtained through the camera device provided on the mobile device. For example, when the mobile device is a lawn mower, the image to be recognized can be obtained through a camera installed on the lawn mower. In this case, the image to be recognized is a grass image captured by the lawn mower. Or upload the image to be recognized through Bluetooth, WiFi or locally. The method of obtaining the image to be recognized is not limited here. The image feature map is used to characterize the image features of the image to be recognized. It can be the original image of the image to be recognized, or the image obtained after feature extraction of the image to be recognized. The image features can include the color features, texture features, and shape of the image. features and spatial relationship features.

The image feature map includes multiple image channels, and the channel maps corresponding to each image channel are superimposed to form an image feature map. Please refer to Figure 2. Figure 2 is a schematic diagram of an image feature map provided by an embodiment of the present application. As shown in Figure 2, the image channel may include three channels: image channel 1, image channel 2, and image channel 3. The corresponding channel maps of image channel 1, image channel 2, and image channel 3 are superimposed to form an image feature map. In the specific implementation process, the multiple image channels may be, for example, three image channels: B image channel, G image channel and R image channel read using opencv (Open Source Computer Vision Library, a cross-platform computer vision library).

S102. Sample the pixels of each image channel to obtain multiple channel feature maps corresponding to each image channel. The total number of pixels of the multiple channel feature maps corresponding to each image channel is equal to the total number of pixels of each image channel. .

For each image channel in the image feature map, the pixel points of the channel map of the image channel are sampled, thereby obtaining multiple channel feature maps corresponding to each image channel.

After sampling, the total number of pixels in the multiple channel feature maps obtained must be the same as the number of pixels in the channel map corresponding to the image channel. That is to say, when sampling, each pixel in the channel map must be Sampling is performed to ensure the integrity of the pixels in the sampled channel feature map, thereby ensuring the integrity of the image information in the image to be recognized without losing image information.

Please refer to Figures 3a and 3b. Figure 3a is a schematic diagram of a channel diagram of an image channel provided by an embodiment of the present application, and Figure 3b is a schematic diagram of a channel feature map corresponding to the image channel provided by an embodiment of the present application. As shown in Figure 3a, the numbers on the pixels in the channel map represent the labels of the pixels.

When sampling pixels for this image channel, pixel 1, pixel 3, pixel 9 and pixel 11 can be used as sampled pixels to obtain the channel feature map 11 corresponding to the image channel; pixel 2 , pixel 4, pixel 10 and pixel 12 are used as sampled pixels to obtain the channel feature map 12 corresponding to the image channel; pixel 5, pixel 7, pixel 13 and pixel 15 are used as sampled pixel point, the channel feature map 13 corresponding to the image channel is obtained; using pixel point 6, pixel point 8, pixel point 14 and pixel point 16 as the sampled pixel points, the channel feature map 14 corresponding to the image channel is obtained.

After multiple samplings are performed until all the pixels in the channel map of the image channel are sampled, the sampling is stopped, and the obtained channel feature map 11, channel feature map 12, channel feature map 13 and channel feature map 14 are used as the The channel feature map corresponding to the image channel.

In one embodiment, please refer to FIG. 4 , which is a schematic flow chart for obtaining a channel characteristic map provided by an embodiment of the present application. As shown in Figure 4, step S102 may include step S1021 and step S1022.

S1021. Group the pixels of each image channel to obtain multiple sampled pixel groups of each image channel.

For each image channel, the pixels of the channel map of the image channel are grouped to obtain multiple sampling pixel groups corresponding to each image channel. Among them, the number of pixels in each sampling pixel group is the same.

In the specific implementation process, the pixels of the channel map of each image channel can be sampled and grouped in an orderly manner. The sampling method may be to perform interval sampling with one pixel as the starting point, or to perform interval sampling with a group of adjacent pixels as the starting point. In a specific implementation process, when performing interval sampling with a group of adjacent pixels as the starting point, multiple adjacent pixels may be grouped as a group, and interval sampling may be performed with this group of pixels as the starting point. For example, the

adjacent pixels

1 and 5 in Figure 3a can be taken as a group and sampled every other pixel. Then, the sampled pixel group includes

pixels

3 and 7.

In one embodiment, the step of grouping the pixels of each image channel to obtain multiple sampled pixel groups of each image channel may include:

The starting pixel point set is determined among all pixels in the image channel according to the preset sampling interval N. The starting pixel point set is a pixel matrix including (N+1)*(N+1) pixels. The first pixel point set of the pixel matrix Each pixel point is any boundary vertex of the corresponding image channel; taking each pixel point in the starting pixel point set as the starting pixel point, the pixel points of the image channel are sampled at the preset sampling interval, and the pixel point of each starting pixel point is obtained. Sample a group of pixels.

When sampling, for an image channel, first determine the starting pixel set from the pixels in the channel map of the image channel. The starting pixel set includes multiple pixels. Each pixel in the starting pixel set The point is the starting point of sampling every time the pixels in the channel map of the image channel are sampled, that is, the starting pixel of each sampling.

In the embodiment of the present application, the starting pixel point set can be determined according to the boundary of the channel map of the image channel and the preset sampling interval N, that is, it is a pixel matrix including (N+1)*(N+1) pixel points, The first pixel point of the pixel matrix is any boundary vertex of the corresponding image channel.

In the specific implementation process, a starting pixel point can be determined based on any boundary vertex of the channel graph of the image channel, and then based on the starting pixel point, pixel points near the starting pixel point are selected according to the preset sampling interval. As the starting pixel point, the starting pixel point set is obtained.

For example, as shown in Figure 3a, the channel map of the image channel can be based on the pixel point 1 in the upper left boundary vertex, the pixel point 4 in the upper right boundary vertex, the pixel point 13 in the lower left boundary vertex, and the pixel point 16 in the lower right boundary vertex. , you can select any boundary vertex as the first pixel point of the pixel matrix.

If the preset sampling interval N is 1, and pixel 1 in the upper left boundary vertex is the first pixel, the pixel matrix of (1+1)*(1+1) is obtained, which is the pixel in the pixel matrix where 1 is located. Point 2, pixel point 5 and pixel point 6, and these three pixel points are also used as the starting pixel points. At this time, the obtained starting pixel point set has a total of four pixel points, namely pixel point 1, pixel point 2, Pixel 5 and Pixel 6.

If the preset sampling interval N is 1, and pixel 4 in the upper right boundary vertex is the first pixel, then the pixel matrix of (1+1)*(1+1) is obtained, which is the pixel in the pixel matrix where 4 is located. Point 3, pixel point 7 and pixel point 8, and these three pixel points are also used as the starting pixel points. At this time, the obtained starting pixel point set has a total of four pixel points, namely pixel point 3, pixel point 4, Pixel point 7 and pixel point 8.

After determining the starting pixel point set, sampling is performed based on the starting pixel point in the starting pixel point set and the preset sampling interval. Specifically, a pixel in the starting pixel set is used as the starting point for sampling, the pixels in the channel map of the image channel are sampled according to the preset sampling interval, and when this sampling is completed, the pixels obtained by this sampling are The pixels are used as a sampling pixel group, and then another pixel in the starting pixel set is used as the sampling starting point for the next sampling, until all pixels in the starting pixel set are used, and an image channel can be obtained The corresponding multiple sampling pixel groups. By executing the above process for each image channel, multiple sampling pixel groups corresponding to each image channel can be obtained.

In a specific implementation process, the preset sampling interval may be set in advance, and the value of the preset sampling interval needs to be set so that after the sampling is completed, the number of pixels in each sampling pixel group is the same.

Since the setting of the preset sampling interval value needs to ensure that the number of pixels in each sampling pixel group is the same after the sampling is completed, therefore, in one embodiment, the step of obtaining the image feature map of the image to be recognized can be It includes: preprocessing the image to be recognized, the preprocessing including adjusting the resolution of the image to be recognized; performing feature extraction on the preprocessed image to be recognized to obtain an image feature map.

Among them, preprocessing the image to be recognized can also include cropping, smoothing, filtering, etc. of the image to be recognized. By preprocessing the image to be recognized, irrelevant information in the image to be recognized can be eliminated, useful real information can be extracted, and relevant information can be enhanced. Detectability of information and maximizing data simplification, thereby increasing the reliability of image recognition.

For an image, the higher the resolution of the image, the more pixels there are in the image, and the clearer the image. Before image recognition is performed on the image to be recognized, the resolution of the image to be recognized is adjusted, and the image to be recognized that needs to be image recognized is adjusted to the preset resolution, so that when the image to be recognized is subjected to image recognition, the pixels involved in the image recognition are The number of samples is the same, so there is no need to adjust the value of the preset sampling interval according to the resolution of the image to be recognized, which can reduce the calculation amount of image recognition during subsequent image recognition. That is to say, by preprocessing the image to be recognized, the number of pixels in the image to be recognized is made consistent, and then when sampling the image to be recognized, it is ensured that the same sampling interval can adapt to all the images to be recognized, without the need to The sampling interval is adjusted according to the size of each image to be recognized, thereby reducing the amount of calculation during image recognition and thereby improving the efficiency of image recognition.

The above-mentioned sampling method will be illustrated below with reference to Figures 3a and 3b. If the preset sampling interval is set to 1, and the starting pixels in the starting pixel set include pixel 1, pixel 2, pixel 5 and pixel 6, then the sampling process is:

Pixel 1 in the starting pixel set is used as the starting pixel for sampling. Starting from pixel 1, horizontal sampling is performed every other pixel. The resulting sampled pixel is pixel 3. At this time, there is no pixel in the first row of pixels. Other sampleable points; perform longitudinal sampling every other pixel from the position of pixel 1, and the resulting sampled pixel is pixel 9, and then perform horizontal sampling every other pixel, and the obtained sampled pixel is pixel 11. At this time, the sampled pixel is pixel 11. There are no other sampleable points in the three rows of pixels; at this time, when pixel 1 is used as the starting pixel, the pixels in the channel map of the image channel have no other sampleable points. After completing this sampling, the pixels are obtained 1 is the sampling pixel group of the starting pixel, and the sampling pixel group includes pixel 1, pixel 3, pixel 9 and pixel 11.

Pixel 2 in the starting pixel set is used as the starting pixel for sampling. Starting from pixel 2, horizontal sampling is performed every 1. The resulting sampled pixel is pixel 4. At this time, there is no pixel in the first row of pixels. Other sampleable pixels; perform vertical sampling every 1 from the position of pixel 2, and the resulting sampled pixel is pixel 10, and then perform horizontal sampling every 1, and the resulting sampled pixel is pixel 12 , at this time, there are no other sampleable points in the third row of pixels; at this time, when pixel 2 is used as the starting pixel, the pixels in the channel map of the image channel have no other sampleable points, and this sampling is completed. A sampling pixel group starting from pixel 2 is obtained. The sampling pixel group includes pixel 2, pixel 4, pixel 10 and pixel 12.

Pixel 5 in the starting pixel set is used as the starting pixel for sampling, and horizontal sampling is performed every 1 starting from pixel 5. The resulting sampled pixel is pixel 7. At this time, there is no pixel in the first row of pixels. Other sampleable pixels; perform vertical sampling every 1 from the position of pixel 5, and the pixel of the sampled image obtained is pixel 13, and then perform horizontal sampling every 1, and the sampled pixel obtained is pixel 13. 15. At this time, there are no other sampleable pixels in the third row of pixels; at this time, when pixel 5 is used as the starting pixel, the pixels in the channel map of the image channel have no other sampleable pixels. After completing this sampling, a sampling pixel group starting from pixel 5 is obtained. The sampling pixel group includes pixel 5, pixel 7, pixel 13 and pixel 15.

Pixel 6 in the starting pixel set is used as the starting pixel for sampling. Starting from pixel 6, horizontal sampling is performed every 1. The resulting sampled pixel is pixel 8. At this time, in the first row of pixels There are no other pixels that can be sampled; perform vertical sampling every 1 from the position of pixel 6, and the resulting sampled pixel is pixel 14, and then perform horizontal sampling every 1, and the resulting sampled pixel is pixel 14. 16. At this time, there are no other sampleable points in the third row of pixels; at this time, when pixel 6 is used as the starting pixel, the pixels in the channel map of the image channel have no other sampleable pixels. This completes the process. After sub-sampling, a sampling pixel group starting from pixel 6 is obtained. The sampling pixel group includes pixel 6, pixel 8, pixel 14 and pixel 16.

S1022. Combine the pixels in the sampled pixel group of each image channel to obtain multiple channel feature maps corresponding to each image channel.

After obtaining multiple sampling pixel groups of each image channel, the pixels in each sampling pixel group are combined into images to obtain multiple channel feature maps corresponding to each image channel. That is to say, for each image channel, the number of sampled pixels is the same as the number of obtained channel feature maps. For example, if an image channel samples four pixel groups, then the channel feature map corresponding to the image channel The number of is also 4.

In one embodiment, the step of combining pixels in the sampled pixel group of each image channel to obtain multiple channel feature maps corresponding to each image channel may include:

For each sampled pixel group, according to the position of each pixel in the sampled pixel group in the image feature map, the individual pixels are spliced to obtain the channel feature map of the sampled pixel group; complete the pixels of all sampled pixel groups Point splicing obtains multiple channel feature maps corresponding to each image channel.

For each sampled pixel group, when combining the pixels in the sampled pixel group, the combination can be based on the position of each pixel in the sampled pixel group in the image feature map, so that the resulting image feature map The characteristic information is not lost. For the pixels in multiple pixel groups in each image channel, the pixels are spliced according to the position of each pixel in the image feature map, and after all the pixel groups are spliced, each image is obtained Multiple channel feature maps corresponding to the channel.

As shown in Figure 3a and Figure 3b, for example, when combining pixel 6, pixel 8, pixel 14 and pixel 16 in a sampled pixel group, the image feature map can be calculated based on these four pixels. position in , that is, pixel point 6 is on the left side of pixel point 8, pixel point 6 is on the upper side of pixel point 14, pixel point 16 is on the right side of pixel point 14, and pixel point 16 is on the lower side of pixel point 8, Based on the positional relationship, these four pixels are spliced to obtain the channel feature map 14 as shown in Figure 3b.

S103. Use all channel feature maps corresponding to the image feature map as input feature maps for convolution processing.

All channel feature maps corresponding to the image feature map are a collection of multiple channel feature maps corresponding to each image channel. The multiple channel feature maps corresponding to each image channel are superimposed, and the superimposed channel feature map is used as a volume. The input feature maps of convolution processing are used to enable the image recognition convolutional network to convolve these channel feature maps.

After sampling the pixel points of the image feature map according to the image channel, the sampled pixel points are spliced as the channel feature map, reducing the size of each channel feature map, so that the size of the obtained channel feature map is relative to the original image feature The map is reduced by a factor of n, and the number of image channels of the input feature map is increased.

For example, for an image feature map with a resolution of 320*320 and a channel number of 3, the pixels are sampled every other sample and the sampled pixels are combined to obtain the size of each channel feature map. is 160*160, which is reduced to 1/4 times of the original image feature map. The width and height of the channel feature map are reduced to 1/2 times of the original. At the same time, after the above sampling and combination, each channel map can get 4 Channel feature maps, after superimposing multiple channel feature maps corresponding to each image channel, the number of image channels of the input feature map used for convolution processing is expanded to 4 times the original.

It can be seen that the channel feature map obtained by sampling and splicing pixels according to the image channel ensures that there is no information loss in the input feature map of the image recognition convolution network, and can reduce the input without ensuring that the image information is not lost. The size of the input feature map for the image recognition convolutional network. And overlaying multiple channel feature maps corresponding to each image channel increases the number of image channels of the input feature map used for convolution processing, and can also reduce the calculation amount of the image recognition convolution network when performing image recognition. , to achieve the purpose of improving image recognition speed.

S104. Perform preset subconvolution processing on the input feature map to obtain the recognition result of the image to be recognized.

The image recognition convolution network performs a preset number of convolution processes on the input feature map to obtain the recognition result of the image to be recognized. The recognition results of the image to be recognized include identifying image boundaries of different areas in the image to be recognized, and the recognition results after image segmentation of the image to be recognized.

In the specific implementation process, since the input feature map includes multiple channel feature maps, when performing convolution processing, each channel feature map is convolved separately, and multiple convolution processing results will be obtained. Feature fusion is performed on multiple convolution processing results to obtain the final recognition result of the image to be recognized. After convolution processing is performed on each channel feature map, a pre-recognition result will be obtained. Multiple pre-recognition results corresponding to multiple channel feature maps need to be spliced to obtain the final recognition result.

In the specific implementation process, for example, the resnet50 residual network can be used as the backbone network of the image recognition convolution network to perform image recognition on the image to be recognized with a resolution of 320*320 and a number of image channels of 3, and in the resnet50 network In the second layer, the image feature map of the image to be recognized is processed to obtain the channel feature map, and the channel feature map is used as the input data of the resnet50 network for convolution processing.

As shown in Table 1, they are the size and number of channels of the output data of the image recognition convolution network in each layer when the channel feature map is used as the input data of the resnet50 network, and the image feature map of the image to be recognized is used as the resnet50 network. Comparison of the size and number of channels of the output data of each layer of the image recognition convolutional network when inputting data.

Table 1

层数Number of layers	通道特征图为输入数据The channel feature map is the input data	图像特征图为输入数据The image feature map is the input data
11	3203203232032032	3203203232032032
22	1601606416016064	3203206432032064
33	80801288080128	160160128160160128
44	40402564040256	80802568080256
55	20205122020512	40405124040512
66	1010102410101024	2020102420201024

Since the image feature map of the image to be recognized is processed in the second layer of the resnet50 network, the channel feature map is obtained, and the obtained channel feature map is used as the input data of the second layer. Therefore, it can be seen from Table 1 that using the image Compared with using the feature map as the input data of the resnet50 network and using the channel feature map as the input data of the resnet50 network, although the output data obtained by the two in each layer of the resnet50 network has the same number of output channels, the size of the output feature map differs by 4 times. , that is, when using the channel feature map as the input data of the resnet50 network, the size of the output feature map of each layer of the resnet50 network starting from the second layer is only the output feature map obtained by using the image feature map as the input data of the resnet50 network. 1/4 of the size, the smaller size of the output feature map makes the calculation amount of convolution smaller and the model inference speed is faster.

It can be understood that for convolutional networks of different depths, you can choose to process the image feature map of the image to be recognized at different depth positions of the convolutional network to obtain the channel feature map. For example, you can process it at the second layer of the resnet50 network. The image feature map of the identified image is processed to obtain the channel feature map.

It can be seen that processing the image feature map of the image to be recognized and using the obtained channel feature map as the input feature map can reduce the size of the output data of each layer of the convolutional network. According to the model parameter calculation formula, the model parameter amount param The smaller it is, the smaller the calculation amount of convolution is, which in turn makes the inference speed of the image recognition convolutional network faster.

Among them, the model parameter calculation formula is: param=n*(c*h*w), where n represents the number of layers of the image recognition convolution network, c represents the number of image channels of input data for each layer of the image recognition convolution network, and w represents the width of the input data of each layer of the image recognition convolutional network, and h represents the height of the input data of each layer of the image recognition convolutional network.

In the embodiment of this application, continuing to take the lawn mowing robot as an example, in addition to using the above-mentioned resnet50 network as the backbone network of the image recognition convolution network, the instance segmentation algorithm model Yolact can also be used, and the resolution is 320* 320. Perform image recognition on the image to be recognized with an image channel number of 3, and sample the pixel points of the image channel of the image feature map in the shallow network layer in the Yolact model to obtain the channel feature map, and then convert the channel feature map As the input data of the next network layer of the shallow network layer, convolution processing is performed, and the recognition speed reaches between 6FPS and 8FPS, and the accuracy rate is almost unchanged, while the existing recognition speed of the feasible area is between 2FPS and 3FPS.

In one embodiment, the step of performing a preset number of convolution processes on the input feature map to obtain the recognition result of the image to be recognized may include: performing a preset number of convolution calculations on the input feature map; The results are normalized to obtain the recognition result of the image to be recognized.

Perform a preset number of convolution calculations on the input feature map of the input image recognition convolution network, and then normalize the results of the convolution calculation. After the normalization process, the recognition result of the image to be recognized is obtained.

In one embodiment, the recognition result of the image to be recognized includes the target area. After step S104, the image recognition method also includes: obtaining the color characteristics of the target area; performing masking processing on the target area according to the color characteristics to obtain a mask map of the target area. In one embodiment, the recognition result of the image to be recognized includes the target area. After step S104, the image recognition method also includes: obtaining a label of the target area; performing masking processing on the target area according to the label of the target area to obtain a mask image of the target area.

Among them, the target area refers to the movable area from the mobile device. When the mobile device is a lawn mower robot, the image to be recognized obtained from the mobile device is shown in Figure 5a. The green grass area and the yellow dead grass area can be used as the movable area of the lawn mower robot, and the movable area represents the lawn mower. The grass area where the robot operates, the immovable area is the non-grass area, such as trees, side paths, moving vehicles, etc. After image recognition, the obtained image recognition result is shown in Figure 5b, where the target area is the white area in Figure 5b, which is the movable area, the black area is the immovable area, and the boundary between the white area and the black area This is the edge of the grassland. Or mark the target area according to the label. For example, mark the movable areas as 1 and mark the immovable areas as 0. Make the same mask for the target area with the same color feature, or use the same mask for the target area with the same label. When the lawn mower robot moves in the movable area, since there may be obstacles and other objects in the movable area that affect the movement of the lawn mower robot, the color characteristics of the target area can be obtained, so that the target area can be mapped based on the color characteristics of the target area. Mask processing is performed, or the target area is masked according to the label of the target area to obtain a mask image of the target area, and the edge of the grass can be presented based on the mask image, so that when identifying objects in the target area, It prevents the lawn mower robot from walking out of the edge of the grass, which facilitates path planning or obstacle recognition, thereby improving the efficiency of path planning and automatic obstacle avoidance. The image recognition method provided by the above embodiment samples the pixel points of each image channel in the image feature map, so that the pixel points in the obtained channel feature map not only retain the information of the image feature map, but also increase the image channel quantity, ensuring the integrity of the image information in the image to be recognized. At the same time, since the size of the channel feature map obtained after sampling is smaller than the image feature map, when the channel feature map is used as the input feature map of the convolution process, no loss is achieved. On the basis of image information, the calculation amount of image recognition is greatly reduced, thereby improving the efficiency of image recognition.

Please refer to Figure 6. Figure 6 is a schematic block diagram of an image recognition device further provided by an embodiment of the present application. The image recognition device is used to execute the aforementioned image recognition method. Wherein, the image recognition device can be configured in a mobile device.

Among them, the server can be an independent server or a server cluster. The terminal can be an electronic device such as a mobile phone, tablet, laptop, desktop computer, personal digital assistant, and wearable device.

As shown in FIG. 6 , the image recognition device 200 includes: an image acquisition module 201 , a channel sampling module 202 , an input determination module 203 and a convolution processing module 204 .

The image acquisition module 201 is configured to acquire an image feature map of an image to be recognized, where the image feature map includes multiple image channels.

The channel sampling module 202 is configured to sample the pixel points of each image channel to obtain multiple channel feature maps corresponding to each image channel. The pixel points of the multiple channel feature maps corresponding to each image channel are The total number is equal to the total number of pixels in each image channel.

The input determination module 203 is configured to use all channel feature maps corresponding to the image feature map as input feature maps for convolution processing.

The convolution processing module 204 is configured to perform preset subconvolution processing on the input feature map to obtain the recognition result of the image to be recognized.

It should be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described image recognition device and each module can be referred to the corresponding processes in the foregoing image recognition method embodiments. I won’t go into details here.

The above image recognition device can be implemented in the form of a computer program, and the computer program can be run on a mobile device as shown in Figure 7.

Please refer to FIG. 7 , which is a schematic structural block diagram of a mobile device provided by an embodiment of the present application. The self-mobile device can be a small self-mobile device such as a lawnmower, a patrol robot, or a mine-clearing robot.

Referring to Figure 7, the computer device includes a processor, a memory, and a network interface connected through a system bus, where the memory may include a non-volatile storage medium and an internal memory.

Non-volatile storage media stores operating systems and computer programs. The computer program includes program instructions, which when executed, can cause the processor to perform any image recognition method.

The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

The internal memory provides an environment for the execution of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, it can cause the processor to execute any image recognition method.

This network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure shown in Figure 6 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Specific computer equipment can May include more or fewer parts than shown, or combine certain parts, or have a different arrangement of parts.

It should be understood that the processor can be a central processing unit (Central Processing Unit, CPU), and the processor can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general processor may be a microprocessor or the processor may be any conventional processor.

Wherein, in one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

An image feature map of the image to be recognized is obtained, where the image feature map includes multiple image channels.

The pixels of each image channel are sampled to obtain multiple channel feature maps corresponding to each image channel. The total number of pixels of the multiple channel feature maps corresponding to each image channel is equal to each of the image channels. The total number of pixels in the image channel.

All channel feature maps corresponding to the image feature map are used as input feature maps for convolution processing.

In one embodiment, when the processor implements the sampling of pixels of each of the image channels to obtain multiple channel feature maps corresponding to each of the image channels, the processor is used to implement:

The pixel points of each image channel are grouped to obtain multiple sampled pixel point groups of each image channel.

The pixels in the sampling pixel group of each image channel are combined to obtain multiple channel feature maps corresponding to each image channel.

In one embodiment, when the processor implements the grouping of pixel points of each of the image channels to obtain multiple sampled pixel point groups of each of the image channels, the processor is configured to implement:

A starting pixel point set is determined among all pixels of the image channel according to the preset sampling interval N, and the starting pixel point set is a pixel matrix including (N+1)*(N+1) pixels, The first pixel point of the pixel matrix is any boundary vertex of the corresponding image channel.

Taking each pixel point in the set of starting pixel points as a starting pixel point, the pixel points of the image channel are sampled at a preset sampling interval to obtain a sampled pixel point group for each starting pixel point.

In one embodiment, when the processor implements the combination of pixels in the sampled pixel group of each image channel to obtain multiple channel feature maps corresponding to each image channel, , used to implement:

For each sampled pixel group, each pixel is spliced according to the position of each pixel in the sampled pixel group in the image feature map to obtain a channel feature map of the sampled pixel group.

Complete the pixel splicing of all the sampled pixel groups to obtain multiple channel feature maps corresponding to each of the image channels.

In one embodiment, when implementing the acquisition of the image feature map of the image to be recognized, the processor is configured to implement:

The image to be recognized is preprocessed, and the preprocessing includes adjusting the resolution of the image to be recognized.

Feature extraction is performed on the preprocessed image to be recognized to obtain an image feature map.

In one embodiment, when performing the preset subconvolution process on the input feature map to obtain the recognition result of the image to be recognized, the processor is configured to:

Perform preset times of convolution calculations on the input feature map.

The results obtained by the convolution calculation are normalized to obtain the recognition result of the image to be recognized.

In one embodiment, after the processor realizes that the recognition result of the image to be recognized includes the target area, and performs a preset convolution process on the input feature map to obtain the recognition result of the image to be recognized, Used to implement:

Obtain the color features of the target area.

Mask processing is performed on the target area according to the color characteristics to obtain a mask image of the target area.

Obtain the label of the target area;

Mask processing is performed on the target area according to the label of the target area to obtain a mask image of the target area.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement the present application. Any image recognition method provided by the embodiment.

The computer-readable storage medium may be an internal storage unit of the mobile device described in the previous embodiment, such as a hard disk or memory of the mobile device. The computer-readable storage medium may also be an external storage device of the mobile device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), or a secure digital device equipped on the mobile device. , SD) card, flash card (Flash Card), etc.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person familiar with the technical field can easily think of various equivalent methods within the technical scope disclosed in the present application. Modification or replacement, these modifications or replacements shall be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image recognition method, the method includes:

Obtain an image feature map of the image to be recognized, where the image feature map includes multiple image channels;

The pixels of each image channel are sampled to obtain multiple channel feature maps corresponding to each image channel. The total number of pixels of the multiple channel feature maps corresponding to each image channel is equal to each of the image channels. The total number of pixels in the image channel;

Use all channel feature maps corresponding to the image feature map as input feature maps for convolution processing;

Perform preset subconvolution processing on the input feature map to obtain the recognition result of the image to be recognized.
The image recognition method according to claim 1, wherein the pixel points of each image channel are sampled to obtain multiple channel feature maps corresponding to each image channel, including:

Group the pixels of each image channel to obtain multiple sampled pixel groups of each image channel;

The pixels in the sampling pixel group of each image channel are combined to obtain multiple channel feature maps corresponding to each image channel.
The image recognition method according to claim 2, wherein said grouping the pixel points of each of the image channels to obtain a plurality of sampled pixel point groups of each of the image channels includes:

A starting pixel point set is determined among all pixels of the image channel according to the preset sampling interval N, and the starting pixel point set is a pixel matrix including (N+1)*(N+1) pixels, The first pixel point of the pixel matrix is any boundary vertex of the corresponding image channel;

Taking each pixel point in the set of starting pixel points as a starting pixel point, the pixel points of the image channel are sampled at a preset sampling interval to obtain a sampled pixel point group for each starting pixel point.
The image recognition method according to claim 2, wherein the pixel points in the sampled pixel point group of each of the image channels are combined to obtain multiple channel feature maps corresponding to each of the image channels. ,include:

For each sampled pixel group, splice the individual pixels according to the position of each pixel in the sampled pixel group in the image feature map to obtain a channel feature map of the sampled pixel group;

Complete the pixel splicing of all the sampled pixel groups to obtain multiple channel feature maps corresponding to each of the image channels.
The image recognition method according to claim 1, wherein said obtaining the image feature map of the image to be recognized includes:

Preprocessing the image to be recognized, the preprocessing including adjusting the resolution of the image to be recognized;

Feature extraction is performed on the preprocessed image to be recognized to obtain an image feature map.
The image recognition method according to claim 1, wherein said performing preset sub-convolution processing on the input feature map to obtain the recognition result of the image to be recognized includes:

Perform a preset number of convolution calculations on the input feature map;

The results obtained by the convolution calculation are normalized to obtain the recognition result of the image to be recognized.
The image recognition method according to any one of claims 1 to 6, wherein the recognition result of the image to be recognized includes a target area, and the input feature map is subjected to preset convolution processing to obtain the image to be recognized. After the recognition result of the image, the method further includes:

Obtain the color characteristics of the target area;

Mask processing is performed on the target area according to the color characteristics to obtain a mask image of the target area.
The image recognition method according to claim 3, wherein the recognition result of the image to be recognized includes a target area, and after performing a preset subconvolution process on the input feature map to obtain the recognition result of the image to be recognized , the method also includes:

Obtain the label of the target area;

Mask processing is performed on the target area according to the label of the target area to obtain a mask image of the target area.
A self-mobile device, the self-mobile device includes a memory and a processor;

The memory is used to store computer programs;

The processor is configured to execute the computer program and implement the image recognition method according to any one of claims 1 to 8 when executing the computer program.
A computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to implement image recognition as claimed in any one of claims 1 to 8 method.