CN116109922A

CN116109922A - Bird recognition method, bird recognition apparatus, and bird recognition system

Info

Publication number: CN116109922A
Application number: CN202211655434.6A
Authority: CN
Inventors: 罗欢; 陈明权; 何涛; 李青
Original assignee: Hangzhou Ruisheng Software Co Ltd
Current assignee: Hangzhou Ruisheng Software Co Ltd
Priority date: 2022-12-21
Filing date: 2022-12-21
Publication date: 2023-05-12

Abstract

The invention provides a bird recognition method, bird recognition equipment, a bird recognition system and a readable storage medium, which are characterized in that an acquired bird-containing image to be recognized is firstly recognized to obtain an area image of the bird where the bird is located, then the bird area image is subjected to image optimization, so that the bird in the optimized bird area image is clearly displayed, and further, a bird recognition engine can recognize the bird in the bird area image by combining the information of a shooting area corresponding to the image to be recognized, so that the accuracy of a bird recognition result is improved.

Description

Bird recognition method, bird recognition apparatus, and bird recognition system

Technical Field

The present invention relates to the field of bird recognition technology, and in particular, to a bird recognition method, bird recognition apparatus, bird recognition system, and readable storage medium.

Background

Birds are important indicators for biodiversity monitoring and ecological environmental impact evaluation. The current situation of bird resources can be known through investigation and monitoring of bird species, the characteristics of the composition, the number, the diversity and the like of the bird species can be summarized, and the characteristics can be utilized to directly reflect the environmental quality of habitats, the health degree of an ecological system, the biodiversity condition, the interference degree of human activities on the ecological system, the influence degree of land utilization and landscape change on the ecological system and the like, so that the birds are required to be identified and supervised, and the birds and the human can be ensured to be in harmony.

Current bird recognition schemes have the situation of inaccurate recognition, so how to adjust the bird recognition scheme to improve recognition accuracy is one of technical hotspots studied in the field.

Disclosure of Invention

The invention aims to provide a bird recognition method, bird recognition device, bird recognition system and readable storage medium, which can improve the accuracy of bird recognition results.

In order to achieve the above object, the present invention provides a bird recognition method comprising:

acquiring an image to be identified containing birds;

acquiring an avian region image in which the birds are located from the image to be identified;

performing image optimization on the bird region image to enable birds in the optimized bird region image to be clearly displayed;

inputting the optimized bird region image and the information of the shooting region corresponding to the image to be recognized into a corresponding bird recognition engine to obtain a final bird recognition result.

Optionally, focusing the birds automatically by using shooting equipment, and performing amplification shooting of pictures to obtain the images to be identified; or automatically focusing the birds by using shooting equipment, performing video amplification shooting, and obtaining the image to be identified by acquiring each frame of picture from the shot video.

Optionally, the step of performing zoom-in shooting includes:

after the birds are automatically focused by utilizing shooting equipment, the birds are automatically amplified to an automatic amplification threshold preset in the shooting equipment, and then a user is guided to perform manual amplification operation through corresponding amplification gestures and/or texts;

judging whether the shooting equipment is provided with a tele lens or not;

if yes, the user manually enlarges the picture after the shooting equipment automatically focuses according to the guidance, and calls the tele lens to shoot the birds;

if not, the user manually enlarges the picture after the automatic focusing of the shooting equipment according to the guidance, and the user selects a lens with a proper focal length to shoot the birds according to the enlarging operation of the user.

Optionally, the step of acquiring the bird region image where the bird is located from the image to be identified includes:

adopting a corresponding bird region identification method to automatically identify the region where birds are located from the image to be identified, and forming a labeling frame;

slicing the region where the birds in the image to be identified are located according to the marking frame so as to form the bird region image containing the birds.

Optionally, after the marking frame is formed and before the region where the birds in the image to be identified are located is sliced, the size of the marking frame is automatically or manually adjusted, and the region where the birds in the image to be identified are sliced according to the adjusted marking frame, so that the bird region image is formed.

Alternatively, when a plurality of bird targets exist in the image to be recognized, the bird region recognition method is adopted, and a region where one bird target is located is selected as the bird region image from regions where the plurality of bird targets are respectively located in the image to be recognized.

Optionally, when the bird region image is subjected to image optimization, the bird region image with different specific parameters is subjected to different image optimization operations, and the specific parameters include at least one of image resolution, image area, image long side size and image wide side size.

Optionally, the image optimization operations employed by the bird region images with different specific parameters include:

when the specific parameter of the bird region image is smaller than a first threshold value, performing ultra-cleaning treatment on the bird region image, wherein the resolution ratio of the bird region image after optimization is higher than that of the bird region image before optimization;

when the specific parameter of the bird region image is larger than or equal to the first threshold value and smaller than a second threshold value, beautifying the bird region image, wherein the resolution of the bird region image is the same before and after optimization;

When the specific parameter of the bird region image is greater than or equal to the second threshold, image optimization is not performed.

Optionally, performing ultra-cleaning treatment on the corresponding bird region images by using an ultra-cleaning engine; and beautifying the corresponding bird region image through a beautifying engine or a computer vision algorithm.

Optionally, inputting the optimized bird region image and the information of the shooting area corresponding to the image to be recognized into a corresponding bird recognition engine, and obtaining the final bird recognition result includes:

when the image to be identified is a local real shot image, the bird identification engine outputs a final identification result and a final identification confidence of birds in the bird area image according to the geographic area corresponding to the image to be identified and all bird types observed by the geographic area after receiving the optimized bird area image;

when the image to be identified is a non-local real shot image obtained by a user for the existing image in a flipping or screenshot mode, the bird identification engine respectively obtains global identification results and global identification confidence coefficients of birds in the bird area image corresponding to all bird types observed globally after receiving the optimized bird area image, and the birds in the bird area image correspond to the area identification results and the area identification confidence coefficients of all bird types observed by the geographic area where the user is located, compares the global identification confidence coefficient with the area identification confidence coefficient, and outputs final identification results and final identification confidence coefficients of birds in the bird area image according to the comparison results.

Optionally, the global recognition result and the regional recognition result are arranged in order from high confidence to low confidence respectively, top1 of global and top2 of global are ranked in the top two bits in the global recognition result, top1 of region and top2 of region are ranked in the top two bits in the regional recognition result respectively; if the global recognition confidence of the top1 of global is larger than the region recognition confidence of the top1 of region by a preset coefficient, the output final bird recognition result in the bird region image consists of the top1 and the top2 of global and the top1 of region; if the global recognition confidence of the top1 of the global is smaller than or equal to the region recognition confidence of the top1 of the region by a preset coefficient, the output final bird recognition result in the bird region image consists of the top1 of the region, the top2 and the top1 of the global.

Based on the same inventive concept, the present invention also provides a readable storage medium having stored thereon a program which when executed implements the bird recognition method according to the present invention.

Based on the same inventive concept, the invention also provides a bird recognition device comprising a processor and a memory, wherein the memory stores a program, and the program is executed by the processor to realize the bird recognition method.

Based on the same inventive concept, the present invention also provides a bird recognition system, comprising:

an image acquisition unit configured to acquire an image to be recognized containing birds;

an area acquisition unit configured to acquire a bird area image in which the bird is located from the image to be recognized;

an image optimizing unit configured to perform image optimization on the bird region image so as to make birds in the optimized bird region image clearly displayed;

the bird recognition engine can be configured to obtain a final bird recognition result according to the optimized bird region image and the information of the shooting region corresponding to the image to be recognized.

Optionally, the image acquisition unit includes a shooting device, and the shooting device is configured to automatically zoom in on the birds, then automatically zoom in to an automatic zoom-in threshold preset in the shooting device, and then guide the user to perform manual zoom-in operation through corresponding zoom-in gestures and/or texts;

when the shooting equipment is provided with a long-focus lens, after a user manually enlarges an image after automatic focusing of the shooting equipment according to guidance, the long-focus lens is called to shoot the birds;

When the shooting equipment does not have a tele lens, after a user manually amplifies a picture after automatic focusing of the shooting equipment according to guidance, a lens with a proper focal length is selected to shoot the birds according to the amplifying operation of the user.

Optionally, the area acquisition unit includes:

the bird region recognition module is provided with a bird region recognition model and is configured to automatically recognize the region where the bird is located from the image to be recognized through the bird region recognition model, and form a labeling frame;

and the slicing module is configured to slice the region where the birds in the image to be identified are located according to the marking frame or the adjusted marking frame so as to form the bird region image containing the birds.

Optionally, the image optimization unit includes:

a judging module configured to judge the magnitude of a specific parameter of the bird region image relative to a first threshold and a second threshold, and when the specific parameter is determined to be greater than or equal to the second threshold, not performing image optimization on the bird region image;

the super-cleaning engine is configured to perform super-cleaning processing on the bird region image when the judging module judges that the specific parameter is smaller than or equal to a first threshold value;

And a beautifying engine configured to beautify the bird region image when the judging module judges that the specific parameter is greater than the first threshold value and less than a second threshold value.

Compared with the prior art, the technical scheme of the invention has at least the following beneficial effects:

through the method, the bird region image is obtained by firstly identifying the region where the birds are located through the obtained bird-containing image to be identified, then the bird region image is subjected to image optimization, so that the birds in the optimized bird region image can be clearly displayed, and further, the bird identification engine can identify the birds in the bird region image by combining the information of the shooting region corresponding to the image to be identified, so that the accuracy of the bird identification result is improved.

Drawings

Those of ordinary skill in the art will appreciate that the figures are provided for a better understanding of the present invention and do not constitute any limitation on the scope of the present invention. Wherein:

FIG. 1 is a schematic flow chart of a bird identification method according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of an enlarged guidance operation in step S1 of the bird recognition method according to the embodiment of the present invention.

Fig. 3 to 5 are schematic views of image changes in the bird region recognition process in step S2 of the bird recognition method according to the embodiment of the present invention.

Fig. 6 is a flowchart illustrating a step S3 of the bird recognition method according to an embodiment of the present invention.

Fig. 7 to 9 are schematic views of image changes during the ultra-cleaning process in step S3 of the bird recognition method according to the embodiment of the present invention.

Fig. 10 is a schematic diagram showing comparison of images before and after the super-cleaning treatment in step S3 of the bird recognition method according to the embodiment of the present invention.

Fig. 11 is a schematic view of a sample image generated at the time of the beautifying process in step S3 of the bird recognition method according to the embodiment of the present invention.

Fig. 12 is a schematic diagram of the final bird recognition result outputted in step S4 of the bird recognition method according to the embodiment of the present invention.

Fig. 13 is a schematic structural view of a bird recognition device according to an embodiment of the present invention.

Fig. 14 is a schematic structural view of a bird recognition system according to an embodiment of the present invention.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without one or more of these details. In other instances, well-known features have not been described in detail in order to avoid obscuring the invention. It should be understood that the present invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout. It will be understood that when an element is referred to as being "connected to," "coupled to" another element, it can be directly connected to the other element or intervening elements may be present. In contrast, when an element is referred to as being "directly connected to" another element, there are no intervening elements present. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups. As used herein, the term "and/or" includes any and all combinations of the associated listed items.

The terms "instruction," "application," "process," "step," and "program" are used interchangeably herein. The instructions may be stored in an object code format for direct processing by one or more processors, or in any other computer language, including scripts or collections of separate source code modules that are interpreted or compiled in advance, as desired. The instructions may include instructions that cause, for example, one or more processors to act as neural networks herein. The functions, methods and routines of the instructions are explained in more detail elsewhere herein.

The technical scheme provided by the invention is further described in detail below with reference to the attached drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description. It should be noted that the drawings are in a very simplified form and are all to a non-precise scale, merely for convenience and clarity in aiding in the description of embodiments of the invention.

Referring to fig. 2, an embodiment of the present invention provides a bird identification method, which includes:

s1, acquiring an image to be identified containing birds;

s2, acquiring an avian region image where the birds are located from the image to be identified;

S3, performing image optimization on the bird region image to enable birds in the optimized bird region image to be clearly displayed;

and S4, inputting the optimized bird region image and the information of the shooting region corresponding to the image to be recognized into a corresponding bird recognition engine to obtain a final bird recognition result.

In step S1, please refer to fig. 2, the shooting device used may provide a plurality of ways for obtaining the image to be identified containing birds for selection by the user, as shown in a region 204a in fig. 2, and may also provide content cues 205 such as shooting aids or shooting descriptions to help the user obtain the image to be identified with higher quality. The first method is a video camera method, in which video shooting is performed directly by using a camera of a shooting device, specifically, in this method, the shooting device performs automatic focusing (i.e. forms a focusing frame 200) after facing birds, and a user performs enlarged shooting of video by pressing a shooting key 204b, and further obtains a required image to be identified by a method of acquiring each frame of picture from the shot video. The second mode is a By photo mode, which is a mode of taking a picture directly By using a camera of a photographing apparatus, specifically, in this mode, the photographing apparatus performs auto-focusing (i.e., forms a focusing frame 200) against birds, and a user performs enlarged photographing of a picture By pressing a photographing button 204b, so as to obtain a desired image to be recognized. In both ways, the camera device may also provide the function of automatically turning on the flash 203 depending on the ambient light.

In addition, the photographing apparatus provides a third way, specifically, a desired picture may be selected from the gallery 204c stored inside the photographing apparatus as a desired image to be recognized.

Referring to fig. 2, taking a local real-time shooting as an example, when a user turns on a shooting device (e.g., a digital camera or a mobile phone camera) carried by the user, the shooting device shoots birds 100 flying in the sky, and an auto-focusing function of the shooting device automatically generates a focusing frame 200. At this time, the step of magnifying the bird 100 to be recognized in step S1 includes:

(1) After the shooting device automatically focuses the bird 100, the shooting device automatically zooms into a preset automatic zoom-in threshold (for example, the automatic zoom-in threshold is 5 times, that is, the zoom-in magnification at 201b is 5 times at this time), when the bird is far away from the user, the zoom-in magnification needs to be more than 5 times, and at this time, in order to shoot a relatively clear bird image or video, the user can only zoom in manually, in this embodiment, for convenience of operation, a corresponding zoom-in gesture 201a and/or

text

202a and 202b is generated on the shooting interface, so as to guide the user to complete the required manual zoom-in operation, where the manual zoom-in operation can be realized by touching and sliding the thumb and index finger of the user in the zoom-in gesture 201a region on the screen of the shooting device, or by clicking "+" in the zoom-in region 201b by the finger of the user, and by clicking the circle in the zoom-in region 201b along a specific straight line.

(2) It is determined whether the photographing apparatus has a tele lens (not shown).

(3) When the photographing device has a tele lens, switching the tele lens to photograph the bird 100; when the photographing apparatus does not have a tele lens, a lens of an appropriate focal length is selected according to a manual zoom-in operation of a user to photograph the bird 100.

It should be noted that in the above step S1, the user is guided to perform the multi-purpose amplifying operation first, and meanwhile, the plurality of cameras are called under the condition that the photographing device allows, so that a more clear picture can be photographed, and the quality of the photographed picture or video is improved. The long-focus lens has a larger focal length, and can shoot more clear pictures. For example, the magnification is 10 times, the long focal lens can be realized through optical zooming without causing the reduction of definition, and the definition of an image cannot be ensured because a common camera needs to be magnified through digital zooming.

Taking an image to be recognized (not shown) taken by the shooting interface shown in fig. 3 as an example, in step S2, firstly, a multi-object recognition (object detection), a convolutional neural network CNN, or a mask-rcnn recognition method may be adopted to recognize an area where birds are located from the image to be recognized, and the area is marked by a marking frame, where the marking frame (may be rectangular, circular or any other suitable shape) encloses the area, that is, a required bird area, as shown in fig. 4; then, the recognized bird region is sliced by acquiring a labeling frame labeled after recognizing the region, thereby forming a corresponding bird region image, as shown in fig. 7.

In this step, after the area where birds are located is automatically identified from the image to be identified, the system can automatically adjust the size of the labeling frame, and the user can also manually adjust the size of the labeling frame, and then slice the bird area according to the adjusted labeling frame to form a bird area image, as shown in fig. 5.

It should be noted that, when a plurality of birds exist in the image to be recognized, for example, a plurality of birds may be photographed by the photographing interface shown in fig. 2, at this time, in step S2, a plurality of bird regions may be automatically recognized, each bird region may include only one bird, and a region is further selected from the bird regions according to the area of each bird region recognized and the position thereof from the center point of the whole image of the image to be recognized, so as to form a corresponding bird region image. For example, regions with a region area larger than a preset threshold value can be screened out of the bird regions, and a region closest to the center point of the full image of the image to be identified is further selected from the screened regions for slicing, so that a corresponding bird region image is formed. For another example, areas with a distance smaller than a preset threshold value from the center points of the whole image of the image to be recognized can be screened out from the bird areas, and a region with the largest area is further selected from the screened areas for slicing, so that a corresponding bird area image is formed. For example, the areas of the areas and the distances between the areas and the center point of the whole image of the image to be recognized are weighted, one area is selected according to the weighted calculation result, and slicing is carried out, so that a corresponding bird area image is formed.

It is noted that, the steps S1 and S2 achieve the operation effect of "zoom-in before shooting, then region slicing", and the above-described shooting apparatus can obtain a higher quality bird region image than the operation of "shoot before region slicing" even without a telephoto lens.

In step S3, when the bird region image is subjected to image optimization, the bird region images with different specific parameters are subjected to different image optimization operations, so as to improve the recognition accuracy. Wherein the specific parameter includes at least one of image resolution, image area, image long side size, image wide side size, and the like.

Referring to fig. 6, taking a specific parameter as a sum of a long side size of an image and a wide side size of the image, a first threshold is 1024, and a second threshold is 2048 as an example, in step S3, performing an image optimization operation on the bird region image obtained in step S2 includes:

s31, judging whether the sum of the image long side size and the image wide side size of the bird region image is smaller than 2048, and if not, executing S32; if yes, executing S33;

s32, outputting an original image (namely, the bird region image obtained by slicing in the step S2 has high definition and can be directly used for the following step S4) without optimization treatment;

S33, judging whether the sum of the image long side size and the image wide side size of the bird region image is smaller than 1024, if yes, executing S34, and if not, executing S35;

s34, performing ultra-cleaning treatment on the bird region image, wherein the resolution ratio of the bird region image after optimization is higher than that of the bird region image before optimization;

s35, carrying out beautifying treatment (such as image sharpening) on the bird region images, wherein the resolution ratios of the bird region images before and after optimization are the same;

and S36, displaying the optimized bird region image.

In this embodiment, the super-cleaning processing in step S34 may be implemented by a super-cleaning engine, specifically, the bird region image obtained in step S2 is input into the super-cleaning engine, as shown in fig. 7, and the super-cleaning engine performs super-cleaning processing on the bird region image, as shown in fig. 8, and outputs a high-definition bird region image after the processing is completed, as shown in fig. 9. As can be seen from fig. 10, the image resolution after the super-resolution processing is much higher than the image resolution before the super-resolution processing.

The super-resolution engine may use a model for image super-resolution, such as SwinIR, drln+, RCAN, and the like, where SwinIR is composed of three modules: the device comprises a shallow layer feature extraction module, a depth feature extraction module and a high-quality image reconstruction module. The shallow feature extraction module extracts shallow features by adopting a convolution layer and directly transmits the shallow features to the image reconstruction module so as to retain low-frequency information. The depth feature extraction module outputs the shallow feature map as a deep feature map. Finally, the shallow layer and depth features are fused in a reconstruction module, and high-quality image reconstruction is carried out. The image reconstruction module generates a high-quality image according to the shallow feature map and the deep feature map, the shallow features are responsible for containing low-frequency information, and the deep features are focused on recovering lost high-frequency information.

In addition, the super-cleaning engine can generate a relatively blurred sample picture by downsampling (the downsampling mode is quite large, such as PyrDown, MATLAB function dyaddown in opencv, etc.) the bird region image obtained in the step S2, and can further take the sample picture as input and the high-definition picture as output to train the corresponding model. In this embodiment, during the training of the ultra-cleaning engine, the length and width of the bird region image obtained in the step S2 are reduced by 1/2, and the length and width of the bird region image output are consistent with those of the bird region image obtained in the step S2, so that the bird region image obtained in the step S2 can be generated into a 4-fold up-sampled picture after being processed by the ultra-cleaning engine.

In this embodiment, the beautifying process in step S35 may be implemented by a beautifying engine (e.g., sharpening engine), or may be implemented by using only computer vision algorithms (e.g., mean filtering, gaussian filtering, etc.). In addition, referring to fig. 11, a prompt may be added to the result page output in step S35 to tell the user that the bird region image obtained in step S2 is clear, and the system is more suitable for processing blurred pictures and displaying sample images.

The beautifying engine can adopt an IPT model, RIDNet, dnCNN or other beautifying models, and can obtain training samples (i.e. sample pictures) of the beautifying engine by a method of blurring (e.g. image smoothing) high-definition pictures. And taking the sample picture as input, taking the beautified picture as output, and training the model. The sample picture can select a picture containing birds so as to improve the beautifying effect of the bird picture. The picture processed by the beautifying engine does not improve the resolution, so that the problem of occupying excessive video memory does not exist.

In step S4, comparing the images obtained in the step S3 with all bird types in the corresponding region in the bird recognition engine according to the image shooting region, and obtaining a final recognition result.

Current bird recognition engines generally divide bird activity areas into regional regions, which may be regions of north america (NorthAmerica, NA), europe (EU), asia (AS), etc., or specifically to a country, and global. Each region sets the type of birds that the corresponding region can observe, and the bird recognition engine outputs a corresponding recognition result topk and a recognition confidence value according to the region (i.e. shooting region) where the image provided by the user is located.

When the image to be recognized obtained in step S1 is a local real shot image (i.e. obtained by directly shooting birds actually flying in a local sky), the image to be recognized will simultaneously include region information (i.e. geographical position information) of the shooting region, at this time, in step S4, after the bird region image finally obtained in step S3 is input into the bird recognition engine, the bird recognition engine will first find all bird types that can be observed by the region according to the region information, for example, the region information is NA region, all bird types in the world are 8015 types at present, and the bird types in the NA region are 2000 types, accounting for about 1/4 of all bird types in the world. The bird recognition engine determines an area NA according to the area region information, extracts 2000 bird types corresponding to the NA, further compares birds in the bird area image finally obtained in the step S3 with the 2000 bird types, calculates the recognition confidence coefficient of each bird type, finally outputs the bird types k (k=1 or 2 or 3 or 5 or 10, and the like) before the recognition confidence coefficient is arranged, and the displayed final recognition results are arranged as top1 and top2 … topk of the region according to the recognition confidence coefficient from high to low.

In other embodiments of the present invention, in the case where the image to be identified is a local real shot image, the birds in the bird region image obtained in the step S3 may be directly compared with 8000 bird types worldwide without combining with shot region information, the identification confidence of each bird type is calculated, the bird type k before the arrangement of the identification confidence is finally output, and the final identification result is displayed as top1 and top2 … topk of global according to the arrangement of the identification confidence from high to low. At this time, the above-mentioned top1 to topk of region and top1 to topk of global are substantially identical.

However, when the image to be identified obtained in step S1 is a non-local real shot image obtained by the user by flipping an existing image (for example, by shooting a television, a book, etc.) or by capturing a shot image (for example, by capturing a mobile phone end), an inaccurate problem may occur in the identification result returned to the user according to the shooting region information of the image to be identified, for example, the top1 of a region is inconsistent with the top1 of a global. In this case, in order to improve the recognition accuracy, in step S4 of the present embodiment, the output logic of the recognition result is adjusted to the integration of global and region. Specifically:

Firstly, after receiving the optimized bird region image, the bird recognition engine respectively obtains global recognition results and global recognition confidence of birds in the bird region image, which correspond to all bird types observed globally, and region recognition results and region recognition confidence of birds in the bird region image, which correspond to all bird types observed in a geographic region where a user is located. The global recognition results are arranged into global top1 and top2 … … topk according to the order of the global recognition confidence, and the region recognition results are arranged into region top1 and top2 … topk according to the order of the region recognition confidence.

Then, comparing the region identification confidence of the top1 of the region with the global identification confidence of the top1 of the global, and if the global identification confidence of the top1 of the global > the region identification confidence of the top1 of the region by a preset coefficient, respectively forming the first three bits of the returned final identification result by the top1, the top2 of the global and the top1 of the region; if global confidence of global top1 < regional confidence of region top1 of region preset coefficient, the first three bits of the returned final recognition result will be composed of region top1, region top2 and global top1, respectively. Obviously, when the winner top1 of both top1 of the region and top1 of the global is adjusted to be in bit 1 in the output recognition result list, the loser top1 is adjusted to be in bit 2 or in bit 3 in the output recognition result list, and the loser top1 is specifically in bit 2 or in bit 3, it may be determined by further comparison, for example, when the global recognition confidence of top1 of the global > the region recognition confidence of top1 of the region, and the global recognition confidence of top2 of the global > the region recognition confidence of top1 of the region, the top1 of the region is placed as the loser top1 in bit 3 of the final recognition result, that is, the top three bits of the returned final recognition result are arranged in sequence of top1 of the global, top2 of the global, and top1 of the region.

Referring to fig. 12, in step S4, not only the first few recognition results with higher confidence may be presented to the user, but also various examples of Sounds (Sounds) made by the bird category, a suitable feed can (Feeder Type), a suitable Food (Food Type), and other related descriptions (descriptions) about the bird category in the first few recognition results may be presented to the user.

Specifically, the bird recognition method can help a user recognize birds and improve the accuracy of bird recognition.

Based on the same inventive concept, an embodiment of the present invention also provides a readable storage medium having a program stored thereon, which when executed, implements the bird recognition method according to the present invention.

Similarly, the readable storage medium in embodiments of the present disclosure may be any medium that can contain, store, communicate, propagate, or transport the computer program. For example, the readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium, specific examples of which include magnetic storage devices such as magnetic tape or Hard Disk (HDD); optical storage devices such as Compact Discs (CDROM); a memory, such as a Random Access Memory (RAM) or a flash memory; and/or wired, wireless communication links.

The present disclosure also proposes a computer program product which may comprise instructions or code which, when executed by a processor, may implement the steps of a plant identification method as described above. The instructions may be any set of instructions, such as machine code, to be executed directly by one or more processors, or any set of instructions, such as scripts, to be executed indirectly.

Based on the same inventive concept, please refer to fig. 13, an embodiment of the present invention further provides a bird recognition device 300, which includes a processor 301 and a memory 302, wherein a program is stored in the memory 302, and when the program is executed by the processor 301, the bird recognition method according to the present invention is implemented.

Wherein the processor 301 may perform various actions and processes in accordance with instructions stored in the memory 302. In particular, the processor 301 may be an integrated circuit chip with signal processing capabilities. The processor may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. Various methods, steps, and logic blocks disclosed in the embodiments of the present disclosure may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and may be an X86 architecture or an ARM architecture or the like.

The memory 302 stores an executable program (which may be code or instructions) that, when executed by the processor 301, performs the bird identification method described above. The memory 302 may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (ddr SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link Dynamic Random Access Memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Based on the same inventive concept, an embodiment of the present invention also provides a bird recognition system 400 capable of implementing the bird recognition method of the present invention. Specifically, please refer to fig. 14, which includes an image acquisition unit 401, an area acquisition unit 402, an image optimization unit 403, and a bird recognition engine 404.

The image acquisition unit 401 is configured to acquire an image to be recognized containing birds, wherein the image acquisition unit 401 may include, but is not limited to, a photographing device 401a, the photographing device 401a may include, but is not limited to, a camera, a video camera, a camera included in a mobile device such as a smart phone, a tablet computer, etc., and an image capture card, a video capture card, etc. The image acquisition unit 401, when in an operation state, can automatically focus, automatically zoom in to an appropriate multiple, and can provide an operation of manual zoom in by a user, which can display an image of a thing to be photographed on a display screen of a camera, video camera, mobile device, computer, or the like, on which the user can know a photographed image to be obtained by photographing in the current state, thus facilitating observation and adjustment during photographing.

The image acquisition unit 401 of the present embodiment can realize some or all of the operations in step S1 of the bird recognition method of the present invention described above. For example, after the shooting device 401a in the image obtaining unit 401 is configured to automatically focus the birds, the birds are automatically amplified to an automatic amplification threshold preset in the shooting device, and then the user is guided to perform manual amplification operation through corresponding amplification gestures and/or texts; when the shooting device 401a is provided with a tele lens, after a user manually enlarges an image after the shooting device is automatically focused according to guidance, the tele lens is called to shoot the birds; when the photographing device 401a has no tele lens, after the user manually enlarges the image after the photographing device is automatically focused according to guidance, the user selects a lens with a proper focal length according to the enlarging operation of the user to photograph the bird.

The region acquisition unit 402 is configured to acquire a bird region image in which birds are present from the image to be recognized acquired by the image acquisition unit 401. The region acquiring unit 402 may internally set a bird region recognition model such as a multi-object detection (object detection) model, a convolutional neural network CNN recognition model or a mask-rcnn recognition model, which can recognize a region where birds are located from an image to be recognized and mark the region by a marking frame, as shown in fig. 4, and further may further slice by acquiring the marking frame, thereby forming a corresponding bird region image, as shown in fig. 7.

The region acquisition unit 402 of the present embodiment may realize part or all of the functions in step S2 of the bird recognition method of the present invention described above. For example, the region acquisition unit 402 includes a slicing module 402b and a bird region recognition module 402a provided with a bird region recognition model. The bird region recognition module 402a is configured to automatically recognize a region where birds are located from the image to be recognized acquired by the image acquisition unit 401 through a corresponding bird region recognition model, and form a labeling frame; the slicing module 402b is configured to slice the region where the birds in the image to be identified are located according to the labeling frame or the adjusted labeling frame, so as to form a bird region image containing the birds.

The image optimizing unit 403 is configured to perform image optimization on the bird region image output by the region acquiring unit 402 so that birds in the optimized bird region image are clearly displayed. The image optimizing unit 403 performs different image optimizing operations when performing image optimization on the bird region images of different image optimizing units 403, so as to improve the recognition accuracy. The specific parameter includes at least one of an image resolution, an image area, an image long side size, an image wide side size, and the like.

The image optimizing unit 403 of the present embodiment may realize part or all of the functions in step S3 of the bird recognition method of the present invention described above. For example, the image optimization unit 403 includes a judgment module 403a, a super-resolution engine 403b, and a beautification engine 403c. Wherein the judging module 403a is configured to judge the magnitudes of the specific parameters of the bird region image output by the image optimizing unit 403 with respect to the first threshold value and the second threshold value, and when it is judged that the specific parameters of the bird region image are greater than or equal to the second threshold value, image optimization is not performed on the bird region image; the super-cleaning engine 403b is configured to perform super-cleaning processing on the bird region image when the determination module 403a determines that a specific parameter of the bird region image is less than or equal to a first threshold value; the beautification engine 403c is configured to beautify the bird region image when the judgment module 403a judges that a specific parameter of the bird region image is greater than the first threshold value and less than the second threshold value.

The bird recognition engine 404 is configured to obtain a final bird recognition result based on the optimized bird region image output by the image optimizing unit 403 and the information of the shooting region corresponding to the image to be recognized acquired by the image acquiring unit 401.

The bird recognition engine 404 of the present embodiment may implement part or all of the functions in step S4 of the bird recognition method of the present invention described above.

It will be appreciated that image acquisition unit 401, region acquisition unit 402, image optimization unit 403, and bird recognition engine 404 may be incorporated into one device for implementation, or any one of the units may be split into multiple devices, or at least some of the functionality of one or more of the units may be combined with at least some of the functionality of the other units and implemented in one device. According to embodiments of the invention, at least one of image acquisition unit 401, region acquisition unit 402, image optimization unit 403, and bird recognition engine 404 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), programmable Logic Array (PLA), system-on-chip, system-on-substrate, system-on-package, application Specific Integrated Circuit (ASIC), or in any other reasonable manner of integrating or packaging circuitry, or in any other reasonable combination of hardware, software, and firmware. Alternatively, at least one of the image acquisition unit 401, the region acquisition unit 402, the image optimization unit 403, and the bird recognition engine 404 may be at least partially implemented as computer program modules, which may perform the functions of the respective modules when the program is run by a computer.

For example, the image acquisition unit 401 and the area acquisition unit 402 are provided on a local device (e.g., a mobile device such as a cellular phone, a digital camera, a smart watch, etc.), and may be integrated into one application client APP. The image optimizing unit 403 and the bird recognition engine 404 are arranged on the same or different servers in the cloud, and can communicate with the application client APP of each user in the manner of internet and the like so as to realize bird recognition, thereby greatly improving the resource utilization rate and reducing the cost of the user terminal equipment.

For another example, the image acquisition unit 401, the area acquisition unit 402, and the determination module 403a of the image optimization unit 403 are provided on a local device (for example, a mobile device such as a mobile phone, a digital camera, a smart watch, etc.), and may be integrated as one application client APP. The ultra-cleaning engine 403b, the beautifying engine 403c and the bird recognition engine 404 of the image optimization unit 403 are arranged on the same or different servers in the cloud, and can communicate with the application client APP of each user in the internet or the like to realize bird recognition, so that the resource utilization rate can be greatly improved, and the cost of the user terminal equipment can be reduced.

For another example, the image acquisition unit 401, the area acquisition unit 402, and the image optimization unit 403 are provided on a local device (e.g., a mobile device such as a mobile phone, a digital camera, a smart watch, etc.), and may be integrated as one application client APP. The bird recognition engine 404 is separately arranged on the cloud server and can communicate with the application client APP of each user in the manner of internet and the like so as to realize bird recognition, thereby greatly improving the resource utilization rate and reducing the cost of the user terminal equipment.

In summary, according to the technical scheme provided by the invention, the bird region image in which the birds are located is obtained by identifying the acquired bird-containing image to be identified, and then the bird region image is subjected to image optimization, so that the birds in the optimized bird region image can be clearly displayed, and further, the bird identification engine can identify the birds in the bird region image by combining the information of the shooting region corresponding to the image to be identified, so that the accuracy of the bird identification result is improved.

The foregoing description is only illustrative of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention in any way, and any changes and modifications made by those skilled in the art in light of the foregoing disclosure will be deemed to fall within the scope and spirit of the present invention.

Claims

1. A method of bird identification comprising:

acquiring an image to be identified containing birds;

2. The bird recognition method according to claim 1, wherein the bird is automatically focused by a photographing device, and an enlarged photograph of a picture is taken to obtain the image to be recognized; or automatically focusing the birds by using shooting equipment, performing video amplification shooting, and obtaining the image to be identified by acquiring each frame of picture from the shot video.

3. The bird recognition method of claim 2, wherein the step of performing the zoom-in photographing includes:

Judging whether the shooting equipment is provided with a tele lens or not;

4. The bird recognition method of claim 1, wherein the step of acquiring an image of a bird region where the bird is located from the image to be recognized includes:

5. The bird recognition method of claim 4, wherein after the marking frame is formed and before the region in which the birds in the image to be recognized are located is sliced, the size of the marking frame is automatically or manually adjusted, and the region in which the birds in the image to be recognized are sliced according to the adjusted marking frame to form the bird region image.

6. The bird recognition method according to claim 4, wherein when a plurality of bird targets are present in the image to be recognized, the bird region recognition method is adopted to select, as the bird region image, a region in which one bird target is present from regions in which the plurality of bird targets are respectively present in the image to be recognized.

7. The bird identification method of claim 1, wherein the bird region images are different in specific parameters including at least one of image resolution, image area, image long side size, image wide side size, and image optimization operations are different when the bird region images are image optimized.

8. The bird identification method of claim 7, wherein the image optimization operation adopted for the bird region image having the different specific parameters includes:

9. The bird recognition method of claim 8, wherein the respective bird region images are subjected to the super-cleaning process by a super-cleaning engine; and beautifying the corresponding bird region image through a beautifying engine or a computer vision algorithm.

10. The bird recognition method of claim 1, wherein inputting the optimized bird region image and the information of the photographed region corresponding to the image to be recognized into the corresponding bird recognition engine, the step of obtaining the final bird recognition result comprises:

11. The bird recognition method of claim 10, wherein the global recognition result and the regional recognition result are arranged in order of high confidence, respectively, and top1 and top2 of global are ranked two times in the global recognition result, and top1 and top2 of region are ranked two times in the regional recognition result; if the global recognition confidence of the top1 of global is larger than the region recognition confidence of the top1 of region by a preset coefficient, the output final bird recognition result in the bird region image consists of the top1 and the top2 of global and the top1 of region; if the global recognition confidence of the top1 of the global is smaller than or equal to the region recognition confidence of the top1 of the region by a preset coefficient, the output final bird recognition result in the bird region image consists of the top1 of the region, the top2 and the top1 of the global.

12. A readable storage medium having a program stored thereon, wherein the program when executed implements the bird identification method according to any one of claims 1 to 11.

13. A bird recognition device comprising a processor and a memory, the memory having stored thereon a program which, when executed by the processor, implements the bird recognition method of any one of claims 1 to 11.

14. A bird identification system, comprising:

and the bird recognition engine is configured to obtain a final bird recognition result according to the optimized bird region image and the information of the shooting region corresponding to the image to be recognized.

15. The bird recognition system of claim 14, wherein the image acquisition unit comprises a photographing device configured to automatically zoom in to an automatic zoom-in threshold preset in the photographing device after automatically focusing the bird, and then guide a user to perform a manual zoom-in operation through a corresponding zoom-in gesture and/or a document;

16. The bird recognition system of claim 14, wherein the region acquisition unit comprises:

17. The bird recognition system of claim 14, wherein the image optimization unit comprises: