WO2023093480A1

WO2023093480A1 - Species touring method and system, and readable storage medium

Info

Publication number: WO2023093480A1
Application number: PCT/CN2022/129296
Authority: WO
Inventors: 徐青松; 李青
Original assignee: 杭州睿胜软件有限公司
Priority date: 2021-11-23
Filing date: 2022-11-02
Publication date: 2023-06-01
Also published as: CN114154699A

Abstract

The present invention provides a species touring method and system, and a readable storage medium. The method comprises: displaying, according to position information of a region selected by a user, species information and/or path information in the region; generating a corresponding species touring initial path according to species and/or a path selected by the user; and acquiring time information and current season information of the current region, screening the species on the species touring initial path, and generating a final species touring path.

Description

Species tour method, tour system and readable storage medium

technical field

The invention relates to the technical field of object recognition, in particular to a species tour method, a tour system and a readable storage medium.

Background technique

At present, more and more people like to go hiking in the wild, and there are usually a large number of different species distributed in the wild, such as plants, animals, birds, mushrooms, minerals, etc., and are located in different locations. If people are not familiar with the distribution of species in the wild, it takes more time to find the location of the species they want to watch, resulting in a poor experience when visiting species.

Contents of the invention

One of the purposes of the present disclosure is to provide a species tour method, including:

According to the location information of the area selected by the user, display species information and/or path information in the area;

According to the species and/or routes selected by the user, generate corresponding species tour initial routes;

The time information of the current area and the current season information are obtained, the species on the initial path of the species tour are screened, and the final species tour route is generated.

In some embodiments, the final species tour route includes a plurality of species information marked in the first manner.

In some embodiments, the species information and route information are obtained from a species and tour information database, and users can add their own tour route information and photographed species images to the species and tour information database.

In some embodiments, the species information and route information include location information and altitude information.

In some embodiments, the route information, the initial species tour route and/or the final species tour route are displayed in the form of a 3D route.

In some embodiments, the method further includes acquiring the species image taken by the user, and acquiring the species information through the species recognition model, and marking it on the initial path of the species tour and/or the final route according to the position information of the species image. on the species tour route.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the location of the corresponding species, giving reminder information based on interaction.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species image taken by the user, and acquiring the identification information of the species through the species identification model , when the identification information of the species is inconsistent with the species information at the current location on the final tour route of the species, a reminder message is given based on an interactive manner.

In some embodiments, when the identification information of the species and the species information at the current position on the final tour route of the species belong to the preset confusing species, the distinguishing features between the two are provided based on the interaction.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species environment image taken by the user, and judging its relationship with the species and tour through the environment recognition model The similarity of the historical environmental images of the current location in the information database, in order to judge whether the user can accurately find the location of the corresponding species.

In some embodiments, acquiring the environmental image of the species taken by the user, and judging its similarity with the historical environmental image of the current location in the species and tourist information database through the environment recognition model, so as to determine whether the user can accurately find the position of the corresponding species also includes :

Identify multiple species information in the species environment images and historical environment images taken by the user and the positional relationship information between them through the species recognition model, so as to judge the current position in the species environment images taken by the user and the species and tour information database The similarity of historical environmental images.

In some embodiments, the method further includes updating the final species tour route according to the user's actual tour route and species images captured on the user's actual tour route.

According to another aspect of the present disclosure, a readable storage medium is provided, on which a program is stored, and when the program is executed, the above-mentioned species tour method is implemented.

According to yet another aspect of the present disclosure, a species tour system is proposed, including a processor and a memory, the memory stores a program, and when the program is executed by the processor, the species tour method as described above is implemented.

Other features of the present disclosure and advantages thereof will become more apparent through the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which constitute a part of this specification, illustrate the embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

Fig. 1 shows the flow chart of the species tour method of the embodiment of the present invention;

FIG. 2 is a schematic diagram of path information in a display area according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of species information in a display area according to an embodiment of the present invention;

Fig. 4 is a schematic diagram of generating the final species tour route according to the embodiment of the present invention;

Fig. 5 is a schematic diagram of the final species tour route according to another embodiment of the present invention.

Note that in the embodiments described below, the same reference numerals may be used in common between different drawings to denote the same parts or parts having the same functions, and repeated descriptions thereof will be omitted. In some instances, similar reference numerals and letters are used to denote similar items, so that once an item is defined in one figure, it does not require further discussion in subsequent figures.

In order to facilitate understanding, the position, size, range, etc. of each structure shown in the drawings and the like may not represent the actual position, size, range, and the like. Therefore, the present disclosure is not limited to the positions, dimensions, ranges, etc. disclosed in the drawings and the like.

Detailed ways

Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that relative arrangements of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and in no way intended as any limitation of the disclosure, its application or uses. That is, the structures and methods herein are presented by way of example to illustrate various embodiments of the structures and methods of this disclosure. However, those skilled in the art will appreciate that they illustrate merely exemplary, and not exhaustive, ways in which the disclosure may be practiced. Furthermore, the figures are not necessarily to scale and some features may be exaggerated to show details of particular components.

Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered part of the Authorized Specification.

In all examples shown and discussed herein, any specific values should be construed as illustrative only, and not as limiting. Therefore, other examples of the exemplary embodiment may have different values.

Fig. 1 shows the flow chart of the species tour method according to the embodiment of the present invention, which can be realized in the application program (app) installed on smart terminals such as mobile phones and tablet computers. As shown in Figure 1, the species tour method includes:

Step S1: According to the location information of the area selected by the user, display species information and/or path information in the area;

Step S2: According to the species and/or route selected by the user, generate a corresponding initial route for species tour;

Step S3: Obtain the time information of the current area and the current season information, filter the species on the tour route, and generate the final species tour route.

Please refer to FIG. 2 and FIG. 3 . FIG. 2 is a schematic diagram showing path information in a display area according to an embodiment of the present invention, and FIG. 3 is a schematic diagram showing species information in a display area according to an embodiment of the present invention. Users can switch different display options in the user interface to display species information or path information in the area respectively. For example, in FIG. 2 , there are

multiple path information

110 , 120 , 130 , and 140 in the current area 100 selected by the user, and the user can select one of them as a species tour path. In some embodiments, the plurality of route information includes one or more related information of route name, route location, route length, route time, species information on the route, etc., and the user can click on the icon or slide to switch between different display modes. Detailed related information 150 of the route information.

As shown in Figure 3, when the user switches to displaying species information,

multiple species information

210, 220, 230, 240 in the current area 200 will be displayed on the user interface, and the user can click on the icon or slide to switch to view different species information 250 for details.

Please refer to FIG. 4 again. FIG. 4 is a schematic diagram of generating a final species tour route according to an embodiment of the present invention. In some embodiments, firstly, according to multiple species or a certain route selected by the user, a corresponding species tour initial route is generated, and then the time information of the current area and the current season information are obtained, and the species tour initial route is The species are screened, and a final species tour route 300 is generated, which includes

species

310 and 320 , and detailed information 330 of the species tour route 300 can also be displayed. In some embodiments, the species includes at least one of plants, animals, birds, mushrooms, and minerals. Of course, other species may also be included, which is not limited in this application. In some embodiments, when a certain species on the initial tour route is judged comprehensively based on the time information of the current area and the current season information, it is not in the period for users to watch, and the species can be placed in the Deletion processing is carried out on the initial tour route, for example, some plants may be in the withering or dormant period, some animals may be in the hibernation period, some birds may be in the migration period, and some mushrooms are in the non-growing period. Deletion can be performed on the initial tour route to avoid confusion when the user cannot find the corresponding species when they arrive at the corresponding location point. When deleting the corresponding species, a prompt message can be given to inform the user that the current species is not available within the current time period. viewing period.

In some embodiments, the final species tour route includes a plurality of species information marked in the first manner, and the user can check which species information is included in different positions on the current species tour route, and the distance from the user's current position to the next or other Species distance information and mutual positional relationship, etc., users can also choose to click on different markers to view the corresponding species.

In some embodiments, the species information and route information are obtained from the species and tour information database, which has basic species information and tour route information, and can automatically obtain relevant information in the current area according to the user's location And display, the species and tour information database also includes related information uploaded by other users, and users can also choose to add their own tour route information and photographed species images to the species and tour information database.

In some embodiments, the species information and route information include location information and altitude information, and the route information and species tour route are displayed in the form of a 3D route. For example, in areas such as mountains or canyons with a certain altitude difference in the wild, locations with the same GPS location information may have different altitudes, which need to be distinguished accordingly. Therefore, adding altitude information and a 3D path including altitude information for display can be The location information of different species is displayed more accurately, which is convenient for users to view and search.

In some embodiments, the method further includes acquiring a species image taken by a user, acquiring species information through a species recognition model, and marking it on the species tour route according to the location information of the species image. Users may find the species they are interested in on the species tour path, and it will be recorded in the form of taking photos. The species identification model can be used to identify and obtain the relevant information of the species, and at the same time, its location information can be recorded on the species tour path , so that subsequent users can view the specific location information where the species was found.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the location of the corresponding species, giving reminder information based on interaction. Due to the complex field environment and many interference factors, users can easily miss the location of species on the tour path, so it is necessary to give a timely reminder when the user arrives at the specific location of the corresponding species. A method based on interactive reminders can be used to facilitate users to discover the species they choose to watch in time, and avoid accidentally missing or omitting the corresponding viewing location.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species image taken by the user, and acquiring the identification information of the species through the species identification model , when the identification information of the species is inconsistent with the species information at the current location on the final tour route of the species, a reminder message is given based on an interactive manner. Users may not be particularly familiar with the species they choose to watch. When they arrive at the location of the corresponding species, they need to take pictures of the species and use the species recognition model to assist in judging whether the corresponding species has been found accurately. The species recognition model will be based on the user's The captured species image is identified and processed to obtain the identification information of the species, and then judge whether it is consistent with the species information at the current location on the species tour path. When the two are inconsistent, it is judged that the user has not found the accurate species and needs to be reminded It is convenient for the user to search for the corresponding species again.

In some embodiments, when the identification information of the species and the species information at the current position on the final tour route of the species belong to the preset confusing species, the distinguishing features between the two are provided based on the interaction. In real environments, there are usually multiple species with similar morphology. In order to judge whether the user has accurately found the corresponding species, the system pre-creates a database of confusing species, which contains the names of multiple groups of confusing species and the distinguishing characteristics of these confusing species. Features should take the rules of the shoot. When the species image taken by the user belongs to a certain confusing species in the confusing species database after being identified and processed by the species recognition model, a reminder message will be output, and the distinguishing features between the two will be given to facilitate the user to judge whether the corresponding species has been found accurately. Species, in addition, the shooting rules that should be used for distinguishing features can be given, so that users can re-shoot and confirm their species information again through the species identification model.

In some embodiments, the method further includes, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species environment image taken by the user, and judging its relationship with the species and tour through the environment recognition model The similarity of the historical environmental images of the current location in the information database, in order to judge whether the user can accurately find the location of the corresponding species. Due to the complexity of the wild environment, there may be many different species in the same location, but the existing location positioning technology solutions (such as GPS or Beidou positioning system) cannot reach a very accurate level. When the user arrives at the location of the corresponding species, in the current In the environment of multiple species near the location, it is impossible to effectively determine which direction needs to be searched in a small area. Therefore, certain technical means are required to accurately determine whether the user can accurately find the location of the corresponding species. After arriving at the location of the corresponding species, the user can take a picture of the species environment, and use the environment recognition model to judge its similarity with the historical environment image of the current location in the species and tour information database, that is, compare the species environment image taken by the user through the environment recognition model The similarity between the feature information and the feature information of the historical environmental images stored in the database, when the similarity of the feature information between the two is greater than a preset threshold, for example, when the similarity is greater than 70%, it is judged that the user has found the exact location of the corresponding species.

In some embodiments, acquiring the environmental image of the species taken by the user, and judging its similarity with the historical environmental image of the current location in the species and tourist information database through the environment recognition model, so as to determine whether the user can accurately find the position of the corresponding species also includes : Recognize multiple species information in the species environment images and historical environment images taken by the user and their mutual positional relationship information through the species recognition model, so as to judge the species environment images taken by the user and the current species and tour information database The similarity of the location's historical environment image. For example, through the species recognition model, obtain multiple species information in the species environment image taken by the user, especially the species information in fixed positions such as plants, and obtain the positional relationship between them, and at the same time obtain the species and tour information database through the species recognition model Multiple species information in the historical environmental image of the current location, also obtain the positional relationship between them, and then compare the image taken by the user with the species information of the historical image, and the similarity of the positional relationship between species, so as to judge whether the user is accurate The corresponding species environment location was found. The user can take multiple directions of species environment images at the currently arrived species location, or take multiple images of different species environments at multiple locations near the current location, and identify and judge through the environment recognition model in turn. Species information in fixed positions and positional relationship information among them are comprehensively judged until an accurate corresponding species environmental position is found, which is convenient for users to accurately find corresponding species in this species environmental position for viewing.

Please refer to FIG. 5 , in some embodiments, the method further includes updating the final species tour route according to the user's actual tour route and species images captured on the tour route. During the tour, the user may find multiple species 340 of interest other than the pre-selected

ornamental species

310, 320 on the tour path 300. When the user takes an image of the corresponding species, the system will record the position of the species , and mark it on the final species tour route in the same way, the user can click on the species icon to view its detailed information 350, and the detailed information 330 of the tour route; at the same time, the user may change the tour route during the tour, so when the user At the end of the tour, the actual tour path will be recorded, and the information of each species taken by the user will be marked on the actual tour path.

In some embodiments, the species identification model may be a neural network model, specifically a convolutional neural network model or a residual network model. The convolutional neural network model is a deep feedforward neural network, which uses a convolution kernel to scan the species image, extracts the features to be identified in the species image, and then identifies the features to be identified in the species. In addition, in the process of identifying species images, the original species images can be directly input into the convolutional neural network model without preprocessing the species images. Compared with other recognition models, the convolutional neural network model has higher recognition accuracy and recognition efficiency.

Compared with the convolutional neural network model, the residual network model has more identity mapping layers, which can avoid the phenomenon of saturation or even decline in the accuracy rate as the network depth (the number of stacked layers in the network) increases. The identity mapping function of the identity mapping layer in the residual network model needs to satisfy: the sum of the identity mapping function and the input of the residual network model is equal to the output of the residual network model. After the introduction of identity mapping, the residual network model has more obvious changes in the output, so the recognition accuracy and recognition efficiency of species can be greatly improved.

In some embodiments, training a species recognition model may include:

Obtaining a first sample set of species images with a first preset number of species annotated;

Determining species images of the first proportion from the first sample set as the first training set;

training a species recognition model using the first training set; and

When the first training accuracy rate is greater than or equal to the first preset accuracy rate, the training ends, and a trained species recognition model is obtained.

Specifically, a large number of species images may be included in the first sample set, and each species image is correspondingly marked with a species. Input the species image into the species recognition model to generate the output species, and then according to the comparison results between the output species and the marked species, the relevant parameters in the species recognition model can be adjusted, that is, the species recognition model is trained until the species The training ends when the first training accuracy rate of the identification model is greater than or equal to the first preset accuracy rate, so as to obtain the trained species identification model. According to a species image, the species recognition model can also output multiple candidate species, where each candidate species can have its corresponding species confidence level for further analysis and screening.

Further, the trained species recognition model can also be tested, which can specifically include:

Determining species images of a second scale from the first sample set as a first test set;

Using the first test set to determine a first model accuracy rate of the trained species identification model; and

When the accuracy of the first model is lower than the second preset accuracy, the first training set and/or the species recognition model are adjusted for retraining.

In general, the species images in the first test set and the first training set are not exactly the same, so the first test set can be used to test whether the species recognition model has a good recognition effect on species images other than the first training set . During the test, the first model accuracy rate of the species recognition model is calculated by comparing the output species generated from the species images in the first test set with the annotated species. In some examples, the calculation method of the first model accuracy rate may be the same as the calculation method of the first training accuracy rate. When the accuracy of the first model obtained by the test is less than the second preset accuracy, it indicates that the recognition effect of the species recognition model is not good enough, so the first training set can be adjusted, for example, the number of species marked in the first training set can be increased. The number of species images, or adjust the species recognition model itself, or both, and then retrain the species recognition model to improve its recognition performance. In some embodiments, the second preset accuracy rate may be set equal to the first preset accuracy rate.

This embodiment also provides a readable storage medium on which a program is stored, and when the program is executed, the above-mentioned species tour method is realized. Further, this embodiment also provides a species tour system, including a processor and a memory, the memory stores a program, and when the program is executed by the processor, the species tour method as described above is realized.

Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art should understand that the above examples are for illustration only, rather than limiting the scope of the present disclosure. The various embodiments disclosed herein can be combined arbitrarily without departing from the spirit and scope of the present disclosure. Those skilled in the art will also understand that various modifications may be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

A kind of species tour method is characterized in that, comprising:

According to the location information of the area selected by the user, display species information and/or path information in the area;

According to the species and/or routes selected by the user, generate corresponding species tour initial routes;

The time information of the current area and the current season information are obtained, the species on the initial path of the species tour are screened, and the final species tour route is generated.
The species tour method according to claim 1, characterized in that the final species tour route includes a plurality of species information marked in the first manner.
The species tour method according to claim 1, wherein the species information and path information are obtained from the species and tour information database, and users can add their own tour route information and photographed species images to the species and tour information. information database.
The species tour method according to claim 1, wherein the species information and path information include location information and altitude information.
The species tour method according to claim 4, wherein the route information, the species tour initial route and/or the final species tour route are displayed in the form of a 3D route.
The species tour method according to claim 1, characterized in that the method also includes acquiring species images taken by users, and acquiring species information through species identification models, and marking them on the species list according to the position information of the species images Tour the initial route and/or the final species tour route.
The species tour method according to claim 1, further comprising, according to the species information on the final species tour route, when the user arrives at the location of the corresponding species, giving reminder information based on interaction.
The species tour method according to claim 1, characterized in that the method further comprises, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species image taken by the user, and passing the species The identification model acquires the identification information of the species, and when the identification information of the species is inconsistent with the species information at the current location on the final species tour route, a reminder message is given based on an interaction.
The species tour method according to claim 8, characterized in that, when the identification information of the species and the species information at the current position on the final species tour route belong to the pre-set confusing species, the interactive method is given distinguishing features between the two.
The species tour method according to claim 7, characterized in that, the method further comprises, according to the species information on the final species tour route, when the user arrives at the position of the corresponding species, acquiring the species environment image taken by the user, and passing The environment recognition model judges its similarity with the historical environment image of the current location in the species and tourist information database, so as to judge whether the user can accurately find the location of the corresponding species.
The species tour method according to claim 10, characterized in that the species environment image taken by the user is obtained, and the similarity between the species and the historical environment image of the current location in the species and tour information database is judged by the environment recognition model, so as to judge the user Whether the location of the corresponding species is accurately found also includes:

Identify multiple species information in the species environment images and historical environment images taken by the user and the positional relationship information between them through the species recognition model, so as to judge the current position in the species environment images taken by the user and the species and tour information database The similarity of historical environmental images.
The species tour method according to claim 1, further comprising updating the final species tour route according to the user's actual tour route and species images captured on the user's actual tour route.
A readable storage medium, on which a program is stored, is characterized in that, when the program is executed, the species tour method according to any one of claims 1-12 is realized.
A species tour system, characterized in that it includes a processor and a memory, and a program is stored in the memory, and when the program is executed by the processor, the species according to any one of claims 1-12 is realized. Tour method.