WO2024005070A1

WO2024005070A1 - Fish species discrimination method, server, program and fish species discrimination system

Info

Publication number: WO2024005070A1
Application number: PCT/JP2023/023973
Authority: WO
Inventors: Akinori KASAI; Yuta HIRABAYASHI; Masashi Muragaki
Original assignee: Furuno Electric Co., Ltd.
Priority date: 2022-06-30
Filing date: 2023-06-28
Publication date: 2024-01-04
Also published as: JP2024005623A

Abstract

To provide a fish species discrimination method, a server, a program that can improve the accuracy of fish species discrimination results by machine learning models, and a fish species discrimination system. To solve this problem, a fish species discrimination method acquires feedback information associating echo data with fish species, stores the acquired feedback information by associating the feedback information with attributes including at least the fishing styles are used, generates an individual model (312) for fish species discrimination for each attribute by machine learning using multiple feedback information for which the attributes are identical, and uses the generated individual models to determine fish species by echo signal being transmitted into water body and reflected by an object in the water body corresponding to the attributes.

Description

FISH SPECIES DISCRIMINATION METHOD, SERVER, PROGRAM AND FISH SPECIES DISCRIMINATION SYSTEM

The present disclosure relates to a fish species discrimination method for discriminating fish species using a machine learning model, a server for generating the machine learning model for fish species discrimination, a program for making a computer execute a function for generating the machine learning model for fish species discrimination, and a fish species discrimination system equipped with the server, and.

Background

Fish finders have been known to detect fish school in the water. In this type of fish finder, ultrasonic waves are transmitted underwater and the reflected echo signal (reflected waves) are received. Echo data is generated according to the intensity of the reflected echo signal received, and an echo image is displayed based on the echo data generated. A user may confirm the fish school from the echo image, and the capture of the fish school can proceed smoothly.

In this case, it is preferable that the fish species may be further discriminated for the fish school on the echo image. Thus, the user may efficiently capture the fish of the fish species that the user wants.

To discriminate the fish species, for example, a machine learning algorithm can be used. In this case, a learned model is generated by learning the machine learning algorithm using the echo data output from the fish finder as input data and the fish species of the fish school on the echo data as teacher data. The fish species of the fish school (teacher data) on the echo data is input by the user based on the actual capture, for example. A Patent publication describes a fish species estimation system for this species.

The Patent publication is JP2019-200175.

Summary

In the method described above, a unique machine learning model can be generated for each user. However, in the method described above, the number of teacher data is limited because the teacher data is input by the user based on the actual capture. For this reason, it is difficult to accurately determine the fish species by the machine learning model.

One possible solution is to aggregate standard data generated by experts for machine learning, for example, and use the aggregated standard data as teacher data to learn machine learning algorithms.

However, when the fishing style used by fish finder is different, the characteristics of the echo data acquired by fish finders differ. That is, when fish finders are installed in fixed nets and fishing boats, the speed and depth of fish swimming differ, and the characteristics of the echo data acquired by fish finders differ. For this reason, even if standard data is used as it is for learning machine learning algorithms, it is difficult to obtain highly accurate results for identifying fish species.

In view of this problem, the present disclosure aims to provide a fish species discrimination method, a server, a program that can provide highly accurate results for identifying fish species using a machine learning model, and a fish species discrimination system.

A first embodiment of the present disclosure relates to a fish species determination method. The fish species determination method according to this embodiment acquires feedback information associating echo data with fish species, stores the acquired feedback information by associating it with attributes including at least the fishing styles used, generates individual models for fish species determination for each of the attributes by machine learning using the multiple pieces of feedback information with the same attributes, and uses the generated individual models to determine fish species by echo signal being transmitted into water body and reflected by an object in the water body corresponding to the attributes.

According to the fish species determination method, according to this embodiment, since the feedback information is acquired and aggregated from the multiple underwater detection devices, the number of feedback information used for the machine learning can be increased. Also, among the aggregated feedback information, the feedback information with the same attributes including the fishing style is used for the machine learning, so that the echo data with different characteristics can be suppressed from being used for the machine learning of individual models. Therefore, it is possible to improve the accuracy of fish species discrimination results by individual models.

In the fish species determination method, in this embodiment, the attributes may further include the area to which the user using the underwater detection device belongs.

If the fishing styles are the same but the areas are different, the characteristics of the fish may differ and the characteristics of the echo data may differ. Therefore, by further restricting the feedback information used to learn the individual models in the areas, the results of fish species discrimination by the individual models can be further refined.

The fish species determination method in this embodiment may be configured to generate a standard model for fish species discrimination by the machine learning using standard data prior to the generation of the individual model, and to generate the individual model by acquiring information indicating the user's modification to the fish species discrimination result by the standard model from the underwater detection device as the feedback information.

With this configuration, while providing the user with the fish species discrimination result by the standard model, the user's modification to the fish species discrimination result can be smoothly acquired from the underwater detection device as the feedback information. Therefore, the individual model can be generated appropriately and efficiently while maintaining the user's convenience.

The fish species determination method in this embodiment can be configured to update the individual model by acquiring as the feedback information indicating the user's modification to the fish species determination result by the individual model.

According to this configuration, the individual model can be updated to gradually adapt to the user's attributes. Therefore, the user's convenience can be enhanced.

The fish species determination method in this embodiment may be configured to receive customization information, including changes in the output conditions of the individual model, entered by the user of each of the underwater detection devices, and to change the output conditions of the individual model for the underwater detection devices based on the received customization information.

According to this configuration, the individual model can be customized for user-friendly use.

In this case, the modification of the output condition may include preferentially outputting the specific fish species as the discriminated result even if the prediction probability of the individual model for the specific fish species specified by the user is lower than that of other fish species, provided that the prediction probability of the specific fish species is not less than a predetermined threshold.

Thus, even if the prediction probability of the specific fish species is lower than that of other fish species, the discriminated result of the specific fish species is outputted, and the output frequency of the discriminated result of the specific fish species can be increased. Therefore, the user is less likely to miss the catch of the specific fish species the user desires, and the catch of the specific fish species can be increased.

The change in the output condition may also include changing the lower limit of the predicted probability for outputting the discriminant result for the specific fish species specified by the user.

Thus, for example, by lowering the lower limit value for the specific fish species, the user can output the discriminant result for the specific fish species even if the predicted probability of the specific fish species is low, thereby more reliably catching the fish that the user wants to catch. Alternatively, by raising the lower limit value for the specific fish species, the user can reduce the frequency with which the discriminant result for the specific fish species is output, thereby more efficiently confirming the fish that the user wants to catch.

The change in the output condition may also include not outputting the discriminant result for the specific fish species specified by the user.

This enables the user to suppress outputting the discriminant result for the fish species that the user is not interested in and to smoothly and efficiently grasp the fish school of the fish species that the user is interested in.

The second embodiment of the present disclosure relates to a server capable of communicating with multiple underwater detection devices. The server according to this embodiment acquires feedback information associating an echo data and the fish species from each of the underwater detection devices, stores the acquired feedback information by associating it with attributes including at least the fishing style in which the underwater detection devices are used, and generates an individual model for fish species discrimination for each of the attributes by machine learning using multiple pieces of feedback information with the same attributes. The server according to this embodiment may include a communication interface configured to acquire feedback information associating echo data with fish species from multiple underwater detection devices, a storage configured to store the acquired feedback information in a storage unit by associating the acquired feedback information with attributes including at least the fishing styles for which the underwater detection devices are used, and processing circuitry configured to generate an individual model for fish species discrimination for each of the attributes by machine learning using the multiple feedback information for which the attributes are identical.

According to the server according to this embodiment, the same effect as the first embodiment is achieved.

The third embodiment of the present disclosure relates to a program that makes a computer perform a prescribed function. The program according to the present embodiment includes a function that acquires feedback information associating echo data with fish species from a plurality of underwater detection devices, a function that stores the acquired feedback information in a storage unit by associating it with at least one attribute including a fishing style in which the underwater detection device is used, and a function that generates an individual model for fish species discrimination for each attribute by machine learning using the multiple pieces of feedback information with the same attribute.

According to the program according to this embodiment, the same effect as in the first embodiment is achieved.

The fourth embodiment of the present disclosure relates to a fish species discriminant system. The fish species discrimination system according to this embodiment is provided with the server according to the second embodiment and the underwater detection device.

According to the fish species discrimination system according to this embodiment, the same effect as the first embodiment is achieved.

In the fish species discrimination system according to this embodiment, the underwater detection device may be configured to include a display unit, an input unit, and a control unit that makes the display unit display the fish species discrimination result according to the individual model, accepts the correction of the discriminated result via the input unit, and transmits the correction to the server as feedback information.

According to this configuration, the correction of the fish species discrimination result according to the individual model can be provided to the server side at any time. Therefore, the individual model can be updated to gradually adapt to the attributes of the user, and the convenience of the user can be enhanced.

In this configuration, the control unit accepts, via the input unit, the input of customization information, including a change in the output conditions of the individual model, and transmits the input customization information to the server, which can be configured to change the output conditions of the individual model to the underwater detection device based on the received customization information.

According to this configuration, the individual models can be customized for user-friendly use.

As described above, according to the present disclosure, a fish species determination method, a server, a program that can make the result of fish species determination by a machine learning model highly accurate, and a fish species determination system can be provided.

The effect or significance of the present disclosure is further clarified by the following description of the embodiment. However, the following embodiment is only one example when implementing the present disclosure, and the present disclosure is not in any way limited to those described in the following embodiment.

Fig. 1 is a diagram showing a configuration of a fish species discrimination system, according to an embodiment. Fig. 2 is a block diagram showing the configuration of the fish species discrimination system, according to an embodiment. Fig. 3 is a diagram showing management status of various information in a storage unit of a server, according to an embodiment. Fig. 4 (a) is a diagram showing the configuration of user management information, according to an embodiment. Fig. 4 (b) to Fig. 4 (d) are diagrams showing the configuration of individual data, according to an embodiment. Fig. 5 is a diagram schematically showing the fish species discrimination processing by a neural network, according to an embodiment. Fig. 6 (a) is a flowchart showing application processing of a machine learning model, according to an embodiment. Fig. 6 (b) is a flowchart showing the fish species discrimination processing, according to an embodiment. Fig. 7 is a diagram schematically showing a display example of an echo image including the fish species discrimination result, according to an embodiment. Fig. 8 (a) is a flowchart showing the transmitting processing of feedback information executed by a control unit of the underwater detection device, according to an embodiment. Fig. 8 (b) is a flowchart showing the receiving process of the feedback information executed by the control unit of the server, according to the embodiment. Fig. 9 is a diagram schematically showing a screen for receiving modification of fish species from a user, according to the embodiment. Fig. 10 (a) is a flowchart showing the customized information transmitting processing performed by the control unit of the underwater detection device, according to the embodiment. Fig. 10 (b) is a flowchart showing the customized information receiving processing performed by the control unit of the server, according to the embodiment. Fig. 11 is a diagram schematically showing a display example of the fish species determination result when the output conditions of an individual model are changed based on the customized information, according to the embodiment.

DETAILED DESCRIPTION

Fig. 1 is a diagram showing a configuration of a fish species discrimination system (1).

The fish species discrimination system (1) is equipped with an underwater detection device (10) and a server (20). The underwater detection device (10) is a fish finder installed on a vessel or ship (2). The underwater detection device (10) may communicate with the server (20) via an external communication network (30) (For example, the Internet) and a base station (40). The underwater detection device (10) and the server (20) each hold address information for communicating with each other. The respective address information is set in the underwater detection device (10) and the server (20) at the initial setting.

The underwater detection device (10) is equipped with a transmitter/receiver (11) and a control unit (12). The transmitter/receiver (11) is installed on the bottom of the ship (2), and the control unit (12) is installed in the wheelhouse or the like of the ship (2). The transmitter/receiver (11) and the control unit (12) are connected by a signal cable (not shown). The transmitter/receiver (11) is equipped with an ultrasonic transducer for transmitting and receiving waves. The transmitter/receiver (11) transmits an ultrasonic wave (3) (transmitted wave) towards a seabed (4) and receives its reflected wave by the ultrasonic transducer in response to control from the control unit (12). The receiver (11) transmits a received signal based on the received reflected wave to the control unit (12).

The control unit (12) processes the received signal to generate echo data indicating the echo intensity at each depth. The control unit (12) arranges the echo intensity at each depth based on the echo data in time series to generate an echo image for one screen. The control unit (12) displays the generated echo image on the display unit. The control unit (12) updates the echo image for each ultrasonic wave transmitted and received. The user can grasp the presence and location of a fish school (5) by referring to the echo image.

Furthermore, the control unit (12) transmits the generated echo data to the server (20) at any time. The server (20) stores the received echo data and generates an echo image similar to that of the control unit (12). The server (20) determines the fish species of the school of fish included in the echo image by a machine learning model applied to the underwater detection device (10) of recipient. The server (20) transmits the determination result of the fish species to the underwater detection device (10) of recipient of the echo data together with the range (Depth, Time) of the school of fish to be determined.

Based on the received determination result and the range (Depth, Time) of the school of fish, the underwater detection device (10) superimposes the determination result of the fish species on the corresponding range on the echo image. Thus, the user can confirm the fish species of each school of fish on the echo image and smoothly go ahead to capture the desired fish.

When the determination result of the fish species provided by the server (20) is different from the fish species of the fish actually caught, the user sends feedback information for correcting the fish species to the server (20). For example, after the user finishes fishing for a day, the user performs an operation to acquire echo data for a predetermined time period of the day from the server (20) via the input unit of the underwater detection device (10). Accordingly, the server (20) transmits the echo data for the designated day and time period to the underwater detection device (10) along with the result of the determination of the fish species (including the range of the fish school to be determined). Based on the received echo data and the result of the determination, the underwater detection device (10) causes the display unit to display the echo image including the result of the determination.

Through the input unit, the user performs an operation to correct the discriminated result of the fish species displayed on the echo image to the fish species of the fish he has captured. With this, the underwater detection device (10) transmits feedback information including the user’s correction of the discriminated result and the range (Depth, Time) of the fish school corresponding to the discriminated result to the server (20). Using the received feedback information as teacher data, the server (20) performs machine learning of the machine learning model applied to the underwater detection device (10). Thus, the machine learning model is optimized to reflect the user’s fishing results.

The machine learning is performed using the feedback information from the underwater detection device (10), as described later, as well as the feedback information transmitted to the server (20) from other underwater detection devices with the same attributes (including at least fishing styles) as the underwater detection device (10). As a result, the accuracy of fish species discrimination in the machine learning model applied to the underwater detection device (10) is enhanced.

Further, the user inputs customized information including the change of output conditions of the machine learning model of the underwater detection device (10) via the input unit of the underwater detection device (10) as appropriate. The underwater detection device (10) transmits the input customization information to the server (20). The server (20) changes the output conditions of the machine learning model to the underwater detection device (10) based on the received customization information. With this, the user can make the echo image display the discriminated result of the fish species according to the user's own preference.

Although only one underwater detection device (10) is shown in Fig. 1, in fact, many underwater detection devices (10) can communicate with the server (20) via the external communication network (30) and the nearest base station (40). In addition, the underwater detection devices (10) that communicate with the server (20) include those installed in the vessel (2) as shown in Fig. 1, as well as several types of underwater detection devices with different fishing styles, such as underwater detection devices installed in fixed net.

Fig. 2 is a block diagram showing the configuration of the fish species discrimination system (1).

The underwater detection device (10) includes a control unit (101), a display unit (102), an input unit (103), a transmitter/receiver unit (104), a signal processing unit (105), a communication unit (106), and a position detection unit (107).

The control unit (101) is composed of a microcomputer, a memory, etc. The control unit (101) controls each part of the underwater detection device (10) according to a program stored in the memory. The program includes functions for receiving and displaying fish species determination results described below and receiving and transmitting feedback and customization information.

The display unit (102) is equipped with a monitor and displays a prescribed image by control from the control unit (101). The input unit (103) is equipped with a trackball for moving a cursor (not shown) on the image displayed on the display unit (102), an operation key, etc., and outputs a signal corresponding to the operation from the user to the control unit (101). The display unit (102) and the input unit (103) may be integrally constituted by a liquid crystal touch panel, etc.

The transmitter/receiver unit (104) includes the transmitter/receiver (11) shown in Fig. 1, a transmission circuit for supplying a transmission signal to the transmitter/receiver (11), and a reception circuit for processing the received signal output from the transmitter/receiver (11) and outputting it to the signal processing unit (105). The transmitting and receiving circuits are included in the control unit (12) of Fig. 1.

The transmitter/receiver unit (104) transmits the transmitted wave (ultrasonic wave) according to the control from the control unit (101). Here, in 1 sequence, two kinds of transmitted waves with different frequencies are transmitted sequentially in time division. The transmitter/receiver unit (104) receives the reflected waves of the transmitted waves of each transmitted frequency and outputs the received signal. The receiving circuit extracts the received signal of the frequency of each transmitted wave and outputs it to the signal processing unit (105).

The reason why the transmitted and received waves are performed at two different frequencies is to perform the fish species discrimination described later with higher accuracy. For example, the presence or absence of an airbladder causes a difference in the echo intensity at each frequency. Therefore, by referring to the difference in the echo intensity from a school of fish, the species of fish in that school can be accurately identified.

The signal processing unit (105) generates echo data indicating the intensity of the reflected wave according to the depth from the received signal of each frequency input from the transmitter/receiver unit (104), and outputs the generated two kinds of echo data to the control unit (101). The elapsed time from the timing of transmitting the transmitted wave of each frequency corresponds to the depth. Here, the intensity of the reflected wave attenuates as the depth increases. Therefore, the signal processing unit (105) corrects the intensity of the reflected wave that attenuates according to the elapsed time and outputs the corrected intensity echo data to the control unit (101) so that echo data can be quantitatively handled regardless of the difference in depth.

The control unit (101) generates an echo image based on the received echo data and causes the display unit (102) to display it. The control unit (101) generates echo data using the echo data corresponding to either frequency. The user may be able to switch as appropriate which frequency of echo data is used to generate the echo image. The control unit (101) generates one row of images in the depth direction, in which the echo intensity at each depth is expressed in gradation by a color scale, from the echo data. The control unit (101) integrates the images of each column from the present time to a predetermined time ago in the time direction to generate an echo image for one screen.

In the following explanation, when we refer to echo data (including echo data included in feedback information), we mean echo data of two different frequencies unless otherwise specified.

The communication unit (106) is a communication module capable of wireless communication with the base station (40). The position detection unit (107) is equipped with GPS and detects the position of the underwater detection device (10). The position detection unit (107) outputs the detected position information to the control unit (101).

As described with reference to Fig. 1, the control unit (101) transmits echo data, feedback information and customization information to the server (20) via the communication unit (106) at any time and receives the fish species determination result from the server (20) via the communication unit (106). The control unit (101) may further transmit the position information detected by the position detection unit (107) to the server (20).

As shown in Fig. 2, in addition to the underwater detection device (10), a number of underwater detection devices (10a, 10b and ...) can communicate with the server (20) via the external communication network (30) and the nearest base stations (40a, 40b and ....). As shown above, the underwater detection device (10) that communicates with the server (20) includes the one installed on the vessel (2) as shown in Fig. 1, as well as several types of underwater detection devices with different fishing styles, such as underwater detection devices installed in fixed net. The basic configuration of the other underwater detection devices is similar to that of the underwater detection device (10) in Fig. 2.

However, the underwater detection device (10) installed in the fixed net may consist of an offshore unit installed in the fixed net and a terminal capable of communicating with the offshore unit via the external communication network (30) in order for the user to remotely monitor the status of fish in the fixed net. The echo data acquired by the offshore unit is transmitted to the terminal via the external communication network (30). This causes the terminal to display the echo image. The terminal may be a personal computer, or a mobile terminal owned by the user, such as a mobile phone or tablet.

In this case, the terminal may transmit the echo data to the server (20), or the offshore unit may also transmit the echo data to the server (20) in parallel with the transmission of the echo data to the terminal. Also, the feedback and customization information may be input through the terminal and may be transmitted from the terminal to the server (20). The fish species determination result may be transmitted directly from the server (20) to the terminal without going through the offshore unit. Also, the echo data may be transmitted to the terminal from the server (20). That is, the server (20) may receive the echo data from the offshore unit and transmit the received echo data to the terminal.

The server (20) is provided with a control unit (201), a storage unit (202), and a communication unit (203). The control unit (201) is composed of a CPU or the like. The storage unit (202) is composed of a ROM, a RAM, a hard disk or the like. The storage unit (202) stores a program for fish species discrimination. The control unit (201) controls each section according to a program stored in the storage unit (202). The communication unit (203) communicates with the underwater detection device (10) via the external communication network (30) and the base station (40) under control from the control unit (201).

The control unit (201) generates a machine learning model applied to each underwater detection device (10) by the above program. The control unit (201) also stores the echo data, feedback information and customization information received from each underwater detection device (10) in the storage unit (202) in association with each underwater detection device (10). The control unit (201) uses the feedback information and customization information received from each underwater detection device (10) to update the machine learning model applied to the underwater detection device (10).

It should be noted that the echo data transmitted from each underwater detection device (10) to the server (20) may be decimated to a predetermined granularity from the reduction of communication traffic and the capacity load of the server (20). In this case, the server (20) performs fish species discrimination and machine learning using the echo data with the decimation corrected by interpolation processing. Alternatively, fish species discrimination and machine learning may be performed using the echo data in the decimated state. However, in order to perform more accurate fish species discrimination and machine learning, it is preferable to use echo data corrected for thinning by interpolation processing for fish species discrimination and machine learning.

In addition, in order to quantitatively handle the echo data received from each underwater detection device (10), the server (20) may perform fish species discrimination and machine learning by correcting the echo data received from each underwater detection device (10) based on underwater acoustic theory and taking into account the characteristics (For example, sensitivity, amplification, etc.) of the underwater detection device (10) and the transmitter/receiver (11). This enables fish species discrimination by the machine learning model and machine learning for the machine learning model to be performed with greater accuracy.

Fig. 3 is a diagram showing the management state of various kinds of information in the storage unit (202) of the server (20).

The storage unit (202) stores a standard data (301), a standard model (302), a user management information (303), individual data (311) and (321), and individual models (312) and (322).

The standard data (301) is standard teacher data for machine learning. The standard data (301) is a combination of echo data for the range (Depth, Time) of a school of fish and the species of fish in that school. The standard data (301) is sequentially generated by professional officers and registered by managers. This gradually increases the number of standard data.

The standard model (302) is a machine learning model generated by machine learning using standard data (301). In response to the update of the standard data (301), machine learning is performed on the standard model (302) and the standard model (302) is updated.

In this embodiment, machine learning using a neural network is applied as machine learning. For example, a neural network with deep learning combining neurons in multiple stages is applied. However, the machine learning applied is not limited to this, and other machine learning such as support vector machines and decision trees may be applied.

The user management information (303) is information for managing the user of the underwater detection device (10).

Fig. 4 (a) is a diagram showing the configuration of the user management information (303).

The user management information (303) is configured by associating user IDs, usernames, regions, fishing styles, equipment types, and application models with each other.

The user IDs are information for identifying users (underwater detection equipment (10). For example, the product code of the underwater detection equipment (10) may be used as the user ID, or a randomly assigned code may be used as the user ID. The username is the name (Name, etc.) of the user. Region is the region to which the user belongs. Region is, for example, a state or province. Region may be a local name, such as Kinki, or a municipality.

Fishing style is information to identify the fishing style in which the underwater detection device (10) is used. For example, fishing style is identified by the type of fishing style, such as roll net fishing or set net fishing. Equipment type is information indicating the type of equipment. Equipment type can be, for example, a model code. If the underwater detection device (10) is specialized for any fishing style, the type of the underwater detection device (10) may be used to identify the fishing style.

The applied model is information indicating whether the machine learning model used for fish species discrimination is a standard model or an individual model. The standard model is a standard machine learning model generated by machine learning using standard data, as described above. The individual model is a machine learning model generated for each user (for each underwater detection device (10)) by machine learning using individual data (feedback information), as described below.

Among the user management information in Fig. 4 (a), information other than the applicable model is registered in the server (20) by the service man when the underwater detection device (10) is set in the fish species determination system (1). At this time, the service man sets the user ID included in the registered user management information in the underwater detection device (10). When communicating with the server (20), the underwater detection device (10) performs communication by including its own user ID in the communication header.

Returning to Fig. 3, the individual data (311), (321) are data acquired from each user’s underwater detection device (10). The individual models (312), (322) are machine learning models applied to each user’s underwater detection device (10). In Fig. 3, individual data (311) and individual model (312) are for user U1, and individual data (321) and individual model (322) are for user U2. Similarly, individual data and individual models are managed for each user other than users U1 and U2.

Figs. 4 (b) to 4 (d) show the structure of individual data.

As shown in Figs. 4 (b) to 4 (d), various types of individual data are managed in association with user IDs. Fig. 4 (b) shows individual data related to echo data. The underwater detection device (10) sequentially transmits echo data obtained by 1 sequence of transmitted and received waves to the server (20) along with the date and time of acquisition. In the storage unit (202) of the server (20), the acquisition start date and time and the acquisition end date and time of the echo data, and a group of echo data acquired during that time and the acquisition date and time are stored corresponding to the user ID.

Furthermore, the result of fish species discrimination obtained by the machine learning model from a group of echo data from the start date and time to the end date and time is further matched to the echo data of each group. When multiple results of fish species discrimination are obtained, the multiple results of fish species discrimination are matched to the group of echo data. Each result of fish species discrimination consists of a range (Depth, Time) of fish schools and a discrimination result (fish species).

Fig. 4 (c) shows individual data on feedback information. Here, the feedback information acquired from the underwater detection device (10) corresponding to the relevant user ID is stored in time series. Fig. 4 (d) is the individual data on the customization information. Here, the customization information acquired from the underwater detection device (10) corresponding to the user ID is stored in time series.

Fig. 5 is a diagram schematically showing the fish species discrimination processing by the neural network.

The control unit (201) of the server (20) extracts the range (Depth, Time) of the fish school from the echo data of one screen to be processed. The range of the fish school is extracted as the range where the echo intensity is above a predetermined threshold and is connected to the echo intensity on the echo image. For the extraction method of the range of the fish school, the description in the applicant’s earlier application, International Publication No. 2019/003759, may be incorporated by reference.

The control unit (201) applies the echo data of the extracted range of the fish school to an input (401a) of a machine learning algorithm (neural network) (401) in Fig. 5. Items of fish species such as sardine, horse mackerel and mackerel are assigned to an output (401b) of the machine learning algorithm (401). When echo data of a range of fish schools are applied to the input (401a) of the machine learning algorithm (401), the probability (prediction probability) that the fish species of the fish school is the fish species of each item is output from each item of the output (401b) of the machine learning algorithm (401). In the example in Fig. 5, the prediction probability of 80% is output from the sardine item and the prediction probability of 8% is output from the horse mackerel item.

The predicted probability of each item is checked against an output condition (402). For the output condition (402), for example, a condition is applied in which the fish species of the item whose predicted probability is equal to or greater than a predetermined lower limit and which has the highest rank (maximum) is output as a fish species discrimination result (403). The lower limit is set in order to prevent the fish species with low probability from being output as the discriminated result. In the example of Fig. 5, a sardine with a prediction probability of 80% is output as the fish species discrimination result (403).

Machine learning for the machine learning algorithm (401) is performed by sequentially applying a series of teacher data to the inputs (401a) and outputs (401b) of the machine learning algorithm (401). That is, the echo data of the fish school included in one teacher data is input to the inputs (401a) of the machine learning algorithm (401), and the items corresponding to the fish species included in this teacher data are set to 100% in the outputs (401b) of the machine learning algorithm (401), and the other items are set to 0% to perform machine learning.

The standard model in Fig. 3 is generated by setting the standard data (Echo data for the range of fish school, fish species) sequentially to the inputs (401a) and outputs (401b) of the machine learning algorithm (401) to perform machine learning. The individual model in Fig. 3 is generated by setting the feedback information (Echo data for the range of fish school, fish species) included in the individual data sequentially to the inputs (401a) and outputs (401b) of the machine learning algorithm (401) to perform machine learning. If the number of feedback information available for machine learning is small, more standard data may be used to learn individual models.

In addition to the echo data of the fish school, other information that can be used for fish species discrimination, such as the location where the echo data is obtained and the water temperature, salinity concentration and flow rate at that location, may be input to the input (401a) of the machine learning algorithm (401). In this case, the underwater detection device (10) is provided with a configuration to acquire the other information further, and together with the echo data, the other information is further transmitted to the server (20). The control unit (201) of the server (20) causes the storage unit (202) to store these other received information as individual data in Fig. 3. In this case, the standard data may also further include the other information.

Fig. 6 (a) is a flowchart showing the application process of the machine learning model.

When a user is newly registered in the user management information (303) (see Fig. 3), the control unit (201) of the server (20) sets the standard model as the application model for the user (user ID) (S101). In this case, the application model of the user (user ID) in Fig. 4 (a) is set to the standard model. Thereafter, until the feedback information is received from the underwater detection device (10) of the user (S102: NO), the application model for the user (user ID) is maintained in the standard model (S101).

When the feedback information is received from the underwater detection device (10) of the user (S102: YES), the control unit (201) extracts the feedback information of another user with the same attributes as the user from the storage unit (202) (S103). In step S103, the control unit (201) extracts the feedback information associated with the other user (user ID) with at least the same fishing style in the individual data of Fig. 4 (a) from the storage unit (202) as the feedback information with the same attributes.

Using the extracted feedback information of the other user and the feedback information of the user received in step (S102) as the teacher data, the control unit (201) performs learning on the machine learning algorithm of Fig. 5 to generate an individual model for the user (S104). Here, if the number of feedback information extracted in step (S103) is not yet sufficient, the standard data may be further used as the teacher data when learning the machine learning algorithm.

Thus, when the individual model for the user is generated, the control unit (201) sets the generated individual model as the applied model for the user (user ID) (S105). As a result, the user’s application model in Fig. 4 (a) is changed from the standard model to the individual model. The generated individual model is associated with the user and stored in the storage unit (202) as shown in Fig. 3. For example, the parameter values and output condition (402) of the machine learning algorithm (401) in Fig. 5 may be stored as individual models.

It should be noted that the identity of the attributes in step (S103) is not necessarily limited to the fact that the fishing styles are identical, and other elements may be further included as long as feedback information (echo data) with different characteristics can be appropriately excluded. For example, if the fishing styles are the same but the region (fishing ground) to which the user belongs is different, the characteristics of the fish may differ and the characteristics of the echo data may differ. Therefore, other feedback information used for machine learning of individual models may be further limited in the region in Fig. 4 (a). This allows the generation of even more accurate individual models.

Other feedback information used for machine learning of individual models may also be further limited by the device type in Fig. 4 (a). This generates individual models using feedback information of similar characteristics obtained from underwater detection devices 10 of the same equipment type. Thus, individual models with higher accuracy can be generated.

Thus, when the individual model is set as the applied model (S105), the control unit (201) returns the processing to step (S102). Thus, every time feedback information is newly received from the user (underwater detection device (10)) (S102: YES), the control unit (201) updates the individual model of the user (S103 to S105) using the received feedback information and the feedback information newly received from another underwater detection device (10) with the same attributes as the teacher data.

It should be noted that after the accuracy of the individual model is enhanced (For example, the frequency of modification of fish species' discriminated results due to feedback information was less than a predetermined threshold) by the repeated updating of the individual model, step (S103) is omitted, and in step (S104), the individual model may be updated only from the feedback information received in step (S102). Thus, the individual model of the user can be updated to be more suitable for the fishing ground of the user.

In the flowchart of Fig. 6 (a), the individual model was generated and updated by receiving feedback information from the underwater detection device (10) of the user as a trigger, but the timing of generation and update of the individual model is not limited to this. For example, the individual model may be updated with feedback information received from the underwater detection device (10) of the user and feedback information received from the underwater detection device (10) of another user having the same attributes as the underwater detection device (10) of the user as teacher data every certain period.

Further, if a sufficient number of feedback information from other users with the same attributes as the user has already been aggregated in the server (20) at the time of the registration of the user in the server, the individual model may be generated using the feedback information from another user (user ID) with the same attributes as the teacher data without applying the standard model to the user (user ID), and the generated individual model may be applied to the user (user ID).

Fig. 6 (b) is a flowchart showing the fish species discrimination process.

When the control unit (201) of the server (20) starts receiving echo data from the underwater detection device (10) (S201: YES), the control unit (201) stores the received echo data in the storage unit (202) as individual data in Fig. 3 (S202). Further, the control unit (201) sequentially constructs an echo image from the received echo data and specifies the range (Depth, Time) of the fish school on the echo image. Then, the control unit (201) applies the echo data of the specified range of the fish school to the application model (standard model/individual model) of the underwater detection device (10) (user ID) to determine the fish species of the fish school (S203). The control unit (201) transmits the determination result of the fish species, together with the range (Depth, Time) of the fish school for which the determination result is obtained, to the underwater detection device (10), and further stores this information in the storage unit (202) (S204).

Thereafter, the control unit (201) repeatedly executes the processing of steps (S202 to S204) until the reception of echo data from the underwater detection device (10) is terminated (S205: NO). As a result, each time the range (Depth, Time) of the fish school is newly identified from the echo image, the species of the fish school is discriminated by the applied model. The result of discrimination of the newly obtained fish school and the range (Depth, Time) of the fish school are transmitted to the underwater detection device (10) and stored in the storage unit (202) as needed.

Thus, when the reception of the echo data from the underwater detection device (10) is finished (S205: YES), the control unit (201) terminates the processing of Fig. 6 (b). Thus, the individual data of one row of Fig. 4 (b) is stored in the storage unit (202). As described above, the echo data column of Fig. 4 (b) holds all the echo data received from the underwater detection device (10) in the processing of Fig. 6 (b). Also, the fish species determination results column of Fig. 4 (b) holds all the fish species determination results obtained by the processing of Fig. 6 (b) along with the range (Depth, Time) of the fish school.

Fig. 7 is a diagram schematically showing a display example of echo image P1 including the result of fish species discrimination. For convenience, in Fig. 7, a line in the depth direction is attached only to the part where the echo intensity is high.

When the control unit (201) of the underwater detection device (10) receives the determination result and the range (Depth, Time) of the fish school from the server (20), a frame-shaped marker M0 indicating the range of the fish school is displayed in the area on the echo image P1 corresponding to the depth width and time width corresponding to the range of the received fish school. Furthermore, the control unit (201) further displays a label L0 indicating the determination result of the received fish school around this marker M0. In the example of Fig. 7, based on the determination result received from the server (20) and the range (Depth, Time) of the fish school, a marker M0 is displayed for the fish schools F1 to F8, and further, a label L0 indicating the determination result of the fish species is displayed around these markers M0. The current date and time is displayed near the upper left corner of the echo image P1.

Here, in the example of Fig. 7, the marker and label are not displayed in the fish school F9 because the determination result of the fish species is not output by the applied model. This can happen, for example, if the determination result of the fish species of the fish school F9 by the applied model does not satisfy the output condition of Fig. 5. For example, if the output condition is to output a fish species with a prediction probability of the first rank (highest) that is greater than or equal to a specified lower limit value, and the prediction probability of the first rank among the prediction probabilities that occur in the output items of each fish species of the machine learning algorithm is less than this lower limit value, the discriminated result for this fish school is not output. In such a case, the discriminated result for this fish school is not transmitted from the server (20) to the underwater detection device (10), so that the discriminated result for the fish species is not displayed as in the case of fish school F9 in Fig. 7.

Fig. 8 (a) is a flow chart showing the feedback information transmitting processing performed by the control unit (101) of the underwater detection device (10). Fig. 8 (b) is a flow chart showing the feedback information receiving processing performed by the control unit (201) of the server (20).

The user performs an operation (feedback operation) to transmit feedback information to the server (20) for correcting the fish species of the given fish school to the underwater detection device (10) via the input unit (103) of Fig. 2, for example, when the fish species determination result of the given fish school displayed in the echo image P1 differs from the fish species of the fish school the user actually captured, or when the user actually captured the fish school for which the fish species determination result was not displayed in the echo image P1 and grasped the fish species of the fish school. In this case, the user inputs the date and time range of the echo image including the fish school to be corrected via the input unit (103), and further performs an operation to acquire the fish species determination result and echo data (Hereafter referred to as "historical information") of the range from the server (20).

Referring to Fig. 8 (a), when a feedback operation is input from the user via the input unit (103) (S301: YES), the control unit (101) of the underwater detection device (10) sends a request to transmit history information including the date and time range input by the user to the server (20) and acquires the history information from the server (20) (S302). Referring to Fig. 8 (b), when the control unit (201) of the server (20) receives a request to transmit the history information transmitted in step (S302) of Fig. 8 (a) (S401: YES), it extracts the history information (echo data and fish species determination result) in the range of dates and times included in the transmission request from the individual data of the underwater detection device (10) of the transmission source of the transmission request, and transmits the extracted history information to the underwater detection device (10) of the transmission source (S402).

Referring to Fig. 8 (a), when the control unit (101) of the underwater detection device (10) receives the history information from the server (20) (S302), it causes the display unit (102) to display an echo image based on the received history information and accepts the modification of the fish species from the user (S303).

Fig. 9 schematically shows a screen for accepting the modification of the fish species from the user in step (S303) of Fig. 8 (a).

The control unit (101) of the underwater detection device (10) causes the display unit (102) to display the echo image and the determination result of the leading time zone in the time range specified by the user when requesting the transmission of history information. The user operates the scroll bar B0 via the input unit103 to transition the echo image in the time direction and display a screen containing the echo image and the determination result of the desired time zone. Thus, the screen of the time zone in Fig. 9 is displayed.

In this screen, the user specifies the marker M0 of the fish school that the user intends to modify via an input unit (103). In the screen of Fig. 9, the marker M0 of the fish school F5 with the determination result of mackerel is specified by the user. With this, the marker M0 of the fish school F5 is highlighted, and the selection candidate C0 of the fish species is displayed around this marker M0 with a scroll bar. The user manipulates the scroll bar of the selection candidate C0 to display the desired fish species, and then selects the fish species that the user intends to change. In the example of Fig. 9, Sea bream is selected as the fish species of the fish school F5.

In addition, when the user inputs the fish species for the fish school to which the discriminated result is not attached on this screen, the user designates the range of the fish school via the input unit (103). As shown in Fig. 7, the fish school F9 does not have the discriminated result of the fish species. When the user inputs the fish species of this fish school F9, the user specifies the range of the fish school F9 via the input unit (103). As a result, as shown in Fig. 9, a new marker M1 is displayed in the range of the specified fish school F9, and the selection candidate C0 of the fish species is displayed around this marker M1 with a scroll bar. The user manipulates the scroll bar of the selection candidate C0 to display the desired fish species, and then selects the fish species the user intends to input. In the example of Fig. 9, Spanish mackerel is selected as the fish species of the fish school F9.

Thus, after performing an operation to change or set the fish species, the user inputs an operation to confirm these operations via the input unit (103).

Referring to Fig. 8 (a), when a confirmation operation is input from the user (S304: YES), the control unit (101) transmits feedback information including the range of the fish school designated by the user in step S303 and the fish species input by the user for the fish school to the server (20) (S305). With this, the control unit (101) terminates the processing of Fig. 8 (a).

Referring to Fig. 8 (b), when the control unit (201) of the server (20) receives feedback information from the control unit (101) of the underwater detection device (10) (S403: YES), it causes the storage unit (202) to store the received feedback information as individual data of the underwater detection device (10) (S404). As a result, the feedback information of one line in Fig. 4 (c) is stored in the storage unit (202). Thus, the control unit (201) ends the processing shown in Fig. 8 (b).

When the feedback information is received in step S403 of Fig. 8 (b), the determination in step (S102) of Fig. 6 (a) becomes YES. With this, the processing after step (S103) of Fig. 6 (a) is executed as described above.

Fig. 10 (a) is a flowchart showing the customized information transmitting processing executed by the control unit (101) of the underwater detection device (10). Fig. 10 (b) is a flowchart showing the customized information receiving processing executed by the control unit (201) of the server (20).

The user can input and transmit the customized information at any time. The customized information is for adjusting the fish species determination result to the user’s preference of the underwater detection device (10) and includes the modification of the output condition (402) of the individual model in Fig. 5.

Referring to Fig. 10 (a), the control unit (101) of the underwater detection device (10), when a customization operation is input from the user via the input unit (103) (S501: YES), causes the display unit (102) to display a prescribed input screen and accepts input of customization information from the user (S502). After input of customization information from the user, when a definite operation is input (S503: YES), the control unit (101) transmits the input customization information to the server (20). With this, the control unit (101) terminates the processing shown in Fig. 10 (a).

As described above, the customization information includes a change in the output condition of the individual model. Here, the change in the output condition may include preferentially outputting this specific fish species as a discriminated result if the prediction probability of the specific fish species is equal to or greater than a predetermined threshold (a value greater than zero) even if the prediction probability of the individual model for the specific fish species specified by the user is lower than that of other fish species. Alternatively, the change in the output condition may include changing the lower limit of the prediction probability for outputting the discriminated result for the specific fish species. Also, the change in the output condition may include not outputting the discriminated result for the specific fish species specified by the user.

Referring to Fig. 10 (b), when the control unit (201) of the server (20) receives customization information from the control unit (101) of the underwater detection device (10) (S601: YES), the received customization information is stored in the storage unit (202) as individual data of the underwater detection device (10) (S602). As a result, the feedback information of one line in Fig. 4 (d) is stored in the storage unit (202). Furthermore, the control unit (201) changes the output condition of the individual model of the underwater detection device (10) based on the received customized information (S603). With this, the control unit (201) terminates the processing shown in Fig. 10 (b).

Fig. 11 is a diagram schematically showing a display example of the fish species determination result when the output conditions of the individual model are changed based on the customized information.

Here, the output conditions of the determination result are changed so that even if the prediction probability of the individual model for the specific fish species specified by the user is lower than that of other fish species, if the prediction probability of the specific fish species is at or above a specified threshold (a value greater than 0), this specific fish species is preferentially output as the determination result. Here, the specific fish species preferentially output is designated as sea bream. Therefore, in the screen of Fig. 7, the fish species determination result of fish school F5 is mackerel, whereas in the screen of Fig. 11, the fish species determination result of fish school F5 is sea bream.

That is, in the fish species discrimination of the fish school F5 by the machine learning algorithm (401) in Fig. 5, the prediction probability of mackerel was the highest, but since the prediction probability of sea bream was above the threshold, sea bream was output as the fish species discrimination result (403) by the modified output condition (402). Here, the threshold may be specified by the user as customized information. With this, the user can preferentially output the discriminated result of the sea bream if the predicted probability of the sea bream is equal to or greater than the threshold value desired by the user and can confirm the possible fish school F5 of the sea bream by the screen in Fig. 11.

In this case, the fact that the predicted probability of the sea bream is lower (not the first rank) than that of any other fish species may be further displayed for this fish school F5. For example, as shown in Fig. 11, the symbol “?” may be included in the label L0 along with the indication of the sea bream to indicate that the predicted probability of the sea bream for the fish school F5 is not of the first rank. Alternatively, the symbol “?” may be replaced by a numerical ranking of the predicted probabilities for sea bream. This allows the user to appropriately determine whether or not to fish for the fish school F5.

In the display example in Fig. 11, the output condition is changed so that the lower limit of the prediction probability for outputting the discriminated result is lower for the specified fish species specified by the user than for other fish species. Here, the specified fish species whose lower limit is changed is specified as Spanish mackerel. Therefore, in the screen of Fig. 7, the fish species determination result for the fish school F9 was not displayed because the prediction probability of the first rank Spanish mackerel for the fish school F9 was lower than the lower limit value of the standard, whereas in the screen of Fig. 11, the fish species determination result for the fish school F9 is displayed as Spanish mackerel because the prediction probability of the first rank Spanish mackerel for the fish school F9 was higher than the lower limit value specified by the user. This enables the user to increase the frequency with which the determination result of the Spanish mackerel that the user assumes as the target of capture is displayed on the echo image P1, and thus, the fishing of Spanish mackerel can proceed efficiently.

Here, the lower limit value for outputting the determination result for the specific fish species is set lower than the lower limit value of the standard applicable to other fish species, but the lower limit value for outputting the determination result for the specific fish species may be set higher than the lower limit value of the standard. With this, the user can reduce the frequency with which the fish species determination result of the specific fish species not assumed to be captured is displayed on the echo image P1.

In the display example in Fig. 11, the output condition is changed so that the determination result of the specific fish species specified by the user is not output. Here, the specific fish species for which the determination result is not output is specified for sardines. Therefore, in the screen of Fig. 7, the marker M0 and the sardine label L0 are displayed for the fish schools F1 and F2, while in the screen of Fig. 11, these displays are omitted for the fish schools F1 and F2. Thus, the user can suppress the display of the identification result of the specific fish species (here, sardines) that is not assumed as the target of capture in the echo image P1, and can smoothly confirm the range of the fish species the user wants in the echo image P1.

Effect of Embodiment
According to the embodiment, the following effects can be achieved.

As shown in Fig. 1 to Fig. 6 (a), the server (20) acquires feedback information associating the echo data with the fish species from each underwater detection device (10), stores the acquired feedback information by associating it with attributes including at least the fishing style used by the underwater detection device (10), and generates an individual model for fish species discrimination for each attribute by machine learning using multiple feedback information with the same attributes.

According to this configuration, the server (20) acquires and aggregates feedback information from the multiple underwater detection devices (100, thereby increasing the number of feedback information available for machine learning. Also, among the aggregated feedback information, feedback information with the same attributes including fishing style (see Fig. 4 (a)) is used for machine learning, so echo data with different characteristics can be suppressed from being used for machine learning of individual models. Therefore, it is possible to improve the accuracy of the fish species discrimination results by the individual model.

Here, the attributes may further include the region (see Fig. 4 (a)) to which the user using the underwater detection device (10) belongs. If the fishing styles are the same but the regions are different, the fish characteristics may differ and the characteristics of the echo data may differ. Therefore, by further restricting the feedback information used to learn the individual models in the regions, the results of fish species discrimination by the individual models can be further refined.

As shown in Fig. 3 and Fig. 6 (a), the server (20) generates the standard model (302) for fish species discrimination by machine learning using the standard data (301) (see Fig. 3) prior to the generation of the individual models (312) and (322), applies the standard model to the user (underwater detection device (10)), acquires information indicating the user’s correction to the results of fish species discrimination by the standard model (302) from the underwater detection device (10) as feedback information (Fig. 6 (a): step S102), and generates the individual models (312) and (322) (steps S103 to S105). This enables the user’s correction to the 302 results of fish species discrimination to be smoothly obtained from the underwater detection device (10) as feedback information. Therefore, the individual models (312) and (322) can be generated appropriately and efficiently while maintaining the user’s convenience.

As shown in Fig. 6 (a), the server (20) acquires as feedback information (S102) information indicating the user’s modification to the fish species determination result by the individual models (312) and (322) (see Fig. 3), and updates the individual models (312) and (322). With this, the individual models (312) and (322) can be updated to gradually adapt to the user’s attributes. Therefore, the user’s convenience can be enhanced.

As shown in Fig. 10 (b), the server (20) receives customization information (S 601), including a change in the output condition (402) (see Fig. 5) of the individual models (312), (322) (see Fig. 3) entered by the user of each underwater detection device (10), and changes the output condition (402) of the individual models (312), (322) for the underwater detection device (10) based on the received customization information. Thus, the individual models (312), (322) can be customized so as to be easy for the user to use.

In this case, the modification of the output condition (402) may include preferentially outputting the specific fish species as the discriminated result even if the prediction probability of the individual models (312), (322) for the specific fish species specified by the user is lower than that of other fish species, provided that the prediction probability of the specific fish species is equal to or greater than a predetermined threshold. Thus, as shown in Fig. 11, even if the prediction probability of the specific fish species (Here, sea bream) is lower than that of other fish species, the discriminated result of the specific fish species (label L0 of fish school F5) is outputted, and the output frequency of the discriminated result of the specific fish species can be increased. Therefore, the user is less likely to miss the catch of the specific fish species that the user desires, and the catch of the specific fish species can be increased.

Alternatively, changing the output condition (402) may involve changing the lower limit of the predicted probability for outputting the discriminant result. Thus, for example, by lowering the lower limit value for a specific fish species, the user can output the discriminant result (label L0 of fish school F9) for the specific fish species (Here, Spanish mackerel) even if the predicted probability of the specific fish species is low, as shown in Fig. 11, and can catch the fish he wants to catch more reliably. Alternatively, by raising the lower limit value for the specific fish species, the user can reduce the frequency with which the discriminant result for the specific fish species is output, and can more efficiently confirm the fish that the user wants to catch.

Alternatively, a change in the output condition (402) may include not outputting the discriminant result for the specific fish species specified by the user. With this, as shown in Fig. 11, the user can suppress outputting the discriminant result (Here, the discriminant results for the fish schools F1 and F2) for the fish species he is not interested in (Here, sardines), and can smoothly and efficiently grasp the fish school of the fish species he is interested in.

As shown in Fig. 2, the underwater detection device (10) includes a display unit (102), an input unit (103), and a control unit (101). Here, the control unit (101) makes the display unit (102) display the result of discrimination of the fish species by the individual model by the processing shown in Fig. 8 (a), accepts the correction of the discriminated result via the input unit (103) (S301 to S303), and transmits the correction as feedback information to the server (20) (S305). Thus, the user can provide the server (20) side with the correction of the fish species determination result by the individual model at any time. Thus, the individual model can be updated to gradually adapt to the attributes of the user, thereby enhancing the convenience of the user.

Here, by the processing of Fig. 10 (a), the control unit (101) further accepts the input of customization information including the change of the output condition (402) of the individual model (see Fig. 5) via the input unit (103) (S501, S502), and transmits the input customization information to the server (20) (S504), and by the processing of Fig. 10 (b), the server (20) changes the output condition (402) of the individual model to the underwater detection device (10) based on the received customization information (S601: YES). Thus, the individual model can be customized to make it easy for the user to use.

Example of changes
The present disclosure is not limited to the above embodiment, and the embodiment of the present disclosure can be changed in various ways other than the above configuration.

For example, in the above embodiment, two types of ultrasonic waves are transmitted and received in one sequence, but the types of frequencies transmitted and received in one sequence are not limited to two. For example, only one frequency of ultrasound may be transmitted and received per sequence, or only three or more frequencies of ultrasound may be transmitted and received per sequence.

In the above embodiment, the determination of fish species using the standard or individual model was performed on the server (20) side, but this determination may be performed on the underwater detection device (10) side. In this case, the server (20) sends the standard model or the individual model generated for each underwater detection device (10) to each underwater detection device (10), and each underwater detection device (10) performs fish species determination using the standard or individual model received from the server (20). Again, the server (20) aggregates the individual data (including feedback and customization information) from each underwater detection device (10) as in the above embodiment, and updates and customizes the individual model based on the aggregated individual data by the same processing as in the above embodiment. The server (20) transmits the updated and customized individual model to each underwater detection device (10) at any time, and each underwater detection device (10) uses the updated and customized individual model to perform fish species discrimination processing.

In addition, in the above embodiment, fishing style, region, and device type (see Fig. 4 (a)) are shown as attributes for limiting feedback information used for machine learning, but other parameters may be included as attributes.

For example, sea areas where echo data are obtained may be included as attributes for limiting feedback information used for machine learning. In this case, the server (20) receives from the underwater detection device (10) the position information detected by the position detection unit (107) of the underwater detection device (10) together with the echo data, and further restricts the feedback information (echo data) used for machine learning of the individual model to the feedback information including the position information in the sea area specified by the user. With this, the individual model can be updated to one adapted to the sea area (the fishing area of the user) specified by the user.

In addition, in the above embodiment, feedback information from the user is input by the screen shown in Fig. 9, but the method of input of feedback information is not limited to this. Also, the feedback information does not necessarily need to correct the result of discrimination by the standard model or the individual model, and the user may specify an arbitrary area (area of a fish school) on the echo image and input the fish species in that area.

Moreover, the customization information need not be information related to the change of the output condition (402) shown above, or it may be information related to the change of the output condition (402) other than the above. For example, the customization information may simply be information for changing the lower limit value of the output condition, regardless of the species of fish.

In addition, the customization of the individual model by the customization information may include the modification of the individual model other than the modification of the output condition (402), for example, the user may specify feedback information to be used for machine learning of the individual model. The user may also specify by customized information that the standard data along with feedback information of the same attributes is to be used for machine learning of his or her individual model, and in this case, the ratio of the feedback information and the standard data to be used for machine learning and the number of the feedback information and the standard data to be used for machine learning.

In the above embodiment, the individual model of the user is customized by the customization information from the user, but the standard model may be customized by the customization information from the user at the stage where the user is using the standard model.

In the above embodiment, the underwater detection device (10) is a fish finder, but the underwater detection device (10) may be a device other than a fish finder such as sonar.

In addition, the embodiment of the present disclosure may be modified as appropriate to the extent stated in the claims.

List of Reference Numerals
1 fish species discrimination system
10 underwater detection device
20 server
101 control unit
102 display unit
103 input unit
301 standard data
302 standard model
312, 322 individual model
402 output conditions
403 discrimination results

Claims

A fish species discrimination method, comprising:
acquiring feedback information associating echo data with fish species;
storing the acquired feedback information in association with attributes including at least the fishing styles are used;
generating an individual model (312) for fish species discrimination for each of the attributes by machine learning using the multiple feedback information for which the attributes are identical; and
performing fish species discrimination by echo signal, being transmitted into water body and reflected by an object in the water body, corresponding to the attributes by using the generated individual model (312).
The fish species discrimination method of claim 1, wherein:
the attributes further include a sea area where the echo signal is transmitted.
The fish species discrimination method of claim 1, further comprising:
generating a standard model (302) for fish species discrimination prior to the generation of the individual model by machine learning using standard data (301);
acquiring correction information indicating the user's correction to the result of fish species discrimination by the standard model (302) from the underwater detection devices (10) as the feedback information; and
generating the individual model (312) based on the correction information.
The fish species discrimination method of claim 1, further comprising:
updating the individual model (312) based on the correction information.
The fish species discrimination method of claim 1, further comprising:
receiving customized information including change of the output condition of the individual model (312) entered by the user of each of the underwater detection devices (10); and
changing the output condition of the individual model (312) for the underwater detection devices (10) based on the received customized information.
The fish species discrimination method of claim 5, wherein:
the change of the output condition includes preferentially outputting the specific fish species as the discriminated result when the prediction probability of the individual model (312) for the specific fish species specified by the user is lower than that of other fish species and when the prediction probability of the specific fish species is not less than a predetermined threshold.
The fish species discrimination method of claim 5, wherein:
the change of the output condition includes changing the lower limit of the predicted probability for outputting the discriminated result for the specified fish species.
The fish species discrimination method of claim 5, wherein:
the change of the output condition includes not outputting the discriminated result of the specific fish species.
A server comprising functions which, when executed by a computer, cause the computer:
to acquire feedback information associating echo data with fish species from multiple underwater detection devices (10);
to store the acquired feedback information in a storage unit (202) by associating the acquired feedback information with attributes including at least the fishing styles for which the underwater detection devices (10) are used; and
to generate an individual model (312) for fish species discrimination for each of the attributes by machine learning using the multiple feedback information for which the attributes are identical.
A server comprising:
a communication interface configured to acquire feedback information associating echo data with fish species from multiple underwater detection devices (10);
a storage configured to store the acquired feedback information in a storage unit (202) by associating the acquired feedback information with attributes including at least the fishing styles for which the underwater detection devices (10) are used; and
processing circuitry configured to generate an individual model (312) for fish species discrimination for each of the attributes by machine learning using the multiple feedback information for which the attributes are identical.
A program comprising functions which, when executed by a computer, cause the computer:
to acquire feedback information associating echo data with fish species from multiple underwater detection devices (10);
to store the acquired feedback information in a storage unit (202) by associating the acquired feedback information with attributes including at least the fishing styles for which the underwater detection devices (10) are used; and
to generate an individual model (312) for fish species discrimination for each of the attributes by machine learning using the multiple feedback information for which the attributes are identical.
A fish species discrimination system (1), comprising:
the server (20) according to any one of claims 1 to 8; and
the underwater detection device (10).
The fish species discrimination method of any one of claims 1 to 8, wherein:
the fishing style includes roll net fishing and set net fishing.
The fish species discrimination method of any one of claims 1 to 8, wherein:
the individual model (312) is configured to be updated to one adapted to the sea area where the echo signal is transmitted.