JP7243986B2 - Seafood Identification Method, Seafood Identification Program, and Seafood Identification System - Google Patents

Description

TECHNICAL FIELD The present invention relates to a marine product identification method, a marine product identification program, and a marine product identification system for identifying marine products moving along a migration route of marine products.

Conventionally, in this type of invention, for example, as described in Patent Document 1, a camera is installed at a position to take an image from the side in the area where the fish running up the fishway jumps over the weir, and an image captured by the camera. means for extracting a detection target area through which a fish passes from an image, preprocessing an image of the extracted detection target area, binarizing it with a predetermined threshold value, and extracting a size of the extracted area within a predetermined range. There is a fish quantity measuring device comprising means for detecting a detected object having a thickness as a fish shadow, and means for counting the detected fish shadow and outputting a count value.
In this prior art, sweetfish candidates are detected based on the characteristics of ayu jumping and shining silvery white, and the characteristics of the splash when the jumping ayu hits the water, and the detected object having a specific size is identified as ayu. , to count.

Japanese Patent No. 3783835

However, in the above-described prior art, since the fish is recognized and counted based on the image characteristics of the moment when the fish jumps, misrecognition and/or , may cause an erroneous count.
In particular, when the above-described prior art is applied to, for example, the counting of cultured fish transported in a gutter or the counting of cultured fish moving in a fish tank, it is difficult to obtain the above-described image characteristics, which further results in erroneous recognition. Also, there is a risk that the number of erroneous counts will increase.

In order to solve at least part of the above problems, the present invention has the following configurations.
A step of capturing a video of a predetermined imaging range in the moving route of the fishery product, and analyzing the video data obtained by the video recording to determine whether or not the individual image that has entered the imaging range has specific visual features. attaching an identification label to the individual image if it is determined in the step that the individual image has a specific visual characteristic; and tracking the individual image with the identification label so that the individual image is the determining that the individual has exited from the imaging range; and after the step of determining that the individual has exited from the imaging range, linking the visual features to the individual image that has exited from the imaging range and storing the visual features. Seafood identification method.

Since the present invention is configured as described above, it is possible to reduce erroneous recognition of marine products.

BRIEF DESCRIPTION OF THE DRAWINGS It is a side view which shows typically an example of the marine product identification system which concerns on this invention. It is a top view which shows the principal part of the same marine product identification system typically. It is a flow chart which shows an example of a seafood identification method concerning the present invention. Fig. 10 is a cross-sectional view of a main part schematically showing another example of the marine product identification system according to the present invention; Fig. 10 is a side view schematically showing another example of the marine product identification system according to the present invention;

The present embodiment discloses the following features.
A first feature is a method for identifying marine products, comprising the steps of taking a video of a predetermined imaging range in the moving route of the marine products, analyzing the video data acquired by the video shooting, and analyzing the individual that has entered the imaging range. determining whether the image has a specific visual characteristic; attaching an identification label to the individual image if it is determined that the image has the specific visual characteristic; tracking the individual image to determine that the individual image has exited the imaging range; and determining the visual characteristics of the individual image that has exited the imaging range (FIGS. 1 to 1). See Figure 5).
Here, the "marine products" include fish and shellfish such as shellfish, shrimp, crab, octopus, squid, etc., and seaweed such as wakame seaweed and kelp.
The "visual features" include shapes, sizes, scars, diseases that can be grasped from the appearance, and the like.

A second feature is that, prior to the step of attaching the identification label, a virtual circumscribing quadrilateral is set in contact with the contour of the individual image determined to have the specific visual feature, and the virtual circumscribing quadrilateral is The step of obtaining a center point, and the step of determining that the imaging range is out of the imaging range includes determining that the individual image with the identification label is out of the imaging range when the center point is out of the imaging range. (See Figures 2 and 3).

A third feature counts the individual images after the step of determining that the individual images have exited the imaging range (see FIG. 3).

As a fourth feature, the step of determining whether the individual image that has entered the imaging range has specific visual features uses an estimation model generated by deep learning.

The fifth feature is to recognize the type of marine product as the visual feature.

The sixth feature is to recognize wounds or diseases of the marine product as the visual feature.

The seventh feature is to recognize the size of the marine product as the visual feature.

The eighth feature constitutes a marine product identification program for causing a computer to execute the marine product identification method.

A ninth feature is a marine product identification system that executes the marine product identification method described above, comprising a video camera that captures a moving image of the imaging range and a computer that performs the steps (FIGS. 1 and 4). and FIG. 5).

A tenth feature is that a light-shielding case is provided to cover the imaging range and block external light rays, and the video camera and a light for illuminating the imaging range are provided in the light-shielding case (see FIG. 4).

As an eleventh feature, the movement path is formed by arranging a plurality of gutter-shaped members arranged in parallel in a direction crossing the flow direction so as to allow marine products to flow obliquely downward, and the imaging range is a plurality of gutter-shaped members. (see FIGS. 1 and 2).

A twelfth feature is that the movement route is set in the water inside the fish tank (see FIG. 5).

<First embodiment>
Next, specific embodiments having the above characteristics will be described in detail with reference to the drawings.
1 to 3 are diagrams illustrating examples of a marine product identification method, a marine product identification program, and a marine product identification system according to the present invention.

The marine product identification system A includes a plurality of gutter-shaped members 1 configured to allow individual marine products F to flow obliquely downward, and a predetermined imaging range 2 straddling the plurality of gutter-shaped members 1 at a predetermined frame rate. A video camera 3 for capturing moving images and a computer 4 for counting marine products F passing through an imaging range 2 based on moving image data acquired by the video camera 3 are provided.

According to an example shown in FIG. 1, this fishery product identification system A is used in the migration route of fishery product F in a facility that injects fishery vaccine into fishery product F and returns this fishery product F to the water.
The marine product F is fish according to the illustrated example.
In the figure, reference numeral 11 denotes a fish preserve that covers the marine products F in the water. Reference numeral 12 is a boat that floats on the water inside the fish tank 11 .

The gutter-shaped member 1 constitutes a movement path of the fishery product F, and is arranged in parallel in a direction perpendicular to the flow direction, which is the movement direction of the fishery product F (see FIG. 2).
Each gutter-shaped member 1 has a concave cross-section with an open top and is continuous in the flow direction in an elongated shape. The downstream end of each trough-like member 1 is positioned above the water surface in the cage 11, and the upstream end is positioned and fixed on the ship 12. As shown in FIG.
The width dimension of each gutter-like member 1 is set wider than the body length of the fishery product F so that the fishery product F flows without clogging and stagnation.

According to the plurality of gutter-shaped members 1, it is possible to prevent a plurality of marine products F from contacting or adjoining each other and flowing together, thereby improving recognition accuracy of each marine product F.
That is, if the gutter-like member is a single member having a relatively wide width, a plurality of fish will approach one side wall of this wide gutter-like member, and these fish will be recognized as a single fish. However, by using a plurality of gutter-shaped members 1 as described above, such erroneous recognition can be reduced.

An imaging range 2 is a range in which moving images are captured by a video camera 3, which will be described later.
This imaging range 2 has an appropriate length so that processing by the marine product identification system A described later can be performed smoothly, and is formed to a dimension that includes the entire length in the width direction so as to straddle a plurality of gutter-shaped members 1. be.

The video camera 3 is a digital video camera capable of shooting moving images at a predetermined frame rate (eg, 30 fps).
The video camera 3 is fixed to an immovable portion such as the ship 12 or the gutter-shaped member 1 with the optical axis directed toward the approximate center of the imaging range 2 . According to the example shown in FIG. 1 , the video camera 3 is positioned rearward and obliquely above the imaging range 2 .
Moving image data captured by the video camera 3 is transmitted to the computer 4 on the ship 12 by wire or wirelessly.

The computer 4 causes the CPU to function according to a program stored in advance, controls devices such as the video camera 3, performs image processing on moving image data obtained from the video camera 3, and outputs the processing results. As the computer 4, a general-purpose personal computer connected to the Internet, a tablet computer, a portable terminal, or the like can be used. In addition, a display device, an input/output device, other electronic devices, etc. are provided as necessary.

Next, an example of processing by the computer 4 will be described along the flowchart shown in FIG.
First, the computer 4 controls the video camera 3 to start capturing a moving image of a predetermined imaging range 2 in the moving route of the marine product F (step S1).

Next, the computer 4 analyzes the moving image data obtained by the moving image capturing, determines whether or not the individual image that has entered the imaging range 2 has specific visual characteristics (step S2), If it has visual characteristics, the process proceeds to the next step S3, and if not, the process proceeds to step S13.

More specifically, step S2 will be described in detail. Each time the computer 4 sequentially inputs frames of moving image data from the video camera 3, it executes an algorithm using an estimation model generated by deep learning in artificial intelligence. Then, it is determined whether or not the individual images present in the frame image have predetermined specific visual characteristics.
According to the illustrated example, the "predetermined specific visual characteristic" is a visual characteristic of a fish.

More specifically, the computer 4 performs learning processing on a large number of sample images of fish by deep learning in advance to generate an estimation model that quantifies the possibility that the input image is a fish.
The computer inputs the individual image obtained from the frame of the moving image data into the estimation model, and quantifies the possibility that the individual image is a fish image (hereinafter referred to as "fish-likeness"). Then, if this "fish-likeness" is equal to or greater than a predetermined threshold, the individual image is regarded as a fish image.

Note that the learning process by deep learning is performed when a single fish is used as sample data, or when a plurality of fish that are in contact with or close to each other are used as sample data.
In the process of step S2, when a plurality of individual images are in contact or close to each other, each individual image constituting a plurality of individual images is not determined to be an image of a single fish. is an image of a single fish.

In the next step S3, for the individual image (marine product F) determined to have the specific visual characteristics, a virtual circumscribing quadrilateral Fs is set to contact the contour thereof, and the center point Fp of this virtual circumscribing quadrilateral is determined.

In the next step S4, a unique identification label is attached to the marine product F for which the center point Fp has been set.
In addition, for example, when a plurality of fishery products F are in contact or close to each other, a central point Fp and an identification label are set for each of the individual images of the plurality of fishery products F.

Then, in the next step S5, the computer 4 substantially continuously tracks (tracks) the central point Fp of the labeled individual image (marine product F) at a predetermined frame rate. In other words, the coordinates of each central point Fp are monitored substantially continuously.

Also, in step S13, it is determined whether or not the central point Fp of the individual image (marine product F) to which the identification label is attached has exited the imaging range 2 (step S13). If so, it is assumed that the individual image (marine product F) having that center point Fp has left the imaging range 2, and the process proceeds to the next step 14. If not, the process returns to step S2.
Here, the fact that the center point Fp exits the imaging range 2 means that the center point Fp crosses the most downstream line L of the imaging range 2 to the downstream side.

In the next step S14, the visual feature determined in step S2 (the feature of being a fish according to the illustrated example) is determined for the individual image that has escaped from the imaging range 2. FIG.
Specifically, the computer 4 associates the individual image that has escaped from the imaging range 2 with the visual feature determined in step S2 (the feature of being a fish in the illustrated example), and stores the image. . An individual image to which visual features are linked in this way can be used as a sample image for the deep learning.

In the next step S15, the individual images whose visual features have been determined in step S14 are counted.
More specifically, the computer 4 stores in advance a variable indicating the number of individual images, and adds 1 to the variable each time a visual feature is determined in step S14. The initial value of the variable may be zero.

Note that, for example, when a plurality of fishery products F are in contact with or close to each other, all the center points Fp of the individual images corresponding to these plurality of fishery products F are monitored, and any one of these plurality of center points Fp is monitored. Each time the imaging range 2 is exited, the processing of steps S14 and S15 is performed.

Therefore, according to the marine product identification system A configured as described above, as described above, after an individual image determined to have a specific visual characteristic passes through the imaging range 2, the visual characteristic is determined, and the visual characteristic is determined. Since the individual images having the characteristic features are counted, erroneous recognition, erroneous counting, etc. can be reduced.
For example, even when the posture of the marine product F changes, or when a plurality of marine products F overlap, the marine product F can be identified and counted with high accuracy.

According to the illustrated example, it is determined whether or not the individual image that has entered the imaging range 2 has the visual characteristics of a fish. For example, it is possible to determine whether or not it has the visual characteristics of shellfish, shrimp, crab, octopus, squid, wakame seaweed, kelp, etc.).

As another example, it is also possible to recognize the type of marine product (tuna, yellowtail, tiger prawn, spiny lobster, turban shell, clam, etc.) for the individual image that has entered the imaging range 2 .

Furthermore, as another example, it is determined whether or not the individual image that has entered the imaging range 2 has a size within a specific range, or whether there is a wound or disease based on visual characteristics. It is also possible to set it as a mode.

Further, according to the above-described embodiment, all the individual images that have escaped from the imaging range 2 are counted without being classified. It is also possible to classify into a plurality of groups and count the individual images for each group.

<Second embodiment>
Next, other embodiments according to the present invention will be described. In addition, since the embodiment shown below adds a part of configuration to the above-described embodiment, the additional part will be mainly described in detail, and redundant detailed description will be omitted.

The fishery product identification system B shown in FIG. , and a light 6 for illuminating the imaging range 2 .

As shown in FIG. 4, the light-shielding case 5 is formed with a concave cross-sectional shape with an open bottom by side walls on both sides in the width direction and a ceiling wall extending between the upper ends of these side walls. , covering from its upper side.
In the longitudinal direction of the gutter-shaped member 1 , the light-shielding case 5 may be partially provided only at a position corresponding to the imaging range 2 , or may be provided in a long shape including the imaging range 2 . It is possible to adopt a mode in which it is provided over the entire length.
The light-shielding case 5 is made of metal, hard synthetic resin, or the like, and fixed to an immovable portion such as the gutter-shaped member 1 or the ship 12 .

One of the plurality of video cameras 3, 3' has its optical axis directed toward the approximate center of the imaging range 2, similar to that of the marine product identification system A, above the fish movement path of the marine product F. It is arranged and fixed to the inner surface of the light shielding case 5 . The other video camera 3 ′ is arranged in a slanting manner on the side of the imaging range 2 with its optical axis directed substantially toward the center of the imaging range 2 and fixed to the inner surface of the light shielding case 5 .

The light 6 is composed of, for example, an LED or the like, and is fixed to the inner surface of the light shielding case 5 so as to illuminate the imaging range 2 .

According to the marine product identification system B configured as described above, it is possible to prevent the adverse effects of external light rays such as sunlight. Moreover, by using the video cameras 3, 3' in multiple directions, occurrence of overlapping of the fishery products F can be grasped, and more accurate discrimination and counting can be performed.

According to the illustrated example, a single light 6 is provided, but as another example, a plurality of lights 6 may be provided, and the marine product F may be irradiated from a plurality of directions by the plurality of lights 6. According to this configuration, , the accuracy of discriminating and counting the marine product F can be further improved.

<Third Embodiment>
The fishery product identification system C shown in FIG. 5 eliminates all the gutter-shaped member 1 from the fishery product identification system A, sets the movement path of the fishery product F in the water in the fish tank 11, and part of this movement path has a predetermined The imaging range 2' is set.

In this fishery product identification system C, the video camera 3 is positioned so as to photograph the fishery product F in the imaging range 2' from above the water surface (according to the illustrated example, obliquely upward with respect to the direction of movement of the fish). is fixed via a support member or the like (not shown).

According to the fishery product identification system C configured as described above, it is possible to count the fishery products F swimming in the water. By setting the range 2', the cultured fish can be counted with high accuracy.
In addition, if necessary, if an underwater video camera that takes a picture of the marine products F in the imaging range 2' from the side is provided, the positional relationship of a plurality of farmed fish that overlap in the vertical direction can be grasped, and the marine products F can be distinguished and counted. Accuracy can be further improved.

According to the above embodiment, the imaging range 2 (or 2′) is set in the moving route of the fish for vaccination or in the moving route of the fish in the cage. It is also possible to set an imaging range in the transportation route for processing, etc., in the migration route of fish, in the upstream route of fish in a river, etc., and provide a video camera 3 so as to photograph this imaging range. It is possible.

Furthermore, as other examples of the marine product identification system according to the present invention, there is an aspect configured in a marine product transportation route on a ship or on land, a mode configured in a marine product transportation route in a market, an underwater camera etc. It is possible to adopt a mode or the like configured as follows.

Moreover, the present invention is not limited to the embodiments described above, and can be modified as appropriate without changing the gist of the present invention.

1: gutter-shaped member 2, 2': imaging range 3, 3': video camera 4: computer 5: light shielding case 6: light 11: fish tank A, B, C: fishery product identification system F: fishery product (individual)

Claims (12)

a step of taking a video of a predetermined imaging range in the moving route of the marine product;
a step of analyzing the moving image data obtained by the moving image capturing, and determining whether or not the individual image that has entered the imaging range has specific visual characteristics;
a step of attaching an identification label to the individual image when it is determined in the step that it has a specific visual characteristic;
a step of tracking the individual image with the identification label and determining that the individual image has exited the imaging range;
and after the step of determining that the individual has escaped, the step of linking and storing the visual features to the individual image that has escaped from the imaging range. setting a virtual circumscribing quadrilateral in contact with the outline of the individual image determined to have the specific visual feature, and determining the center point of the virtual circumscribing quadrilateral, prior to the step of attaching the identification label; have
In the step of determining that the imaging range has been exited, the individual image with the identification label is deemed to have exited the imaging range when the center point has exited the imaging range. The method for identifying marine products according to claim 1. 3. The marine product identification method according to claim 1 or 2, wherein the individual images are counted after the step of determining that the individual images have exited the imaging range. 4. An estimation model generated by deep learning is used in the step of determining whether or not the individual image that has entered the imaging range has specific visual features. Seafood identification method as described. 5. The marine product identification method according to any one of claims 1 to 4, characterized in that the type of the marine product is recognized as the visual feature. 6. The marine product identification method according to any one of claims 1 to 5, wherein wounds or diseases of the marine product are recognized as the visual features. The marine product identification method according to any one of claims 1 to 6, wherein the size of the marine product is recognized as the visual feature. A marine product identification program for causing a computer to execute the marine product identification method according to any one of claims 1 to 7. A marine product identification system for executing the marine product identification method according to any one of claims 1 to 7, comprising: a video camera for capturing moving images of the imaging range; and a computer for performing each of the steps. Seafood identification system characterized. 10. A marine product identification according to claim 9, further comprising a light-shielding case covering said imaging range and shielding external light rays, and said video camera and a light for illuminating said imaging range are provided in said light-shielding case. system. The moving path is formed by arranging a plurality of gutter-shaped members in a direction crossing the flow direction of the gutter-shaped members so as to make the marine products flow obliquely downward, and the imaging range is set across the plurality of gutter-shaped members. The marine product identification system according to claim 9 or 10, characterized in that: 11. The fishery product identification system according to claim 9 or 10 , wherein the moving route is set in the water inside the fish tank.

Images