JP2017133901A - Monitoring device and monitoring method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To monitor the occurrence of an offshore current using a captured marine image.SOLUTION: An image at a prescribed angle of view is captured from a prescribed altitude by a camera provided in a drone. This image is separated into a luminance signal and a color difference signal, the image of the luminance signal is divided into small areas, and a luminance range is obtained for each of the small areas from a displacement such as the minimum value, and average value, or the maximum value of the luminance signal at a prescribed time. The image of the color difference signal is divided into small areas, and a difference of the average value of colors of the entire image with an average color is obtained for each of the small areas. Then, it is assumed that there are no waves due to that the luminance range is lower than a prescribed value, and that there is occurrence of an offshore current in a small area in which a difference with an overall average color is large and a sea surface color is different from others. The present disclosure can be applied to a drone.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to a monitoring device, a monitoring method, and a program, and more particularly, to a monitoring device, a monitoring method, and a program that can detect the occurrence of a rip current based on an image obtained by imaging the sea.

  2. Description of the Related Art Conventionally, there has been proposed a technique for detecting an object on the sea, a water current, a wave direction, a flow velocity, and the like by imaging the sea with a camera and analyzing the captured image (see Patent Documents 1 to 4).

Japanese Patent Laid-Open No. 02-248863 JP2015-127669A JP 2008-232860 A JP 2009-074968 A

  However, in Patent Technologies 1 to 4, it is possible to monitor a swimmer to some extent based on an image of the sea, but in order to detect the occurrence of a danger such as a rip current, It was necessary to keep monitoring visually.

  The present disclosure has been made in view of such a situation, and in particular, enables generation of a rip current to be monitored using a marine image captured by a camera or the like.

  A monitoring device according to one aspect of the present disclosure includes an imaging unit that captures an image of a sea surface, and a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and the color of the sea surface in the image. A monitoring device including a specifying unit that specifies a region in which the current area is located.

  A separation unit that separates the image of the sea surface into a luminance signal and a color difference signal; a wave detection unit that detects the presence or absence of waves on the sea surface based on the luminance signal; and a color of the sea surface based on the color difference signal A sea surface color detection unit that detects sea surface wave detection results obtained from the wave detection result of the wave detection unit and the sea surface color detection result of the sea surface color detection unit. Based on the obtained sea surface color, it is possible to identify a region where a rip current is generated on the sea surface.

  The wave detection unit is configured to divide the image composed of the luminance signal into small regions, measure a luminance range in the displacement of the luminance value at a predetermined time for each small region, and the specific unit has the predetermined luminance range. A range of a small area smaller than the value can be specified as an area where the rip current may occur.

  The displacement of the luminance value at a predetermined time for each small area may include any of the maximum value, the average value, and the minimum value of the luminance value at the predetermined time for each small area.

  The sea surface color detection unit divides the image composed of the color difference signals into small regions, measures an average value of colors for a predetermined time for each of the small regions, and causes the specifying unit to perform a predetermined time for each of the small regions. A range of a small area in which the difference between the average value of the color of the image and the average value of the entire image is larger than a predetermined threshold is specified as an area where the rip current may occur.

  The wave detection unit is configured to divide the image composed of the luminance signal into small regions, measure a luminance range in a luminance value displacement at a predetermined time for each small region, and the sea surface color detection unit includes the color difference signal. And dividing the image into small regions, measuring an average value of colors for a predetermined time for each of the small regions, and the specifying unit is a small region range in which the luminance range is smaller than a predetermined value, and The range of the small area in which the difference between the average value of the color at the predetermined time for each small area and the average value of the entire image is larger than a predetermined threshold is specified as the area where the rip current is generated.

  When the region where the rip current is generated is specified by the specifying unit, a communication unit that transmits information on the region where the shore current is generated can be further included.

  A human detection unit for detecting a person can be further included based on the image.

  When the area where the rip current is generated is specified, the specifying unit can control the person detecting unit to detect the person based on the image.

  The human detection unit can be regarded as detecting a person in an area where the skin color is detected among the pixels in the image.

  The human detection unit is configured to detect a marine color, a wave color, and a skin color among the pixels in the image, and the skin color region, the marine color, the wave color, and the skin color other than It can be made to consider that the person was detected in the area | region of the color of.

  A driving unit for moving the main body of the monitoring device may be further included. When the person detecting unit detects a person, the specifying unit controls the driving unit to control the person. It can be made to move to the position corresponding to the area where the is detected.

  The drive unit can include a plurality of rotor blades.

  An audio output unit that announces the occurrence of the rip current and evacuation guidance may be further included. When the person detection unit detects a person, the identification unit controls the drive unit. The person can be moved to a position corresponding to the detected area and the voice output unit can be controlled to announce the occurrence of the rip-off flow and evacuation guidance.

  A dropping unit for dropping a lifesaving device can be further included, and when the person detecting unit detects a person, the specifying unit controls the driving unit so that the person is detected in an area where the person is detected. It can be moved to the corresponding position, and the throwing unit can be controlled so that the life-saving device is dropped on the detected person.

  A monitoring method according to one aspect of the present disclosure captures an image of a sea surface, and in the image, a region where a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and the color of the sea surface. A monitoring method including a step of identifying

  A program according to an aspect of the present disclosure includes an imaging unit that captures an image of a sea surface, and in the image, a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and the color of the sea surface. This is a program that causes a computer to function as a specifying unit that specifies a region in which the computer is located.

  In one aspect of the present disclosure, an image of a sea surface is captured, and in the image, a region where a rip current is generated on the sea surface is identified based on the presence or absence of waves on the sea surface and the color of the sea surface. Is done.

  According to one aspect of the present disclosure, it is possible to monitor the occurrence of a rip current using a marine image captured by a camera or the like.

It is a figure explaining a banking flow. It is a figure explaining the example of composition of the embodiment of the monitoring device to which this indication is applied. It is a flowchart explaining the monitoring process by the monitoring apparatus of FIG. It is a flowchart explaining the wave detection process of FIG. It is a figure explaining the wave detection process of FIG. It is a figure explaining the wave detection result of the wave detection process of FIG. It is a flowchart explaining the sea surface color detection process of FIG. It is a figure explaining the sea surface color detection result of the sea surface color detection process of FIG. It is a flowchart explaining the person detection process of FIG. It is a flowchart explaining the other process of the person detection process of FIG. And FIG. 11 is a diagram illustrating a configuration example of a general-purpose personal computer.

  Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially the same function, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<About offshore flow>
The monitoring device of the present disclosure is a so-called drone, which is a wirelessly operable flying device provided with an imaging unit including a camera. , To inform.

  In describing the monitoring device of the present disclosure, first, the rip current will be described.

  The offshore current is a strong tidal current that occurs locally at a width of about 10m from the coastal coast to the offshore.

  More specifically, as shown in FIG. 1, the rip current is seawater carried by a sea current (counter shore current) that strikes toward the coast G in the front part of the figure indicated by the solid line arrow of the sea S. However, by concentrating on a part of the range due to the undulations and depth of the coast G, for example, due to the ocean current flowing from the coast G in the front part of the figure indicated by the dotted arrow in the range Z toward the sea S It is a strong ebb tide that occurs.

  In FIG. 1, a solid line arrow on the sea S indicates the direction of the ocean current hitting the coast G, and a dotted arrow indicates the direction of the ocean current from the coast G toward the sea S. Moreover, in the range Z, the mode that the swimmer H is made to flow toward the offing by the rip current is drawn.

  The width of the offshore flow is generally said to be about 10 m, and the flow is very fast, so it is difficult for humans swimming to resist the flow. It is said that the speed of the offshore current may exceed 1m per second, and it is said that it is difficult even for swimming Olympic athletes to swim faster than that against the offshore current.

  However, the characteristic of the offshore current is that it occurs near the coast, in a range where almost no waves are generated in the sea where white waves are generated in the vicinity, and in the range where the seawater is cloudy. Therefore, it is difficult to recognize the occurrence while the rip current is generated, and beach accidents due to the rip current are unending.

  Therefore, the monitoring device of the present disclosure detects the characteristics of the rip current by detecting the characteristics of the rip current from an image of the sea, and prompts the swimmer to generate the rip current. This is to reduce the damage caused by.

<Configuration Example of Monitoring Device of the Present Disclosure>
Next, a configuration example of the monitoring device of the present disclosure will be described with reference to FIG.

  2 includes an imaging unit 31, a YC separation unit 32, a wave detection unit 33, a sea surface color detection unit 34, a rip current identification unit 35, a human detection unit 36, a position information detection unit 37, and a warning activation unit 38. , A drive unit 39, an audio output unit 40, and a communication unit 41.

  The imaging unit 31 is an image sensor made up of a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc., and takes an image in the sea surface direction that is incident through an optical system (not shown). Output to the YC separation unit 32.

  The YC separation unit 32 separates the image supplied from the imaging unit 31 into a luminance signal Y and a color difference signal C (Cb, Cr), and supplies the luminance signal Y to the wave detection unit 33, respectively. Is supplied to the sea surface color detection unit 34.

  The wave detection unit 33 specifies an area where a wave is generated in the imaging range based on the image formed of the luminance signal Y, and outputs the wave detection result to the offshore flow identification unit 35. In other words, on the sea, the area where no wave is generated is surrounded by the area where the wave is generated. The information on the area where the area and the information on the area where no wave is generated can be identified is supplied to the rip current identifying unit 35 as a wave detection result.

  The sea surface color detection unit 34 specifies an area in which the sea surface color is different from other areas in the imaging range based on the image including the color difference signal C, and supplies the sea surface color detection result to the rip current detection unit 35. That is, on the sea, the area where the sea surface color is cloudy (the area different from other areas) is an area where there is a high possibility that a rip current is generated, so there is information on the area where the sea surface color is different from other areas. The sea surface color detection result is supplied to the offshore current specifying unit 35.

  The rip current identifying unit 35 identifies the region where the rip current is generated in the imaging region by the imaging unit 31 from the wave detection result and the sea surface color detection result, and supplies the identification result to the warning activation unit 38. . More specifically, the rip current identification unit 35 superimposes the wave detection result and the sea surface color detection result, and identifies the region where either is detected as a region that is likely to have a rip current. In addition, the region where both overlap is specified as the region where the rip current is generated. As a result, the offshore flow identification unit 35 uses the information on the area where the possibility of the offshore flow is high and the area identified as the offshore flow is generated as the identification result as the warning activation unit 38. To supply. Of course, there are cases where neither the wave detection result nor the sea surface color detection result includes an area indicating the occurrence of a rip current, and in such a case, the occurrence of a rip current is generated in the imaging area. Specific information indicating that there is no data is output.

  The human detection unit 36 detects a person who is a swimmer on the sea based on the image captured by the imaging unit 31 in response to a request from the warning activation unit 38 and supplies the detection result to the warning activation unit 38. . More specifically, the person detection unit 36 detects the presence or absence of a skin color pixel in an image obtained by imaging the sea, and if there is a skin color pixel, it is considered that a person exists.

  The position information detection unit 37 includes, for example, a GPS (Global Positioning System) and the like, receives signals from a satellite (not shown) at predetermined time intervals, and detects latitude and longitude indicating its own position on the earth. The position information is supplied to the warning issuing unit 38.

  When the warning activation unit 38 identifies a region where a rip current is generated or a region that may be generated based on the specific information supplied from the shore flow identification unit 35 The human detection unit 36 is caused to detect a region where a rip current may be generated or a swimmer on the sea in the vicinity of the generated region by the human detection process. And when the person who is a swimmer is detected based on a person detection result, the warning issuing part 38 controls the drive part 39, and controls it to move to the position where the swimmer was detected.

  In addition, when the offshore current is generated or may be generated according to the specific information, the warning issuing unit 38, when the person who is a swimmer arrives at the detected position, the audio output unit 40 Is controlled to notify the swimmer at sea by information that warns that a rip current is or is likely to occur in the surrounding area. Thereby, a swimmer who is swimming at sea can evacuate immediately. The rip current is known to be able to evacuate from the rip current by swimming in a direction perpendicular to the direction of the flow. By outputting by voice, it is possible to more reliably protect the swimmer from damage caused by the offshore current.

  Furthermore, when the offshore current is generated or may be generated according to the specific information, the warning issuing unit 38 controls the communication unit 41 to generate the offshore current, or The management server of the management center (not shown) used by the user who manages himself / herself is notified of the location information together with the specific information indicating the information on the marine area that may be generated. By this notification, the user who manages and operates the monitoring device 11 may or may have generated a rip current based on information notified to a management server (not shown). It is possible to recognize this, and it is possible to prompt further attention and assist a quick rescue request.

  The drive unit 39 is operated by a user who manages the monitoring device 11 via the communication unit 41, based on an operation signal supplied from a remote operation device (not shown) or a management server. The flight is controlled by controlling the number of rotations and the rotation direction (pitch, roll, yaw, etc.) of each of the plurality of rotor blades for flying 11.

<Monitoring process>
Next, the monitoring process by the monitoring device 11 in FIG. 2 will be described with reference to the flowchart in FIG. In this monitoring process, the monitoring device 11 is flying in a state where the drive unit 39 is controlled from a remote location via the communication unit 41 and can capture an image of the sea at a predetermined height and a predetermined angle of view. Assuming that

  In step S <b> 11, the imaging unit 31 captures the sea to be monitored from a height having a predetermined angle of view, and supplies the captured image to the YC separation unit 32 and the human detection unit 36.

  In step S12, the YC separation unit 32 performs YC separation on the marine image to be monitored supplied from the imaging unit 31, outputs the luminance signal Y to the wave detection unit 33, and outputs the color difference signal C to the sea surface color detection unit. 34.

  In step S <b> 13, the wave detection unit 33 executes wave detection processing based on the luminance signal Y, and identifies and identifies a region where a rip current may occur on the sea within the imaging range. Information is output as a wave detection result.

<Wave detection processing>
Here, with reference to the flowchart of FIG. 4, the wave detection process in the wave detection part 33 is demonstrated.

  In step S51, the wave detection unit 33 stores the image formed of the luminance signal Y in a memory (not shown) and divides the image into, for example, small rectangular areas. The wave detection unit 33 stores an image composed of a luminance signal Y for a predetermined time in a memory having a predetermined capacity. When the wave detection unit 33 is full, the oldest image is deleted and a new image is stored. However, when a new image is always supplied, a plurality of images up to a predetermined time after the timing are stored in a state of being divided into small areas.

  In step S52, the wave detection unit 33 calculates a luminance value in a small area of the current image. More specifically, the wave detection unit 33 calculates the maximum value, the average value, or the minimum value of the luminance signal Y of the pixel in each small area. The luminance signal Y in the small area may be other than the maximum value, the average value, and the minimum value as long as the characteristics of the luminance signal Y of the pixels in the small area can be extracted.

  In step S <b> 53, the wave detection unit 33 acquires a change in the time direction of the maximum value, the average value, or the minimum value of the luminance values in each small region in all images for a predetermined time accumulated in a memory (not shown).

  In step S54, the wave detection unit 33 determines the luminance range (amplitude) for each small region from the change in the time direction of the maximum value, the average value, or the minimum value of the luminance values in the image for a predetermined time accumulated in the memory. Are mapped, and a portion having a luminance range (amplitude) smaller than a predetermined value is specified.

  In step S <b> 55, the wave detection unit 33 outputs a mapping result indicating that there is a high possibility that a rip current is generated in the specified region to the rip current detection unit 35 as a wave detection result.

  That is, as shown by the solid line waveform in FIG. 5, the fact that the range of the amplitude H of the luminance value for a predetermined time is large can be regarded as a wave generated in the small area. On the other hand, as shown by the dotted waveform in FIG. 5, the fact that the range of the amplitude H of the luminance value is small can be considered that no wave is generated in the small region. In FIG. 5, the vertical axis represents the luminance value amplitude H, and the horizontal axis represents time t.

  Therefore, as shown in FIG. 6, the wave detection unit 33 maps a distribution with a diagonal line rising to the right with respect to a small area of which the luminance value change range is smaller than a predetermined threshold among the small areas. This is output as a wave detection result. That is, in FIG. 6, the entire monitoring range is divided into rectangular small areas, and in the area with a diagonal line rising to the right, it is shown that the area where there is a high possibility that a rip current is generated. It is a wave detection result. Note that B in FIG. 6 indicates a coastline. In FIG. 6, the upper side of the coastline B is the sea S and the lower side is the coast G.

  Now, the description returns to the flowchart of FIG.

  In step S <b> 14, the sea surface color detection unit 34 performs sea surface color detection processing based on the color difference signal C, and identifies an area where a rip current may be generated on the sea within the imaging range. The identified information is output as the sea surface color detection result.

<Sea surface color detection processing>
Here, sea surface color detection processing will be described with reference to the flowchart of FIG.

  In step S71, the sea surface color detection unit 34 stores the image formed of the color difference signal C in a memory (not shown) and divides the image into, for example, small rectangular regions. Similarly to the wave detection unit 33, the sea surface color detection unit 34 stores an image composed of a color difference signal C for a predetermined time in a predetermined capacity memory. When a new image is deleted and accumulated, and a new image is always supplied, an image for a predetermined time from the timing is stored in a state of being divided into small areas.

  In step S72, the sea surface color detection unit 34 calculates an average value of the sea surface color for a predetermined time. That is, the sea surface color detection unit 34 calculates an average value of the sea surface colors of all the stored sea surface images for a predetermined time.

  In step S73, the sea surface color detection unit 34 obtains a difference between the average value of the average value of the color of the pixels in the small area for a predetermined time and the average value of the predetermined time for the colors of all the small areas for each small area. A small region having a difference from the average value larger than a predetermined threshold and a large color difference is identified and mapped as a region where the sea surface color is cloudy.

  In step S <b> 74, the sea surface color detection unit 34 outputs a mapping result indicating that there is a high possibility that a rip current is generated in the specified region to the shore current detection unit 35 as a sea surface color detection result.

  That is, for each small area, when the difference between the average value of the color for a predetermined time and the average value of the color for the entire image is large and the color difference is large, the small area is different from the other areas in color. However, the sea surface color is cloudy, indicating that there is a high possibility that a rip current has occurred.

  In view of this, the sea surface color detection unit 34 determines that the difference between the average value of the color value for a predetermined time and the average value of the color for the predetermined time of the entire image shown in FIG. A distribution with a slanting line to the lower right and a black color is mapped to a larger small area, and this is output as a sea surface color detection result.

  That is, in FIG. 8, the entire monitoring range is divided into rectangular small areas, and the small area indicated by the hatched portion that rises to the right is an area where there is a high possibility that a rip current is generated by the luminance signal Y. , And a small area with a diagonally slanting portion to the right indicates an area where there is a high possibility that a rip current is generated by the color difference signal C, and further, a small area indicated in black indicates the luminance signal Y and Since there is a possibility that a rip current is generated in any of the color difference signals C, an area where a shore current is considered to be generated is shown.

  Therefore, in FIG. 8, the range of the small area specified as the area where the rip current may have occurred due to the absence of the wave based on the luminance signal Y is shown by an upward-slashed shaded area and a black area. Based on the color difference signal C, the seawater was considered cloudy due to a different sea surface color, and was identified as an area where a rip current might have occurred. The range of the small area is a range of the small area with a diagonally downward slanting portion and black.

  Now, the description returns to the flowchart of FIG.

  In step S <b> 15, the shore current specifying unit 35 generates the shore current and the presence of the shore current based on the wave detection result supplied from the wave detection unit 33 and the sea surface color detection result. The position is specified, and the specified result is supplied to the warning issuing unit 38.

  That is, when a mapping result as shown in FIG. 8 is obtained, there is a high possibility that a rip current is generated in the shaded area that rises to the right due to the wave detection result, and the lower right according to the sea surface color detection result. There is a high possibility that offshore currents occur in the shaded area. Furthermore, in the black range, it is considered that there is a high possibility that a rip current is generated from the wave detection result and the sea surface color detection result, so it is considered that a rip current is actually generated. It is.

  From this, the offshore current specifying unit 35 specifies that the offshore current is generated in the black region, and the offshore current is shown in the upward-sloping oblique line part or the downward-sloping oblique line part. Identify as an area that may have occurred.

  That is, in step S15, for example, a mapping result as shown in FIG. 8 is output as a specific result.

  In step S <b> 16, the warning issuing unit 38 determines whether or not a rip current is generated according to the specific result. More specifically, when the warning issuing unit 38 includes at least one of a region where a rip current is likely to be generated or a region where a shore current is generated within the specific result, It is determined that a rip current is generated. Otherwise, it is determined that there is no rip current.

  That is, when the specific result is as shown in FIG. 8, since there is a region where a rip current may be generated and a region where a shore current is generated, The process proceeds to step S17 because it is regarded as occurring.

  In step S <b> 17, the warning issuing unit 38 instructs the human detection unit 36 to detect the presence or absence of a human in the captured image. In response to this instruction, the human detection unit 36 executes human detection processing to detect the presence or absence of a human in the captured image. At this time, the warning issuing unit 38 acquires its own position information supplied from the position information detecting unit 37 at the same time.

<Human detection processing>
Here, the human detection process will be described with reference to the flowchart of FIG.

  In step S <b> 91, the human detection unit 36 searches for a skin color region in the image supplied from the imaging unit 31. More specifically, for each pixel, the human detection unit 36 determines whether or not the pixel value is a pixel value corresponding to the skin color, and adds information indicating that the skin color is detected at the corresponding position. Mapping.

  In step S <b> 92, the human detection unit 36 outputs a human detection result including a mapping result in which information indicating whether or not skin color is detected for each pixel is mapped to the warning issuing unit 38.

  Through the above processing, the human detection result is output as mapping information indicating whether or not a pixel having a flesh color exists in the captured image. That is, when a person such as a swimmer exists in the image area, the image includes a skin color, and thus it is possible to determine whether or not a person is captured in the image by such processing. Become.

<Other human detection processing>
In the above, an example in which the presence / absence of a human in the imaging range at sea is extracted as skin color pixel mapping information has been described. However, it may not be detected. Therefore, in the image, the color of the sea, the color of the wave, and the skin color are detected, and information indicating that a human is present is mapped to the position where the skin color and other colors are detected. Humans may be detected with higher accuracy.

  Therefore, referring to the flowchart of FIG. 10, the marine color, wave color, and skin color pixels are detected in the captured image, and the skin color pixels and other pixels are detected by humans. A human detection process in which mapping is performed on the assumption that the data exists is described.

  In step S <b> 111, the human detection unit 36 determines whether or not the pixel value is a maritime color, a wave color, and a skin color for all pixels.

  In step S <b> 112, the human detection unit 36 regards pixels that are not any of the maritime color, wave color, and skin color (different color pixels) and skin color pixels among all the pixels as pixels that image humans. Mapping.

  In step S113, the human detection unit 36 supplies the mapping result as the human detection result to the warning issuing unit 38.

  With the above processing, in the captured image, the mapping result considered to be human for the skin color pixels or the different color pixels that are not classified as any of the marine color, wave color, and skin color Is detected as a human detection result. As a result, even when a person wearing clothing of a color other than the skin color exists on the sea, it can be detected with high accuracy.

  Now, the description returns to the flowchart of FIG.

  When the human detection result is acquired by the process of step S17, in step S18, the warning issuing unit 38 determines whether a human is detected on the sea based on the human detection result. For example, if the mapping result as the human detection result includes a mapping result indicating that a human is present, the warning issuing unit 38 regards that a human is present at sea, and the process proceeds to step S19. .

  In step S 19, the warning issuing unit 38 controls the communication unit 41, and along with its own location information supplied from the location information detection unit 37, a rip-off flow has occurred in a server or the like of a management center (not shown). The identification information of the rip current including the mapping information indicating the information, the information indicating that there is a person who may be a rescuer, and the person detection result indicating the position of the rescuer are notified.

  Through this process, the server of the management center that manages the monitoring device 11 generates the rip current, the position where the rip current is generated, the presence or absence of a rescuer who may be shed by the rip current, and Information on the position of a person who may be a rescuer is notified. As a result, the user who manages the management center can promptly arrange for alerting and rescue accompanying the occurrence of a rip current, and can reduce the damage caused by the rip current.

  In step S <b> 20, the warning issuing unit 38 determines the position of the person on the sea detected as the human detection result based on the position information and the human detection result, and moves the driving unit 39 so as to move toward the corresponding position. Control. Further, the warning activation unit 38 controls the sound output unit 40 by approaching the monitoring device 11 serving as the main body device in order from a person existing at a position close to the position where the rip-off flow is detected. The information that prompts the evacuation is provided by transmitting the voice that the flow has occurred. For example, for swimmers who are not involved in the rip current, an announcement is made to convey the direction in which the rip current is generated and to encourage them not to approach the sea area where the rip current is generated. You may make it do. In addition, it is known that the rip current can be separated by swimming in a direction perpendicular to the direction of the flow, so it is easy to detach for a swimmer or the like caught in the rip current. You may make it announce the direction which should swim.

  In Step S18, when a swimmer who may be swept away by the rip current is not detected, in Step S21, the warning activation unit 38 generates a rip current along with the occurrence of the rip current. Only the specific information for specifying the position is notified to the server or the like of the management center, and the process proceeds to step S22. That is, in this case, there is a high possibility that there is no victim due to the offshore current, so only the occurrence of the offshore current and its position are notified.

  Furthermore, when it is determined in step S16 that no rip current has occurred, the process proceeds to step S22. That is, the processes in steps S17 to S21 are skipped.

  In step S22, the warning issuing unit 38 controls the communication unit 41 to determine whether or not an operation end has been instructed. If the end has not been instructed, the process returns to step S11, and thereafter The process is repeated. In step S22, when the end of the operation is instructed from the server of the management center (not shown) via the communication unit 41, the drive unit 39 is controlled to return to a predetermined position such as the management center. The processing is terminated as soon as it arrives.

  By the above processing, the occurrence of offshore currents and their positions that cannot be recognized from the coastal area are detected from the wave detection result and the sea surface color detection result based on the luminance signal Y and the color difference signal C of the captured image. It is possible to detect and notify appropriately. As a result, it is possible to alert a nearby swimmer to the occurrence of a rip current, and to reduce the damage caused by the rip current.

  In addition, by performing a human detection process using the captured image, it is possible to detect and notify the presence or absence of a swimmer in the vicinity and the position thereof when a rip current occurs. As a result, it is possible to promptly alert nearby swimmers, prompt evacuation, rescue, etc., and to reduce damage caused by rip currents.

  Further, the monitoring device 11 approaches the swimmer detected at sea and notifies the occurrence of the rip-off current and the position of the occurrence by voice announcement to prevent the rip-off flow from being caught in advance. It becomes possible.

  In addition, it becomes possible to instruct the swimmer who is involved in the offshore current in the direction to leave the offshore current, and it is possible to reduce damage to the offshore current.

  In the above, the monitoring process by one monitoring device 11 has been described. However, since the monitoring range at sea is wide, a plurality of monitoring devices 11 are used to configure a wide range offshore monitoring system. It may be realized. At this time, a plurality of monitoring devices 11 may be used in cooperation. For example, when the occurrence of a rip current is detected in any of the plurality of monitoring devices 11, the plurality of other monitoring devices 11 are used. In addition, the information may be notified to alert the swimmer in each monitoring range of the offshore current.

  In the above description, an example has been described in which the monitoring device 11 approaches a swimmer who is likely to be a victim due to a rip current and announces the occurrence of the rip current and the evacuation direction. May be equipped with a life-saving device or the like, and may be dropped to a swimmer who is washed away by the offshore current to assist the rescue.

  Furthermore, in the above, when the monitoring device 11 detects the occurrence of a rip current or when a swimmer is detected by the human detection process, the communication unit 41 is controlled to notify the server of the management center. Alternatively, an example has been described in which the drive unit 39 is controlled to approach the vicinity of the swimmer and to notify the occurrence and position of the rip current and the evacuation direction by voice.

  However, for example, when a swimmer wears a wristwatch-type wearable terminal or the like having a waterproof function and a communication function, the wearable terminal or the like is notified of information such as the occurrence of a rip-off flow and the evacuation direction. You may do it. In addition, the server of the management center or the like may notify the wearable terminal of information such as the occurrence and position of the offshore current from the monitoring device 11 and the evacuation direction. Furthermore, a swimmer may transmit via a wearable terminal and interact with an operator who operates the server of the management center to generate a rip-off flow or guide evacuation.

  In the above description, an example in which the warning issuing unit 38 controls the communication unit 41 to issue a warning based on whether or not a rip current has occurred and its position has been described. The function is given to the server of the management center etc., and the monitoring device 11 is made to notify only the specific information of the offshore flow, the human detection result, and the location information, and the warning and evacuation guidance are managed. You may make it notify from the server of a center.

  Furthermore, the flying drone is equipped with only the imaging unit 31, and only the captured image is notified to the server of the management center so that all the functions after the YC separation unit 32 in FIG. May be.

  Further, since the monitoring device 11 can be realized by a flying device such as a drone, for example, a conventional monitoring system can be realized at low cost without using a large and expensive device such as a helicopter. Can be realized.

  Further, in the above description, the configuration example of the monitoring device 11 having a flight function such as a drone has been described. However, the imaging unit 31 is fixed to a hill capable of monitoring the sea or a monitoring pedestal provided on a marine support. Thus, the same processing may be executed by providing a configuration other than the drive unit 39 in FIG. In this case, since a flying device such as a drone is not required, it is possible to realize a further offshore flow monitoring system at a low cost.

<Example executed by software>
By the way, the series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 11 shows a configuration example of a general-purpose personal computer. This personal computer incorporates a CPU (Central Processing Unit) 1001. An input / output interface 1005 is connected to the CPU 1001 via a bus 1004. A ROM (Read Only Memory) 1002 and a RAM (Random Access Memory) 1003 are connected to the bus 1004.

  The input / output interface 1005 includes an input unit 1006 including an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 for outputting a processing operation screen and an image of the processing result to a display device, programs, and various types. A storage unit 1008 including a hard disk drive for storing data, a LAN (Local Area Network) adapter, and the like, and a communication unit 1009 for performing communication processing via a network represented by the Internet are connected. Also, a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk (including an MD (Mini Disc)), or a semiconductor A drive 1010 for reading / writing data from / to a removable medium 1011 such as a memory is connected.

  The CPU 1001 is read from a program stored in the ROM 1002 or a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed in the storage unit 1008, and loaded from the storage unit 1008 to the RAM 1003. Various processes are executed according to the program. The RAM 1003 also appropriately stores data necessary for the CPU 1001 to execute various processes.

  In the computer configured as described above, the CPU 1001 loads the program stored in the storage unit 1008 to the RAM 1003 via the input / output interface 1005 and the bus 1004 and executes the program, for example. Is performed.

  The program executed by the computer (CPU 1001) can be provided by being recorded on the removable medium 1011 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

  In the computer, the program can be installed in the storage unit 1008 via the input / output interface 1005 by attaching the removable medium 1011 to the drive 1010. Further, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. In addition, the program can be installed in advance in the ROM 1002 or the storage unit 1008.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  In this specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Accordingly, a plurality of devices housed in separate housings and connected via a network and a single device housing a plurality of modules in one housing are all systems. .

  The embodiment of the present disclosure is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present disclosure.

  For example, the present disclosure can take a configuration of cloud computing in which one function is shared by a plurality of apparatuses via a network and is jointly processed.

  In addition, each step described in the above flowchart can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

  Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by being shared by a plurality of apparatuses in addition to being executed by one apparatus.

In addition, this indication can also take the following structures.
<1> an imaging unit that captures an image of the sea surface;
A monitoring device comprising: a specifying unit that specifies a region where a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and a color of the sea surface in the image.
<2> a separation unit that separates the image of the sea surface into a luminance signal and a color difference signal;
Based on the luminance signal, a wave detection unit that detects the presence or absence of waves on the sea surface,
A sea surface color detection unit that detects the color of the sea surface based on the color difference signal;
The identification unit is obtained based on the wave detection result of the wave detection unit, based on the presence or absence of waves on the sea surface and the sea surface color obtained from the sea surface color detection result of the sea surface color detection unit, The monitoring device according to <1>, wherein an area where a rip current is generated on the sea surface is specified.
<3> The wave detector
Dividing the image consisting of the luminance signal into small areas, measuring the luminance range in the displacement of the luminance value at a predetermined time for each small area,
The monitoring unit according to <2>, wherein the specifying unit specifies a range of a small region in which the luminance range is smaller than a predetermined value as a region where the rip current may occur.
<4> The monitoring device according to <3>, wherein the displacement of the luminance value at a predetermined time for each small region includes any one of a maximum value, an average value, and a minimum value of the luminance value at the predetermined time for each small region. .
<5> The sea surface color detection unit
Dividing the image consisting of the color difference signal into small areas, measuring the average value of the color at a predetermined time for each small area,
The identifying unit may cause the rip current to flow in a small area where the difference between the average value of the color in the predetermined time for each small area and the average value of the entire image is larger than a predetermined threshold. The monitoring device according to <2>, which is specified as an area.
<6> The wave detection unit is
Dividing the image consisting of the luminance signal into small areas, measuring the luminance range in the displacement of the luminance value at a predetermined time for each small area,
The sea surface color detection unit
Dividing the image consisting of the color difference signal into small areas, measuring the average value of the color at a predetermined time for each small area,
The specifying unit is a small area range in which the luminance range is smaller than a predetermined value, and
A range of a small area in which the difference between the average value of the color at a predetermined time for each small area and the average value of the entire image is larger than a predetermined threshold is specified as the area where the rip current is generated <2> The monitoring device described in 1.
<7> When the area where the rip current is generated is specified by the specifying unit, further includes a communication unit that transmits information on the area where the shore current is generated. <1> to <6 > The monitoring apparatus in any one of>.
<8> The monitoring device according to any one of <1> to <7>, further including a human detection unit that detects a person based on the image.
<9> The monitoring according to <8>, wherein the specifying unit controls the person detection unit to detect the person based on the image when the region where the rip current is generated is specified. apparatus.
<10> The monitoring device according to <8>, wherein the human detection unit considers that a person has been detected in an area where skin color is detected among the pixels in the image.
<11> The human detection unit detects a marine color, a wave color, and a skin color among the pixels in the image, and the skin color region, the marine color, the wave color, and the The monitoring device according to <8>, in which it is assumed that a person is detected in a color region other than the skin color.
<12> Further includes a drive unit for moving the main body of the monitoring device,
The monitoring device according to <8>, wherein when the person detection unit detects a person, the specifying unit controls the drive unit to move the position to a position corresponding to a region where the person is detected.
<13> The monitoring device according to <12>, wherein the driving unit includes a plurality of rotor blades.
<14> further includes an audio output unit that announces the occurrence of the rip current and evacuation guidance,
When the person detecting unit detects a person, the specifying unit controls the driving unit to move to a position corresponding to the area where the person is detected, and controls the audio output unit to perform the separation. The monitoring device according to <8>, which announces the occurrence of shore current and evacuation guidance.
<15> It further includes a throwing part for dropping a life preserver,
When the person detecting unit detects a person, the specifying unit controls the driving unit to move the person to a position corresponding to a region where the person is detected, and controls the projecting portion to detect the person. The monitoring device according to <8>, wherein the lifesaving device is dropped on the person.
<16> Take an image of the sea surface,
In the image, the monitoring method includes a step of identifying a region where a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and the color of the sea surface.
<17> An imaging unit that captures an image of the sea surface;
In the image, based on the presence or absence of waves on the sea surface and the color of the sea surface, as a specifying unit for specifying a region where a rip current is generated on the sea surface,
A program that allows a computer to function.

  DESCRIPTION OF SYMBOLS 11 Monitoring apparatus, 31 Image pick-up part, 32 YC separation part, 33 Wave detection part, 34 Sea surface color detection part, 35 Offshore flow specific part, 36 Person detection part, 37 Position information detection part, 38 Warning issuing part, 39 Drive part , 40 Audio output unit, 41 Communication unit

Claims (17)

An imaging unit for capturing images of the sea surface;
A monitoring device comprising: a specifying unit that specifies a region where a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and a color of the sea surface in the image. A separation unit for separating the image of the sea surface into a luminance signal and a color difference signal;
Based on the luminance signal, a wave detection unit that detects the presence or absence of waves on the sea surface,
A sea surface color detection unit that detects the color of the sea surface based on the color difference signal;
The identification unit is obtained based on the wave detection result of the wave detection unit, based on the presence or absence of waves on the sea surface and the sea surface color obtained from the sea surface color detection result of the sea surface color detection unit, The monitoring apparatus according to claim 1, wherein an area where a rip current is generated on the sea surface is specified. The wave detector is
Dividing the image consisting of the luminance signal into small areas, measuring the luminance range in the displacement of the luminance value at a predetermined time for each small area,
The monitoring device according to claim 2, wherein the specifying unit specifies a range of a small region in which the luminance range is smaller than a predetermined value as a region where the rip current may occur. The monitoring apparatus according to claim 3, wherein the displacement of the luminance value at a predetermined time for each small region includes any one of a maximum value, an average value, and a minimum value of the luminance value at the predetermined time for each small region. The sea surface color detection unit
Dividing the image consisting of the color difference signal into small areas, measuring the average value of the color at a predetermined time for each small area,
The identifying unit may cause the rip current to flow in a small area where the difference between the average value of the color in the predetermined time for each small area and the average value of the entire image is larger than a predetermined threshold. The monitoring device according to claim 2 specified as a region. The wave detector is
Dividing the image consisting of the luminance signal into small areas, measuring the luminance range in the displacement of the luminance value at a predetermined time for each small area,
The sea surface color detection unit
Dividing the image consisting of the color difference signal into small areas, measuring the average value of the color at a predetermined time for each small area,
The specifying unit is a small area range in which the luminance range is smaller than a predetermined value, and
The range of the small area where the difference between the average value of the color for each small area at a predetermined time and the average value of the entire image is larger than a predetermined threshold is specified as the area where the rip current is generated. The monitoring device described in 1. The monitoring apparatus according to claim 1, further comprising: a communication unit that transmits information on a region where the rip current is generated when the region where the shore current is generated is specified by the specifying unit. The monitoring apparatus according to claim 1, further comprising a person detection unit that detects a person based on the image. The monitoring device according to claim 8, wherein the specifying unit controls the person detection unit to detect the person based on the image when the region where the rip current is generated is specified. The monitoring apparatus according to claim 8, wherein the human detection unit considers that a person has been detected in an area where skin color is detected among the pixels in the image. The human detection unit detects sea color, wave color, and skin color among the pixels in the image, and the skin color region, the sea color, the wave color, and the skin color other than the skin color are detected. The monitoring apparatus according to claim 8, wherein a person is detected in the color area. A drive unit for moving the main body of the monitoring device;
The monitoring apparatus according to claim 8, wherein when the person detection unit detects a person, the specifying unit controls the drive unit to move the position to a position corresponding to a region where the person is detected. The monitoring device according to claim 12, wherein the drive unit includes a plurality of rotor blades. A voice output unit for announcing the occurrence of the offshore current and evacuation guidance;
When the person detecting unit detects a person, the specifying unit controls the driving unit to move to a position corresponding to the area where the person is detected, and controls the audio output unit to perform the separation. The monitoring apparatus according to claim 8, wherein the occurrence of a shore current and evacuation guidance are announced. It further includes a lowering part for dropping a life preserver,
When the person detecting unit detects a person, the specifying unit controls the driving unit to move the person to a position corresponding to a region where the person is detected, and controls the projecting portion to detect the person. The monitoring device according to claim 8, wherein the life-saving device is dropped on the person. Take an image of the sea surface,
In the image, the monitoring method includes a step of identifying a region where a rip current is generated on the sea surface based on the presence or absence of waves on the sea surface and the color of the sea surface. An imaging unit for capturing images of the sea surface;
In the image, based on the presence or absence of waves on the sea surface and the color of the sea surface, as a specifying unit for specifying a region where a rip current is generated on the sea surface,
A program that allows a computer to function.

Images