KR101781757B1 - Underwater image processing device for object recognition and method there of - Google Patents

Abstract

Various embodiments of the present invention relate to an underwater image processing device for detecting an object using an underwater sonar image, and a method thereof. According to an embodiment, the underwater image processing device comprises a sensing part for sensing a sonar image through at least one sensor; a storage part including a convolutional neural network; and a processing part for detecting a target object designated under water by using the sonar image and the convolution neural network. The processing part comprises an image collection module for collecting the sonar image through the sensing part; an object checking module for detecting an object from the collected sonar image; and an object determination module for determining the target object among the detected objects by using the detected object and the convolution neural network. It is possible to search for a specific object under water by using a sonar.

Description

[0001] UNDERWATER IMAGE PROCESSING DEVICE FOR OBJECT RECOGNITION AND METHOD THERE OF} [0002]

The present invention relates to an apparatus for processing an underwater image for recognizing an object and a method thereof, and more particularly, to an apparatus and method for processing an underwater image for recognizing an object in an underwater view.

Conventionally, in order to obtain information necessary for position recognition and navigation in an underwater environment, various researches on object search, recognition, and navigation using image information have been conducted.

In recent years, research has been carried out to recognize objects using underwater image processing, to measure relative positions, and to correct errors in other sensor results. In many studies, a method of acquiring images using an optical camera has been proposed, The optical image has a problem that it is difficult to overcome a clock limited by the influence of turbidity and light.

As an alternative to the above-mentioned optical camera, a method of determining the underwater situation through an image obtained using an underwater image sonar is widely used.

In recent years, techniques have been researched to reduce the difficulty of securing a visual field due to turbidity and to observe an image of an underwater environment by performing post-processing (image processing) on the sonar image and the like through a computer or the like.

Here, the image sonar is an instrument for measuring the surrounding environment using ultrasonic waves, and is useful for grasping the underwater environment. However, the sonar image is a characteristic that is highly influenced by the environment such as the material of the object Therefore, there is a limit to clearly expressing objects in water.

As described above, since the sonar image has a limitation that it can not clearly convey the information of the object, it is difficult to search and recognize the object only by the sonar image itself, and in addition to the image having a lower image quality and noise than the optical camera And it was difficult for the post-processing of the image to make it easier to identify the object.

Patent Document 10-2014-0083195 discloses a method of searching for a candidate object while moving the floor in water, moving to a predetermined position with respect to a candidate object when a candidate object is searched, Obtaining a candidate group by calculating a correlation coefficient between the obtained sonar image and a simulated sonar image template for a plurality of objects, and obtaining a sonar image while rotating at a predetermined angle around the candidate object Calculating a correlation coefficient between the obtained sonar image by angle and a sonar image template map including a sonar image for each of a plurality of objects by angle, and determining a target object when the correlation coefficient is equal to or greater than the first threshold value; An underwater object recognition method using an image template is disclosed.

However, the patent document discloses a simple method of recognizing an object by photographing a sonar image at various angles and comparing the correlation with the simulated sonar image template many times for each angle, and the resolution is low due to noise There is still a limit to present a method for clearly recognizing a specific object using a sonar image which is difficult to confirm by a simple comparison.

10-2014-0083195 (Patent Document 1)

Therefore, according to various embodiments of the present invention, an underwater image processing apparatus and method for object recognition for recognizing a specific object to be searched by using a sonar in water whose turbidity is high and light is not visible and visibility is not ensured is provided can do.

In addition, according to various embodiments of the present invention, a true / false label is set for various objects, and a specific object among unlabeled images is set using an image of objects with labels set to a predetermined true / false It is possible to provide an apparatus for processing an underwater image for recognizing an object and a method thereof.

According to an embodiment of the present invention, there is provided an image sensing apparatus comprising: a sensing unit for sensing a sonar image through at least one sensor; A storage unit including a convolutional neural network; And a processing unit for detecting a target object in the water using the sonar image and the convolutional neural network, wherein the processing unit comprises: an image collection module for collecting the sonar image through the sensing unit; An object checking module for detecting an object from the collected sonar image; And an object determining module for determining the object among the detected objects using the detected object and the convolution neural network.

According to various embodiments, the underwater image processing apparatus for object recognition further includes at least one communication unit for communicating with an external device, and the image acquisition module can receive the sonar image through the communication unit.

According to various embodiments, the processing unit may include: a neural network generating the convolution neural network based on neural network data according to the convolution of the input image and the result of subsampling, And a processing module.

According to various embodiments, max-pooling may be performed with the subsampling.

According to various embodiments, when the image extracted from the sonar image includes the target object, the object confirmation module may set a true label on the extracted image and transmit the true label to the neural network processing module.

According to various embodiments, the object identification module may be configured to determine whether the image extracted from the sonar image includes at least one of a part of the object, an object, a part of the object, A false label can be set and transmitted to the neural network processing module.

According to various embodiments, the object identification module may extract at least a portion of the sonar image and convert the size of the extracted image.

According to various embodiments, the object determination module may determine the target object among the objects whose labels have not been determined according to the sub-sampling result of the convolution neural network and the true / false labels.

According to an embodiment of the present invention, there is provided a method of processing an object using an underwater image, comprising: collecting a sonar image; Detecting an object from the collected sonar image; And determining a previously designated target object among the detected objects using the detected object and the convolution neural network. The present invention provides an underwater image processing method for object recognition.

According to various embodiments, the sonar image may be photographed through a sensing unit or received from at least one external device.

According to various embodiments, an underwater image processing method for object recognition may include generating an image with an input true / false label set therein and the convolutional neural network based on neural network data according to convolution and sub-sampling results of the input image, ; ≪ / RTI >

According to various embodiments, max-pooling may be performed with the subsampling.

According to various embodiments, the step of detecting an object from the collected sonar image may set a true label on the extracted image if the image extracted from the sonar image includes the target object.

According to various embodiments, detecting the object from the collected sonar image may extract at least a portion of the sonar image and convert the size of the extracted image.

According to various embodiments, the step of detecting an object from the sonar image may include detecting at least one of a part of the target object, an object, at least one of a part of the object, , A false label can be set on the extracted image.

According to various embodiments, the step of determining a pre-determined target object among the detected objects using the detected object and the convolution neural network may include determining a label according to a sub-sampling result of the convolution neural network and a true / false label And determine the target object among the non-target objects.

In accordance with various embodiments of the present invention, an underwater image processing apparatus is provided that prior to shooting an image of an underwater terrain, an object, etc., generates a convolutional neural network using the captured image, By using the learned information about the false to detect the target object among the unlabeled images, accurate recognition of the object in the undersea feature can be realized.

According to various embodiments of the present invention, the convolution neural network of an underwater image processing apparatus used for detecting a target object continuously learns a variety of images labeled as true / false to detect a target object, Thereby improving the recognition speed and accuracy of the target object to be searched.

1 shows a structure of an underwater image processing apparatus according to an embodiment of the present invention.
2 shows a detailed structure of a processing unit in an underwater image processing apparatus according to an embodiment of the present invention.
FIG. 3 illustrates a situation in which a target object is photographed in an underwater image processing apparatus according to an embodiment of the present invention.
Figs. 4 and 5 illustrate a flow in which an underwater image processing apparatus according to an embodiment of the present invention processes an underwater image to generate a convolution neural network.
6 is a structural diagram of a convolutional neural network generated by an underwater image processing apparatus using an underwater image according to an embodiment of the present invention.
FIG. 7 is a flowchart illustrating an operation of an underwater image processing apparatus according to an exemplary embodiment of the present invention to generate a convolutional neural network using an image of a target object and an underwater image.
FIG. 8 is a flowchart illustrating an operation of detecting a target object from an image that has not been set by using the convolution neural network generated by the underwater image processing apparatus according to an embodiment of the present invention.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, it is to be understood that the invention is not limited to the specific embodiments thereof, And equivalents and alternatives falling within the spirit and scope of the invention. In order to clearly illustrate the present invention in the drawings, parts not related to the description may be omitted, and the same reference numerals may be used for the same or similar components throughout the specification.

In various embodiments of the present invention, the expressions "or "," at least one ", etc. may denote one of the words listed together or may represent a combination of two or more. For example, "A or B "," at least one of A and B "may include only one of A or B, and may include both A and B.

In various embodiments of the present invention, expressions such as "first", "second", "first", "second", etc. may modify various components, . For example, the first device and the second device are both devices and may represent different devices. Also, unless the elements of the configuration, function, operation, etc. of the first device are the same as or similar to the second device, the first device can be named as the second device, without departing from the scope of the various embodiments of the present invention, Similarly, the second device may also be termed the first device.

In the various embodiments of the present invention, when an element is referred to as being "connected" or "connected" to another element, the elements may be directly connected or connected, It should be understood that there may be one and the same time. On the other hand, if an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that no other element exists between the elements.

The terms used in various embodiments of the present invention are intended to illustrate a specific embodiment and are not to be construed as limiting the invention, for example, the singular forms "a," "an, ≪ / RTI >

An apparatus according to various embodiments of the present invention, for example, an underwater image processing apparatus for object recognition (hereinafter, an underwater image processing apparatus), can be described as a manned or unmanned submersible. An underwater image processing apparatus that does not explicitly state the limitations may be applied to various types of devices such as an autonomous underwater vehicle (AUV), a human-operated submersible vehicle, a remotely operated vehicle , ROV), etc. It should be appreciated that an underwater image processing apparatus according to various embodiments of the present invention may also be configured with one or more of the various devices described. For example, the device may be provided as a structure that includes at least a portion of the devices described, or at least some of the functionality of the device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following drawings and description, according to various embodiments of the present invention, an underwater image processing apparatus may be an apparatus for detecting a specified object (a target object) from at least one image. At this time, the underwater image processing apparatus can use an artificial neural network (hereinafter referred to as a convolution neural network (CNN)) that uses convolution to detect a target object from an image.

1 shows components of an underwater image processing apparatus 10 according to an embodiment of the present invention. The underwater image processing apparatus 10 may include at least one of a processing unit 101, a sensing unit 103, an input unit 105, a communication unit 107, and a storage unit 109.

The processing unit 101 receives data from the above-described other components (for example, the sensing unit 103, the input unit 105 communication unit 107, and the storage unit 109), decodes the received data, It is possible to execute processing of the data.

For example, the processing unit 101 may control the components included in the underwater image processing apparatus 10 and / or the connected external apparatus 130 that move underwater in the sea or river, along a predetermined path, As shown in FIG. The processing unit 101 may apply various path tracking methods such as a radio navigation method and a speculative navigation method in processing the underwater image processing apparatus 10 and / or the external apparatus 130 in the water.

At this time, the processing unit 101 is configured to perform a predetermined process based on at least one image sensor (e.g., a camera), a position measuring sensor (hereinafter referred to as a position sensor), a depth measuring sensor The image processing apparatus 10 and / or the external apparatus 130 can be moved or moved.

The processing unit 101 can take images of objects, background images that are background, and / or images containing a target object through at least one of the sensors included in the sensing unit 103.

The processing unit 101 may generate the convolution neural network algorithm using the images captured and / or collected. Wherein the image for generating the convolution neural network algorithm comprises at least some of an image of a target object for underwater movement confirmation, an underwater image comprising a recycled object, various underwater images, and an underwater image comprising at least one object .

Here, the image to be applied to the convolutional neural network may include at least one sonar image. The sensing unit 103 of the underwater mobile device 10 may include at least one sound navigation ranging sonar such as a forward-looking sonar image and / or an obstacle avoidance sonar ), And the like. In addition, when the sensing unit 103 includes a camera, the sensing unit 103 may further include at least one optical image photographed through the camera.

The processing unit 101 can detect a target object from an image collected based on the generated convolution neural network and an object recognition algorithm using the same.

The processing unit 101 generates a learned convolution neural network according to a convolution neural network algorithm for a target object, an object and a background image using the collected image, and detects a designated target object during underwater movement using the generated convolution neural network . Here, the operation for generating the convolutional neural network may include an operation for learning (or training) the convolutional neural network already created using the sonar image, without limiting to the operation for generating the new convolutional neural network.

The generation of the convolutional neural network of the underwater image processing apparatus 10 and the object recognition operation using the same can be described with reference to the detailed components of the processing unit 101 shown in FIG.

2, in the operation of generating a convolutional neural network and the recognition of a target object using the operation, an image acquisition module 201, an object verification module 203, a neural network processing Module 205, and object determination module 207. In one embodiment,

In other words, the image acquisition module 201 may collect background images that do not include a target object, at least one object other than the target object, and / or a specific object. Here, the image may be photographed through the sensing unit 103, received through the communication unit 107, or may use the image stored in the storage unit 109. [

Here, the image of the target object is preferably arranged so as to include a shape for at least one direction with respect to all directions of the target object, preferably, at least as shown in Fig. 3 so as to specify a two- Images taken from more than one direction can be collected.

For example, the image to be collected may include at least some of the images 301-1, 301-3, and 301-5 taken at the different positions 30-1, 30-3, and 30-5 . As the direction of taking a target object and the number of times of shooting are increased, the accuracy of the target object can be improved. In addition, the comparison group (or the control group) is improved in the process of detecting the target object from the image, thereby improving the accuracy of detection of the target object.

3, the image 301-1, 301-3, and 301-5 of the target object is photographed at three different positions. However, the present invention is not limited to this, It is obvious that the position and the number of times of shooting can be changed.

The object identification module 203 can identify the objects included in the image to generate a data set of the convolutional neural network using the image. According to one embodiment, the object identification module 203 can use the collected images to identify an image including a target object, an image including an object, and / or a background image.

The object identification module 203 can collect the extracted image according to the shape of the object or extract the shape of the objects from the image or mark the shape of the objects. Here, in using the term of "extraction ", it can be expressed as an operation of copying and cutting at least a part of an image in the form of a closed figure. It may also include the same or similar meaning of trimming, cropping, and the like.

Objects may include terrain, artificial or natural structures (or structures), objects, devices, etc., that are distinguished from images. The object may include a corner point and / or a contour for at least a portion, whereby the shape is specified and / or its shape may be determined to be a closed shape using the detected corner point and / or contour as described above.

The target object can be defined as an object designated to check the existence and position during underwater movement through the underwater image processing apparatus 10 or the external apparatus 130 connected to the underwater image processing apparatus 10. [ In setting (or determining) the target object, it is possible to set among the objects extracted from the image, or to set a new object as the target object.

The object identification module 203 may generate a data set for the convolutional neural network using the collected images and the object and the captured image of the target object. According to one embodiment, the object identification module 203 may extract at least a portion of the collected image and determine it as input data of the convolution neural network algorithm.

For example, the object validation module 203 may extract at least some images of a specified size from the collected original image. According to one embodiment, the size of a sonar image (e.g., an initial image) photographed through the sensing unit 103 may be composed of 96x512 pixels. Here, the object identification module 203 can extract at least a part from the sonar image into an image of a specified size, for example, 32x128 pixels. The object check module 203 can convert the extracted image into an image of 32x32 pixels (e.g., image resizing), and determine the converted image as input data.

At this time, if the first image is an image extracted based on the shape of the object, for example, it differs from the designated size (for example, the size of photographed sonar or image, 96x512) 32x128 pixels, and the 32x32 pixel image generated by performing the image conversion can be determined as the input data.

In extracting the image of 32x128 pixels from the initial image, the object check module 203 extracts the image of the extracted 32x128 pixels (hereinafter, extracted image) and the image of the converted 32x32 pixels (hereinafter referred to as " , Converted image) can be generated with reference to FIGS. 4 and 5. FIG. At this time, it is possible to set a true label depending on whether or not the target object is included in the extracted image to be generated.

4, when the target object is included in the first image 401, the object identification module 203 determines whether the extracted image (i.e., 411-3), and can set a true label on the extracted image 411-3.

The object identification module 203 can perform image conversion on the extracted image 411-3 to generate the converted image 411-5 and set the true label on the converted image 411-5. Here, in setting the true label to the extracted image 411-3 and the converted image 411-5, it is described that the label is set for each image generated. However, the present invention is not limited to this, and the predetermined label can be maintained even during image conversion.

In addition, a false label can be set for the first image not including the target object and the extracted image not including the target object.

According to one embodiment, the object identification module 203 determines whether the image 511-i extracted from the first image 501,503, when not including the target object in the first image 401, 11, 511-13, 511-15, 511-17, 513-3, and 523-3), and a false label can be set on the extracted image

At this time, when the image extracted from the original image is generated, the object confirmation module 203 determines whether or not the images are overlapped with each other such that the images do not overlap with each other, such as the extracted images 511-11, 511-13, 511-15, 511-1 can be set, or the regions 513-1 and 523-1 can be set such that at least some images overlap, such as the extracted images 513-3 and 5123-3.

The object confirmation module 203 performs image conversion on the extracted images 511-11, 511-13, 511-15, 511-17, 513-3, and 523-3 to convert the converted images 511-21, 511-23, 511-25, 511-27, 513-5, and 523-5, and sets a false label on the converted image 411-5.

Additionally, if it includes at least a portion of the target object but does not include the entire shape of the target object, for example, only a portion of the target object may be included, or some portion may be obscured by another object, and / You can optionally set a true or false label on an extracted 32x128 pixel image if it falls. According to various embodiments of the present invention, a false label can be set on an extracted 32x128 pixel image that does not include the entire shape of the target object.

The neural network processing module 205 may learn the convolution neural network in time with the input data determined through the object identification module 203 and the convolutional neural network algorithm. One embodiment of the convolutional neural network structure and algorithm can now be described with reference to FIG.

Referring to FIG. 6, the neural network processing module 205 can verify the transformed image 601 from the input data of the data set, and perform convolution on each transformed image.

According to one embodiment of the convolution for the transformed image 601, it is possible to move (e.g., shift) the transformed image in units of a two-dimensional region 603 of a specified size and perform a series of regular operations have.

The neural network processing module 205 can sub-sample the convolution image (hereinafter, convoluted image) 603 'and store the sampling result as information on the transformed image. For example, the neural network processing module 205 may apply max-pooling in subsampling the convoluted image 603 '. The neural processing module 205 determines the result of subsampling of the convoluted image 603 'as neural network data 603' 'for the transformed image 601 and outputs information about the transformed image 601 Can be stored.

That is, the neural network data may include true labels or false labels based on the result of subsampling and the extracted image.

The underwater image processing apparatus 10 can compare the extracted images including the images and / or objects to be unlabeled with the neural network data including the labeled images of the convolution neural network and the set labels.

The underwater image processing apparatus 10 can determine true or false of an object whose label is not set according to the comparison result and can determine the object as a target object when the object is determined to be true.

Referring to FIG. 6, the neural network data 603 '' determined by performing one convolution and one sub-sampling on the transformed image 601 are determined as neural network data for the transformed image 601, Without limitation, convolution and subsampling can be repeated at least once.

The neural network processing module 205 may perform convolution and subsampling on the transformed image to form a convolutional neural network for the data set. At this time, the information about the transformed image 601 included in the convolutional neural network may include at least a part of neural network data based on the convolution performed and the number of times of subsampling, convolution and subsampling performed.

According to one embodiment of the present invention, the neural network processing module 205 may generate a convolutional neural network, as described above, in processing the convolutional neural network algorithm and generating the convolutional neural network, but is not limited thereto, To generate a convolutional neural network.

According to one embodiment, the convolutional neural network may be formed using at least a portion of a regression analysis algorithm. For example, the neural network processing module 205 may separate the data set into a training set and a test set and apply it to the convolution neural network, and apply the result to the true / false label The accuracy of the convolutional neural network can be improved.

The object determination module 207 can detect various objects and a designated target object based on a sonar image captured during underwater movement of the underwater image processing apparatus 10 or the external apparatus 130. [

According to one embodiment, the object determination module 207 determines whether the image that is collected (e.g., photographed) during underwater movement of the underwater image processing apparatus 10 or the external apparatus 130 as an initial image, Generating an image, and applying the transformed image to the convolutional neural network to detect the target object.

According to the above description, a configuration for detecting a target object using a sonar image to be photographed has been described. However, the present invention is not limited to this, and it is obvious that a target object can be detected using a sonar image.

Referring again to FIG. 1, the sensing unit 103 may include at least one sensor for measuring information on the state of the underwater image processing apparatus 10 and surrounding conditions. For example, the sensing unit 103 may include a Doppler speed sensor, a global positioning system (GPS), a pressure measurement sensor (hereinafter referred to as a pressure sensor), a temperature measurement sensor A sensor (or a gyroscope), and a tilt measuring sensor (hereinafter, a tilt sensor).

In addition, the sensing unit 103 may include at least one image sensor. For example, the image sensor may be an image sensor using ultrasound (e.g. sonar), an image sensor using visible light (e.g. a camera, a camera), an image sensor using infrared and / or visible light An image sensor may be included.

The sensing unit 103 may measure the status of the underwater image processing apparatus 10 and the surrounding conditions through various sensors and transmit the measured information to the processing unit 101 as described above. At this time, the processing unit 101 can determine the position of the underwater image processing apparatus 10 based on the data received through the sensing unit 103. [

According to one embodiment, when water is represented by a three-dimensional coordinate system having x, y, and z axes, the water depth can be determined as a z-axis. At this time, the water depth corresponding to the z-axis of the underwater image processing apparatus 10 can be determined using at least one of a Doppler velocity sensor, a pressure sensor, and a temperature sensor gyro sensor.

It can be assumed that the depth z of the underwater image processing apparatus 10 is relatively accurately measured and therefore the correction for the position of the underwater image processing apparatus 10 is determined by the GPS, the gyroscope, or the like The horizontal movement with respect to the x-axis and / or the y-axis, and the correction of the position.

The input unit 105 may include at least one of, for example, a keyboard, a keypad, a touch screen, at least one button, and a microphone for inputting data and control commands to the underwater image processing apparatus 10 have. Further, the input unit 105 may further include a steering unit for directly controlling the movement of the underwater image processing apparatus 10 and / or the external apparatus 130.

The communication unit 107 can connect external communication with the underwater image processing apparatus 101. For example, the communication unit 107 may be connected to the network 120 via wireless communication or wired communication, and / or may communicate with the external device 130. [

Wireless communication may include, for example, wireless fidelity, Bluetooth, near field communication (NFC), global positioning system (GPS), orthogonal frequency division multiplexing (OFDM), orthogonal frequency division multiple access (OFDMA) (LTE), long term evolution-advanced (LTE), code division multiple access (CDMA), wideband code division multiple access (WCDMA), universal mobile telecommunications system (UMTS), WiBro or GSM mobile communications, and at least some of the above-described communication standards and communication schemes may be provided in cellular communication.

The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232) or a plain old telephone service (POTS). The communication unit 107 may be included in one integrated chip (IC) or an IC package.

According to one embodiment of the present invention, the network 120 may be a telecommunications network. The communication network may include at least one of a computer network, an internet, an internet of things, or a telephone network.

1, the external device 130 directly communicates with the underwater image processing apparatus 10 through the communication unit 107. However, the present invention is not limited to this, and the underwater image processing may be performed through the network 120 It is clear that it is possible to communicate with the device 10.

The storage unit 109 may store instructions and / or data received from the processing unit 101 or other components or generated by the processing unit 101 or other components. The storage unit 109 may include, for example, a programming module such as a kernel, a middleware, an application programming interface (API) or an application. Each of the above-described programming modules may be stored in the storage unit 109 as software, firmware, hardware, or a combination of at least two of them.

In addition, the storage unit 109 stores an image collected through the sensing unit 103 and / or the communication unit 107, stores the image extracted from the initial image through the processing unit 101, .

The storage unit 109 also includes a convolutional neural network and may include at least some of the convolutional data, subsampling data, and neural network data of the transformed images processed through the convolutional neural network.

Although not shown in Fig. 1, the underwater image processing apparatus 10 may further include a driving unit for moving. The driving unit may include at least one of a propeller, a screw, and a turbine, for example, as an element for generating a propulsion force to move the underwater image processing apparatus 10. In addition, the driving unit may include a direction key for switching the direction of the underwater image processing apparatus 10.

The external device 130 may be connected to the underwater image processing apparatus 10 in a wireless communication and / or a wired communication, and may be provided in the same or similar apparatus as the underwater image processing apparatus 10. For example, the external device 130 may be provided with an autonomous underwater vehicle (AUV), a human-operated submersible vehicle, a remotely operated vehicle (ROV), or the like. In addition, the external device 130 may be configured as one or more of various devices described in the same manner as the underwater image processing apparatus 10. For example, the device may be provided as a structure that includes at least a portion of the devices described, or at least some of the functionality of the device.

The external device 130 can shoot at least one sonar image through the sensing unit and transmit the photographed sonar image to the underwater imaging device 10. [

In addition, when the target object is detected, the external device 130 can perform an operation designated to the target object based on the data received from the underwater image processing apparatus 10. [

According to one embodiment, the underwater image processing apparatus 10 generates a convolutional neural network using an image collected through the sensing unit 103 and / or an image received from the external apparatus 130, And the target object included in the image can be detected by processing using the generated convolution neural network.

Here, the flow of the operation of generating the convolutional neural network of the underwater image processing apparatus 10 can be described as shown in Fig.

Referring to FIG. 7, the underwater image processing apparatus 10 may collect an image of a target object and a background image. At this time, the underwater image processing apparatus 10 may collect at least one sonar image photographed using the sonar of the sensing unit 103 (S701).

In the underwater image processing apparatus 10, in collecting the image of the object object, the underwater image processing apparatus 10 may acquire an image captured in correspondence to at least two directions for constructing an image, preferably a two-dimensional or three- . ≪ / RTI >

The underwater image processing apparatus 10 may generate a data set for generating a convolutional neural network using the collected images (S703). At this time, the data set may generate an extracted image by extracting at least a part of the collected image, and a converted image by transforming (e.g., resizing) the extracted image.

The underwater image processing apparatus 10 can set a true label on the extracted image including the entirety of the target object and can include a part of the target object or set a false label on the extracted image that does not include the target object . Here, the converted image generated by converting the extracted image with the true / false label set can also be set to the same true / false label set in the extracted image.

The underwater image processing apparatus 10 may generate a convolutional neural network using the transformed image and convolutional neural network algorithm (S705). According to one embodiment, the underwater image processing apparatus 10 can determine the converted image as input data for generating a convolutional neural network.

The underwater image processing apparatus 10 can perform convolution and sub-sampling on the transformed image according to the convolution neural network algorithm to generate neural network data. For example, if a true label is set on the transformed image, the data generated through convolution and subsampling is also labeled as true, in which case the result of the subsampling of the neural network data can be represented as true.

Similarly, when a false label is set on the transformed image, in the case of data generated through convolution and subsampling, a false label is set and, as described above, the result of subsampling of the neural network data can be represented as false.

The underwater image processing apparatus 10 may constitute a convolutional neural network including at least some of the convolutional neural network algorithm, the transformed image, and the neural network data, as described above.

The flow of the operation of detecting the target object using the convolutional neural network of the underwater image processing apparatus 10 can be described as shown in Fig.

The underwater image processing apparatus 10 may collect the sonar image from the external apparatus 130 connected to the underwater image processing apparatus 10 and / or the underwater image processing apparatus 10 via wired / wireless communication (S801). The underwater image processing apparatus 10 can detect the object from the collected image (S803). For example, the underwater image processing apparatus 10 may generate an extracted image (and a transformed image based thereon) including at least a portion of the object when the object includes the object.

The underwater image processing apparatus 10 can detect the target object using the transformed image for the object and the data of the generated convolution neural network. For example, the underwater image processing apparatus 10 can compare the converted image, the neural network data of the converted image with the image of the generated convolution network and the neural network data (S805) (S807). ≪ / RTI >

At this time, when the underwater image processing apparatus 10 confirms that at least a part of the collected image and the image of the object detected in the collected image and the neural network data of the object are matched with the image of the convolution neural network and the neural network data, The object can be determined as the target object.

When determining that the object detected from the collected image is a target object, the underwater image processing apparatus 10 may perform an operation specified for the target object, or may cause the external apparatus 130 to perform processing to perform the specified operation (S807). On the other hand, if the underwater image processing apparatus 10 determines that the detected object from the collected image is not a target object, the underwater image processing apparatus 10 may repeat the operations (S801) to (S807).

According to the various embodiments described above, the underwater image processing apparatus 10 precedes the capturing of an image of an underwater terrain, an object, etc., generates a convolutional neural network using the captured image, / It is possible to realize accurate recognition of the object in the undersea feature by detecting the target object among the unlabeled images using the learned information about false.

According to the various embodiments, the convolution neural network of the underwater image processing apparatus 10 used for detecting the target object continuously learns the input various images in which the true / false label is set to detect the target object, It is possible to improve recognition accuracy and thus to improve recognition speed and accuracy of a target object to be searched.

According to various embodiments, at least some of the devices and methods according to the various embodiments described in the claims of the present invention and / or the specification are in the form of hardware, software, firmware, or a combination of two or more of hardware, For example, modules, units). A module may be a minimum unit or a portion thereof that performs various embodiments of the present invention as a minimum unit or a part of an integrally constructed component. The module may be implemented mechanically or electronically. When implemented in software, a computer-readable storage medium (or computer-readable storage medium) for storing one or more programs (or programming modules, applications) may be provided. For example, the software may be embodied in instructions stored on a computer-readable storage medium in the form of a programming module. The one or more programs may include instructions to cause the underwater image processing apparatus to perform the methods according to the embodiments of the invention and / or the claims of the present invention. When executed by one or more processors (e.g., processing unit 101), the one or more processors may perform functions corresponding to the instructions. The computer-readable storage medium may be, for example, a storage unit 110. [ At least a portion of the programming module may be implemented (e.g., executed) by, for example, the processing unit 101. At least some of the programming modules may include, for example, modules, programs, routines, sets of instructions or processes, etc. to perform one or more functions.

The computer-readable recording medium includes a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc) A magneto-optical medium such as a floppy disk and a program command such as a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, (EEPROM), a magnetic disc storage device or other type of optical storage device, or any other form of optical storage device, A magnetic cassette may be included. Or a combination of some or all of these. In addition, a plurality of constituent memories may be included.

In addition, a communication network, such as the Internet, an intranet, a LAN (Local Area Network), a WLAN (Wide Area Network), or a communication network such as a SAN (Storage Area Network) Or in an attachable storage device that can be accessed through the storage device. Such a storage device can be connected to an underwater image processing apparatus through an external port. Further, a separate storage device on the communication network may be connected to the portable underwater image processing apparatus. The hardware devices described above may be configured to operate as one or more software modules to perform operations on the various embodiments of the present invention, and vice versa.

Modules or programming modules according to various embodiments of the present invention may include at least one or more of the elements described above, some of which may be omitted, or may further include other additional elements. Operations performed by modules, programming modules, or other components in accordance with various embodiments of the invention may be performed in a sequential, parallel, iterative, or heuristic manner. Also, some operations may be performed in a different order, omitted, or other operations may be added.

The embodiments of the present invention disclosed in the present specification and drawings are merely illustrative examples of the present invention and are not intended to limit the scope of the present invention in order to facilitate understanding of the present invention. Accordingly, the scope of the present invention should be construed as being included in the scope of the present invention, all changes or modifications derived from the technical idea of the present invention.

10: underwater image processing apparatus 101:
103: sensing unit 105: input unit
107: communication unit 109:
130: External device 201: Image acquisition module
203: Object Identification Module 205: Neural Network Processing Module
207: Object Decision Module

Claims (16)

A sensing unit for sensing a sonar image through at least one sensor;
A storage unit including a convolutional neural network; And
And a processor for detecting a target object in the water using the sonar image and the convolution neural network,
Wherein,
An image collection module for collecting the sonar image through the sensing unit;
An object checking module for detecting an object from the collected sonar image;
A neural network processing module for generating the convolutional neural network based on neural network data according to the convolution of the input image and the result of subsampling inputted through the object verification module; And
And an object determining module for determining the object among the detected objects using the detected object and the convolution neural network. The method according to claim 1,
At least one communication unit for communicating with an external device,
Wherein the image acquisition module receives the sonar image through the communication unit. delete The method according to claim 1,
And performs max-pooling with the sub-sampling. The method according to claim 1,
Wherein the object confirmation module sets a true label on the extracted image and delivers the label to the neural network processing module when the image extracted from the sonar image includes the target object. The method according to claim 1,
Wherein the object identification module sets a false label in the extracted image if the image extracted from the sonar image includes at least one of a part of the target object, an object, and a part of the object, An underwater image processing device for object recognition, which is transmitted to a neural network processing module. The method according to claim 1,
Wherein the object identification module extracts at least a portion of the sonar image and converts the size of the extracted image. The method according to claim 1,
Wherein the object determining module determines the target object among the objects whose labels have not been determined according to a sub-sampling result of the convolution neural network and a true / false label. A method for recognizing an object using an underwater image,
Collecting sonar images;
Detecting an object from the collected sonar image; And
Determining a previously designated target object among the detected objects using the detected object and the convolution neural network,
Wherein the convolution neural network is generated based on neural network data based on convolution and sub-sampling results of an input image with an input true / false label and the input image. 10. The method of claim 9,
Wherein the sonar image is photographed through a sensing unit or received from at least one external device. delete 10. The method of claim 9,
And performing max-pooling with the sub-sampling. 10. The method of claim 9,
Wherein the step of detecting an object from the collected sonar image sets a true label on the extracted image when the image extracted from the sonar image includes the target object. 10. The method of claim 9,
Wherein the detecting the object from the collected sonar image comprises extracting at least a portion of the sonar image and converting the size of the extracted image. 10. The method of claim 9,
Wherein the step of detecting an object from the collected sonar image comprises: if the image extracted from the sonar image includes at least one of a part of the target object, an object, and a part of the object, A method for processing an underwater image for object recognition, which sets a false label on an object. 10. The method of claim 9,
Wherein the step of determining a pre-determined target object among the detected objects using the detected object and the convolution neural network comprises the steps of: determining, from among the objects whose labels have not been determined according to a result of subsampling of the convolution neural network and a true / false label, An underwater image processing method for object recognition that determines a target object.

Images