JP2018156416A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection method with high processing speed and high accuracy in order to process object detection processing faster and to detect an object with high accuracy, so as to quickly check the object detection result when object detection is performed relating to an image photographed with a camera-equipped device such as a smartphone.SOLUTION: The designation of the position is accepted by accepting a touch operation on an image displayed on the touch panel with respect to an input image and to define the position where designation was accepted as a starting point and to enlarge the area, thereby an area including an object to be detected is specified, and the information of the specified area is output.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  Conventionally, object detection methods for recognizing what an object is in the image and where the position is in the image have been studied.In particular, in recent years, an object using machine learning technology called deep learning has been applied. Many detection methods have been proposed.

  Non-Patent Document 1 discloses an object detection method characterized by detecting the size of an object more accurately and with good accuracy compared to a conventional object detection method applying deep learning.

Donggeun Yoo et al., “Action-Driven Object Object detection with Top-Down Visual Attentions”

  However, in Non-Patent Document 1, a number of images in which the resolution and aspect ratio of a detection target image are changed are created, and all of these images are processed many times by a neural network in order to detect an object. Therefore, there is a problem that the detection speed is slow. In particular, when object detection is performed on an image captured by a camera-equipped device such as a smartphone and the result is confirmed, there is a problem that convenience is lowered. Further, depending on the object to be detected, there is a problem that the detection accuracy is low.

  Therefore, for example, when object detection is performed on an image captured with a camera-equipped device such as a smartphone, it is desired to perform the object detection process faster in order to quickly confirm the object detection result. It is also desired to detect an object with higher accuracy.

  Accordingly, an object of the present invention is to provide an object detection method with high processing speed and accuracy.

  The information processing apparatus of the present invention

  According to the present invention, it is possible to provide an object detection method with high processing speed and accuracy.

It is a figure which shows an example of the system configuration | structure of an object detection system in embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of the portable terminal 100 and the object detection server 200 in embodiment of this invention. It is a figure which shows an example of a function structure of the object detection system in embodiment of this invention. It is a flowchart which shows an example of the process which performs an object detection with respect to the image | photographed image and displays a detection result in embodiment of this invention. It is a flowchart which shows an example of the process which detects an object using the object position hint in embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a configuration of an information processing system (object detection system) according to an embodiment of the present invention.

  The object detection system has a configuration in which a portable terminal 100 including an imaging unit such as a smartphone and an object detection server 200 are connected via a LAN 300.

  The portable terminal 100 captures an image to be subjected to object detection processing using a camera (imaging means) provided in the device. A user's object position hint instruction is received for the captured image, and the image and the object position hint are transmitted to the object detection server 200. Then, the object detection result is received from the object detection server 200.

  The object detection server 200 performs an object detection process using the image transmitted from the mobile terminal 100 and the object position hint, and transmits the object detection result to the mobile terminal 100.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing apparatus applicable to the mobile terminal 100 and the object detection server 200 according to the embodiment of the present invention.

  As shown in FIG. 2, the mobile terminal 100 and the object detection server 200 are connected via a system bus 204 to a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an input controller 205. , A video controller 206, a memory controller 207, and a communication I / F controller 208 are connected.

  The CPU 201 comprehensively controls each device and controller connected to the system bus 204.

  The ROM 202 or the external memory 211 includes a BIOS (Basic Input / Output System) and an OS (Operating System) that are control programs executed by the CPU 201, and a computer-readable program and various necessary programs for realizing the information processing method. Holds data (including data table).

  The RAM 203 functions as a main memory, work area, and the like for the CPU 201. The CPU 201 implements various operations by loading a program or the like necessary for executing the processing from the ROM 202 or the external memory 211 to the RAM 203 and executing the loaded program.

  The input controller 205 controls input from an input device such as a keyboard 209 or a pointing device such as a mouse (not shown). When the input device is a touch panel, the user can perform various instructions by pressing (touching with a finger or the like) in accordance with an icon, a cursor, or a button displayed on the touch panel.

  The touch panel may be a touch panel capable of detecting a position touched with a plurality of fingers, such as a multi-touch screen.

  The video controller 206 controls display on an external output device such as the display 210. The display includes a display of a notebook computer integrated with the main body. The external output device is not limited to a display, and may be a projector, for example. An input device is also provided for the device that can accept the touch operation described above.

  Note that the video controller 206 can control a video memory (VRAM) for display control, and a part of the RAM 203 can be used as a video memory area, or a dedicated video memory can be provided separately. Is possible.

  The memory controller 207 controls access to the external memory 211. The external memory is connected via an adapter to an external storage device (hard disk), flexible disk (FD), or PCMCIA card slot that stores boot programs, various applications, font data, user files, editing files, and various data. A compact flash (registered trademark) memory or the like can be used.

  The communication I / F controller 209 connects and communicates with an external device via a network, and executes communication control processing on the network. For example, communication using TCP / IP, telephone lines such as ISDN, and communication using 3G lines of mobile phones are possible.

Note that the CPU 201 enables display on the display 210 by executing outline font rasterization processing on a display information area in the RAM 203, for example. Further, the CPU 201 enables a user instruction with a mouse cursor (not shown) on the display 210.

Next, with reference to FIG. 3, an example of a functional configuration of various apparatuses according to the embodiment of the present invention will be described.

  The object position hint receiving unit 101 has a function of acquiring a position designated by the user with respect to an image. In step S402, designation of the position is received from the user, and the designated position is acquired (S404).

  The transmission control unit 102 has a function of transmitting, to the object detection server 200, an image photographed by the camera photographing unit 105 and information related to the position designated by the object position hint accepting unit 101.

  The reception control unit 103 has a function of receiving a processing result by the object detection processing unit 202 of the object detection server 200.

  The object detection result confirmation unit 104 has a function of displaying information such as a generated rectangle based on the result received by the reception control unit 103. Details will be described in step S407.

  The reception control unit 201 receives (inputs) the data transmitted from the transmission control unit 102 (the process in step S404).

  The object detection processing unit 202 is a functional unit that executes the process shown in the flowchart of FIG. 5 in step S405. The object detection processing unit 202 expands the region from the specified position as a starting point, thereby detecting the region including the object to be detected. Specify and output information indicating the specified area. Specifically, a rectangle centered at a specified position is generated, and it is determined whether or not the detected object is included in the generated rectangle. Identifies as an included area. Further, when it is determined that the detection target object is not included in the generated rectangle, a rectangle obtained by enlarging the rectangle is generated. Specific contents will be described later using a flowchart.

  The transmission control unit 203 has a function of transmitting the processing result in the object detection processing unit 202 to the mobile terminal 100.

  Next, processing executed by the object detection system according to the embodiment of the present invention will be described using the flowcharts of FIGS. 4 and 5.

  The flowchart of FIG. 5 is a flowchart showing a process of detecting an object, which is a process in which the CPU 201 of the mobile terminal 100 and the object detection server 200 reads and executes a predetermined control program.

  In step S <b> 401, the CPU 201 of the mobile terminal 100 receives a shooting instruction from the user and takes an image with a built-in camera.

  In step S402, the CPU 201 of the mobile terminal 100 displays the image captured in step S401 on the display unit, and accepts an object position hint instruction (position specification) from the user. The object position hint is instructed by an operation such as a user performing a touch operation on the displayed image with a finger.

  Further, it is possible to guide the object to be photographed in accordance with a predetermined position (area) at the time of photographing, and the predetermined position may be set as the designated position.

  Further, in this embodiment, it is assumed that a mobile terminal with a camera function such as a smartphone accepts an object position hint instruction, but an image taken on a PC or the like is displayed, and the object position hint is displayed by a mouse click or the like. May be accepted.

  As described above, any configuration may be used as long as the specification of the object position or the hint (reference) of the object position is received from the user.

  In step S403, the CPU 201 of the mobile terminal 100 transmits the image captured in step S401 and the object position hint received in step S402 to the object detection server 200.

  In step S404, the CPU 201 of the object detection server 200 receives (acquires) the image and the object position hint (information indicating the position designated by the user) transmitted in step S403.

  In step S405, the CPU 201 of the object detection server 200 performs object detection processing on the image using the image received in step S404 and the object position hint. Detailed processing in step S405 will be described later with reference to the flowchart of FIG.

  In step S406, the CPU 201 of the object detection server 200 transmits the object detection result obtained by the object detection process in step S405 and the image received in step S404 to the mobile terminal 100. The object detection result is expressed by the coordinates of the upper left point and the lower right point of the rectangle indicating the position and size of the object on the image. If no object is detected, the coordinates of the upper left point and the lower right point are all expressed as 0.

  In step S407, the CPU 201 of the mobile terminal 100 receives the object detection result and the image transmitted in step S406, for example, draws a rectangle as the object detection result with a red line on the image, and displays the image. To do.

  Next, processing executed by the object detection server 200 according to the embodiment of the present invention will be described using the flowchart of FIG.

  The flowchart in FIG. 5 is a process in which the CPU 201 of the object detection server 200 reads out and executes a predetermined control program, and is a flowchart showing a process in which an object detection process is performed using an image and an object position hint.

  In step S501, the CPU 201 of the object detection server 200 generates coordinate information of a rectangle that is an initial processing target from the coordinates of the object position hint. The coordinate information of the rectangle is obtained by subtracting 5 from the X and Y coordinates of the object position hint as the upper left point coordinate of the rectangle, and adding the X and Y coordinates of the object position hint as 5 to the lower right of the rectangle. And the point coordinates. Therefore, the rectangle that is the initial processing target has a width and a height of 10, and the center position is the object position hint. Note that the size of the rectangle that is the initial processing target described in the present embodiment, the correction amount in S505, and the like are examples, and can be set as appropriate according to the object to be detected.

  In step S502, the CPU 201 of the object detection server 200 generates an image (partial image) including an area indicated by the coordinate information of the processing target rectangle. And the produced | generated partial image is input into AttentionNet which is an object detection method shown by the nonpatent literature 1, and the identification result with respect to the upper left point and the lower right point of a partial image is acquired. Further, the width and height of the partial image are stored in the memory area.

  Here, “AttentionNet” identifies one of “down”, “right”, “bottom right”, “stop”, and “FALSE” with respect to the upper left point of the image depending on whether or not the size of the detection target object has been accurately detected. It is a deep neural network that identifies one of top, left, top left, stop, and FALSE for the lower right point.

  When the detected object is included in the target image (partial image in this embodiment) with an accurate size (the detected object is included in the target image), “stop” is output. That is, when a stop is output for both the upper left and lower right, it means that a rectangle circumscribing the detected object has been generated.

  FALSE indicates that the detection object is not included in the target image.

  Except for stop and FALSE, if the coordinates are moved in this direction and the area of the target image is changed (reduced), the detection target object can be surrounded with a more accurate size (a rectangle circumscribing the detection object can be generated). It is shown that.

  In addition, as shown in Non-Patent Document 1, the AttentionNet used here uses a deep neural network learned in advance so that a detection target object can be detected.

  In step S503, the CPU 201 of the object detection server 200, if the identification result of the upper left point acquired in step S502 is lower right or stopped, and if the identification result of the lower right point is upper left or stopped, the detection target object is It is determined that a rectangle that can be accommodated has been generated, and the coordinate information of the current processing target rectangle is used as the object detection result, and the process ends. Otherwise, the process proceeds to step S504.

  Note that in this embodiment, by gradually expanding the size of the rectangle, a rectangle in which the object to be detected is accommodated is generated, and thus “stop” may be passed. That is, the image may be enlarged past a rectangle just circumscribing the detection target object. Therefore, in the determination of S503, control is performed not only in the case of “stop” but also when the identification result of the upper left point is lower right and when the identification result of the lower right point is upper left, it is determined to be YES. Yes.

  As described in the processing of S504, which will be described later, since the possibility of false detection is high while the rectangle is small, it is not determined YES in the case of “stop”, and “the identification result of the upper left point is lower right. In such a case, control may be performed so as to determine YES only when the identification result of the lower right point is upper left.

  In step S504, if the identification result of the upper left point acquired in step S502 is FALSE, the CPU 201 of the object detection server 200 proceeds to step S505. Otherwise, the process proceeds to step S506.

  As described in the above-described processing of S503, since the possibility of erroneous detection is high while the rectangle is small (for example, when the rectangle is less than a predetermined size), control is performed so as to determine YES even in the case of “stop”. May be.

  In step S505, the CPU 201 of the object detection server 200 subtracts 5 from the X coordinate and Y coordinate of the upper left point of the coordinate information of the processing target rectangle, and when the X coordinate is smaller than 0, it is 0, and the Y coordinate is 0. If it is smaller, the coordinates are corrected to 0, and the process proceeds to step S510.

  In step S506, the CPU 201 of the object detection server 200 proceeds to step S507 if the identification result of the upper left point acquired in step S502 is lower. Otherwise, the process proceeds to step S508.

  In step S507, the CPU 201 of the object detection server 200 subtracts 5 from the X coordinate of the upper left point of the coordinate information of the processing target rectangle, and corrects the coordinate to 0 when the X coordinate is smaller than 0, and proceeds to step S510. Proceed with That is, when the identification result of the upper left point is lower, the detection target object is included in the Y coordinate direction, but the X coordinate direction needs to be further enlarged to include the detection target object. It means that there is. Therefore, a process for enlarging the X coordinate direction is executed.

  In step S508, the CPU 201 of the object detection server 200 proceeds to step S509 if the identification result of the upper left point acquired in step S502 is right. Otherwise, the process proceeds to step S510.

  In step S509, the CPU 201 of the object detection server 200 subtracts 5 from the Y coordinate of the upper left point in the coordinate information of the processing target rectangle, and corrects the coordinate to 0 when the Y coordinate is smaller than 0, and proceeds to step S510. Proceed with That is, when the identification result of the upper left point is right, the X coordinate direction includes the detection target object, but the Y coordinate direction needs to be further enlarged to include the detection target object. It means that there is. Therefore, a process for enlarging in the Y coordinate direction is executed.

  In step S510, if the result of identifying the lower right point acquired in step S502 is FALSE, the CPU 201 of the object detection server 200 proceeds to step S511. Otherwise, the process proceeds to step S512.

  As in the above-described processing of S504, while the rectangle is small (for example, when it is less than the predetermined size), there is a high possibility of erroneous detection. Also good.

  In step S511, the CPU 201 of the object detection server 200 adds 5 to the X coordinate and the Y coordinate of the lower right point of the coordinate information of the processing target rectangle, and when the X coordinate is equal to or larger than the width of the image, If the value obtained by subtracting 1 from the width is equal to or larger than the height of the image, the coordinate is corrected to a value obtained by subtracting 1 from the height of the image, and the process proceeds to step S516.

  In step S512, the CPU 201 of the object detection server 200 proceeds to step S513 if the identification result of the lower right point acquired in step S502 is up. Otherwise, the process proceeds to step S514.

  In step S513, the CPU 201 of the object detection server 200 adds 5 to the X coordinate of the lower right point of the coordinate information of the processing target rectangle, and when the X coordinate is equal to or larger than the width of the image, the X coordinate is set to 1 from the image width. The coordinates are corrected to the value obtained by subtracting, and the process proceeds to step S516.

  That is, when the identification result of the upper right point is up, the Y coordinate direction includes the detection target object, but the X coordinate direction needs to be further enlarged to include the detection target object. It means that there is. Therefore, a process for enlarging the X coordinate direction is executed.

  In step S514, if the identification result of the lower right point acquired in step S502 is left, the CPU 201 of the object detection server 200 proceeds to step S515. Otherwise, the process proceeds to step S502.

  In step S515, the CPU 201 of the object detection server 200 adds 5 to the Y coordinate of the lower right point of the coordinate information of the processing target rectangle, and when the Y coordinate is equal to or higher than the image height, the Y coordinate is set to the image height. The coordinate is corrected to a value obtained by subtracting 1 from 1, and the process proceeds to step S516.

  That is, when the identification result of the upper right point is left, the X coordinate direction includes the detection target object, but the Y coordinate direction needs to be further expanded to include the detection target object. It means that there is. Therefore, a process for enlarging in the Y coordinate direction is executed.

  In step S516, the CPU 201 of the object detection server 200 compares the width and height of the partial image stored in the memory area in step S502 with the width and height of the area indicated by the coordinate information of the current processing target rectangle. If the width and height are the same (that is, the size has not changed before and after coordinate correction by S604 etc., for example, the size of the entire image and the size of the rectangular area may be the same), the object is It determines with having not detected, and complete | finishes a process by making into the object detection result the coordinate information whose X coordinate and Y coordinate of the upper left point and the lower right point are all 0.

  As described above, the image of the region specified by the rectangle is repeatedly input to the AttentionNet shown in Non-Patent Document 1 while gradually expanding the rectangle information using the object position hint by the processing of the flowchart of FIG. Thus, the size of the object can be detected more accurately at high speed.

  The embodiment as the information processing apparatus has been described above, but the present invention can take an embodiment as a system, apparatus, method, program, recording medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  The program according to the present invention is a program that allows a computer to execute the processing methods of the flowcharts shown in FIGS. 5 and 6. The storage medium according to the present invention is a program that allows the computer to execute the processing methods of FIGS. Is remembered. Note that the program in the present invention may be a program for each processing method of each apparatus shown in FIGS.

  As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) operating on the computer based on an instruction of the program is actually It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the processing and the processing is included.

  Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The present invention may be applied to a system constituted by a plurality of devices or an apparatus constituted by a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

100 Information processing apparatus 200 Object detection server 300 LAN

Claims (6)

Position acquisition means for acquiring a position designated by the user with respect to the image;
An area specifying means for specifying an area including an object to be detected by expanding the area starting from the position acquired by the position acquiring means;
Output means for outputting information indicating the area specified by the area specifying means;
An information processing system comprising: Rectangle generating means for generating a rectangle centered on the position acquired by the position acquiring means;
Determining means for determining whether the object to be detected is included in the rectangle generated by the rectangle generating means;
With
The information processing system according to claim 1, wherein when it is determined that a detection target object is included in the generated rectangle, the information processing system is specified as a region including the detection target object.   The rectangle generation unit further generates a rectangle obtained by enlarging the rectangle when the determination unit determines that a detection target object is not included in the generated rectangle. Information processing system. Display means for displaying the input image on a touch panel;
The position acquisition means receives a touch operation on an image displayed on a touch panel by the display means, thereby receiving the designation of the position and acquiring the position where the designation is accepted. The information processing system according to any one of the above. A position acquisition step of acquiring a position specified by the user with respect to the image;
A region specifying step for specifying a region including an object to be detected by expanding the region starting from the position acquired in the position acquiring step;
An output step of outputting information indicating the region specified by the region specifying step;
An information processing method comprising: A program executable in the information processing apparatus,
The information processing apparatus;
Position acquisition means for acquiring a position designated by the user with respect to the image;
An area specifying means for specifying an area including an object to be detected by expanding the area starting from the position acquired by the position acquiring means;
A program for functioning as output means for outputting information indicating an area specified by the area specifying means.

Images